| 2 |
Sequence-to-Sequence Predictive Model: From Prosody To Communicative Gestures
|
|
|
|
In: Proceeding of the 24th International Conference on Human-Computer Interaction ; https://hal.archives-ouvertes.fr/hal-03428910 ; Proceeding of the 24th International Conference on Human-Computer Interaction, 2021 (2021)
|
|
BASE
|
|
Show details
|
|
|
| 3 |
Sequence-to-Sequence Predictive models: from Prosody to Communicative Gestures
|
|
|
|
In: Workshop sur les Affects, Compagnons artificiels et Interactions ; https://hal.inria.fr/hal-02933487 ; Workshop sur les Affects, Compagnons artificiels et Interactions, CNRS, Université Toulouse Jean Jaurès, Université de Bordeaux, Jun 2020, Saint Pierre d'Oléron, France (2020)
|
|
Abstract:
International audience ; Communicative gestures and speech prosody are tightly linked. Our aim is to predict when gestures are performed based on prosody. We develop a model based on a seq2seq recurrent neural network with attention mechanism. The model is trained on a corpus of natural dyadic interaction where the speech prosody and the gestures have been annotated. Because the output of the model is a sequence, we use a sequence comparison technique to evaluate the model performance. We find that the model can predict certain gesture classes. In our experiment, we also replace some input features with random values to find which prosody features are pertinent. We find that the F0 is pertinent. Lastly, we also train the model on one speaker and test it with the other speaker to find whether the model is generalisable. We find that the models which we train on one speaker also works for another speaker of the same conversation.
|
|
Keyword:
[SCCO.COMP]Cognitive science/Computer science; [SCCO.NEUR]Cognitive science/Neuroscience; [SCCO.PSYC]Cognitive science/Psychology; [SHS.PSY]Humanities and Social Sciences/Psychology; [SHS.SOCIO]Humanities and Social Sciences/Sociology; Gesture; Machine Learning; Neural Network
|
|
URL: https://hal.inria.fr/hal-02933487/file/wacai_2020_7_.pdf
https://hal.inria.fr/hal-02933487
https://hal.inria.fr/hal-02933487/document
|
|
BASE
|
|
Hide details
|
|
|
| 4 |
The ISO Standard for Dialogue Act Annotation, Second Edition
|
|
|
|
In: Proceedings of the 12th Language Resources and Evaluation Conference ; https://hal.archives-ouvertes.fr/hal-03011688 ; Proceedings of the 12th Language Resources and Evaluation Conference, May 2020, Marseille, France (2020)
|
|
BASE
|
|
Show details
|
|
|
| 5 |
Multimodal Analysis of Cohesion in Multi-party Interactions
|
|
|
|
In: LREC ; https://hal.archives-ouvertes.fr/hal-02953469 ; LREC, 2020, Marseille, France (2020)
|
|
BASE
|
|
Show details
|
|
|
| 6 |
A multimodal corpus of simulated consultations between a patient and multiple healthcare professionals
|
|
|
|
In: ISSN: 1574-020X ; EISSN: 1574-0218 ; Language Resources and Evaluation ; https://hal.sorbonne-universite.fr/hal-03209106 ; Language Resources and Evaluation, Springer Verlag, 2020, ⟨10.1007/s10579-020-09526-0⟩ (2020)
|
|
BASE
|
|
Show details
|
|
|
| 7 |
How Confident are You? Exploring the Role of Fillers in the Automatic Prediction of a Speaker's Confidence
|
|
|
|
In: Workshop sur les Affects, Compagnons artificiels et Interactions ; https://hal.inria.fr/hal-02933476 ; Workshop sur les Affects, Compagnons artificiels et Interactions, CNRS, Université Toulouse Jean Jaurès, Université de Bordeaux, Jun 2020, Saint Pierre d'Oléron, France (2020)
|
|
BASE
|
|
Show details
|
|
|
| 8 |
HOW CONFIDENT ARE YOU? EXPLORING THE ROLE OF FILLERS IN THE AUTOMATIC PREDICTION OF A SPEAKER’S CONFIDENCE
|
|
|
|
In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) ; https://hal.archives-ouvertes.fr/hal-02942182 ; ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2020, Barcelona, Spain. pp.8104-8108, ⟨10.1109/ICASSP40776.2020.9054374⟩ (2020)
|
|
BASE
|
|
Show details
|
|
|
| 9 |
Disfluencies and Teaching Strategies in Social Interactions Between a Pedagogical Agent and a Student: Background and Challenges
|
|
|
|
In: SEMDIAL 2018 (AixDial), The 22nd workshop on the Semantics and Pragmatics of Dialogue ; https://hal.telecom-paris.fr/hal-02292437 ; SEMDIAL 2018 (AixDial), The 22nd workshop on the Semantics and Pragmatics of Dialogue, Nov 2018, Aix-en-Provence, France. pp.188-191 (2018)
|
|
BASE
|
|
Show details
|
|
|
| 11 |
Automating the Production of Communicative Gestures in Embodied Characters
|
|
|
|
BASE
|
|
Show details
|
|
|
| 12 |
Proceedings of the 18th International Conference on Intelligent Virtual Agents, Sydney, NSW, Australia, November 05 - 08, 2018
|
|
|
|
BASE
|
|
Show details
|
|
|
| 13 |
A Methodology for the Automatic Extraction and Generation of Non-Verbal Signals Sequences Conveying Interpersonal Attitudes
|
|
|
|
In: ISSN: 1949-3045 ; IEEE Transactions on Affective Computing ; https://hal.archives-ouvertes.fr/hal-01793271 ; IEEE Transactions on Affective Computing, Institute of Electrical and Electronics Engineers, 2017, XX, pp.1 - 1. ⟨10.1109/TAFFC.2017.2753777⟩ (2017)
|
|
BASE
|
|
Show details
|
|
|
| 14 |
Laughter Animation Generation
|
|
|
|
In: Handbook of Human Motion ; https://hal.sorbonne-universite.fr/hal-01528400 ; Bertram Müller; Sebastian I. Wolf; Gert-Peter Brueggemann; Zhigang Deng; Andrew McIntosh; Freeman Miller; William Scott Selbie. Handbook of Human Motion, Springer, 2017, ⟨10.1007/978-3-319-30808-1_190-1⟩ (2017)
|
|
BASE
|
|
Show details
|
|
|
| 15 |
The NoXi database: multimodal recordings of mediated novice-expert interactions
|
|
|
|
BASE
|
|
Show details
|
|
|
| 20 |
Analysis of Verbal and Nonverbal Communication and Enactment.The Processing Issues : COST 2102 International Conference, Budapest, Hungary, September 7-10, 2010, Revised Selected Papers
|
|
|
|
UB Frankfurt Linguistik
|
|
Show details
|
|
|
|
|
