1 |
The Seeds of the Noun–Verb Distinction in the Manual Modality: Improvisation and Interaction in the Emergence of Grammatical Categories
|
|
|
|
In: Languages; Volume 7; Issue 2; Pages: 95 (2022)
|
|
BASE
|
|
Show details
|
|
2 |
Sequence-to-Sequence Predictive Model: From Prosody To Communicative Gestures
|
|
|
|
In: Proceeding of the 24th International Conference on Human-Computer Interaction ; https://hal.archives-ouvertes.fr/hal-03428910 ; Proceeding of the 24th International Conference on Human-Computer Interaction, 2021 (2021)
|
|
Abstract:
International audience ; Communicative gestures and speech acoustic are tightly linked. Our objective is to predict the timing of gestures according to the acoustic. That is, we want to predict when a certain gesture occurs. We develop a model based on a recurrent neural network with attention mechanism. The model is trained on a corpus of natural dyadic interaction where the speech acoustic and the gesture phases and types have been annotated. The input of the model is a sequence of speech acoustic and the output is a sequence of gesture classes. The classes we are using for the model output is based on a combination of gesture phases and gesture types. We use a sequence comparison technique to evaluate the model performance. We find that the model can predict better certain gesture classes than others. We also perform ablation studies which reveal that fundamental frequency is a relevant feature for gesture prediction task. In another sub-experiment, we find that including eyebrow movements as acting as beat gesture improves the performance. Besides, we also find that a model trained on the data of one given speaker also works for the other speaker of the same conversation. We also perform a subjective experiment to measure how respondents judge the naturalness, the time consistency, and the semantic consistency of the generated gesture timing of a virtual agent. Our respondents rate the output of our model favorably.
|
|
Keyword:
[INFO.INFO-HC]Computer Science [cs]/Human-Computer Interaction [cs.HC]; Communicative Gesture; Machine Learning; Prosody
|
|
URL: https://hal.archives-ouvertes.fr/hal-03428910/file/hcii_2021_shortened.pdf https://hal.archives-ouvertes.fr/hal-03428910 https://hal.archives-ouvertes.fr/hal-03428910/document
|
|
BASE
|
|
Hide details
|
|
5 |
Video recordings of static hand gestures for gesture based interaction ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Parenting and Early Language Development of Young Children in South Korea
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Do women and men use language differently in spoken face-to-face interaction? A scoping review
|
|
|
|
In: Review of Communication Research ; 9 ; 43-79 (2021)
|
|
BASE
|
|
Show details
|
|
8 |
Can you see what I mean? A multimodal approach to interactional competence in English-Medium Instruction
|
|
|
|
In: TDX (Tesis Doctorals en Xarxa) (2021)
|
|
BASE
|
|
Show details
|
|
9 |
Building An Automated Gesture Imitation Game For Teenagers with ASD
|
|
|
|
In: ISSN: 0973-7006 ; Far East Journal of Electronics and Communications ; https://hal-imt-atlantique.archives-ouvertes.fr/hal-02894314 ; Far East Journal of Electronics and Communications, 2020, 23 (1), pp.1 - 10. ⟨10.17654/EC023010001⟩ (2020)
|
|
BASE
|
|
Show details
|
|
10 |
Automata in antiquity : sources, functions and virtual restitution ; Les automates dans l'antiquité : sources, fonctions et restitution virtuelle
|
|
|
|
In: https://tel.archives-ouvertes.fr/tel-02997617 ; Linguistique. Normandie Université, 2020. Français. ⟨NNT : 2020NORMC009⟩ (2020)
|
|
BASE
|
|
Show details
|
|
11 |
Joint Attention in Hearing Parent-Deaf Child and Hearing Parent-Hearing Child Dyads.
|
|
|
|
In: IEEE transactions on cognitive and developmental systems, vol 12, iss 2 (2020)
|
|
BASE
|
|
Show details
|
|
12 |
Gesten - gestern, heute, übermorgen. Vom Forschungsprojekt zur Ausstellung
|
|
|
|
BASE
|
|
Show details
|
|
13 |
The emergence of word order conventions: improvisation, interaction and transmission ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
The emergence of systematic argument distinctions in artificial sign languages (dataset) ...
|
|
Unkn Unknown. - : University of Edinburgh. Centre for Language Evolution, 2020
|
|
BASE
|
|
Show details
|
|
15 |
Le corps comme médium d’interaction dans le processus d’appropriation ou de résistance au dispositif de jeu vidéo ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Evaluating approaches to improve upon a Leap Motion-based hand-gesture recognition system
|
|
|
|
BASE
|
|
Show details
|
|
17 |
Language in the body: Multimodality in grammar and discourse
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Exploring Cognitive Load of Single and Mixed Mental Models Gesture Sets for UAV Navigation
|
|
|
|
In: 1st International Workshop on Human-Drone Interaction ; https://hal.archives-ouvertes.fr/hal-02128398 ; 1st International Workshop on Human-Drone Interaction, Ecole Nationale de l'Aviation Civile [ENAC], May 2019, Glasgow, United Kingdom (2019)
|
|
BASE
|
|
Show details
|
|
19 |
T-Voks: the Singing and Speaking Theremin
|
|
|
|
In: NIME 2019 International Conference on New Interfaces for Musical Expression ; https://hal.archives-ouvertes.fr/hal-02197063 ; NIME 2019 International Conference on New Interfaces for Musical Expression, UFRGS, Jun 2019, Porto Alegre, Brazil. pp.110-115 ; http://www.nime.org/proceedings/2019/nime2019_022.pdf (2019)
|
|
BASE
|
|
Show details
|
|
20 |
Gestures and construction of referential chains in a corpus of tandem interactions ; Gestualité et construction des chaînes de référence dans un corpus d’interactions tandem
|
|
|
|
In: ISSN: 0765-4944 ; EISSN: 2111-5044 ; Les cahiers de praxématique ; https://hal.archives-ouvertes.fr/hal-02171598 ; Les cahiers de praxématique, Montpellier : Presses universitaires de la Méditerranée, 2006-, 2019, La gestion de l'anaphore en discours : complexités et enjeux, 72 (72), ⟨10.4000/praxematique.5576⟩ ; https://journals.openedition.org/praxematique/5576# (2019)
|
|
BASE
|
|
Show details
|
|
|
|