| 1 |
Mutual Understanding in Situated Interactions with Conversational User Interfaces : Theory, Studies, and Computation
|
|
|
|
Abstract:
This dissertation presents advances in HCI through a series of studies focusing on task-oriented interactions between humans and between humans and machines. The notion of mutual understanding is central, also known as grounding in psycholinguistics, in particular how people establish understanding in conversations and what interactional phenomena are present in that process. Addressing the gap in computational models of understanding, interactions in this dissertation are observed through multisensory input and evaluated with statistical and machine-learning models. As it becomes apparent, miscommunication is ordinary in human conversations and therefore embodied computer interfaces interacting with humans are subject to a large number of conversational failures. Investigating how these inter- faces can evaluate human responses to distinguish whether spoken utterances are understood is one of the central contributions of this thesis. The first papers (Papers A and B) included in this dissertation describe studies on how humans establish understanding incrementally and how they co-produce utterances to resolve misunderstandings in joint-construction tasks. Utilising the same interaction paradigm from such human-human settings, the remaining papers describe collaborative interactions between humans and machines with two central manipulations: embodiment (Papers C, D, E, and F) and conversational failures (Papers D, E, F, and G). The methods used investigate whether embodiment affects grounding behaviours among speakers and what verbal and non-verbal channels are utilised in response and recovery to miscommunication. For application to robotics and conversational user interfaces, failure detection systems are developed predicting in real-time user uncertainty, paving the way for new multimodal computer interfaces that are aware of dialogue breakdown and system failures. Through the lens of Theory, Studies, and Computation, a comprehensive overview is presented on how mutual understanding has been observed in interactions with humans and between humans and machines. A summary of literature in mutual understanding from psycholinguistics and human-computer interaction perspectives is reported. An overview is also presented on how prior knowledge in mutual understanding has and can be observed through experimentation and empirical studies, along with perspectives of how knowledge acquired through observation is put into practice through the analysis and development of computational models. Derived from literature and empirical observations, the central thesis of this dissertation is that embodiment and mutual understanding are intertwined in task-oriented interactions, both in successful communication but also in situations of miscommunication. ; QC 20220216
|
|
Keyword:
common ground; conversational failures; dialogue and discourse; embodiment; Human Computer Interaction; human-computer interaction; joint-construction tasks; Människa-datorinteraktion (interaktionsdesign); multimodal behaviours; smart-speakers; social robots; social signal processing
|
|
URL: http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-308927
|
|
BASE
|
|
Hide details
|
|
|
| 2 |
Demonstratives in Nsélišcn ‘Montana Salish’
|
|
|
|
In: Graduate Student Theses, Dissertations, & Professional Papers (2022)
|
|
BASE
|
|
Show details
|
|
|
| 3 |
Towards a unified account of quidem and ne.quidem
|
|
|
|
In: Linguisticae dissertationes: current perspectives on Latin grammar, lexicon and pragmatics. Selected Papers from the 20th International Colloquium on Latin Linguistics (Las Palmas de Gran Canaria, Spain, June 17-21, 2019) ; https://hal.archives-ouvertes.fr/hal-03327155 ; Antonio María Martín Rodríguez. Linguisticae dissertationes: current perspectives on Latin grammar, lexicon and pragmatics. Selected Papers from the 20th International Colloquium on Latin Linguistics (Las Palmas de Gran Canaria, Spain, June 17-21, 2019), Ediciones Clásicas, 2021 (2021)
|
|
BASE
|
|
Show details
|
|
|
| 4 |
Contrast, Verum Focus and Anaphora: The Case of et pourtant si/non in French
|
|
|
|
In: ISSN: 0022-2267 ; EISSN: 1469-7742 ; Journal of Linguistics ; https://hal.archives-ouvertes.fr/hal-03251593 ; Journal of Linguistics, Cambridge University Press (CUP), 2021, 57 (2), pp.279-319. ⟨10.1017/S0022226720000195⟩ (2021)
|
|
BASE
|
|
Show details
|
|
|
| 5 |
Taboo: Analysis of referring expressions in an interactive communication game ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 6 |
Modal and discourse properties of utterance-final particles in Mandarin
|
|
|
|
BASE
|
|
Show details
|
|
|
| 7 |
Investigating the Influence of Local and Personal Common Ground on Memory for Conversation Using an Online Referential Communication Task
|
|
|
|
BASE
|
|
Show details
|
|
|
| 8 |
Rational Redundancy in Referring Expressions: Evidence from Event‐related Potentials
|
|
|
|
BASE
|
|
Show details
|
|
|
| 9 |
PACO : A corpus to analyze the impact of common ground in spontaneous face-to-face interaction ; PACO: un corpus pour analyser l'impact du common ground dans les interactions spontanées en face à face
|
|
|
|
In: Language Resources and Evaluation Conference ; https://hal.archives-ouvertes.fr/hal-02558217 ; Language Resources and Evaluation Conference, May 2020, Marseille, France (2020)
|
|
BASE
|
|
Show details
|
|
|
| 11 |
Epistemic primacy, Common Ground management and epistemic perspective ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 12 |
Epistemic primacy, Common Ground management and epistemic perspective ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 14 |
Where do you place your argument?
|
|
|
|
In: OSSA Conference Archive (2020)
|
|
BASE
|
|
Show details
|
|
|
| 15 |
“. to grasp the native's point of view.” - A Plea for a Holistic Documentation of the Trobriand Islanders' Language, Culture and Cognition
|
|
|
|
In: Russian Journal of Linguistics, Vol 24, Iss 1, Pp 7-30 (2020) (2020)
|
|
BASE
|
|
Show details
|
|
|
| 16 |
Quelle méthodologie pour analyser le sourire lors d'interactions conversationnelles ?
|
|
|
|
In: JAD 2019 ; Journée Annuelle des Doctorants 2019 ; https://hal.archives-ouvertes.fr/hal-02412788 ; Journée Annuelle des Doctorants 2019, Laboratoire Parole et Langage, Dec 2019, Aix-en-Provence, France (2019)
|
|
BASE
|
|
Show details
|
|
|
| 17 |
Metonymy and frame integration: Interfacing between concepts and discourse
|
|
|
|
In: Topics in Linguistics, Vol 20, Iss 1, Pp 1-23 (2019) (2019)
|
|
BASE
|
|
Show details
|
|
|
| 18 |
Juk and gi, and “particles” in contemporary Lithuanian: Explaining language-particular elements in a cross-linguistic context
|
|
|
|
In: Kalbotyra, Vol 72 (2019) (2019)
|
|
BASE
|
|
Show details
|
|
|
| 19 |
O constituinte-QU in situ e os efeitos de Common Ground no português brasileiro infantil / The WH-Constituent in situ and the effects of Common Ground in Child Brazilian Portuguese
|
|
|
|
In: Revista de Estudos da Linguagem, Vol 27, Iss 2, Pp 839-884 (2019) (2019)
|
|
BASE
|
|
Show details
|
|
|
| 20 |
« Parlons d'autre chose » ; « Parlons d'autre chose »: L'impact du Common Ground sur les sourires des interlocuteurs lors des transitions thématiques
|
|
|
|
In: JAD - Journée Annuelle des Doctorants ; https://hal.archives-ouvertes.fr/hal-01953892 ; JAD - Journée Annuelle des Doctorants, Dec 2018, Aix-en-Provence, France (2018)
|
|
BASE
|
|
Show details
|
|
|
|
|
