| 1 |
Hybrid models of cognition: The influence of modal and amodal cues in language processing tasks
|
|
|
|
Abstract:
In recent years, a growing amount of evidence has been accumulated suggesting that at least some part of our cognition and especially language comprehension is embodied in actions, perceptions, and emotions, and is therefore multimodal in nature. While the debate in previous decades was focused on whether cognition indeed is embodied, today the discussion revolves more around the question when and how embodied representations are used and what their exact role is. The present dissertation is aimed at shedding light on this discussion, investigating the presence and the role of multimodal representations across different tasks and contexts. At first, a series of anagram-solving tasks investigating the influence of different modal cues on subsequent solving of anagrams of words associated with either the ocean (e.g., shark -> SARHK) or the sky (e.g., cloud -> CUOLD) was conducted. Combining a background picture depicting an ocean-sky scene with a shift of attention towards the upper half of the computer screen resulted in faster solution times for words associated with sky compared to words associated with ocean, while the reverse was true for a downward attentional shift. This finding was extended to emotional valence, using pictures either associated with a positive or negative emotional valence to prime words with a matching emotional valence. Indeed, anagrams were solved faster when the emotional valence of the picture matched the associated emotional valence of the solution word. Going back to the domain of vertical space, we tried to replicate the findings of the first set of experiments with another set of stimuli and the use of linguistic cues in the form of adjectives or sentences preceding the anagrams, paired with a vertical shift of attention. In contrast to pictorial cues, these linguistic cues did not influence solution times. In another set of anagram-solving experiments, we directly compared the influence of linguistic (amodal) and pictorial colour (modal) cues, using written colour words or coloured rectangles as primes for solution words associated with a certain colour (e.g., specific types of fruit or vegetable, such as “cherry”). These were solved faster when a matching colour cue was presented before the anagram, regardless of whether the colour cue was linguistic or pictorial. Combining both cues by showing a written colour word inside a coloured rectangle only facilitated anagram solving of anagrams when both cues matched the solution word, e.g. the word “green” written inside a green rectangle facilitating solution of an anagram for “cucumber”. Neither a symbolic, amodal colour word, nor a colour patch seem to be responsible for this match effect exclusively, but instead both cues seem to activate the same superimposed conceptual colour representation. In a last line of research, it was investigated in how far hemispheric differences come into play during embodied word representations. A divided visual field study by Zwaan and Yaxley (2003a), who found a match effect regarding visual-spatial relations between objects to be confined to the right hemisphere in a semantic-relatedness judgment task was replicated - with the addition of the factor response side. Word pairs were shown very briefly either to the left or right visual field in a vertical arrangement on the screen either matching or mismatching the canonical spatial relation between the word’s referents (“nose” being above “mustache” in a canonical view of a face, thus seeing “nose” written above “mustache” would be a match). In contrast to the original study there was no interaction between visual field and the spatial compatibility effect. Instead an interaction between response side and visual field and an additional main effect of match - independent of visual field - was found. This leads us to assume that multimodal concepts are not confined to either hemisphere but instead seem to be spread over large scale networks across the whole brain. Taking all of these results together, a hybrid-view of cognition seems to be the most fertile: superimposed conceptual representations seem to be at the core of semantic meaning, and can be influenced by both modal and amodal contextual information, with neither type of information exerting clear dominance over the other.
|
|
Keyword:
150; Grounded Cognition; Kognition; Kognitive Psychologie; Simulationsansatz des Sprachverstehens; Sprachverarbeitung; Sprachverstehen
|
|
URL: https://doi.org/10.15496/publikation-66983
http://nbn-resolving.de/urn:nbn:de:bsz:21-dspace-1256204
http://hdl.handle.net/10900/125620
|
|
BASE
|
|
Hide details
|
|
|
| 3 |
Perspectives on dog–human interactions ; Perspektiven von Mensch-Hund-Interaktionen
|
|
|
|
BASE
|
|
Show details
|
|
|
| 4 |
The role of occipital temporal cortex in the formation of face familiarity and identity
|
|
|
|
BASE
|
|
Show details
|
|
|
| 7 |
Wild jackdaws respond to their partner's distress, but not with consolation
|
|
|
|
BASE
|
|
Show details
|
|
|
| 9 |
Nichtbewußtsein : über die affektiv-digitale Transformation eines humanen Kernbereichs : ein erster Entwurf
|
|
|
|
In: <Titel nicht gefunden> (2020)
|
|
UB Frankfurt Linguistik
|
|
Show details
|
|
|
| 10 |
Human face recognition in horses: data in favor of a holistic process
|
|
|
|
In: ISSN: 1664-1078 ; Frontiers in Psychology ; https://hal.inrae.fr/hal-03151453 ; Frontiers in Psychology, Frontiers, 2020, 11, pp.1-7. ⟨10.3389/fpsyg.2020.575808⟩ (2020)
|
|
BASE
|
|
Show details
|
|
|
| 11 |
The importance of fast auditory processing for central learning mechanisms and related feedback adaptation processes to the auditory periphery ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 12 |
Neurocognitive functioning and health-related quality of life in adult medulloblastoma patients: long-term outcomes of the NOA-07 study
|
|
|
|
In: Journal of neuro-oncology. - 148, 1 (2020) , 117-130, ISSN: 1573-7373 (2020)
|
|
BASE
|
|
Show details
|
|
|
| 13 |
The importance of fast auditory processing for central learning mechanisms and related feedback adaptation processes to the auditory periphery
|
|
|
|
BASE
|
|
Show details
|
|
|
| 14 |
Theoretical and corpus-based investigation of the relation between sensorimotor processes and linguistic structure
|
|
|
|
BASE
|
|
Show details
|
|
|
| 15 |
Vermin, parasites, puppeteers - the contribution of antisemitic metaphors to social categorization
|
|
|
|
BASE
|
|
Show details
|
|
|
| 16 |
Different or the Same? Determination of Discriminatory Power Threshold and Category Formation for Vague Linguistic Frequency Expressions
|
|
|
|
BASE
|
|
Show details
|
|
|
| 17 |
Effects of system response delays on elderly humans’ cognitive performance in a virtual training scenario
|
|
|
|
BASE
|
|
Show details
|
|
|
| 18 |
Iconicity in Cognition and Communication
|
|
|
|
In: Historical Social Research, Supplement ; 31 ; 66-77 ; Models and Modelling between Digital & Humanities - A Multidisciplinary Perspective (2019)
|
|
BASE
|
|
Show details
|
|
|
| 19 |
Soziologie der Steinzeit - die Anfänge des Denkens von der Höhlenmalerei zur Hochkultur am Göbekli Tepe
|
|
|
|
BASE
|
|
Show details
|
|
|
| 20 |
Lenguaje y referencia ; Language and reference
|
|
|
|
In: Sophia : colección de filosofía de la educación. - (2019) , 141-166, ISSN: 1390-3861 (2019)
|
|
BASE
|
|
Show details
|
|
|
|
|
