| 21 |
Building the validity foundation for interpreter certification performance testing
|
|
Han, Chao. - : Sydney, Australia : Macquarie University, 2015
|
|
Abstract:
Thesis by publication. ; Includes bibliographical references. ; Chapter 1. Introduction -- Chapter 2. Building a validity argument for interpreter certification performance testing -- Chapter 3. An interactionalist approach to construct definition for English/Chinese interpreter certification performance testing -- Chapter 4. Profiling conference interpreting practice in China : a sequential-exploratory mixed-methods design study -- Chapter 5. The effects of interpreting task characteristics on the quality of simultaneous interpreting : a convergent parallel mixed-methods approach -- Chapter 6. Exploring the relationship between task characteristics, strategy use and performance in English-to-Chinese simultaneous interpreting -- Chapter 7. Investigating rater severity/leniency in interpreter performance testing : using multifacted Rasch measurement -- Chapter 8. Investigating score reliability in English/Chinese interpreter certification performance testing : a generalizability theory approach -- Chapter 9. Summary and conclusions. ; Interpreter certification performance testing (ICPT) has developed rapidly over the past decade. Yet there has been very limited discussion and systematic research conducted to enhance reliability and validity of high-stakes interpreter certification performance tests (ICPTs). This interdisciplinary mixed-methods research was therefore initiated to build theoretical and methodological foundations for rater-mediated ICPTs, with a special focus on test validation, construct definition, and rater/score reliability for English/Chinese ICPTs in China. ; Presented in a thesis-by-publication format, the research follows a multi-phase mixed-methods research (MMR) design, in which research results from a previous study inform and build to a subsequent study. ; To begin with, given the lack of guidance on rigorous validation of ICPTs, this thesis draws upon an argument-based approach to build a validity argument for ICPTs. The validity argument could serve as a roadmap to help testers collect validity evidence. Based on Interpreting Studies literature, two particular types of evidence are generally lacking: evidence supporting substantive score interpretations based on a strong construct theory, and evidence supporting test score generalizability, especially across raters. ; To help generate evidence that justifies the substantive score interpretations intended by certification authorities in China, an interactionalist approach to construct definition is therefore proposed and articulated for English/Chinese ICPTs. Essentially, the interactionalist construct model contends that performance consistency (i.e., interpreting performance) is as a function of context (i.e., characteristics of test tasks), trait (i.e., interpreting ability), and interactions between the two. The theoretical construct model gives rise to two research questions (RQ). RQ 1: What are the characteristics of interpreting tasks in the real-life practice domain in China? RQ 2: What is the possible interplay between characteristics of interpreting tasks, interpreting ability, and interpreting performance quality? ; To address RQ 1, an exploratory qualitative diary (n = 11) and a follow-up quantitative survey (n = 140) were conducted to generate empirical data that describe the characteristics of the interpreting practice in China. Main results include that the interpreters performed a greater variety of simultaneous interpreting (SI) tasks than previously thought, and encountered a number of prominent factors contributing to SI difficulty, such as fast speech rate (FSR) and strong accent (StrA). ; To investigate RQ 2, a factorial repeated-measures experiment was conducted. Specifically, informed by the diary and the survey findings, the experiment sought to address the interactions between SI tasks (characterized by FSR and StrA), strategy use (regarded as a crucial component of interpreting ability), and SI performance quality (measured by information completeness, fluency of delivery and target language quality). In the experiment, 32 interpreters were asked to perform English-to-Chinese SI in four manipulated tasks. A crossed measurement design was then implemented in which nine trained raters assessed each performance by each interpreter on each rating dimension. Results show that 1) the speed factor had a pattern of mixed impacts on information completeness, fluency of delivery and target language quality of SI performance, while the accent factor had a consistent pattern of detrimental impacts across the three dimensions; 2) the strategies of syntactic transformation and substitution were used most frequently. It also would appear that while the speed factor greatly influenced the use of the two strategies, the accent factor did not; and 3) there seemed to be a general trend that the more strategies were used, the better SI performance was. ; Finally, to help produce evidence supporting rater reliability and score generalizability (i.e., RQ 3), a methodological exploration was conducted to evaluate the utility of multifaceted Rasch measurement and generalizability theory in analyzing rater behavior, rater variability and its effects on score dependability. Data for the analyses were the rater-generated scores from the experiment. Results indicate that although the rating design produced reliable results, one of the raters was problematic, as s/he was not self-consistent, and provided significantly biased scores to a large proportion of the interpreters. The findings also show that increasing the number of raters and/or tasks would generally improve score reliability for each rating dimension, but the relative efficiency was different across the dimensions. ; Ultimately, the empirical and methodological findings would contribute evidence to the ICPT validity argument, and their implications on ICPT design and validation were also discussed. ; Mode of access: World wide web ; 1 online resource (xx, 272 pages)
|
|
Keyword:
generalizability theory; interpreter certification; multifaceted Rasch measurement; performance testing; simultaneous interpreting; Translating and interpreting -- Testing; Translators -- Certification; validity
|
|
URL: http://hdl.handle.net/1959.14/1068520
|
|
BASE
|
|
Hide details
|
|
|
| 22 |
Investigating rater severity/leniency in interpreter performance testing : a multifaceted Rasch measurement approach
|
|
Han, Chao. - : John Benjamins Publishing, 2015
|
|
BASE
|
|
Show details
|
|
|
| 23 |
Ten Years of Rich Internet Applications: A Systematic Mapping Study, and Beyond
|
|
|
|
BASE
|
|
Show details
|
|
|
| 24 |
Green supply chain performance measurement using fuzzy-ANP based balanced scorecard: A collaborative decision-making approach
|
|
|
|
In: Bhattacharya, Arijit, Mohapatra, Priyabrata, Kumar, Vikas orcid:0000-0002-8062-7123 , Dey, Prasanta Kumar, Brady, Malcolm orcid:0000-0002-4276-3976 , Tiwari, Manoj Kumar and Nudurupati, Sai S. (2014) Green supply chain performance measurement using fuzzy-ANP based balanced scorecard: A collaborative decision-making approach. Production Planning & Control: The Management of Operations, 25 (8). pp. 698-714. ISSN 1366-5871 (2014)
|
|
BASE
|
|
Show details
|
|
|
| 26 |
Determinanten der Lesekompetenz bei Jugendlichen nicht deutscher Herkunftssprache. Zur Bedeutung der Spracherwerbsreihenfolge im Falle von Mehrsprachigkeit
|
|
|
|
In: Jude, Nina [Hrsg.]; Klieme, Eckhard [Hrsg.]: PISA 2009 - Impulse für die Schul- und Unterrichtsforschung. Weinheim u.a. : Beltz 2013, S. 111-131. - (Zeitschrift für Pädagogik, Beiheft; 59) (2013)
|
|
BASE
|
|
Show details
|
|
|
| 27 |
PISA 2009 - Impulse für die Schul- und Unterrichtsforschung
|
|
|
|
In: Weinheim u.a. : Beltz 2013, 246 S. - (Zeitschrift für Pädagogik, Beiheft; 59) (2013)
|
|
BASE
|
|
Show details
|
|
|
| 28 |
Determinanten der Lesekompetenz bei Jugendlichen nicht deutscher Herkunftssprache. Zur Bedeutung der Spracherwerbsreihenfolge im Falle von Mehrsprachigkeit ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 29 |
PISA 2009 - Impulse für die Schul- und Unterrichtsforschung ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 30 |
Processes for Assessing Outcomes of Multi-national Missions (Processus d'evaluation des resultats de missions multinationales)
|
|
|
|
In: DTIC (2013)
|
|
BASE
|
|
Show details
|
|
|
| 31 |
Measuring Collective Intelligence in Human-Machine Systems
|
|
|
|
In: DTIC (2013)
|
|
BASE
|
|
Show details
|
|
|
| 32 |
Measurement of mental attention: Assessing a cognitive component underlying performance on standardized intelligence tests
|
|
|
|
In: Faculty of Social Sciences - Papers (Archive) (2013)
|
|
BASE
|
|
Show details
|
|
|
| 35 |
Efficient in-memory top-k document retrieval
|
|
|
|
In: http://goanna.cs.rmit.edu.au/~e76763/publications/cps12-sigir.pdf (2012)
|
|
BASE
|
|
Show details
|
|
|
| 36 |
Automatic refinement of patent queries using concept importance predictors
|
|
|
|
In: http://www.inf.usi.ch/phd/keikha/papers/fp099-mahdabi.pdf (2012)
|
|
BASE
|
|
Show details
|
|
|
| 37 |
Expertise "Erfolgreiche Sprachförderung unter Berücksichtigung der besonderen Situation Berlins"
|
|
|
|
In: Berlin 2012, 205 S. (2012)
|
|
BASE
|
|
Show details
|
|
|
| 38 |
Expertise "Erfolgreiche Sprachförderung unter Berücksichtigung der besonderen Situation Berlins" ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 39 |
Speech and oromotor outcome in adolescents born preterm: relationship to motor tract integrity.
|
|
|
|
In: The Journal of Pediatrics , 160 (3) 402 - 408.e1. (2012) (2012)
|
|
BASE
|
|
Show details
|
|
|
| 40 |
Rediscovering Missing Web Pages Using Link Neighborhood Lexical Signatures
|
|
|
|
In: http://www.cs.odu.edu/%7Emln/pubs/jcdl11/jcdl2011-link-neighborhood.pdf (2011)
|
|
BASE
|
|
Show details
|
|
|
|
|
