Methods of Scoring Elicited Imitation Items: an Empirical Study
Elicited imitation (EI) has received attention in recent years as a tool for measuring oral language development in adult second language learners (Chaudron, Prior, & Kozok, 2005; Erlam 2006). Some attention has been paid to the role of syntax in determining the degree to which learners' errors during sentence imitation mirror those which occur during spontaneous speech. However, in the use of elicited imitation as a tool for measuring language proficiency, little work has been done to examine variables which may affect test design. Some of these include the mode of administration of items (delay or no delay; test for comprehension or not; timed or open-ended responses), the methods of scoring (four point scale, counting correct syllables; holistic scoring of quality of utterances; fluency measures), and the nature of the items themselves (item length, lexical choices, morphological forms, syntactic patterns). The focus of this paper will be on examining methods of scoring items—specifically, whether it is better to score items by syllable or by word, whether items should be scored holistically or by length, whether items should be weighted according to difficulty, etc.
In the current paper we report the results of analyses of different approaches to scoring items. Using previously collected data bases as training items we developed algorithms for scoring items which gave optimal ratings as compared with OPI scores. Then using the algorithms thus developed, we applied the model to 60 new learners who had taken both the OPI and the Elicited Imitation test. In this session, we will discuss the results of the various ways of scoring EI tests.
[Deryle Lonsdale, Dan P. Dewey, Jeremiah McGhee, Ross Hendrickson and Aaron Johnson]
Approximating Oral Language Proficiency Using Elicited Imitation
In recent years there has been a renewed interest in using elicited imitation (EI) as a tool for measuring oral language development in adult second language learners. In earlier studies by the authors involving hundreds of students, correlations between scores from elicited imitation instruments and those of other measures of oral language proficiency have been encouraging. Certainly, it can be said that scores from elicited imitation tests can approximate scores derived from measures with greater face validity. If such is indeed the case, there are many uses to which EI tests can be put, other than as high stakes measures of oral proficiency. EI tests have several advantages over conventional ways of measuring speaking ability in a second language: 1) they can be administered by computer to large numbers of test takers in a single administration, 2) the entire test requires only a few minutes to complete, and 3) methods of automatic scoring using speech recognition are feasible. Thus EI can become an efficient way of estimating learners’ actual speaking ability in a second language.
In the current paper we present the results of a new study involving 160 adult learners of English as a Second language in which the ACTFL OPI was administered to a stratified random sample of 60 learners, ranging in English proficiency from novice to superior. All 160 participants had taken a 60-item EI test. Various statistical procedures were used to determine the degree to which Elicited Imitation scores could be used to predict OPI scores.
[C. Ray Graham, Ben Millard, Meghan Eckerson and Carl Christensen]