SLRF 2008

Jump to: navigation, search

The Role of Lexical Choice in Elicited Imitation (presentation)

Abstract: In recent years, elicited imitation (EI) has become a viable way to measure language proficiency (Vinther, 2002, Chaudron, Nguyen, Prior, & Kozok, 2005). But up to the present selection of items to be used in these tests has been based largely on sentence length and syntactic complexity. There has been little investigation into the effects of lexical choice on EI test items.

The purpose of this study was to investigate the effects of lexical density and lexical difficulty on the difficulty of EI items. Lexical density is the ratio of content words to the total words in the sentence (McCarthy, 1990). There are several ways of estimating lexical difficulty. The most common of these and the ones that we have chosen to use are frequency and morphological complexity.(Nation, 1991; Gardner, 2007).

The EI instrument used in the study consisted of 60 sentences, ten sets of six sentences each, varying in length from four syllables to 22 syllables. In each set of six sentences, length and syntactic complexity was kept constant while vocabulary difficulty was varied from easy to difficult. The instrument, along with a previously calibrated set of items, was administered to 185 adult ESL learners from a variety of native language backgrounds and English proficiency levels. Item difficulty was calculated using IRT analyses and a multiple linear regression analysis was used to examine the relative contribution of lexical density and lexical difficulty to item difficulty.

Some complex relationships were discovered between item difficulty and lexical density and difficulty. Implications of these findings as they relate to the design of EI tests will be discussed.


[Ray Graham, Ben Millard and Jeremiah McGhee]

Paper available in SLRF 2008 Proceedings here.

What makes test items difficult? - A syntactic, lexical, and morphological study of Elicited Imitation test items (presentation)

Abstract: In recent years there has been a renewed interest in using elicited imitation as a tool for examining oral language skills in second language learners (Chaudron, Prior, & Kozok, 2005; Erlam 2006). Some attention has been paid to the role of syntax in determining the degree to which learners’ errors during sentence imitation mirror those which occur during spontaneous speech. However, in the use of elicited imitation as a tool for measuring language proficiency, little work has been done to examine variables which may affect test design. Some of these include the mode of administration of items (delay or no delay; test for comprehension or not; timed or open-ended responses), the methods of scoring (four point scale, counting correct syllables; holistic scoring of quality of utterances; fluency measures), and the nature of the items themselves (item length, lexical choices, morphological forms, syntactic patterns). The focus of this paper will be on examining the nature of item difficulty—specifically, what are the factors which contribute to the degree to which learners can successfully repeat items.

In the current paper we report the results of a principal components analysis of approximately 150 Elicited imitation items administered to approximately 380 adult learners of English as a second language in an intensive English program. Item analyses were performed on these scores and reliability coefficients ranging from .96 to .98 were found. Using item response theory (IRT) analyses of the items we determined item difficulty scores for each item. We then identified the lexical, morphological, and syntactic features of each sentence, along with general measures of sentence length and complexity and performed a multiple linear regression analysis on the data to determine which factors were contributing to item difficulty. Implications of the results for test design will be discussed.


[Ross Hendrickson, Meghan Eckerson, Aaron Johnson and Jeremiah McGhee]

Paper available in SLRF 2008 Proceedings here.