Nasim Ghanbari

Performance assessment has been very problematic and a cause of concern for a long time. Outcomes significantly depend on the raters, so rater-related factors considerably affect the evaluation of writing. So, the rater and the rating scale are two influential components of writing assessment which interact with each other to reach a final outcome. Hence, this study aims to identify and classify raters’ behavior in an EFL context when using two analytic and holistic rating scales in writing assessment. So, nine expert raters were trained and then asked to use think-aloud protocols when rating student essays by applying the analytic scale of Jacobs et al. ESL Composition Profile (1981) and IELTS holistic scale. When the think-aloud protocols were analyzed, two themes appeared that showed the raters’ behaviors toward the rating scales. Using the holistic scale, the raters read the text first to get an overall impression. Then assess the text based on their own criteria. After that, they refer to the scale for scoring and at last they provide evidence for their scoring. Using an analytic scale, the raters first scan the text for surface features, second Read the text for initial impression, third read each component and its descriptor for scoring and finally, provide evidence for the scoring. Then, the raters’ behaviors were classified so that the researcher could better understand the process of rating. The study presents a number of implications for EFL rating writing and rater training programs.