نسیم قنبری

The present study reports the process of development and validation of a local rating scale in the Iranian EFL writing assessment context. To achieve this goal, the study was conducted in three distinct phases. In the first exploratory phase of the study, a questionnaire was administered to thirty experienced raters from ten major state universities in Iran. Later in the same phase, five of the raters participated in a think-aloud session which aimed to provide evidence about the results of the questionnaire. Development of a rating scale was triggered on the grounds that there was no explicit and common rating scale in use among Iranian EFL writing raters. Therefore, in the second phase of the study, fifteen of the raters were asked to verbalize their thoughts when rating five undergraduate sample essays. At the end of this phase, an empirical rating scale was developed. Finally, in the last validation phase of the study, ten raters were asked to rate a body of twenty writing essays based on the newly-developed rating scale. Next, eight of the raters participated in a follow-up retrospective interview to elaborate on the process of their rating and their perception regarding the functionality of the rating scale. The quantitative analysis of raters’ performance using FACETS showed high profile of reliability and validity for the new scale. However, the qualitative findings of the interviews spotted some problematic areas regarding the structure of the whole scale and the individual components. It is discussed that although the novelty of rating with a rating scale caused a conservative rating practice and homogenous performance among the raters, the detailed qualitative data revealed that the introduction of a common, empirically rating scale was quite welcome in the context. The findings are discussed in terms of their pedagogical implications in the context. The study also suggests that further validation studies of the scale in the context are deemed necessary.