One of the complexities in writing assessment relates to raters who affect the validity of the writing assessment significantly. In fact, due to the high stakes of writing scores, it is very important to identify and reduce the sources of errors especially rater errors. Therefore, present study was conducted to seek for any possible changes in the rating behavior of the raters when they face different proficiency levels of the student writers. For this aim, ten experienced raters were asked to verbalize their thoughts while rating four essays, two of them belonged to two BA students and the other two were written by two MA students in two different rating states (i.e. aware and unaware) by using Jacobs et al ESL Composition Profile (1981). Upon the content analysis of the think-aloud protocols (TAPs), a body of 35 types of rating behaviors emerged. Results of the t-test showed that there was a significant difference between the mean scores of the raters assigned to BA texts in aware and unaware states, but no significant difference between the mean scores of the raters given to MA texts was found in aware and unaware states. Findings of TAPs were also discussed in the two rating states. Upon the analysis, there were differences in the way the raters approached the rating task in the two rating situations. The findings of the T-test along with those of TAPs confirmed that the way raters perceive the writing ability of the writer affect their rating both quantitatively and qualitatively. It also warns us against awkward interpretations of the students’ writing proficiency level based on a single score; they are not true representatives of the cognitive processes and the construct the raters have in their minds. Students’ proficiency level differences were also found to be effective in the raters’ ideologies about rating learners’ writing tasks. Pedagogical implications of the results for the rating practice of the Iranian EFL raters are discussed.