I've been thinking a lot in the past few years about the validity of instructor-assessed final grades in English for Academic Purposes courses and how evidence can be gathered to contribute to the validity of these types of grades. One on-going study that I've been involved with has been looking at the relationship between instructor-assessed EAP final grades and standardized English language proficiency test scores administered at the end of an EAP course. The first paper published out of this study is in the Autumn 2014 issue of the International Student Experience Journal (http://isejournal.weebly.com/), and the paper looks at the concurrent validity of instructor-assessed EAP final grades. Correlational analysis was used to compare the EAP final grades with TOEFL ITP scores that were gathered at the end of the semester. There were statistically significant moderate correlations, contributing to the idea of concurrent validity, but the was also enough of a divergence to point to meaningful differences in what the instructors were assessing and what the TOEFL ITP was assessing.
The article can be found in the current issue here: http://isejournal.weebly.com/current-issue.html
2 comments:
I'm a little surprised at the moderate correlation. I admit I haven't read the linked text yet, but I suppose it depends on what "EAP" skills are being assessed by these scores used since EAP measures much more than language-related skills and TOEFL does not.
Our recent program evaluation at UofT actually noticed a small predictive element to incoming IELTS scores and student behaviour in our program, but no statistical correlation in how our students performed in their undergraduate degree programs. We'll probably present on it at TESL Canada this year. See you there?
It was an interesting exercise to see how the numbers would correlate. It looks like both measures are generally in somewhat of the same constructs, but they are diverging in interesting ways, which underlines how the EAP scores are representing something different than the standardized scores. I'm hoping the funding will come through to expand on this study and incorporate other measures, including qualitative ones. Fun stuff!
Post a Comment