TESL Canada Conference 2009

TESL Canada 2009

I can’t believe the Teachers of English as a Second Language (TESL) Canada 2009 Conference that was just held in Banff from October 1 – 3 is over. We’ve been planning it for a year and a half, and now we can sit back and reflect on a conference well done. There were a few glitches at the start, but by the end, everything seemed to smooth itself out.

This year, I was on the planning committee working to help organize the Exhibition Hall with the publishers’ displays. This year there were 44 tables in the Exhibition Hall, with all the major ESL publishers doing business in Canada being represented. I was quite proud of us to be able to pull this off and put it together. I’d just like to say thanks to all of the publishers and people I worked with to make this happen. You are an amazing group of people!

On the academic side, I was involved in three events. The first event was a symposium on “Supporting Transitions: Assessment, Literacy and Academic Achievement”. I had the honour of sharing the stage with Drs. Janna Fox, Liying Cheng, and Hetty Roessingh as well as another PhD student like me, Christine Doe. I felt bad because I had to run in and out of the symposium (I was helping to simultaneously set up the publishers’ displays), so I missed the discussion part of the symposium. However, I’m posting a copy of my presentation to my webpage for people to have a look at.

The next event that I was involved in was the first ever poster presentations at a TESL Canada conference. I was very proud to be part of this inaugural event. My poster topic was on university level writing competence, vocabulary, and academic success, and it mirrored much of what I talked about in the symposium. You can have a look at my poster below.

Finally, I was involved in the first ever panel to bring together coordinators, managers and directors of EAP programs at the Universities of Calgary, Alberta, Athabasca and Lethbridge. It was very exciting to see what is going on at our various programs, and I’m hoping to do this again at the next ATESL conference in Edmonton, but this time also inviting the two new universities in Alberta: Mount Royal University and Grant MacEwan University.

Type Token Ratios and Success on the Effective Writing Test

I was just reading about the components of lexical richness in John Read’s book Assessing Vocabulary (Read, 2000) when I thought I’d write about one aspect of lexical richness in relation to the performance of NNES students on the Effective Writing Test at the University of Calgary. One of my key findings is that while about 70% of native English speaking (NS) students pass this test on their first attempt, only 23% of NNES students whose first language is of East Asian origin (Chinese, Japanese, Korean, Vietnamese, Laotian) actually pass on their first attempt. Obviously something is affecting the scores of these NNES students. Roessingh’s work (2008) with NNES high school students looking at the role vocabulary plays as an underlying variable in determining success on the written response component of the Grade 12 English 30-1 Diploma examination is one of the factors that has led me to analyse the vocabulary use of novice undergraduate writers at university.

One aspect of my analysis is the type to token ratios of NS versus NNES students on the test. Type to token ratios look at the number of unique words students use in comparison to the total number of words that they use. In his book, Read (2000) explains how students with a high type to token ratio use a variety of different words in their writing. Students with lower type to token ratios are using a limited number of words repetitively in their writing. Through the use of low frequency words, synonyms, hyponyms, and specific vocabulary items, good writers are able to tap into their larger vocabulary knowledge in order to convey a more precise meaning. Another term for type to token ratios is lexical variation.

Looking at my own research into lexical variation (please see the chart accompanying this blog), I found that NS students wrote significantly shorter essays than NNES students. However, the number of unique words employed by each of these groups was statistically the same. This resulted in NS students having a higher type to token ratio than their NNES counterparts. In other words, the NS students were employing a greater range of expression than the NNES students in their essays, and this significant difference may be one of the factors contributing to the low success rate of NNES students on the Effective Writing Test.

It was interesting to see that the NS students were writing shorter essays with more variation than the NNES students. In addition to being repetitive, another reason behind why the NNES students were writing longer essays includes the necessity of using circumlocution when a precise term wasn’t lexically available. More words are needed than necessary to express ideas due to a lack of vocabulary. However, the NS students typically have access to a much larger lexicon, enabling them to employ greater lexical variation in their writing.

On the Effective Writing Test rubric, repetitious diction is one of the aspects of word use that markers are evaluating. While circumlocution isn’t overtly marked according to the rubric, the use of “too many words” is a key component of the word use category. This is what drew me to consider the lower type to token ratios as being a possible underlying factor to the overall quality of undergraduate compositions.

These findings seem to mirror the findings of Cheryl Engber (1995) who was looking at the relationship between lexical proficiency and reader judgments of the overall quality of timed essays. Engber found that reader judgements of the overall quality of these compositions did reflect lexical variation. This leads me to conclude that a similar effect may be taking place in the Effective Writing Test, with the lower lexically varied essays of NNES students having less success than the more lexically varied essays of their NS counterparts.

Some References:

Engber, C. (1995). The relationship of lexical proficiency to the quality of ESL compositions. Journal of Second Language Writing 4(2), 139-155.

Read, J. (2000). Assessing Vocabulary. Cambridge: CUP.

Roessingh, H. (2008). Variability in ESL Outcomes: The Influence of Age on Arrival and Length of Residence on Achievement in High School. TESL Canada Journal 26(1), 87-107.

Creating a corpus of first year university academic writing

Preparing my corpus

This past Friday I presented at the British Columbia Teachers of English as an Additional Language (BC TEAL) conference. The topic was “Comparing non-native and native English undergraduate vocabulary in writing”. The first part of my talk dealt with the creation of the corpus that I used for my analyses.

The focus of my presentation was on what lexical frequency based analyses reveal about active vocabulary breadth of knowledge in novice native English speaking (NS) and non-native English speaking (NNES) undergraduate writing. By novice, I mean I am looking at first year students at the University of Calgary who have not yet passed the Effective Writing Proficiency Requirement (http://www.ucalgary.ca/efwr/requirement).

In order to investigate this question, I’ve gathered a corpus of writing, which I’m calling the Effective Writing Corpus. The writing samples in the corpus come from the Alberta Universities’ Writing Competence Test, also called, the Effective Writing Test (EWT). The EWT is a test administered to first year university students at the University of Calgary, the University of Lethbridge and Athabasca University. The test is designed to look for university level writing competence, and it is administered to all students who are entering university with less than a score of 75% on the English Language Arts 30-1 Diploma exam, or less than a blended grade of 80% on the blended grade of the diploma exam and the class score (50-50 split). Students who enter university with higher scores than these are exempt from the EWT. There are also other ways students are exempt, such as achieving a score of B- in a first year English course (http://www.ucalgary.ca/efwr/requirement/exemptions). In total the test is sat approximately 2250 times each year, with some of those sittings being repeated attempts to pass the test by the same students.

The EWT itself takes the form of a persuasive or expository essay answering one of four questions. These questions tap on a general body of knowledge, and no specialized knowledge is needed to answer the questions. An example of a question on the EWT might be along the lines of “Should the Government of Alberta institute mandatory physical education courses from kindergarten to Grade 12?” The essay answer written by the student should be around 400 words, and the markers are looking for university level writing competence. Some of the key points markers pay attention to include logical arguments, clear organizations, well developed paragraphs, well constructed sentences, accurate word use, and correct grammar, spelling and punctuation. English language dictionaries are permitted in the test, and the students have two and a half hours to complete their essays.

The corpus I am building focuses on the academic year of 2003/2004. Out of the approximately 2250 tests that were written that year, 561 NS students and 184 NNES gave permission for their tests to be used for research purposes. This is approximately 33% of the total amount of tests written in that year. Out of the NNES papers, 40 different languages were represented in the raw data. Out of these 40 languages, by far the greatest numbers of students had Chinese, Arabic, Spanish, and Punjabi as their first languages. Chinese was the largest group of all NNES students.

Breaking the students down into their constituent first languages reveals some interesting results in terms of performance on the EWT. 70% of all NS students who write the EWT pass on their first attempt. If we look at all the NNES students, except for those whose first language is of East Asian origin, 47% of NNES students (minus East Asian languages) pass the EWT on their first attempt. Finally, if we look only at students with a first language originating in East Asia, only 23% of those students pass the EWT on their first attempt. It is also interesting to note, that at the end of the academic year, there are about 700 students who still have not completed the Effective Writing Requirement. out of those 700 students, approximately 90% (630) are NNES. If approximately 75% of NNES students are of East Asian origin, that means about 470 NNES students of East Asian origin are still struggling to complete the Effective Writing Requirement by the end of the school year, and face being blocked from registering in their second year classes.

It is because of the struggles NNES whose first language is of East Asian origin face in passing the EWT that I have decided to focus on this group of students for my study. By focusing on this group of students, 79% of the papers in the NNES sub-corpus are written by students with Chinese (Cantonese and Mandarin) as their first language. The rest of the NNES sub-corpus is made up of Korean, Vietnamese, Japanese and Laotian. The NNES students have varying lengths of residence in Canada, ranging from 14+ years, 10-13 years, 7-9 years, 4-6 years, and less than 3 years. Each of these cohorts contains between 11 and 20 students.

The two sub-corpora (NS and NNES) also revealed some differences in faculty enrolment and topic choice between the two groups of students. The top three faculties of enrolment for NS students at the time of writing were Communication and Culture, Science, and Social Science. The top three faculties of enrolment for NNES students at the time of writing were Social Science, Science and Engineering. The top three topics for NS students were Physical Education, Computers, and Urban Growth. The top three topics for NNES students were Physical Education, Computers, and Being Ready for the Workforce.

Before I could begin my analysis, I had to prepare the raw data. The EWT is a handwritten test in official University of Calgary exam booklets. All the tests were typed and converted into text files for computer storage and analysis. As the papers were being typed, they were corrected for spelling, with spelling errors being noted on the original raw data. Proper nouns, such as of people and places, were recategorized into the first one thousand most frequent words of English. Semantic and derivational errors were also recategorized into the first one thousand most frequent words of English. Doing this prepared the data for linguistics analysis using various tools found on the Compleat Lexical Tutor website (Cobb, 2009).

The U of C Graduate Conference

This is the poster I'm presenting at the U of C Graduate Conference this May. I'm basically looking at the breadth of productive vocabulary in the writing of native English speaking and non-native English speaking first year university students. For the conference, I've had to weave in the themes of innovation and sustainability as well.

Possible Vocabulary Errors in Academic Writing

Keeping in mind that I am looking at advanced users of English, all of whom have passed the English language proficiency requirements to enter the Univeristy of Calgary, here is a chart to go with a post I made a while ago when I was trying to think of all the different possible errors that can be associated with vocabulary in academic writing. I think I have basically narrowed it down to six categories: meaning, appropriacy, derivation, form, omission, and style. Out of those six categories, I think five of them have the greatest impact on the quality of student writing: meaning, appropriacy, derivation, form, and style. I'm thinking of leaving out omission from my final list when I begin to comb through the writing samples in my thesis project. I can then read through each of my writing samples, tagging the samples for five categories of vocabulary error. My plan to then to see if any patterns arise from the samples as a whole, and between the different groups in my corpus (native vs. non-native English speakers, different groups of NNES based on length of residence). I finally think I'm ready to start. Already, I'm fairly sure of what I'm going to find. So far, my NNES papers show are demonstrating a lack of depth of knowledge of vocabulary. The students seem to have a breadth of knowledge that stretches to about 11,000 word families of active vocabulary usage (compared to 17,000 for the native English speakers), but the problems lie in the categories of appropriacy, derivation, and form - in particular choosing the correct part of speech.

I'll keep you posted as to what I find. Enjoy my chart of possible vocabulary errors in academic writing.

P.S. It's May 4th. I just added spelling errors to my chart - I've decided orthography is an important part of vocabulary knowledge as well.

What University Student Corpora Reveal About Vocabulary

One of the best parts of the TESOL Convention this year in Denver was the Doctoral Forum. The Doctoral Forum is a chance for doctoral students from all around the world to spend a day together presenting and getting to know one another and what sorts of doctoral research are going on around the world. This year the Doctoral Forum consisted of two panel discussions, two poster sessions (with a total of almost 60 posters being presented), and a networking session with professor-mentors from a variety of different universities from Temple University Japan, to the University of Texas to my own University of Calgary (my supervisor, Dr. Hetty Roessingh was one of the mentors).

All in all, it was a really great time. I did a poster presentation on my research in progress, and it was a great opportunity for me to focus my thoughts and to pause and think about where I am currently in my thesis research project. This poster is the result of that pause, and I even won second prize for the best poster at the Doctoral Forum. I was really pleased, but I realised that I still have a lot of work to do before I get finished.

Anyway, here is my poster - enjoy!

Lexical Error in Novice Academic Writing

Lately, I have been thinking quite a lot about what can possible go wrong with someone’s vocabulary usage when they are writing an academic essay. I’ve started to keep a running tally of all the errors I can think of, and this is my list so far. I started off by looking at the detailed marking code for the Effective Writing Test (http://www.efwr.ucalgary.ca/markingcode), and I went on from there. I think some of the items in the list might overlap, but I think I’m off to a good start.

Semantic error
A word is used that had the wrong meaning for the context.

Miscollocation
Words are used in inappropriate combinations, as in the difference between a torrential rainstorm and a torrential snowstorm*, or highly educated vs. greatly educated*.

Topic constraints
Register constraints
Genre constraints
Certain words can only be used in certain situations, depending on the topic, register and genre.

Inappropriate synonym choice
Synonyms have different connotations. An example of this is the difference between Scott is a famous teacher at the University of Calgary, and Scott is a notorious* teacher at the University of Calgary. (or maybe I am notorious . . . )

Trite language
This involves the use of clichés and overused expressions such as rabbits are not a valid food choice because they are as cute as a button. Another example would be, it’s important to think outside the box when dealing with financial problems.

Excessive jargon
This occurs when overly technical and specialized vocabulary is used when writing for a general audience.

Pretentious word choice
Big words are not necessarily better words. Overly complicated words and language are not better than simple and precise more common words and language.

Immature word choice
This occurs when writing about daddies instead of fathers, bunnies instead of rabbits, and choo-choos instead of trains.

Over repetition
The same words are used over and over again. For example: Completing high school should be mandatory. Mandatory high school classes will keep young people off the streets. If high school becomes mandatory, students will learn more. If school is mandatory, society will benefit. If it is not mandatory, there will continue to be problems. That is why a high school education must be mandatory.

Artificial variation
Too many synonyms are used making the writing seem unnatural. For example, Cats are important pets for senior citizens. Without their kitties, many old people feel lonely. Once arriving in the golden years, a feline companion is a necessity. Without pussy cats, oldsters won’t have the same quality of life. Granny and grandpa can’t do without these mini lions and tigers.

Word form
The wrong part of speech is used. Scott is a success* teacher, instead of Scott is a successful teacher.

Derivational error
This occurs when words are put together incorrectly, usually with inappropriate affixes. for example: After completing my analization* of the problem, I realize there was no solution.

Inaccurate lexical bundling
Some words operate in lexical bundles that are fairly inflexible. For example: on another hand* vs. on the other hand.

Mixed metaphor
This occurs when two different metaphors are combined. For example: We need to stop swimming against the current and follow the herd.

Inappropriate metaphor
This often happens when metaphors are translated from the first language, but they don’t quite work in the second. For example: After making many mistakes the government took a different tunnel.

Omission
This occurs when the writing is lacking transitions and connectors.

Imprecision
This happens when a general type of word, such as people, humans, things and stuff, is used in place of a more specific word.

Omnibus words
This occurs when the writer tries to incorrectly bring together many different ideas into a single word such as factor, aspect, situation, or concept.

My New Favourite Book: Proust and the Squid

I just read an amazing book by Maryanne Wolf called Proust and the Squid. Basically the book is about the history of reading, how people learn to read, the implications reading has on people and the development of their brains, and what happens when people have trouble learning how to read.

While a lot of the book really struck me as I was reading it, one part in particular I found very interesting in relation to my work in the area of undergraduate writing competence. I often wonder about what are the implications of not being a competent writer in university. One issue is that students who are not competent writers are not going to be able to convey their thoughts in a precise and meaningful manner. However, Maryanne Wolf takes this one step further when she discusses the ideas of Lev Vygotski and how he believed that the act of writing does not just convey thoughts, but when spoken words and unspoken thoughts are put into writing, this act releases and, in the process, changes the thoughts themselves. He felt that as people learn to use written language more and more precisely to convey their thoughts, their capacity for abstract thought and novel ideas accelerates. In other words, there is “a germinating relationship between writing language and new thought” (pp. 65-66).

I really was intrigued by the idea of a generative relationship between word and thought and how the process of writing down thoughts leads people to refine those thoughts and to discover new ways of thinking (p. 73). Writing enables students to think of new things that they would not have thought of if they weren’t able to write with academic competency. Taken further, being a fluent writer is going to aid students in generating new thoughts they would never have had if they weren’t fluent. Based on Wolf’s interpretation of Vygotski, writing isn’t just about reporting what people think, it’s about generating new ideas through the act of writing.

The implications for my research would be that if students haven’t reached a certain level of undergraduate writing competence, it isn’t just about them not being able to convey their thoughts in a precise manner. They will be missing out on the generative nature of the writing act. The less able the students can write fluently, the less able they will be to come up with novel ideas as they write. It is the transition from laboriously reporting thoughts to creatively putting together thoughts as they write. This idea of a continuum from simply reporting thoughts to generating creative ideas has cast a new light on the Effective Writing exams I’ve been looking at in my doctoral research. There always seemed to be a certain panache to the papers that did very well on the exam. I am always struck by their novel way of putting together words and coming up with ideas. I am particularly struck by how easy it seemed for the writer to write compared to papers which did not pass. Thinking about it again, it looks like the better writers were coming up with creative responses to the essay prompts during the act of writing, while the poorer writers were perhaps painstakingly trying to write down what was in their heads, or they were going at it one idea at a time. Hmmm, maybe as I write, I’m generating new ideas . . . . Anyway, I’m going to have to think a bit more about this, but I’m starting to worry that the repercussions of not being a fluent writer are much greater that I first thought.

Typing Away

I can’t believe that it is already about two months since my last blog. It’s amazing how time flies by when you are working on your PhD. The latest thing that I have been focusing on is typing up hand written essays for my thesis project. So far, I have about 800 essays, with about 250 having been written by non-native English speakers, and another 550 written by native English speakers. All of the students in my study are first year undergraduate students. Right now, I am focusing on typing up the essays written by the native speakers. It’s so fascinating doing this because I am really entering into the mind of a first year undergraduate student. It’s been almost 20 years since I first went to university, and I think I had forgotten what it was like to be a first year student. What has struck me the most so far is the ease with which they express themselves. I guess that with almost 10 years of looking only at papers written by non-native English speaking students, it was surprising how fluid and unlaboured the writing of the native speaking students is. Although I am focusing on a comparison of the use of less frequent vocabulary items between the two groups of students, again and again, it is the use of the simple clear language used by the native English speakers that is standing out. In addition to that, I have noticed the use of lexical bundling and collocations that provide clarity and precision to the native English speaking writing that seems to be lacking the non-native English speaking papers. However, taken as single words, these are not low frequency lexical items. Instead, the native speakers seem to be putting together the high frequency vocabulary differently than the non-native speakers. Interestingly enough, it seems to be the non-native speakers who are trying to use the “big academic” words, and then slightly misfiring with the meaning, while it is the native speakers who are using a high frequency set of vocabulary, but in such as way to convey precise meaning. I’ll have to think more about this.

Anyway, that is what I am up to right now. I figure I’ll be typing away for the next month (hopefully I’ll be done by the end of February). As soon as I’m done typing up enough native English papers, I provide some examples to support my musings above . . .