Coursework‎ > ‎

Quantitative data and analysis

This assignment was the first I handed in for my PhD coursework. When I was planning my assignment, I already knew that my research would be mostly, if not completely, qualitative, so this was not an assignment that could contribute towards my thesis. I therefore decided to work on a topic that was of more personal interest. As a student of language assessment, I had always felt that my understanding of Rasch measurement was lacking. I asked my lecturer if I could write about reliability and that seemed acceptable to him (then).

It took quite a lot of time to write this paper, from looking for relevant books and articles to the actual writing. While I'm reasonably comfortable with quantitative methods, I have never received actual training in statistics, so the more mathematical materials, such as formulas, were intimidating. Different writers seemed to put things differently, so I often had to look for third or more sources of information to confirm my understanding. A lot of rereading was called for. I had to ensure that I did not get things wrong when paraphrasing. 

Writing the paper was nonetheless very rewarding, both intellectually and practically. I have had to explain to colleagues previously why Rasch was preferred over Classical Test Theory, but was never able to do so satisfactorily. The year after I handed in this assignment, I took Statistics for Language Testing, and my understanding of Rasch came in very useful then too. I have no doubt that I will have opportunities to apply Rasch in future research, if I continue working in this field.

Reading my paper again now, with other assignments I have since written in mind, I see that I did well in making sure I had a title and clear section headings. I was also able to paraphrase complicated concepts (relatively) simply and clearly, even though my lecturer felt it was still too jargon-laden. His suggestion that I relate Rasch to more commonly known statistical methods makes a lot of sense -- except that my knowledge of these methods is limited. Outside of the most common and basic methods taught in quant courses, my statistical knowledge is very testing-centred.

I found my supervisor's comments easier to take on board, probably because his expectations (as a language tester) were different from my lecturer's (as a corpus linguist). I have since made more determined attempts (perhaps not always successful) to 'synthesise and evaluate'; this requires real depth of knowledge and true familiarity that only time can provide, I now realise. My supervisor's comment about unpacking dense information such as in tables and diagrams is one that I take to heart also (they are not devices for 'saying' a lot without saying much at all!) His comment about mixing the two approaches taught me something new. Clearly, reading dozens of theoretical literatures is no substitute for actual experience. 

Overall, I see this as an experimental paper of sorts for me. While my supervisor commented some time later that this paper still remained within my comfort zone, I feel that it gave me an opportunity to stretch myself writing a technical paper, while extending existing knowledge further. Even though I'm unlikely to write anything similar for my thesis, this new expertise should allow me to better introduce Rasch measurement to other researchers and practitioners, which I very much hope to do in the near future.