Category Archives: DESIGN CRITERIA

What goes into responsible language test design?

languageandcommunicationReturning to the still unresolved issue of how best to conceptualize test validation and validity, I attempt to answer this question in a special issue of Language & Communication that commemorates the work of the late Alan Davies. In particular, I argue that responsible test design encompasses ethicality and accountability, and is a conceptually clearer way of thinking about the quality of a language test.

Elsevier, the publisher of the journal, has generously, though for a limited period, provided unlimited access to the article that I contributed to this commemorative issue. The final published version of the article, “Does responsibility encompass ethicality and accountability in language test design?” is available until 17 December to anyone who clicks on the following link: https://authors.elsevier.com/a/1Vy-wzlItpy~5. No sign up, registration or fees are required – you can simply click and read.

Advertisements

Ethics in language testing, local practices, once more in the spotlight

This is to share two thoughts with you.

First, that it’s time for language test developers in South Africa to begin to evaluate the ethicality of their assessment practices with reference to international codes, and second, to consider what local and contextual conditions might further shape our design work.

ilta

Does the South African language testing community need a Code of Ethics? That need has just been brought into focus again by an invitation from Tineke Brunfaut, coordinator of a project of the International Language Testing Association (ILTA; http://www.iltaonline.com/) aiming to have the ILTA Code of ethics translated into more languages, to translate it into Afrikaans. Continue reading

Is a theory of applied linguistics necessary?

9783319417295Is a theory of applied linguistics desirable? And if so, is it possible? My new book, Responsible design in applied linguistics: theory and practice (2017; Springer) proceeds from the thesis that applied linguistics needs a theoretical foundation. It is indeed possible to delineate its work (and specifically distinguish it from linguistics). Providing it with a theoretical foundation might additionally yield new insight into the principles that underlie applied linguistic designs. Those designs we encounter as the interventions that we call language courses, language tests and language policies. Continue reading

Make the information available, then accountability will follow

Construct Refinement in Tests of Academic LiteracyAvasha Rambiritch of the University of Pretoria and I have just written a chapter for a book edited by John Read (Post-admission Language Assessment of University Students, Springer, 2016) that shows how making sufficient information available about the conception, design, development, refinement and eventual administration of a test of language ability — in other words “telling the story of a test” — is the first step towards ensuring accountability for such tests. The test in question, the Test of Academic Literacy for Postgraduate Students (TALPS), is used to determine the academic literacy of prospective postgraduate students. For the full reference, see the bibliography on this site.   Continue reading

Fairness and credibility in high stakes examinations

fairness_in_language_assessmentThe assessment of the 11 “home languages” at the end of secondary school in South Africa is patently unfair. That is the finding of a recent investigation that Colleen du Plessis (UFS), Sanet Steyn (NWU) and I report on in an article that has just been published on LitNet Akademies. The Grade 12 exit examinations are high stakes assessments, since the Home Language mark contributes disproportionately to the index on the basis of which access is granted to higher education (or entry into the world of work). They are unfair, because they are not equivalent: in some languages one has a much better chance to pass than in others. Continue reading

A dual accountability

fail-passThe solutions proposed by applied linguists are defensible in two ways:

First, the designs must be justified with reference to theory – not only linguistic theory, but also those found in other disciplines, like psychology and pedagogy, and across disciplines.

AccountabilitySecond, applied linguists are accountable to the public for the designs they produce. This is because their plans affect ever-increasing numbers of human beings. A bad and inappropriate language test may prevent one from earning a decent living or gaining legitimate access to a country or resources. An inadequate language course may stultify personal and professional growth. An ineffective and inappropriate language policy or arrangement can inhibit optimal performance within an institution or the workplace. Continue reading