Towards Adaptive CALL:
Natural Language Processing for Diagnostic Language Assessment
Many advances in computer-assisted language learning (CALL) require an increase in the technical knowledge about diagnostic assessment, student models, and natural language processing in order to design adaptive instruction. This conference brings together researchers and graduate students working to address questions pertaining to each of these complex areas.
The papers beginning on Friday afternoon discuss issues and projects attempting to connect computer-assisted language assessment to language instruction. The first two papers discuss Spanish and ESL test task design in view of practices and tasks used in teaching. The third paper presents a framework for connecting cognitive diagnostic assessment to second language learning contexts, and the fourth describes a materials development project attempting to create diagnostic testing and alternative learning paths in ESL materials to be published by a major international company. The final two papers of the afternoon discuss the nature and development of a student model which is intended to guide instructional decisions in Portuguese materials and the type of linguistic feedback that can and should be provided for the benefit of learners. All conference attendees are invited to a reception following the Friday papers to provide an opportunity for continued discussion among participants.
The papers on Saturday morning begin with a plenary by Professor Robert Mislevy, who is well-known for his work drawing on statistical methods and cognitive psychology to develop new ways of integrating work in assessment, student modeling, and natural language processing in order to make use of technology for learning. His plenary will provide insight into the powerful concept of adaptivity as it pertains to language assessment and learning, where relevant evidence about learners’ knowledge is typically obtained from analysis of their language. Refreshments will be provided before and after this presentation.
The following two papers in the morning session describe how analysis of learners’ language is used in ESL tests of writing and speaking. The final paper in the morning session discusses motivations and methods for using NLP in diagnostic ESL tests of writing. Box lunches will then be available for conference participants.
The Saturday afternoon session continues the theme of automated scoring of linguistic production with a paper discussing the links between scoring decisions and the test’s intended construct definition. The next paper discusses cognitive response theory as a source of insight for decisions about automatic scoring of test responses and the following one places the seemingly micro issue of automatic scoring of linguistic responses into the large context of validity argumentation.
The next four papers shift the focus from assessment to learning. The first one examines the extent to which existing research demonstrates that NLP has been used to promote effective second language learning. The following two discuss the use of data gathered to document learners’ performance during language learning tasks. The final paper describes research investigating second language acquisition over time through the use of NLP of learners’ writing and student models.
The contribution of each presenter promises to embroider aspects of our understanding of complex facets of CALL. Authors are invited to submit their papers immediately following the conference for publication of the proceedings on Iowa State University’s Applied Linguistics Website by the end of December 2007.
We are grateful for a grant from the TOEFL Grants and Awards Program at Educational Testing Service for making this conference possible. We also thank the Departments of English and World Languages and Culture as well as the Program in Linguistics at Iowa State University for their financial support.