About the project
This ongoing collaboration between InterMedia and Hong Kong Baptist University (HKBU) has the aim to develop and evaluate a student essay critiquing system for English as a second language. Essay writing is characterized as a design activity, involving composition and organization of essay themes (i.e. topics the essay is about).This is supported by a critic-based feedback component that uses latent semantic techniques (LSA) to analyze student essays and compute feedback based on how much of an assigned topic they have completed. LSA is a computational technique for comparing two segments of texts using matrix algebra, one created from the student text and the other from texts collected and stored on similar topics by domain experts (teachers and research assistants) to serve as reference. LSA works by constructing and comparing conceptual structures in a large semantic space.
EssayCritic, phase 1 is a pilot study at Skien vgs. in 2007, and EssayCritic2 is a design experiment at Sandvika vgs. in 2009.
Objectives
To incrementally develop and evaluate an LSA-based English essay critiquing system, compare our results with similar studies in Hong Kong, and to create the requirements for the next generation of collaborative essay writing support that takes empirical findings, socio-cognitive theories of learning, and the technological possibilities of a semantic web (domain-oriented environments and adaptive feedback) as the basis for design.
In EssayCritic2 we analyze the "evolution of knowledge artifacts," i.e. the process and outcome of the modification of a text in response to critique over time. When a text is incrementally modified and resubmitted it becomes larger (in number of words) and (hopefully) richer in number of ideas.
Theory and methods
Programming (system, LSA integration, multiple user interfaces), algorithm refinement (LSA), quantitative (questionnaire) and qualitative (participatory observation, video analysis, interview, text analysis) empirical methods. Application of theory from multiple fields: communication, intersubjectivity, design, social formation of mind, cognition, learning.
Partners
Hong Kong Baptist University (computer science department and language center)
Funding and timeframe
Funded by: Småforsk, UiO
Timeframe: 2004-2010
Related projects
There are many projects of this kind around (often aimed at product development), especially in the USA. At UiO the project is probably closest to the English Wiki and the special needs for language projects.
Outcomes
A running system in several releases with a web interface (student and teacher/adm interfaces) that have incrementally improved in speed (feedback turnaround time) and accuracy (essay theme detection) over time since the project started (2004). The system has been employed in different classroom settings (individual and cooperative use), in effect taking the unique teaching practices in Hong Kong and Norway into account.
Publications
1. Cheung, W.K., Mørch, A.I., Wong, K.C., Lee, C., Liu, J., Lam, M.H. (2007). Grounding Collaborative Learning in Semantics-based Critiquing. Int'l J. Distance Education Technologies, 5(2), pp. 40-55. (PDF)
2. Kelvin C.K.Wong, Fion S.L. Lee, Cynthia F.K. Lee, William K.W. Cheung, Anders I. Mørch and Jiming Liu.(2007). A Pilot Study on the Impact of the Web-based Essay Critiquing System on Writing at the Tertiary Level. Proceedings of the 2007 International Joint Conference on e-Commerce, e-Administration, e-Society, and e-Education, Hong Kong, 2007. (PDF).
Masters theses and reports
1. Otnes, J.A. (2008). EssayCritic: A Pilot Study of a Computer Supported Essay Critiquing System. Master's thesis, Department of Informatics, University of Oslo, 02/08. (PDF)
2. Möllenkamp, E. (2010). From talk to text production with EssayCritic: Scaffolds in the early phases of the writing process. Master's thesis, TOOL master program, Department of Informatics, University of Oslo, 05/10. (PDF)
Presentations
Presentation at Interactive Media congress, University of Medellin, Columbia, October 5, 2011 (PDF)
Presentation at PhD seminar for Nordic PhD students, March 2, 2011 (PDF)
Presentation at Faglig-Pedagogisk dag UiO, Jan 8th, 2010 (in Norwegian) (PDF)
Demo of system created at HKBU
Video of design experiment in Norwegian high school with English subtext: View video