Investigating the impact of automated instruments used for assessing the writing skill: Perspectives of language e-learners
Main Article Content
Abstract
The main goal of using Automated Writing Evaluation (AWE) instruments is to help language learners learn more quickly and more effectively. In addition to assigning writing-related assignments, these technologies give language learners a suitable forum where they can receive general comments on their written pieces. In the same vein as the benefits of using AWE instruments, this study aims to comprehend foreign language e-learners' perspectives regarding the impact of these tools in their learning environment. Sixty-seven (67) research participants, who are currently e-learning foreign languages at King Khalid University, participated in an online survey to achieve this fundamental goal. Their answers to the questions were compiled and presented using a quantitative method, and they serve as the foundation for the research results. Moreover, the data distribution was precisely computed using a descriptive statistics table. Nonetheless, the study's findings demonstrate that AWE tools come with suitable measurements that non-native English speakers might utilize to improve their writing skills. This is predicated on the scoring propensity and feedback propensity—two crucial aspects of these instruments. Additionally, the study shows that e-learners occasionally run into difficulties when using these resources, which are mostly remediable by the programmers of these applications as well as language teachers. Lastly, to achieve a balance in the appropriate integration and use of AWE tools in the language educational setting, this research recommends using these tools in addition to human feedback and instruction.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This open-access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.
You are free to: Share — copy and redistribute the material in any medium or format. Adapt — remix, transform, and build upon the material for any purpose, even commercially. The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
No additional restrictions You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
How to Cite
Share
References
Aysel Saricaogl, Zeynep & Bilkii (2021), Voluntary use of automated writing evaluation by content course students. Cambridge University Press Volume 33, pp. 265-265
Burstein, J., Chodorow, M. & Leacock, C. (2003) Criterion SM online essay evaluation: An application for automated evaluation of student essays. In Riedl, J. & Hill, R. (eds.), Proceedings of the Fifteenth Innovative Applications of Artificial Intelligence Conference. Menlo Park: AAAI Press, 3–10.
Burstein, J., & Marcu, D. (2003). A machine learning approach for identification of thesis and conclusion statements in student essays. Computers and the Humanities, 37(4), 455–467.
Chapelle, C. A. & Voss, E. (2016) 20 years of technology and language assessment in language learning & technology. Language Learning & Technology, 20(2): 116–128.
Chen, C.-F. E. & Cheng, W.-Y. E. (2008) Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning & Technology, 12(2): 94–112.
Cotos, E., Huffman, S., & Link, S. (2020). Understanding graduate writers’ interaction with and impact of the Research Writing Tutor during revision. Journal of Writing Research 12(1), 187-232.
Council of Europe (2001) Common European framework of reference for languages: Learning, teaching, assessment. Strasbourg: Language Policy Unit.
Dang, T. K. A., Nguyen, H. T. M. & Le, T. T. T. (2013) The impacts of globalization on EFL teacher education through English as a medium of instruction: An example from Vietnam. Current Issues in Language Planning, 14(1): 52–72.
Delsa Miranty, Utami Widiati. Pegem (2021).Automated Writing Evaluation (AWE) in Higher Education: Indonesian EFL Students’ Perceptions about Grammarly Use across Student Cohorts. Journal of Education and Instruction, Vol. 11, No. 4, 2021 (pp. 126-137).
Don Yao (2021), Automated Writing Evaluation for ESL Learners: A Case Study of Pigai System. The Journal of Asia TEFL Vol. 18, No. 3, Fall 2021, 949-958.
Dyson, A. H., & Freedman, S. W. (1990). On teaching writing: A review of the literature. Berkeley, CA: National Center for the Study of Writing.
El-Ebyary, K. & Windeatt, S. (2010) The impact of computer-based feedback on students’ written work. International Journal of English Studies, 10(2): 121–142.
Feng, H.-H. (2015) Designing, implementing, and evaluating an automated writing evaluation tool for improving EFL graduate students’ abstract writing: A case in Taiwan. Iowa State University, unpublished Ph.D. https://lib.dr.iastate.edu/etd/14824/
Ferris, D. R., Liu, H., Sinha, A. & Senna, M. (2013) Written corrective feedback for individual L2 writers. Journal of Second Language Writing, 22(3): 307–329.
Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32(4), 365–387.
Ghanizadeh, A., Razavi, A., & Jahedizadeh, S. (2015). Technology-enhanced language learning (TELL): A review of resources and upshots. International Letters of Chemistry, Physics and Astronomy, 54, 73-87.
Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. The Journal of Technology, Learning and Assessment, 8(6).
Grimes, D. C. (2008). Middle school use of automated writing evaluation: A multi-site case study (Unpublished doctoral dissertation). University of California, Irvine, CA.
Guo, L. (2020). A study of the application of Pigai software in teaching university-level English writing: Taking a freshman class in Chongqing as an example. Theory and Practice in Language Studies, 10(8), 925-958.
Hibert, A. I. (2019). Systematic literature review of automated writing evaluation as a formative learning tool. In Transforming Learning with Meaningful Technologies: 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, The Netherlands, September 16–19, 2019, Proceedings 14 (pp. 199-212). Springer International Publishing.
Hockly, N. (2019). Automated writing evaluation.
Ho, O. K., Yao, D., & Kunnan, A. J. (2021). An analysis of Macau’s joint admission examination-English. The Journal of Asia TEFL, 18(1), 208-222.
Hsu, C. K. (2015). Learning motivation and adaptive video caption filtering for EFL learners using handheld devices. ReCALL, 27(1), 84-103
Joshua Wilson, Cristina Ahrendt, Emily A. Fudge, Alexandria Gaysha Beard, Chares MacArthur (2021). Elementary teachers’ perceptions of automated feedback and automated scoring: Transforming the teaching and learning of writing using automated writing evaluation. Computer and Education, Volume 168.
Koh, C. (2016) Translating motivational theory into the application of information technology in the classroom. In Liu, W. C., Wang, J. C. K. & Ryan, R. M. (eds.), Building autonomous learners: Perspectives from research and practice using self-determination theory. New York: Springer, 245–258.
Koltovskaia, S. (2020) Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44: 1–12.
Lai, C., Yeung, Y. & Hu, J. (2016) University student and teacher perceptions of teacher roles in promoting autonomous language learning with technology outside the classroom. Computer Assisted Language Learning, 29(4): 703–723.
Lai, Y.-h. (2010) Which do students prefer to evaluate their essays: Peers or computer programs? British Journal of Educational Technology, 41(3): 432–454.
Laviolette, E., Polio, C. & Kahng, J. (2015) The accuracy of computer-assisted feedback and students’ responses to it. Language Learning & Technology, 19(2): 50–68.
Lee, I. (2020). Utility of focused/comprehensive written corrective feedback research for authentic L2 writing classrooms. Journal of Second Language Writing, 49, 1-7
Leki, I. (2006) “You cannot ignore”: L2 graduate students’ response to discipline-based written feedback. In Hyland, K. & Hyland, F. (eds.), Feedback in second language writing: Contexts and issues. New York: Cambridge University Press, 266–285.
Link, S., Dursun, A., Karakaya, K., & Hegelheimer, V. (2014). Towards better ESL practices for implementing automated writing evaluation. Calico Journal, 31(3), n3.
Link, S., Mehrzad, M., & Rahimi, M. (2022). Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement. Computer Assisted Language Learning, 35(4), 605-634.
Li, J., Link, S. & Hegelheimer, V. (2015) Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27: 1-18
Li, R., Meng, Z., Tian, M., Zhang, Z., Ni, C. & Xiao, W. (2019) Examining EFL learners’ antecedents on the adoption of automated writing evaluation in China. Computer Assisted Language Learning, 32(7):
Li, Z., Feng, H.-H. & Saricaoglu, A. (2017) The short-term and long-term effects of AWE feedback on ESL students’ development of grammatical accuracy. CALICO Journal, 34(3): 355–375.
Liao, H.-C. (2016) Using automated writing evaluation to reduce grammar errors in writing. ELT Journal, 70(3): 308–319.
Mackey, A. & Polio, C. (2009) Introduction. In Mackey, A. & Polio, C. (eds.), Multiple perspectives on interaction: Second language research in honor of Susan M. Gass. New York: Routledge, 1–10.
Norris, J. & Ortega, L. (2003) Defining and measuring SLA. In Doughty, C. J. & Long, M. H. (eds.), Handbook of second language acquisition. Malden: Blackwell, 717–761.
Parra G, L., & Calero S, X. (2019). Automated writing evaluation tools in the improvement of writing skills. International Journal of Instruction, 12(2), 209-226.
Palermo, C., & Wilson, J. (2020). Implementing automated writing evaluation in different instructional contexts: A mixed-methods study. Journal of Writing Research, 12(1), 63-108.
Plonsky, L. & Ziegler, N. (2016) The CALL–SLA interface: Insights from a second-order synthesis. Language Learning & Technology, 20(2): 17–37.
Polio, C. (2012) The relevance of second language acquisition theory to the written error correction debate. Journal of Second Language Writing, 21(4): 375–389.
Quinlan, T., Higgins, D. & Wolff, S. (2009) Evaluating the construct-coverage of the e-rater scoring engine. Princeton: Educational Testing Service.
Saricaoglu, A. (2019) The impact of automated feedback on L2 learners’ written causal explanations. ReCALL, 31(2): 189–203.
Saricaoglu, A., & Bilki, Z. (2021). Voluntary use of automated writing evaluation by content course students. ReCALL, 33(3), 265-277.
Shadiev, R., Hwang, W. Y., & Liu, T. Y. (2018). A study of the use of wearable devices for healthy and enjoyable English as a foreign language learning in authentic contexts. Journal of Educational Technology & Society, 21(4), 217-231.
Türk, E., & Erçetin, G. (2014). Effects of interactive versus simultaneous display of multimedia glosses on L2 reading comprehension and incidental vocabulary learning. Computer Assisted Language Learning, 27(1), 1-25.
Wang, P.-l. (2013) Can automated writing evaluation programs help students improve their English writing? International Journal of Applied Linguistics & English Literature, 2(1): 6–1.
Ware, P. (2011). Computer-generated feedback on student writing. TESOL Quarterly, 45(4), 769–774.
Wilson, J. & Czik, A. (2016) Automated essay evaluation software in English Language Arts classrooms: Effects on teacher feedback, student motivation, and writing quality. Computers & Education, 100: 94–109
Wilson, J., & Roscoe, R. D. (2020). Automated writing evaluation and feedback: Multiple metrics of efficacy. Journal of Educational Computing Research, 58, 87-125
Zhang, S. (2021). Review of automated writing evaluation systems. Journal of China Computer-Assisted Language Learning, 1(1), 170-176.
Zhu, M., Liu, O. L., & Lee, H. S. (2020). The effect of automated feedback on revision behavior and learning gains in formative assessment of scientific argument writing. Computers & Education, 143, 103668.