MS Copilot in Academic Writing Instruction: Effects on Performance & Self-Efficacy in COM101/111 | Scholarship of Teaching and Learning | Seneca Polytechnic

In this project, Chia-Lin (Lance) Chang (Principal Investigator) and Plamen Kushkiev (Co-Investigator), faculty in the School of English and Liberal Studies, investigate the impact of generative AI — specifically Microsoft Copilot — on student writing development, with a focus on both writing self-efficacy and performance. The study also explores innovative instructional methods that integrate AI tools to enhance writing skills among college students, particularly within the context of Seneca Polytechnic’s COM101 and COM111 courses.

The project is guided by the following research questions:

1. How does the integration of Microsoft Copilot into writing instruction influence overall writing performance among students in COM101/111 courses at Seneca?
2. How does the integration of Microsoft Copilot impact students’ self-reported writing self-efficacy in these courses?
3. To what extent does writing self-efficacy predict writing performance among students, with and without the integration of Microsoft Copilot?

Data were collected from 427 students across 17 sections of COM101 and COM111, taught by eight instructors between September and December 2024. Participants were assigned to either an experimental group or a control group. The experimental group received explicit instruction and practice using Microsoft Copilot as a writing support tool, while the control group followed the standard curriculum without AI integration.

To measure self-efficacy, the Self-Efficacy for Writing Scale (SEWS) was administered in Week 1 and Week 13. Writing performance was evaluated using the first and final mandatory course writing assignments, which took place at the beginning and end of the semester, respectively. Data were analyzed using Multivariate Analysis of Covariance (MANCOVA).

The findings indicate that integrating Microsoft Copilot into writing instruction did not significantly impact students’ writing performance or self-efficacy, and students' self-efficacy did not significantly predict their writing performance. The project did, however, provide meaningful benefits for students, such as early exposure to generative AI within an academic context, enhancement of their digital literacy, and critical reflection on the writing process. The integration of Microsoft Copilot also promoted equity of access to writing support, empowered students to experiment with writing strategies, and opened important classroom discussions about the ethical and practical implications of AI in education.

Special thanks to participating faculty members and research assistants: Professors Christine Dalton, Dagmar Kulivova, Densie Doll, Ekjot Kaur, Janet Santagada, Kasia Kasztenna and Sonja Penzo and Research Assistants Annalisa Canale-Parola, Chun San Wong, Yelin Su, Saba Halabisaz and Kiran Tahir.