The impact of AI writing tools on the content and organization of students’ writing: EFL teachers’ perspective – Taylor & Francis Online

2.1. Teaching writing in the artificial intelligence era

The evolution of teaching writing in the current age of technological proliferation cannot be dissociated from the rapid advancement of digital tools and. As eloquently discussed in Haleem’s et al. (Citation2022) research, this shift is precipitated by an increasing fusion of digital utilities within the educational realm, effectively transforming traditional pen-and-paper methodologies into far more creative and dynamic pedagogical experiences. Garlinska et al. (Citation2023) discuss how virtual classrooms, online workshops, and cloud-based writing tools, for example, are revolutionizing the domain of writing instruction. These platforms offer functionalities such as real-time feedback, collaborative editing, and plagiarism checks. Such features not only enhance the writing acumen of students, as per Nykyporets’s (Citation2023) study, but also inspire them to engage in critical thinking and independent reasoning.

Furthermore, as illustrated by Bhutoria’s (Citation2022) work, AI-driven platforms and applications offer personalized learning opportunities for students, identifying their writing strengths and weaknesses. This empowers educators to tailor their teaching strategies to each student’s unique needs and preferences, leading to more effective learning outcomes (Dogan et al., Citation2023). In the same vein, Cahyono et al. (Citation2023) explored of the workable approach mediated by mobile technology in teaching writing uncovers another layer of tech-enabled pedagogical innovation. Students are emboldened to publish their work in public forums, nurturing their confidence and writing abilities. Such platforms also promote peer review and feedback mechanisms, fostering a sense of community and collaborative learning (Umamah & Cahyono, Citation2022). However, as the literature acknowledges, the digitization of writing instruction is not without its challenges. As Duncan and Joyner (Citation2022) articulated, educators must grapple with issues related to digital equity, privacy, and the potential for distraction. These issues underscore the necessity for an ongoing dialogue and proactive approaches in crafting pedagogical policies and strategies as we continue to navigate the teaching of writing in this artificial intelligence era.

2.2. The impact of AI writing tools on students writing quality

There has been burgeoning research on the impact of AI writing tools on students’ writing ability. While several studies indicate positive outcomes, others highlight potential negative impacts. On the positive side, AI writing tools such as Grammarly, QuillBot, Wordtune, and Jenni have been found to significantly improve students’ writing skills. These tools utilize advanced algorithms to identify common errors in grammar, punctuation, and syntax and provide suggestions to improve clarity and style. They also offer unique capabilities such as paraphrasing and refining sentences for enhanced effectiveness. A study by Tambunan et al. (Citation2022) showed that using Grammarly improved students’ grammar and punctuation. Grammarly’s AI analyzes the user’s text, provides real-time suggestions for enhancing grammar, spelling, punctuation, clarity, engagement, and delivery. This has the effect of turning the process of writing into a learning opportunity. QuillBot, another AI tool, specializes in paraphrasing, helping students avoid plagiarism while maintaining the original meaning of their content. Kurniati and Fithriani (Citation2021) found that QuillBot aided students in developing better paraphrasing skills, which is a critical competency in academic writing. WordTune, on the other hand, focuses on refining and improving the tone and style of the text. A study by Lam and Moorhouse (Citation2022) found that WordTune effectively helped students identify their writing weaknesses, thereby promoting self-assessment and learning. This tool goes beyond simple grammar correction and delves into the stylistic elements of writing. Jenni is another AI-based writing assistant that provides predictive text suggestions to help users compose emails, reports, articles, and other documents more efficiently. Although no specific research was cited in this regard, user feedback suggests a positive impact on writing productivity and creativity. GPT-3, developed by OpenAI, represents a significant advancement in language model technology. Capable of generating coherent and contextually relevant sentences, GPT-3 has been found to stimulate students’ creative and critical thinking (Mhlanga, Citation2023). It can be an excellent tool for students to experiment with different writing styles and ideas. These studies highlight the potential of AI tools in enhancing the writing abilities of students.

In terms of the content and the structure of writing, a number of research has shown that AI writing tools can significantly enhance the efficiency and quality of writing tasks. AI-based tools, like GPT-3 and its successor GPT-4, have been leveraged to suggest the next word or paragraph in a text, aiding authors in creating human-like compositions (Y. Liu et al., Citation2023). This process is facilitated by machine learning algorithms that predict text sequences based on the vast amount of data they have been trained on. Furthermore, there is ongoing research exploring the impact of these tools on different aspects of writing. For instance, a study found that AI writing tools like “AI KAKU”, a popular Japanese-developed English language learning tool, could potentially reduce cognitive barriers for English as a Foreign Language (EFL) learners when producing written text in English (Gayed et al., Citation2022). This is achieved by providing real-time translation and grammar suggestions, making the writing process less daunting. Another important application of AI in writing is its ability to expedite the literature review process, which deals with large volumes of partially structured (meta)data (Wagner et al., Citation2022). This is particularly valuable in fields like healthcare and social sciences, where researchers must often sift through enormous bodies of literature for their studies. In conclusion, the evolution and application of AI writing tools like have undoubtedly marked a significant stride in the world of writing and literature. These advancements hold substantial potential in not only streamlining the writing process but also in improving the quality and efficiency of written output.

Despite these advantages, certain studies have indicated potential drawbacks. For instance, AI writing tools may inadvertently promote over-reliance among students, as they might lean too heavily on these tools for correction without thoroughly understanding their mistakes. This dependency can stunt their natural learning process and development of self-editing skills. Iskender (Citation2023) provided a critical perspective on this matter, arguing that the use of AI writing tools could lead to diminished critical thinking skills if students become overly dependent on them. They expressed concern that students might prioritize quick fixes from AI tools over deeply understanding and learning from their mistakes, which would essentially negate the process of learning, growth, and development in writing. In addition to concerns about over-reliance, the impact of AI writing tools on creativity has also been questioned. Some educators worry that students might use these tools not just for refining their language, but also for generating ideas, potentially curtailing their creative thinking and originality (Johinke et al., Citation2023). Furthermore, Farrokhnia et al. (Citation2023) pointed out that while AI tools can improve certain writing skills, they may not be as effective in addressing higher-order writing elements, such as argument structure and coherence. These aspects require a deep understanding of the topic, logical thinking, and the ability to connect ideas, which are currently beyond the capacity of AI tools. In terms of tone and context, AI tools may also falter. These tools might not fully grasp the subtleties and nuances of human language and emotion, potentially leading to suggestions that may not be appropriate or accurate in a given context (Haleem et al., Citation2022). Finally, issues of digital equity have been raised. Not all students have equal access to these technologies, leading to potential disparities in learning outcomes. In their study, Mozumder et al. (Citation2023) noted that students without access to high-speed internet or modern digital devices might be unable to utilize these AI tools, widening the educational achievement gap. Therefore, while the application of AI in writing instruction has promising benefits, educators need to be mindful of these potential drawbacks and address them proactively in their teaching strategies.

3.1. Research design

To address the research questions, a qualitative approach was employed, specifically utilizing a case study methodology. The case study methodology, which typically involves conducting interviews and/or observations, enables the researcher to uncover important factors that emerge from the open-ended process (Yin, Citation2009). Through this methodology, we were able to explore in-depth the types of AI writing tools used by EFL teachers, as well as their perceptions about the impact of these tools on students’ writing quality, particularly in terms of content and organization.

3.2. Research participants

This study focused on gaining insights from EFL teachers experienced with AI writing tools, specifically their views on how these tools affect student’s writing quality, particularly in terms of content and organization. Four writing teachers were recruited from three different universities, one being a public institution and the other two, private. The selection was primarily based on their practical experience with AI writing tools in the classroom, accessibility, willingness to participate, and teaching expertise in advanced courses such as essay writing and argumentative essay writing. These advanced courses often demand a higher level of grammatical knowledge and a deeper understanding of discourse structure.

All the recruited teachers possessed at least three years of experience in teaching EFL and a minimum of one year’s experience in applying AI writing tools in their classrooms. These AI tools served as supplementary teaching resources, aiding in grammar checking, paraphrasing, plagiarism detection, generating content, and offering suggestions to enhance the clarity and coherence of students’ writing. This ensured the teachers’ experiences transcended mere theoretical knowledge, encapsulating practical applications of AI tools in real-life teaching scenarios.

Teachers’ classes varied in size, ranging from intimate, personalized instruction settings to larger, conventional classrooms. This range in teaching environments was intentional, allowing the study to account for various factors potentially impacting the efficacy of AI tools. These factors could include class size, available institutional resources, the composition of the student population, and individual teaching styles. Such a diverse selection of participants enabled a comprehensive understanding of EFL teachers’ experiences and perspectives while using AI writing tools.

Participants were designated as T1, T2, T3, and T4 to maintain confidentiality. The recruitment process began with contacting prospective participants via email, outlining the study’s purpose, their roles, data collection methods, and their rights, including the ability to withdraw from the study at any point without any repercussions. This ensured adherence to ethical research guidelines. They were also provided with an informed consent form to read, comprehend, and sign if they agreed to participate. Through this careful recruitment process, the study maintained the highest ethical standards while ensuring the inclusion of relevant experiences and perspectives. The demographic information of the participants is presented in Table 1.

3.3. Instrument

We used interviews as our research instrument to collect the required data for this study. This instrument allowed us to gather rich, qualitative data related to EFL teachers’ use of AI writing tools, and their perspectives on the impact of these tools on students’ writing quality- specifically, in terms of content and organization. To construct pertinent interview questions, we turned to several relevant pieces of literature. Works by Nazari et al. (Citation2021), Dale and Viethen (Citation2021), M. Lee et al. (Citation2022), and Zhao (Citation2022) served as valuable references. By undertaking an in-depth review of these sources, we focused on the topics and themes that aligned with our research objectives.

This thorough literature review helped us craft our interview protocol. It included specific questions such as “What AI writing tools have you used in your classroom?” and “In what ways have these tools impacted your students’ writing, specifically with respect to content and organization?” This interview guide allowed us to investigate both the practical applications and theoretical implications of AI writing tools in the EFL teaching context.

3.4. Data collection procedure

After designing the interview protocol centered on the types of AI writing tools utilized by EFL teachers and their views on the impact of these tools on student writing quality (specifically, content and organization), we organized interviews with each participant. These were scheduled according to their convenience and availability. Before starting each interview, we briefed the participants about the study’s objectives, the interview process, and our rigorous approach to data protection. We secured their informed consent to proceed, ensuring they comprehended their rights, including the freedom to withdraw from the study at any time. During the interview, we began with general questions about their experiences teaching EFL and using AI writing tools, thus fostering a relaxed environment conducive for open discussions. Gradually, we introduced more specific questions aligned with our interview protocol, delving into the particular AI writing tools they had employed and their impact on students’ writing quality, particularly in terms of content and organization improvements.

To address potential social-desirability bias inherent in self-reported data, we emphasized to the participants that there were no right or wrong answers and encouraged them to be as open and honest as possible. This was done to ensure the reliability and authenticity of the responses gathered. Each interview was recorded with the participant’s permission, to maintain accuracy in capturing their responses, an integral part of reliable transcription and subsequent data analysis. Note-taking during the interviews was also employed as a means to highlight significant points, capture nuanced details, and document immediate observations.

Upon the conclusion of each interview, we expressed our gratitude to the participants for their contributions, also providing them with an opportunity to pose any questions about the study or to add any further comments. Following the interviews, we carefully transcribed the recorded dialogues, transforming verbal insights into textual data. Accuracy during this stage is critical, as the transcribed data underpin our subsequent analysis. All collected data, encompassing the interview recordings, transcriptions, and notes, were securely stored, respecting participant privacy and enabling efficient data management for ensuing stages of data analysis and research reporting. Adhering to these practices ensured a comprehensive, ethical, and effective data collection procedure for our research on AI writing tools’ use and impact in EFL teaching.

3.5. Data analysis

We employed a thematic case study to analyze the qualitative data. As outlined by Braun and Clarke (Citation2006), the goal of this thematic analysis was to identify patterns and connections among themes extracted from the qualitative data, thereby addressing our research questions. We divided the analysis into five distinct steps.

In the first step, we thoroughly read the interview transcript multiple times to familiarize ourselves with its content. The second step involved developing preliminary codes that grouped related words and phrases in line with our research criteria. Following a detailed analysis of these codes, we then categorized each one. In this step, we also compared the results to identify any relationships or patterns. Finally, we established a set of categories from these grouped codes.

4.1. Types of AI writing tools use by the EFL teachers

Drawing from the data collated during the interview phase of our research, it was evident that the four EFL teachers under study employed a range of AI writing tools in their teaching practice. These tools were chosen and used based on their distinctive features and capabilities, aligning with the unique demands of their teaching environments and objectives. To provide a detailed yet succinct overview of these AI writing tools, we have organized this data into a clear and informative format. Refer to Table 2, which serves as a comprehensive catalog, capturing the variety of AI writing tools adopted by these EFL teachers. This table effectively highlights the names of the AI tools in use and their specific purposes, providing a comprehensive snapshot of the current landscape of AI utilization in EFL teaching.

The provided dataset in Table 2 reveals a notable pattern of how four teachers (T1, T2, T3 and T4), integrated Artificial Intelligence (AI) tools in their teaching methodologies. These teachers had shared a common interest in utilizing a combination of AI tools in enhancing students’ writing skills, hinting at the fundamental trend of embedding technology into education.

Two tools, QuillBot and Wordtune, had emerged as universally used across all four teachers. QuillBot, known for its proficiency in paraphrasing, was an important part of teaching students how to express the same idea in different ways, effectively aiding in the avoidance of plagiarism. Wordtune, on the other hand, was employed to improve sentence structure and clarity, making it an essential tool in teaching English and enhancing students’ written expression.

A second layer of commonality could be seen with Jenni AI and ChatGPT, which were employed by three out of the four teachers (T1, T3, and T4). Jenni AI, renowned for its topic modeling capabilities, assisted students in brainstorming and organizing their thoughts prior to writing, thereby encouraging critical thinking and structured writing. Chat-GPT simulated conversations, assisting students in improving their conversational writing skills, which was a crucial aspect of English learning in a globalized and digitally connected world.

While the aforementioned tools were the common threads woven into the teaching methodologies of these teachers, they also diversified their approach with specific tools unique to their classrooms. For instance, T2 used Copy.ai and Paperpal. Copy.ai generated creative writing samples, serving as examples for different styles of writing and thus, fostering creativity in students. Paperpal was utilized for its strength in grammar checks and providing real-time corrections, which was a vital practice in English learning. T3 employed Essay Writer, which helped students structure their essays logically and provided them with ideas for improvements, a tool that seemed uniquely suited to a classroom focused on academic writing. T4 also brought in Copy.ai and Essay Writer, leveraging these tools to show students how to craft compelling pieces of writing in various styles and formats.

When considering the demographic information of participants given in Table 1 and analyzing it against the teachers’ utilization of AI tools, several initial observations become apparent. It appears that the teachers with more years of experience in using AIWT, T2 and T3, utilize a broader set of AI tools. Notably, T2, who is the oldest, holds the highest qualification (a Ph.D.) and has the most teaching experience, utilizes unique tools like Copy.ai and Paperpal. Similarly, T3, who has comparable AIWT experience, also uses a unique tool, Essay Writer. This pattern suggests that more seasoned and experienced teachers in using AIWT may exhibit greater confidence in exploring diverse AI tools for varying educational purposes. While it is challenging to conclude any significant gender-based differences in AI tool usage from this dataset, it is worth mentioning that the only female teacher, T2, uses a unique set of tools that could be more attributable to her experience and qualifications than her gender. Furthermore, it is worth noting that the level of teaching experience and qualifications may influence the use of AI tools. T2, having the highest teaching experience, experiments with unique tools, suggesting that more experienced teachers could be more exploratory. Similarly, the teacher with a Ph.D. degree, T2 again, utilizes unique AI tools, hinting that the level of education might have some influence on the choice of AI tools. However, given the small dataset, these observations need to be treated as preliminary trends rather than definitive conclusions.

4.2. Teachers perspective toward the impact of AI writing tools

In this section, we delved into the findings from our interviews regarding the teachers’ perspectives on the impact of AI writing tools on students’ writing quality, specifically focusing on the aspects of content and organization.