Human-AI collaboration patterns in AI-assisted academic writing

ABSTRACT Artificial Intelligence (AI) has increasingly influenced higher education, notably in academic writing where AI-powered assisting tools offer both opportunities and challenges. Recently, the rapid growth of generative AI (GAI) has brought its impacts into sharper focus, yet the dynamics of its utilisation in academic writing remain largely unexplored. This paper focuses on examining the nature of human-AI interactions in academic writing, specifically investigating the strategies doctoral students employ when collaborating with a GAI-powered assisting tool. This study involves 626 recorded activities on how ten doctoral students interact with GAI-powered assisting tool during academic writing. AI-driven learning analytics approach was adopted for three layered analyses: (1) data pre-processing and analysis with quantitative content analysis, (2) sequence analysis with Hidden Markov Model (HMM) and hierarchical sequence clustering, and (3) pattern analysis with process mining. Findings indicate that doctoral students engaging in iterative, highly interactive processes with the GAI-powered assisting tool generally achieve better performance in the writing task. In contrast, those who use GAI merely as a supplementary information source, maintaining a linear writing approach, tend to get lower writing performance. This study points to the need for further investigations into human-AI collaboration in learning in higher education, with implications for tailored educational strategies and solutions.


Introduction
Given the rapid evolution of technology and its infusion into various sectors of life, it is difficult to ignore its undeniable impact on educational paradigms (Ashour 2020;Buhl-Wiggers, Kjaergaard, and Munk 2023).Academic writing, a fundamental skill across educational systems, has experienced significant transformation due to these technological innovations.In fact, academic writing has been influenced by the emergence of various digital tools designed to assist scholars in their research, writing, and composition processes (Schcolnik 2018;Strobl et al. 2019).These tools, ranging from digital libraries and online collaboration platforms to specialised writing software, have transformed how academic writing is produced and disseminated (Strobl et al. 2019).
Although technology-assisted writing or intelligent writing assistants are not new concepts (e.g.O'Neill & Russell 2019; Zhang et al. 2016), the recent advancements in artificial intelligence (AI), particularly in the domain of natural language processing (NLP), mark a significant evolution in this field.
Deep learning models, a subset of machine learning, have shown immense promise in understanding and generating human-like text, which has a considerable impact on academic writing (Yan 2023).OpenAI's GPT (Generative Pretrained Transformer) series is a prime example of such advancements.The ChatGPT variant, in particular, offers a spectrum of capabilities, from responding to queries to content generation (Dwivedi et al. 2023).Although the potential of these technological breakthroughs has been shown to enhance learning experiences, nurture creativity, and streamline knowledge management, there are growing concerns about their implications for academic integrity and the possible erosion of academic writing skills (Kasneci et al. 2023;Lund et al. 2023).
The integration of state-of-the-art AI-assisted writing assistants into the academic writing process represents a paradigm shift.These tools not only provide assistance in drafting and revising text but also in conducting literature reviews and synthesising information, which are critical components of scholarly writing.However, this convenience comes with its own set of challenges.The ease of generating coherent and sophisticated text using AI can potentially lead to over-reliance, raising questions about the originality and authenticity of scholarly work.Furthermore, the use of AI in academic writing necessitates a reevaluation of pedagogical approaches to ensure that students develop critical thinking and analytical skills, rather than merely relying on AI for content generation.
The importance of academic writing proficiency has always been a consistent requirement, especially for doctoral students, and focus of discussion in higher education research, predating the widespread use of generative AI (Caffarella and Barnett 2000).Doctoral students, engaged in the pursuit of advanced research, must possess the capability to effectively communicate their research findings.This involves not only the clear articulation of these findings but also the synthesis of existing literature and the generation of new insights.The ability to write academically is not merely a functional skill but a critical component of a doctoral student's intellectual toolkit, enabling them to contribute meaningfully to their respective fields..With the emergence of technological advancements like generative AI, there are potential unknown effects on the quality and integrity of doctoral students' academic writing, raising both opportunities and challenges (Dwivedi et al. 2023;Kishore et al. 2023).While AI, particularly in its generative form, does not possess the ability to fully synthesise literature or independently engage in critical writing, it has shown considerable proficiency in aiding these processes.Specifically, generative AI can assist by aggregating and summarising relevant literature and generating written content based on specific prompts.This capability, however, is supplementary in nature; it serves to support and streamline the initial stages of academic writing, such as literature review and ideation, rather than replace the nuanced, critical thinking and analytical skills required for comprehensive synthesis and critical writing.These latter skills remain distinctly within the human domain, necessitating a depth of understanding and intellectual engagement that AI has yet to replicate.Furthermore, unlike other technological innovations, generative AI possesses the unique ability to respond according to previous engagements between the AI mechanism and its human counterparts.Accordingly, assessing how academic writing processes are shaped in human-AI collaboration is becoming increasingly essential.
In light of the pressing importance of understanding human-AI collaboration in academic contexts (Järvelä, Nguyen, and Hadwin 2023), this paper investigates the dynamics between doctoral students and AI-assisted writing tools in their academic writing tasks.This research diverges from the conventional approach of assessing the direct impact of GAI-powered assistants on academic writing, such as whether they enhance or impair writing quality.Instead, it concentrates on revealing the patterns of human-AI interaction within academic writing.The study's focus is to discern which strategies are most effective in the context of GAI-assisted academic writing, thereby understanding how these practices can be optimised.We provide a thorough, process-oriented examination based on screen recordings from these writing sessions, aiming to examine the role and implications of GAI-assisted writing tools.In particular, this study aims to assess doctoral students' regulation of academic writing processes when engaged in human-AI collaboration, particularly with ChatGPT, through the following research questions: . RQ1.What are the strategies and patterns doctoral students employ in their academic writing processes when collaborating with GAI-assisted writing tools?o RQ1a) What are the key hidden states in GAI-assisted academic writing?o RQ1b) What are the predominant patterns in GAI-assisted academic writing? .RQ2.How is the effectiveness of each identified strategic pattern in GAI-assisted academic writing characterised by its specific nature and underlying attributes?
The following sections of this paper provide a literature review of the cognitive process of writing and ChatGPT in academic writing.We then present the applied methods, followed by a detailed analysis of our findings.We conclude with a discussion of the implications of our study for the fields of academic writing and AIED and suggest directions for future research.By investigating the intersection of AI and academic writing, we hope to shed light on a critical area of AIED research, contributing valuable insights to the discourse surrounding AI's role in self-regulated academic writing.

Writing as a complex cognitive activity
Writing is a complex activity (Graham 2018), encompassing the navigation of language rules, communication norms, and the transition from the writer's to the reader's perspective.Specifically, it demands intense cognitive effort (Piolat, Olive, and Kellogg 2005) for tasks such as expressing ideas, maintaining logical flow and structure, and reorganising content to suit the given context.Moreover, writing processes share the working memory capacity (McCutchen 1996;2000), and if the cognitive load surpasses this capacity, these processes can be hindered (Sweller 1994).
Hence, the writer's objective is to minimise the demands on working memory, facilitating smooth processing and interactions among writing processes within working memory.According to Flower & Hayes' framework (1981), shown in Figure 1, writing processes encompass planning, translating, and reviewing, all of which are influenced by the writer's working memory capacity and the task environment.An efficient writer is capable of enhancing their performance (Kellogg 2008), necessitating deliberate effort and control over the activity.However, the constraints imposed by the limitations of working memory capacity significantly impact writing processes.

Regulation and coordination of writing
As a result, there is an adaptive shift in the regulation and coordination of these processes, tailored to meet specific task requirements and enhance the fluidity of composition.Efficient regulation of the writing processes is central to producing good-quality texts (Beauvais, Olive, and Passerault 2011).For example, proficient writers, often considered good regulators, can coordinate their writing processes by transitioning from sequential to more concurrent methods, mitigating the risk of overloading the working memory capacity (Olive 2014).
To be more specific, these skilled writers exhibit a higher level of executive control over the flow and transition of information across various cognitive and linguistic resources in a timely manner.This heightened executive control not only facilitates seamless flow and smooth information transitions but also paves the way for harnessing the potential of advanced technologies like AI.Through the automation and streamlining of lower-level processes in academic writing, AI can effectively release cognitive resources, allowing writers to dedicate more attention to higher-order aspects of composition, such as content organisation, argument development, and creative expression.This collaborative partnership between skilled writers and AI exemplifies the evolving landscape of writing, where technology serves as a supportive tool, ultimately enhancing the overall efficiency and quality of the writing process.

AI-assisted academic writing
The integration of AI technologies in academic writing has been a subject of increasing interest within the academic community.The initial phase of this integration, marked by the use of AIdriven applications such as grammar and style checkers to enhance the quality of written work, has been well-documented (Graesser et al. 2004).The landscape of AI assistance in academic writing has evolved significantly with recent advancements in natural language processing (NLP).These developments have given rise to a new generation of sophisticated AI-driven tools that offer extensive support in various facets of the writing process (Yan 2023).Notably, the advent of generative AI (GAI) technology, incorporating large language models (LLMs) like GPT-3 and GPT-4, has expanded the capabilities of AI in academic writing to include content generation, summarisation, feedback provision, and offering suggestions for improvements (Radford et al. 2021).These tools, when strategically employed, can assist writers throughout the entire writing process, from idea generation and outlining to drafting and revising manuscripts (Enriquez et al. 2023).
The effectiveness of these generative AI tools in supporting learners from diverse cultural backgrounds has also been a subject of research (Kasneci et al. 2023).These tools have been found to aid in overcoming language and style-related challenges in complex academic writing, facilitating translation, vocabulary selection, sentence structuring, and adherence to scholarly tone, thereby enhancing the speed and clarity of the writing process (Dwivedi et al. 2023;Radford et al. 2021).In academic discourse, where critical analysis and the formulation of counterarguments are crucial, generative AI tools have shown potential in alleviating writer's block, enhancing creativity, and improving the overall coherence and quality of academic texts (Yan 2023).
Despite these advancements, the ethical implications of employing generative AI in academic writing necessitate careful consideration to promote responsible use of this technology (Nguyen et al. 2023).Moreover, there is a notable research gap in the study of human-AI collaboration dynamics in academic writing.Current research is still nascent and primarily focuses on the capabilities and immediate effects of generative AI tools (Gašević et al. 2023;Kishore et al. 2023).There is a critical need for in-depth investigations into how human writers interact with these AI-driven writing assistants.Such research is essential to fully understand the complexities of incorporating generative AI tools in academic settings.This understanding will be instrumental in developing strategies that ensure the responsible and effective use of AI in supporting self-regulated academic writing practices.

Research methods
By applying process mining techniques to the analysis of human-AI collaboration in an academic writing task, this study seeks to provide a novel perspective on the dynamics of this interaction and offer insights into potential improvements and challenges associated with AI-assisted writing.The writing task and the unrestricted use of tools aim to provide an ecologically valid context for exploring the dynamics of human-AI collaboration in academic writing.

Participants and procedures
The participants in this study consisted of ten doctoral students (N = 10) from Finland and New Zealand enrolled in an English doctoral program in either Information Systems or Learning and Educational Technology.As the study aims to examine how ChatGPT is used as an AI-driven writing assistant, focusing on this particular population was intentional.Doctoral students were selected as a particularly relevant population for the study due to their familiarity with academic writing.The data collected from these students can provide valuable insights into the dynamics of human-AI collaboration in the context of advanced academic writing, where high standards and complex ideas are expected.
The study employed an online experimental design conducted via the Zoom video conferencing platform.The experiment was structured around a writing task requiring participants to compose a short essay of approximately 500 words on artificial intelligence (AI) use in education.Participants were instructed to convey their opinions, supporting their arguments with evidence and examples.Ensuring that the task was representative of the types of writing tasks participants would encounter in their academic pursuits, the task design and rubric were kept closely similar to those used in typical university writing assignments.
Participants were given 30 min to complete the writing task, during which they were allowed to use any tools deemed necessary, including ChatGPT and Google Scholar.The unrestricted access to tools was intended to mimic real-life writing situations and to investigate the extent to which participants chose to utilise AI-driven writing assistants during the task.During the experiment, screen recording was employed to capture participants' interactions with the writing task and the various tools they chose to use.The Zoom session was recorded, providing a comprehensive view of participants' behaviour and tools used during the writing task.A pre-survey questionnaire was initially conducted at the start of each session to collect information about participants' backgrounds.All identifying information was removed from the recordings and survey responses to ensure data privacy and participant anonymity.The assessment of the final product of the writing task was conducted by a university lecturer immediately following the data collection phase, prior to engaging in any sequential analysis or process mining.This procedural decision was strategically made to minimise the potential for biases in the performance assessment, while simultaneously mirroring the conventional evaluation practices prevalent in the higher education context.

Data analysis
The study's primary aim is to understand the dynamic interactions between doctoral students and ChatGPT, especially the strategic patterns and their effectiveness in academic writing tasks.In order to capture the complexities of these human-AI collaborative processes, we employ an AI-driven learning analytics approach (Järvelä, Nguyen, and Hadwin 2023; Ouyang, Xu, and Cukurova 2023) that allows for the computational extraction and interpretation of rich behavioural data during writing sessions, thereby enabling the identification of recurring strategies and patterns.
In particular, the analytical framework for this study is informed by the artificial intelligence-driven learning analytics method proposed by Ouyang, Xu, and Cukurova (2023), which adopts the complex adaptive systems perspective.This analytical approach involves three layers: (1) data pre-processing and analysis, (2) multi-channel sequence analysis, and (3) pattern analysis.In the first layer, quantitative content analysis was conducted to code the writing behaviour associated with the micro-processes undertaken by students.In the second layer, Hidden Markov Model (HMM) and sequence clustering were utilised to identify and categorise the underlying patterns exhibited by doctoral students in their writing processes with ChatGPT.Finally, process mining techniques are applied in the third layer to further examine the identified patterns.This study's choice of the HMM, sequence clustering and process mining methods is grounded in theoretical relevance and empirical rigour.The HMM is particularly suitable for modelling sequential processes that involve a certain degree of uncertainty, as is the case with human-AI interactions in academic writing.HMMs are adept at capturing latent states that cannot be directly observed but influence observable variables.In the context of this study, these latent states may include cognitive processes like planning, revising, and editing, which are not directly visible but impact the final written output.By employing HMM, we can more accurately model the hidden cognitive states that govern the visible interactions between doctoral students and ChatGPT.Sequence clustering and process mining, on the other hand, offer a complementary approach.While HMM provides insights into individual latent states and transitions, Sequence Clustering groups similar sequences of actions allowing us to identify common patterns or strategies employed across different students.Process Mining enriches this by providing the temporal aspect, showing what strategies are employed and how they evolve over an academic writing session.

Writing performance scoring
A structured marking rubric (Appendix A) mirroring typical higher education academic evaluation norms was instituted to appraise doctoral compositions enhanced by GAI-powered tools.This rubric comprises five key criteria: content, analysis, organisation and structure, quality of writing, and word Limit and referencing.Content, with a 30% weightage, critically assesses the student's clarity and depth in understanding AI's role in education.Analysis, also weighted at 30%, critically examines the rigour and relevance of the evidence in discussing AI's educational implications.Organisation and structure, accounting for 15%, examine the logical flow and coherence of the essay.Quality of writing, another 15%, scrutinises grammatical precision and stylistic suitability for a scholarly audience.Finally, word limit and referencing, weighted at 10%, check adherence to specified word counts and accuracy in APA referencing.

Quantitative content analysis
The screen recording with audio data was coded to document the temporal pattern of student writing tactics when employing ChatGPT.To characterise the writing behaviour associated with the micro-processes undertaken by students, we conducted a qualitative content analysis utilising the constant comparison approach as delineated by Onwuegbuzie et al. (2009).In the initial open coding phase, each student action associated with the writing process received a descriptor, highlighting a facet of the writing procedure.Subsequently, the research team distilled these descriptors into broader categories and formulated overarching themes that encapsulated the essence of individual or cluster codes.In total, 626 events were coded.Table 1 describes our code scheme for microlevel processes in AI-assisted writing.

Hidden Markov model
We utilised the Hidden Markov Model (HMM) to capture the dynamic alterations in doctoral students' management of their writing processes while collaboratively working with the AI tool, ChatGPT.The HMM, an extension of the basic Markov chain, provides insights into the probabilities of sequences composed of specific random variables or states (Eddy 2004).Hidden Markov Model (HMM) techniques have been frequently used in various studies within the field of learning sciences to scrutinise how learning processes unfold (Dang et al. 2023;Malmberg et al. 2021).In this study, we focus on understanding how doctoral students adjusted their writing strategies when working with AI assistants.Using HMM helps us model hidden cognitive activities like planning or editing, which, although not directly seen, affect the final written product and the interactions between doctoral students and the GAI-powered tool.The Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) metrics were applied to selecting the optimal number of states.

Sequence clustering & process mining
To examine the main patterns of AI-assisted writing, we employed Agglomerative Hierarchical Clustering (AHC) (Bouguettaya et al. 2015), leveraging the Python programming language in conjunction with the scikit-learn library.AHC is an unsupervised machine learning technique aiming to discern the dataset's structure, pinpoint inherent patterns, and classify data points into clusters.The Silhouette Coefficient was initially computed to assess the quality of cluster assignments.In our study, Type own text that is be part of the final writing TypeNote Type own notes of the read content, articles, information TypeOutline Type ideas for writing but with the structure that serves as outline sequences representing the hidden states associated with writing tactics for a cohort of 10 students were integrated into a clustering framework.We explored a range of cluster quantities (from 2 to 10) and evaluated each using its corresponding silhouette score.Our analysis indicated that a twocluster model was the most appropriate, achieving an optimal fit with a silhouette score of 0.372.The agglomerative coefficient, which reflects the tightness of clustering, was 0.77.
In order to describe the patterns of doctoral students' regulation of writing processes in human-AI collaboration with ChatGPT, we employed a process-mining technique to the coded actions.Specifically, our analysis was rooted in the Fuzzy Miner approach, as detailed by Günther and van der Aalst (2007), and utilised Fluxicon's Disco analysis software, a recognised tool in process mining.Notably, this software has been previously employed in research to examine learning event processes, as evidenced by works (e.g.Dindar, Järvelä, and Haataja 2020).

RQ1a) What are the key hidden states in GAI-assisted academic writing? Hidden Markov model results
Based on the fitting scores, where the lowest values for BIC are preferred (see Table 2), our HMM model identified 3 hidden states in the sequence of writing tactics students employ while interacting with ChatGPT.The data were then fitted to a three-state model -below is the transition matrix and are plotted against States with probability < 0.05 (as shown in Table 3).
The characteristics of each state are described as follows: . State 1: This state captures 14.6% of the total data on writing tactics and predominantly features activities connected to pasting -such as PasteToEssay, PasteToPrompt, PasteToWord, and Paste-ToGrammarTool -which collectively represent 82.6% of actions within this state.Accordingly, this state is labelled as Content Pasting behaviour.Component Shaping (State 3), as visualised in Figures 2 and 3. Figure 2 explains the hidden states with probabilities whereas Figure 3 visualises the HMM paths.Additionally, State 3, associated with Component Shaping behaviour, was characterised by less frequent but extended periods of activity.In contrast, States 1 and 2 -Content Pasting and Content Copying, respectively -were marked by brief and sequentially adjacent occurrences.This sequential patterning aligns with the anticipated behaviour of copy-pasting.However, its frequency and distribution are different for some students compared to others.

RQ1b) What are the predominant patterns in GAI-assisted academic writing?
Following the HMM analysis, Agglomerative Hierarchical Clustering (AHC) was applied to the sequences of hidden states for 10 students' writing tactics.The optimal clustering yielded two distinct writing strategy types: Type 1 with 5 sequences and Type 2 with 5 sequences, as shown in Figure 4.The process-mining analysis results reveal significant differences between Type 1 (Structured Adaptivity) and Type 2 (Unstructured Streamlinece) regarding the writing tactics pathways they went through.Figures 5 and 6 present the pathway of the writing tactics in the process maps for Type 1 and Type 2, with the most dominant process flow among those activities.The maps show the absolute frequencies and case coverages in the percentage of the number of activities and the connection among them.
In Type 1, individuals exhibited a higher familiarity and multifaceted engagement strategy with ChatGPT.After reviewing the task requirements, these high achievers often start by prompting ChatGPT for content while searching for articles (f PromptContent → SearchArticle = 40%).This indicates a preference for multitasking and reading content to stimulate their thought rather than just waiting for GPT to finish generating its responses, thereby maximising productivity.A typical sequence of actions in this type involves reading generated content → copying such content → pasting it to the essay then → either directly editing it or typing their content.This demonstrated a reflective and effective approach to processing and applying the generated material before integrating it into their final output.Their use of ChatGPT was characterised by more sophisticated operations with more PromptFollowUp (f Type 1 = 21; f Type 2 = 9) and EditPrompt (f Type 1 = 5, f Type 2 = 2) compared to their counterparts in Type 2. On the other hand, the students in Type 2 display a less refined interaction strategy with the GAIpowered tool.After reviewing task guidelines, they commonly insert these requirements into their essays, followed by TypeContent and TypeNotes.While this indicates the process of pre-thought (analysis of the task requirement to define objectives), prolonged engagement in this phase could negatively impact productivity, especially within the time constraints of the current task.When they use the GAI-powered tool, the procedure is usually linear: prompt for content → copy the generated text → and paste it into their essay.This hints at a lack of critical assessment of the GAI-generated content's relevance or accuracy.For some such actions are succeeded by more TypeContent and then DeletePreviouslyPastedGeneratedContent (fType 2 = 8, fType 1 = 0), suggesting that the GAI-powered tool's output serves mainly as a springboard for their own writing rather than as final text.Overall, their interactions with the GAI-powered tool reflect unexamined adoption or a supplemental note-taking approach that may not contribute to the final draft.

RQ2. How is the effectiveness of each identified strategic pattern in GAI-assisted academic writing characterised by its specific nature and underlying attributes?
Regarding the final performance of the students' writing, the data's normality was first verified using a Shapiro-Wilk test, which indicated a normally distributed performance measure (W = 0.91534, p > 0.05).A t-test was conducted to compare the mean performance scores between the two clusters.The analysis revealed a significant difference between Type 1's and Type 2's performance (t = 2.4011, df = 6.0267, p ≤ 0.05).Table 4 shows the summary of different descriptive statistics across two types.
The descriptive statistics in Table 4 can be interpreted as follows: Students in cluster Type 1 demonstrated a higher average performance score (M = 79.75,SD = 20.64)than Type 2 (M = 54.75,SD = 10.77).Although both types expressed comparable confidence levels regarding digital skills (M = 4.40), they diverged in their assessments of task difficulty and interest, which also changed after the task.Overall, Type 1 was correlated with higher performance and higher usage of the GAI-powered tool, while Type 2 was associated with lower performance with limited use and  familiarity with the GAI-powered tool.Despite both groups demonstrating similar confidence in their digital skills (M = 4.40), they differed in their evaluations of task difficulty and interest, both of which evolved over the course of the task.In general, Type 1 participants exhibited a correlation with enhanced performance and more frequent usage of the GAI-powered tool.Conversely, Type 2 participants were characterised by reduced performance, which was linked to their limited usage and familiarity with the GAI-powered tool.

Discussion
Academic writing in higher education is facing disruptive changes due to the emergence of GAIpowered writing assistants.Academic writing not only serves as a medium to communicate their research but also as a reflection of their critical thinking and analytical capabilities (Hyland 2021;Kellogg 2008).The rigour and depth required at the doctoral level demand a mastery of writing conventions, clarity of expression, and the ability to engage meaningfully with their immediate academic community and the broader intellectual discourse (Caffarella and Barnett 2000;Graham 2018).In this context, improving academic writing skills is not merely a formality but a crucial component of their academic and professional development (Caffarella and Barnett 2000).Our investigation identified two main strategies employed by doctoral students in their academic writing in collaboration with AI and how these strategies exhibited significant variations in their performances.Our findings contribute to the growing body of research concerning AI's application in education, specifically related to academic writing.This investigation has the potential to enhance our understanding of the writing process, offer perspectives on how AI tools can be effectively integrated into teaching practices, and influence the design of future academic writing support tools.As technology becomes more embedded in the educational landscape (Ashour 2020;Nguyen, Gardner, and Sheridan 2020), it becomes vital to comprehend its significance and effects.In alignment with this perspective, this paper examines the use and influence of the GAI-powered toolin the academic writing activities of doctoral students.
A key finding from our research is that the iterative coordination of writing processes with AI assistance typically yields better results than linear methods.When examining our findings through the Cognitive Process Theory of Writing (Flower and Hayes 1981), distinct similarities and differences emerge, emphasising the executive demands that arise when integrating AI tools into academic writing.The literature highlights the importance of the efficacy of executive control in modulating the flow and transition of information across diverse cognitive and linguistic resources (McCutchen 1996;2000).Notably, writers who exhibit proficiency are often identified as adept regulators.Such individuals are adept at coordinating their writing processes, shifting fluidly from sequential approaches to more concurrent strategies.This adaptive coordination is pivotal, particularly given that limited working memory capacity often poses a significant constraint in the writing process (McCutchen 1996).Our findings, in line with prior studies, show that the higher-performing doctoral students (Type 1) showcase a strategic and dynamic interaction with the GAI-powered tool.They seamlessly shift between sequential and concurrent strategies, demonstrating adaptive coordination crucial for overcoming the constraints posed by limited working memory capacity.The integration of AI complements the cognitive demands of planning and accentuates executive functions like attention control and strategic planning.Our study shows that the higher-performing doctoral students' engagement in AI-assisted writing is multifaceted, suggesting a higher familiarity with the tool.The observed sequence of actions, starting with prompting the GAI-powered toolfor content and subsequently searching articles, reflects a proactive approach to information gathering.This tactic, contrasting with merely waiting for generated responses, optimises productivity and stimulates cognitive processes.The subsequent sequence of reading, copying, pasting, and editing or integrating content indicates a methodical approach whereby the students critically assess, adapt, and incorporate the AI-generated material into their writing.The higher frequency of sophisticated operations such as PromptFollowUp and EditPrompt further accentuates the depth of their interaction and the value they derive from the the GAIpowered tool.This finding emphasises the potential benefits of adopting a cyclical approach to writing coordination with the GAI-powered tool (Dwivedi et al. 2023;Yan 2023).Furthermore, when employed adaptively and judiciously, AI not only aids in reducing cognitive demands but also serves as a strategic ally in optimising the utilisation of working memory during academic writing.By efficiently offloading certain cognitive tasks to AI, writers can free up cognitive resources, allowing for a more focused and meaningful engagement in the creative aspects of the writing process.This symbiotic relationship between AI and working memory enhances the overall efficiency and efficacy of academic writing, fostering an environment conducive to thoughtful and purposeful content creation.
Generative AI distinguishes itself from other technological advancements in its capacity to react based on prior interactions between human users and the AI system (Kasneci et al. 2023;Kishore et al. 2023).Consequently, in line with recent studies (Järvelä, Nguyen, and Hadwin 2023;Molenaar 2022), this study points out an imperative to devise strategic methodologies for effective human-AI collaboration to integrate such AI technologies into academic writing in higher education.In contrast with the higher-performing group, our study shows that the lower-performing students demonstrate a more simplistic and potentially inefficient engagement with the GAI-powered tool.Their initial approach of directly inserting task guidelines into essays, followed by content typing and note-taking, might signal a preliminary phase of understanding and outline.However, extended duration in this phase could be counterproductive, considering the task's time-bound nature.Their linear interaction with the GAI-powered tool-prompting, copying, and pasting -lacks the depth observed in the higher-performing group, hinting at a possible shortfall in critical engagement with the AI-generated content.It suggests that for these students, the GAI-powered tool's output primarily serves as an initial scaffold or placeholder rather than a substantive contribution to their final writing.The more linear and potentially superficial engagement of low-performing students with the GAI-powered toolmight indicate challenges in executing this translation effectively, possibly due to limited executive control or insufficient integration of the AI tool into their cognitive writing schema.
This study offers several implications for different stakeholders in higher education concerning the integration of AI tools like the GAI-powered toolin academic writing.For educators, the data reveal the specific cognitive and metacognitive skills contributing to effective collaboration with AI.This information is vital for developing targeted pedagogical strategies and interventions to enhance students' skill sets.For instructional designers and developers in the EdTech industry, the study identifies key functionalities particularly useful to students in the writing process.The research suggests that AI-enabled writing tools should promote adaptive coordination among users, as this approach correlates with higher writing performance.In terms of policymaking, the findings offer a substantial foundation for formulating policies on the inclusion of AI technologies in academic settings.Policymakers must consider the academic benefits and ethical considerations such as data privacy and potential misuse.Accordingly, comprehensive frameworks must be developed to govern the responsible use of these technologies.Finally, for students, particularly at the doctoral level, the study provides invaluable guidance for optimising their use of AI tools in academic writing.The research underscores the need for a proactive, dynamic engagement with the AI tool, as this mode of interaction has been shown to result in higher-quality academic work.

Conclusions and final remarks
This research investigated how doctoral students manage and control their academic writing activities in collaboration with the generative AI tool the GAI-powered tool.The distinct differences observed in the pathways between the two identified types of writing tactics in human and AI collaboration highlight the crucial importance of purposeful engagement with AI tools in academic writing.High-performing students effectively utilise the GAI-powered tool's functionalities to enrich their writing processes, whereas low-performing students appear to either not make full use of the tool or view it only as an additional resource.Future studies could explore the cognitive processes driving these behaviours in greater depth, providing educators with guidance for focused interventions and instruction.
The limitations of this study are notably underscored by the small cohort size, which necessitates a cautious interpretation of the statistical outputs.This constraint is particularly salient in the context of our research, which focuses on a specific and inherently limited population: doctoral students.The nature of this target group inherently restricts the potential pool of participants, rendering the attainment of a large sample size a significant challenge.While the sample size we achieved may be considered reasonable given the narrow scope of our study population, it is imperative to recognise the limitations this imposes on the generalizability and robustness of our findings.The small cohort size not only limits the statistical power of the study but also heightens the risk of sampling bias.This, in turn, could potentially restrict the diversity and representativeness of the sample, thereby impacting the breadth and depth of the insights gleaned from the study.Furthermore, the limited sample size may constrain the extent to which we can confidently extrapolate our findings to the broader population of doctoral students.It also potentially limits the ability to detect subtle but potentially significant patterns within the data.As such, any conclusions drawn must be approached with an understanding of these limitations, and the findings should be viewed as exploratory, necessitating further investigation with larger and more diverse samples to validate and extend these initial observations.
Another limitation of this study is the variability in factors influencing academic writing performance among doctoral students.This variability encompasses differences in interest in the writing task, and familiarity with the subject matter.Such differences can significantly affect the outcomes of the study, as they may not uniformly represent the broader student population's skills and engagement levels.Consequently, this variability must be considered when interpreting the findings, as it introduces potential variations that could impact the generalizability and applicability of the results.Furthermore, in our study, we adhered to a common assessment procedure prevalent in higher education to evaluate the outcomes of the writing tasks.However, it is acknowledged that this approach may not fully capture the transformational potential of the tool in question.The focus on efficiency gains in text production, while significant, only scratches the surface of the broader implications of generative AI technologies on learning processes.
To truly understand the depth and breadth of these technologies' impact, future research should delve into the transformational effects of generative AI on learning paradigms.This exploration is crucial for uncovering how such technologies can fundamentally alter the way knowledge is acquired, understood, and applied by learners.Moreover, there is a pressing need for the development and implementation of innovative assessment tools that are capable of more accurately evaluating student learning outcomes in the context of higher education.These tools should be designed to not only assess traditional metrics of academic performance but also to measure the nuanced ways in which generative AI can enhance critical thinking, creativity, and problem-solving abilities.
Nevertheless, despite these constraints, this study provides valuable insights into the intricate dynamics of human-AI collaboration in academic writing.It underscores the potential roles and impacts of generative AI, illustrating how, when utilised optimally, such tools can significantly bolster the writing process.The observed behaviours and tactics offer a preliminary understanding of how doctoral students navigate the confluence of traditional academic writing skills and AI-assisted content generation.Despite its limitations, this research lays the groundwork for further in-depth investigations, fostering a clearer understanding of best practices in integrating AI tools in academic writing.The results of this study point to several areas where more research is needed.Future research could further examine the cognitive and metacognitive factors influencing how students work with AI in academic writing.Methods like eye-tracking or verbal reports might give a more detailed picture of students' decision-making while writing.Additionally, issues like data privacy and potential AI bias are increasingly important.Future research could explore these ethical issues more thoroughly, guiding potential changes in higher education policy.

Figure 1 .
Figure 1.The cognitive process of writing (Flowers and Hayes 1981).

Figure 3 .
Figure 3.Most probable HMM paths of the hidden states.

Figure 4 .
Figure 4. Index plots depicting the hidden states characteristics for the two cluster types.

Figure 5 .
Figure 5.The sequence of writing tactics among the Type 1 -Structured Adaptively.

Figure 6 .
Figure 6.The sequence of the writing tactics among the Type 2 -Unstructured Streamline.
This state encompasses 70.8% of the writing tactics data.Unlike the other two states, this one presents a broader range of writing tactics.These activities include, but are not limited to, AddReference, TypeContent, EditContent, PromptContent, and PromptFollowUp.Accordingly, this state is most accurately described as Component Shaping behaviour.Among the HMM results, a regular cyclical transition pattern was observed between the Content Copying (State 2) and Content Pasting (State 1), as well as between Content Pasting (State 1) and . State 2: This state occupies 14.5% of the data on writing tactics and is primarily composed of copying-related activities.Actions such as CopyArticleContent, CopyContent, CopyGenerated-Content, CopyParaphrasedContent, and CopyTaskRequirement contribute to 97.8% of this state's activities.Thus, it is best described as Content Copying behaviour. .State 3:

Table 2 .
HMM model fit statistics.
Figure 2. HMM graphs of the transitional structures between states.

Table 4 .
Summary of different criteria across two types.