Student Use and Teacher Practice: A Scoping Review of Generative AI in Computing Education Using PICRAT Carrie Anne Philbin Raspberry Pi Computing Education Research Centre University of Cambridge Cambridge, United Kingdom cap90@cam.ac.uk Sue Sentance∗ Raspberry Pi Computing Education Research Centre University of Cambridge Cambridge, United Kingdom Ss2600@cam.ac.uk Abstract Technology has promised to transform teaching and learning for over 25 years, from interactive whiteboards to personalised learning platforms. Generative AI large languagemodel (LLM) chatbots, such as ChatGPT, are the latest innovation poised to reshape computing education. However, much of the research has taken a technology- driven approach to their integration rather than a student-centred one. This scoping review examines studies published between Oc- tober 2022 and October 2024, using the PICRAT framework to map student activity with LLMs as passive, interactive, or creative, and educator use as replacing, amplifying, or transforming traditional teaching methods. Analysing 32 studies across K-12 and higher education, sourced following PRISMA-ScR guidelines, we find that use of AI chatbots primarily replicates or amplifies conventional instructional patterns, with most student interactions being passive or interactive rather than creative. While these tools improve effi- ciency, their potential for transformative learning, particularly in K-12 settings, remains under explored. This review offers insights for computing educators seeking to integrate AI into their teaching and highlights the value of using PICRAT, not only for planning and designing AI-enhanced learning activities, but also for evaluating their pedagogical impact over time. CCS Concepts • Applied computing → Computer-assisted instruction; • So- cial and professional topics → Adult education; K-12 edu- cation; • Human-centered computing → Natural language interfaces. Keywords K-12 education, Generative AI, PICRAT, Scoping review ACM Reference Format: Carrie Anne Philbin and Sue Sentance. 2025. Student Use and Teacher Practice: A Scoping Review of Generative AI in Computing Education ∗This author has a conflict of interest due to their involvement in the conference proceedings as a Programme Chair. An independent review process, including final decision, was managed by Rosanne English, University of Strathclyde, who served as Proxy Programme Chair for this submission. The process reflects the ‘Conflict of Interest Policy for ACM Publications’ and the Proxy Programme Chair was appointed by the UKICER Steering Committee. This work is licensed under a Creative Commons Attribution 4.0 International License. UKICER 2025, Edinburgh, United Kingdom © 2025 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2078-9/25/09 https://doi.org/10.1145/3754508.3754513 Using PICRAT. In UK and Ireland Computing Education Research Conference (UKICER 2025), September 04–05, 2025, Edinburgh, United Kingdom. ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/3754508.3754513 1 Introduction Education has long been intrigued by new and emerging technolo- gies, and in a field where technology is central to the subject matter, they are frequently experimented with and implemented in practice [41]. This is the case for generative AI, particularly Large Language Models (LLMs) in recent years. The rapid proliferation of gener- ative AI LLMs such as Open AI’s ChatGPT and Google’s Gemini has prompted educators across various disciplines to examine their potential impact on student learning especially where they may offer new opportunities for personalised learning and interactive engagement [7]. Research indicates a growing interest in leveraging generative AI LLMs to enhance computing education, particularly in the context of programming instruction. These tools offer po- tential benefits such as personalised feedback, automated tutoring, and fostering creative problem-solving. However, they also present challenges, including ethical concerns, issues of equitable access, and the preparedness of educators [9]. This paper aims to critically examine the use of generative AI LLMs in computing education through a pedagogy-first lens, prioritising teaching and learning approaches over a technology-driven impact perspective. 2 Background and Related Work Selwyn’s work on technology in education [40, 41] highlights the ways in which digital tools mediate teaching and learning, sug- gesting that their integration can reinforce or challenge traditional pedagogical models. He asserts that technological innovations in ed- ucation often create superficial changes rather than fundamentally disrupting existing structures. The rise of MOOCs, learning analyt- ics, adaptive testing, and personalised learning networks has been widely perceived as transformative. However, these technologies have largely reinforced rather than resolved persistent issues such as educational inequalities, inefficiencies, and institutional inertia [41]. This perspective suggests that without deliberate pedagogical innovation, technology is more likely to replicate existing practices than drive substantive change. On the other hand, some suggest that the future of education will be shaped by collaboration with generative AI and that the role of the educator is changing from a holder of expertise and knowledge towards a model where they signpost and guide [38]. Since the release of ChatGPT in 2022, numerous studies have examined the role of generative AI LLM chatbots in computing education, leading to several systematic literature reviews on its https://orcid.org/0009-0002-9974-3965 https://orcid.org/0000-0002-0259-7408 https://creativecommons.org/licenses/by/4.0 https://creativecommons.org/licenses/by/4.0 https://doi.org/10.1145/3754508.3754513 https://doi.org/10.1145/3754508.3754513 http://crossmark.crossref.org/dialog/?doi=10.1145%2F3754508.3754513&domain=pdf&date_stamp=2025-09-03 UKICER 2025, September 04–05, 2025, Edinburgh, United Kingdom C.A. Philbin and S. Sentance opportunities and challenges [3, 31, 37]. Prather et al. categorised existing research surfaced through a literature review into five key areas: assessing the capabilities and limitations of AI, impact on generating teaching materials, using LLMs to assess student work, studying interactions between student and AI, and theoretical posi- tion papers [37]. This categorisation reflects what Selwyn describes as a “technologically determinist perspective” [40], where research is shaped by the assumption that digital technology inevitably drives change – whether positive or negative. Similarly, through a literature review, Mahon et al. investigated the motivations behind CS1 instructors’ consideration of genera- tive AI integration, finding that concerns over academic integrity were the primary driver rather than a desire for long-term peda- gogical transformation [31]. Much of the research has approached AI adoption from a top-down perspective, focusing on how these tools could or should be implemented within existing institutional structures, rather than examining their real-world use in specific educational contexts [40]. 3 The PICRAT Framework This study uses the PICRAT model to evaluate how generative AI LLM chatbots, are integrated into K–12 and higher education com- puting contexts. PICRAT was selected for its systematic approach to assessing technology use from both student and educator per- spectives and its suitability for synthesising insights from a scoping review of 32 studies. The PICRAT framework is a model for evaluat- ing and integrating technology in education. It is a student-focused, pedagogy-driven model that can help teachers reflect on their tech- nology integration practices and make informed choices about how to use technology to enhance learning [22, 43]. PICRAT builds on and combines two perspectives: PIC (passive, interactive, creative), which examines the student’s interaction with technology in a specific educational context, and RAT (replacement, amplification, transformation), which evaluates how technology influences and shapes existing teacher practices [19, 22]. Figure 1: The PICRAT framework matrix and flow diagram When using PICRAT, each lesson plan, activity, or instructional practice can be categorised into one of nine cells in the matrix (Figure 1). The matrix progresses hierarchically from bottom-left to top-right – from passive replacement to creative transformation [22]. Teachers can use the PICRAT framework to reflect on their practices, consider new strategies and approaches, and move their technology integration practices toward more effective uses [43]. PICRAT has been used in a number of research studies to evalu- ate technology integration in K-12 schools and higher education. For instance, one study used PICRAT to analyse how often K–12 teachers used technology in their classrooms in response to the COVID-19 pandemic [43]. Researchers asked 76 teachers about their grade level, frequency of technology use, the types of activities they implemented, and their support levels. They then categorised 26 different technology-integrated activities, such as students attend- ing virtual discussions or creating online presentations, according to the PICRAT framework matrix. The study found that in general teachers used technology more frequently for passive activities such as watching videos and for activities that replaced traditional teaching methods, and less frequently for activities that encour- aged creativity or transformed teaching practices. This pattern was particularly pronounced at the K–6 level. The study also found that the level of support teachers received was a significant predictor of how often they integrated technology into their teaching, especially at the K–6 level [43]. While research on the PICRAT model is still developing, it offers several potential advantages. For teachers, it serves as a valuable framework for reflecting on their technology integration strate- gies and for making deliberate, informed decisions about using technology to enhance student learning [19]. For students, PICRAT positions technology as a tool to support broader educational ob- jectives, thus enriching student experiences and outcomes [22]. For researchers, it offers a structured method for assessing the effec- tiveness of technology integration and fosters a shared language to discuss and compare different approaches. 4 Research questions This study has three primary objectives: first, to identify and cate- gorise recent empirical research on the use of generative AI chat- bots in computing education contexts; second, to analyse how the technology was used to teach and learn computing; and third, to identify gaps in the existing literature and suggest directions for future research. To address these objectives, the following research questions have been formulated: • RQ1: How have generative AI LLM chatbots been used in computing education? • RQ2: What types of student use are exhibited when using generative AI LLM chatbots in computing education con- texts? • RQ3: To what extent do generative AI LLM chatbots replace, amplify, or transform traditional teaching practices in com- puting education contexts? Generative AI in Computing Education: A PICRAT Scoping Review UKICER 2025, September 04–05, 2025, Edinburgh, United Kingdom 5 Methods To answer the research questions, a scoping literature review was developed using the Preferred Reporting Items for Systematic Re- views andMeta-analysis Protocols (PRISMA-ScR) recommendations and guidelines [35]. In the following sections, the systematic meth- ods undertaken are outlined, and the search strategy, inclusion criteria, and data analysis procedure are described. 5.1 Search Strategy The search was designed to identify peer-reviewed articles and other relevant literature on the use of generative AI chatbots in computing education across K-12 and higher education contexts covering the period from October 2022 to October 2024. The ra- tionale was to cast a wide search net while employing detailed eligibility criteria to screen for empirical studies. A comprehensive list of keywords was developed by the authors to construct the database search strings. The following search string was applied: “generative AI” OR ChatGPT OR “large language models” OR “AI chatbots” AND K-12 OR “higher education” AND “computing educa- tion” OR “computer science education” OR “programming education”, along with a publication date filter to limit results to the specified time frame. Grey literature was excluded from the search to focus solely on peer-reviewed sources. The search strategywas implemented across three academic databases – ACM Digital Library, SCOPUS, and ERIC – in October 2024. A total of 265 articles were retrieved: 143 from ACM, 122 from SCOPUS, and none from ERIC. The retrieved results were exported to the reference management tool Rayyan [34] to facilitate the removal of duplicates and the initial screening of abstracts. 5.2 Inclusion Criteria Articles were excluded based on several criteria. Only peer- reviewed journal articles and full conference proceedings were included; while literature reviews, posters, abstracts, and extended abstracts were excluded. Studies were removed if their population setting was outside of computing education or if they were not conducted within a school or university environment. Additionally, articles that did not focus on teaching computing, computer sci- ence (CS) or programming were deemed out of scope and excluded. This included studies focused on AI literacy and those targeting assessment or testing practices. Finally, research investigating be- spoke generative AI tools specifically developed and tested within educational contexts was excluded. This screening process ensured that the review remained focused on studies directly relevant to the research questions (Figure 2) and resulted in 32 articles for analysis as part of this scoping review. 5.3 Analysis Procedure To enable a thorough analysis of the 32 studies included in this scoping review, a structured data extraction table was developed to systematically collect and organise relevant information. The table was designed to capture key details about each study, focusing on its characteristics, context, and findings related to the integra- tion of generative AI LLM chatbots in computing education. Data were extracted across several key fields to ensure consistency and Figure 2: Search Process comprehensiveness. First, information about the study’s general characteristics was collected, including the author(s), title, publica- tion source, year of publication, and geographical context. Data on the target population were recorded, including sample size (n), age range, gender distribution, and other demographic characteristics. The environmental context of each study was noted, including the date of data collection and the educational setting in which the in- tervention occurred (e.g., K-12 classroom, university lecture). This was followed by the study’s aim and the specific research questions it sought to address. Key findings were summarised, particularly impacts on teaching practices, and student engagement. To address RQ1, we conducted an inductive thematic analysis of the extracted data [5]. To answer RQ2 and RQ3 the first author coded each study according to the PICRAT framework (Figure 1). To enhance reliability, the second author independently reviewed and coded a small number of studies and the authors iteratively discussed their interpretations to ensure a consistent application of the PICRAT framework. After all iterations, the second author independently coded a random 50% sample of the studies (𝑛 = 16) initially coded by the first author, yielding a Krippendorff’s Alpha of 0.91 using a nominal weight, indicating a high level of interrater reliability [32]. 6 Results and Interpretation Results are organised into three categories aligned with the research questions: (1) an overview of study populations and contexts; (2) categorisation of teaching approaches and learning activities; and analysis of these activities using the PICRAT framework, consider- ing both (3) student and (4) educator perspectives. 6.1 Sample The scoping review included 32 empirical studies exploring the use of generative AI chatbots in computing education. They were conducted across 16 different countries: North America (11 stud- ies), Europe (9), Asia (7), Oceania (4), and South America (1). The populations studied in the 32 included articles primarily consisted of higher education students (29). These included first-year stu- dents enrolled in introductory programming courses across various languages (e.g., C, Java, Python, C++), and those with no prior programming experience (11). Many studies targeted students in core computer science courses, such as CS1 (introductory program- ming), CS2 (data structures and algorithms), and object-oriented programming (13). Some studies also focused on students in spe- cialised programs like computer networking (1), data science (1), and robotics courses (1). A few studies included graduate students, UKICER 2025, September 04–05, 2025, Edinburgh, United Kingdom C.A. Philbin and S. Sentance particularly in robotics and data science fields (2). In contrast, a smaller portion of studies involved K-12 students and teachers, such as high school students in introductory programming courses and sixth-grade students and elementary school teachers (3). Addition- ally, some studies examined students in vocational schools and those enrolled in open online programming courses (4). 6.2 RQ1 How the technology was used Teaching and learning activities identified across the studies can be summarised and grouped into four main categories: student acquisition of computing knowledge, personalised learning support, collaboration and co-creation, and programming- specific competencies. Table 1 provides a detailed list of the types of activities within each category. Most prevalent activities were related to programming compe- tencies, such as error detection, debugging, syntax learning, and problem-solving. The studies reviewed suggest that LLMs, particu- larly ChatGPT and Codex, significantly improved students’ ability to detect and resolve bugs. While LLM-generated code was often correct, it sometimes contained minor issues that students over- looked, and an overreliance on AI-generated solutions led to re- duced engagement with complex problem solving. For example, Xue et al. [45] tasked participants with creating UML diagrams and implementing Java class skeletons using ChatGPT. They found that students relied less on traditional resources such as lecture slides and were more likely to depend solely on generative AI [45]. In the acquisition of knowledge category, activities focused on exploring concepts, generating explanations, and reviewing learn- ing materials, with generative AI LLMs frequently used to clarify programming concepts, suggest algorithms, and provide person- alised learning content. A major advantage of LLM-based tools was their personalised learning support, demonstrated by Abolnejadian et al. [1] in their study. Personalised explanations, examples and ex- ercises were generated as part of a Python programming course for high school students. Those who received AI feedback performed significantly better in subsequent quizzes, highlighting the value of tailored, descriptive feedback with actionable suggestions. The abil- ity of Generative AI LLMs to tailor responses to individual learning needs also enhanced engagement, particularly for students strug- gling with syntax-based errors. However, for complex conceptual challenges, such as data type conversions, LLM-based hints did not always lead to improvement [26]. Limited activities were categorised within collaboration and co- creation. The intervention by Dos Santos and Cury [10] focused on facilitating collaborative programming tasks, with ChatGPT providing instant feedback and tailored support. Students appre- ciated its ability to provide immediate feedback and reduce wait times associated with asking for help. However, concerns about trust, accuracy, and ethical implications remained. Student fears of dependency, misleading answers, or perceived declines in program- ming proficiency were also reported by Rogers et al. [39]. Hou et al. [16] additionally suggest that social pressures tied to asking peers or instructors for help were alleviated by using ChatGPT. However, the approach lacked the community and mutual learning benefits of collaborative help-seeking. Despite these varied impacts, several studies found no signifi- cant differences in overall academic performance between ChatGPT and non-ChatGPT users [6, 23]. This suggests that while LLMs en- hance certain aspects of learning, they do not necessarily replace traditional instruction. Furthermore, differences in usage patterns were observed across student groups, with more experienced pro- grammers benefiting from iterative feedback to deepen their under- standing [20], while novice users sometimes struggled to effectively communicate with AI models [11]. 6.3 RQ2 Computing Students’ Use (PIC) The PICRAT mapping (Table 2) shows that most student activities involving generative AI chatbots fall into the Passive (P) and Inter- active (I) categories. While AI is often used to support engagement and assist with problem-solving, these interactions typically do not involve higher-order thinking or creative application. These activities include: • Scaffolding programming exercises, where explanations and hints adjust to student input. For example, in Fenu et al. [12], a portion of students using ChatGPT during C programming training sessions engaged in interactive problem-solving with generative AI by requesting code modifications and improvements. • Debugging and code improvement, where AI helps trou- bleshoot errors and optimise code. In Kosar et al. [23], stu- dents with ChatGPT access used it to refine code and com- pare solutions, promoting more reflective and iterative de- velopment. • Exploring computing concepts, where students deepen un- derstanding through tailored explanations. In Brender et al. [6] A group of participants of a graduate-level robotics course used ChatGPT to provide conceptual explanations. These interactive applications improve accessibility but primarily reinforce existing learning models rather than introducing new approaches for students. By contrast, relatively few activities fall into the Creative (C) cat- egory, where students are using higher-order thinking to produce digital outputs [22]. These activities often involved personalisation by the student in order to clarify concepts. In Bernstein et al. [4], students received analogies aligned with their interests or cultural context from an LLM. The limited presence of creative applications suggests that AI is primarily serving as an adaptive tutor rather than a catalyst for innovation in computing education. 6.4 RQ3 Computing Teacher Practice (RAT) The mapping exercise indicates that, in most cases, educators’ use of generative AI primarily replicated traditional teaching methods. For example, generative AI LLM chatbots were frequently employed to generate programming exercises or produce explanatory mate- rials, effectively mirroring conventional instructional approaches. In these instances, the technology altered the format rather than fundamentally changing the pedagogical approach [22]. This was demonstrated by Sun et al. [42], where students participated in a Python programming course focusing on foundational topics – Python basics, data structures, control structures, and functions – delivered through traditional lectures and self-directed practice. Generative AI in Computing Education: A PICRAT Scoping Review UKICER 2025, September 04–05, 2025, Edinburgh, United Kingdom Table 1: Categories of teaching and learning activities from 32 reviewed studies Category Teaching and Learning Activities Reviewed Studies Acquiring Knowledge Generating explanations [1, 15, 25, 26, 42, 47] Exploring & understanding concepts [6, 12, 39, 42, 46, 47] Clarifying concepts through real-world application [17, 39] Personalised Learning Support Generating personalised materials [1, 4, 29, 44, 47] Virtual teaching assistants [10, 16, 28] Self-reflection dialogue [11, 24] Feedback on student assignments & quizzes [33, 44] Peer instruction [10] Exam preparation [17] Collaboration/Co-creation Creating real-world scenarios and user personas [21] Collaborative programming [10] Programming Competencies Learning syntax [12, 15, 20, 26, 42] Suggesting algorithms to solve problems [2, 11, 15, 45, 46] Identifying & resolving programming errors [6, 13, 15, 20, 26, 30, 36, 42] Improving code quality [14, 23] Reducing complexity [26] Solution code generation [1, 2, 6, 8, 11–15, 26, 42, 46, 47] Exploring starter code & programming exercises [1, 8, 20, 29] Although the only distinction between the control and experimen- tal groups was the use of ChatGPT in the latter, the integration of generative AI primarily served to support existing instructional methods rather than transform them, aligning with the “replace- ment” categories of the PICRAT framework. A smaller subset of studies demonstrated amplification, where generative AI enhanced existing teaching practices by introducing additional functionalities that improved efficiency and personalisa- tion. Notable examples included the following: • Personalising scaffolding for students, where LLMs dynami- cally adjusted their explanations and support based on stu- dent responses, offering more tailored assistance than static resources, and reducing the load on educators. For instance, in the study by Abolnejadian et al. [1], ChatGPT was cus- tomised to provide dynamically generated explanations, ex- amples, exercises, and solutions tailored to each student’s background and needs. • AI-powered virtual teaching assistants, which provided real- time, context-aware feedback on programming assignments, again reducing educator workload. For example, in Hsin [17] students in a computer networking course used ChatGPT as a virtual assistant for exam preparation, specifically clarifying concepts and gaining insights into real-world applications. • Collaborative brainstorming and guided learning, where students used AI to explore coding approaches or receive adaptive hints during problem-solving. In Dos Santos and Cury [10], ChatGPT was used as a virtual peer in pair pro- gramming activities. Students received immediate, tailored feedback and support while completing collaborative pro- gramming tasks, allowing them to explore alternative coding strategies more effectively than through traditional peer in- struction alone. While these applications enhanced learning experiences, they still functioned within pre-existing pedagogical frameworks, focusing on efficiency and accessibility rather than fundamentally altering instructional methods. Notably, none of the studies demonstrated a truly transformative use of generative AI — where the technology enabled entirely new teaching and learning paradigms that would otherwise be impos- sible [22]. This suggests that LLMs are currently being adopted primarily as supportive tools rather than as catalysts for pedagog- ical innovation. However, whether we should expect – or even desire – generative AI to transform traditional practice remains an open question. What might truly transformative applications look like, and how could they benefit students and teachers alike? Factors such as limited teacher training, ethical and trust concerns, and the risk of over-reliance on AI-generated solutions may hinder the exploration of possibilities [38]. Table 2: Activities from 32 studies mapped using PICRAT CR [45] CA [4, 8, 21] CT [-] IR [2, 6, 11, 15, 18, 23, 24, 26–28, 36, 42, 44, 46] IA [10, 13, 14, 16, 17, 20, 29, 47] IT [-] PR [12, 25, 30, 33, 39] PA [1] PT [-] 7 Limitations Our scoping review has some limitations. The search protocol was limited to a two-year period, from October 2022 to October 2024. Given the speed of technological advancement, some findings may already be outdated in terms of the tools examined and how they were applied in practice. The research landscape itself may have evolved significantly since the studies were conducted. The PICRAT framework can be difficult to apply in practice. Determining where a particular lesson falls in the nine-box matrix can be a matter of subjective interpretation, as there are no precise definitions for each level of the model. Some studies provided vague descriptions of activities which may have led to inaccurate mapping on the PICRAT matrix. To mitigate, the authors discussed their UKICER 2025, September 04–05, 2025, Edinburgh, United Kingdom C.A. Philbin and S. Sentance level of confidence and degree of inference needed while making decisions about matrix placing and this was used to help form a consensus. At some level, even an agreed decision between two authors is still subjective. Another limitation is that the framework does not take into account the quality of the technology being used or the specific learning goals of the lesson. The model focuses primarily on how technology is being used, rather than why it is being used. There- fore, it is worth noting that although AI chatbots may not have been used in a transformative or creative way, that does not mean that student outcomes were determinately affected or vice versa. In particular, the PICRAT framework is not designed to evaluate whether an activity or lesson delivered by an educator is good, or the attainment of student learning. 8 Recommendations and Future Work The overarching aim of this scoping literature reviewwas to explore how generative AI LLM chatbots were being used in computing education, and whether those applications were superficial and replicated existing practice or were revolutionary and disruptive [41]. The results of the PICRAT mapping analysis indicate that the existence of the technology may be currently driving usage of the applications, rather than an examination or reflection of desired pedagogy. Frameworks such as PICRAT could help determine whether tech- nology enhances or transforms teaching and learning, potentially opening up new approaches that include everyone. Before inte- grating generative AI chatbots, educators may wish to consider whether it genuinely improves student engagement and fosters deeper learning. Instead of using AI primarily for automation, fu- ture research could explore how it can support problem-solving, computational creativity, and innovative approaches to learning in computing. Most of the identified studies focused on generative AI’s role in student acquisition of computing knowledge, personalised learning support, collaboration and co-creation, and programming-specific competencies. Research could expand beyond these areas to con- sider AI’s influence on metacognition, self-regulated learning, eth- ical decision-making, and interdisciplinary problem-solving. A wider perspective may provide a clearer understanding of AI’s potential to support meaningful and transformative learning expe- riences in computing education. Only three of the 32 studies reviewed focused on K-12 settings, revealing a research gap on generative AI chatbot use in schools. Research into how students in different age groups respond to activities with generative AI LLMs may further the development of pedagogy in our field. 9 Conclusion The PICRAT framework provided a new lens through which to re- view how generative AI LLM chatbots impact computing education from both the student and educator perspectives. Much of the focus around generative AI LLM chatbots in computing education has been on the effect or impact of the technology on existing teaching approaches and exercises (such as programming or assessments), as evidenced by this review and others. This study has implications for computing educators in higher education or K-12 settings who wish to use generative AI in their teaching. We suggest using the PICRAT framework in the planning and design of learning activities may help educators maximise the value of generative AI LLM chatbots. It offers a structured way to assess their impact on students over time, while also fostering a shared language for comparing and discussing different pedagogical approaches. References [1] Mohammad Abolnejadian, Sharareh Alipour, and Kamyar Taeb. 2024. Leveraging ChatGPT for Adaptive Learning through Personalized Prompt-based Instruction: A CS1 Education Case Study - Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. In Extended Abstracts of the CHI Con- ference on Human Factors in Computing Systems (CHI EA ’24). Association for Computing Machinery, New York, NY, USA. doi:10.1145/3613905.3637148 [2] N. Baláž, J. Porubän, M. Horváth, and T. Kormaník. 2024. Using ChatGPT During Implementation of Programs in Education. OpenAccess Series in Informatics 122 (2024). doi:10.4230/OASIcs.ICPEC.2024.18 [3] Erik Barendsen, Violetta Lonati, Keith Quille, Rukiye Altin, Monica Divitini, Sara Hooshangi, Oscar Karnalim, Natalie Kiesler, Madison Melton, Calkin Suero Mon- tero, and Anna Morpurgo. 2024. AI in and for K-12 Informatics Education. Life after Generative AI.. In Proceedings of the 2024 on ACM Virtual Global Computing Education Conference V. 2 (Virtual Event, NC, USA) (SIGCSE Vir- tual 2024). Association for Computing Machinery, New York, NY, USA, 279–280. doi:10.1145/3649409.3691073 [4] Seth Bernstein, Paul Denny, Juho Leinonen, Lauren Kan, Arto Hellas, Matt Little- field, Sami Sarsa, and Stephen Macneil. 2024. "Like a Nesting Doll": Analyzing Recursion Analogies Generated by CS Students Using Large Language Models - Proceedings of the 2024 on Innovation and Technology in Computer Science Ed- ucation V. 1. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 (ITiCSE 2024). Association for Computing Machinery, New York, NY, USA, 122–128. doi:10.1145/3649217.3653533 [5] Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psy- chology. Qualitative Research in Psychology 3, 2 (2006), 77–101. doi:10.1191/ 1478088706qp063oa [6] J. Brender, L. El-Hamamsy, F. Mondada, and E. Bumbacher. 2024. Who’s Helping Who? When Students Use ChatGPT to Engage in Practice Lab Sessions. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 14829 (2024), 235–249. doi:10.1007/978-3- 031-64302-6_17 [7] Miguel A Cardona, Roberto J Rodríguez, and Kristina Ishmael. 2023. Artificial Intelligence and Future of Teaching and Learning: Insights and Recommendations. Technical Report. Office for Educational Technology, U.S. Department of Edu- cation. https://tech.ed.gov/files/2023/05/ai-future-of-teaching-and-learning- report.pdf [8] P. Denny, J. Leinonen, J. Prather, A. Luxton-Reilly, T. Amarouche, B.A. Becker, and B.N. Reeves. 2024. Prompt Problems: A New Programming Exercise for the Generative AI Era. SIGCSE 2024 - Proceedings of the 55th ACM Technical Symposium on Computer Science Education 1 (2024), 296–302. doi:10.1145/3626252. 3630909 [9] Paul Denny, James Prather, Brett A. Becker, James Finnie-Ansley, Arto Hellas, Juho Leinonen, Andrew Luxton-Reilly, Brent N. Reeves, Eddie Antonio Santos, and Sami Sarsa. 2024. Computing Education in the Era of Generative AI. Commun. ACM 67, 2 (Jan. 2024), 56–67. doi:10.1145/3624720 [10] O.L. Dos Santos and D. Cury. 2023. Challenging the Confirmation Bias: Us- ing ChatGPT as a Virtual Peer for Peer Instruction in Computer Program- ming Education. Proceedings - Frontiers in Education Conference, FIE (2023). doi:10.1109/FIE58773.2023.10343247 [11] Molly Q Feldman and Carolyn Jane Anderson. 2024. Non-Expert Programmers in the Generative AI Future - Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work. In Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work (CHIWORK ’24). Association for Computing Machinery, New York, NY, USA. doi:10.1145/3663384.3663393 [12] G. Fenu, R. Galici, M. Marras, and D. Reforgiato. 2024. Exploring Student Interac- tions with AI in Programming Training. UMAP 2024 - Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization (2024), 555–560. doi:10.1145/3631700.3665227 [13] M.B. Garcia, T.F. Revano, R.R. Maaliw, P.G.G. Lagrazon, A.M.C. Valderama, A. Happonen, B. Qureshi, and R. Yilmaz. 2023. Exploring Student Preference be- tween AI-Powered ChatGPT and Human-Curated Stack Overflow in Resolving Programming Problems and Queries. 2023 IEEE 15th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management, HNICEM 2023 (2023). doi:10.1109/HNICEM60674. https://doi.org/10.1145/3613905.3637148 https://doi.org/10.4230/OASIcs.ICPEC.2024.18 https://doi.org/10.1145/3649409.3691073 https://doi.org/10.1145/3649217.3653533 https://doi.org/10.1191/1478088706qp063oa https://doi.org/10.1191/1478088706qp063oa https://doi.org/10.1007/978-3-031-64302-6_17 https://doi.org/10.1007/978-3-031-64302-6_17 https://tech.ed.gov/files/2023/05/ai-future-of-teaching-and-learning-report.pdf https://tech.ed.gov/files/2023/05/ai-future-of-teaching-and-learning-report.pdf https://doi.org/10.1145/3626252.3630909 https://doi.org/10.1145/3626252.3630909 https://doi.org/10.1145/3624720 https://doi.org/10.1109/FIE58773.2023.10343247 https://doi.org/10.1145/3663384.3663393 https://doi.org/10.1145/3631700.3665227 https://doi.org/10.1109/HNICEM60674.2023.10589162 https://doi.org/10.1109/HNICEM60674.2023.10589162 Generative AI in Computing Education: A PICRAT Scoping Review UKICER 2025, September 04–05, 2025, Edinburgh, United Kingdom 2023.10589162 [14] P. Haindl and G. Weinberger. 2024. Does ChatGPT Help Novice Programmers Write Better Code? Results From Static Code Analysis. IEEE Access 12 (2024), 114146–114156. doi:10.1109/ACCESS.2024.3445432 [15] P. Haindl and G. Weinberger. 2024. Students’ Experiences of Using ChatGPT in an Undergraduate Programming Course. IEEE Access 12 (2024), 43519–43529. doi:10.1109/ACCESS.2024.3380909 [16] Irene Hou, Sophia Mettille, Owen Man, Zhuo Li, Cynthia Zastudil, and Stephen MacNeil. 2024. The Effects of Generative AI on Computing Students’ Help- Seeking Preferences - Proceedings of the 26th Australasian Computing Education Conference. In Proceedings of the 26th Australasian Computing Education Con- ference (ACE ’24). Association for Computing Machinery, New York, NY, USA, 39–48. doi:10.1145/3636243.3636248 [17] Wen-Jung Hsin. 2024. The Effect of ChatGPT: Student Perspective and Perfor- mance Achievement. J. Comput. Sci. Coll. 39, 6 (5 2024), 20–29. [18] Minjie Hu, Tony Assadi, and HamidMahroeian. 2023. Explicitly Introducing Chat- GPT into First-year Programming Practice: Challenges and Impact. In 2023 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE). 1–6. doi:10.1109/TALE56641.2023.10398297 [19] Donald H. Heberer Jr. 2021. Teacher Perceptions and Practice of Technology Inte- gration Before and After PICRAT Matrix Professional Development Intervention. Ph. D. Dissertation. The School of Education, St. John’s University, New York, NY. [20] M. Kazemitabaar, J. Chow, C.K.T. Ma, B.J. Ericson, D. Weintrop, and T. Grossman. 2023. Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming. Conference on Human Factors in Computing Systems - Proceedings (2023). doi:10.1145/3544548.3580919 [21] Ahmed Kharrufa and Ian Johnson. 2024. The Potential and Implications of Generative AI on HCI Education - Proceedings of the 6th Annual Symposium on HCI Education. In Proceedings of the 6th Annual Symposium on HCI Education (EduCHI ’24). Association for Computing Machinery, New York, NY, USA. doi:10. 1145/3658619.3658627 [22] Royce Kimmons, Charles R Graham, and Richard E West. 2020. The PICRAT Model for Technology Integration in Teacher Preparation. Contemporary Issues in Technology and Teacher Education (2020). https://citejournal.org/volume-20/issue-1-20/general/the-picrat-model- for-technology-integration-in-teacher-preparation/ [23] T. Kosar, D. Ostojić, Y.D. Liu, and M. Mernik. 2024. Computer Science Education in ChatGPT Era: Experiences from an Experiment in a Programming Course for Novice Programmers. Mathematics 12, 5 (2024). doi:10.3390/math12050629 [24] Harsh Kumar, Ruiwei Xiao, Benjamin Lawson, IlyaMusabirov, Jiakai Shi, Xinyuan Wang, Huayin Luo, Joseph Jay Williams, Anna N. Rafferty, John Stamper, and Michael Liut. 2024. Supporting Self-Reflection at Scale with Large Language Models: Insights from Randomized Field Experiments in Classrooms - Proceed- ings of the Eleventh ACM Conference on Learning @ Scale. In Proceedings of the Eleventh ACM Conference on Learning @ Scale (L@S ’24). Association for Computing Machinery, New York, NY, USA, 86–97. doi:10.1145/3657604.3662042 [25] S. Lee and K.-S. Song. 2024. Teachers’ and students’ perceptions of AI-generated concept explanations: Implications for integrating generative AI in computer science education. Computers and Education: Artificial Intelligence 7 (2024). doi:10. 1016/j.caeai.2024.100283 [26] Juho Leinonen, Paul Denny, Stephen MacNeil, Sami Sarsa, Seth Bernstein, Joanne Kim, Andrew Tran, and Arto Hellas. 2023. Comparing Code Explanations Created by Students and Large Language Models - Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (ITiCSE 2023). Association for Computing Machinery, New York, NY, USA, 124–130. doi:10.1145/3587102.3588785 [27] J. Liao, L. Zhong, L. Zhe, H. Xu,M. Liu, and T. Xie. 2024. Scaffolding Computational Thinking With ChatGPT. IEEE Transactions on Learning Technologies 17 (2024), 1668–1682. doi:10.1109/TLT.2024.3392896 [28] Mengqi Liu and FatenM’Hiri. 2024. Beyond Traditional Teaching: Large Language Models as Simulated Teaching Assistants in Computer Science - Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1. In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2024). Association for Computing Machinery, New York, NY, USA, 743–749. doi:10.1145/3626252.3630789 [29] Evanfiya Logacheva, Arto Hellas, James Prather, Sami Sarsa, and Juho Leinonen. 2024. Evaluating Contextually Personalized Programming Exercises Created with Generative AI - Proceedings of the 2024 ACM Conference on International Computing Education Research - Volume 1. In Proceedings of the 2024 ACM Conference on International Computing Education Research - Volume 1 (ICER ’24). Association for Computing Machinery, New York, NY, USA, 95–113. doi:10.1145/ 3632620.3671103 [30] Stephen Macneil, Paul Denny, Andrew Tran, Juho Leinonen, Seth Bernstein, Arto Hellas, Sami Sarsa, and Joanne Kim. 2024. Decoding Logic Errors: A Comparative Study on Bug Detection by Students and Large Language Models - Proceedings of the 26th Australasian Computing Education Conference. In Proceedings of the 26th Australasian Computing Education Conference (ACE ’24). Association for Computing Machinery, New York, NY, USA, 11–18. doi:10.1145/3636243.3636245 [31] Joyce Mahon, Brian Mac Namee, and Brett A. Becker. 2024. Guidelines for the Evolving Role of Generative AI in Introductory Programming Based on Emerging Practice. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 (Milan, Italy) (ITiCSE 2024). Association for Computing Machinery, New York, NY, USA, 10–16. doi:10.1145/3649217.3653602 [32] Giacomo Marzi, Marco Balzano, and Davide Marchiori. 2024. K-Alpha Calcula- tor–Krippendorff’s Alpha Calculator: A user-friendly tool for computing Krip- pendorff’s Alpha inter-rater reliability coefficient. MethodsX 12 (2024), 102545. doi:10.1016/j.mex.2023.102545 [33] Ha Nguyen, Nate Stott, and Vicki Allan. 2024. Comparing Feedback from Large Language Models and Instructors: Teaching Computer Science at Scale - Proceed- ings of the Eleventh ACM Conference on Learning @ Scale. In Proceedings of the Eleventh ACM Conference on Learning @ Scale (L@S ’24). Association for Com- puting Machinery, New York, NY, USA, 335–339. doi:10.1145/3657604.3664660 [34] M Ouzzani, H Hammady, Z Fedorowicz, and A Elmagarmid. 2017. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016; 5: 210. https://www. rayyan.ai/ [35] Matthew J Page, Joanne E McKenzie, Patrick M Bossuyt, Isabelle Boutron, Tammy C Hoffmann, Cynthia D Mulrow, Larissa Shamseer, Jennifer M Tetzlaff, Elie A Akl, Sue E Brennan, Roger Chou, Julie Glanville, Jeremy M Grimshaw, Asbjørn Hróbjartsson, Manoj M Lalu, Tianjing Li, Elizabeth W Loder, Evan Mayo- Wilson, Steve McDonald, Luke A McGuinness, Lesley A Stewart, James Thomas, Andrea C Tricco, Vivian A Welch, Penny Whiting, and David Moher. 2021. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. (2021), n71. doi:10.1136/bmj.n71 [36] M. Pankiewicz and R.S. Baker. 2024. Navigating Compiler Errors with AI Assis- tance - A Study of GPT Hints in an Introductory Programming Course. Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE 1 (2024), 94–100. doi:10.1145/3649217.3653608 [37] James Prather, Paul Denny, Juho Leinonen, Brett A. Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton- Reilly, Stephen MacNeil, Andrew Petersen, Raymond Pettit, Brent N. Reeves, and Jaromir Savelka. 2023. The Robots Are Here: Navigating the Generative AI Revolution in Computing Education. In Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education (Turku, Finland) (ITiCSE-WGR ’23). Association for Computing Machinery, New York, NY, USA, 108–159. doi:10.1145/3623762.3633499 [38] B. Mairéad Pratschke. 2024. Generative AI and education. Springer. [39] Michael P. Rogers, Hannah Miller Hillberg, and Christopher L. Groves. 2024. Attitudes Towards the Use (and Misuse) of ChatGPT: A Preliminary Study - Pro- ceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1. In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2024). Association for Computing Machinery, New York, NY, USA, 1147–1153. doi:10.1145/3626252.3630784 [40] N. Selwyn. 2010. Looking beyond learning: notes towards the critical study of educational technology. Journal of Computer Assisted Learning 26, 1 (1 2010), 65–73. doi:10.1111/j.1365-2729.2009.00338.x [41] Neil Selwyn. 2016. Is technology good for education? John Wiley & Sons. [42] D. Sun, A. Boudouaia, C. Zhu, and Y. Li. 2024. Would ChatGPT-facilitated pro- gramming mode impact college students’ programming behaviors, performances, and perceptions? An empirical study. International Journal of Educational Tech- nology in Higher Education 21, 1 (2024). doi:10.1186/s41239-024-00446-5 [43] Woonhee Sung, Heejung An, and Christopher L. Thomas. 2024. PICRAT Analysis of Technology-Integration Activities in U.S. K–12 Public Schools. (2024), 1–24. doi:10.1080/07380569.2024.2338243 [44] M. Wieser, K. Schöffmann, D. Stefanics, A. Bollin, and S. Pasterk. 2023. Investi- gating the Role of ChatGPT in Supporting Text-Based Programming Education for Students and Teachers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 14296 (2023), 40–53. doi:10.1007/978-3-031-44900-0_4 [45] Yuankai Xue, Hanlin Chen, Gina R. Bai, Robert Tairas, and Yu Huang. 2024. Does ChatGPT Help With Introductory Programming?An Experiment of Students Using ChatGPT in CS1 - Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training. In Pro- ceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET ’24). Association for Computing Machinery, New York, NY, USA, 331–341. doi:10.1145/3639474.3640076 [46] R. Yilmaz and F.G. Karaoglan Yilmaz. 2023. The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, pro- gramming self-efficacy and motivation. Computers and Education: Artificial Intelligence 4 (2023). doi:10.1016/j.caeai.2023.100147 [47] Yong Zheng. 2023. ChatGPT for Teaching and Learning: An Experience from Data Science Education - Proceedings of the 24th Annual Conference on In- formation Technology Education. In Proceedings of the 24th Annual Conference on Information Technology Education (SIGITE ’23). Association for Computing Machinery, New York, NY, USA, 66–72. doi:10.1145/3585059.3611431 https://doi.org/10.1109/HNICEM60674.2023.10589162 https://doi.org/10.1109/ACCESS.2024.3445432 https://doi.org/10.1109/ACCESS.2024.3380909 https://doi.org/10.1145/3636243.3636248 https://doi.org/10.1109/TALE56641.2023.10398297 https://doi.org/10.1145/3544548.3580919 https://doi.org/10.1145/3658619.3658627 https://doi.org/10.1145/3658619.3658627 https://citejournal.org/volume-20/issue-1-20/general/the-picrat-model-for-technology-integration-in-teacher-preparation/ https://citejournal.org/volume-20/issue-1-20/general/the-picrat-model-for-technology-integration-in-teacher-preparation/ https://doi.org/10.3390/math12050629 https://doi.org/10.1145/3657604.3662042 https://doi.org/10.1016/j.caeai.2024.100283 https://doi.org/10.1016/j.caeai.2024.100283 https://doi.org/10.1145/3587102.3588785 https://doi.org/10.1109/TLT.2024.3392896 https://doi.org/10.1145/3626252.3630789 https://doi.org/10.1145/3632620.3671103 https://doi.org/10.1145/3632620.3671103 https://doi.org/10.1145/3636243.3636245 https://doi.org/10.1145/3649217.3653602 https://doi.org/10.1016/j.mex.2023.102545 https://doi.org/10.1145/3657604.3664660 https://www.rayyan.ai/ https://www.rayyan.ai/ https://doi.org/10.1136/bmj.n71 https://doi.org/10.1145/3649217.3653608 https://doi.org/10.1145/3623762.3633499 https://doi.org/10.1145/3626252.3630784 https://doi.org/10.1111/j.1365-2729.2009.00338.x https://doi.org/10.1186/s41239-024-00446-5 https://doi.org/10.1080/07380569.2024.2338243 https://doi.org/10.1007/978-3-031-44900-0_4 https://doi.org/10.1145/3639474.3640076 https://doi.org/10.1016/j.caeai.2023.100147 https://doi.org/10.1145/3585059.3611431 Abstract 1 Introduction 2 Background and Related Work 3 The PICRAT Framework 4 Research questions 5 Methods 5.1 Search Strategy 5.2 Inclusion Criteria 5.3 Analysis Procedure 6 Results and Interpretation 6.1 Sample 6.2 RQ1 How the technology was used 6.3 RQ2 Computing Students' Use (PIC) 6.4 RQ3 Computing Teacher Practice (RAT) 7 Limitations 8 Recommendations and Future Work 9 Conclusion References