Students’ Use of ChatGPT in an Introductory Programming Course: A Deep Dive into Chat Protocols and the Student Perspective

Abstract

Within just a few years, Generative AI (GenAI) and related tools proved their impact on higher education, including computing disciplines. Their performance and feedback capabilities are convincing, especially for introductory classes, e.g., CS1 or Introduction to Programming. It is, therefore, no surprise that students are using GenAI tools at a great scale. In this invited research paper, we investigate students’ use and perception of ChatGPT in an Introductory Programming class at Goethe University, Frankfurt, during the winter term 2023/24. To address this goal, we asked students to solve programming exercises with the assistance of ChatGPT as part of their weekly introductory course exercises. 213 students submitted their chat protocols (with 2335 prompts in sum) as a data basis for this analysis. The data was analyzed w.r.t. the prompts, frequencies, the chats’ progress, contents, and other use patterns. In addition, students were asked to provide information regarding their use of ChatGPT and their evaluation of the tool via an online survey (n=298). The chat protocols revealed a great variety of interactions and (follow-up) prompts, both potentially supportive and concerning. Students’ responses to the survey added a diverse range of perceptions, indicating a wide adoption but also critical engagement. Learning about students’ interactions with ChatGPT will help inform and align teaching practices and instructions for future introductory programming courses in higher education. Therefore, this work has implications for tool creators and educators who want to design pedagogical instruction or guardrails for students and help inform their reflected use of GenAI tools.