Document Type
Article
Publication Date
1-9-2026
Department
Department of Engineering Fundamentals; Department of Computer Science
Abstract
This study investigated the impact of leveraging generative artificial intelligence (GenAI) to assist 1st-year engineering and computer science (CS) students in reading code in a new (to them) language. Students were asked to comment code in FORTRAN. They were then asked to run the code through ChatGPT-4.0 for its comments and reflect on what they learned from the experience. Participants completed survey items from Ramalingam and Wiedenbeck’s Computer Programming Self-Efficacy Scale (CPSES) prior to and after the intervention. Additional open-ended reflective (qualitative) questions were added to the quantitative questions in the postintervention questionnaire. This study documents increases in self-efficacy for programming independence and persistence (Factor 1 of the CPSES) as well as for complex programming tasks (Factor 2) after students used ChatGPT for generating code explanations. Both CS and engineering students showed improvements in programming independence and persistence; but only engineers showed significant improvements in their confidence regarding complex programming tasks. Men experienced a significant increase in self-efficacy on Factor 1 while women experienced a significant increase on Factor 2. The qualitative data point to an increase in student understanding of the new code and suggest that although students may be more likely to use GenAI for assistance as they progress through programming courses, guiding students in using GenAI to understand code may shift students’ intent away from using GenAI to write code for them. Thus, we recommend that programming faculty instruct students how to interact with GenAI.
Publication Title
Journal of Teaching and Learning with Technology
Recommended Citation
Jarvie-Eggart, M.,
Teahen, J.,
Masker, D. T.,
Padilla, J.,
Ureel, L. C.,
Brown, L.,
Pomerville, S.,
&
Sticken, J.
(2026).
Jiving with LLMs: Assessing First Year Students’ Computer Programming Self Efficacy After Reading Code With LLMs.
Journal of Teaching and Learning with Technology,
14(1).
http://doi.org/10.14434/jotlt.v14i1.41701
Retrieved from: https://digitalcommons.mtu.edu/michigantech-p2/2685
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.
Version
Publisher's PDF
Publisher's Statement
© 2026. Authors retain copyright and grant the Journal of Teaching and Learning with Technology (JoTLT) right of first publication with the work simultaneously licensed under a Creative Commons Attribution License, (CC-BY) 4.0 International, allowing others to share the work with proper acknowledgement and citation of the work's authorship and initial publication in JoTLT.