Emerging tensions around learning with LLM-based chatbots

A CHAT approach

Abstract

This paper investigates the tensions introduced by the introduction of large language model (LLM)-based chatbots in higher education, particularly within a distance university that embraces a networked learning approach. LLM-based chatbots such as ChatGPT are applications that simulate conversations with humans, powered by AI-trained large language models. The study is grounded in a socio-material perspective, considering the digital as inherently intertwined with social, material, and broader contextual factors. Utilising cultural-historical activity theory (CHAT) as a framework, the research examines the changing human-technology relationship and the emerging tensions within the learning activity system.
A survey conducted in Spring 2023 at OnlineUni (name changed) revealed that while students appreciate the immediate, personalised responses from LLM-based chatbots, there is significant distrust in the output, and they raised concerns about the quality of learning and the value of their diplomas. Teachers share these concerns and question the evolving roles and responsibilities in education due to such chatbots. The survey, which included responses from 584 students and 171 teachers, highlighted the use of LLM-based chatbots for various purposes, such as obtaining quick answers to generating content, and underscored the need for support in integrating them into learning and professional activities.
The CHAT framework allows for an analysis of the activity system, identifying tensions in tool mediation, division of labour, and rules. Students are grappling with the balance between human and AI contributions to learning, the potential for chatbots to undermine the learning process, and the need for clear guidelines on using these technologies. The paper discusses the implications of these tensions on the learning objectives, the role of the community, and the division of labour within the educational setting.
The findings suggest that the introduction of LLM-based chatbots is forcing a re-evaluation of what should be taught and how learning could be facilitated. There is a call for critical AI literacy and a better understanding of the redefinition of the division of labour between students, AI, and the community. The study highlights the necessity for collaborative rule-making to navigate the complexities of integrating LLM-based chatbots into higher education, ensuring fairness and accessibility while maintaining the quality of learning.
In conclusion, the paper emphasises the need for ongoing research to understand the evolving dynamics of human and non-human actors in the learning process. It calls for a closer examination of the interactions within the learning activity system, particularly the instrumental genesis of student-tool relationships, to inform the development of pedagogical strategies that leverage the capabilities of LLM-based chatbots without compromising the collaborative values of education.