Beyond conventional teaching towards networked learning

The role of generative AI chatbots in enhancing program evaluation skills

Abstract

With Generative Artificial Intelligence (GenAI) adoption growing, education has seen the emergence of innovative technologies like chatbots. However, little research has examined the impacts of GenAI integration in specialized higher education contexts. This study explored graduate students’ experiences using a GenAI chatbot, PEARL, within a graduate-level teacher education course focused on teaching students how to collaboratively conduct program evaluations using practice cases. Four students participated in our study and shared perceptions of interviewing personas with PEARL when evaluating the practice cases. Thematic analysis identified advantages like enhanced efficiency and accessibility, plus limitations regarding authenticity of artificial interactions. Findings emphasized the continued importance of human guidance and peer learning to enrich GenAI-enabled education aligning with principles of networked learning. Students highlighted the need for ethical considerations despite interacting with artificial entities, underscoring nuanced understanding. The significance of collaborative analysis and ongoing iterative improvements also emerged as themes integral to meaningful learning. Although GenAI presents transformational potential in instructional designs, findings support the use of blended approaches that strategically integrate its advantages with human activity and collaborative inquiry. The study makes contributions by elucidating domain-specific nuances of integrating GenAI into teaching in higher education. Practical implications encourage scaffolding GenAI curricula to promote authenticity and collaborative knowledge construction. Further research could examine variations across disciplines, technologies, and demographics. Overall, as GenAI shapes academia’s evolution, reflective pedagogical examination will be key to evidence-guided integration. This exploratory study presents a preliminary yet important step, unveiling opportunities for networked learning and complexities of GenAI adoption in teaching program evaluation skills in education contexts.