“Remember to hand out medals”

Value and peer rating in an online open study group

Abstract

Rating systems are popular in recommender systems and ecommerce websites, but recently there have been some implementations in online educational settings as well. This paper presents an exploratory qualitative study of the way a peer rating system influences value creation within a large, teacherless online open study group where the participants connect to and help each other in their process of learning Python, a programming language. In this study group, the participants ask questions and receive answers from their peers. The participants are strongly encouraged to acknowledge good answers from their peers by rewarding them with medals. Using a combination of preexisting and inductive codes, qualitative content analysis was used to examine instances of the value provided by good answers, which were rated as “best responses” and awarded medals. The analysis focused on 108 closed questions asked by a homogeneous sample of participants whose title indicated the status of beginners. Closed questions are threads including a top-level question about a topic, followed by one or more answers and/or other questions. Turning attention on the materiality of the rating system, this study attempts to bring into focus how this device contributes to value creation. The analysis suggests that the peer rating system makes visible what the participants find immediately valuable. By making good responses recognizable, the rating system makes more visible what the participants can gain from each other and what they can achieve by helping each other. The medals awarded to participants giving good responses act as “tokens of appreciation” and partake of a mechanism aimed at supporting motivation, engagement, and commitment to participation in the study group. The rating system contributes to a rank measured in capacity to be committed to the study group, help others and solve problems.