Content Analysis as a Means of Quality Assurance as Exemplified in a Course on Organisational Development

Abstract

In this paper, we describe the qualitative content analysis (QCA) of students' written reactions to a blended learning course on organisational development. The organisational development course was held at the University of Vienna as part of the "business informatics" master's programme in the summer term 2006 with 20 participating students. The course was based on the principles of Person-Centred e-Learning (PCeL), which integrates Person-Centred Teaching as developed by the American psychologist Carl Rogers with elements of e-learning. PCeL is a blended learning approach aiming to address the whole person, i.e., his/her intellect, skills and personal intentions and feelings. The main learning goals of the course were to develop personal visions, to be introduced to the concepts of a learning organisation, to improve communication, to increase openness to experience, and to improve one's effectiveness in use of new media.

Online reaction sheets, which were analysed using the QCA method presented in this paper, offered students the opportunity to reflect on their experiences, cognitions and feelings about the preceding course unit. Reaction sheets were requested after each course unit. They were accessible to all participants of the course via the learning platform and were discussed at the beginning of the following course unit.

The main question of interest underlying the qualitative analysis of course reaction sheets was: What were crucial experiences of the students participating the course (i.e., positive and negative aspects, significant events, etc.)? The QCA as a systematic, rule-guided qualitative text analysis method was performed using a procedure of nine steps proposed by Mayring: collection and determination of text material, analysis of the situation in which the text originated, formal characterisation of the material, specification of the subject of analysis, theoretically informed differentiation of research questions to be answered, definition of the unit of analysis, selection of a qualitative analytical technique, verification of the category system, and interpretation. The content body comprised 71 reaction sheets with a total of 10,750 words. We used summarization as qualitative analytical technique and developed the category system in an inductive way.

The results of the qualitative content analysis were compared with quantitative data of the online questionnaire. For example, the most profitable aspects as judged by students in the online questionnaire were also those that were frequently mentioned as positive aspects in the reaction sheets, e.g. nearly all students experienced discussions and dialogues as important and useful because these allowed, for example, expressing different opinions, and supported better understanding of topics. Some students argued that discussions stimulated them to participate more actively. Yet, open reaction sheets brought up aspects that were not explicitly included in the structured online questionnaire, e.g. most students tended to complain about the arrangement of the course because of, for example, too few breaks and some long sessions. Some students argued that there was not enough time to discuss certain topics and to do certain exercises. This way we were able to produce a comprehensive, multi-perspective evaluation to be used in Action Research-procedures that have proven effective to investigate and improve blended learning courses.