Keywords
Individualized assessment
Group work assessment
Tutorial assessment
Observational assessment
Collaborative learning
Performance indicators
How to Cite
Abstract
Problem-based learning (PBL) tutorials are designed to make learning visible through collaborative inquiry, yet the evidence generated in these settings is often collective, distributed and difficult to attribute to individual students. Programmes must nevertheless make defensible individual judgements for feedback, remediation, progression and, in some contexts, summative assessment. This creates a persistent assessment problem: the process evidence most relevant to PBL is difficult to capture, while the scores required for institutional decision-making risk compressing collaborative performance into overly simple or performative measures. Existing approaches, including peer ratings, group products, post-session tutor judgements and lengthy item-based instruments, may provide limited individualisation or impose substantial rater burden. This pilot instrument-development study reports the design and initial evaluation of EIPA-PBL, an Embedded Individualised Performance Assessment approach for PBL tutorials. EIPA-PBL uses a schematic seating map and a small set of observable indicators to record individual learning-relevant behaviours during tutorials. Individualisation refers to the generation of individual evidence profiles within a shared indicator framework, rather than the use of different criteria for different students. Data from 37 medical students across five PBL modules yielded 43 student-module enrolment records and approximately 7,000 coded indicator events. Intraclass correlation coefficients showed indicator-specific cross-session patterns, while exploratory factor analysis suggested coherent clustering of background knowledge, ideas and minor contributions, with major contributions and questions requiring refinement. Student feedback provided preliminary acceptability evidence. The study broadly supports feasibility and preliminary construct exploration, but inter-rater reliability, response-process evidence and consequences research remain necessary. EIPA-PBL is presented as a candidate framework for formative, portfolio and bounded low-stakes summative use.
References
Alimoglu, M. K., Sarac, D. B., Alparslan, D., Akman Karakas, A., & Altintas, L. (2014). An observation tool for instructor and student behaviors to measure in-class learner engagement: A validation study. Medical Education Online, 19, 24037. https://doi.org/10.3402/meo.v19.24037
Barrows, H. S., & Tamblyn, R. M. (1980). Problem-based learning: An approach to medical education. Springer.
Bearman, M., & Ajjawi, R. (2018). From seeing through to seeing with: Assessment criteria and the myths of transparency. Frontiers in Education, 3, 96. https://doi.org/10.3389/feduc.2018.00096
Borsboom, D., Cramer, A. O. J., Kievit, R. A., Zand Scholten, A., & Franic, S. (2009). The end of construct validity. In R. W. Lissitz (Ed.), The concept of validity (pp. 135–170). Information Age Publishing Inc. https://doi.org/10.1108/978-1-61735-269-020251010
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071. https://doi.org/10.1037/0033-295x.111.4.1061
Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 41(3), 400–413. https://doi.org/10.1080/02602938.2014.999746
Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823
Chin, C., & Osborne, J. (2008). Students' questions: A potential resource for teaching and learning science. Studies in Science Education, 44(1), 1–39. https://doi.org/10.1080/03057260701828101
Cook, D. A., Kuper, A., Hatala, R., & Ginsburg, S. (2016). When assessment data are words: Validity evidence for qualitative educational assessments. Academic Medicine, 91(10), 1359–1369. https://doi.org/10.1097/acm.0000000000001175
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(1), 7. https://doi.org/10.7275/jyj1-4868
Cowan, N. (2010). The magical mystery four: How is working memory capacity limited, and why? Current Directions in Psychological Science, 19(1), 51–57. https://doi.org/10.1177/0963721409359277
Czabanowska, K., Moust, J. H., Meijer, A. W., Schröder-Bäck, P., & Roebertsen, H. (2012). Problem-based learning revisited: Introduction of active and self-directed learning to reduce fatigue among students. Journal of University Teaching & Learning Practice, 9(1), Article 6. https://doi.org/10.53761/1.9.1.6
de Hei, M., Strijbos, J.-W., Sjoer, E., & Admiraal, W. (2016). Thematic review of approaches to design group learning activities in higher education: The development of a comprehensive framework. Educational Research Review, 18, 33–45. https://doi.org/10.1016/j.edurev.2016.01.001
Dijkstra, J., Latijnhouwers, M., Norbart, A., & Tio, R. A. (2016). Assessing the 'I' in group work assessment: State of the art and recommendations for practice. Medical Teacher, 38(7), 675–682. https://doi.org/10.3109/0142159x.2016.1170796
Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches (pp. 1–19). Elsevier.
Dolmans, D. H. J. M., De Grave, W., Wolfhagen, I. H. A. P., & van der Vleuten, C. P. M. (2005). Problem-based learning: Future challenges for educational practice and research. Medical Education, 39(7), 732–741. https://doi.org/10.1111/j.1365-2929.2005.02205.x
Eva, K. W. (2001). Assessing tutorial-based assessment. Advances in Health Sciences Education, 6(3), 243–257. https://doi.org/10.1023/A:1012743830638
Eva, K. W., Solomon, P., Neville, A. J., Ladouceur, M., Kaufman, K., Walsh, A., & Norman, G. R. (2007). Using a sampling strategy to address psychometric challenges in tutorial-based assessments. Advances in Health Sciences Education, 12(1), 19–33.
https://doi.org/10.1007/s10459-005-2327-z
Gingerich, A., Kogan, J., Yeates, P., Govaerts, M., & Holmboe, E. (2014). Seeing the 'black box' differently: Assessor cognition from three research perspectives. Medical Education, 48(11), 1055–1068. https://doi.org/10.1111/medu.12546
Govaerts, M. J. B., & van der Vleuten, C. P. M. (2013). Validity in work-based assessment: Expanding our horizons. Medical Education, 47(12), 1164–1174. https://doi.org/10.1111/medu.12289
Hébert, R., & Bravo, G. (1996). Development and validation of an evaluation instrument for medical students in tutorials. Academic Medicine, 71(5), 488–494. https://doi.org/10.1097/00001888-199605000-00020
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266. https://doi.org/10.1023/B:EDPR.0000034022.16470.f3
Hmelo-Silver, C. E., & Barrows, H. S. (2006). Goals and strategies of a problem-based learning facilitator. Interdisciplinary Journal of Problem-Based Learning, 1(1), 21–39. https://doi.org/10.7771/1541-5015.1004
Hmelo-Silver, C. E., & Barrows, H. S. (2008). Facilitating collaborative knowledge building. Cognition and Instruction, 26(1), 48–94. https://doi.org/10.1080/07370000701798495
Hutchins, E. (1995). Cognition in the wild. MIT Press. https://doi.org/10.7551/mitpress/1881.001.0001
Johnson, D. W., & Johnson, R. T. (2005). New developments in social interdependence theory. Genetic, Social, and General Psychology Monographs, 131(4), 285–358.
https://doi.org/10.3200/MONO.131.4.285-358
Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130–144. https://doi.org/10.1016/j.edurev.2007.05.002
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
Karau, S. J., & Williams, K. D. (1993). Social loafing: A meta-analytic review and theoretical integration. Journal of Personality and Social Psychology, 65(4), 681–706. https://doi.org/10.1037/0022-3514.65.4.681
King, A. (1994). Guiding knowledge construction in the classroom: Effects of teaching children how to question and how to explain. American Educational Research Journal, 31(2), 338–368. https://doi.org/10.3102/00028312031002338
Kozlowski, S. W. J., & Ilgen, D. R. (2006). Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest, 7(3), 77–124. https://doi.org/10.1111/j.1529-1006.2006.00030.x
Kulhavy, R. W., & Stock, W. A. (1996). How cognitive maps are learned and remembered. Annals of the Association of American Geographers, 86(1), 123–145. https://doi.org/10.1111/j.1467-8306.1996.tb01748.x
Latané, B., Williams, K., & Harkins, S. (1979). Many hands make light the work: The causes and consequences of social loafing. Journal of Personality and Social Psychology, 37(6), 822–832. https://doi.org/10.1037/0022-3514.37.6.822
MacDonald, R. (2005). Assessment strategies for enquiry and problem-based learning. In T. Barrett, I. Mac Labhrainn, & H. Fallon (Eds.), Handbook of enquiry and problem-based learning: Irish case studies and international perspectives (pp. 85–94). Centre for Excellence in Learning and Teaching, NUI Galway, in association with the All Ireland Society for Higher Education.
Macfarlane, B. (2015). Student performativity in higher education: Converting learning as a private space into a public performance. Higher Education Research & Development, 34(2), 338–350. https://doi.org/10.1080/07294360.2014.956697
Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework and taxonomy of team processes. Academy of Management Review, 26(3), 356–376. https://doi.org/10.5465/amr.2001.4845785
Mercer, N. (2000). Words and minds: How we use language to think together. Routledge.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066X.50.9.741
Nendaz, M. R., & Tekian, A. (1999). Assessment in Problem-Based Learning Medical Schools: A Literature Review. Teaching and Learning in Medicine, 11(4), 232–243. https://doi.org/10.1207/S15328015TLM110408
Nijstad, B. A., & Stroebe, W. (2006). How the group affects the mind: A cognitive model of idea generation in groups. Personality and Social Psychology Review, 10(3), 186–213. https://doi.org/10.1207/s15327957pspr1003_1
Norman, G., Bordage, G., Page, G., & Keane, D. (2006). How specific is case specificity? Medical Education, 40(7), 618–623. https://doi.org/10.1111/j.1365-2929.2006.02511.x
Pervaz Iqbal, M., Velan, G. M., O'Sullivan, A. J., & Balasooriya, C. (2020). The collaborative learning development exercise (CLeD-EX): An educational instrument to promote key collaborative learning behaviours in medical students. BMC Medical Education, 20(1), 62. https://doi.org/10.1186/s12909-020-1977-0
Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435–448. https://doi.org/10.1080/02602930902862859
Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education, 34(2), 159–179. https://doi.org/10.1080/02602930801956059
Salas, E., Sims, D. E., & Burke, C. S. (2005). Is there a big five in teamwork? Small Group Research, 36(5), 555–599. https://doi.org/10.1177/1046496405277134
Savery, J. R. (2006). Overview of problem-based learning: Definitions and distinctions. Interdisciplinary Journal of Problem-Based Learning, 1(1), 9–20. https://doi.org/10.7771/1541-5015.1002
Savin-Baden, M. (2004). Understanding the impact of assessment on students in problem-based learning. Innovations in Education and Teaching International, 41(2), 221–233. https://doi.org/10.1080/1470329042000208729
Schmidt, H. G. (1983). Problem-based learning: Rationale and description. Medical Education, 17(1), 11–16.
https://doi.org/10.1111/j.1365-2923.1983.tb01086.x
Schmidt, H. G., Rotgans, J. I., & Yew, E. H. J. (2011). The process of problem-based learning: What works and why. Medical Education, 45(8), 792–806. https://doi.org/10.1111/j.1365-2923.2011.04035.x
Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19(4), 460–475. https://doi.org/10.1006/ceps.1994.1033
Schuwirth, L. W. T., & van der Vleuten, C. P. M. (2011). Programmatic assessment: From assessment of learning to assessment for learning. Medical Teacher, 33(6), 478–485. https://doi.org/10.3109/0142159X.2011.565828
Scott, B. M., & Schwartz, N. H. (2007). Navigational spatial displays: The role of metacognition as cognitive load. Learning and Instruction, 17(1), 89–105. https://doi.org/10.1016/j.learninstruc.2006.11.008
Segers, M., & Dochy, F. (2001). New Assessment Forms in Problem-based Learning: The value-added of the students' perspective. Studies in Higher Education, 26(3), 327–343. https://doi.org/10.1080/03075070120076291
Sim, S. M., Azila, N. M. A., & Lee, M. S. (2006). A simple instrument for the assessment of student performance in problem-based learning tutorials. Annals of the Academy of Medicine, Singapore, 35(9), 634–641. https://doi.org/10.47102/annals-acadmedsg.V35N9p634
Slavin, R. E. (1996). Research on cooperative learning and achievement: What we know, what we need to know. Contemporary Educational Psychology, 21(1), 43–69. https://doi.org/10.1006/ceps.1996.0004
St-Onge, C., Frenette, E., Cote, D. J., & De Champlain, A. (2014). Multiple tutorial-based assessments: A generalizability study. BMC Medical Education, 14, 30. https://doi.org/10.1186/1472-6920-14-30
Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. MIT Press. https://doi.org/10.7551/mitpress/3372.001.0001
Stewart, D. W., Shamdasani, P. N., & Rook, D. W. (2007). Focus groups: Theory and practice, 2nd edSage Publications, Inc. https://doi.org/10.4135/9781412991841
Strijbos, J.-W. (2011). Assessment of (computer-supported) collaborative learning. IEEE Transactions on Learning Technologies, 4(1), 59–73. https://doi.org/10.1109/tlt.2010.37
Tavares, W., & Eva, K. W. (2013). Exploring the impact of mental workload on rater-based assessments. Advances in Health Sciences Education, 18(2), 291–303. https://doi.org/10.1007/s10459-012-9370-3
Torrance, H. (2007). Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post-secondary education and training can come to dominate learning. Assessment in Education: Principles, Policy & Practice, 14(3), 281–294. https://doi.org/10.1080/09695940701591867
Uijtdehaage, S., & Schuwirth, L. W. T. (2018). Assuring the quality of programmatic assessment: Moving beyond psychometrics. Perspectives on Medical Education, 7(6), 350–351.
https://doi.org/10.1007/s40037-018-0485-y
Van Berkel, H. J. M., & Schmidt, H. G. (2000). Motivation to commit oneself as a determinant of achievement in problem-based learning. Higher Education, 40(2), 231–242. https://doi.org/10.1023/A:1004022116365
van der Vleuten, C. P. M., & Schuwirth, L. W. T. (2019). Assessment in the context of problem-based learning. Advances in Health Sciences Education, 24(5), 903–914. https://doi.org/10.1007/s10459-019-09909-1
van Merriënboer, J. J. G., & Sweller, J. (2010). Cognitive load theory in health professional education: Design principles and strategies. Medical Education, 44(1), 85–93. https://doi.org/10.1111/j.1365-2923.2009.03498.x
Visschers-Pleijers, A. J. S. F., Dolmans, D. H. J. M., De Grave, W. S., Wolfhagen, I. H. A. P., Jacobs, J. A., & van der Vleuten, C. P. M. (2006). Student perceptions about the characteristics of an effective discussion during the reporting phase in problem-based learning. Medical Education, 40(9), 924–931. https://doi.org/10.1111/j.1365-2929.2006.02548.x
Webb, N. M. (1989). Peer interaction and learning in small groups. International Journal of Educational Research, 13(1), 21–39. https://doi.org/10.1016/0883-0355(89)90014-1
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright (c) 2026 Colin Greengrass