Introducing a Problem Analysis Tool Implies Increasement in Understanding Problem Analysis Among Students: a PBL Case

Problem-based learning (PBL) is the through-going didactics at Aalborg University, but literature shows how integrating PBL into project work is challenging for students. Studies indicate that students especially struggle with the problem analysis section, i.e., what it consists of, how the structure of the analysis should be, etc. Moreover, literature shows that ignorance among students leads to conflicts among group members. The aim of the study was to evaluate the consequences of introducing a problem analysis tool to master students working with a PBL project. Data analysis showed an increase (with significant p values) in the following 5 topics: 1) the problem analysis term, 2) problem analysis structure, 3) scientific argumentation, 4) learn to analyze instead of explaining, and 5) using literature to argue for a scientific problem. Significant results showed that students believed that they had increased their understanding of the term problem analysis after being introduced to the problem analysis tool.

Students with nonacademic bachelor's degrees, such as nurses, occupational therapists, and physiotherapists, have the possibility to be enrolled as master's students at AAU.After two years, they must achieve the same level of PBL didactics knowledge as students who have been studying their bachelor's degree at AAU for three years.Literature indicate how these students challenge with integrating PBL in the project work, especially work with the problem analysis part (Chen et al., 2021;Sukackė et al., 2022).
An attempt to overcome the lack of understanding the PBL principles has been to introduce a problem analysis tool.Therefore, the aim of this study was to evaluate the consequences of introducing a problem analysis tool to master students working with a PBL project.

Participants
In total, 28 students were included.17 participants were students from the master's program Clinical Science and Technology at AAU.Five participants were students from the master's program Digital Health Services (DHS) at The Arctic University of Tromsø, Norway, and the remaining participants were students from the bachelor's program in medicine at AAU.

The problem analysis tool
The problem analysis tool aims to give an overview of how a problem analysis section can be structured in a PBL project report.Moreover, it gives an idea of which elements need to be covered.Below, a presentation of the different elements is visualized.A further elaboration on the development process of the problem analysis tool will not be given in this study.

The introduction -Presentation of the initial problem
The problem analysis

An evaluation questionnaire
To evaluate a progression in the participants' understanding of the problem analysis term, we developed a questionnaire.According to Beaton et al., it can be a comprehensive and time-consuming task to develop a validated questionnaire (Beaton et al., 2000).Therefore, Beaton et al. was not followed strictly, but used as inspiration in the development process.

Development of the questionnaire
First step in developing the questionnaire was to clarify what to measure.In this study, the participants' understanding of the problem analysis was to be investigated.Thereafter, relevant questions were formulated.Each question was rated on a Likert scale going from 1-5, where 1 = totally disagree, 2 = disagree, 3 = neither nor, 4 = agree, and 5 = totally agree.

Validation and adjustments of the questionnaire
The final questionnaire was validated by two individual evaluators.Both evaluators had been supervisor before and employees at the department of health science and technology, AAU.First evaluator was associate professor with a PhD and a Master of Science degree in biomedical engineering and informatics.Second evaluator was assistant professor with a PhD and a Master of Science in Clinical Science and Technology.Both went through each question and came up with suggestions.Subsequently, the suggestions were implemented in the final questionnaire.

The final questionnaire
In total, the final questionnaire consisted of 20 questions divided in three categories and a comment section.Question 1-5 had focus on the students' understanding of the problem analysis term before being introduced to the problem analysis tool Question 6-10 had focus on the students' understanding of the problem analysis term after being introduced to the problem analysis tool Question 11-20 had focus on when to introduce the problem analysis tool

Data processing of the results
The final validated questionnaire was distributed to all participants.Microsoft Excel version 2022 was applied to conduct postprocessing.The program was used to make descriptive statistics and produce plots to visualize the results.MATLAB (r2021b, Natick, Massachusetts: The MathWorks inc.) to conduct a statistical test for differences in scores.The Mann-Whitney U test was used to assess differences in scores prior to the introduction of the problem analysis tool vs. scores post-introduction.

RESULTS
In total, 28 students received an introduction to the problem analysis tool.The introduction took approximately 1 hour.Based on experience from previous years of supervising student groups, the supervisor presented the elements in the problem analysis tool on a big monitor in the room, where the students received the supervision.The supervisor went through each section with detailed description on how to understand the reason why the specific section should be included and how it could be covered.After going through each section in the problem analysis tool and all questions the students had, the supervisor helped the students to apply the problem analysis tool on their problem analysis.
Two to three weeks after the introduction, all participants were asked to evaluate their understanding of the problem analysis term.The results are presented in Figure 1-5

Topic 1: Problem analysis term
The participants were asked to describe how well on the Likert scale they understood the problem analysis term.The question was asked before they received the problem analysis tool, and the same question was asked after the problem analysis tool was introduced.
In the 'before' category, most answered 'doubt' or 'great doubt' about content of the term.
In the 'after'-category, most students answered they no doubt about the content of the term.The participants were asked to describe how well on the Likert scale they understood the problem analysis structure.The question was asked before they received the problem analysis tool and after the problem analysis tool was introduced.Figure 2 shows two categories: 'before' and 'after'.
In the 'before' category, many answered 'doubt'.In the 'after'-category, most answered no doubt about the problem analysis structure.

Topic 3: Scientific argumentation
The participants were asked to describe how well on the Likert scale they knew scientific argumentation.The question was asked before they received the problem analysis tool and again after the problem analysis tool was introduced.Figure 3 shows two categories: 'before' and 'after'.
In the 'before' category, most answered, 'neither or no' closed, followed by 'agree' i.e., some were unsure, and some knew.In the 'after'-category, most answered no doubt.Topic 4: From explaining to analyzing The participants were asked to describe how well on the Likert scale they knew how to analyze a scientific problem.The question was asked before they received the problem analysis tool and again after the problem analysis tool was introduced.Figure 2 shows two categories: 'before' and 'after'.
In the 'before'-category, most did not become better informed on the term or had great doubt about how to analyze a scientific problem.In the 'after'-category, most answered 'no doubt' how to analyze a scientific problem.Topic 5: Using literature to argue for a scientific problem Students were asked to describe how well on the Likert scale they knew how to use literature to argue for a scientific problem.The question was asked before they received the problem analysis tool and after the problem analysis tool was introduced.Figure 5 shows two categories: 'before' and 'after'.
In the 'before' category, the answers were divided into two groups.Most 'knew how to' use literature to argue, but many also had 'great doubt'.In the 'after'-category, most answered 'no doubt' how to argue for a scientific problem.

Introducing the problem analysis tool
The participants were asked to state when they thought it would be the best time to be introduced to the problem analysis tool. Figure 6 shows the answers.Some wanted to have the tool from the beginning of the semester, while others said it was the right time.
Nevertheless, almost all participants disagreed on introducing the tool later in their semester.
Figure 6.An overview of when the problem analysis tool should be introduced according to the students.

DISCUSSION
The aim of the study was to evaluate the consequences of introducing a problem analysis tool to master students working with a PBL project.Moreover, the students were presented to a problem analysis tool to become better to understand what a problem analysis was, what it consisted of, how to manage scientific argumentation, etc.Data analysis showed an increase (with significant p values) in the following 5 topics: 1) the problem analysis term, 2) problem analysis structure, 3) scientific argumentation, 4) learn to analyze instead of explaining, and 5) using literature to argue for a scientific problem.

Problem analysis term and structure
Figure one showed a comparison of how students understood the term 'problem analysis' before and after being introduced to the problem analysis tool.Some students knew the term, some had heard about the term without knowing what it included, and some had never heard about the term.The answers were expected since the students represented many different education programs.The opposite picture was seen after introducing the problem analysis tool.With significant p value (p<0.001),students no longer were in doubt about the definition of the term.Literature indicates that the problem analysis section is crucial for further progression (Hasslacher et al., 2009;Yew & Goh, 2016), and therefore, it may be inexpedient if the students are in doubt of the basis of the project, e.g., the problem analysis.
Most students agreed on having great doubt about how well they knew the problem analysis structure (83% answered 3 or more) before the problem analysis tool introduction.A significant p value, p<0.001, showed how most students no longer were in doubt about the problem analysis structure after being introduced to the problem analysis tool.The lack of being acquainted with the structure may be correlated with that this was the students' first semester at AAU (Holgaard et al., 2021).It may also be correlated to their background and educational institution where the problem analysis tool has not been a part of the methodological toolbox when writing projects.It would be relevant to spend resources on students, e.g., implementing a crash course where students have the possibility to practice the problem analysis structure in small project reports before writing a large project (Holgaard et al., 2021;Kolmos et al., 2008).Since understanding the problem analysis is the basis for working problem based, it makes sense to help students become better to this part.

Scientific argumentation
Figure three demonstrates how well students knew how to conduct scientific argumentation.Most students initially answered, 'neither or no' closed followed by 'agree', which refers to that students were unsure, and only knew a little.After introducing the problem analysis tool, a significant p value (p<0.01)showed that most students no longer were in doubt about how to argue for a scientific problem.Being able to make use of the argument model by Toulmin (claim, data or grounds and warrant) requires practice and a high academic level (Kneupper, 1978;Nielsen, 2013;Zohar & Nemet, 2002).At present, students' focus is probably on acquiring academic knowledge within specific areas, and therefore, they are not yet able to utilize the model by Toulmin since the model requires training.However, the more competent and qualified the students become, the better qualifications students have to make use of Toulmin' argument model (Kneupper, 1978;Nielsen, 2013).

Implications for practice
Studying a master program at university level requires students to be able to analyze.Moreover, students must improve their competences to analyze according to Blooms Taxonomy.Most students had great doubt about how to analyze a scientific problem before they were introduced to the problem analysis tool, which is very expectable since their prerequisites to analyze were untrained.Significant p value of p<0.001 showed how most students no longer were in doubt about how to analyze after receiving the introduction to the problem analysis tool.The significant changes may indicate that it is matters to spend extra resources on giving students the problem analysis tool (Nielsen, 2013).
The exact time for when a problem analysis tool should be introduced was not clear in the results, e.g., some students thought would like the introduction earlier, i.e., in the beginning of a semester, others thought it was the right time.90% of the students thought it should be introduced later in the semester.From a pedagogical perspective, there are pros and cons related to handling out the tool at the beginning of a semester.On the one hand, students feel safe if they have models or tools they can rely on when navigating in a new academic field.On the other hand, it can be educative for students if they by themselves try and thereby gain experience, which they do not get if they get the model from the beginning of the semester.According to the psychologist Lev Vygotsky's theory zone of proximal development, learning is about supporting students to the extent that is necessary to motivate and encourage them to pass across a comfort zone (Eun, 2019;Wass & Golding, 2014).When it comes to the students included in this study, for a period test if the problem analysis tool could be introduced in the beginning of the semester for some years and then evaluate the effects.Vygotsky's zone of proximal development theory is also relevant when it comes to understanding why a crash course on practicing small projects is relevant.It helps students to find comfort in writing projects, taking risks, overcome small projects before they must write large projects.

STRENGHTS AND LIMITATIONS
A strength of the study is, to our knowledge, that there is no problem analysis tool developed.Another strength is that the problem analysis tool has been tested on students from different master programs.A limitation of the study is that the problem analysis tool is only tested on 28 students.Even though the results are significant, it would have increased the validity of the problem analysis tool if more students had been included.

CONCLUSION
Significant results showed how students believed that they had increased their understanding of the problem analysis term, problem analysis structure, scientific argumentation, and abilities to analyze instead of explaining.When to introduce the problem analysis tool depends on different pedagogical perspectives.Future work includes testing the problem analysis tool on a larger group of students.

Figure 1 .
Figure 1.The participants' perception of the problem analysis term before/after the introduction (question 1 and 6).

Figure 2 .
Figure 2. The participants' perception of the problem analysis structure before/after the introduction (question 2 and 7).

Figure 4 .
Figure 4.The participants' perception of their ability to analyze a problem before/after the introduction (question 4 and 9).

Figure 5 .
Figure 5.The participants' perception of their ability to use literature to argue for a scientific problem (question 5 and 10).

Name of the education Level Number of students in the groups
Table1gives an overview of the distribution and number of the included participants.

Table 2 .
. 5 topics were identified, all with significant p values (table2).An overview of calculated p values related to identified topics in the analysis.All p values were significant.