Satisfaction level and correlation between performance and self-evaluation of physical therapy students in an objective structured clinical examination (OSCE) designed to assess clinical reasoning

Nivel de satisfacción y correlación entre el desempeño y la autoevaluación de los estudiantes de fisioterapia en una evaluación clínica objetiva estructurada (ECOE) diseñada para evaluar el razonamiento clínico

Introduction: The Objective Structured Clinical Evaluation (OSCE) is an instrument that allows evaluating clinical reasoning among physical therapy students, considering that knowing the correlation between their performance and their self-evaluation, as well as their level of satisfaction with this instrument, is essential.

Objectives: To determine the correlation between performance and self-evaluation (perceived performance) of physical therapy students in an OSCE designed to assess clinical reasoning, and to evaluate their level of satisfaction with this instrument.

Materials and methods: Analytical cross-sectional study conducted in 159 physical therapy students from a university in Chile who participated in an 11-station OSCE. Performance checklists and answer sheets were used for performance evaluation (passing score: 70% of the maximum score per station and in the OSCE). Two perception surveys were also used, one for self-evaluation of performance (for each station) and another to determine the students’ level of satisfaction with the OSCE. The correlation between performance in the OSCE (overall score, score by station, and score by type of station) and perceived performance was determined using the Spearman’s correlation coefficient.

Results: The level of satisfaction with the OSCE was high in 57.23% of the students. The correlation between the total score in the OSCE and perceived performance was significant, but weak (Rho=0.31; p<0.001). In students with a low satisfaction level with the OSCE, the correlation between performance and perceived performance was not significant (Rho=0.15; p=0.23), while in those with high satisfaction level it was significant and moderate (Rho=0.48; p<0.001).

Conclusions: Most students reported a high level of satisfaction with the OSCE. However, in most stations, the correlation between observed and perceived performance was weak and very weak.

Introducción. La Evaluación Clínica Objetiva Estructurada (ECOE) es un instrumento que permite evaluar el razonamiento clínico en estudiantes de fisioterapia, por lo que saber la correlación entre su desempeño y su autoevaluación, así como su nivel de satisfacción con esta herramienta, es esencial.

Objetivos. Determinar la correlación entre el desempeño y la autoevaluación (desempeño percibido) de estudiantes de fisioterapia en una ECOE diseñada para evaluar el razonamiento clínico, y evaluar su nivel de satisfacción con esta herramienta.

Materiales y métodos. Estudio transversal analítico realizado en 159 estudiantes de fisioterapia de una universidad de Chile que participaron en una ECOE de 11 estaciones. Se utilizaron listas de verificación de desempeño y hojas de respuesta para la evaluación del desempeño (nota aprobatoria: 70% de la nota máxima por estación y de la ECOE) y 2 encuestas de percepción, una para la autoevaluación del desempeño (para cada estación) y otra para determinar el nivel de satisfacción de los estudiantes con la ECOE. La correlación entre el desempeño en la ECOE (puntuación global, por estaciones y por tipo de estación) y el desempeño percibido (autoevaluación) se determinó mediante el coeficiente de correlación de Spearman.

Resultados. El nivel de satisfacción con la ECOE fue alto en el 57.23% de los estudiantes. La correlación entre el puntaje total en la ECOE y el desempeño percibido fue significativa, pero débil (Rho=0.31; p<0.001). En estudiantes con nivel de satisfacción bajo, la correlación entre el desempeño y el desempeño percibido no fue significativa (Rho=0.15; p=0.23), mientras que en aquellos con un nivel de satisfacción alto fue significativa y moderada (Rho=0.48; p<0.001).

Conclusiones. La mayoría de los estudiantes manifestaron tener un alto nivel de satisfacción con la ECOE. Sin embargo, en la mayoría de las estaciones la correlación entre el desempeño observado y el percibido fue débil y muy débil.

Original research

Paola Figueroa-González1 Nicole Figueroa-Arce1 Luis Gómez-Miranda1 Ruvistay Gutiérrez-Arias1,2,3 Viviana Contreras-Pizarro1

1 Universidad Andrés Bello - Faculty of Rehabilitation Sciences - School of Physical Therapy - Exercise and Rehabilitation Science Institute - Santiago - Chile.

2 Instituto Nacional del Tórax - Comprehensive Cardiopulmonary Rehabilitation Support Department - Santiago - Chile.

3 Instituto Nacional del Tórax - INTRehab Research Group - Santiago - Chile.

Open access

Received: 22/02/2023

Accepted: 14/09/2023

Corresponding author: Nicole Figueroa-Arce. Escuela de Kinesiología, Facultad de Ciencias de la Rehabilitación, Universidad Andrés Bello. Santiago. Chile. Email: nicolefiguearce@gmail.com.

Keywords: Physical Therapy; Competency-based education; Self-evaluation programs; Educational assessment (MeSH).

Palabras clave: Fisioterapia; Educación basada en competencias; Programas de autoevaluación; Evaluación educacional (DeCS).

How to cite: Figueroa-González P, Figueroa-Arce N, Gómez-Miranda L, Gutiérrez-Arias R, Contreras-Pizarro V. Satisfaction level and correlation between performance and self-evaluation of physical therapy students in an objective structured clinical examination (OSCE) designed to assess clinical reasoning. Rev. Fac. Med. 2023;71(4):e107397. English. doi: https://doi.org/10.15446/revfacmed.v71n4.107397.

Cómo citar: Figueroa-González P, Figueroa-Arce N, Gómez-Miranda L, Gutiérrez-Arias R, Contreras-Pizarro V. [Nivel de satisfacción y correlación entre el desempeño y la autoevaluación de los estudiantes de fisioterapia en una evaluación clínica objetiva estructurada (ECOE) diseñada para evaluar el razonamiento clínico]. Rev. Fac. Med. 2023;71(4):e107397. English. doi: https://doi.org/10.15446/revfacmed.v71n4.107397.

Copyright: Copyright: ©2023 Universidad Nacional de Colombia. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, as long as the original author and source are credited.

Abstract

Resumen

Introduction

Clinical reasoning is a skill that involves logical thinking and decision-making to effectively diagnose and treat patients in healthcare practice. In physical therapy, it is defined as the application of cognitive and psychomotor skills, as well as reflection and knowledge processes.1,2 The goal of this collaborative, adaptive, and iterative process is to frame the intended outcome within a biopsychosocial framework that considers the perspectives of the patient and the therapist.3

In this sense, the diagnostic process must be based on adequate clinical reasoning, given that physicians must establish therapeutic objectives and the most appropriate interventions using the information reported by the patient, as well as that obtained from physical examination, medical records, and imaging and laboratory tests.2,4-6 Therefore, clinical reasoning is considered a core skill for health professionals and, consequently, a necessary component of their training, including physical therapists.5,7

The Objective Structured Clinical Evaluation (OSCE) is a test designed to assess standardized clinical skills and is used for training health professionals.8 The early incorporation of this type of evaluation in the training of undergraduate health science students allows them to experience a real professional practice environment, which, in turn, promotes the development of skills, reinforces theoretical knowledge, facilitates the clinical reasoning process, and strengthens the students’ sense of security and reflective practice.1,9-12 In view of the foregoing, it has been reported that the OSCE favors self-evaluation and feedback, actions that allow the identification of weaknesses and strengths observed during the evaluation process using this instrument.13,14

Self-evaluation is defined as the act of assessing oneself in order to make decisions about the next steps to take based on the conclusions obtained, which helps to improve knowledge acquisition and performance, and provides security and motivation.15 Thus, conducting a self-evaluation using the OSCE encourages students to identify the factors that may be helping or hindering their learning process.10

On the other hand, several studies have inquired about the opinion of students regarding the OSCE, which has not only facilitated the development of new knowledge to improve its quality, but also provided insights into the value that students give to the use of this instrument in their education.16,17 Concerning this, it has been found that students with a high OSCE score give greater meaning and relevance to the implementation of this instrument in their training process and value the contents, response times, influence and relevance of this instrument in their development as health professionals, strengthening the link between performance and self-evaluation.18-22

Providing methodologies to assess clinical competence in a reliable, consistent and valid manner is a challenge in health sciences education. On this point, Yusuf,23 in a mixed convergent parallel study involving 5 groups of final year students from a medical school in Lahore (Pakistan) assigned for rotation in gynecology wards (each group consisting of 18 students), showed that the OSCE responds to these needs.

Although some studies, such as that of Figueroa-Arce et al.,24 have reported experiences regarding the implementation of this evaluation instrument in physical therapy students, evidence is scarce. This highlights the need to generate knowledge about the self-evaluation and satisfaction of students with the performance of an OSCE that focuses on evaluating and promoting clinical reasoning in physical therapy, as stated by de la Barra-Ortiz et al.25

Therefore, the objectives of the present study were to determine the correlation between performance and self-evaluation (expected performance) of physical therapy students in an OSCE designed to assess clinical reasoning, and to evaluate their level of satisfaction with this instrument.

Materials and methods

Study type and sample

Analytical cross-sectional study with an exploratory scope. The study population comprised 163 fourth-semester students (of a total of 10 semesters) enrolled in the physical therapy program offered at a university in Santiago (Chile) and who were taking the course “Reasoning in physical therapy” during 2018-2 (age range between 18 and 20 years). All students completed the OSCE because it was a summative exam of the course; however, 4 students did not sign the informed consent, so the final sample included 159 students.

Procedures, study variables, and instruments

OSCE

A committee comprising 7 professors of the course Reasoning in Physical Therapy was created and was in charge of designing and implementing the OSCE, including the design of its 11 stations (each one with a complexity that addresses the achievement of the learning objectives of the course), the performance checklists (scenario and standardized patient stations), and the answer sheets (mailbox stations). They also validated its contents using a 4-point Likert scale that evaluated sufficiency, clarity, coherence, and relevance criteria. It should be noted that the members of the OSCE design committee had experience in the application of this type of evaluation instrument.

Subsequently, a pilot test was conducted on 30 volunteer students who had already taken the course and signed an informed consent form authorizing their participation in the test. In this pilot test, the students completed the 11 OSCE stations (plus 2 rest stations). Their performance at each station was evaluated depending on the type of station (checklists for those involving a clinical scenario or a standardized patient and answer sheets for the mailbox stations) by the same 7 professors of the committee mentioned above, who, through a checklist, issued their observations on the adjustments to be made, as follows: improving the wording of the checklist contents, increasing the time allotted to perform the activities at each station, and eliminating a rest station. These adjustments were fully implemented.

Once these modifications were made, an OSCE consisting of 11 evaluation stations was obtained (S1. Hand washing, S2. Safety in health care, S3. Clinical interview, S4. Cardiorespiratory assessment 1, S5. Musculoskeletal assessment, S6. Neurological assessment, S7. Cardiorespiratory assessment 2, S8. Clinical record, S9. Physical therapy diagnosis 1, S10. Physical therapy diagnosis 2, S11. Resolution of clinical case) and a rest period located between the fourth and sixth.

The stations were classified into three types: “scenario” station (S1 and S2), in which procedures associated with an intervention are recreated; “standardized patient” station (S3-S7), where an actor with a standardized script simulates being a patient in a clinical situation; and “mailbox” station (S8-S11), in which students must solve a clinical case and leave their answer in a mailbox. At the standardized patient and scenario stations, students were evaluated using checklists, while at the mailbox stations they were evaluated using answer sheets. The time allotted to complete each station was 5 minutes, for a total time of 60 minutes.

At each station, different minimum and maximum scores were assigned based on the defined objectives and the level of complexity of the station, with a total score of 198 points. A student was considered to pass the OSCE (passing score as per the passing criteria of the university’s school of physical therapy) if their score was >70% of the maximum grade, i.e., 134 points.

The main characteristics of the 11 OSCE stations are described in Table 1.

Table 1. Stations of the Objective Standardized Clinical Evaluation for clinical reasoning in physical therapy.

Station	Type of station	Objective	Description	Dimensions assessed	Score
S1. Hand washing	Scenario	To identify standard precautions to prevent healthcare associated infections	The student recognizes and identifies in the proposed scenario the distractors that may interfere with proper hand washing according to WHO standards *	Cognitive, psychomotor, and affective	0-21
S2. Safety in health care	Scenario	To choose personal protective equipment considering the etiologic agent	Based on the clinical case, the student chooses and correctly uses personal protective equipment depending on the type of isolation and the agent of contagion as per WHO standards *	Cognitive, psychomotor, and affective	0-17
S3. Clinical interview	Standardized patient	To retrieve and identify relevant aspects of the clinical interview	The student is able to conduct an interview and extract information relevant to the patient’s health condition	Cognitive, psychomotor, and affective	0-21
S4. Cardiorespiratory assessment 1	Standardized patient	To identify cardiorespiratory monitoring based on the exposed health condition	The student must examine and assess the user’s vital signs based on their health condition	Cognitive, psychomotor, and affective	0-24
S5. Musculoskeletal assessment	Standardized patient	To identify musculoskeletal signs and symptoms consistent with the health condition	The student must examine and evaluate the joint exposed in the clinical case	Cognitive, psychomotor, and affective	0-23
S6. Neurological assessment	Standardized patient	To identify neurological signs and symptoms consistent with the health condition	The student must examine and evaluate the central nervous system based on the proposed clinical situation	Cognitive, psychomotor, and affective	0-8
S7. Cardiorespiratory assessment 2	Standardized patient	To identify cardiorespiratory signs and symptoms consistent with the health condition	The student must examine and auscultate the specific segment requested based on the health condition	Cognitive, psychomotor, and affective	0-17
S8. Clinical record	Mailbox	To sort the information provided in the clinical record according to the case presented	The student must identify, organize, construct and record information based on the clinical case presented	Cognitive, psychomotor, and affective	0-16
S9. Physical therapy diagnosis 1	Mailbox	To make a physical therapy diagnosis	The student must analyze the information provided to make a physical therapy diagnosis using the ICF †	Cognitive, psychomotor, and affective	0-15
S10. Physical therapy diagnosis 2	Mailbox	To make a physical therapy diagnosis	The student must analyze the information provided to make a physical therapy diagnosis using the ICF †	Cognitive, psychomotor, and affective	0-15
S11. Clinical case resolution	Mailbox	To identify clinical problems using the RPS-form ‡ and the ICF † based on the information provided	The student must analyze the information provided to make a physical therapy diagnosis using the ICF †	Cognitive and psychomotor	0-21

* World Health Organization.26

† International Classification of Functioning, Disability and Health.6,27

‡ Rehabilitation Problem-Solving form.6,27

Source: Own elaboration.

The OSCE was administered on the same day to the entire sample. For this purpose, the students entered in groups of 12 people and each group was allocated to a station (including the rest station). The groups rotated every 5 minutes, ensuring that all participants had the opportunity to be evaluated. It is important to point out that the OSCE was administered to 163 students at the end of the course.

Self-evaluation and satisfaction surveys

Upon completion of the OSCE, students were asked to answer two surveys in order to obtain data regarding the perception of their performance on the test (self-evaluation of performance) and their level of satisfaction with the instrument. It is worth mentioning that while all students responded to the first survey, this was not the case for the second survey.

For the self-evaluation performance survey, a 1 to 5 Likert scale (1: very poor, 2: poor, 3: fair, 4: good, and 5: very good) was used to assess the perceived performance of each student after answering the OSCE at each of the 11 stations (maximum score of 55 points).

Furthermore, the satisfaction survey consisted of 5 questions on, among other things, the general structure and usefulness of the OSCE, as well as the relevance of the stations. As in the self-evaluation survey, a 1 to 5 Likert scale (1: strongly disagree, 2: disagree, 3: neither agree nor disagree, 4: agree and 5: strongly agree) was used for each question to establish the level of student satisfaction with the instrument (maximum score of 25 points). Satisfaction was classified into two levels based on the median score obtained in the survey: low: <20 points and high: ≥20 points.28

Statistical analysis

The scores obtained in the OSCE and the two surveys were systematized in a database created in Microsoft Excel. A descriptive analysis of the data was performed by calculating absolute and relative frequencies for qualitative variables, and medians and interquartile ranges (IQR) with 25th and 75th percentiles (p25-p75) for quantitative variables, since the distribution of the data was not normal (Shapiro-Wilk test).

The correlation between OSCE performance (overall score, by station, and by type of station) and perceived performance (self-evaluation) was determined using Spearman’s correlation coefficient (Rho) because self-evaluation is an ordinal qualitative variable. The correlation between performance in the OSCE (total score) and self-evaluation of performance was determined based on the level of satisfaction in accordance with the two established categories (low and high). All analyses were performed in the STATA 13.0 statistical package and a significance level of p<0.05 was considered. The graph of the correlation between the global score and the performance self-evaluation was made using the JASP 0.14.1.0 software.

Ethical considerations

The study was approved by the Bioethics Committee of the Faculty of Rehabilitation Sciences of the Universidad Andrés Bello through Certified Project A.138 of July 2020. It also followed the ethical principles for biomedical research involving human subjects established in the Declaration of Helsinki.29 All participants signed an informed consent form.

Results

The median total score obtained in the OSCE was 142 points (IQR: 132-150), and 97 (61.01%) students passed the exam (>134 points). The pass rate was ≥50% in 8 stations (S1: 51.57%; S2: 54.09%; S3: 78.62%; S4: 58.49%; S5: 96.86%; S6: 50.94; S7: 85.53; and S11:57.23%), and the pass rate was very low (2.52%) only in S8 (Table 2).

Table 2. Scores obtained by students in each of the 11 stations of the objective structured clinical evaluation (n=159).

Station	Maximum score (points)	Students obtaining the maximum score (%)	Passing score (points)	Students who obtained a passing score (%)	Median	IQR (p25-p75)
S1. Hand washing	21	3.14	15	51.57	15	14-17
S2. Safety in health care	17	23.27	12	54.09	15	0-16
S3. Clinical Interview	21	26.42	15	78.62	18	15-21
S4. Cardiorespiratory assessment 1	24	4.40	17	58.49	18	15-21
S5. Musculoskeletal assessment	23	29.56	16	96.86	21	19-23
S6. Neurological assessment	8	50.94	6	50.94	8	6-8
S7. Cardiorespiratory assessment 2	17	20.75	12	85.53	14	13-16
S8. Clinical record	16	0.00	12	2.52	6	4-7
S9. Physical therapy diagnosis 1	15	7.55	11	44.03	9	7-11
S10. Physical therapy diagnosis 2	15	8.23	11	42.14	9	9-11
S11. Clinical case resolution	21	0.63	15	57.23	15	13-17

IQR: interquartile range.

Source: Own elaboration.

With respect to perceived performance, the median total score of the self-evaluation was 38 points (IQR: 33-42), with S1 being the station with the highest percentage of students (59.75%) who rated their performance as very good. In contrast, S8 was the station in which the highest proportion of students perceived a very poor performance (8.97%) (Table 3).

Table 3. Performance self-evaluation in the objective clinical assessment by station.

Station	Responses received (n)	Likert Scale (%)
		Very poor	Poor	Fair	Good	Very good
S1. Hand washing	156	0	1.92	4.49	33.97	59.62
S2. Safety in health care	156	4.55	13.64	21.43	29.22	31.17
S3. Clinical Interview	154	5.13	16.67	33.97	25.64	18.59
S4. Cardiorespiratory assessment 1	156	6.45	12.26	25.16	36.13	20
S5. Musculoskeletal assessment	156	5.77	17.95	29.49	28.21	18.59
S6. Neurological assessment	156	5.19	5.84	26.62	31.82	30.52
S7. Cardiorespiratory assessment 2	155	4.55	11.69	28.57	36.36	18.83
S8. Clinical record	155	8.97	26.28	43.59	18.59	2.56
S9. Physical therapy diagnosis 1	154	7.10	20.65	30.97	34.19	7.10
S10. Physical therapy diagnosis 2	154	8.33	23.72	32.05	28.21	7.69
S11. Clinical case resolution	156	8.33	30.77	39.1	19.23	2.56

Source: Own elaboration.

A positive and weak correlation was observed between the total score obtained in the OSCE and the performance self-evaluation (Rho=0.31; p<0.001) (Figure 1).

Figure 1. Correlation between the total score obtained and the performance self-evaluation in the objective structured clinical evaluation.

Source: Own elaboration.

Regarding the correlations between performance self-evaluation and the scores obtained for each station, the correlation was positive in all stations, being moderate in one station (S2), weak in seven stations (S3, S4, S5, S6, S7, S1, and S11), and very weak in three stations (S1, S8, and S9) (Table 4). Furthermore, these correlations were statistically significant at all stations, except for S1 (Rho=0.02; p=0.818) and S8 (Rho=0.08; p=0.296) (Table 4).

As for the correlations between the self-evaluation and the scores obtained by type of station, the correlation was positive and moderate in the scenario stations (Rho=0.42; p<0.001) and positive and weak in the standardized patient and mailbox stations (both with Rho=0.28; p<0.001).

Table 4. Correlation between the score obtained in the stations of the objective structured clinical assessment and the performance self-evaluation in each station.

Station	Rho Spearman	p-value
S1. Hand washing	0.02	0.818
S2. Safety in health care	0.51	<0.001
S3. Clinical Interview	0.23	0.004
S4. Cardiorespiratory assessment 1	0.38	<0.001
S5. Musculoskeletal assessment	0.30	<0.001
S6. Neurological assessment	0.25	0.002
S7. Cardiorespiratory assessment 2	0.24	0.003
S8. Clinical record	0.08	0.296
S9. Physical therapy diagnosis 1	0.17	0.032
S10. Physical therapy diagnosis 2	0.29	<0.001
S11. Clinical case resolution	0.29	<0.001

Source: Own elaboration.

On the other hand, the median score on the ECOE satisfaction scale was 20. In addition, 91 (57.23%) students expressed having a high level of satisfaction with the OSCE (Table 5). In the group of students with a low level of satisfaction, the correlation between the total score obtained in the OSCE and the self-evaluation score was positive and very weak (Rho=0.15; p=0.234), while in the group with high satisfaction, the correlation was positive and moderate (Rho=0.48; p<0.001).

Table 5. Survey of satisfaction with the Objective Structured Clinical Evaluation.

Question	Strongly disagree (%)	Disagree (%)	Neither agree nor disagree (%)	Agree (%)	Strongly agree (%)
Q1. The overall setup of the OSCE is adequate	12.58	22.64	13.21	44.03	7.55
Q2. The objective of the station is suitable for my level of knowledge	1.89	15.09	8.81	52.83	21.38
Q3. The OSCE has been useful for my training as a physical therapist	2.52	4.4	6.29	48.43	38.36
Q4. Implementing similar tests improves my training process	1.89	5.03	8.81	45.91	38.36
Q5. Implementing tests that assess my clinical skills is important	2.56	0	1.89	36.48	59.12

OSCE: Objective Structured Clinical Evaluation.

Source: Own elaboration.

Discussion

The present study aimed to determine the correlation between performance and self-evaluation (perceived performance) of physical therapy students from a Chilean university in an OSCE designed to assess clinical reasoning, and to evaluate their level of satisfaction with this instrument.

In this regard, it was found that 57.23% of the participants reported a high level of satisfaction with the OSCE and that a high percentage indicated that they agreed or strongly agreed with the questions in the satisfaction survey, with the exception of question 1, where this proportion slightly exceeded 50%. The students stated that this instrument is useful for their training process and that it is important to evaluate their clinical skills with this type of tool.

These findings are similar to those of other studies in which it has been reported that between 63% and 94.5% of the students have a positive attitude and/or a high level of satisfaction with the OSCE.25,30-31 For example, in a study conducted between February 2012 and February 2013 in 76 undergraduate medical students at a university in Iran, Khosravi-Khorashad et al.30 reported that 94.5% of the participants had a positive attitude toward a 13-station OSCE and mentioned that this assessment format was more appropriate than other examination methods. In turn, in a 2017 study of 54 medical students at the University of the West Indies (Barbados, Jamaica), Majumder et al.31 found that the majority of participants (63-91%) perceived positively the attributes of a 24-station OSCE.

These findings are also confirmed by Doloresco et al.,32 who conducted a study in 2018 on 124 third-year pharmacy students at the University of Buffalo (Buffalo, USA) and found that, based on the mean scores of their responses, students agreed with most of the instrument’s quality assessment items, namely: “The OSCE measured application of skills and abilities required in pharmacy practice” (M=5.8, SD=1.1); “The level of difficulty of the OSCE cases today was appropriate” (M=5.3, SD=1.5); “The topics covered in the session today were relevant to me” (M=5.6, SD=1.3); “The OSCE cases today were a fair measure of my communication skills” (M=5.7, SD=1.3); and “I prefer the OSCE over examinations” (M=4.9, SD=2.0).

On the other hand, in the present study, a positive and weak correlation (Rho=0.31) was found between the total score obtained in the OSCE and self-evaluation. Moreover, when performing this analysis for each station, positive correlations were observed in all stations, being moderate only in one station (S2; Rho=0.51), weak in seven (Rho=0.17-0.38), and very weak in three (S1, S8, and S9; Rho=0.02-0.08). These findings are comparable to those reported by de la Barra-Ortiz et al.,25 who, in a study conducted on 111 Chilean physical therapy students enrolled in the Physical Agents course, found positive correlations between objective performance (score) and perceived performance (self-evaluation) in the 7 stations implemented in the OSCE, being this correlation very weak in 3 stations (Rho=0.11-0.19), weak in 2 (Rho=0.36), and moderate in 2 (Rho=0.47-0.56).

However, such a finding differs from the findings reported by Garza et al.,33 who, in a study of 733 pharmacy students at a U.S. university who completed a 5-station OSCE designed to assess new prescription counseling skills, found a significant but moderate correlation between self- and peer-evaluation and scores obtained in the information gathering, management strategies, and monitoring and follow-up domains (r=0.43-0.51), but not in the communication domain (r=0.12).

It should be noted that in the present study the correlations between the scores obtained in the OSCE and the performance self-evaluation in S1 and S8 were very weak and had no statistical significance. This may be due to the fact that the contents and procedures of these stations were covered in early stages of the course and, consequently, students were familiar with them and, therefore, responded in an automated way, relying on repetition and being less reflective, which could influence a perception of good student performance.34,35

On the contrary, standardized patient stations require ongoing reflection and greater attention from the students, making them more aware when making decisions in clinical situations.34,36 This is supported by the conclusions obtained in another study in which an OSCE was performed on 32 third-semester physical therapy students at the Federal University of Minas Gerais (Belo Horizonte, Brazil) who had previous experience with this type of evaluation.37 In that study, the objective was to measure the correlation between performance levels and self-perception. For this purpose, 3 stations were created with an integrative and simulation approach (which could be comparable to the standardized patient stations of the present study), and a moderate correlation was reported (r=0.475; p=0.007) between OSCE and self-perception scores. This result was attributed to the fact that the OSCE is an instrument that allows to address realistic situations and to receive feedback on physiological and psychological status, which contributes to the development of a positive self-perception.37 Therefore, incorporating standardized patient stations that allow for an integration of the physiological and psychological components would facilitate decision making, thus improving students’ confidence and self-perception.

Finally, it was observed that the correlation between performance and self-evaluation was higher in the group of students with a high level of satisfaction with the OSCE, compared to those with a low level of satisfaction. This finding should be confirmed in future studies, which should also explore the possible reasons that explain a greater congruence between the observed and perceived performance of students taking a test such as the OSCE using qualitative methodologies.

One of the limitations of the present study was that variables such as the communication skills and emotional state (e.g., anxious symptomatology and stress) of the students during the OSCE, which could influence performance and self-perception of performance, were not considered. In this regard, several studies have described a high proportion of students reporting negative emotional distress, although the effect on OSCE scores is inconclusive given the presence of protective factors such as self-confidence and self-efficacy.37,38 Consequently, further studies could explore potential factors associated with the consistency between obtained performance and expected performance in the OSCE by physical therapy students, and their impact on the evaluative processes considering communicational and emotional variables.

Conclusions

Most students reported a high level of satisfaction with the OSCE. However, in most stations, the correlation between observed and perceived performance was weak and very weak.

Conflicts of interest

None stated by the authors.

Funding

None stated by the authors.

Acknowledgments

None stated by the authors.

References

1.Willis BW, Campbell AS, Sayers SP, Gibson K. Integrated clinical experience with concurrent problem-based learning is associated with increased clinical reasoning of physical therapy students in the United States. J Educ Eval Health Prof. 2018;15:30. https://doi.org/jdxs.

2.Atkinson HL, Nixon-Cave K. A tool for clinical reasoning and reflection using the international classification of functioning, disability and health (ICF) framework and patient management model. Phys Ther. 2011;91(3):416-30. https://doi.org/dtxr3p.

3.Huhn K, Gilliland SJ, Black LL, Wainwright SF, Christensen N. Clinical Reasoning in Physical Therapy: A Concept Analysis. Phys Ther. 2019;99(4):440-56. https://doi.org/gm2c9k.

4.McClure P, Tevald M, Zarzycki R, Kantak S, Malloy P, Day K, et al. The 4-Element Movement System Model to Guide Physical Therapist Education, Practice, and Movement-Related Research. Phys Ther. 2021;101(3):pzab024. https://doi.org/k8jp.

5.Baker SE, Painter EE, Morgan BC, Kaus AL, Petersen EJ, Allen CS, et al. Systematic Clinical Reasoning in Physical Therapy (SCRIPT): Tool for the Purposeful Practice of Clinical Reasoning in Orthopedic Manual Physical Therapy. Phys Ther. 2017;97(1):61-70. https://doi.org/gnf44v.

6.Jiménez-Tordoya EJ. Guía metodológica para elaborar el diagnóstico fisioterapéutico según la Clasificación Internacional del Funcionamiento (CIF), de la discapacidad y de la salud. Gac Med Bol. 2016;39(1):46-52.

7.Young M, Thomas A, Gordon D, Gruppen L, Lubarsky S, Rencic J, et al. The terminology of clinical reasoning in health professions education: Implications and considerations. Med Teach. 2019;41(11):1277-84. https://doi.org/gh2jnw.

8.Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. Br Med J. 1975;1(5955):447-51. https://doi.org/fvvpkw.

9.McCallum C, Bayliss J, Becker E, Nixon-Cave K, Colgrove Y, Kucharski-Howard J, et al. The integrated clinical education strategic initiatives project-development of parameters to guide harmonization in clinical education: A scoping review. Phys Ther. 2019;99(2):147-72. https://doi.org/k8jq.

10.Madrazo L, Lee CB, McConnell M, Khamisa K. Self-assessment differences between genders in a low-stakes objective structured clinical examination (OSCE). BMC Res Notes. 2018;11(1):393. https://doi.org/ggqjbz.

11.Sánchez-Ferrer F, González-de Dios J, Juste-Ruiz M, Compañ AF, Ramos JM. Encuesta de percepción de la evaluación clínica objetiva y estructurada (ECOE) por los estudiantes de sexto grado en medicina de la Universidad Miguel Hernández. FEM (Ed. impresa). 2017;20(3):137-40. https://doi.org/jdx5.

12.Tawanwongsri W, Phenwan T. Reflective and feedback performances on Thai medical students’ patient history-taking skills. BMC Med Educ. 2019;19(1):141. https://doi.org/jdzc.

13.Larsen T, Jeppe-Jensen D. The introduction and perception of an OSCE with an element of self- and peer-assessment. Eur J Dent Educ. 2008;12(1):2-7. https://doi.org/bqbhpq.

14.Taylor CA, Green KE. OSCE Feedback: A Randomized Trial of Effectiveness,Cost-Effectiveness and Student Satisfaction. Creative Education. 2013;4(6A):9-14. https://doi.org/jdx6.

15.Inayah AT, Anwer LA, Shareef MA, Nurhussen A, Alkabbani HM, Alzahrani AA, et al. Objectivity in subjectivity: do students’ self and peer assessments correlate with examiners’ subjective and objective assessment in clinical skills? A prospective study. BMJ Open. 2017;7(5):e012289. https://doi.org/k8js.

16.Behrens C, Morales V, Parra P, Hurtado A, Fernández R, Giaconi E, et al. Diseño e implementación de OSCE para evaluar competencias de egreso en estudiantes de medicina en un consorcio de universidades chilenas. Rev Med Chile. 2018;146(10):1197-204. https://doi.org/jdxw.

17.Cyr PR, Schirmer JM, Hayes V, Martineau C, Keane M. Integrating Interprofessional Case Scenarios, Allied Embedded Actors, and Teaching into Formative Observed Structured Clinical Exams. Fam Med. 2020;52(3):209-12. https://doi.org/jdxv.

18.Alemán-Belando S, Marín-Silvente C, Soriano-Giménez V, González-Gil A, Flores-Funes D, Ramírez-Romero P, et al. Cómo perciben los estudiantes del primer curso clínico del Grado en Medicina la prueba de Evaluación Clínica Objetiva Estructurada (ECOE). Educ Médica. 2019;20(Suppl 2):144-7. https://doi.org/gsrn.

19.McKay A, McCall J, Cairns AM. Peer assessment: Development and delivery of the OSCE. Eur J Dent Educ. 2023;27(2):234-9. https://doi.org/k8jt.

20.Villegas N, Cianelli R, Fernandez M, Henderson S, Sierra S, Alfonso Y, et al. Assessment of breastfeeding clinical skills among nursing students using the Objective Structured Clinical Examination (OSCE). Inv Educ Med. 2016;5(20):244-52. https://doi.org/gsrj.

21.Khan A, Ayub M, Shah Z. An Audit of the Medical Students’ Perceptions regarding Objective Structured Clinical Examination. Educ Res Int. 2016;2016:4806398. https://doi.org/jjv3.

22.Egloff-Juras C, Hirtz P, Luc A, Vaillant-Corroy AS. An Objective Structured Clinical Examination (OSCE) for French Dental Students: Feedback after 2 Years. Dent J (Basel). 2021;9(11):136. https://doi.org/k8jv.

23.Yusuf L. Objective Structured Clinical Examination? How students perceive their learning after OSCE. Pak J Med Sci. 2021;37(4):1206-10. https://doi.org/k8jd.

24.Figueroa-Arce N, Figueroa-González P, Gómez-Miranda L, Gutiérrez-Arias R, Contreras-Pizarro V. Implementation of Objective structured clinical examination (OSCE) as a tool to evaluate the development of clinical reasoning in physical therapy students. Rev. Fac. Med. 2021;70(2):e90746. https://doi.org/jjwb.

25.de la barra-Ortiz HA, Gómez-Miranda LA, de la Fuente-Astroza JI. Level of satisfaction and correlation between the performance and self-evaluation of physical therapy students in the objective structured clinical examination (OSCE) when using physical agents. Rev. Fac. Med. 2021;70(3):e92473. https://doi.org/k8jf.

26.Organización mundial de la Salud (OMS). Manual técnico de referencia para la higiene de las manos. OMS; 2010 [cited 2023 Dec 13]. Available from: https://bit.ly/3tikZpw.

27.Steiner WA, Ryser L, Huber E, Uebelhart D, Aeschlimann A, Stucki G. Use of the ICF model as a clinical problem-solving tool in physical therapy and rehabilitation medicine. Phys Ther. 2002;82(11):1098-107.

28.Matas A. Diseño del formato de escalas tipo Likert: un estado de la cuestión. Revista Electrónica de Investigación Educativa. 2018;20(1):38-47. https://doi.org/c86w.

29.World Medical Association. (WMA). WMA Declaration of Helsinki – Ethical principles for medical research involving human subjects. Fortaleza: 64th WMA General Assembly; 2013.

30.Khosravi-Khorashad A, Salari S, Baharvahdat H, Hejazi S, Lari SM, Salari M, et al. The assessment of undergraduate medical students’ satisfaction levels with the objective structured clinical examination. Iran Red Crescent Med J. 2014;16(8):e13088. https://doi.org/k8nn.

31.Majumder MAA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019;10:387-97. https://doi.org/gnnxss.

32.Doloresco F, Maerten-Rivera J, Zhao Y, Foltz-Ramos K, Fusco NM. Pharmacy Students’ Standardized Self-Assessment of Interprofessional Skills During an Objective Structured Clinical Examination. Am J Pharm Educ. 2019;83(10):7439. https://doi.org/k8np.

33.Garza KB, Hohmann NS, Kavookjian J, Kleppinger EL. Assessment of student performance on a mock new prescription counseling session and an objective structured clinical examination across five years. Curr Pharm Teach Learn. 2020;12(9):1046-55. https://doi.org/k8nq.

34.Medina MS, Castleberry AN, Persky AM. Strategies for Improving Learner Metacognition in Health Professional Education. Am J Pharm Educ. 2017;81(4):78. https://doi.org/gm2c2d.

35.Hall ACG, Evans DG, Higginbotham L, Thompson KS. The effects of mood and retrieval cues on semantic memory and metacognition. Scand J Psychol. 2020;61(3):333-47. https://doi.org/ghg3sk.

36.O’Carroll PJ, Fisher P. Metacognitions, worry and attentional control in predicting OSCE performance test anxiety. Med Educ. 2013;47(6):562-8. https://doi.org/k8nr.

37.Ferreira ÉMR, Pinto RZ, Arantes PMM, Vieira ÉLM, Teixeira AL, Ferreira FR, et al. Stress, anxiety, self-efficacy, and the meanings that physical therapy students attribute to their experience with an objective structured clinical examination. BMC Med Educ. 2020;20(1):296. https://doi.org/gmd37r.

38.Sánchez-Conde P, Clemente-Suárez VJ. Autonomic Stress Response of Nurse Students in an Objective Structured Clinical Examination (OSCE). Sustainability. 2021;13(11):5803. https://doi.org/k8ns.