Study design and context
This study employed a cross-sectional explanatory design integrating self-assessed understanding, structured explanatory tasks, calibration analysis, and diagnostic accuracy testing. Multi-component metacognitive designs of this type are recommended when investigating discrepancies between perceived and actual understanding in health professions education, because they allow simultaneous examination of subjective judgments, explanatory performance, and applied clinical reasoning(Veenman, 2011).
The study was conducted in the Department of Pediatric Dentistry at a dental school in the Middle East during two consecutive academic years, providing a stable curricular context and comparable instructional exposure for all participating students.
Participants
The target population comprised final-year (5th year) Bachelor of Dental Surgery (BDS) students enrolled in the pediatric dentistry clinical course. Across the two academic years, 148 students were eligible to participate.
Inclusion criteria were:
active enrollment in the pediatric dentistry course,
completion of all core theoretical modules related to pediatric caries, pulp therapy in primary teeth, and behaviour guidance,
no ongoing remedial status or prolonged absence from clinical duties.
Participation was voluntary, and all students provided written informed consent.
Sample sizes exceeding approximately 100 participants are considered adequate for stable estimation of metacognitive indices and reliability of performance-based measures in educational research(Kline, 2015).
Selection of pediatric topics
Three core pediatric domains were selected as IOED targets:
Caries risk assessment in children
Pulp diagnosis and management in primary teeth
Behaviour guidance for anxious pediatric patients
These domains were chosen because they require integration of biological, developmental, and behavioural concepts, and previous work has shown that dental students frequently display errors or inconsistencies in clinical judgment specifically in pediatric scenarios(Moura et al., 2016). The topics align with the existing pediatric dentistry curriculum, ensuring content familiarity for all final-year students.
Measures
1. Perceived Explanatory Understanding Score (PEUS)
PEUS captured students’ self-rated explanatory understanding prior to any explanatory or diagnostic task. For each of the three topics, students were asked:
How well do you believe you can explain this concept to a peer?
Each response was recorded on a 10-point numerical rating scale (1 = “not at all well”, 10 = “extremely well”). Single-item metacognitive judgments of this type are widely used as global indicators of perceived understanding and have been shown to be informative for calibration research(Dunlosky & Metcalfe, 2009).
PEUS values were obtained for each topic separately (range 1–10). A global PEUS was computed as the arithmetic mean of the three topic-specific scores.
2. Observed Explanatory Performance Score (OEPS)
OEPS represented students’ objectively rated explanatory ability. Following the canonical Illusion of Explanatory Depth (IOED) paradigm[1,18], students were asked to produce short written explanations for each of the three pediatric topics.
Explanatory prompts
Students responded in writing (4–6 sentences) to the following prompts:
Explain the process of caries risk assessment in a 6-year-old child.
Explain the diagnostic approach and treatment decision-making for pulp involvement in a primary molar.
Explain the steps of behaviour guidance for an anxious child attending the dental clinic for the first time.
These prompts were designed to elicit causal, conceptual, and procedural knowledge, consistent with prior research demonstrating that explanation tasks are highly sensitive to gaps in understanding(Chi et al., 1994).
Scoring rubric and rating procedure
Two pediatric dentistry faculty members independently rated each explanation using a 5-point analytic rubric, adapted from explanatory assessment frameworks in cognitive and educational research[19,20]. The rubric evaluated:
Causal accuracy (correctness of key mechanisms and relationships),
Conceptual completeness (coverage of essential elements),
Logical structure (clarity and coherence of reasoning),
Clinical relevance (appropriateness for pediatric practice).
Each dimension was scored from 0 to 5, and an overall OEPS score (0–5) per topic was assigned based on holistic integration of these criteria, following established practices in performance-based explanation scoring(Lombrozo, 2006).
Inter-rater reliability for OEPS was evaluated using a two-way random intraclass correlation coefficient (ICC), which is recommended for reliability studies involving multiple raters and continuous scores(Shrout & Fleiss, 1979). Discrepant ratings were resolved by discussion, but only scores from the independent ratings were used for ICC computation.
A global OEPS was calculated as the mean of the three topic-specific scores.
3. Explanatory Calibration Index (ECI)
The Explanatory Calibration Index (ECI) quantified the gap between perceived and actual explanatory depth, providing a direct operationalization of IOED. Calibration metrics similar to ECI have a long tradition in judgment and decision-making research and are widely used to assess metacognitive accuracy(Lichtenstein et al., 1982).
Because PEUS was recorded on a 1–10 scale and OEPS on a 0–5 scale, OEPS values were first linearly transformed to a 0–10 metric by multiplication with two. For each topic, ECI was computed as:
ECI = PEUS – (OEPS × 2)
Interpretation:
ECI > + 2 → overestimation (strong IOED pattern),
ECI between − 1 and + 1 → good calibration,
ECI < − 2 → underestimation (impostor-like pattern).
ECI was calculated per topic and as a global index (mean of three ECI values). This allowed analysis of both topic-specific and overall metacognitive calibration.
4. Diagnostic Accuracy Score (DAS)
DAS measured applied pediatric diagnostic accuracy, independent of self-perception or explanatory performance. Diagnostic accuracy tests are recognized as valid indicators of clinical reasoning quality in health professions education(Eva, 2005).
Students completed six pediatric case-based micro-vignettes, constructed in alignment with key-feature format principles(Page & Bordage, 1995):
2 vignettes focused on caries risk assessment,
2 on pulp diagnosis and management in primary teeth,
2 on behaviour guidance in common pediatric scenarios.
Each vignette was followed by a single best-answer question targeting the critical diagnostic or management decision. Responses were scored 0 (incorrect) or 1 (correct), yielding a total DAS range from 0 to 6. Higher scores indicated better diagnostic accuracy.
Data collection procedure
Data collection followed a fixed sequence, closely aligned with established IOED protocols[1,18]:
PEUS ratings: students first rated their perceived explanatory understanding for each of the three topics.
Explanatory tasks: students then wrote 4–6 sentence explanations for each topic, without access to notes or electronic resources.
OEPS scoring: explanations were subsequently rated independently by two faculty members using the analytic rubric.
Diagnostic vignettes: students completed the six pediatric micro-vignettes (DAS).
All activities were conducted in a supervised classroom environment, during a single scheduled session, to standardize conditions and prevent consultation of external resources. The total time required to complete all components was approximately 35–40 minutes, which is comparable to durations reported in explanation-based metacognitive studies and educational assessment sessions(Lodge & Kennedy, 2013).
Ethical considerations
The study protocol was reviewed and approved by the institutional research ethics committee. Participation was voluntary, and students were informed that they could withdraw at any time without academic consequences. Data were anonymized prior to analysis; individual scores were not shared with course instructors in a way that could influence summative assessment. These procedures are consistent with best-practice recommendations for ethics in educational research involving students(Kass et al., 2007).