Research & Evaluation

A scholarly, mixed-methods design

CaseStep is built as an educational research project — from needs assessment and expert consensus to a piloted, evaluated intervention informed by implementation science.

Study overview

Background

Community Medicine teaching often remains fact-heavy and didactic, leaving a gap between knowledge and applied, context-aware clinical reasoning — the skill undergraduate assessment and practice ultimately demand.

Educational problem

There is no structured, scalable way for MBBS students in this setting to repeatedly rehearse applied clinical reasoning that integrates individual patient care with a community/public-health lens, with standardised, measurable feedback.

Innovation

CaseStep: branching digital cases, Script Concordance Testing, and OSCE/OSPE stations combined in one platform, explicitly designed against Kern’s 6-step model, ADDIE, and TPACK, and validated through expert (Delphi) review.

Aim

To develop, validate, and pilot-evaluate a digital case-based learning platform that measurably strengthens applied clinical reasoning among MBBS students in Community Medicine.

Objectives

Conduct a targeted needs assessment of students and faculty against NMC CBME reasoning competencies.
Develop and Delphi-validate a bank of digital cases, SCT items, and OSCE/OSPE stations.
Pilot the platform within the Community Medicine posting.
Evaluate its effect on clinical-reasoning outcomes using a mixed-methods design.
Disseminate findings to inform scale-up within the institution and beyond.

Study design

Design

Mixed-methods, quasi-experimental (single-arm pre/post) with a preceding development & validation phase.

Participants

MBBS students in the Community Medicine clinical posting (illustrative cohort size: ~50/batch); Community Medicine faculty as facilitators; external subject/assessment experts for Delphi validation.

Intervention

Structured use of the CaseStep platform (digital cases, SCT, OSCE/OSPE practice) integrated into the existing posting schedule, alongside usual teaching.

Comparator / control

Within-subject pre/post comparison (each student is their own control) in the pilot phase; a parallel-group comparison against a standard-teaching cohort is a possible extension once resources allow — not assumed at this stage.

Outcome measures

Case branching-decision accuracy (per case and pooled).
Script Concordance Test concordance score against the expert panel.
OSCE/OSPE checklist and global-rating scores.
Pre/post confidence and self-reported reasoning-skill ratings.
Engagement analytics: completion rate, time-on-task, reflection submission.
Qualitative themes from reflections, focus groups, and faculty feedback.

Research phases

01
Needs assessment
Cross-sectional survey · competency mapping
Survey MBBS students and Community Medicine faculty; map gaps in applied clinical reasoning against NMC CBME competencies.
02
Delphi consensus
Modified Delphi · median/IQR & % agreement
Panel of experts rates candidate cases across seven dimensions (relevance, validity, feasibility, CBME alignment, reasoning authenticity, feedback quality, community integration) across iterative rounds to a defined consensus threshold.
03
Module development
Instructional design · blueprinting
Author digital branching cases, SCT items, and OSCE/OSPE stations using ADDIE and Kern’s model; align with TPACK.
04
Pilot implementation
Implementation science · fidelity monitoring
Deploy the platform in the Community Medicine posting; run guided and self-directed sessions with the student cohort.
05
Evaluation
Mixed-methods · pre/post · thematic analysis
Assess reasoning outcomes, engagement, and satisfaction using validated tools and qualitative feedback; iterate.

Kirkpatrick evaluation mapping

Level 1 — Reaction

Level 2 — Learning

Level 3 — Behaviour

Level 4 — Results

Level	What is measured	Tool
Level 1 — Reaction	Student satisfaction and engagement with the platform.	Post-module feedback survey; engagement analytics (completion, time-on-task).
Level 2 — Learning	Change in clinical-reasoning knowledge/skill.	Pre/post SCT concordance; OSCE/OSPE scores; case-decision accuracy.
Level 3 — Behaviour	Transfer to applied reasoning in subsequent clinical/skills settings.	Facilitator-observed reasoning during ward-based/skills-lab encounters (planned); reflective-practice quality.
Level 4 — Results	Broader curricular/programmatic impact.	Uptake across batches; faculty adoption; contribution to programme-level CBME assessment evidence (longer-term, aspirational).

Logic model

A planned, illustrative program logic — Inputs flow through Activities and Outputs to the Outcomes and Impact the project aims to demonstrate.

Inputs

FAIMER fellowship time & mentorship
Community Medicine faculty expertise
External expert (Delphi) panel
Digital platform (CaseStep)

Activities

Needs assessment
Case/SCT/OSCE authoring
Delphi validation
Pilot deployment in the posting
Mixed-methods evaluation

Outputs

Validated digital case bank (11 topics)
SCT item bank
OSCE/OSPE station bank
Evaluation dataset & reports

Outcomes

Improved applied clinical-reasoning scores
Higher engagement & confidence
Faculty capacity to author digital cases

Impact

Stronger CBME-aligned reasoning competence at graduation
A reusable, scalable model for other departments

Evaluation matrix

Objective	Data source	Method	Timing
Validate case content and design	Expert (Delphi) review — relevance, validity, feasibility, CBME alignment, feedback quality, community integration	Modified Delphi, 2 rounds; median/IQR & % agreement	Before pilot deployment
Assess change in clinical reasoning	SCT scores, case-decision accuracy, OSCE/OSPE scores	Pre/post comparison; descriptive & inferential statistics as appropriate	Start and end of the posting
Understand learner experience	Reflections, focus-group discussions	Thematic analysis	End of posting
Understand facilitator experience	Faculty feedback interviews/survey	Thematic analysis / descriptive summary	End of pilot
Monitor engagement & usability	Platform analytics (completion, time-on-task)	Descriptive analytics	Throughout the pilot

Ethical considerations & data privacy

Commitments

Ethics committee approval will be obtained before any real student or expert data are collected; the current build uses no real participant data.
Participation will be voluntary with informed consent; non-participation will not disadvantage any student academically.
No patient-identifiable data are used anywhere in the platform — all clinical cases are synthetic/composite teaching scenarios.
Student performance data collected during a real pilot will be de-identified for analysis and reported only in aggregate.
Any future database (e.g. Supabase) will apply authentication and row-level security so learners can access only their own records; policy details are documented in supabase/schema.sql.

No ethics-approval number is quoted anywhere on this platform. One will be obtained, and cited here, before any real participant data are collected.

Limitations & mitigation strategies

Single-institution, single-arm pilot limits generalisability.

Mitigation: Report findings as a pilot/feasibility study; plan a multi-site or comparator-arm extension for scale-up.

Illustrative SCT expert-panel keys are placeholders until a real panel completes scoring.

Mitigation: Run a dedicated expert-panel SCT scoring exercise as part of Delphi validation before using scores for high-stakes decisions.

Self-reported confidence may not equal demonstrated competence.

Mitigation: Triangulate with objective measures (OSCE scores, case-decision accuracy) rather than relying on self-report alone.

Novelty effect may inflate early engagement metrics.

Mitigation: Track engagement across multiple cohorts/terms rather than a single pilot window.

Scale-up plan

1Expand from the pilot batch to all Community Medicine postings at the institution.
2Extend the validated case-authoring workflow to other departments (e.g. Medicine, Paediatrics) using the same schema.
3Formalise the expert panel and complete Round 2 Delphi validation for all 11 case topics.
4Integrate a database backend (Supabase, already scaffolded) for real multi-cohort analytics.
5Pursue inter-institutional collaboration once single-site feasibility is demonstrated.

Expected scholarly outputs

Dissemination plan

Needs-assessment paper (planned): Gaps in applied clinical reasoning among MBBS students in Community Medicine — a mixed-methods needs assessment.

Delphi validation paper (planned): Development and expert consensus validation of a digital case bank for Community Medicine.

Module development report (planned): An ADDIE/Kern-informed process for authoring digital case-based learning modules.

Pilot evaluation paper (planned): Effect of a digital case-based learning platform on clinical reasoning — a mixed-methods pilot.

Educational innovation report / FAIMER poster (planned): CaseStep as a scalable model for digital case-based learning in Community Medicine.

Publication details, DOIs and links will be added here as outputs are produced.

Evaluation tools at a glance

Quantitative outcomes

Case branching-decision accuracy (per case and pooled).
Script Concordance Test concordance score against the expert panel.
OSCE/OSPE checklist and global-rating scores.
Pre/post confidence and self-reported reasoning-skill ratings.
Engagement analytics: completion rate, time-on-task, reflection submission.

Qualitative feedback

Semi-structured student interviews / focus groups
Open-ended reflection analysis (thematic)
Faculty facilitator feedback
Expert (Delphi) qualitative suggestions