22  Activity: Chained Workflows

Now let’s work through an example of how you might implement a chained workflow. We’ll start with the work we did in a previous section where we applied analytic rubrics to an OSCE note. What can one do with that note? Many things! Below is an image demonstrating how one could build a chained workflow for a comprehensive approach to evaluating student notes from OSCE assessments.

We won’t complete all of these tasks. Just working through a few steps will be sufficient to give you a sense of how a chained workflow might work in practice.

22.1 Task: Generate Feedback Report

We have a report that we obtained by applying an analytic rubric to the student’s OSCE note, but that report is not sufficient to provide the student with helpful feedback on their performance. Yes, it does provide a score and an indication of what information was present or not in their note, but this doesn’t translate to actionable feedback. We can now craft a prompt that uses this generated report as the basis for a prompt to generate feedback that would be more appropriate to provide to the student.

Code
load("data/analytic_response.Rdata")

build_feedback_prompt <- function(completed_rubric){
  glue::glue(
    "Using the following completed rubric as a guide, provide constructive feedback to the student about their clinical case documentation. 
    
    The feedback should consist of two well-developed paragraphs:
    The first paragraph should offer a balanced assessment of the student's documentation. 
    Begin by acknowledging elements that were documented well (if any), then identify the most critical missing or inadequately addressed components based on the rubric. 
    Be specific about what was missing and, where applicable, note the quality or completeness of what was documented.
    
    The second paragraph should provide educational context by explaining why the missing or inadequate elements are clinically important for evaluating and managing a patient presenting with this specific chief concern. 
    Connect these documentation gaps to potential impacts on patient care, differential diagnosis, treatment planning, or patient safety. 
    Use this as a teaching opportunity to help the student understand the clinical reasoning behind thorough documentation.
    
    Here is the complete rubric on the student's performance: {completed_rubric}
    ")
}

feedback_prompt <- build_feedback_prompt(analytic_rubric)
feedback_response <- claude_plus(feedback_prompt,
                                 temperature = 0)
Code
library(stringr)
library(knitr)

# Some cleaning for the quarto output; not strictly necessary
source('downloads/format_for_qmd.R')

feedback_response <- format_for_qmd(feedback_response)

knitr::asis_output(feedback_response)

Constructive Feedback on Clinical Case Documentation

Your documentation demonstrates attention to several critical elements of this cardiovascular presentation. You appropriately identified the chief concern of chest pain and documented the episodic pattern of symptoms, which is essential for characterizing the temporal nature of cardiac events. You also captured the patient’s poorly controlled hypertension history and recorded vital signs indicating current hypertensive values. However, your documentation has significant gaps that could compromise patient care. Most critically, you failed to document whether the pain radiates to the back, which is a key distinguishing feature in chest pain evaluation. Additionally, and most concerning, you did not formulate or document a working diagnosis such as acute coronary syndrome (ACS), NSTEMI, or STEMI. Without documenting your clinical impression, there is no clear indication of your clinical reasoning process or the urgency level you assigned to this presentation.

The missing elements in your documentation have serious clinical implications for this patient presenting with chest pain. Radiation of chest pain to the back is a red flag symptom that could indicate aortic dissection—a life-threatening condition that can mimic ACS but requires completely different management (contraindicated thrombolytics, need for emergent imaging and surgical consultation). Failing to assess and document this characteristic could lead to catastrophic treatment errors. Furthermore, not documenting a working diagnosis of ACS/NSTEMI/STEMI represents a critical gap in clinical reasoning documentation. In a patient with chest pain, hypertension, and episodic symptoms, ACS must be at the top of your differential diagnosis, driving immediate interventions such as ECG interpretation, cardiac biomarkers, antiplatelet therapy, and cardiology consultation. Your documentation should reflect the urgency of the situation and guide other healthcare providers who may assume care. Complete documentation protects patient safety, ensures care continuity, and demonstrates your clinical decision-making process—all essential components of competent clinical practice.

22.2 Task: Generate Instructor Report

Now that we have student feedback, let’s use this to provide information to the instructor about where they might need to improve their course materials or provide additional instruction / information / attention.

Code
build_remediation_prompt <- function(feedback_output){
  glue::glue("
  You will analyze feedback that has been provided to a student about their performance. 
  Your task is to identify any potential gaps in the course materials or instruction that could be improved to further help the learning. 
  Based on the documentation deficiencies and missing clinical elements identified in the student feedback:
  
  1. Pattern Analysis: Determine whether these gaps likely reflect:
  • Individual student oversight or understanding issues
  • Systematic instructional gaps that may affect multiple students
  • Unclear expectations in the assignment or rubric
  
  2. Curricular Recommendations: If the gaps suggest instructional needs, provide 2-3 specific, actionable recommendations for strengthening the course materials. 
  For each recommendation, specify:
  • Which course component to enhance (e.g., lecture content, practice cases, rubric clarity, pre-assignment resources)
  • What specific content or skill should be emphasized
  • Why this would address the observed documentation gap
  
  3. Context Considerations: Note whether the missing elements are:
  • Foundational knowledge that should have been covered previously
  • Advanced concepts that may need more instructional time
  • Clinical reasoning skills requiring additional practice opportunities

  Format your response as a brief analysis followed by concrete action items that an instructor can implement.
  
  Complete this task using this feedback: {feedback_output}
  ")
}

remediation_prompt <- build_remediation_prompt(feedback_response)
remediation_response <- claude_plus(remediation_prompt,
                                    temperature = 0)
Code
remediation_response <- format_for_qmd(remediation_response)

knitr::asis_output(remediation_response)

Analysis of Feedback and Instructional Gaps

Pattern Analysis

This feedback reveals systematic instructional gaps rather than simple student oversight. The missing elements fall into two critical categories:

  1. Structured clinical reasoning documentation - The absence of a working diagnosis suggests students may not understand that documentation must explicitly demonstrate their clinical thought process
  2. Red flag symptom assessment - Missing the back pain radiation inquiry indicates potential gaps in teaching systematic approaches to high-risk chief complaints

These are unlikely to be individual oversights because they represent fundamental clinical reasoning steps that should be automatic in chest pain evaluation. The pattern suggests students may be completing tasks (gathering some history, recording vitals) without understanding the clinical decision-making framework that should guide comprehensive assessment.

Curricular Recommendations

Recommendation 1: Enhance Pre-Assignment Resources with Clinical Reasoning Templates

Component: Pre-assignment preparatory materials **

Specific Content: ** - Provide a “Clinical Reasoning Documentation Framework” that explicitly shows: Chief Complaint → Systematic Assessment (including red flags) → Working Diagnosis → Clinical Rationale - Include annotated exemplars showing how experienced clinicians document their differential diagnosis thinking - Create a chest pain-specific checklist highlighting life-threatening differentials (ACS, aortic dissection, PE) with their distinguishing features

Rationale: Students appear to be documenting data collection without understanding they must also document clinical synthesis. This addresses the complete absence of working diagnosis documentation and provides scaffolding for systematic red flag assessment.**

Recommendation 2: Integrate Deliberate Practice Cases for High-Risk Chief Complaints

Component: Practice cases with formative feedback **

Specific Content: - Develop 3-4 progressive chest pain scenarios requiring students to: (a) identify must-ask questions for life-threatening conditions, (b) formulate and document differential diagnoses, (c) explain clinical reasoning - Include cases where missing key questions (like back radiation) leads to different diagnoses - Provide immediate feedback showing consequences of incomplete assessment (e.g., “Without asking about back radiation, you cannot rule out aortic dissection”)

Rationale: This directly addresses the missing back pain radiation assessment by making the clinical consequences explicit and memorable. Repeated practice with feedback builds automatic inclusion of critical elements.**

Recommendation 3: Clarify Rubric Expectations for Clinical Reasoning Documentation

Component: Assignment rubric and instructions **

Specific Content: - Add explicit rubric criteria: “Documents working diagnosis with supporting clinical rationale” (separate from data collection) - Include rubric item: “Assesses and documents red flag symptoms specific to chief complaint” - Provide rubric language: “Demonstrates systematic approach to ruling out life-threatening conditions” - In assignment instructions, explicitly state: “Your documentation must include your clinical impression/working diagnosis and the reasoning behind it”

Rationale: Students may believe their task is complete after data collection if expectations for clinical reasoning documentation aren’t explicit. This clarifies that documentation of thinking is as important as documentation of findings.**

Context Considerations

Foundational Knowledge Gaps: - Understanding that documentation serves as evidence of clinical reasoning (not just data recording) - this is foundational professional competency - Knowledge of red flag symptoms for common chief complaints - should be covered in basic clinical assessment courses

Advanced Skills Requiring More Support: - Differential diagnosis formulation in ambiguous presentations - this is an evolving skill requiring substantial practice - Translating clinical reasoning into clear, actionable documentation - bridges cognitive and communication skills

Clinical Reasoning Development: - The gap between knowing facts (aortic dissection exists) and applying systematic approaches (always assess back radiation in chest pain) requires deliberate practice with specific feedback - Students need multiple opportunities to practice the complete cycle: assess → synthesize → document reasoning → receive feedback on clinical logic

Implementation Priority

Immediate action: Clarify rubric expectations (Recommendation 3) - can be implemented before next assignment **

Short-term: Develop clinical reasoning templates (Recommendation 1) - provides immediate scaffolding **

Ongoing development: Build practice case library (Recommendation 2) - most resource-intensive but highest impact for skill development**

The pattern suggests students may be approaching this as a documentation exercise rather than a clinical reasoning demonstration. All recommendations aim to make the expectation explicit: document not just what you found, but what you’re thinking and why.

22.3 Task: All 3 Steps at Once!

Let’s put this all together in a single workflow. I’ve written a different simulated OSCE note so we can see the variation in what the model produces given a different basis for the workflow.

I saved the functions that we used earlier so we can easily load them. This allows us to pay more attention to the chained workflow and not get caught up in the details of the lengthy prompt text. Saving the individual prompts as files to be loaded by source() later is a good practice because you can refine pieces of the chained workflow individually, catching errors and optimizing each step in the workflow. I also saved analytic_rubric in the build_osce_analytic_prompt file instead of repeating it here.

Code
source('downloads/build_osce_analytic_prompt.R')
source('downloads/build_feedback_prompt.R')
source('downloads/build_remediation_prompt.R')

osce_note_2 <- "
45yo m presents with shortness of breath due to intermittent chest pain.
Reports that the pain more noticable on exertion this morning (walking up stairs). 
Hypertensive, although on medication, suggesting it is poorly controlled.
Diagnostic testing to rule out ACS should be completed first."


analytic_response_2 <- claude_plus(
  build_osce_analytic_prompt(osce_note_2, analytic_rubric),
  temperature = 0)
save(analytic_response_2, file = 'data/analytic_response_2.Rdata')
  
feedback_response_2 <- claude_plus(
  build_feedback_prompt(analytic_response_2),
  temperature = 0)
save(feedback_response_2, file = 'data/feedback_response_2.Rdata')

remediation_response_2 <- claude_plus(
  build_remediation_prompt(feedback_response_2),
  temperature = 0)
save(remediation_response_2, file = 'data/remediation_response_2.Rdata')

full_report <- paste(analytic_response_2,
                     feedback_response_2,
                     remediation_response_2,
                     collapse = "\n\n  ***  \n\n")

full_report <- format_for_qmd(full_report)
Code
knitr::asis_output(full_report)

OSCE Post-Encounter Note Scoring Rubric

Criterion 1: Chief concern of chest pain

Status: Included

Justification: The note clearly states “45yo m presents with shortness of breath due to intermittent chest pain.” While shortness of breath is mentioned first, chest pain is explicitly identified as part of the presenting complaint.**

Criterion 2: Episodic pattern of symptoms

Status: Included

Justification: The note describes “intermittent chest pain” and specifies that “the pain more noticable on exertion this morning (walking up stairs),” which indicates an episodic pattern with exertional triggers.**

Criterion 3: Poorly controlled history of hypertension

Status: Included

Justification: The note explicitly states “Hypertensive, although on medication, suggesting it is poorly controlled,” which directly addresses both the history of hypertension and its poor control status.**

Criterion 4: Vitals indicate hypertension

Status: Not Included

Justification: While the note mentions the patient is hypertensive, there is no documentation of actual vital sign measurements (e.g., specific blood pressure readings) to support this finding.**

Criterion 5: Pain radiates to the back

Status: Not Included

Justification: There is no mention of pain radiation to the back or any other location in the note.**

Criterion 6: Likely diagnosis of acute coronary syndrome (ACS), NSTEMI, or STEMI

Status: Included

Justification: The note states “Diagnostic testing to rule out ACS should be completed first,” which indicates ACS is being considered as a likely diagnosis requiring immediate evaluation.**


Summary: 4 of 6 criteria met ** Constructive Feedback**

Your clinical documentation demonstrates several strengths, particularly in capturing the essential presenting complaint and relevant medical history. You clearly identified both the chest pain and shortness of breath, recognized the episodic and exertional nature of the symptoms, and appropriately documented the patient’s poorly controlled hypertension despite medication use. Most importantly, you correctly prioritized acute coronary syndrome (ACS) in your differential diagnosis and appropriately indicated the need for immediate diagnostic testing to rule out this life-threatening condition. However, there are two critical gaps in your documentation that significantly limit the completeness of your clinical assessment. First, you failed to document the actual vital sign measurements, particularly the blood pressure reading that would objectively demonstrate the degree of hypertension present during this acute presentation. Second, you did not document whether the chest pain radiates to the back or any other location—a key characteristic that was presumably elicited during your history-taking.

These omissions have important clinical implications for this patient presenting with chest pain. Documenting specific vital signs, especially blood pressure values, is essential because the degree of hypertension can help differentiate between diagnoses (such as hypertensive emergency, aortic dissection, or ACS) and guides immediate treatment decisions, including whether antihypertensive therapy should be initiated or adjusted during the acute presentation. The pattern of pain radiation is equally critical in this scenario—radiation to the back is a red flag symptom for aortic dissection, a catastrophic condition that can mimic ACS but requires completely different management. Administering thrombolytics or anticoagulation for presumed ACS in a patient with unrecognized aortic dissection could be fatal. In emergency presentations with chest pain, these specific details aren’t just documentation formalities; they are essential data points that directly impact your differential diagnosis, determine which diagnostic tests are ordered, and influence potentially life-saving treatment decisions. Always document objective measurements and specific symptom characteristics to ensure safe, appropriate care. * Analysis of Feedback and Instructional Gaps

Pattern Analysis

This feedback reveals systematic instructional gaps rather than simple student oversight. The student demonstrated strong clinical reasoning (correctly prioritizing ACS, recognizing the need for immediate testing) but failed to document two fundamental clinical elements:

  1. Objective vital sign values (specific BP measurements)
  2. Critical symptom characteristics (pain radiation pattern)

The disconnect between good clinical thinking and incomplete documentation suggests students may not fully understand: - Which specific data elements are non-negotiable in documentation - Why certain details have life-or-death clinical implications - How documentation directly drives differential diagnosis refinement

This pattern indicates students are learning clinical reasoning but not the parallel skill of translating clinical assessment into complete, actionable documentation.


Curricular Recommendations

*Recommendation 1: Create a “High-Stakes Documentation Checklist” Resource**

Component to enhance: Pre-assignment resources and rubric materials**

Specific content: Develop a symptom-specific documentation checklist for common chief complaints (chest pain, shortness of breath, abdominal pain, headache). For chest pain specifically, create a visual checklist highlighting:** - Mandatory vital signs with rationale (BP values → differentiates dissection/hypertensive emergency) - Critical symptom descriptors (radiation pattern → rules in/out dissection) - “Cannot proceed without” documentation elements marked distinctly

Why this addresses the gap: Students need explicit guidance on which elements are clinically mandatory versus supplementary. The feedback shows the student captured history but missed critical specifics—a checklist bridges the gap between “taking a history” and “documenting decision-critical details.” This makes implicit expectations explicit.**


*Recommendation 2: Integrate “Documentation Consequences” Case Scenarios**

Component to enhance: Lecture content and practice cases**

Specific content: Add 2-3 brief case vignettes demonstrating adverse outcomes from incomplete documentation:** - Case A: Chest pain documented without radiation pattern → aortic dissection missed, anticoagulation given, patient deteriorates - Case B: “Elevated BP” documented without values → treatment delayed, hypertensive emergency progresses

Follow each vignette with guided reflection: “What specific documentation element was missing? What was the clinical consequence? How does this change your documentation approach?”

Why this addresses the gap: The feedback emphasizes clinical implications, but students may not viscerally understand the connection between documentation omissions and patient harm. Concrete examples of “what goes wrong” when specific elements are missing transforms documentation from a bureaucratic task to a patient safety imperative.**


*Recommendation 3: Add “Documentation Translation” Practice Exercises**

Component to enhance: Practice cases with structured feedback**

Specific content: Provide 3-4 practice scenarios where students receive a clinical vignette with complete information, then must:** 1. Identify which elements are “mandatory to document” vs. “supplementary” 2. Write the documentation 3. Receive immediate feedback comparing their documentation against an expert exemplar with annotations explaining why each element matters clinically

Include specific practice with vital sign documentation (writing actual values, not interpretations) and symptom characterization (specific descriptors, not generalizations).

Why this addresses the gap: This student likely heard the information during history-taking but didn’t recognize it as documentation-critical. Deliberate practice in translating clinical data into complete documentation—with immediate feedback—builds the habit of capturing specific, actionable details rather than general impressions.**


Context Considerations

Foundational knowledge requiring reinforcement: - Documentation of objective measurements (vital signs as numbers, not interpretations) - This is basic clinical documentation but may need explicit re-teaching in the context of decision-making

Clinical reasoning skills requiring additional practice: - Connecting specific documentation elements to differential diagnosis refinement (e.g., pain radiation → changes DDx from ACS alone to ACS vs. dissection) - Understanding documentation as a clinical reasoning tool, not just a record-keeping task

Advanced concept needing more emphasis: - The concept of “red flag symptoms” and their documentation priority - How specific details change management pathways (dissection vs. ACS requiring opposite treatments)


Implementation Priority

Immediate action: Implement Recommendation 1 (checklist) before the next assignment—low effort, high impact.**

Short-term: Develop Recommendation 3 (practice exercises) for ongoing skill-building.**

Curriculum revision: Integrate Recommendation 2 (consequence cases) into core lecture

22.4 Task: Brainstorm a Chained Workflow

Now it’s time for you to think about how you can utilize a chained workflow!

What sophisticated tasks / workflows could benefit from integrating a generative AI chained workflow? What processes could you automate?

In what steps does the model output flow directly into another model call? Are there places where you’d like to have human review before the model completes additional steps?

For this activity, identify at least 3 steps where a generative AI model could be useful in your workflow.