# Activity: Prompt Engineering {#sec-act-prompting}
It's time to practice prompt engineering!
## Task 1: Prompt Formulas
Pick a particular use case and try fitting that use case into the 3 different prompt formulas:
- **Role + Task + Output**
- **Action, Context, Tone (ACT) Formula**
- **The C-A-R-E Framework**
- Context: [situation]
- Action: [what to do]
- Result: [desired output]
- Explanation: [why or how]
For my example, I'm going to use the use case of explaining test reliability to K-12 educators who do not have a psychometrics background and want to better understand why student test scores vary between test administrations.
::: {.callout-note collapse="true"}
## Role + Task + Output
```{r, eval = FALSE}
source('downloads/claude_plus.R')
source('downloads/format_for_qmd.R')
rto_prompt <-
"Role: You are en educational measurement specialist.
Task: Explain the concept of *test reliability* to K–12 educators who are unfamiliar with psychometrics and want to better understand why student scores sometimes vary between test administrations.
Output: Provide a concise explanation (2-3 short paragraphs) followed by one classroom-based example in bullet form."
reliability_rto <- claude_plus(rto_prompt)
reliability_rto <- format_for_qmd(reliability_rto)
```
```{r, eval = FALSE, echo = FALSE}
save(reliability_rto, file = 'data/reliability_rto.Rdata')
```
```{r, echo = FALSE}
load('data/reliability_rto.Rdata')
```
```{r}
knitr::asis_output(reliability_rto)
```
:::
::: {.callout-note collapse="true"}
## "ACT" (Action - Context - Tone)
```{r, eval = FALSE}
act_prompt <- "
Action: Explain the concept of test reliability.
Context: You’re writing to K–12 educators who are unfamiliar with psychometrics and want to better understand why student scores sometimes vary between test administrations.
Tone: Friendly, conversational, and encouraging — as if you’re helping teachers connect a familiar classroom experience to an underlying measurement idea."
reliability_act <- claude_plus(act_prompt)
reliability_act <- format_for_qmd(reliability_act)
```
```{r, eval = FALSE, echo = FALSE}
save(reliability_act, file = 'data/reliability_act.Rdata')
```
```{r, echo = FALSE}
load('data/reliability_act.Rdata')
```
```{r}
knitr::asis_output(reliability_act)
```
:::
::: {.callout-note collapse="true"}
## "CARE" (Context - Action- Result - Explanation)
```{r, eval = FALSE}
care_prompt <- "
Context: Teachers have noticed that their students’ scores fluctuate across testing sessions and are unsure what that means.
Action: Explain the concept of test reliability in a way that helps K–12 educators unfamiliar with psychometrics make sense of these score variations.
Result: They should understand that reliability reflects the consistency of test scores and why it matters for interpreting student performance.
Explanation: Include one concrete example that links reliability to real classroom assessment practices."
reliability_care <- claude_plus(care_prompt)
reliability_care <- format_for_qmd(reliability_care)
```
```{r, eval = FALSE, echo = FALSE}
save(reliability_care, file = 'data/reliability_care.Rdata')
```
```{r, echo = FALSE}
load('data/reliability_care.Rdata')
```
```{r}
knitr::asis_output(reliability_care)
```
:::
What did you notice when writing the prompts?
What information was more easily apparent that needed to be included in the prompt by using the different formulas?
Which one did you like best, and why?
## Task 2: Progressive Constraints
Let's now do a similar activity - that of adding **progressive constraints**. This process probably most closely mirrors how I iterate and refine prompts before I start to empirically test them.
Start with a vague prompt, and then add another bit of detail and see how this changes the output. And then another bit of detail, see how it changes the output, and so on. I'll do this so there are a total of 4 interactions with the model.
I'll also add a system prompt of "You are a measurement educator who explains things concisely in a single paragraph" to decrease the amount of output you'll have to read through. 😅
::: {.callout-note collapse="true"}
## Progressive 1
```{r, eval = FALSE}
progressive_1 <- "Explain test validity."
validity_p1 <- claude_plus(progressive_1,
system = "You are a measurement educator who explains things concisely in a single paragraph")
validity_p1 <- format_for_qmd(validity_p1)
```
```{r, eval = FALSE, echo = FALSE}
save(validity_p1, file = 'data/validity_p1.Rdata')
```
```{r, echo = FALSE}
load('data/validity_p1.Rdata')
```
```{r}
knitr::asis_output(validity_p1)
```
:::
::: {.callout-note collapse="true"}
## Progressive 2
```{r, eval = FALSE}
progressive_2 <- "Explain test validity to K–12 educators who are not familiar with psychometrics."
validity_p2 <- claude_plus(progressive_2,
system = "You are a measurement educator who explains things concisely in a single paragraph")
validity_p2 <- format_for_qmd(validity_p2)
```
```{r, eval = FALSE, echo = FALSE}
save(validity_p2, file = 'data/validity_p2.Rdata')
```
```{r, echo = FALSE}
load('data/validity_p2.Rdata')
```
```{r}
knitr::asis_output(validity_p2)
```
:::
::: {.callout-note collapse="true"}
## Progressive 3
```{r, eval = FALSE}
progressive_3 <- "Explain test validity to K–12 educators unfamiliar with psychometrics, focusing on helping them understand why some test results may not reflect true student ability."
validity_p3 <- claude_plus(progressive_3,
system = "You are a measurement educator who explains things concisely in a single paragraph")
validity_p3 <- format_for_qmd(validity_p3)
```
```{r, eval = FALSE, echo = FALSE}
save(validity_p3, file = 'data/validity_p3.Rdata')
```
```{r, echo = FALSE}
load('data/validity_p3.Rdata')
```
```{r}
knitr::asis_output(validity_p3)
```
:::
::: {.callout-note collapse="true"}
## Progressive 4
```{r, eval = FALSE}
progressive_4 <- "Explain test validity to K–12 educators unfamiliar with psychometrics, focusing on helping them understand why some test results may not reflect true student ability. Write in plain language suitable for a short teacher newsletter and avoid using the words “psychometrics” or “construct.”"
validity_p4 <- claude_plus(progressive_4,
system = "You are a measurement educator who explains things concisely in a single paragraph")
validity_p4 <- format_for_qmd(validity_p4)
```
```{r, eval = FALSE, echo = FALSE}
save(validity_p4, file = 'data/validity_p4.Rdata')
```
```{r, echo = FALSE}
load('data/validity_p4.Rdata')
```
```{r}
knitr::asis_output(validity_p4)
```
:::
## Task 3: Including Examples (Zero-shot v Few-shot)
This activity demonstrates how large language models can learn implicitly from examples provided within the prompt, even when the task involves completely unfamiliar or made-up categories (e.g., information not well represented in their training data).
In a **zero-shot** setting, the model receives only instructions, so it must guess without any contextual grounding.
In a **few-shot** setting, however, the model can observe patterns in the examples — inferring a latent rule or decision boundary and then applying that inferred rule to new inputs.
::: {.callout-note collapse="true"}
## Zero-Shot
```{r, eval = FALSE}
make_zero <- "
Classify the following sentences as wamples or doglets:
1. The concert was amazing and everyone was smiling.
2. The student felt frustrated after failing the exam.
3. The sunset filled the sky with brilliant colors.
4. The meeting dragged on and everyone was bored.
Respond _only_ with the sentence and either (wample) or (doglet) after the sentence."
zeroshot <- claude_plus(make_zero)
zeroshot <- format_for_qmd(zeroshot)
```
```{r, eval = FALSE, echo = FALSE}
save(zeroshot, file = 'data/zeroshot.Rdata')
```
```{r, echo = FALSE}
load('data/zeroshot.Rdata')
```
```{r}
knitr::asis_output(zeroshot)
```
:::
::: {.callout-note collapse="true"}
## Few-Shot
```{r, eval = FALSE}
make_few <- "
A doglet sentence describes a pleasant or positive experience.
A wample sentence describes an unpleasant or negative experience.
Examples:
– The people were having an enjoyable day. (doglet)
– It was raining and the woman was sad. (wample)
– The person was happy to be eating their favorite food. (doglet)
– The person had a stomach ache after eating too fast. (wample)
Classify the following sentences as wamples or doglets:
1. The concert was amazing and everyone was smiling.
2. The student felt frustrated after failing the exam.
3. The sunset filled the sky with brilliant colors.
4. The meeting dragged on and everyone was bored.
Respond _only_ with the sentence and either (wample) or (doglet) after the sentence."
fewshot <- claude_plus(make_few)
fewshot <- format_for_qmd(fewshot)
```
```{r, eval = FALSE, echo = FALSE}
save(fewshot, file = 'data/fewshot.Rdata')
```
```{r, echo = FALSE}
load('data/fewshot.Rdata')
```
```{r}
knitr::asis_output(fewshot)
```
:::
## Task 4 : Prompt Improvement
- Step 1: Write your own prompt - whatever topic you'd like, or re-use one that we've already used today. Don't submit the prompt to the model yet.
- Step 2: Using the prompt below to ask the model for help with your prompt.
I'd like your help improving the prompt below. Please review it and suggest ways to make it clearer and more effective.
Specifically:
• Identify any missing context or details that would help generate better results
• Point out areas where the prompt might be vague or ambiguous
• Suggest how to better structure the request
• Recommend any information I should add about my goals, audience, or desired format
Here is my prompt: {paste your prompt here}
```{r, eval = FALSE}
prompt_prompt <- "I'd like your help improving the prompt below. Please review it and suggest ways to make it clearer and more effective.
Specifically:
• Identify any missing context or details that would help generate better results
• Point out areas where the prompt might be vague or ambiguous
• Suggest how to better structure the request
• Recommend any information I should add about my goals, audience, or desired format
Here is my prompt:
What are the pros and cons of Bayesian vs Frequentist statistics? For each point, include a brief explanation."
improved_prompt <- claude_plus(prompt_prompt)
improved_prompt <- format_for_qmd(improved_prompt)
```
```{r, eval = FALSE, echo = FALSE}
save(improved_prompt, file = 'data/improved_prompt.Rdata')
```
```{r, echo = FALSE}
load('data/improved_prompt.Rdata')
```
::: {.callout-note collapse="true"}
## Improved Prompt Result
```{r}
knitr::asis_output(improved_prompt)
```
:::