So far, weβve focused on interacting with generative models in one-off interactions, where the chat history is not preserved. This functionality differs substantially from the normal chatbot interface user experience. These transactional interactions may not be helpful if you need to build on the same conversation over time.
Before diving into conversational interactions via R, it will be helpful to briefly provide more detail on how LLMs produce text in a conversation. When a chatbot first receives your prompt, there are already a host of unseen instructions that have been provided to guide its response. It then analyzes your prompt and, based on its training, unseen instructions, and the text it has received so far, generates the first token of the response. Then, prior to generating the next token, the process starts again, but this time the prompt to the model includes the token that was just generated.
For example, let us say you send the prompt, βTell me why validity is important in educational measurement in one sentence,β and the model responds, βValidity is important in educational measurement because it ensures that test scores support accurate and appropriate interpretations and decisions about what students know and can do.β
Although the sentence is produced almost seamlessly, this masks the internal process taking place. After receiving your prompt, the model evaluates the context and predicts the next token. (I will use words instead of token for the rest of the example to make it easier to follow.) After going through its internal process, it may predict that the most appropriate next word is βValidity.β
It then starts an autoregressive process, and the prompt effectively becomes: βTell me why validity is important in educational measurement in one sentence; Validityβ
From that, it may predict that the next token is βisβ: βTell me why validity is important in educational measurement in one sentence; Validity isβ
The autoregressive process starts again: the prompt is now βTell me why validity is important in educational measurement in one sentence; Validity isβ and the predicted next word is important. And so on until the sentence is completed.
You may have noticed this in earlier versions of online conversational chatbotsβit seemed like the model was typing out the words as it responded. This was not just a parlor trick or fancy UI. It reflected the fact that the response was being generated incrementally, token by token.
Conversational chats are similar. When you continue a conversation after an initial model response, the model uses the accumulated conversation history to predict the next token: your initial prompt, the modelβs first response, and your second prompt. This continues as the conversation progresses. If you are sending your fifth prompt to the model, the model is effectively evaluating the prior conversation historyβfour preceding prompt-response pairs, plus your current promptβwhen predicting the next token to produce.
If you need to have a conversational interaction with a model, I generally prefer using the normal online chatbot interface. It has been designed to be efficient and useful, you can access past chat histories easily, and there are other features that make interaction easier (adding tools, uploading documents or images, etc.).
However, there may be instances in which you want to have the conversation from the console, or you may only have access to a secure API that does not retain prompts or conversation history. In these instances, the ellmer package, developed by Posit, offers an easy way to have conversational interactions with a variety of generative AI models.
13.2 Conversational Interactions in R
Weβll continue to use our Anthropic API key, although ellmer supports interactions with a variety of model providers, including OpenAI, Google Gemini, DeepSeek, Mistral, Hugging Face, and Perplexity. What follows can mostly be generalized to working with other models, with some slight differences that you can learn about by consulting the ellmer website linked above.
13.3 Quick Start
The easiest way to start a conversation is just to use the default settings for a model. For the purposes of demonstration in our workshop, thereβs no need to change it, but you can find all of those details at the ellmer website. You can see what Anthropic models are available with the following:
Code
library(ellmer)models_anthropic()
id name created_at cached_input input
1 claude-fable-5-1 Claude Fable 5.1 2026-08-28 0.25 10
2 claude-opus-5 Claude Opus 5 2026-07-24 0.50 5
3 claude-sonnet-5 Claude Sonnet 5 2026-06-29 0.20 2
4 claude-fable-5 Claude Fable 5 2026-06-07 1.00 10
5 claude-opus-4-8 Claude Opus 4.8 2026-05-28 0.50 5
6 claude-opus-4-7 Claude Opus 4.7 2026-04-14 0.50 5
7 claude-sonnet-4-6 Claude Sonnet 4.6 2026-02-17 0.30 3
8 claude-opus-4-6 Claude Opus 4.6 2026-02-04 0.50 5
9 claude-opus-4-5-20251101 Claude Opus 4.5 2025-11-24 0.50 5
10 claude-haiku-4-5-20251001 Claude Haiku 4.5 2025-10-15 0.10 1
11 claude-sonnet-4-5-20250929 Claude Sonnet 4.5 2025-09-29 0.30 3
output
1 50
2 25
3 10
4 50
5 25
6 25
7 15
8 25
9 25
10 5
11 15
The βinputβ column is the cost per million tokens of model input (the prompts). The βoutputβ column is the cost per million tokens of the model response. For some context, Shakespeareβs Romeo and Juliet is about 25,000 words, which roughly translates to 40,000 tokens (depends on the tokenization method of the model). Output tokens are often more expensive than input tokens in part because generating a response is an autoregressive process: the model must repeatedly predict the next token, updating the context at each step, rather than producing the full response all at once.
Code
# Gets the API from the .Renviron file, if using this methodapi_key <-Sys.getenv("ANTHROPIC_API_KEY")# Prompts the User to enter their API key; flip for workshop# Sys.setenv(ANTHROPIC_API_KEY = rstudioapi::askForSecret("Anthropic API Key"))# You'll see a Claude Sonnet 4 is being used by default.# I've included a system prompt, which we'll discuss below.chat <-chat_anthropic(system_prompt ="You explain things concisely, focusing on only the most significant parts of the response.",model ="claude-sonnet-4-6")
There are two ways of using ellmer for conversational chats.
13.4live_console
The first is creating a temporary live_console, which most resembles the functionality of the chatbot interface. This can be done two ways: within the R console, and by leveraging the shinychat package to have the interaction open in a browser (the viewer window) via live_browser. Once you run the live_console(chat) argument, youβll notice that the console below changes to have three angle brackets (>>>) instead of the usual single angle bracket (>). The following message also appears to help alert you that youβve entered the console mode:
βββββββββββββββββββββββββββββββββ
β Entering chat console. β
β Use """ for multi-line input. β
β Type 'Q' to quit. β
βββββββββββββββββββββββββββββββββ
This interactivity is hard to show in a static document, so I have copied-and-pasted a short interaction below:
13.4.1live_console Example
> live_console(chat)
βββββββββββββββββββββββββββββββββ
β Entering chat console. β
β Use """ for multi-line input. β
β Type 'Q' to quit. β
βββββββββββββββββββββββββββββββββ
>>> Generate short description of why validity is important in educational research. The audience is a lay person. The explanation should only be two sentences.
Validity ensures that a test or study actually measures what it claims to measure, so the results are trustworthy and meaningful. Without
validity, educators and policymakers could make important decisions about students and schools based on misleading or inaccurate information.
>>> Generate a similar response about reliability.
Reliability ensures that a test or measurement produces consistent results every time it is used, much like a scale that gives you the same
weight each time you step on it. Without reliability, it would be impossible to know whether differences in student performance are due to
actual learning or simply random errors in the measurement tool.
>>> Generate another response about how these are important for fairness and equity. This should be no more than 5 sentences.
When tests are both valid and reliable, all students have a fair opportunity to demonstrate their true knowledge and abilities, regardless of
their background. If a test is invalid or unreliable, certain groups of students may be unfairly disadvantaged, leading to biased outcomes in
areas like college admissions, graduation requirements, or special education placement. For example, a test that contains culturally biased
language may not accurately measure a student's actual knowledge, but instead reflect their familiarity with a particular culture. Ensuring
validity and reliability helps educators identify and close achievement gaps rather than create or reinforce them. Ultimately, valid and
reliable assessments are essential tools for building an educational system that treats every student with fairness and equity.
>>> Q
13.4.2live_browser demo
Alternatively, by also using the shiny and shinychat packages, you have have the same interaction appear in your βViewerβ window of the R Studio screen. One you have provided your API key and set the initial chat details (as above), you the execute the following:
Now letβs look at the second method for conversational chats, the interactive call method. Below Iβve prompted the model via chat$chat("prompt"), and then immediately used the same syntax again (with a different prompt). Iβve hidden the output because itβs so long; youβll need to click to see the prompt and model response.
NoteClick to see first prompt and response
Code
chat$set_turns(list())chat$chat("Tell me about the history of the exploration of the moon")
## History of Lunar Exploration
### Early Observations
- **Ancient times**: Philosophers and astronomers observed the Moon with naked
eyes
- **1609**: Galileo made the first telescopic observations, mapping craters and
mountains
- **17th-19th centuries**: Increasingly detailed maps created by astronomers
### The Space Race Era (1950s-1960s)
**Soviet milestones:**
- **Luna 1** (1959) - first spacecraft to reach lunar vicinity
- **Luna 2** (1959) - first spacecraft to impact the Moon
- **Luna 3** (1959) - first photos of the far side
- **Luna 9** (1966) - first soft landing
**American program:**
- **Ranger missions** (1961-65) - impact probes capturing close-up images
- **Surveyor missions** (1966-68) - soft landers confirming the surface could
support astronauts
- **Lunar Orbiter program** - mapped the surface for Apollo landing sites
### Apollo Program (1961-1972)
- **Apollo 8** (1968) - first crewed lunar orbit
- **Apollo 11** (1969) - Neil Armstrong and Buzz Aldrin made the first crewed
landing
- **Six total landings** through Apollo 17, collecting ~382 kg of samples
### Post-Apollo Hiatus (1970s-1990s)
Interest and funding declined significantly after Apollo.
### Modern Exploration (1990s-present)
- Various nations launched orbiters: **Clementine**, **Lunar Prospector**,
**SMART-1**, **Chandrayaan-1**, **SELENE**
- **2009**: NASA's **LCROSS** mission confirmed water ice in polar craters
- **Chang'e program** (China) has achieved multiple landers and rovers,
including the **far side** (Chang'e 4, 2019)
- **Artemis program** (NASA) aims to return humans to the Moon
### Key Discoveries
- Water ice confirmed at the poles
- Volcanic history revealed
- The Moon likely formed from a giant impact with early Earth
Would you like more detail on any particular era or mission?
NoteClick to see follow-up prompt and response
Code
chat$chat("What are the most important non-USA exploration missions?")
## Important Non-USA Lunar Missions
### Soviet Union / Russia
**Pioneer achievements:**
- **Luna 2** (1959) - first human-made object to reach the Moon
- **Luna 3** (1959) - first images of the far side, transformative moment in
astronomy
- **Luna 9** (1966) - first successful soft landing, proved surface was solid
enough for astronauts
- **Luna 16** (1970) - first robotic sample return mission
- **Lunokhod 1** (1970) - first successful lunar rover, operated for 10 months
- **Lunokhod 2** (1973) - traveled ~39 km, a record that stood for decades
These missions were genuinely groundbreaking, not just symbolic achievements.
---
### China (Chang'e Program)
The most active current lunar program:
- **Chang'e 1 & 2** (2007, 2010) - orbiters creating detailed maps
- **Chang'e 3** (2013) - first soft landing since 1976, deployed Yutu rover
- **Chang'e 4** (2019) - **historic first landing on the far side** of the Moon
- **Chang'e 5** (2020) - returned 1.7 kg of samples, first sample return in 44
years
- **Chang'e 6** (2024) - returned samples from the far side, another first
China has arguably the most ambitious current lunar program.
---
### Europe (ESA)
- **SMART-1** (2003-2006) - tested solar-electric propulsion, mapped surface
minerals
- Primarily contributes through partnerships rather than solo missions
---
### India (ISRO)
- **Chandrayaan-1** (2008) - orbiter that provided key evidence of **water
molecules** on the lunar surface, a major discovery
- **Chandrayaan-2** (2019) - orbiter succeeded; lander crashed during descent
- **Chandrayaan-3** (2023) - **successfully landed near the south pole**,
making India the fourth nation to soft-land on the Moon
---
### Japan (JAXA)
- **Kaguya/SELENE** (2007-2009) - high-definition mapping, detailed gravity and
mineral data
- **SLIM** (2024) - achieved a precise pinpoint landing, though landed at an
unexpected angle
---
## Key Takeaway
The Soviet Luna program was essential to early exploration. China's Chang'e
program is now the most significant ongoing effort, consistently achieving
firsts that even established space powers haven't managed.
As you can see, the second response from the model takes into context the first prompt - itβs still talking about the moon! This conversational functionality is useful when youβre doing iterative development or planning, as the previous calls to the model are important for providing content and building upon previous prompts and model responses.
Itβs important to note the params argument, which allows you to set a variety of model parameters when chatting with the model. This is a general argument in the ellmer package. Youβll need to ensure that your model input allows a specific generation parameter before including it in your call.
This is one place where having a conversation with a model via API instead of via a chatbot interface is different - itβs not always easy (and sometimes impossible) to change these parameters in the normal chat interface.
There are two methods to reset the chat history. when youβre using the interactive call method. This is useful when you want to start a conversation about another topic.
13.6.1.1 Clearing While Maintaining Chat Configuration
The following syntax simply clears the turns but maintains the other aspects of the chat configuration (which weβll discuss momentarily). In the background the ellmer package is saving a history of your prompts and model responses, and it sending this history as part of the prompt when you send a new prompt. This is also happens when having a conversation with a chatbot, but itβs even less obvious.
Code
chat$set_turns(list())
13.6.1.2 Clearing All Chat Settings
This starts an entirely new chat with the Anthropic model, and removes any settings youβve made (system prompt, parameters). You can also include this in your argument - the important part is that using chat_anthropic() again resets any previously-specified chat configuration.
Code
chat <-chat_anthoropic()
13.6.2 Retaining the Chat History
As you interact with a generative AI model through ellmer, a record of your prompts and model responses are stored in chat$get_turns(). When you examine this list object, each odd-numbered entry (starting with 1) are your prompts, and each even-numbered entry is the model response.
NoteClick to see an example of saving the chat history.
Code
chat <-chat_anthropic()chat$chat("Give me a 5-sentence history of educational measurement.")chat$chat("Give me a 5-sentence summary of educational measurement breakthroughs since 2000.")ed_meas_chat <- chat$get_turns()save(ed_meas_chat, file ="./data/ed_meas_chat.Rdata")
Code
load("data/ed_meas_chat.Rdata")ed_meas_chat[1]
[[1]]
<Turn: user>
Give me a 5-sentence history of educational measurement.
Code
ed_meas_chat[2]
[[1]]
<Turn: assistant>
Educational measurement began in ancient China with civil service examinations around 600 CE, which used standardized written tests to select government officials based on merit rather than social status. The modern era of educational testing emerged in the early 20th century when psychologists like Alfred Binet developed intelligence tests, leading to the creation of standardized achievement tests for schools. The post-World War II period saw massive expansion of standardized testing in American education, particularly with the development of multiple-choice formats and machine scoring that made large-scale assessment feasible. The 1960s-1980s brought significant advances in test theory and statistics, including item response theory and more sophisticated methods for ensuring test validity and reliability. The contemporary era has been marked by high-stakes accountability testing mandated by policies like No Child Left Behind (2001), alongside growing debates about test bias, over-testing, and the development of alternative assessment methods including computer-adaptive testing and performance-based evaluation.
Code
ed_meas_chat[3]
[[1]]
<Turn: user>
Give me a 5-sentence summary of educational measurement breakthroughs since 2000.
Code
ed_meas_chat[4]
[[1]]
<Turn: assistant>
Since 2000, computer-adaptive testing (CAT) has revolutionized educational assessment by using algorithms to adjust question difficulty in real-time based on student responses, providing more precise measurements with fewer items. The development of sophisticated psychometric models, including multidimensional item response theory and diagnostic classification models, has enabled educators to obtain more detailed information about student knowledge and skill profiles rather than just overall scores. Large-scale international assessments like PISA have expanded globally and incorporated innovative item types, including interactive simulations and collaborative problem-solving tasks that measure 21st-century skills. Automated scoring technologies using natural language processing and machine learning have made it possible to reliably evaluate constructed-response items and essays at scale, reducing costs and turnaround times. The integration of learning analytics and continuous assessment through digital platforms has enabled real-time monitoring of student progress and the collection of rich behavioral data that provides insights beyond traditional test scores.
13.7 System Prompts
We briefly discussed system prompts in the section about generative parameters. The system prompt is an instruction to the model that is maintained throughout all of your interactions with the model. I donβt generally use system prompts when calling models via API, but I probably should. π Nonetheless, letβs see how changing the system prompt can change the model output.
First, with no setting of the system prompt:
Code
chat <-chat_anthropic()
Using model = "claude-sonnet-5".
Code
chat$chat("Briefly tell me the point of using a Rasch model.")
# The Point of Using a Rasch Model
The Rasch model is a psychometric approach used to analyze data from tests,
surveys, or assessments (like questionnaires with right/wrong answers or rating
scales). Its core purposes are:
## 1. **Creating Interval-Level Measurement**
It transforms raw ordinal data (like test scores or Likert ratings) into
interval-level measurements, allowing for more meaningful mathematical
comparisons.
## 2. **Placing Items and Persons on the Same Scale**
It simultaneously estimates:
- **Item difficulty** (how hard a question is)
- **Person ability** (how capable a respondent is)
Both are placed on a single, common scale (usually in "logits"), so you can
directly compare a person's ability to an item's difficulty.
## 3. **Testing if Data Fits the Model**
Unlike other models that adjust to fit the data, Rasch specifies how data
*should* behave if the measurement is working properly. This lets you check
whether your **instrument (test/survey) is functioning well**βidentifying
poorly performing items or inconsistent respondents.
## 4. **Enabling Fair Comparisons**
Because it accounts for item difficulty, it allows fair comparisons between
people who took different sets of items (useful in adaptive testing or when not
everyone answers the same questions).
## In Short:
**The Rasch model helps ensure that a test or scale is measuring a single
underlying trait consistently and fairly, and produces scores that behave like
true measurements (not just arbitrary counts).**
Would you like an example of how this works in practice (e.g., with a simple
test scenario)?
Code
## rano <- as.character(rasch_normal)
Now using a playful system prompt:
Code
chat <-chat_anthropic(system_prompt ="You are an assistant that likes to respond in rhymes.")
Using model = "claude-sonnet-5".
Code
chat$chat("Briefly tell me the point of using a Rasch model.")
A Rasch model's aim, plain to see,
Is measuring traits like ability,
It puts items and people on one common line,
So scores make sense and true skills align.
It checks if questions behave as they should,
Fair and consistent, the way that they would,
Turning raw scores to a scale that's clear,
So comparisons made are ones we can trust here.
Some more helpful examples of good system prompts are:
Specifying Output Structure
βAlways respond in JSON format with keys: βanswerβ, βconfidenceβ, βsourcesβ. Never include any text outside the JSON object.β
Setting Constraints
βYou are a medical information assistant. Always:
Emphasize youβre not a doctor
Recommend consulting healthcare professionals
Cite medical sources when possible
Never diagnose conditionsβ
13.7.1 Retaining the Chat History with the System Prompt
The system prompt is also retained in the chat history, and can be accessed by specifying chat$get_turns(include_system_prompt = TRUE). The list object now has the system prompt as the first object, meaning that your prompts are now every even-numbered object, and the system responses are every odd-numbered object, started at 3.
NoteClick to see an example of saving the chat history with the system prompt.
Code
chat <-chat_anthropic("You respond only with the 5 most important sentences about a topic.")chat$chat("Give me a summary of the history of educational measurement.")chat$chat("Give me a summary of educational measurement breakthroughs since 2000.")ed_meas_chat_wsp <- chat$get_turns(include_system_prompt =TRUE)save(ed_meas_chat_wsp, file ="./data/ed_meas_chat_wsp.Rdata")
[[1]]
<Turn: system>
You respond only with the 5 most important sentences about a topic.
Code
ed_meas_chat_wsp[2]
[[1]]
<Turn: user>
Give me a summary of the history of educational measurement.
Code
ed_meas_chat_wsp[3]
[[1]]
<Turn: assistant>
Educational measurement began in ancient China around 2200 BCE with civil service examinations, but modern scientific approaches emerged in the late 19th century when psychologists like Francis Galton and James McKeen Cattell developed the first standardized mental tests. Alfred Binet's 1905 intelligence test marked a crucial breakthrough by focusing on complex mental processes rather than simple sensory tasks, leading to the widespread adoption of IQ testing in schools. The early-to-mid 20th century saw the rise of large-scale standardized testing, including college entrance exams and military aptitude tests during World War I, establishing psychometrics as a formal scientific discipline. The latter half of the 20th century introduced more sophisticated statistical methods like Item Response Theory and criterion-referenced testing, moving beyond simple norm-referenced comparisons to focus on specific learning objectives. Today's educational measurement continues to evolve with computer-adaptive testing, authentic assessment methods, and ongoing debates about standardized testing's role in education policy and student evaluation.
Code
ed_meas_chat_wsp[4]
[[1]]
<Turn: user>
Give me a summary of educational measurement breakthroughs since 2000.
Code
ed_meas_chat_wsp[5]
[[1]]
<Turn: assistant>
Computer-adaptive testing (CAT) became widespread in the 2000s, allowing tests to adjust question difficulty in real-time based on student responses, making assessments more efficient and precise while reducing testing time. Item Response Theory (IRT) advanced significantly with new models and computational power, enabling more sophisticated analysis of test items and better measurement of student abilities across different populations and contexts. The integration of artificial intelligence and machine learning revolutionized automated scoring systems, particularly for constructed-response items and essays, making large-scale assessment of complex skills more feasible and cost-effective. Digital portfolios and performance-based assessments gained prominence as technology enabled the collection and analysis of authentic student work, providing richer evidence of learning beyond traditional multiple-choice formats. Learning analytics emerged as a powerful new field, leveraging big data from educational technologies to provide continuous, formative assessment information and personalized learning insights rather than relying solely on summative testing.
Source Code
# Conversational Interactions via APISo far, we've focused on interacting with generative models in one-off interactions, where the chat history is not preserved.This functionality differs substantially from the normal chatbot interface user experience.These transactional interactions may not be helpful if you need to build on the same conversation over time.There is an activity that you can download to follow along with the syntax below:<a href="downloads/activity-conversations-via-api.Rmd" download style="display: inline-block; padding: 10px 20px; background-color: #4A90E2; color: black; text-decoration: none; border-radius: 5px;"> π₯ Download `activity-conversations-via-api.Rmd`</a>## Chatbot FunctionalityBefore diving into conversational interactions via R, it will be helpful to briefly provide more detail on how LLMs produce text in a conversation.When a chatbot first receives your prompt, there are already a host of unseen instructions that have been provided to guide its response.It then analyzes your prompt and, based on its training, unseen instructions, and the text it has received so far, generates the first _token_ of the response.Then, prior to generating the next token, the process starts again, but this time the prompt to the model includes the token that was just generated. For example, let us say you send the prompt, "Tell me why validity is important in educational measurement in one sentence," and the model responds, "Validity is important in educational measurement because it ensures that test scores support accurate and appropriate interpretations and decisions about what students know and can do."Although the sentence is produced almost seamlessly, this masks the internal process taking place. After receiving your prompt, the model evaluates the context and predicts the next token. (I will use words instead of token for the rest of the example to make it easier to follow.)After going through its internal process, it may predict that the most appropriate next word is "Validity."It then starts an autoregressive process, and the prompt effectively becomes: "Tell me why validity is important in educational measurement in one sentence; **Validity**"From that, it may predict that the next token is "is": "Tell me why validity is important in educational measurement in one sentence; Validity **is**"The autoregressive process starts again: the prompt is now "Tell me why validity is important in educational measurement in one sentence; Validity is" and the predicted next word is **important**.And so on until the sentence is completed.You may have noticed this in earlier versions of online conversational chatbotsβit seemed like the model was typing out the words as it responded. This was not just a parlor trick or fancy UI. It reflected the fact that the response was being generated incrementally, token by token.Conversational chats are similar. When you continue a conversation after an initial model response, the model uses the accumulated conversation history to predict the next token: your initial prompt, the model's first response, and your second prompt. This continues as the conversation progresses.If you are sending your fifth prompt to the model, the model is effectively evaluating the prior conversation historyβfour preceding prompt-response pairs, plus your current promptβwhen predicting the next token to produce.If you need to have a conversational interaction with a model, I generally prefer using the normal online chatbot interface. It has been designed to be efficient and useful, you can access past chat histories easily, and there are other features that make interaction easier (adding tools, uploading documents or images, etc.).However, there may be instances in which you want to have the conversation from the console, or you may only have access to a secure API that does not retain prompts or conversation history. In these instances, the [`ellmer` package](https://ellmer.tidyverse.org/){target="_blank"}, developed by Posit, offers an easy way to have conversational interactions with a variety of generative AI models.## Conversational Interactions in RWe'll continue to use our Anthropic API key, although `ellmer` supports interactions with a variety of model providers, including OpenAI, Google Gemini, DeepSeek, Mistral, Hugging Face, and Perplexity.What follows can _mostly_ be generalized to working with other models, with some slight differences that you can learn about by consulting the `ellmer` website linked above.## Quick StartThe easiest way to start a conversation is just to use the default settings for a model. For the purposes of demonstration in our workshop, there's no need to change it, but you can find all of those details at the `ellmer` website.You can see what Anthropic models are available with the following:```{r anthropic available, warning = FALSE}library(ellmer)models_anthropic()```The "input" column is the cost per million tokens of model input (the prompts).The "output" column is the cost per million tokens of the model response.For some context, Shakespeare's _Romeo and Juliet_ is about 25,000 words, which roughly translates to 40,000 tokens (depends on the [tokenization method](@sec-tokens) of the model).Output tokens are often more expensive than input tokens in part because generating a response is an autoregressive process: the model must repeatedly predict the next token, updating the context at each step, rather than producing the full response all at once.```{r ellmer quick}# Gets the API from the .Renviron file, if using this methodapi_key <-Sys.getenv("ANTHROPIC_API_KEY")# Prompts the User to enter their API key; flip for workshop# Sys.setenv(ANTHROPIC_API_KEY = rstudioapi::askForSecret("Anthropic API Key"))# You'll see a Claude Sonnet 4 is being used by default.# I've included a system prompt, which we'll discuss below.chat <-chat_anthropic(system_prompt ="You explain things concisely, focusing on only the most significant parts of the response.",model ="claude-sonnet-4-6")```There are two ways of using `ellmer` for conversational chats. ## `live_console`The first is creating a temporary `live_console`, which most resembles the functionality of the chatbot interface. This can be done two ways: within the R console, and by leveraging the `shinychat` package to have the interaction open in a browser (the viewer window) via `live_browser`.Once you run the `live_console(chat)` argument, you'll notice that the console below changes to have three angle brackets (`>>>`) instead of the usual single angle bracket (`>`).The following message also appears to help alert you that you've entered the console mode:```ββββββββββββββββββββββββββββββββββ Entering chat console. ββ Use """ for multi-line input. ββ Type 'Q' to quit. ββββββββββββββββββββββββββββββββββ```This interactivity is hard to show in a static document, so I have copied-and-pasted a short interaction below:### `live_console` Example```> live_console(chat)ββββββββββββββββββββββββββββββββββ Entering chat console. ββ Use """ for multi-line input. ββ Type 'Q' to quit. ββββββββββββββββββββββββββββββββββ>>> Generate short description of why validity is important in educational research. The audience is a lay person. The explanation should only be two sentences.Validity ensures that a test or study actually measures what it claims to measure, so the results are trustworthy and meaningful. Without validity, educators and policymakers could make important decisions about students and schools based on misleading or inaccurate information.>>> Generate a similar response about reliability.Reliability ensures that a test or measurement produces consistent results every time it is used, much like a scale that gives you the same weight each time you step on it. Without reliability, it would be impossible to know whether differences in student performance are due to actual learning or simply random errors in the measurement tool.>>> Generate another response about how these are important for fairness and equity. This should be no more than 5 sentences.When tests are both valid and reliable, all students have a fair opportunity to demonstrate their true knowledge and abilities, regardless oftheir background. If a test is invalid or unreliable, certain groups of students may be unfairly disadvantaged, leading to biased outcomes inareas like college admissions, graduation requirements, or special education placement. For example, a test that contains culturally biased language may not accurately measure a student's actual knowledge, but instead reflect their familiarity with a particular culture. Ensuring validity and reliability helps educators identify and close achievement gaps rather than create or reinforce them. Ultimately, valid and reliable assessments are essential tools for building an educational system that treats every student with fairness and equity.>>> Q```### `live_browser` demoAlternatively, by also using the `shiny` and `shinychat` packages, you have have the same interaction appear in your "Viewer" window of the R Studio screen.One you have provided your API key and set the initial `chat` details (as above), you the execute the following:```{r, eval = FALSE}live_browser(chat)``````ββββββββββββββββββββββββββββββ Entering interactive chat ββ Press Ctrl+C to quit. ββββββββββββββββββββββββββββββ```Here is a screenshot of part of an interaction:{fig-align="center"}## Interactive method callNow let's look at the second method for conversational chats, the interactive call method.Below I've prompted the model via `chat$chat("prompt")`, and then immediately used the same syntax again (with a different prompt).I've hidden the output because it's so long; you'll need to click to see the prompt and model response.::: {.callout-note collapse="true"}## Click to see first prompt and response```{r eq 1}chat$set_turns(list())chat$chat("Tell me about the history of the exploration of the moon")```:::::: {.callout-note collapse="true"}## Click to see follow-up prompt and response```{r eq 2}chat$chat("What are the most important non-USA exploration missions?")```:::As you can see, the second response from the model takes into context the first prompt - it's still talking about the moon!This conversational functionality is useful when you're doing iterative development or planning, as the previous calls to the model are important for providing content and building upon previous prompts and model responses.## `chat_anthropic` DetailsLet's look at the details of `chat_anthropic` (from [this page of the `ellmer` package reference.](https://ellmer.tidyverse.org/reference/chat_anthropic.html))```{r, eval = FALSE}chat_anthropic(system_prompt =NULL,params =NULL,model =NULL,cache =c("5m", "1h", "none"),api_args =list(),base_url ="https://api.anthropic.com/v1",beta_headers =character(),api_key =NULL,credentials =NULL,api_headers =character(),echo =NULL)```It's important to note the `params` argument, which allows you to set a variety of model parameters when chatting with the model.This is a [general argument](https://ellmer.tidyverse.org/reference/params.html) in the `ellmer` package. You'll need to ensure that your model input allows a specific generation parameter before including it in your call.This is one place where having a conversation with a model via API instead of via a chatbot interface is different - it's not always easy (and sometimes impossible) to change these parameters in the normal chat interface.```{r, eval = FALSE}params(temperature =NULL,top_p =NULL,top_k =NULL,frequency_penalty =NULL,presence_penalty =NULL,seed =NULL,max_tokens =NULL,log_probs =NULL,stop_sequences =NULL,reasoning_effort =NULL,reasoning_tokens =NULL, ...)```### Options for Clearing the ChatThere are two methods to reset the chat history. when you're using the interactive call method.This is useful when you want to start a conversation about another topic.#### Clearing While Maintaining Chat ConfigurationThe following syntax simply clears the turns but maintains the other aspects of the chat configuration (which we'll discuss momentarily). In the background the `ellmer` package is saving a history of your prompts and model responses, and it sending this history as part of the prompt when you send a new prompt.This is also happens when having a conversation with a chatbot, but it's even less obvious.```{r, eval = FALSE}chat$set_turns(list())```#### Clearing All Chat SettingsThis starts an entirely new chat with the Anthropic model, and removes any settings you've made (system prompt, parameters).You can also include this in your argument - the important part is that using `chat_anthropic()` again resets any previously-specified chat configuration.```{r, eval = FALSE}chat <-chat_anthoropic()```### Retaining the Chat HistoryAs you interact with a generative AI model through `ellmer`, a record of your prompts and model responses are stored in `chat$get_turns()`.When you examine this list object, each odd-numbered entry (starting with 1) are your prompts, and each even-numbered entry is the model response.::: {.callout-note collapse="true"}## Click to see an example of saving the chat history.```{r chat hist, eval = FALSE}chat <-chat_anthropic()chat$chat("Give me a 5-sentence history of educational measurement.")chat$chat("Give me a 5-sentence summary of educational measurement breakthroughs since 2000.")ed_meas_chat <- chat$get_turns()save(ed_meas_chat, file ="./data/ed_meas_chat.Rdata")``````{r view chat hist}load("data/ed_meas_chat.Rdata")ed_meas_chat[1]ed_meas_chat[2]ed_meas_chat[3]ed_meas_chat[4]```:::---## System PromptsWe briefly discussed system prompts in the [section about generative parameters ](08-gen-ai-parameters.qmd#sec-system-prompt).The system prompt is an instruction to the model that is maintained throughout all of your interactions with the model.I don't generally use system prompts when calling models via API, but I probably should.π Nonetheless, let's see how changing the system prompt can change the model output.First, with no setting of the system prompt:```{r no system prompt}chat <-chat_anthropic()chat$chat("Briefly tell me the point of using a Rasch model.")## rano <- as.character(rasch_normal)```---Now using a playful system prompt:```{r playful system prompt}chat <-chat_anthropic(system_prompt ="You are an assistant that likes to respond in rhymes.")chat$chat("Briefly tell me the point of using a Rasch model.")```---Some more helpful examples of good system prompts are:- **Specifying Output Structure** - "Always respond in JSON format with keys: 'answer', 'confidence', 'sources'. Never include any text outside the JSON object."- **Setting Constraints** - "You are a medical information assistant. Always: 1. Emphasize you're not a doctor 2. Recommend consulting healthcare professionals 3. Cite medical sources when possible 4. Never diagnose conditions"### Retaining the Chat History with the System PromptThe system prompt is also retained in the chat history, and can be accessed by specifying `chat$get_turns(include_system_prompt = TRUE)`.The list object now has the system prompt as the first object, meaning that your prompts are now every even-numbered object, and the system responses are every odd-numbered object, started at 3.::: {.callout-note collapse="true"}## Click to see an example of saving the chat history with the system prompt.```{r chat hist w system, eval = FALSE}chat <-chat_anthropic("You respond only with the 5 most important sentences about a topic.")chat$chat("Give me a summary of the history of educational measurement.")chat$chat("Give me a summary of educational measurement breakthroughs since 2000.")ed_meas_chat_wsp <- chat$get_turns(include_system_prompt =TRUE)save(ed_meas_chat_wsp, file ="./data/ed_meas_chat_wsp.Rdata")``````{r view chat hist w system}load("data/ed_meas_chat_wsp.Rdata")ed_meas_chat_wsp[1]ed_meas_chat_wsp[2]ed_meas_chat_wsp[3]ed_meas_chat_wsp[4]ed_meas_chat_wsp[5]```:::
