Anatomy of a conversation

Programming with LLMs in R and Python

posit::conf(2026)

2026-09-14

How to think about LLMs

Let’s talk a bit about how to think at a high level about LLMs before we dive into some code

Think Empirically, Not Theoretically

Think Empirically, Not Theoretically

• It’s okay to (mostly) treat LLMs as black boxes.
  We’re not going to focus on how they work internally

• Just try it! When wondering if an LLM can do something,
  experiment rather than theorize

• You might think they could not possibly do things
  that they clearly can do today

• And you might think surely they can do something
  that it turns out they’re terrible at

Think empirically, not theoretically. There are some things LLMs can clearly do today that you would have thought possible, and things That they can’t do that you would have thought they would have been able to.

You might expect a performance to look something like this.

LLMs are jagged

It really looks something more like this.

Embrace the Experimental Process

Embrace the experimental process.

Start Simple, Build Understanding

Start Simple, Build Understanding

• We’re going to focus on the core building blocks.

• All the incredible things you see AI do
  decompose to just a few key ingredients.

We’re gonna focus on the core building blocks. The goal for today is to really for you to sense those building blocks, feel like you’re exploring, have some fun, build intuition.

Anatomy of a Conversation

First, let’s talk about what happens when we talk to an LLM.

Those conversations often look something like this:

You ask a question and ChatGPT responds. You can keep talking to ChatGPT and it keeps responding, almost like you’re having a conversation with a person.

We can do the same thing from R and Python.

We can do this from R and Python!

R

Python

We can build the same back-and-forth interaction in code. We’ll use ellmer in R and chatlas in Python.

A complete example

ellmer

library(ellmer)

chat <- chat_posit()

chat$chat("Tell me a fact about cats.")

#> **Cats can't taste sweetness.**
#> They lack the specific taste receptor gene
#> (Tas1r2) that allows most mammals to
#> detect sugary flavors.

You just saw a conversation in a chat UI. This is the same basic interaction from R.

At a high level, this loads a package, creates a chat client, and sends it a message.

A complete example

chatlas

import chatlas

chat = chatlas.ChatPosit()

chat.chat("Tell me a fact about cats.")

#> **Cats can't taste sweetness.**
#> They lack the specific taste receptor gene
#> (Tas1r2) that allows most mammals to
#> detect sugary flavors.

The same three steps work in Python: import a package, create a chat client, and send it a message.

How does this work?

Most LLMs are accessible through HTTP APIs

This is pretty much a normal API. If you have queried an API to get data, this is fundamentally the same thing.

so how does this work? Talking to an LLM like ChatGPT happens through HTTP requests and responses. When you send a message, the chat client sends a POST request. The server runs the model, then returns the response.

ellmer

library(ellmer)

chatlas

import chatlas

First, load the package for your language, just as you would any other package.

ellmer

library(ellmer)

chat <- chat_posit()

chatlas

import chatlas

chat = chatlas.ChatPosit()

Next, create a chat client. This is the object we’ll use to send messages and receive responses.

ellmer

library(ellmer)

chat <- chat_posit()

chat$chat("Tell me a fact about cats.")

chatlas

import chatlas

chat = chatlas.ChatPosit()

chat.chat("Tell me a fact about cats.")

Finally, call the chat method and pass the message as a string.

ellmer

chat

<Chat Posit turns=2>
── user ─────────────────────────────────────────────
Tell me a fact about cats.
── assistant ────────────────────────────────────────
Cats have a special reflective layer behind their retinas that
helps them see in very low light.

chatlas

print(chat)

## User turn:

Tell me a fact about cats.

## Assistant turn:

Cats have a special reflective layer behind their retinas that
helps them see in very low light.

Messages have roles.

The chat objects contain the conversation history. What do you notice?

Each message has a role. The user supplies a prompt, and the assistant supplies the model’s response.

Messages have roles

When you send a message to an LLM, you send both the text and a role.

The user role marks a message from the person using the application.

Messages have roles

The assistant role marks the model’s response.

Messages have roles

A conversation alternates between user and assistant messages. The role tells the model who contributed each part of the history.

Messages have roles

The system role contains instructions from the developer. It can set behavior, style, and constraints for the whole conversation. The person using a chat interface usually does not see it.

Message roles

Role	Description
system	Instructions that set the behavior of the assistant
user	Messages from the person interacting with the assistant
assistant	The AI model’s responses to the user

To recap, user and assistant roles identify the two sides of the exchange. The system role gives the model instructions that persist across the conversation.

System prompts

ellmer

library(ellmer)

chat <- chat_posit(
  system_prompt = "Answer in haikus."
)

chat$chat("Tell me a fact about sheep.")

chatlas

import chatlas

chat = chatlas.ChatPosit(
    system_prompt="Answer in haikus."
)

chat.chat("Tell me a fact about sheep.")

Both packages accept a system_prompt argument when you create the chat client. Here we’re using a silly instruction so its effect will be obvious.

System prompts

ellmer

library(ellmer)

chat <- chat_posit(
  # Set the response style
  system_prompt = "Answer in haikus."
)

chat$chat("Tell me a fact about sheep.")
#> Woolly white cloud grows
#> Sheep never sleep flat on ground
#> Fear keeps them standing.

chatlas

import chatlas

chat = chatlas.ChatPosit(
    # Set the response style
    system_prompt="Answer in haikus."
)

chat.chat("Tell me a fact about sheep.")
#> Woolly white cloud grows
#> Sheep never sleep flat on ground
#> Fear keeps them standing.

The model follows the instruction and answers in a haiku. The system prompt shapes every response produced by this chat client.

The chat object

ellmer

chat

<Chat Posit turns=3>
── system ─────────────────────────
Answer in haikus.
── user ───────────────────────────
Tell me a fact about sheep.
── assistant ──────────────────────
Woolly white cloud grows
Sheep never sleep flat on ground
Fear keeps them standing.

chatlas

print(chat)

## System turn:
Answer in haikus.

## User turn:
Tell me a fact about sheep.

## Assistant turn:
Woolly white cloud grows
Sheep never sleep flat on ground
Fear keeps them standing.

When we inspect the chat object again, we can see all three roles. The system prompt is part of the history that guides the assistant’s response.

Your Turn 02_conversation

1. Set up a chat with this system prompt:

   Answer in as few words as possible.

2. Ask: What creates an Anthropic chat in ellmer and chatlas?

3. Ask: What about OpenAI?

4. Then, create a new chat with no system prompt and ask only the second question: What about OpenAI?

5. How do the answers to 3 and 4 differ? Why?

There are two takeaways to discuss afterward. The system prompt persists across calls on the first chat, so its answers stay brief. A new chat has no memory of the first question and no system prompt, so “What about OpenAI?” has less context and may produce a longer or less relevant answer.

Demo: clearbot

👨‍💻 _demos/03_clearbot/app.py

System prompt:

Answer in as few words as possible.

First question:

What creates an Anthropic chat in ellmer and chatlas?

Second question:

What about OpenAI?

Use clearbot to show the full conversation history sent with each request. The demo makes the next point visible: the model itself does not retain the earlier exchange.

How to talk to robots

Those conversations often look something like this:

You ask a question and ChatGPT responds. You can keep talking to ChatGPT and it keeps responding, almost like you’re having a conversation with a person.

Is this actually a conversation?

It feels like a conversation, but it works differently from a conversation between people. Clearbot showed what happens behind the scenes.

LLMs are stateless

• The LLM doesn’t remember anything between requests

• For each response, the LLM receives the conversation history again

• The LLM reconstructs the “conversation” from that history

The LLM has amnesia - it forgets everything after each response. Each generation starts from the context that the LLM receives.

To continue the conversation, the chat system gives the LLM the conversation history again. That’s what clearbot showed you - the full history becomes context for each new response.

ChatGPT

Chatting with a Generative Pre-trained Transformer

LLM → Large Language Model

GPT is a type of LLM, but not all LLMs are GPTs.

LLM: Deep learning on huge amounts of text data to understand and generate human language. GPT: Specific architectural design of LLMs.

From now on, we’ll use LLM to refer to any large language model.

How to make an LLM

LLMs learn patterns from huge amounts of text. That training lets them use context and generate language that resembles the material they learned from.

If you read everything
ever written…

• Books and stories

• Websites and articles

• Poems and jokes

• Questions and answers

…then you could…

• Answer questions
• Write stories
• Tell jokes
• Explain things
• Translate between languages

The cat sat in the ____

• 🎩
• 🛌
• 📦

• 🪟
• 🛒
• 👠

The model builds a response one token at a time, choosing a likely next token from the context it has received.

The breakthrough insight is the attention mechanism in the Transformer architecture. Instead of processing the input one token at a time, attention lets GPT models relate all input tokens in parallel.

The attention mechanism allows the model to:

• Weigh the importance of every word in relation to every other word
• Capture long-range dependencies that traditional models missed
• Process input tokens in parallel

The model still generates new tokens one at a time.

Actually: tokens, not words

• Fundamental units of information for LLMs
• Words, parts of words, or individual characters
  • “hello” → 1 token
  • “unconventional” → 3 tokens: un|con|ventional
• Important for:
  • Model input/output limits
  • API pricing is usually by token
• Not just words, but images can be tokenized too

Tokens are the units the model reads and generates. A token may be a whole word, part of a word, or a character. Token counts matter because models limit how many they can process, and APIs commonly use them to measure usage.

Demo:
token-possibilities

👨‍💻 _demos/04_token-possibilities/app.R

This demo shows a response being built token by token. Pause to look at the possible next tokens and their probabilities. The model repeats this choice until the response is complete.

Programming is fun, but I kind of like ChatGPT…

Transition from calling the model in code to using the same chat client through an interactive interface.

ellmer and chatlas can do that, too!

	Console	Browser
	live_console(chat)	live_browser(chat)
	chat.console()	chat.app()

As we’ll see, everything we’re going to do today can be done in either R or Python and ellmer and chatlas are designed to be similar.

But they’re in different languages and those languages have different conventions and idioms, so they’re not completely identical. If they were, one or the other would start to feel unnatural to use.

Your Turn 05_live

1. Your job: write a groan-worthy roast of Hadley Wickham

2. Bonus points for puns, rhymes, and one-liners

3. Don’t be mean

4. Share your best with a neighbor.