Evals and Prompt Engineering
Getting LLMs to do what you want through effective prompting.
Slides
Outline
- (15m) Evals
- Datasets, solvers, and scorers
- Evals with vitals in R and Inspect with chatlas in Python
- Scorer options and the bluffbench example
- Your Turn
_exercises/15_evals: compare model behavior
- (35m) Prompt engineering
- Your Turn
_exercises/12_plot-image-1: ask an LLM to interpret a scatter plot
- Your Turn