Local models with LM Studio

How to run LLMs locally on your own computer with LM Studio (or ollama).

LM Studio is a desktop app for running LLMs locally on your computer. Local models do not provide the same quality of responses as flagship models from AI providers like OpenAI or Anthropic, but you can run models on your own computer without having to pay for API access or sending your data to a third party.

You do not need to install LM Studio or use local models to participate in the workshop. That said, they’re a fun way to experiment with LLMs without incurring API costs.

Install LM Studio

Download and install LM Studio on your computer. Then, use the app to browse and download models. Here are a few that we like for local testing. For the workshop, we’d recommend [Bonsai 27B][lmstudio-bonsai]1.

ID Size Capabilities
[Bonsai 27B][lmstudio-bonsai] 5GB vision tools thinking
[qwen3.8][lmstudio-qwen38] 18GB vision tools thinking

Once you’ve downloaded a model, start LM Studio’s local server from the Developer tab. You can then use ellmer or chatlas to interact with the model from R or Python.

library(ellmer)

chat <- chat_lmstudio(model = "prism-ml/bonsai-27b")
chat$chat("What is the capital of France?")

The capital of France is Paris.

It’s a global center for art, fashion, gastronomy, and culture. 🌍🇫🇷

from chatlas import ChatLMStudio

chat = ChatLMStudio(model="prism-ml/bonsai-27b")
chat.chat("What is the capital of France?")

The capital of France is Paris.

It’s a global center for art, fashion, gastronomy, and culture. 😊

Do you want to know anything more about Paris?

Prefer ollama?

You can also run local models with ollama, which works from the terminal instead of a desktop app. For the workshop, we’d recommend gemma4:12b (7.4GB), which you can download with ollama pull gemma4:12b and use with chat_ollama() in R or ChatOllama in Python.

Why use a local model?

Local models are just like the famous ChatGPT or Claude models, but they run entirely on your own computer. You can use them for all of the same tasks: conversational AI, code generation, text processing, vision and more. There are a wide range of models available at many different sizes and capabilities, so you can choose one that fits your hardware and performance needs.

Local models are no where near as capable as the flagship models from OpenAI or Anthropic. Still, there are a few reasons you might want to use a local model:

  1. Your conversation never leaves your computer, ensuring complete data privacy.
  2. No API costs, making local models very low-cost (other than the cost of your laptop or the hardware to run the model).
  3. They work offline, so you can use them without an internet connection.

Local models are a great option for prototyping, experimentation, and personal use. They’re also useful to companies that want to use LLMs but have strict data privacy requirements and need to ensure that their data never leaves their network.

On the other hand, because you’re running the model on your own hardware, local models are slower and less powerful than cloud-based models, especially if you’re using a laptop or desktop computer. Personally, I use local models for experiments and testing to avoid API charges and switch to cloud-based models when I’m putting an LLM-powered application into production.

Footnotes

  1. Bonsai 27B is a 5GB download and runs comfortably on a typical laptop with 16GB of RAM.↩︎