Running Local LLMs with LM Studio: Benchmarks and Four Real Integrations

The pitch for local AI is easy to say and easy to overpromise: private, free, offline. The part nobody shows you is whether it's actually fast enough to use, and what it takes to wire a real tool to it. This article does both — the exact setup, real tokens-per-second measured on my machine, and four integrations you can finish today.

The whole thing rests on one fact: LM Studio's server is OpenAI-API-compatible. Learn to make one raw call to it, and every integration below is just that same call wearing a different UI.

Setup

LM Studio is a desktop app (Mac, Windows, Linux) that downloads open-weight models and serves them over HTTP. Four steps and you have a running endpoint:

1. Install from lmstudio.ai and launch it.
2. Search for a model and download it. Any recent small instruct model is a fine start — a current Llama, Qwen, or Phi release in the 3B–8B range runs on modest hardware.
3. Open the Developer tab (the <-> icon), pick the model in the top dropdown.
4. Enable the CORS toggle (browser extensions need it), then Start Server.

You now have an OpenAI-compatible endpoint at http://localhost:1234/v1. Base URL is that; the API key is any non-empty string, because the local server never checks it.

The one call everything is built on

Before any integration, prove the endpoint works with a raw call. This is the entire mechanism — every tool later is a wrapper around it.

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "local-model",
    "messages": [{"role": "user", "content": "Say hi in five words."}]
  }'

The same thing from Python, using the official OpenAI client — note that only the base URL changes from a normal OpenAI script:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")

resp = client.chat.completions.create(
    model="local-model",              # LM Studio routes to the loaded model
    messages=[{"role": "user", "content": "Say hi in five words."}],
)
print(resp.choices[0].message.content)

If that returns text, you're done with the hard part. Every integration below is this call with a different front end bolted on.

What your machine actually does

"Runs locally" is not the same as "usable." Two numbers decide that: time to first token (how long you stare at a blank screen before text appears) and generation throughput in tokens/second (how fast it streams once it starts). Cloud GPT-class models feel instant because both are strong. A local model on a laptop CPU or a small GPU can be perfectly good — or painfully slow — and the only way to know is to measure your hardware.

So I wrote a small benchmark that hits the local endpoint, streams a fixed prompt, and times both. It's OpenAI-compatible, so it works against LM Studio, Ollama, or llama.cpp unchanged:

# bench.py — full script at the repo link below
import sys, time
from openai import OpenAI

client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")

def one_run(model):
    start = time.perf_counter()
    first_at, n = None, 0
    stream = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "Explain quantization in one paragraph."}],
        max_tokens=256, temperature=0, stream=True,
    )
    for chunk in stream:
        delta = chunk.choices[0].delta.content if chunk.choices else None
        if not delta:
            continue
        if first_at is None:
            first_at = time.perf_counter()   # time-to-first-token
        n += 1
    end = time.perf_counter()
    return (first_at - start), n / (end - first_at)   # ttft_s, tokens_per_sec

Here is what it measured on my machine (‹RUN: hardware — e.g. Apple M-series, 16 GB›), averaged over 3 runs after a warm-up, for the models I had loaded:

The trade-off is the whole story: the small models stream fast enough to feel live but reason less; the bigger ones think better but drop below reading speed on a laptop. First-token latency told the same story — ‹RUN: fill TTFT numbers, e.g. "sub-300ms on the 3B, ~1.2s on the 14B"›. Pick per task: a fast small model for autocomplete and summaries, a slower large one for the occasional hard question.

Four integrations worth setting up

Skip the shallow "point the base URL here" list. These four are the ones that actually change how you work, and each is a complete, working config.

AnythingLLM — private RAG over your own documents

AnythingLLM turns a folder of PDFs, notes, and docs into a chat interface with citations, and it can run the whole pipeline locally. In its settings, choose LM Studio as the LLM provider and set the base URL to http://localhost:1234/v1. Point the embedding provider at a local embedding model too, and no part of your documents ever leaves the machine. This is the integration that justifies local AI on its own: you can finally ask questions of medical records, legal contracts, or internal company docs without shipping them to a third party.

Continue.dev — local autocomplete in place of Copilot

Continue is a VS Code / JetBrains extension that does inline completion and chat, backed by whatever model you give it. Recent versions use a config.yaml; add the local server as a model:

# ~/.continue/config.yaml
models:
  - name: Local (LM Studio)
    provider: openai
    apiBase: http://localhost:1234/v1
    apiKey: lm-studio
    model: local-model
    roles:
      - chat
      - autocomplete

Use a small, fast model here — autocomplete is where tokens/sec matters most, and a laggy model is worse than none. Your code never gets sent to a vendor.

n8n — always-on local automation

n8n is a self-hosted automation platform. Wire the local model into any workflow with an HTTP Request node. The one thing that trips everyone: n8n usually runs in Docker, so it cannot reach localhost — use host.docker.internal:

{
  "method": "POST",
  "url": "http://host.docker.internal:1234/v1/chat/completions",
  "sendBody": true,
  "bodyContentType": "json",
  "jsonBody": {
    "model": "local-model",
    "messages": [{"role": "user", "content": "Summarize: {{ $json.text }}" }]
  }
}

Now an RSS post or incoming email can be summarized, classified, and filed to Notion or Slack, 24/7, with the reasoning happening entirely inside your network.

Fabric — the local server as a CLI power tool

Fabric is a CLI with 200+ curated prompt "patterns" (extract_wisdom, summarize, analyze_claims, and more). Run fabric --setup, choose the OpenAI-compatible option, and give it the local base URL. Then pipe anything through a pattern:

pbpaste | fabric --pattern extract_wisdom
yt "https://youtu.be/..." | fabric --pattern summarize

Every pattern that would normally cost API tokens now runs against your local model for nothing.

The other tools people list — Obsidian Copilot, Raycast AI, HARPA, Open WebUI, Whisper, LibreChat — all work the exact same way: find the "custom OpenAI base URL" field and paste http://localhost:1234/v1. Once you've done the four above, the rest are five-minute repeats.

When local loses to cloud

Local AI is not a strict upgrade, and pretending otherwise is how you end up disappointed. Be honest about the ceiling:

Where to start

You don't need all four on day one. The fastest path to feeling the value:

The one thing to remember

The whole ecosystem reduces to a single fact: LM Studio speaks the OpenAI API at localhost:1234/v1, so anything with a custom-base-URL field is already a local-AI client. What decides whether it's usable is speed — tokens/sec and time-to-first-token on your hardware — which is why the benchmark isn't optional. Measure first, then put the fast model on autocomplete and save the slow, smart one for the hard questions.

Benchmark script and integration configs: ‹RUN: repo link, e.g. github.com/MohamedHamedLab/lm-studio-local-ai›