Using Warp’s BYOK with Local LLMs: What Actually Works

If you found Warp’s Bring Your Own API Key (BYOK) page and assumed it was your route to running a local model through Warp’s agents, there’s a catch worth knowing before you start: BYOK is for cloud providers only. It lets you plug in your own Anthropic, OpenAI, or Google keys so inference is billed to your provider account instead of consuming Warp credits. There’s no slot for a local model.

The feature you actually want for a local LLM is Warp’s Custom inference endpoint. It’s a sibling feature with a near-identical setup flow, and it’s what makes Ollama, LM Studio, vLLM, or llama.cpp usable inside Warp. This post covers the distinction, the one architectural gotcha that trips people up, and the actual steps.

BYOK vs. custom inference endpoint

Warp gives you three ways to bring your own inference. Two are relevant here:

• BYOK: Use your own API key for OpenAI, Anthropic, or Google models. Keys live only on your device. There’s no way to specify an arbitrary endpoint, so local models are out.
• Custom inference endpoint: Point Warp at any OpenAI-compatible endpoint: a router like OpenRouter or LiteLLM, a provider like z.ai, an internal gateway, or a local model server exposed at a public URL.

The third, Bring Your Own LLM (BYOLLM), is Enterprise-only managed inference through a cloud provider and isn’t relevant for a local setup.

Since nearly every local runner (Ollama, LM Studio, vLLM, llama.cpp’s server) already speaks the OpenAI Chat Completions API, the custom inference endpoint is the right door.

The gotcha: Warp’s harness runs server-side

Here’s the important part: Warp’s agent harness (the thing that assembles your prompt, system instructions, conversation context, and tool calls) runs on Warp’s backend servers, not on your machine. BYOK and custom endpoints only swap which credential and destination the backend uses; they don’t move the harness onto your laptop.

The practical consequence: localhost won’t work. When you send a prompt, your endpoint URL and key travel from your device up to Warp’s servers, and Warp’s backend calls your endpoint from there. So a model listening on 127.0.0.1:11434 is unreachable. Warp explicitly rejects localhost, 127.0.0.1, and other private network addresses.

To use a local model, you have to expose it at a public HTTPS URL first, typically with a tunnel.

Step-by-step: local LLM with Ollama + a tunnel

This uses Ollama as the example. LM Studio, vLLM, and llama.cpp all expose the same OpenAI Chat Completions surface. The only differences are the port and how you name models, noted at each step.

1. Serve a model with an OpenAI-compatible API

A default Ollama install listens on 11434 and exposes the OpenAI-compatible surface at /v1:

# Pull a tool-capable coding model (tool calling matters; see the caveat below)
ollama pull qwen3-coder:30b

# Ollama's background service usually already runs; if not:
ollama serve

Confirm the OpenAI surface responds before going any further. The two endpoints Warp cares about are /v1/models and /v1/chat/completions:

# List models in OpenAI format
curl http://localhost:11434/v1/models

# A real chat completion; exercises the path Warp will hit
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-coder:30b",
    "messages": [{"role": "user", "content": "Say hello in one word."}]
  }'

If both return JSON, your endpoint is OpenAI-compatible and the rest is just plumbing. For the other runners: LM Studio serves on 1234 (start the server from its Developer tab; base path /v1), vLLM on 8000 via vllm serve <model>, and llama.cpp on 8080 via llama-server. Swap the port in every command below accordingly.

2. Expose it at a public HTTPS URL

Warp’s backend has to reach your machine, so the endpoint must be public HTTPS. Two solid options:

ngrok (quickest, what Warp’s docs use):

ngrok http 11434

ngrok prints a forwarding line like https://abc123.ngrok-free.app -> http://localhost:11434. Your base URL for Warp is that host plus /v1:

https://abc123.ngrok-free.app/v1

One Ollama-specific wrinkle: Ollama validates the Host header, so a raw tunnel can come back with 403 Forbidden. Rewrite the host on the way in:

ngrok http 11434 --host-header="localhost:11434"

Cloudflare Tunnel (stable URL, no random subdomain churn, free):

cloudflared tunnel --url http://localhost:11434

This prints a https://<random>.trycloudflare.com URL for quick use; for a permanent hostname, set up a named tunnel bound to your domain. Tailscale Funnel works similarly if you’d rather keep it inside your tailnet’s public surface.

3. Verify the public URL works

Before touching Warp, confirm the tunnel actually forwards to the model. Re-run the chat-completion curl against the public URL. This is exactly the request Warp’s backend will make:

curl https://abc123.ngrok-free.app/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-coder:30b",
    "messages": [{"role": "user", "content": "ping"}]
  }'

If this succeeds and the local curl in step 1 did too, any later failure inside Warp is a config/auth issue, not a connectivity one, which is a useful thing to have isolated up front.

4. Configure the endpoint in Warp

Open “Settings” and search for inference endpoint (it has no dedicated sidebar page yet, so search is the fastest route). Then:

1. Endpoint URL: the base URL that exposes /v1/chat/completions, i.e. your tunnel URL including /v1 (https://abc123.ngrok-free.app/v1). Don’t append /chat/completions yourself; Warp builds the full path.
2. Credentials: Ollama needs no real key, but Warp’s form may require something; a dummy like ollama is fine. If you added basic auth or a proxy key in step 2, put the real value here instead.
3. Model identifier(s): these must match exactly what your server reports in /v1/models (e.g. qwen3-coder:30b). A mismatched tag is the most common reason a request 404s after everything else looks right.
4. Save: Your custom models now appear in the model picker.

5. Select the model explicitly and test in-agent

Pick your endpoint-routed model from Warp’s model picker, and do not leave it on Auto (Auto always routes through Warp’s own infrastructure and burns credits regardless). Then run a small agent task, like “list the files in this directory and summarize the largest one,” and watch the tunnel logs: you should see POST /v1/chat/completions hits arrive in real time. That confirms the full round trip: Warp backend, to tunnel, to local model, and back.

Billing and the “Auto” trap

Two things worth internalizing:

• Auto always burns Warp credits. Warp’s Auto model routing depends on Warp’s own infrastructure, so it consumes credits regardless of your BYOK or endpoint config. You have to select your specific endpoint-routed model from the picker to actually use your local LLM.
• Inference itself is free of Warp credits when you use your endpoint. With a local model that means it’s genuinely free; you’re just paying in electricity and VRAM. (On Business and Enterprise plans, local agent runs still consume platform credits for run lifecycle and observability, separate from inference.)

A few other Warp features (Codebase Context, cloud agent runs) keep using Warp’s infrastructure no matter what you configure, since they don’t run through your endpoint.

Is it worth it?

Routing a local model through Warp is a real option, but be honest about the tradeoff. Your prompts still leave your machine: up to Warp’s backend, then back down through a public tunnel to your machine. That way, you get Warp’s polished agent harness and tool use driving a model you control and don’t pay per-token for.

If what you want is Warp’s agent experience without spending credits, and you’ve got a capable local model and a tunnel, it works well. If your goal is air-gapped, nothing-leaves-the-laptop inference, a fully local agent tool is the better fit.

Docs referenced: Warp’s Bring Your Own API Key and Custom inference endpoint pages.