Skip to content
Nx IDEDocs

AI Providers & Models3 min read

Ollama (Local)

Run local models fully offline

Ollama runs open-weight models directly on your machine. Paired with Nexgile Code it gives you a fully offline coding agent: no API key, no per-token bill, and no source code leaving your computer — a real option for air-gapped environments and proprietary repositories.

Install Ollama and pull a model

Section titled “Install Ollama and pull a model”
  1. Install Ollama from ollama.com (macOS, Windows, Linux).
  2. Pull a model built for coding and tool use. A solid starting point:
Terminal window
ollama pull devstral:24b
  1. Make sure the Ollama service is running (the desktop app runs it automatically; otherwise ollama serve). By default it listens on http://localhost:11434.

Pick the largest model your RAM/VRAM can hold — agent work punishes small models. Model size, quantization, and your hardware determine both quality and speed.

Configure in Nx IDE

Section titled “Configure in Nx IDE”
  1. Open Settings → Providers and choose Ollama.
  2. Base URL (optional) — leave empty for the default http://localhost:11434; set it if Ollama runs on another port or a different machine on your network.
  3. Ollama API Key — appears only when a base URL is set; use it for authenticated remote or cloud-hosted Ollama instances. Leave empty for local installs.
  4. Model ID — the picker lists the models actually installed in your Ollama instance (it refreshes automatically when you open the panel). If the panel warns that your selected model isn’t available, pull it first.
  5. Context Window Size (num_ctx) — optional override of the model’s default context length. Leave empty to use the model’s own Modelfile configuration; minimum value is 128.

If the service isn’t running you’ll get an explicit error naming the URL it tried — start Ollama and retry.

Context windows matter for agent use

Section titled “Context windows matter for agent use”

This is the number-one local-model gotcha. Nexgile Code sends a substantial system prompt plus file contents, tool results, and conversation history on every request. Many Ollama models default to a small context window, and when the window is too small the prompt gets truncated — the agent then “forgets” its instructions, loses tool syntax, or loops.

  • Set num_ctx high enough for real work (tens of thousands of tokens if your hardware allows).
  • Bigger context costs RAM/VRAM and speed; find the balance your machine sustains.
  • Keep tasks tightly scoped and let context management condense long sessions.

Privacy: fully offline

Section titled “Privacy: fully offline”

With a local Ollama endpoint, prompts, code, diffs, and terminal output never leave your machine. There is no vendor to trust because there is no vendor in the loop. This also composes with private semantic search: codebase indexing supports Ollama as a local embeddings provider alongside a local Qdrant container, keeping the entire retrieval stack offline.

Realistic expectations

Section titled “Realistic expectations”

Nexgile Code’s own settings panel says it plainly: the product uses complex, structured prompts and works best with highly capable models; less capable models may not work as expected. In practice:

  • Local open-weight models trail frontier hosted models at multi-step agentic work — expect more malformed edits, more retries, and weaker long-horizon planning.
  • They shine at focused, well-scoped tasks: explain this file, write a unit test, draft a docstring, small mechanical edits.
  • Approval prompts and checkpoints are your safety net; keep auto-approve conservative with local models.

A common pattern is a two-profile setup: a local Ollama profile for private or bulk work, and a hosted frontier profile for the hard problems — switching is one click. See Providers Overview.

Troubleshooting

Section titled “Troubleshooting”
  • “Ollama service is not running at http://localhost:11434”: exactly what it says — start the Ollama app or ollama serve, then retry. If Ollama listens elsewhere, fix the Base URL.
  • Model picker is empty or your model is flagged unavailable: the picker only lists models Ollama reports as installed. Run ollama list to confirm, ollama pull <model> to fetch, then reopen the panel.
  • Agent loses the plot mid-task (forgets instructions, malformed tool calls): classic small-context symptom — raise num_ctx or pick a model with a larger window.
  • Timeouts on long generations: raise nexgile-code.apiRequestTimeout; local hardware is the bottleneck, not the network.
  • Everything is just slow: try a smaller quantization, a smaller model, or GPU offload in Ollama’s configuration; agent workloads multiply any per-request slowness.
Section titled “Related”