Enodios wires Hermes Agent to vLLM on your machine — uncensored models, full tool access, zero cloud inference. No refusals from a provider you don't control.
Cloud models come with alignment layers, usage policies, and refusals baked in. Enodios is the opposite: a local inference stack where you decide what runs, what tools the agent gets, and which requests cross the boundary of your machine.
Weights, KV cache, and inference stay on your GPU. No data leaves your box unless Hermes tools explicitly do it.
Default model is Nous Hermes 3 — tuned for agent work without corporate alignment blocking your prompts.
Native tool calling via vLLM's Hermes parser. ~3× faster than Ollama for the same model class on a 4090.
Hermes permissions, sandboxing, and model choice are yours. Enodios doesn't phone home or enforce policy.
One OpenAI-compatible endpoint. Hermes handles orchestration; vLLM handles tokens.
Binds 127.0.0.1 by default — use start --lan only when a remote Hermes agent needs access.
Not every open model is suitable. Hermes needs native tool calling and enough context for long agent loops. These models prioritize minimal alignment — abliterated or Nous-tuned — over corporate safety layers.
The Enodios default. Uncensored, fast tool calls, fits 24GB at 64k context.
BF16 weights — higher quality, heavier VRAM. Best on a clean 24GB+ GPU.
Abliterated 3.2B — safety layers stripped. Fastest tool calls, smallest footprint.
Larger abliterated Qwen — strong reasoning, slower. Tool output can be inconsistent.
Aligned cloud-tuned models — nemotron-3-nano, qwen3.6 (stock), gemma —
have strong agent skills but corporate safety refusals. Fine for compliance workloads; wrong fit when you want
the model to follow your instructions without a policy layer in the middle.
Run enodios recommend to detect your GPU and get tuned ENODIOS_* exports.
Swap models via ENODIOS_MODEL — any vLLM-compatible weights with Hermes tool support.
One machine runs vLLM on the GPU; another runs Hermes with tools — Hermes controlling Hermes over your LAN.
enodios start -b --lanenodios urlsenodios configure --url http://<gpu-host>:8000/v1hermes chat
No API auth. --lan exposes an unauthenticated OpenAI-compatible API — trusted local network only.
One Hermes orchestrator plans work and delegates to many client Hermes agents — each client runs scoped sub-tasks with its own tools while inference flows through Enodios vLLM on your GPU hosts.
enodios start -b --lan per inference box
Each client machine: enodios configure --url http://<gpu-host>:8000/v1 then hermes chat.
Full setup →
curl -fsSL https://raw.githubusercontent.com/DataKnifeAI/enodios/main/install.sh | bash enodios install
enodios recommend enodios recommend --apply source ~/.local/share/enodios/recommended.env
enodios start -b enodios status enodios bench
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
hermes setup # → Full setup → Inference Provider → Custom endpoint (enter URL manually) # Base URL: http://127.0.0.1:8000/v1 # API key: (empty) model: hermes3:8b context: 65536 # Or provider only: hermes setup model hermes config show hermes chat
enodios stop # or: enodios pause — releases ~20GB VRAM # game, render, or other GPU work enodios start -b # resume when done
Prefer automatic config? enodios configure writes the same endpoint to ~/.hermes/config.yaml.
vLLM holds most of your VRAM while running — stop it before other GPU-heavy apps.
Install Enodios and start vLLM first, then install Hermes and run hermes setup →
Full setup → Inference Provider → Custom endpoint (enter URL manually).
Use hermes setup model or hermes model to change provider later — not /model
inside chat (that only switches existing providers).
| Prompt | Enodios value |
|---|---|
| Provider | Custom endpoint (enter URL manually) |
| Base URL | http://127.0.0.1:8000/v1 |
| API key | Leave empty (no auth on loopback) |
| Model name | hermes3:8b — served name, not the HuggingFace path |
| Context length | 65536 — Hermes agent minimum |
| Provider name | enodios (optional named provider) |
enodios start -b # vLLM must be running first enodios status # confirm hermes3:8b hermes setup # Full setup → Inference Provider → Custom endpoint # or: hermes setup model hermes config show hermes chat
Remote GPU host: run enodios start -b --lan on the inference box,
then hermes setup model on the controller with the lan: URL from enodios urls,
or enodios configure --url http://<gpu-host>:8000/v1.
enodios start --lan detects UFW/firewalld and prompts to allow TCP 8000 — or enodios firewall --allowtail -f ~/.local/share/enodios/vllm.logenodios bench first; model must be hermes3:8b with context 65536/model custom:enodios:hermes3:8b after wizard setupnvidia-smi working| Default | Value |
|---|---|
| Model | solidrust/Hermes-3-Llama-3.1-8B-AWQ |
| API name | hermes3:8b |
| Bind | 127.0.0.1 (loopback; start --lan → 0.0.0.0) |
| Port | 8000 |
| Context | 65536 (Hermes agent minimum) |
| KV cache | fp8 |
| Tool parser | hermes |
| Command | Description |
|---|---|
enodios deps | Check / install OS GPU packages (curl, git, NVIDIA driver) |
enodios install | Deps check + uv + vLLM venv + CLI link |
enodios update | git pull + upgrade vLLM + refresh CLI |
enodios recommend | GPU detect → model/settings for Hermes |
enodios recommend --apply | Write recommended.env |
enodios start | Foreground vLLM on loopback |
enodios start -b | Background; log + PID file |
enodios start --lan | Expose vLLM on LAN; prompts firewall allow |
enodios firewall | Check / allow TCP port for remote Hermes |
enodios urls | Print local + LAN API URLs |
enodios stop / pause | Stop vLLM — free GPU for games and other tasks |
enodios doctor | GPU / CUDA / health checks |
enodios bench | Tool-call smoke test |
enodios configure | Wire Hermes to local vLLM |
enodios configure --url URL | Wire Hermes to remote vLLM |
enodios status | URLs + /v1/models |
enodios stop before launching — default 64k uses ~20GB VRAM. Resume with enodios start -b.enodios stop first, close other GPU apps, or export ENODIOS_GPU_UTIL=0.65 — run enodios recommend.export ENODIOS_PORT=8001 then enodios configure.enodios start -b --lan (prompts firewall) + enodios urls; controller: configure --url http://<gpu-host>:8000/v1tail -f ~/.local/share/enodios/vllm.logCUDA_ERROR_UNKNOWN in containers.