/guides/Recent content in Guides on Modelplane DocsHugo -- gohugo.ioen-usMon, 01 Jan 0001 00:00:00 +0000/guides/anthropic-messages-api//guides/anthropic-messages-api/<!-- vale write-good.Passive = NO --> <p>A vLLM server registers the Anthropic Messages API at <code>/v1/messages</code> alongside its OpenAI routes, with no extra flag. Modelplane&rsquo;s route matches the <code>/&lt;namespace&gt;/&lt;service&gt;/</code> prefix and preserves the path below it, so the same service URL answers both <code>/v1/chat/completions</code> and <code>/v1/messages</code>. A client that speaks the Messages API, including Claude Code via <code>ANTHROPIC_BASE_URL</code>, talks to the deployment directly. See <a href="/models/model-service/">Alternate APIs</a> for the routing detail.</p> <p>This recipe serves Qwen3-8B on a single NVIDIA H100 on Nebius, with tool calling on: <code>--enable-auto-tool-choice</code> and <code>--tool-call-parser=hermes</code> are what let Claude Code&rsquo;s tool use work. An 8B model needs a fraction of an H100, so the GPU has ample headroom. Apply the platform side first, then the ML side.</p>/guides/collecting-engine-metrics//guides/collecting-engine-metrics/<!-- vale write-good.Passive = NO --> <p>Scraping an inference engine&rsquo;s Prometheus metrics, shown on the smallest serving shape: a 0.5B Qwen chat model on one NVIDIA L4. vLLM publishes metrics at <code>/metrics</code> on its serving port with no extra flag, and Modelplane runs a Prometheus on every workload cluster with <code>PodMonitor</code> discovery open across namespaces, so scraping the engine is a <code>PodMonitor</code> plus a <code>port-forward</code>. The model is only the subject; the same wiring fits any engine, with the SGLang, leader/worker, and prefill/decode differences noted at the end.</p>