Self-Hosting 1M-Context Open Models in 2026: Hardware, Engines & Deployment Guide (DeepSeek, GLM-5.2, Kimi K3, MiniMax M3 & More)

By AiCybr Editorial · Published August 6, 2026

The central fact: Every model in this guide advertises a 1M token context window. Actually serving 1M tokens requires VRAM that scales far beyond what fits the model weights alone — and the weights alone are already substantial. "The weights fit" does not mean "the advertised context fits."

Navigate This Guide

• Who should self-host (and who should not)
• The three meanings of "supports 1M context"
• Understanding the VRAM math
• Hardware tiers at a glance
• NVIDIA DGX Spark: where it fits
• The validated model landscape
• Tier 1 — Datacenter-scale models (100B+)
  • DeepSeek V4 Flash 0731 (284B/13B MoE)
  • MiniMax M3 (428B/23B MoE)
  • GLM-5.2 (753B/40B MoE)
  • Kimi K3 (2.8T/104B MoE)
  • Qwen3-235B-A22B-2507 (235B/22B MoE)
• Tier 2 — Enterprise workstation models
  • Llama 4 Scout (109B/17B MoE)
  • Llama 4 Maverick (400B/17B MoE)
  • Qwen2.5-7B and 14B Instruct 1M
• Tier 3 — Consumer GPU models (under 35B)
• Models often misclassified as 1M models
• Inference engine comparison
• Quantization trade-offs
• Step-by-step deployment workflow
• Cost analysis: self-host vs API
• Decision matrix
• Frequently asked questions
• Official sources

Who Should Self-Host (and Who Should Not)

Self-hosting is not the right answer for everyone. Clarity here prevents expensive mistakes.

Self-hosting is worth it if:

• Data residency is non-negotiable. Compliance obligations (HIPAA, GDPR, defense contracts) prohibit sending data to external APIs.
• You have consistently high volume. At sustained high concurrency with the GPUs staying busy, the economics shift in your favour.
• You need model control. Fine-tuning, custom system prompts baked in at weight level, behavioral red-teaming requiring logit access.
• Offline or air-gapped deployment. Industrial control, secure government environments, or edge deployments with no internet access.

Self-hosting is NOT worth it if:

• Volume is low or bursty. At low throughput, API pricing beats any self-hosted setup after engineering overhead.
• You want 1M context for occasional large-document tasks. APIs are far more economical for infrequent long-context use.
• You lack ML infrastructure expertise. vLLM and SGLang require experience to tune correctly for production.
• Your only motivation is cost. At low utilization, a self-hosted H100 costs more per token than the API.

The utilization trap: A GPU rented at $2.50/hr running at 10% utilization costs ~$0.83 per 1M tokens effectively. The same workload via DeepSeek API costs $0.14. Self-hosting wins only when GPUs stay busy.

The Three Meanings of "Supports 1M Context"

1. Architectural / advertised support

The model was trained, extended, or configured to accept approximately one million tokens. This says little about the hardware needed to run it.

2. Single-sequence technical support

A specific runtime and hardware configuration can process one sequence near the maximum length. This may require max_num_seqs=1, chunked prefill, sparse or hybrid attention kernels, FP8 KV cache, no meaningful concurrency, and extended prefill time.

3. Production support

The system can serve the context length with defined SLA, concurrency, predictable time-to-first-token, fault tolerance, and output headroom. Production 1M serving is substantially harder than a one-request demonstration.

A model should be described as "practical at 1M" only when a documented configuration matches the intended workload.

Understanding the VRAM Math

Every self-hosting plan must account for three separate memory budgets that all compete for the same VRAM.

1. Model weight memory

For a simple dense checkpoint: VRAM_weights ≈ parameters × bits ÷ 8. But this formula breaks for mixed-precision checkpoints:

• DeepSeek V4 Flash 0731: backbone is 284B parameters, but the checkpoint is ~167 GB — not 284 GB — because MoE experts use FP4 while other parameters use FP8, and the release includes a speculative-decoding module.
• GLM-5.2: ~753B parameters reported by Hugging Face, not 744B.

Always use the actual checkpoint size, not a derived estimate from parameter count alone.

2. KV cache memory

The KV cache grows with context length and dominates at 1M. A simplified conventional-attention estimate:

KV bytes ≈ 2 × layers × KV heads × head dimension × tokens × bytes per cache element × concurrent sequences

The factor of 2 represents keys and values. This formula is a starting point only — it does not apply unchanged to DeepSeek's hybrid attention, MiniMax Sparse Attention, Kimi Delta Attention, or GLM's sparse-attention implementation.

Illustrative scale (70B-class, FP16, 1 sequence):

This is in addition to model weights. A 167 GB checkpoint serving 1M context needs **230–250 GB total VRAM** minimum.

3. Runtime overhead

Reserve memory for: quantization scales and metadata, CUDA kernels and graph capture, temporary activations, all-reduce and expert-parallel buffers, multimodal encoders, allocator fragmentation, and the OS when using unified memory.

A configuration leaving only a few gigabytes after loading weights is not a stable serving configuration.

Why "aggregate memory" can mislead

Four 32 GB GPUs provide 128 GB aggregate, but the model must be partitionable across them via tensor or expert parallelism. PCIe-only consumer multi-GPU setups may fit a model while delivering poor performance. Memory capacity is not throughput — DGX Spark's 128 GB is LPDDR5X at 273 GB/s; H100/H200 use much higher-bandwidth HBM with high-speed GPU interconnects.

Hardware Tiers at a Glance

NVIDIA DGX Spark: Where It Fits

NVIDIA DGX Spark uses the GB10 Grace Blackwell Superchip with:

• 128 GB coherent unified LPDDR5X memory;
• 273 GB/s memory bandwidth;
• a 20-core Arm CPU;
• 1 TB or 4 TB NVMe;
• 10 GbE plus ConnectX-7 NIC; and
• up to 1 PFLOP theoretical sparse FP4 tensor performance.

NVIDIA positions one Spark for models up to 200B parameters and two linked Sparks for models up to 405B. Those figures describe model scale, not guaranteed context length, throughput, or compatibility with every architecture.

DGX Spark suitability by model

Good DGX Spark use cases

Private local coding assistants, quantized 30B–120B-class models, Llama 4 Scout at reduced context, Qwen long-context testing below the maximum window, local RAG keeping prompts below 32K–128K, fine-tuning smaller models, and always-on agents where local data handling matters more than maximum throughput.

DGX Spark verdict: Buy it for local capacity, privacy, and compactness — not because the "up to 200B" headline guarantees useful one-million-token inference.

The Validated Model Landscape

Models with downloadable weights and advertised 1M+ context

Qwen3.8-Max note: Announced with 2.4T total / 95B active parameters, but open weights had not been released as of August 6, 2026. Do not include in hardware planning tables until the repository, license, checkpoint format, and runtime recipes are public.

Tier 1 — Datacenter-Scale Models (100B+)

These models require server-grade hardware. Do not attempt to run them on consumer workstations without aggressive quantization and KTransformers-style offload — and expect significant throughput limitations if you do.

DeepSeek V4 Flash 0731 (284B/13B MoE)

The MIT-licensed frontier sweet spot. The most accessible frontier-class MoE for self-hosters by checkpoint size, license, and toolchain support.

Architecture highlights:

• 284B total parameters, ~13B active per token (MoE)
• Mixed FP4 + FP8 checkpoint — actual checkpoint size ~167 GB, not 284 GB
• Hybrid attention: local dense + global sparse
• 1M token context window
• DSpark speculative decoding module
• MIT license — full commercial use permitted

Common VRAM calculation mistake: Multiplying 284B × 1 byte = "284 GB FP8" ignores the actual mixed-format release. Use the actual ~167 GB checkpoint as your weight baseline, then add KV cache, expert-parallel buffers, workspaces, and speculative decoding.

Hardware guidance (derived)

Deploy with vLLM (minimal, from model card)

vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code

Add only flags documented for your installed vLLM release. Do not copy expert-parallel backends or FP4 indexer cache settings from a different checkpoint without confirming compatibility.

Deploy with SGLang (preferred for agentic workflows)

python -m sglang.launch_server \
  --model-path deepseek-ai/DeepSeek-V4-Flash-0731 \
  --tp 2 \
  --context-length 131072 \
  --chunked-prefill-size 4096 \
  --kv-cache-dtype fp8 \
  --enable-torch-compile \
  --port 8000

KTransformers (experimental, single RTX 5090 + 256 GB RAM)

python -m ktransformers.local_chat \
  --model_path deepseek-ai/DeepSeek-V4-Flash-0731 \
  --gguf_path ./DeepSeek-V4-Flash-Q4_K_M.gguf \
  --max_new_tokens 4096

KTransformers keeps the router and dense layers on GPU, offloads individual MoE expert modules to CPU RAM. Enables running the full model on one consumer GPU — but expect ~5–15 tok/s vs 80+ tok/s on a proper multi-GPU setup. Personal use only.

Chat template note: The model card states this release does not use a conventional Jinja chat template. Use the official encoding utilities or the current runtime integration rather than assuming every generic OpenAI-style template produces correct tool calls.

MiniMax M3 (428B/23B MoE)

The large-context specialist. Uses MiniMax Sparse Attention (MSA) that is particularly efficient at very long contexts — but requires careful configuration or the KV cache will OOM.

Architecture highlights:

• ~428B total parameters, ~23B active per token (current vLLM recipe reports 26B under its counting convention)
• Native text, image, and video understanding
• MiniMax Sparse Attention (MSA) — critical for 1M context efficiency
• 1M token context window
• MiniMax Community License — read the actual text before any commercial deployment

Hardware reality

MoE reduces compute per generated token. It does not make inactive expert weights disappear from memory. M3 remains a large checkpoint even with sparse activation.

Critical vLLM flag (REQUIRED)

# Omitting --block-size 128 causes KV cache initialization failure due to MSA block structure
vllm serve MiniMaxAI/MiniMax-M3 \
  --tensor-parallel-size 8 \
  --block-size 128 \
  --enable-auto-tool-choice \
  --tool-call-parser minimax_m3 \
  --reasoning-parser minimax_m3

Use the vLLM version or container image named in MiniMax's current deployment documentation. Support for a new architecture may arrive in a dedicated nightly build before a stable release.

License caution: The MiniMax Community License defines permitted and restricted uses in its own terms — do not reduce it to "non-commercial." Read the current text before exposing a public API, redistributing weights, offering a hosted service, embedding in a paid product, or creating a derivative model.

GLM-5.2 (753B/40B MoE)

The largest MIT-licensed model available. Only organizations with 8×H200 or B200-class clusters can deploy it effectively. For everyone else, quantized variants and the Z.ai API are the practical paths.

Architecture highlights:

• Official repository: zai-org/GLM-5.2
• ~753B parameters reported by Hugging Face checkpoint
• ~39–40B active parameters
• IndexShare attention — reduces redundant attention computation
• Compressed Sparse Attention for 1M context efficiency
• MIT license — commercial use fully permitted
• 1M token context window

Repository correction: The official repository is zai-org/GLM-5.2, not THUDM/GLM-5.2. Always pull from the official source.

Hardware classes

Deploy with vLLM (8×H200, reduced context)

vllm serve zai-org/GLM-5.2 \
  --tensor-parallel-size 8 \
  --max-model-len 131072

For full 1M, follow the current GLM/vLLM recipe rather than merely changing --max-model-len. The full-context path may require a specific checkpoint precision, attention backend, and GPU generation.

1M context reality: Even on 8×H200, serving GLM-5.2 at full 1M context with multiple concurrent users requires careful tuning. For multi-user serving, cap max-model-len at 64K–128K for practical throughput.

Kimi K3 (2.8T/104B MoE)

The largest open-weight model in existence. Kimi K3 at 2.8T parameters is a genuine engineering achievement — and the hardware requirements match.

Architecture highlights:

• 2.8T total parameters, 104B activated parameters
• 896 routed experts, 16 selected per token
• Kimi Delta Attention + gated MLA
• MXFP4 weights, MXFP8 activations (quantization-aware training)
• 1,048,576-token context
• Native text and image input
• Kimi K3 License (broadly permissive with commercial conditions for large businesses)

Memory reality

At 4 bits, 2.8T parameters alone imply roughly 1.4 TB before scales, metadata, multimodal components, cache, activations, and runtime buffers. This rules out consumer workstations, DGX Spark, and many older 8-GPU nodes.

Why 16 active experts does not make it a "small" model: Active-parameter count affects compute per token. Serving still requires all routed expert weights to be accessible. Expert parallelism distributes them but adds communication and routing overhead.

License conditions

The Kimi K3 License is broadly permissive but includes conditions:

• A Model-as-a-Service business with aggregate revenue above $20 million over 12 consecutive months must enter a separate agreement before commercial use
• Qualifying very large products must display "Kimi K3"
• The license includes exceptions for internal use and access through official or certified partners

Read the current license text for exact definitions and thresholds before commercial deployment.

Multi-node vLLM (minimum viable)

# Requires tp=8, pp=8 = 64 GPUs minimum
python -m vllm.entrypoints.openai.api_server \
  --model moonshotai/Kimi-K3 \
  --tensor-parallel-size 8 \
  --pipeline-parallel-size 8 \
  --max-model-len 32768 \
  --port 8000

There is no consumer path to running Kimi K3. Use the Kimi API for individual and small-team access. The open weights matter for large enterprises building proprietary inference infrastructure.

Qwen3-235B-A22B-2507 (235B/22B MoE)

The Apache 2.0 frontier MoE with native long-context extension.

Official specifications:

• 235B total parameters, 22B active
• 262,144-token native context
• ~1,010,000 tokens via extended-context configuration (Dual Chunk Attention + MInference)
• Apache 2.0 license
• Official guidance: ~1,000 GB total GPU memory for 1M operation

The 1M path is not simply a --max-model-len change. It requires loading a different config file and a specific attention backend.

Official long-context preparation

export MODELNAME=Qwen3-235B-A22B-Instruct-2507
huggingface-cli download Qwen/${MODELNAME} --local-dir ${MODELNAME}

mv ${MODELNAME}/config.json ${MODELNAME}/config.json.bak
mv ${MODELNAME}/config_1m.json ${MODELNAME}/config.json

vLLM launch (1M context)

VLLM_ATTENTION_BACKEND=DUAL_CHUNK_FLASH_ATTN VLLM_USE_V1=0 \
vllm serve ./Qwen3-235B-A22B-Instruct-2507 \
  --tensor-parallel-size 8 \
  --max-model-len 1010000 \
  --enable-chunked-prefill \
  --max-num-batched-tokens 131072 \
  --enforce-eager \
  --max-num-seqs 1 \
  --gpu-memory-utilization 0.85

Consult the current model card before deployment — long-context kernels and vLLM integration can change between releases.

Tier 2 — Enterprise Workstation Models

These models sit in the sweet spot for teams with multi-GPU workstations or affordable cloud GPU rentals, without requiring full datacenter infrastructure.

Llama 4 Scout (109B/17B MoE)

The long-context specialist. Scout is one of the best DGX Spark candidates in this guide because 4-bit weights are roughly 55 GB before overhead, and its architecture supports long context natively.

python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Llama-4-Scout \
  --tensor-parallel-size 2 \
  --max-model-len 524288 \
  --gpu-memory-utilization 0.90 \
  --port 8000

Cap max_model_len to the workload you actually need. Reserving a massive cache for the headline 10M window will prevent the model from loading on all but the largest hardware.

Llama 4 Maverick (400B/17B MoE)

The reasoning powerhouse. 128 experts but only 17B active params per token — one of the most parameter-efficient frontier-class models per inference FLOP.

Maverick at ~200 GB at a simple 4-bit estimate before overhead. One DGX Spark is insufficient. Choose Maverick only when you have a well-supported multi-GPU server and a reason to prefer its capability over smaller, newer models.

python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Llama-4-Maverick \
  --tensor-parallel-size 4 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.88 \
  --port 8000

Qwen2.5-7B and 14B Instruct 1M

The clearest entry point for true 1M context testing — because Qwen publishes explicit 1,010,000-token model variants with documented full-context memory requirements and deployment examples.

# Full 1M via vLLM (from official Qwen model card)
vllm serve Qwen/Qwen2.5-7B-Instruct-1M \
  --tensor-parallel-size 4 \
  --max-model-len 1010000 \
  --enable-chunked-prefill \
  --max-num-batched-tokens 131072 \
  --enforce-eager \
  --max-num-seqs 1

Practical recommendation: Use the 7B model to validate the business value of 1M context before attempting a frontier MoE. Start at 64K or 128K, monitor real peak memory, then raise the limit incrementally.

Tier 3 — Consumer GPU Models (Under 35B)

These models run on a single high-end consumer GPU. They do not reach 1M context on consumer hardware — but they are the right starting point for individuals and small teams and are deliberately excluded from the "1M models" category above.

Qwen3 30B-A3B MoE

Apache 2.0. 30B total / 3B active per token. Consumer GPU champion — fits on a single RTX 4090/5090 at INT4.

# Easiest path
ollama run qwen3:30b

# vLLM single GPU
python -m vllm.entrypoints.openai.api_server \
  --model Qwen/Qwen3-30B-A3B \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --port 8000

Qwen2.5 7B–32B (standard releases)

Good for local RAG, personal assistants, and embedded applications. Use 128K context variants. Not to be confused with the dedicated Qwen2.5-7B-Instruct-1M model above, which has a different checkpoint optimized for 1M context.

Mistral Small 3.1 (22B/24B)

Apache 2.0. Fits on a single RTX 4090 at INT4. One of the most practically deployable models for individual developers. 128K context — not a 1M model.

ollama run mistral-small3.1

python -m vllm.entrypoints.openai.api_server \
  --model mistralai/Mistral-Small-3.1-24B-Instruct-2503 \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.90 \
  --port 8000

Models Often Misclassified as 1M Models

Gemma 4

Gemma 4 is useful for local multimodal and agentic workloads, but the official context limits are:

• 128K for smaller E2B/E4B models
• 256K for 12B, 26B-A4B, and 31B models

Present it as a strong 128K–256K local alternative, not a native 1M model.

Mistral Small 3.1

24B parameters, 128K context, vision support, Apache 2.0. A good RTX 4090-class model when quantized — not a million-token model.

Standard Qwen3 releases

Several Qwen3 models have 32K native context or smaller extended windows. Do not generalize the 1M configuration of Qwen3-235B-A22B-Instruct-2507 to every Qwen3 checkpoint.

Qwen3-VL 8B

Advertises 256K native context expandable to 1M. Interesting small multimodal candidate, but the model card does not provide the same explicit full-1M memory recipe as Qwen2.5-1M. Treat 1M as an extended mode requiring validation.

Inference Engine Comparison

Choosing the right inference engine is as important as choosing the right model.

vLLM: The production standard

vLLM's PagedAttention manages the KV cache as non-contiguous memory pages — the same principle as virtual memory. For any production multi-user serving scenario on NVIDIA GPUs, vLLM is the default choice.

SGLang: The agentic specialist

SGLang's RadixAttention is purpose-built for workloads where multiple requests share a large common prefix (e.g., a system prompt containing your entire codebase). SGLang reuses the computed KV cache across those requests — dramatically cutting time-to-first-token. For coding agents, RAG pipelines, and tool-calling intensive workloads, SGLang consistently outperforms vLLM on latency.

There is no universal winner between vLLM and SGLang

Benchmark both with your workload's actual prompt-length distribution, shared-prefix rate, output length, tool-call frequency, and concurrency. A benchmark using 2K prompts and 256-token outputs says little about a 500K-document workload.

Quantization Trade-offs

Practical hierarchy

Fixed quality-loss percentages (e.g., "INT4 loses 3–7%") are not generally valid. Quality impact varies by quantizer, calibration data, architecture, task, prompt length, and whether the model was trained for the target format. Test your actual workload before committing to a hardware budget.

KV cache quantization (independent from weight quantization)

# FP8 KV cache cuts KV VRAM ~50% with minimal perplexity impact
python -m vllm.entrypoints.openai.api_server \
  --model your-model \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 2 \
  --port 8000

Weight precision and KV-cache precision are independent decisions. FP8 KV cache can materially increase available context or concurrency. Validate long-context retrieval accuracy and output quality near the end of the context window after enabling it.

Build a private evaluation set before committing

Include documents at 32K, 128K, 256K, 512K, and your target maximum; relevant facts near the beginning, middle, and end; distractors; exact citation requirements; tool calls and structured output; and multi-turn cache reuse.

Step-by-Step Deployment Workflow

Step 1: Define the real target

Record: maximum input tokens, typical input tokens, maximum output tokens, requests per second, concurrent sequences, latency target, privacy requirement, uptime target, and multimodal requirements. Do not start with "I need 1M" unless the application actually sends 1M-token prompts.

Step 2: Start below the maximum and scale up

32K → 64K → 128K → 256K → 512K → 1M

At every stage record: peak GPU memory, prefill time, time-to-first-token, decode rate, request failures, retrieval accuracy, and output correctness.

Step 3: Complete vLLM setup

# 1. Create environment
python -m venv vllm-env && source vllm-env/bin/activate

# 2. Install vLLM
pip install vllm

# 3. Download model (example: DeepSeek V4 Flash 0731)
huggingface-cli download deepseek-ai/DeepSeek-V4-Flash-0731 \
  --local-dir ./models/deepseek-v4-flash \
  --local-dir-use-symlinks False

# 4. Launch server
python -m vllm.entrypoints.openai.api_server \
  --model ./models/deepseek-v4-flash \
  --tensor-parallel-size 2 \
  --max-model-len 131072 \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.92 \
  --enable-prefix-caching \
  --host 0.0.0.0 \
  --port 8000

# 5. Test
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "deepseek-v4-flash", "messages": [{"role": "user", "content": "Hello!"}]}'

Step 4: SGLang setup (for agentic workloads)

pip install sglang[all]

python -m sglang.launch_server \
  --model-path ./models/deepseek-v4-flash \
  --tp 2 \
  --context-length 131072 \
  --chunked-prefill-size 4096 \
  --kv-cache-dtype fp8 \
  --enable-torch-compile \
  --host 0.0.0.0 \
  --port 8000

Step 5: Ollama (Tier 3 models, easiest path)

curl -fsSL https://ollama.ai/install.sh | sh

ollama run qwen3:30b
ollama run mistral-small3.1

# Expose as API
OLLAMA_HOST=0.0.0.0 ollama serve

Step 6: Python client (works with any server)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed",
)

response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Explain the bug in this code..."}],
    max_tokens=2048,
)
print(response.choices[0].message.content)

Step 7: Pin the software stack

Record exact versions of: model revision or commit, CUDA, GPU driver, PyTorch, vLLM or SGLang, Transformers, FlashAttention or model-specific kernels, and container image digest. "Latest" is not reproducible.

Step 8: Verify license and redistribution terms

Before production: archive the license version, verify commercial-use conditions, check attribution requirements, review acceptable-use terms, and confirm whether a hosted API changes your obligations.

Step 9: Add observability and harden the endpoint

Monitor GPU and host memory, cache utilization, prefill and decode latency separately, queue time, batch size, cache-hit rate, OOM and worker restart count, malformed tool calls, and cost per completed task.

Security minimums: bind privately by default, add authentication and rate limits, isolate model workers, encrypt disks, protect Hugging Face tokens, restrict outbound network access, and log metadata without retaining sensitive prompts unnecessarily.

Cost Analysis: Self-Host vs API

GPU rental prices (August 2026)

A better monthly model

Rather than multiplying one headline tokens-per-second figure by 3,600 (which assumes constant peak throughput and no idle time), use:

monthly self-host cost =
  GPU/server cost
+ storage
+ network
+ engineering and operations (0.5–1 FTE is realistic)
+ power and cooling
+ idle capacity cost
+ redundancy

monthly API cost =
  new input tokens × input rate
+ cached input tokens × cache-read rate
+ output tokens × output rate
+ tool or storage charges

Then divide self-host cost by successfully completed production tasks, not generated tokens alone.

API price snapshot (August 6, 2026)

Verify the linked pricing pages before budgeting — prices are promotional or subject to change.

Factors that favour an API

Low or bursty volume, occasional 500K–1M prompts, rapidly changing model requirements, no infrastructure team, global availability requirements, need to compare multiple frontier models.

Factors that favour self-hosting

Strict on-premises or air-gapped requirements, sustained GPU utilization, sensitive prompts that cannot leave controlled infrastructure, custom weights or logit access, predictable model versioning, and the ability to operate GPUs efficiently.

Hidden costs to budget for

• Engineering time: 0.5–1 FTE for setup, monitoring, model updates, incident response
• Storage: ~167 GB NVMe for DeepSeek Flash weights alone; dedicated NVMe on cloud providers costs extra
• Networking: Multi-GPU tensor parallelism requires InfiniBand or NVLink for maximum throughput
• Idle cost: Bursty workloads (busy 4 hours, idle 20 hours) pay for the idle GPU time at the same rate

Decision Matrix

Frequently Asked Questions

Can a single DGX Spark run a 1M-token model?

It can load several models that advertise 1M and run them at shorter context. Full 1M is different. Qwen's smallest dedicated 1M model calls for at least 120 GB aggregate GPU VRAM, while Spark has 128 GB total unified memory shared by the OS, runtime, weights, cache, and workspaces. Full 1M is borderline and unvalidated as a general Spark configuration.

Why is the DeepSeek V4 Flash 0731 checkpoint ~167 GB when it has 284B parameters?

The 0731 release uses a mixed FP4+FP8 format: MoE experts use FP4 while many other parameters use FP8. The release also bundles a speculative-decoding module. Do not calculate VRAM by multiplying 284B × 1 byte.

What is GLM-5.2's official repository?

zai-org/GLM-5.2. The Hugging Face checkpoint reports approximately 753B parameters. Not THUDM/GLM-5.2.

Is Kimi K3 actually usable for self-hosters?

Not practically without 64+ H100/H200 GPUs in a cluster environment. Use the Kimi API. The open weights are valuable for large enterprises and researchers studying the architecture.

Does Gemma 4 support 1M context?

No. Gemma 4's maximum context is 256K for the 12B–31B models and 128K for smaller ones. It should not be described as a "1M context model."

Is Mistral Small 3.1 a 1M context model?

No. Its maximum context is 128K.

Which inference engine should I start with?

Start with the runtime explicitly recommended for the checkpoint and hardware. For Tier 3 models: Ollama. For production multi-user serving (Tier 1/2): vLLM. If building an agentic system or needing low latency on shared prefixes: benchmark SGLang after stabilizing with vLLM.

Does self-hosting mean full data privacy?

On-premises hardware can keep prompts inside your environment, but privacy still depends on logging, telemetry, remote model downloads, container images, outbound network access, backups, and administrator access. Cloud GPU rental remains third-party infrastructure.

Can I use these models with Cursor, Continue, or Claude Code?

Most tools that accept a custom OpenAI-compatible endpoint can use vLLM or SGLang. Compatibility still depends on chat templates, tool-call schema, reasoning-output handling, and the client's model assumptions.

Is GLM-5.2 really MIT licensed?

Yes. The official Hugging Face model card at huggingface.co/zai-org/GLM-5.2 identifies the license as MIT.

Is MiniMax M3 non-commercial?

That description is too crude. It uses the MiniMax Community License, which defines permitted and restricted uses in its own terms. Read the current text for the intended deployment.

Is Kimi K3 free for commercial use?

The custom license grants broad rights but imposes conditions for certain MaaS businesses above $20M annual revenue. Review the exact definitions and current license before deployment.

Official Sources

Model cards and repositories

• DeepSeek V4 Flash 0731 — official Hugging Face model card
• DeepSeek V4 technical report
• GLM-5.2 — official Hugging Face model card
• Kimi K3 — official GitHub repository
• Kimi K3 License
• MiniMax M3 — official Hugging Face model card
• MiniMax M3 License
• Llama 4 Scout — official Hugging Face model card
• Llama 4 Maverick — official Hugging Face model card
• Qwen2.5-7B-Instruct-1M — official model card
• Qwen2.5-14B-Instruct-1M — official model card
• Qwen3-235B-A22B-Instruct-2507 — official model card
• Gemma 4 31B — official model card
• Mistral Small 3.1 24B — official model card

Hardware

• NVIDIA DGX Spark product page
• NVIDIA DGX Spark hardware guide

Inference engines

• vLLM documentation
• vLLM recipe: DeepSeek V4 Flash 0731
• vLLM recipe: GLM-5.2
• vLLM recipe: MiniMax M3
• vLLM recipe: Kimi K3
• SGLang documentation
• KTransformers
• llama.cpp
• Ollama

Pricing

• Z.AI API pricing (GLM-5.2)
• DeepSeek API documentation
• MiniMax pay-as-you-go pricing
• CoreWeave GPU Cloud
• Lambda Labs GPU Cloud
• Vast.ai Spot GPU Marketplace
• RunPod GPU Cloud

Last reviewed: August 6, 2026. Model repositories, runtime support, cloud prices, and licenses can change. Pin the exact model revision and software versions used in production, and re-check the applicable license before deployment.