Local LLM Model Naming Glossary
- URL: https://www.facebook.com/share/1BDwVLk2SN/?mibextid=wwXIfr
- Date Saved: 2026-06-11
- Source: Facebook (胡嘉璽)
- Tags: ai-engineering
Context
Original post breaks down model names like Qwen3-30B-A3B-QAT-Instruct-Q4_K_M_MTP-NSFW for people new to local model deployment, explaining what each segment means.
Glossary
Parameter Size (參數量)
| Term | Meaning |
|---|---|
| B | Billion parameters (e.g. 30B = 30 billion) |
| M | Million parameters (e.g. 125M = 125 million) |
| 7B, 13B, 30B, 70B | Common model sizes; bigger = smarter but needs more VRAM |
Architecture / Mixture of Experts (MoE)
| Term | Meaning |
|---|---|
| A3B | “Active 3 Billion” — in MoE models, only a subset of parameters are active per token. 30B-A3B means 30B total params but only 3B active at inference time. Much faster than dense 30B. |
| MoE | Mixture of Experts — architecture that routes tokens to specialized sub-networks |
| Dense | Traditional architecture where all parameters are used for every token |
Quantization (量化)
| Term | Meaning |
|---|---|
| GGUF | File format for quantized models (used by llama.cpp, LM Studio, Ollama) |
| Q4_K_M | 4-bit quantization, K-quant method, Medium quality. Smaller file, slight quality loss. |
| Q4_K_S | 4-bit, K-quant, Small — more aggressive compression than M |
| Q5_K_M | 5-bit, K-quant, Medium — good balance of quality and size |
| Q6_K | 6-bit, K-quant — near-lossless for most use cases |
| Q8_0 | 8-bit quantization — very close to full precision |
| F16 | Full 16-bit floating point (no quantization) |
| BF16 | Brain Float 16 — Google’s format, same size as F16 but different precision distribution |
| MXFP4 | Microsoft’s mixed-precision 4-bit format |
| QAT | Quantization-Aware Training — model was trained knowing it would be quantized, so quality holds up better at low bits vs post-training quantization |
| AWQ | Activation-aware Weight Quantization — another quant method |
| GPTQ | Post-training quantization method (older, GPU-focused) |
| EXL2 | ExLlamaV2 quantization format, variable bits-per-weight |
| IQ4_XS | Importance-matrix quantization, 4-bit extra-small — newer, quality-preserving |
Fine-tuning Method (微調方法)
| Term | Meaning |
|---|---|
| Instruct | Fine-tuned to follow instructions (chat format). Without this = base model that just does text completion. |
| Chat | Similar to Instruct — trained for conversational format |
| Base | Raw pretrained model, no instruction tuning |
| LoRA | Low-Rank Adaptation — lightweight fine-tuning that trains small adapter layers |
| DoRA | Weight-Decomposed Low-Rank Adaptation — improved LoRA variant |
| QLoRA | LoRA applied to a quantized base model (saves VRAM during training) |
| SFT | Supervised Fine-Tuning — training on curated instruction/response pairs |
| DPO | Direct Preference Optimization — alignment training using preference data |
| RLHF | Reinforcement Learning from Human Feedback — alignment method (used by ChatGPT) |
| Merge | Model created by mathematically combining weights from multiple models |
| SLERP / TIES / DARE | Different merge strategies |
Inference Optimization (推理優化)
| Term | Meaning |
|---|---|
| MTP | Multi-Token Prediction — model predicts multiple tokens at once for faster generation (speculative-style) |
| Speculative Decoding | Using a small draft model to propose tokens, verified by the main model |
| KV Cache | Key-Value cache — stores past attention computations to avoid recomputation |
| Flash Attention | Memory-efficient attention algorithm |
| Context Length | Max tokens the model can process at once (e.g. 4K, 32K, 128K) |
| RoPE | Rotary Position Embedding — enables context length extension |
Content / Safety Tags
| Term | Meaning |
|---|---|
| NSFW | Not Safe For Work — uncensored model, safety filters removed |
| Uncensored | Same as NSFW — trained without refusal behavior |
| Abliterated | Safety training surgically removed via representation engineering |
| Censored / Aligned | Default — model will refuse harmful requests |
Model Family Prefixes
| Term | Meaning |
|---|---|
| Qwen3 | Alibaba’s Qwen family, version 3 |
| Llama-3 | Meta’s LLaMA family |
| Mistral / Mixtral | Mistral AI models (Mixtral = MoE variant) |
| Gemma | Google DeepMind’s open models |
| Phi | Microsoft’s small efficient models |
| DeepSeek | DeepSeek AI models |
| Command-R | Cohere’s models |
File / Deployment Terms
| Term | Meaning |
|---|---|
| safetensors | Safe file format for model weights (replaces pickle) |
| VRAM | GPU memory needed to load the model |
| Offloading | Putting some layers on CPU RAM when VRAM isn’t enough (slower) |
| Context window | How much text the model can “see” at once |
| Tokens | Sub-word units; ~1 token ≈ 0.75 English words or ~0.5 Chinese characters |
VRAM Quick Reference
| Quant | 7B | 14B | 30B | 70B |
|---|---|---|---|---|
| Q4_K_M | ~5 GB | ~9 GB | ~19 GB | ~42 GB |
| Q8_0 | ~8 GB | ~16 GB | ~34 GB | ~75 GB |
| F16 | ~14 GB | ~28 GB | ~60 GB | ~140 GB |
(Add ~1-2 GB overhead for KV cache at short context)
How to Read a Model Name
Example: Qwen3-30B-A3B-QAT-Instruct-Q4_K_M_MTP-NSFW
Qwen3 → Model family (Alibaba, version 3)
30B → 30 billion total parameters
A3B → Only 3B active per token (MoE)
QAT → Quantization-Aware Training
Instruct → Instruction-tuned (chat-ready)
Q4_K_M → 4-bit K-quant Medium quantization
MTP → Multi-Token Prediction enabled
NSFW → Uncensored / no safety filters