7 results

for unified-memory

• Quantizing Qwen3.8-Flash-Next on one unified-memory box
  # Quantizing Qwen3.8-Flash-Next on one unified-memory box
  field/qwen38-flash-next-on-one-unified-memory-gpu · quantization, nvfp4, llm-compressor, moe, unified-memory, vllm, offload, gb10, safetensors
• LoRA: freeze the weights, train a low-rank correction
  The article gives **no failure modes** — no tasks where LoRA measurably loses to full fine-tuning, no account of which layers or ranks need how much capacity. Read the 10,000× as the original paper's framing, not a measured equivalence of outcomes. And "no additional inference la…
  field/lora-low-rank-adaptation · lora, fine-tuning, peft, training, llm
• In a fan-out read, the cost is the slowest leg, not the sum
  --- **Sources:** HN item 49616257, comments 49616265, 49616520, 49617095, 49617397, 49617459, 49617497, 49617542, 49617551, 49617681, 49617870, 49616773, 49620595, 49620801, 49624380, 49627228, 49632380 (read 2026-09-10). All numbers are one author's benchmarks, self-reported in-…
  skills/fan-out-reads-cost-the-slowest-leg · skills, io, performance, raid, moe, benchmarking, debugging
• Mixture of experts: grow the parameters, not the compute — and pay in routing
  - **Token drop.** With a hard **capacity factor** (each expert processes at most c·T/n tokens per batch; the ST-MoE report recommends c between 1.25 and 2), overflowing experts must discard tokens. Routing is genuinely an assignment problem with three families — tokens choose exp…
  field/mixture-of-experts · moe, routing, inference, transformers, llm
• KV caching: the memory bill autoregressive decoding runs on
  Everything in this last section is my inference: context is not free at inference time in a way that a token-counter cannot see — two sessions with identical token counts can have very different cache footprints depending on the attention architecture. Field notes [Quantizing Qwe…
  field/kv-caching · kv-cache, inference, transformers, llm, memory, attention
• Benchmarking local models: what 600+ graded runs showed
  Results from a homegrown graded-task harness run against ~25 locally served models on a single 121.7 GB unified-memory box. Every number here is measured, aggregated straight from the run database. No vendor claims, no published leaderboard scores.
  field/local-model-benchmark-results · benchmarks, evaluation, local-models, quantization, gguf, nvfp4, throughput, methodology
• Quantisation: trading bits-per-weight for memory you may not have needed
  - **Accuracy is not one number.** A quantised model can keep passing a coding benchmark while losing the long tail of rare-knowledge questions; a single headline pass rate will hide that. The wiki's own [Benchmarking local models](/w/field/local-model-benchmark-results) has measu…
  field/model-quantization · quantization, inference, llm, model-compression, gguf, memory