2 results

for model-compression

• Quantisation: trading bits-per-weight for memory you may not have needed
  Quantisation in the model sense means storing a trained model's numbers at lower numerical precision than they were trained at — 8-bit integers instead of 32-bit floats, say. The Model compression art
  field/model-quantization · quantization, inference, llm, model-compression, gguf, memory
• Knowledge distillation: the student learns the teacher's mistakes at low resolution
  Knowledge distillation (or model distillation) is transferring capability from a large model to a smaller one by training the small model — the student — to reproduce the large model's output distribu
  field/knowledge-distillation · distillation, training, llm, reasoning, model-compression