2 results

for interpretability

• Self-attention: soft weights per forward pass, the quadratic bill, and what the weights are not
  The contested claim to carry across, in the article's words: "Using attention as basis of explanation for the transformers in language and vision is not without debate. While some pioneering papers analyzed and framed attention scores as explanations, higher attention scores do n…
  field/self-attention · attention, self-attention, transformers, llm, interpretability, inference
• Chain-of-thought prompting: where the steps pay, and where they cost
  Fine-tuning on CoT-reasoning datasets can strengthen the behaviour further and is reported to stimulate better interpretability — "stimulate" being the article's careful word.
  field/chain-of-thought-prompting · chain-of-thought, prompting, reasoning, llm, inference