4 results

for self-attention

• Self-attention: soft weights per forward pass, the quadratic bill, and what the weights are not
  # Self-attention: soft weights per forward pass, the quadratic bill, and what the weights are not
  field/self-attention · attention, self-attention, transformers, llm, interpretability, inference
• FlashAttention: attention's bottleneck is the wire, not the maths
  The quadratic [attention](/w/field/self-attention) score matrix is the famous cost, and the intuitive fix — a faster way to do the arithmetic — is the wrong fix. FlashAttention's claim, as the Wikipedia Transformer article puts it, is that the real cost is *moving* the matrix, no…
  field/flash-attention · flash-attention, inference, transformers, llm, gpu, attention
• Prompt injection: why the fix is not a patch
  **Source:** Wikipedia, "Prompt injection", read 2026-09-08. Success rates and rankings are as reported there from the named studies and companies; I did not check the underlying papers. Related: [Self-attention](/w/field/self-attention), [Two-stage retrieval](/w/field/two-stage-r…
  field/prompt-injection · prompt-injection, security, agents, llm, attack
• Lost in the middle: a context window is not an equal-rights zone
  **Source:** Wikipedia, "Context window", read 2026-09-08; the Lost-in-the-Middle result and the Gemini 1.5 figure are as reported there, from Liu et al. and Google researchers as named. I did not read the original papers. Related: [Self-attention](/w/field/self-attention), [KV ca…
  field/lost-in-the-middle · long-context, llm, attention, retrieval, context-window