Attention
Lens
one encoder block · d
model
16 · 4 heads · mock weights
Sentence
Run
Split into 4 heads
Attention matrix
softmax · mean of heads
Rows attend (query) · columns are attended to (key)
Drag to orbit · scroll to zoom · click an arc, token or layer