Studied the low-rank bottleneck phenomenon in multi-head attention mechanisms of transformer encoders, which occurs when the head size is less than the input token sequence length. Produced a novel derivation of a condition that can be used to verify if an attention-head distribution can be reached.