Orhan Demirci, Sezer Aptourachman, Ayd{\i}n Kaya

Researchers introduce Adaptive Dictionary Embeddings (ADE), a framework that scales multi-anchor word representations to large language models by replacing traditional single-vector embeddings with multiple context-aware vectors per word. The approach uses three key techniques: Vocabulary Projection to optimize anchor lookup, Grouped Positional Encoding to preserve semantic coherence, and context-aware reweighting via self-attention. On text classification benchmarks, ADE achieves comparable or better performance than DeBERTa-v3-base while using 98.7% fewer trainable parameters and compressing the embedding layer over 40x.