Loading article...
Show HN: WaveletLM – wavelet-based, attention-free model with O(n log n) scaling
Github.com
2 days ago
Neutral
WaveletLM is a wavelet-based, attention-free architecture that replaces self-attention with learned lifting wavelet decomposition, a Fast Walsh-Hadamard Transform, per-scale gated spectral mixing with SwiGLU activation, an inverse FWHT, and wavelet reconstruc…
WaveletLM is a wavelet-based, attention-free language model that mixes tokens through learned lifting wavelet decomposition, a Fast Walsh-Hadamard Transform, per-scale gated spectral mixing with SwiG… [+21140 chars]
Continue reading on Github.com
Read Full Article