Player FM uygulamasıyla çevrimdışı Player FM !
Re-Introducing LayerNorm: Geometric Meaning, Irreversibility and a Comparative Study with RMSNorm
Manage episode 440815403 series 3524393
This paper explores the geometric implications of LayerNorm in transformers, revealing its irreversibility and redundancy, and advocates for RMSNorm as a more efficient alternative with similar performance.
https://arxiv.org/abs//2409.12951
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1669 bölüm
Manage episode 440815403 series 3524393
This paper explores the geometric implications of LayerNorm in transformers, revealing its irreversibility and redundancy, and advocates for RMSNorm as a more efficient alternative with similar performance.
https://arxiv.org/abs//2409.12951
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1669 bölüm
Semua episod
×Player FM'e Hoş Geldiniz!
Player FM şu anda sizin için internetteki yüksek kalitedeki podcast'leri arıyor. En iyi podcast uygulaması ve Android, iPhone ve internet üzerinde çalışıyor. Aboneliklerinizi cihazlar arasında eş zamanlamak için üye olun.