Player FM uygulamasıyla çevrimdışı Player FM !
[QA] Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
Manage episode 426507867 series 3524393
LLMs process text with tokens, but individual tokens may not relate to word meanings. This study explores how LLMs convert tokens into higher-level representations, revealing an "erasure" effect.
https://arxiv.org/abs//2406.20086
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1236 bölüm
Manage episode 426507867 series 3524393
LLMs process text with tokens, but individual tokens may not relate to word meanings. This study explores how LLMs convert tokens into higher-level representations, revealing an "erasure" effect.
https://arxiv.org/abs//2406.20086
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1236 bölüm
सभी एपिसोड
×Player FM'e Hoş Geldiniz!
Player FM şu anda sizin için internetteki yüksek kalitedeki podcast'leri arıyor. En iyi podcast uygulaması ve Android, iPhone ve internet üzerinde çalışıyor. Aboneliklerinizi cihazlar arasında eş zamanlamak için üye olun.