The Foundations of Tokenization: Statistical and Computational Concerns
Juan Luis Gastaldi, John Terilla, Luca Malagutti, Brian DuSell, Tim Vieira, Ryan Cotterell
January 2025
Publication
Proceedings of the 11th International Conference on Learning Representations
Add the full text or supplementary notes for the publication here using Markdown formatting.