Ultra-FineWeb-L1 Released: Over 1 Trillion Tokens of High-Quality Web Data

Summary

Ultra-FineWeb-L1, part of the UltraData ecosystem, is now open-sourced, offering over 1 trillion tokens of high-quality English web data processed from Common Crawl. The release underscores the importance of high-quality data for training powerful large language models (LLMs).