Loading…
Attending this event?
June 19-20, 2024
Paris, France
View More Details & Registration
Note: The schedule is subject to change.

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for AI_dev Europe to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

This schedule is automatically displayed in CEST (Central European Summer Time) UTC/GMT +2 hours. To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date."

IMPORTANT NOTE: Timing of sessions and room locations are subject to change.

Back To Schedule
Wednesday, June 19 • 12:05 - 12:35
Scaling to the Future: Training 100B+ Parameter Models on Trillions of Tokens - Hagay Lupesko, Databricks

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
In this talk we'll embark on a journey into the realm of ultra large language models (LLMs), as we unravel the complexities of training 100-billion-plus parameter models on an unprecedented scale. We'll delve into the intricacies of orchestrating thousands of GPUs to process trillions of tokens, discuss cutting-edge model architectures, training optimizations, hardware capabilities, and sophisticated orchestration and fault-tolerance strategies required to sustain weeks-long training runs. This talk is suitable for ML researchers, engineers, and anyone curious about the "sausage making" behind large scale LLM training.
Speakers
avatar for Hagay Lupesko

Hagay Lupesko

GenAI Engineering, Databricks Mosaic AI, Databricks
Hagay Lupesko is an engineering lead at Databricks, where he focuses on making generative AI training and inference efficient, fast, and accessible. Prior to Databricks, held led AI engineering at MosaicML (acquired by Databricks), Meta AI, and AWS ML. He shipped products across various... Read More →

Wednesday June 19, 2024 12:05 - 12:35 CEST
Sorbonne Descartes & Lutèce (Level 5)
  AI Research & Methodologies
Feedback form isn't open yet.