Learn about new PyTorch advancements for LLMs and how PyTorch is enhancing every aspect of the LLM lifecycle. In this talk from AI Infra @ Scale 2024, software engineers Wanchao Liang and Evan Smothers are joined by Meta research scientist Kimish Patel to discuss our newest features and tools that enable large-scale training, memory efficient [...]
Read More...
The post How PyTorch powers AI training and inference appeared first on Engineering at Meta.

How PyTorch powers AI training and inference

In this talk from AI Infra @ Scale 2024, Joel Colburn, a software engineer at Meta, technical lead Junqiang Lan, and software engineer Jack Montgomery discuss the second generation of MTIA, Meta’s in-house training and inference accelerator. They cover the co-design process behind building the second generation of Meta’s first-ever custom silicon for AI workloads, [...]
Read More...
The post Inside the hardware and co-design of MTIA appeared first on Engineering at Meta.

Inside the hardware and co-design of MTIA

Llama 3 is Meta’s most capable openly-available LLM to date and the recently-released Llama 3.1 will enable new workflows, such as synthetic data generation and model distillation with unmatched flexibility, control, and state-of-the-art capabilities that rival the best closed source models.  At AI Infra @ Scale 2024, Meta engineers discussed every step of how we [...]
Read More...
The post Bringing Llama 3 to life appeared first on Engineering at Meta.

Bringing Llama 3 to life

Delivering new AI technologies at scale also means rethinking every layer of our infrastructure – from silicon and software systems and even our data center designs. For the second year in a row, Meta’s engineering and infrastructure teams returned for the AI Infra @ Scale conference, where they discussed the challenges of scaling up an [...]
Read More...
The post Aparna Ramani discusses the future of AI infrastructure appeared first on Engineering at Meta.

CoFeed

Posts about #ai-infra-@-scale from the top innovative companies, all in one place.