Meta Platforms Inc. said Monday that its research team built a new artificial intelligence supercomputer that the company maintains will soon be the fastest in the world.
The supercomputer, the AI Research SuperCluster, was the result of nearly two years of work, often conducted remotely during the height of the pandemic, and led by the Facebook parent’s AI and infrastructure teams. Several hundred people, including researchers from partners Nvidia Inc., Penguin Computing Inc. and Pure Storage Inc., were involved in the project, the company said.
Meta, which announced the news in a blog post Monday, said its research team currently is using the supercomputer to train AI models in natural-language processing and computer vision for research. The aim is to boost capabilities to one day train models with more than a trillion parameters on data sets as large as an exabyte, which is roughly equivalent to 36,000 years of high-quality video.
“The experiences we’re building for the metaverse require enormous compute power…and RSC will enable new AI models that can learn from trillions of examples, understand hundreds of languages, and more,” Meta CEO Mark Zuckerberg said in a statement provided to The Wall Street Journal.
Meta’s AI supercomputer houses 6,080 Nvidia graphics-processing units, putting it fifth among the fastest supercomputers in the world, according to Meta.
By mid-summer, when the AI Research SuperCluster is fully built, it will house some 16,000 GPUs, becoming the fastest AI supercomputer in the world, Meta said. The company declined to comment on the location of the facility or the cost.
Artificial intelligence typically requires computers to perform a large number of low-precision calculations at the same time. GPUs are well suited for the task as they have thousands of processing cores that can work simultaneously.
“For us, it’s an order of magnitude larger compute that’s available to our researchers to train a single model with a lot more compute than anybody else in the world has access to,” Jerome Pesenti, vice president of AI at Meta, said in an interview. “We believe that this will really unlock an AI that just understands the world around you much, much better.”
Regular supercomputers are optimized for high-precision activity, whereas AI supercomputers operate on much lower levels of precision, gaining speed without affecting the end results, said Shubho Sengupta, a software engineer at Meta who was one of the leads on the supercomputer initiative.
High-performance computing infrastructure, such as a supercomputer, is necessary to make sense of the volumes of data that Facebook needs, said Bill Gropp, director of the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign.
“In Facebook’s case, I suspect what they’re looking at is the treasure trove that they have of the data that they get from their users, and that’s a huge data set, too big for most of the artificial intelligence systems that researchers typically use,” Dr. Gropp said.
Meta’s supercomputer is being used for research purposes, said Mr. Pesenti, with products not likely arising from it for years. The objective is to ingest troves of data to build AI models that can think like a human brain, with multiple inputs—such as voice and visual recognition—and can deliver contextual understanding of situations.
“You want to have an artificial intelligence that’s a lot more context aware, that understands all the subtleties and a level of granularity that we can’t match today,” said Mr. Pesenti.
Eventually the supercomputer will help Meta’s researchers build AI models that can work across hundreds of languages, analyze text, images and video together and develop augmented reality tools, the company said. The technology also will help Meta more easily identify harmful content and will aim to help Meta researchers develop artificial-intelligence models that think like the human brain and support rich, multidimensional experiences in the metaverse.
“In the metaverse, it’s one hundred percent of the time, a 3-D multi-sensorial experience, and you need to create artificial-intelligence agents in that environment that are relevant to you,” said Mr. Pesenti.
Write to Suman Bhattacharyya at [email protected]
Copyright ©2022 Dow Jones & Company, Inc. All Rights Reserved. 87990cbe856818d5eddac44c7b1cdeb8
