A new path optimization method creates reaction paths using EScAIP trained on Open Catalyst Project data
UC Berkeley College of Chemistry and Lawrence Berkeley National Laboratory (Berkeley Lab) researchers have created a new machine-learning method that makes computer simulations of atoms faster and more efficient. The method, known as Efficiently Scaled Attention Interatomic Potential (EScAIP) reduces memory usage by more than five times and speeds up its results by more than ten times compared to existing models. These calculations, which help us better understand batteries and semiconductors, and lead to drug discovery, for example, require a tremendous amount of computing power and are usually done on the world's biggest supercomputers.
Their research was recently presented at the NeurIPS 2024 conference, a leading publication venue in artificial intelligence and machine learning.
Eric Qu, a UC Berkeley graduate student, explained that their approach is inspired by techniques used in large language models. “With our approach,” he said, “researchers can more efficiently map how atoms move around and interact with each other.”
And while other researchers are also working on solving similar problems, this approach stands out precisely because these methods have typically only been used for natural language processing.
Recently, scientists have made large language models like ChatGPT smarter by increasing their size through a method called scaling, which involves adding more parameters to the model. However, these scaling techniques haven’t been widely used for Neural Network Interatomic Potentials (NNIPs), models that help scientists predict molecular and material properties much faster. Aditi Krishnapriyan, paper co-author, and assistant professor of chemical and biomolecular engineering, notes that applying scaling methods to NNIPs could greatly improve simulations for materials, chemistry, and physics.
Scaling involves making these models bigger and smarter by systematically increasing the number of parameters in the neural networks. How you increase these parameters matters: different parameters contribute to model performance in distinct ways, and optimizing this process can lead to significant improvements. Researchers can also design new operations or components within the neural network architecture—such as novel attention mechanisms—that are more expressive, enabling further increases in parameters while maintaining or improving efficiency. But it’s not just about size; scaling also means finding ways to make these models more efficient, using smarter algorithms to save time and computing power during both training and use. Instead of focusing solely on raw processing power, researchers often measure efficiency by how long it actually takes to train or run these models, prioritizing real-world performance.
“We are saying to the science community, ‘Hey, look over here, let’s explore this idea more,’” Krishnapriyan said. “EScAIP is an initial proof-of-concept for how to think about scaling machine learning models in the context of atomistic systems, and now represents a “lower bound” for what’s possible. We think it’s the direction that we should be thinking about going in the field as we enter a future with more data and computational resources.”
Source: Berkeley Lab News