Toward Machines that Improve with Experience

Estimated reading time: 3 minutes

Self-driving taxis. Cell phones that react appropriately to spoken requests. Computers that outcompete world-class chess and Go players. Artificial Intelligence (AI) is becoming part and parcel of the technological landscape—not only in the civilian and commercial worlds but also within the Defense Department, where AI is finding application in such arenas as cybersecurity and dynamic logistics planning. 

But even the smartest of the current crop of AI systems can’t stack up against adaptive biological intelligence. These high-profile examples of AI all rely on clever programming and extensive training datasets—a framework referred to as Machine Learning (ML)—to accomplish seemingly intelligent tasks. Unless their programming or training sets have specifically accounted for a particular element, situation, or circumstance, these ML systems are stymied, unable to determine what to do. 

That’s a far cry from what even simple biological systems can do as they adapt to and learn from experience. And it’s light years short of how, say, human motorists build on experience as they encounter the dynamic vagaries of real-world driving—becoming ever more adept at handling never-before-encountered challenges on the road. 

This is where DARPA’s new Lifelong Learning Machines (L2M) program comes in. 

The technical goal of L2M is to develop next-generation ML technologies that can learn from new situations and apply that learning to become better and more reliable, while remaining constrained within a predetermined set of limits that the system cannot override. Such a capability for automatic and ongoing learning could, for example, help driverless vehicles become safer as they apply knowledge gained from previous experiences—including the accidents, blind spots, and vulnerabilities they encounter on roadways—to circumstances they weren’t specifically programmed or trained for. 

“Life is by definition unpredictable. It is impossible for programmers to anticipate every problematic or surprising situation that might arise, which means existing ML systems remain susceptible to failures as they encounter the irregularities and unpredictability of real-world circumstances,” said L2M program manager Hava Siegelmann. “Today, if you want to extend an ML system’s ability to perform in a new kind of situation, you have to take the system out of service and retrain it with additional data sets relevant to that new situation. This approach is just not scalable.” 

To get there, the L2M program aims to develop fundamentally new ML mechanisms that will enable systems to learn from experience on the fly—much the way children and other biological systems do, using life as a training set. The basic understanding of how to develop a machine that could truly improve from experience by gaining generalizable lessons from specific situations is still immature. The L2M program will provide a unique opportunity to build a community of computer scientists and biologists to explore these new mechanisms. 

“Enabling a computer to learn even the simplest things from experience has been a longstanding but elusive goal,” said Siegelmann. “That’s because today’s computers are designed to run on prewritten programs incapable of adapting as they execute, a model that hasn’t changed since the British polymath Alan Turing developed the earliest computing machines in the 1930s. L2M calls for a new computing paradigm.” 

The four-year L2M program features two technical areas. The first aims to develop ML frameworks that can continuously apply the results of past experience and adapt “lessons learned” to new data or situations. Simultaneously, it calls for the development of techniques for monitoring an ML system’s behavior, setting limits on the scope of its ability to adapt, and intervening in the system’s functions as needed. The research will encompass network theory, algorithms, software, and computer architectures. The second technical area, which derives from Siegelmann’s longstanding interest in biological learning mechanisms, will focus specifically on how living systems learn and adapt and will consider whether and how those principles and techniques can be applied to ML systems. “Life has had billions of years to develop approaches for learning from experience,” Siegelmann said. “There are almost certainly some secrets there that can be applied to machines so they can be not just computational tools to help us solve problems but responsive and adaptive collaborators.”