DARPA Goes 'Meta' with Machine Learning for Machine Learning
June 20, 2016 | DARPAEstimated reading time: 2 minutes
Popular search engines are great at finding answers for point-of-fact questions like the elevation of Mount Everest or current movies running at local theaters. They are not, however, very good at answering what-if or predictive questions—questions that depend on multiple variables, such as “What influences the stock market?” or “What are the major drivers of environmental stability?” In many cases that shortcoming is not for lack of relevant data. Rather, what’s missing are empirical models of complex processes that influence the behavior and impact of those data elements. In a world in which scientists, policymakers and others are awash in data, the inability to construct reliable models that can deliver insights from that raw information has become an acute limitation for planners.
To free researchers from the tedium and limits of having to design their own empirical models, DARPA today launched its Data-Driven Discovery of Models (D3M) program. The goal of D3M is to help overcome the data-science expertise gap by enabling non-experts to construct complex empirical models through automation of large parts of the model-creation process. If successful, researchers using D3M tools will effectively have access to an army of “virtual data scientists.”
“The construction of empirical models today is largely a manual process, requiring data experts to translate stochastic elements, such as weather and traffic, into models that engineers and scientists can then ask questions of,” said Wade Shen, program manager in DARPA’s Information Innovation Office. “We have an urgent need to develop machine-based modeling for users with no data-science background. We believe it’s possible to automate certain aspects of data science, and specifically to have machines learn from prior example how to construct new models.”
D3M is being initiated at a time when there is unprecedented availability of data via improved sensing and open sources, and vast opportunities to take advantage of those data streams to speed scientific discovery, deepen intelligence collection, and improve U.S. government logistics and workforce management. Unfortunately, the expertise required to build useful models is in short supply. Some experts project deficits of 140,000 to 190,000 data scientists worldwide in 2016 alone, and increasing shortfalls in coming years. Also, because the process to build empirical models is so manual, their relative sophistication and value is often limited.
A recent exercise conducted by researchers from New York University illustrated the problem. The goal was to model traffic flows as a function of time, weather and location for each block in downtown Manhattan, and then use that model to conduct “what-if” simulations of various ride-sharing scenarios and project the likely effects of those ride-sharing variants on congestion. The team managed to make the model, but it required about 30 person-months of NYU data scientists’ time and more than 60 person-months of preparatory effort to explore, clean and regularize several urban data sets, including statistics about local crime, schools, subway systems, parks, noise, taxis, and restaurants.
“Our ability to understand everything from traffic to the behavior of hostile forces is increasingly possible given the growth in data from sensors and open sources,” said Shen. “The hope is that D3M will handle the basics of model development so people can apply their human intelligence to look at data in new ways, and imagine solutions and possibilities that were not obvious or even conceivable before.”
Suggested Items
Connect the Dots: Designing for Reality—The Pre-Manufacturing Process
05/08/2024 | Matt Stevenson -- Column: Connect the DotsI have been working with Nolan Johnson on a podcast series about designing PCBs for the reality of manufacturing. By sharing lessons learned over a long career in the PCB industry, we hope to shorten learning curves and help designers produce better boards with less hassle and rework. Episode 2 deals with the electronic pre-manufacturing process. Moving from CAD (computer-aided design) to CAM (computer-aided manufacturing) is a key step in PCB manufacturing. CAM turns digital designs into instructions that machines can use to actually build the PCB.
Altus Equips SSTL with Advanced Cleaning Technology for Electronics
05/07/2024 | Altus GroupAltus Group, a leading supplier of capital equipment for the electronics manufacturing industry, has announced the successful installation of an advanced PCBA cleaning system at Surrey Satellite Technology Limited (SSTL).
RTX's Advanced Ground System for Space-based Missile Warning Now Operational
05/06/2024 | RTXAn advanced ground system for space-based missile warning developed by Raytheon, an RTX business, is now operational at the U.S. Space Force's Overhead Persistent Infrared Battlespace Awareness Center (OBAC).
BrainChip, Frontgrade Gaisler to Augment Space-Grade Microprocessors with AI Capabilities
05/06/2024 | BUSINESS WIREBrainChip Holdings Ltd, the world’s first commercial producer of ultra-low power, fully digital, event-based, neuromorphic AI IP, and Frontgrade Gaisler, a leading provider of space-grade system-on-chip solutions, announce their collaboration to explore the integration of BrainChip’s AkidaTM neuromorphic processor into Frontgrade Gaisler’s next generation fault-tolerant, radiation-hardened microprocessors.
Argonne, Toyota Collaborate on Cutting-Edge Battery Recycling Process
05/01/2024 | BUSINESS WIREThe U.S. Department of Energy’s (DOE) Argonne National Laboratory has recently launched a collaboration with Toyota Motor North America that could reduce the nation’s reliance on foreign sources of battery materials.