Cracking Open the Black Box of Automated Machine Learning
June 3, 2019 | MITEstimated reading time: 5 minutes
ATMSeer’s interface consists of three parts. A control panel allows users to upload datasets and an AutoML system, and start or pause the search process. Below that is an overview panel that shows basic statistics — such as the number of algorithms and hyperparameters searched — and a “leaderboard” of top-performing models in descending order. “This might be the view you’re most interested in if you’re not an expert diving into the nitty gritty details,” Veeramachaneni says.
ATMSeer includes an “AutoML Profiler,” with panels containing in-depth information about the algorithms and hyperparameters, which can all be adjusted. One panel represents all algorithm classes as histograms — a bar chart that shows the distribution of the algorithm’s performance scores, on a scale of 0 to 10, depending on their hyperparameters. A separate panel displays scatter plots that visualize the tradeoffs in performance for different hyperparameters and algorithm classes.
Case studies with machine-learning experts, who had no AutoML experience, revealed that user control does help improve the performance and efficiency of AutoML selection. User studies with 13 graduate students in diverse scientific fields — such as biology and finance — were also revealing. Results indicate three major factors — number of algorithms searched, system runtime, and finding the top-performing model — determined how users customized their AutoML searches. That information can be used to tailor the systems to users, the researchers say.
“We are just starting to see the beginning of the different ways people use these systems and make selections,” Veeramachaneni says. “That’s because now that this information is all in one place, and people can see what’s going on behind the scenes and have the power to control it.”
Suggested Items
PCB Market Size to Grow by $29.06B from 2024-2028
05/17/2024 | PRNewswireThe global printed circuit board (PCB) market size is estimated to grow by USD 29.06 bn from 2024-2028, according to Technavio. The market is estimated to grow at a CAGR of over 6.6% during the forecast period.
Terran Orbital Joins the Mobile Satellite Services Association (MSSA)
05/16/2024 | BUSINESS WIRETerran Orbital Corporation, a global leader in satellite-based solutions primarily serving the aerospace and defense industries, announced its membership in the Mobile Satellite Services Association (MSSA).
Raytheon-Kongsberg System is Proving Itself – In Demos and In Combat
05/16/2024 | Raytheon TechnologiesOn a rocky island off the coast of Norway, Michael Sadlowski watched as a surface-to-air missile launcher pointed up into a dark nighttime sky.
Un-Jammable Quantum Tech Takes Flight to Boost UK’s Resilience Against Hostile Actors
05/13/2024 | BUSINESS WIREOSI Systems Receives Order for $9 Million to Provide Cargo and Vehicle Inspection Systems
05/13/2024 | BUSINESS WIREOSI Systems, Inc. announced that its Security division received an order from an international customer for approximately $9 million to provide the Company’s Eagle® M60 high energy mobile cargo and vehicle inspection systems including related service and support.