Measuring AI's Ability to Learn is Difficult
January 17, 2019 | University of WaterlooEstimated reading time: 1 minute
Organizations looking to benefit from the artificial intelligence (AI) revolution should be cautious about putting all their eggs in one basket, a study from the University of Waterloo has found.
In a study published in Nature Machine Intelligence, Waterloo researchers found that contrary to conventional wisdom, there can be no exact method for deciding whether a given problem may be successfully solved by machine learning tools.
“We have to proceed with caution,” said Shai Ben-David, lead author of the study and a professor in Waterloo’s School of Computer Science. “There is a big trend of tools that are very successful, but nobody understands why they are successful, and nobody can provide guarantees that they will continue to be successful.
“In situations where just a yes or no answer is required, we know exactly what can or cannot be done by machine learning algorithms. However, when it comes to more general setups, we can’t distinguish learnable from un-learnable tasks.”
In the study, Ben-David and his colleagues considered a learning model called estimating the maximum (EMX), which captures many common machine learning tasks. For example, tasks like identifying the best place to locate a set of distribution facilities to optimize their accessibility for future expected consumers. The research found that no mathematical method would ever be able to tell, given a task in that model, whether an AI-based tool could handle that task or not.
“This finding comes as a surprise to the research community since it has long been believed that once a precise description of a task is provided, it can then be determined whether machine learning algorithms will be able to learn and carry out that task,” said Ben-David.
The study, Learnability can be Undecidable, was co-authored by Ben-David, Pavel Hrubeš from the Institute of Mathematics of the Academy of Sciences in the Czech Republic, Shay Morgan from the Department of Computer Science, Princeton University, Amir Shpilka, Department of Computer Science, Tel Aviv University, and Amir Yehudayoff from the Department of Mathematics, Technion-IIT.
Suggested Items
Europlacer Presents New Range of iineo SMT Placement Machines.
05/01/2024 | EuroplacerFor more than 15 years, the Europlacer iineo placement machines have made their mark on the SMT industry with unique features and unrivalled flexibility. Today, Europlacer announces the launch of the second generation iineo.
Real Time with... IPC APEX EXPO 2024: Factory Insights: A Customizable Data Collection Solution
05/01/2024 | Real Time with...IPCEditor Nolan Johnson and Julie Cliche-Dubois, product manager for Cogiscan, focus on Factory Insights, a new product that emphasizes customization. With years of experience in data collection from various machines and factories, the Factory Insights product offers dashboards and KPI customization, striking a balance between out-of-the-box solutions and customization, ultimately bringing contextualization to machine data.
Real Time with... IPC APEX EXPO 2024: Industrial Quality Solutions from Zeiss
04/23/2024 | Real Time with...IPC APEX EXPOEditor Nolan Johnson and Herminso Gomez of Zeiss Group discuss the company's industrial quality solutions, with a focus on X-ray technology. Zeiss provides a range of microscopy options and Herminso highlights the advantages of X-ray technology for aerospace, medical, and consumer electronics sectors.
VDMA: Machine Vision Navigating Through Uncertain Times
04/18/2024 | VDMAFor over a decade, the European machine vision industry has reported steady growth, with turnover increasing by an average of 9 percent annually between 2012 and 2022. Despite a temporary setback in 2020 (minus 4 percent) due to the Covid-19 pandemic, the industry rebounded strongly in 2021 (plus 17 percent) and 2022 (plus 11 percent).
IPC Bestows Posthumous Hall of Fame Award to Industry Icon Michael Ford
04/18/2024 | IPCIPC honored the late Michael Ford, Aegis Software, for his extraordinary contributions to the global electronics manufacturing industry with the IPC Raymond E. Pritchard Hall of Fame Award at IPC APEX EXPO 2024. IPC’s most prestigious honor, the Hall of Fame Award is given to individuals who have provided exceptional service and advancement to IPC and the electronics industry. Ford, an industry leader and valued IPC volunteer, died suddenly in January 2024.