Measuring AI's Ability to Learn is Difficult
January 17, 2019 | University of WaterlooEstimated reading time: 1 minute

Organizations looking to benefit from the artificial intelligence (AI) revolution should be cautious about putting all their eggs in one basket, a study from the University of Waterloo has found.
In a study published in Nature Machine Intelligence, Waterloo researchers found that contrary to conventional wisdom, there can be no exact method for deciding whether a given problem may be successfully solved by machine learning tools.
“We have to proceed with caution,” said Shai Ben-David, lead author of the study and a professor in Waterloo’s School of Computer Science. “There is a big trend of tools that are very successful, but nobody understands why they are successful, and nobody can provide guarantees that they will continue to be successful.
“In situations where just a yes or no answer is required, we know exactly what can or cannot be done by machine learning algorithms. However, when it comes to more general setups, we can’t distinguish learnable from un-learnable tasks.”
In the study, Ben-David and his colleagues considered a learning model called estimating the maximum (EMX), which captures many common machine learning tasks. For example, tasks like identifying the best place to locate a set of distribution facilities to optimize their accessibility for future expected consumers. The research found that no mathematical method would ever be able to tell, given a task in that model, whether an AI-based tool could handle that task or not.
“This finding comes as a surprise to the research community since it has long been believed that once a precise description of a task is provided, it can then be determined whether machine learning algorithms will be able to learn and carry out that task,” said Ben-David.
The study, Learnability can be Undecidable, was co-authored by Ben-David, Pavel Hrubeš from the Institute of Mathematics of the Academy of Sciences in the Czech Republic, Shay Morgan from the Department of Computer Science, Princeton University, Amir Shpilka, Department of Computer Science, Tel Aviv University, and Amir Yehudayoff from the Department of Mathematics, Technion-IIT.
Testimonial
"We’re proud to call I-Connect007 a trusted partner. Their innovative approach and industry insight made our podcast collaboration a success by connecting us with the right audience and delivering real results."
Julia McCaffrey - NCAB GroupSuggested Items
Driving Innovation: Mastering Panel Warpage
09/23/2025 | Simon Khesin -- Column: Driving InnovationDuring the complex and multi-step process of PCB fabrication, a panel's flatness is constantly at risk. A host of factors can introduce warpage, bending, and unevenness, presenting a fundamental challenge to achieving high-precision results. This deformation (sometimes referred to as “bow and twist”), even on a microscopic scale, can lead to critical defects during subsequent stages, such as component surface mounting (e.g., tombstoning, solder opens) and the PCB's long-term functional reliability.
Secure Semiconductor Manufacturing Acquires Full SMT Line from Manncorp
09/11/2025 | ManncorpSecure Semiconductor Manufacturing, LLC (SSM), an American-owned company dedicated to producing secure printed wiring boards and advanced assembly solutions in the MidWest USA, today announced the acquisition of a complete surface mount technology (SMT) line from Manncorp.
Driving Innovation: Depth Routing Processes—Achieving Unparalleled Precision in Complex PCBs
09/08/2025 | Kurt Palmer -- Column: Driving InnovationIn PCB manufacturing, the demand for increasingly complex and miniaturized designs continually pushes the boundaries of traditional fabrication methods, including depth routing. Success in these applications demands not only on robust machinery but also sophisticated control functions. PCB manufacturers rely on advanced machine features and process methodologies to meet their precise depth routing goals. Here, I’ll explore some crucial functions that empower manufacturers to master complex depth routing challenges.
MS2 Technologies, LLC/P. Kay Focuses on Central America with First Installation in Honduras
08/24/2025 | P. Kay Metal, Inc.This year MS2 Technologies has turned their focused to the growing electronics market in Central America. With that focus came the adaptation of MS2 and the Akila System from a Honduras-based corporation with manufacturing plants in both Honduras and Mexico.
SEL: Revolutionizing PCB Production Through MES, Partnerships, and Vision
08/21/2025 | Barry Matties, I-Connect007Two years ago, we visited Schweitzer Engineering Laboratories (SEL) to better understand its new captive greenfield PCB facility. We recently returned, this time to discuss how this bold vision has transformed the industry. Barry Matties met with John Hendrickson, engineering director, and Jessi Hall, vice president of vertical integration, to discuss the transformative capabilities of Factory Core, SEL’s custom manufacturing execution system (MES), which allows for real-time monitoring of workflow and machine performance, and has led to impressive improvements in quality and cost efficiency.