Robots Learn by Watching How-to Videos
January 4, 2016 | Cornell UniversityEstimated reading time: 2 minutes
When you hire new workers you might sit them down to watch an instructional video on how to do the job. What happens when you buy a new robot?
Cornell researchers are teaching robots to watch instructional videos and derive a series of step-by-step instructions to perform a task. You won’t even have to turn on the DVD player; the robot can look up what it needs on YouTube. The work is aimed at a future when we may have “personal robots” to perform everyday housework – cooking, washing dishes, doing the laundry, feeding the cat – as well as to assist the elderly and people with disabilities.
The researchers call their project ”RoboWatch.” Part of what makes it possible is that there is a common underlying structure to most how-to videos. And, there’s plenty of source material available. YouTube offers 180,000 videos on “How to make an omelet” and 281,000 on “How to tie a bowtie.” By scanning multiple videos on the same task, a computer can find what they all have in common and reduce that to simple step-by-step instructions in natural language.
Why do people post all these videos? “Maybe to help people or maybe just to show off,” said graduate student Ozan Sener, lead author of a paper on the video parsing method presented Dec. 16 at the International Conference on Computer Vision in Santiago, Chile. Sener collaborated with colleagues at Stanford University, where he is currently a visiting researcher.
A key feature of their system, Sener pointed out, is that it is “unsupervised.” In most previous work, robot learning is accomplished by having a human explain what the robot is observing – for example, teaching a robot to recognize objects by showing it pictures of the objects while a human labels them by name. Here, a robot with a job to do can look up the instructions and figure them out for itself.
Page 1 of 2
Testimonial
"The I-Connect007 team is outstanding—kind, responsive, and a true marketing partner. Their design team created fresh, eye-catching ads, and their editorial support polished our content to let our brand shine. Thank you all! "
Sweeney Ng - CEE PCBSuggested Items
PC Graphics Add-in Board Shipments Up 27% QoQ in 2Q25
09/03/2025 | Jon Peddie ResearchAccording to a new research report from the analyst firm Jon Peddie Research, the growth of the global PC-based graphics add-in board market reached 11.6 million units in Q2'25 and desktop PC CPUs shipments increased to 21.7 million units.
PC GPU Shipments Up 8.4% in 2Q25 on Pre-Tariff Demand
09/02/2025 | Jon Peddie ResearchJon Peddie Research reports the growth of the global PC-based graphics processor unit (GPU) market reached 74.7 million units in Q2'25, and PC CPU shipments increased to 66.9 million units.
20 Years of Center Nanoelectronic Technologies (CNT) – Backbone of German Semiconductor Research Celebrates Anniversary
08/14/2025 | Fraunhofer IPMSThe Center Nanoelectronic Technologies (CNT) of the Fraunhofer Institute for Photonic Microsystems (IPMS) is celebrating its 20th anniversary this year. Since its founding in 2005, it has developed into a pillar of applied semiconductor research in Germany and Europe. With its unique research cleanroom and equipment adhering to the 300-mm wafer industry standard, CNT is unparalleled in Germany and serves as a central innovation driver for the microelectronics industry.
Q2 Client CPU Shipments Increased 8% from Last Quarter, Up 13% YoY
08/13/2025 | Jon Peddie ResearchJon Peddie Research reports that the global client CPU market expanded for two quarters in a row, and in Q2’25, it showed unseasonal growth of 7.9% from last quarter, while server CPU shipments increased 22% year over year.
FuriosaAI Closes $125M Investment Round to Scale Production of Next-Gen AI Inference Chip
07/31/2025 | BUSINESS WIREFuriosaAI, a semiconductor company building a new foundation for AI compute, today announced it has completed a $125 million Series C bridge funding round. The investment continues a period of significant momentum for Furiosa as global demand for high-performance, efficient AI infrastructure soars.