Teaching A.I. to See: How Computer Vision Is Reshaping Medicine, Security, YouTube and the NBA

Estimated reading time: 8 minutes

What's the next move in the computer vision game? Taking it up a notch to track in true 3D.

Wang recently joined forces with fellow Stevens computer scientist Dunn in a more comprehensive effort to enhance computer vision that can track objects and scenes in greater detail.

Dunn's own expertise lies in image-based 3D modeling, which reconstructs virtual environmental representations from a set of images observing a common scene.

He does it by developing algorithms to pick out and analyze common features in large sets of images, then piecing the scenes together in order to estimate their geometry. In one project, he was able to recreate hundreds of famous world landmarks in 3D — almost down to the inch — without ever seeing them. The system worked, in part, by harvesting and inspecting some 100 million crowd-sourced photos.

"The challenge of efficiently identifying clusters of images, by observing the same scene and relying only on the image content, had not been previously addressed at that scale in the context of image-based modeling," explains Dunn. "The commonality and the redundancy among all those observations is what allows you to infer their geometric relationships. And estimating those geometric relationships then allows you to build fully unsupervised modeling systems."

Now he hopes to combine his expertise with Wang's, creating new methods to better compensate for moving, shaking or low-quality cameras as well as conditions of lesser visibility.

Applications, the duo say, could include virtual and augmented reality, autonomous driving and aerial surveillance applications where objects are tracked online in live video feeds captured by moving cameras.

In another fascinating iteration of autonomous/visual navigation, Stevens researcher Philippos Mordohai — who specializes in the resolution of multiple images into cohesive, accurate 3D maps — mounts cameras on a motorized wheelchair to help patients with limited upper-body mobility navigate those wheelchairs more intelligently. The work has been supported by the National Institutes for Health (NIH).

Mordohai also recently received support from Google for another intriguing project: the development of better methods to power augmented reality (AR) systems and games such as Pokemon Go that need to continuously track the real 3D scenes and insert virtual objects in motion seamlessly into them.

Making Waves at Microsoft, Adobe

Current faculty and students aren't the only ones probing the frontiers and promise of computer vision. Stevens faculty and alumni have also gone on to make a significant impact in the field elsewhere.

Leading computer vision researcher and former faculty member Gang Hua, for instance — who helped develop the wheelchair co-robot research eventually completed by Mordohai — published key research in 2015 while still at Stevens on the use of a type of AI known as a convolutional neural network to detect faces.

Hua later joined Microsoft to refine and apply those techniques further, then moved on to one of China's very largest AI ventures, where he continues to develop cutting-edge computer vision projects today.

"It is a great pleasure to see computer vision grow healthily at Stevens," says Hua now.

Doctoral recipient Haoxiang Li Ph.D. '16 is another alumnus who has made major contributions to face-recognition technology, such as helping to build Adobe Photoshop's proprietary ability to quickly and efficiently identify and organize individual faces in collections of images. Li's name already appears on five U.S. technology patents, with four more pending approval.

As Stevens' computer vision group grows and continues expanding its participation in major conferences and publishing in key journals, department chair Ateniese predicts further applications, enhancements and breakthroughs.

"We have a world-class team in computer vision developing here at the university," he says. "We will continue to attack new problems and create new approaches."

• < Previous Page