A Camera That Can See Unlike Any Imager Before It

Estimated reading time: 4 minutes

Picture a sensor pixel about the size of a red blood cell. Now envision a million of these pixels—a megapixel’s worth—in an array that covers a thumbnail. Take one more mental trip: dive down onto the surface of the semiconductor hosting all of these pixels and marvel at each pixel’s associated tech-mesh of more than 1,000 integrated transistors, which provide each and every pixel with a tiny reprogrammable brain of its own. That is the vision for DARPA’s new Reconfigurable Imaging (ReImagine) program.

This artist’s rendition depicts a single imaging sensor, in this case one that is aboard an unmanned aerial vehicle, simultaneously operating in three potential ReImagine modes—3D-mapping at the lower left, vehicle detection and tracking, and thermal scanning for industrial activity—in different regions of the same field of view. Today a single camera cannot do all of these things.

“What we are aiming for,” said Jay Lewis, program manager for ReImagine, “is a single, multi-talented camera sensor that can detect visual scenes as familiar still and video imagers do, but that also can adapt and change their personality and effectively morph into the type of imager that provides the most useful information for a given situation.”

This could mean selecting between different thermal (infrared) emissions or different resolutions or frame rates, or even collecting 3-D LIDAR data for mapping and other jobs that increase situational awareness. The camera ultimately would rely on machine learning to autonomously take notice of what is happening in its field of view and reconfigure the imaging sensor based on the context of the situation.

The future sensor Lewis has in mind would even be able to perform many of these functions simultaneously because different patches of the sensor’s carpet of pixels could be reconfigured by way of software to work in different imaging modes. That same reconfigurability should enable the same sensor to toggle between different sensor modes from one lightning-quick frame to the next. No single camera can do that now.

A primary driver here, according to Lewis, who works in DARPA’s Microsystems Technology Office (MTO), is the shrinking size and cost of militarily important platforms that are finding roles in locations that span from orbit to the seas. With multi-functional sensors like the ones that would come out of a successful ReImagine program, these smaller and cheaper platforms would provide a degree of situational awareness that today can only come from suites of single-purpose sensors fitted onto larger airborne, ground, space-based, and naval vehicles and platforms. And with the more extensive situational awareness, Lewis said, would come the most important payoff: more informed decision-making.

Today, DARPA posted a Special Notice (DARPA-SN-16-68) on FedBizOpps.gov with instructions for those who might want to attend a Proposers Day on September 30 in Arlington, VA, as a step toward possibly participating in the ReImagine program. In the coming days, DARPA expects to also post a Broad Agency Announcement that specifies the new program’s technical objectives, milestones, schedule, and deliverables, along with instructions for researchers seeking to submit proposals.

One key feature of the ReImagine program is that teams will be asked to develop software-configurable applications based on a common digital circuit and software platform. During the four-year program, MIT-Lincoln Laboratory—a federally funded research and development center (FFRDC) whose roots date back to the WWII mission to develop radar technology—will be tasked to provide the common reconfigurable digital layer of what will be the system’s three-layer sensor hardware.

Page 1 of 2

• Next Page >