Using a Camera to Spot and Track Drones
August 17, 2017 | EPFLEstimated reading time: 3 minutes
EPFL researchers have shown that a simple camera can detect and track flying drones. Plus, the lightweight, energy-efficient and inexpensive technology could be installed directly on the drones themselves and enhance safety in the skies.
The rising number of drones in air space poses numerous challenges. Topping that list is our ability to simply detect these small unmanned aerial vehicles. Periodic near-misses between drones and large airplanes raise the specter of disaster, and the drones themselves often lack the necessary technology to locate other moving objects. To address these issues, EPFL researchers have developed algorithms capable of detecting and tracking small flying objects using a simple camera. The proof of concept was conducted as part of a PhD dissertation, and a real-time detection and collision avoidance system is now being developed in a project funded by the Commission for Technology and Innovation (CTI).
Today’s collision avoidance systems operate actively: an airplane in flight calculates its position, altitude and course, and communicates this information to other aircraft using the same technology. Those aircraft can then evaluate the risk of a collision based on their own positioning data and, if necessary, alert the pilot. But this system is only effective as long as all aircraft are equipped with the same technology. In reality, drones often lack such systems, which are costly and heavy and consume more power.
Artificial intelligence and Deep learning
A camera can thus be an effective, non-cooperative (i.e., not every aircraft must be equipped with it) addition to that system, provided the camera can successfully detect a flying drone. Therein lay the obstacle that researchers at EPFL's Computer Vision Laboratory (CVLAB) sought to overcome. The biggest challenge for a moving camera is to spot another moving object. This is much more difficult on a drone than it is on a car, which only moves in two dimensions. Drones move in three dimensions, and the camera is called on to detect objects against the sky or the ground, depending on the angle of sight. Plus, drones need to locate objects as quickly as possible, such as when they are still fuzzy black dots against a dark forest. And the fact that no two drones look alike anymore – new models are constantly being developed – meant that the researchers had to find a way to teach the camera to recognize all sorts of drones.
In his thesis, Artem Rozantsev showed that these challenges can be overcome. The first step consisted in using artificial intelligence and deep learning to teach the camera to recognize drones. His method combined information on both appearance (types of drones, position, etc.) and motion (movement in the camera's field of view), as neither alone was capable of achieving sufficiently reliable detection. He therefore proposed a machine-learning technique that operates on spatio-temporal cubes of image intensities where individual patches are aligned using a regression-based motion stabilization algorithm.
Real-time performance and accuracy
But the recognition algorithm on its own was not enough. To train a detector to recognize all types of drones in all kinds of positions, it has to have "seen" as many as possible. The existing database of images, however, is limited. So Rozantsev filled in the gaps by generating realistic synthetic images. The generated images, which are based only on a small set of real examples and a coarse 3D model of the object, are used together with the real examples to train the detector. A key ingredient to his method, the generated images are as close as possible to the real ones – not in terms of image quality, but according to the features used by the machine-learning algorithm.
The researchers managed to develop a reliable algorithm capable of detecting a drone using a lightweight camera similar to those found in smartphones. The aim of the project, now financed by the CTI, is to train a detector using an even larger data set to improve its real-time performance and accuracy. EPFL's CVLAB researchers are working on this in collaboration with FLARM Technology AG, a leading supplier of affordable collision avoidance technology for civil aviation. The first commercial models are expected to be released next year.
Suggested Items
RTX's Raytheon to Design Landsat Next Space Instruments
06/14/2024 | RTXRaytheon, an RTX business, was awarded a $506 million contract from NASA to design and build the Landsat Next Instrument Suite (LandIS), which includes three next generation space instruments, with an option for an additional instrument.
Off Like a Rocket: A Review of the EIPC Summer Conference, Part 1
06/14/2024 | Pete Starkey, I-Connect007Motorway traffic jams, airport car parking, walking for miles, queuing for security, delayed flights—oh, the joys of travelling. I was quickly reminded of my status as a foreigner while waiting patiently in line for a half-hour at passport control to enter the European Union at Amsterdam Schiphol, having flown all the way from London Luton in the United Kingdom, that great distance of 200 miles.
Multicircuits Adds Two More atg Flying Probe Test Systems
06/14/2024 | atg Luther & Maelzer GmbHatg Luther & Maelzer GmbH and IEC USA confirm the order for high-speed bare board testing technology.
Technology Days 2024 at Rehm: #opentochange
06/14/2024 | Rehm Thermal Systems“The only constant is change” – this is particularly true in electronics manufacturing. In an industry that is constantly evolving and changing, it is crucial to be open to change.
Siemens Collaborates with Samsung Foundry to Expand 3D-IC Enablement Tools
06/14/2024 | SiemensSiemens Digital Industries Software today announced that, in collaboration with Samsung Foundry, they have developed compelling new capabilities for the manufacture of multi-die packaged designs at advanced nodes and achieved a host of new product certifications for many of Siemens’ industry-leading IC design and verification technologies.