AI detects illicit vehicle activity

A new deep learning algorithm can detect vehicles linked to illicit activity, including the movement of nuclear materials. The software can identify unusual vehicle patterns by analysing drone, camera and sensor data. It was developed by researchers at the Department of Energy’s Oak Ridge National Laboratory (ORNL).

The software monitors routine traffic over time to establish a baseline for patterns of life, enabling detection of deviations that could signal something out of place. For example, a surge in overnight truck traffic at a facility which is normally only visited during the day could reveal illegal shipments. 

The research builds on a previous ORNL-developed technology for recognising specific vehicles from side views. Sally Ghanem, ORNL lead researcher, says researchers improved the structure of this software’s deep learning network to provide much broader capabilities than any existing recognition systems.

“The majority of the current re-identification models require specific views of the car from the same angles,” says Ghanem. “But our model does not have any of these limitations. We can basically put in any view, from any distance, and determine if it is the same vehicle.”

This means the top of a truck seen from a drone can be matched with a side view from the ground. 

This precision in recognition was achieved by training the software on hundreds of thousands of publicly available images from surveillance cameras, ground sensors and drones, combined with computer-generated images based on vehicle specifications. ORNL researcher John Holliman built 3D digital models of many car and truck brands, varying the paint jobs, perspectives and lighting conditions to create a wide range of training scenarios. The ORNL training images included older vehicle models.

The image set was expanded with footage captured during six data collections around three ORNL campus intersections chosen because vehicles enter and exit by the same route.

“We’re using drones to improve the training data because they are very flexible,” says Ghanem. “Drones can circle a vehicle and change their distance to get many angles, so we can simulate images collected from a satellite or at road level.”

To demonstrate that flexibility, ORNL’s Zach Ryan and Jairus Hines piloted a drone hovering 80 feet over the road to ORNL’s High Flux Isotope Reactor, rotating the drone to follow vehicles through turns for multiple perspectives. They filmed desirable footage of vehicles slightly hidden by tree limbs or traffic lights, and even blurry shots caused by electrical or magnetic interference. 

“The more low-resolution images we include, the more robust the model.”

Unclear footage and nighttime images train the software to more accurately identify vehicles even when visibility is poor, as in some satellite images.

To avoid bias, Ghanem weeded out repetitive images of the same angle or vehicle type. She taught the algorithm with both correct and incorrect matches, making sure the correct pairs represented different perspectives. These methods prevent the algorithm from choosing based only on obvious similarities, such as front views of white sedans.

“By retraining the model on challenging pairs, we make it more capable of tricky matches.”

After training, the team tested the software against 10,000 image pairs, evenly split between correct and incorrect matches. The system proved more than 97% accurate. 

The software leverages a series of neural networks – computational models that function similarly to the brain – which can be trained to not only match different viewpoints but derive long-term patterns from the results.

“The project supports nuclear nonproliferation, enabling us to identify whether shipment activities are happening at a specific place.” 

But the algorithm is also precise enough to track an individual vehicle with stickers, dents or other distinguishing features across a variety of sensors, flagging repeated visits to the same location even if the vehicle takes different routes each time. Researchers are exploring possibilities for adapting the algorithm to incorporate information from non-visual sensors. It could be applied to identifying the shipment of dangerous or illegal substances on other forms of transportation, such as ships and airplanes.

ORNL researchers and staff who contributed to the project, which was funded through ORNL Laboratory Directed Research and Development program, include Ghanem, John Holliman, Ryan Kerekes, Andrew Duncan, Jairus Hines, Ken Dayman, and former staff member Zach Ryan. The High Flux Isotope Reactor is a DOE Office of Science user facility.

UT-Battelle manages ORNL for the Department of Energy’s Office of Science, the largest supporter of basic research in the physical sciences in the United States. The Office of Science says it is working to address some of the most pressing challenges of our time.

* For more information, visit the website here.