AI tech in Amazon vans saves time, effort

It’s called Vision-Assisted Package Retrieval, or VAPR for short, and Amazon plans to deploy it in 1,000 Rivian electric delivery vans in the next few months.

In 2020, a team within Amazon Transportation tried to imagine a world where technology is used as an invisible force to help drivers.

“We had to think about factors that are unique to the delivery experience, such as lighting and space constraints inside vans,” said John Colucci, product manager, Amazon Transportation, speaking to Ivy Levine for the Amazon News site.

Levine writes: “While looking for packages in the back of vans only takes a few minutes at each stop for a delivery driver, that time can add up when delivering packages to over a hundred Amazon customers each day.

“With Vision-Assisted Package Retrieval (VAPR), a new AI-powered solution, delivery drivers will no longer have to spend time organising packages by stops, reading labels, or manually checking key identifiers like a customer’s name or address to ensure they have the right packages. They simply have to look for VAPR’s green light; grab, and go.”

Once the van arrives at a delivery location, VAPR automatically projects a green “O” on all packages that will be delivered at that stop and a red “X” on all other packages. Through an audio and visual cue, VAPR prompts the driver, confirming it has found the right packages, before the driver needs to enter the cargo area.

“The technology also removes the need for drivers to use a mobile device throughout the process.”

Bobby Garcia, a driver with Bloomfield Logistics, who has been testing VAPR in the North Boston, Massachusetts area, told Levine: “Before, it could take me anywhere between 2 and 5 minutes to empty a tote and organise packages for the next stops. Now, with VAPR, this whole step just takes me about a minute. “It’s made my life so much easier.”

An AI-powered assist for drivers

VAPR uses Amazon Robotics Identification (AR-ID), a form of computer vision originally developed for fulfilment centres, to automatically identify items during inventory stow or pick operations. AR-ID replaces the need for manual barcode scanning. As it processes its surroundings, it can locate and decipher multiple barcodes in real time. 

VAPR was built by training machine learning models to recognise different labels and packages, in various lighting conditions and package characteristics. The technology was also optimised for an in-van environment, and paired with automotive light projectors and cameras, all integrated with the van’s delivery route navigation system. 

VAPR is taking several AWS technologies on the road, including machine learning platform SageMaker and IoT Greengrass.

Amazon says its Transportation team has spent hundreds of hours in the field ensuring that these new experiences add value. They saw a 67% reduction in perceived physical and mental effort for drivers and more than 30 minutes saved per route.