An image database of more than a billion products helps power the tool
THE WALL STREET JOURNAL // JAN. 13, 2023
Google Cloud said it has developed a new artificial intelligence tool designed to help big-box retailers better track the inventory on their shelves, aiming to improve a technology that has struggled to work well in the past.
Google Cloud said Friday its algorithm can recognize and analyze the availability of consumer packaged goods products on shelves from videos and images provided by the retailer’s own ceiling-mounted cameras, camera-equipped self-driving robots or store associates. The tool, which is now in preview, will become broadly available in the coming months, it said.
Alphabet Inc.’s cloud business unveiled the technology, along with a series of artificial intelligence tools aimed at e-commerce, ahead of the National Retail Federation conference in New York City.
Lack of timely, accurate information about shelf inventory is a major problem for retailers and so difficult to manage that it is industry standard to just make guesses, said Robert Hetu, VP analyst for retail at IT research and consulting firm Gartner Inc. Having that information would help retailers pad their lines in a variety of ways, including giving them the chance to replenish out-of-stock items faster, and lose fewer sales opportunities, according to Carrie Tharp, Google Cloud’s vice president of retail and consumer.
“If every retailer just knew what they had in their stores and how much was left on the shelves, their lives would be so much simpler,” she said.
The idea of computer vision-enabled shelf-checking technology has been around for several years, but has not taken off in that time. In part, retailers have been deterred by the cost and complexity of large-scale camera deployment, said Mr. Hetu.
Data has also been a problem, said Ms. Tharp. Retailers have not historically had access to thorough, organized and labeled data on all their product offerings, she said. Another challenge has been building the AI model itself, which needs to understand how to recognize a product in imperfect, real-life conditions, including from different angles, in different lighting and when seasonal packaging changes, Google Cloud said.
Google Cloud said its product is trained on a database of over a billion products, including images that are publicly available, licensed and provided directly by manufacturers. Its algorithm is also designed to recognize those products, whether the image is coming from a ceiling-mounted camera or mobile-phone video—the same way the human eye understands it is seeing a cereal box regardless of whether it is looking at the box from above or from head on, Google Cloud said. But there are still challenges.
“It’s probably not entirely solved yet,” said Graham Watkins, executive vice president of supply-chain transformation and retail innovation at supermarket chain Giant Eagle Inc. He said the Google Cloud product has shown above 90% accuracy during early tests in a Giant Eagle innovation lab, which is designed to replicate store conditions. That is high enough to generate continued interest from the supermarket chain, but not high enough for the company to consider deploying it at scale yet.
For now Giant Eagle provides continuous feedback to Google Cloud about where the tool isn’t working so that it can fine-tune, he said. For example, if the camera is too high or too low and the algorithm can’t identify the product, Giant Eagle would provide that image to Google Cloud so it can train the algorithm to recognize that angle the next time, said Mr. Watkins.
The supermarket chain said it plans to begin piloting the technology in an actual store in the next several months, but a deployment across the full chain would take several years to come to fruition, if in fact the company decides to pursue a broad rollout, Mr. Watkins said. In part that is because of the high degree of expense involved, he said.
Camera visibility into every shelf on every aisle—whether it is coming from ceiling cameras or inventory robots that stroll around stores—is a complicated and expensive proposition, said Gartner’s Mr. Hetu.
Walmart Inc. notably ended its effort to use roving robots in store aisles to keep track of its inventory in 2020 because it found different, sometimes simpler solutions that proved just as useful, said people familiar with the situation.
Mr. Hetu said he expects investment in shelf-checking technology to continue, despite cost barriers, because of the growing need to digitize the in-store experience
But it won’t happen overnight. It could take three to six years for computer-vision shelf checking to become more mainstream, he said.
Giant Eagle’s Mr. Watkins said the algorithm may never be perfect and there will always be some unique conditions that it won’t work in.
“There’s going to be a give and take between technology and operational business processes. Anytime you’re in a new space, you’re always going to be balancing. How close is good enough?” he said. “It is just a bit of an iterative process.”
Write to Isabelle Bousquette at isabelle.bousquette@wsj.com
