Computer Vision
7 min read
Computer Vision in Retail Stores: The Future of Retail Industry
Written by
Chinar Movsisyan

In 2016, Amazon stunned the world by releasing a video showcasing what the future of brick-and-mortar stores may look like. In the video, they posed a question: “what would shopping look like if you could walk into a store, grab what you want, and just go?”. Customers were shown scanning their phone to enter the store, proceeding to shop as they normally would, and exit without paying at a cash register for the items they selected. Instead, the application they used to scan in automatically charged them for the items they had selected. These stores, aptly named “Amazon Go”, are nothing short of living organisms. Equipped with a wide array of cameras and sensors, as well as powerful computer vision algorithms, they are capable of tracking a customer’s selections as they make them in the store, removing the need for an employee to then process each item one-by-one and ask the customer to pay. Since Amazon first showcased this technology in 2016, several other companies and start-ups have emerged to compete in the space.

Computer Vision

The technology that is powering the rise of cashier-less retail is computer vision. Computer vision algorithms allow machines to interpret and understand visual information from the world around them, whether that information be in the form of images or video. It is the technology that has brought autonomous vehicles and many other important solutions to market.

In the retail space, cameras and sensors, equipped with computer vision algorithms, are used to track a customer from the moment they walk in the store, detect the products they have chosen to purchase, and process the transaction at the point of leaving the store, all while providing a cashier-less experience.

To understand this better, let’s walk through an example from start to end. When a customer walks into a cashier-less store, they usually scan in with a mobile application associated with the store. This is where the computer vision algorithms begin their work, as they identify the customer and track them throughout their shopping experience in order to accurately identify the products they’ve selected.

The customer begins browsing the store’s selection, looking at various products, comparing them, and making decisions. The computer vision algorithms now have to keep track of the customer as they make their journey through the store. When a customer selects a product from the shelf, the algorithm recognizes that decision, and unless the customer puts the item back on the shelf, they consider that action an intent to buy. This is similar to adding a product to a cart while shopping online. The algorithms will then continue to track the customer’s journey, making note of what items have been selected. Once the customer has gotten everything they need, they exit the store without interacting with a cashier, as one would do at a regular store. The computer vision system simply recognizes that they are leaving, and processes the transaction on their way out.

Challenges

The computer vision tasks associated with making cashier-less stores function, such as object detection, image segmentation, and people tracking, can be extremely complex. There are a wide range of scenarios that need to be accounted for during the development of these models.

For instance, as the cameras track the customer from when they enter the store, they must reliably keep track of the same person, and log that person’s activities (picking items from the shelf or putting something back) correctly. Problems can arise when two people are standing close to one another, as the system may begin to have a hard time distinguishing between the two. The camera’s view of the person may also be blocked by someone or something in the store.

Another important aspect of the customer’s behavior that needs to be correctly identified is the act of picking something off the shelf versus putting something back on the shelf, as each action triggers something very different. Picking something off the shelf signals the system to add it to their cart, while putting something back on the shelf indicates taking it out of the cart. Getting this wrong can cause the customer’s cart to be incorrect.

When it comes to object detection, the computer vision algorithms need to identify the objects the customers interact with correctly. This can be challenging due to similar items being placed close together, such as two different brands of toast bread. The system can easily mistake a $3.99 pack of toast bread from brand A, for a $5.99 pack of toast bread from brand B. Issues such as lighting and damaged products can also cause the system to perform poorly.

To tackle some of these challenges, CVPR, one of the largest computer vision conferences in the world, during the last few years has dedicated an entire workshop to this area called “RetailVision” to encourage researchers to tackle these problems. However, not all solutions will have to wait for better algorithms to be developed. At the data level, some companies have enlisted the help of simulators to create artificial training data in order to be able to capture the dynamic nature of a retail store environment. This allows them to add more training data to the model, especially around scenarios where the model may be performing poorly. Model performance monitoring systems also have an important role to play in identifying the areas where the computer vision systems will perform poorly.  

The Future of Computer Vision in Retail

Cashier-less stores are likely to become more and more common in the coming years. They solve real problems in the shopping experience, and make things more efficient for businesses. In order to see wide-scale adoption, the cost of the technology will need to continue to come down, which it likely will as the technology continues to develop and more companies enter the space. It is not just tech giants such as Amazon that are building cashier-less systems. Several start-ups, such as Grabango, Scandit and AiFi have entered the space as well. More competition will continue to ensure a healthy development of the technology and drive prices down, leading to greater adoption.

Stay up to date !

Subscribe to our newsletter to get inbox notifications.
Sign up to our newsletter ↓
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.