.jpg)
Amazon just unveiled a grocery store without lines or checkout counters. Amazon Go, a 1800-square-foot retail space located in the company’s hometown of Seattle, lets shoppers just grab the items they want and leave; the order gets charged to their Amazon account afterwards.
Amazon Go works by using computer vision and sensors to detect what items you’re taking out of the store. You start by scanning an app as you enter the Amazon Go shop. You do your normal shopping, and the sensors throughout the store identify the items in your cart and charge them to your account when you walk out the door. It’ll feel like shoplifting, except you’re actually being watched by more cameras than you can imagine.
By now many have heard of Amazon’s audacious attempt to shake up the retail world, the cashless, cashierless Go store.
Will it be successful? What if someone was to try to shoplift something and catch these complacent Amazon types napping. But it becomes clear after entering that this isn’t going to be an option. They have already provided against such crude attacks on their system.
As you might have seen in their promo video, you enter the store through a gate that opens when you scan a QR code generated by the Amazon Go app on your phone. At this moment your account is associated with your physical presence and cameras begin tracking your every move.
The many, many cameras.
One might wonder when the idea of Amazon’s cashierless store was first proposed how it would be accomplished. Cameras on the ceiling, behind the display cases, on pedestals? What kind? Proximity and weight sensors, face recognition? Where would this all be collated and processed?
Amazon’s approach isn’t as complex as one might expect. Mainly the system is made up of dozens and dozens of camera units mounted to the ceiling, covering and recovering every square inch of the store from multiple angles. One might guess there are maybe a hundred or so in the store, which was about the size of a gas station mart.
These are ordinary RGB cameras, custom made with boards in the enclosure to do some basic grunt computer vision work, presumably things like motion detection, basic object identification, and so on.
They’re augmented by separate depth-sensing cameras (using a time-of-flight technique) that blend into the background like the rest, all matte black.
The images captured from these cameras are sent to a central processing unit, which does the real work of quickly and accurately identifying different people in the store and objects being picked up or held. Picking something up adds it to your “virtual shopping cart,” and you can pop it in a tote or shopping bag as fast as you like. No need to hold it up for the system to see.
This is where the secret sauce is. As banal a problem as it may seem to determine which similarly dressed person picked up which nearly identical yogurt cup, it’s very difficult to get right at the speed and accuracy level needed in order to base an entire business on it.
A student, after all, with the resources available these days, could probably design a version of this store in a few weeks that would work 80 percent of the time. But to get it right 99.9 percent of the time, frictionlessly and instantly, is a challenge that requires a great deal of work.
Notably, there is no facial recognition used. Amazon perhaps sensed early on that this would earn them rebuke from privacy-conscious shoppers, though the idea of those people coming to Amazon's store strikes one as unlikely. Instead, the system uses other visual cues and watches for continuity between cameras — you’re never not in sight of a lens, so it’s easy for the system to see a shopper move from one camera to another and make the connection.
Should there be a technical problem with a camera, the system doesn’t break down entirely. It’s been tested with cameras missing, though naturally it wouldn’t be long before a replacement is put in place and the system calibrates itself.
In addition to the cameras, there are weight sensors in the shelves, and the system is aware of every item’s exact weight.
Most people aren’t shoplifters, and the system is designed around most people.
There is in fact a human in the loop should the system find itself in a bind. The difficulty of monitoring the store doesn’t increase with square footage, though of course you’ll need more cameras and more processing power.
It’s also been tested with serious crowds; during a lunch rush, when dozens rather than a handful of people could be found walking in and out without doing anything more than showing their phone to a sensor at the entrance.
There may not be cashiers, but there are staff: stockers who replenish inventory; an ID checker in the wine and beer section, and chefs in the back throwing together fresh sandwiches and meal kits. Someone also hovers in the entrance area to help people with the app, answer questions, and take returns.
The selection was mainly grab-and-go lunches and snacks, with the usual handful of household items you grab at the convenience store on the way home. Prices were what you’d expect at a supermarket rather than a convenience store, though.
As for the expected Amazon gambits that leverage its existing properties and hooks, few are to be found. The app is self-contained, and your purchases are tracked there rather than on your “main” Amazon account. Prime members don’t get lower prices. Whole Foods has a little section of its own but there’s no broader partnership.
One would be impressed with the seamlessness of the system, and could see these things successfully operating here and there.
On the philosophical side, of course — a convenience store you just walk out of is a friendly mask on the face of a highly controversial application of technology: ubiquitous personal surveillance.
It’s a bit overkill, to replace a checker or self-checkout stand with a hundred cameras that unblinkingly record every tiny movement.
Like so many ways companies are applying tech today, this seems to be an immense amount of ingenuity and resources being used to “solve” something that few people care about and fewer still consider a problem. As a technical achievement it’s remarkable.
The store works — that much one can say for it. Where Amazon will take it from here one couldn’t say. Amazon Go will be open to the public starting this week, but whether anyone will find it to be anything more than a novelty is yet to be seen.
