Cozmo, the toy robot putting AI at our fingertips

When playing with Cozmo, Anki’s palm-sized artificial intelligence robot, it’s easy to forgot all of the engineering and software running behind the scenes. Every action, from Cozmo’s audible chirps of victory when it wins a game to its childlike mannerisms when it recognizes your face, conceals tens of thousands of lines of code.

While Cozmo sleeps, it snores. The small robot — shaped like a miniaturized bulldozer with a CRT monitor for a cockpit — sits in a charging dock, waiting to be awoken. Like Pixar’s adorably anthropomorphic WALL-E, Cozmo falls somewhere between a Mars rover and an animated woodland creature. It’s lifelike enough to evoke sympathy, but still enough of a toy not to teeter too close to the uncanny valley. 



With the tap of a smartphone screen, Cozmo comes to life. It makes a subtle motion to indicate it’s shaking off its slumber and begins wheeling over to the edge of the table. When it gets too close, it slams to a halt and looks down over the cliff, emitting a series of terrified chirps. When it wheels back and reorients itself, Cozmo takes a hard look at the other faces in the room. Some are new, but others it remembers from before it fell asleep.



The robot is the latest creation from Anki, a Silicon Valley toymaker best known for building small race cars you can control using a mobile app. The company was founded in 2010 by a trio of Carnegie Mellon graduates with PhDs in robotics. Anki has always considered itself an artificial intelligence and robotics company, even if the average consumer could only see a toy car racing around a track. But now, with Cozmo, there is no doubt. Anki’s first robot fits in the palm of your hand, and it also happens to employ some of the most sophisticated AI software ever made available to consumers. 


The Anki crew has been thinking about Cozmo since well before Sofman’s Drive demo at WWDC. From 2005-2010, Sofman was a PhD student in the well-regarded robotics program at Carnegie Mellon (which is now maybe most famous for being the place Uber raided for talent when it started developing self-driving cars). He, along with classmates Mark Palatucci and Hans Tappeiner, wanted to do something unique with their research.

Tappeiner and his colleagues, a gaggle of PhDs who emerged from the robotics group at Carnegie Mellon University, will tell you much the same thing. Like so many others in the field, they admire the impressively mobile robots created by Google-owned Boston Dynamics, whose dog- and human-like droids radiate mechanical charisma. But Tappeiner questions how long it will be before these robots are genuinely useful. “Does it really make sense for us to create a farming robot—or will it take 20 years to really do that well? We can do this,” he says, nodding at Cozmo, “really incredibly well.”

What’s more, he believes, Cozmo can provide a seedbed for the future. Offering tools that even kids could use, a kit like Cozmo’s SDK could help breed a new generation of robotics researchers. But it also gives seasoned robotics researchers a path into the heart of this toy automaton, and that can help advance today’s work. “When we were in grad school,” says Anki CEO Boris Sofman, “you would have to pay $10,000 for a platform that had 10 to 15 percent of the capabilities of Cozmo.”

Cozmo is fully based on computer vision and deep learning. The robot sees the world through a single camera in its face, hidden in a slot that’s meant to look like a mouth. The camera runs at 15 frames per second, sending the footage to your phone, which does all the processing before sending instructions back to the robot. So Cozmo will always have as much processing power as that fancy new computer in your pocket. The downside, of course, is that you need a phone nearby when you’re playing with the li’l bot. The phone trick didn’t solve all of Anki’s problems, either: Stein spent years working out how to compensate for the latency that comes with sending data back and forth.

It would be impossible to hard-code every imaginable playplace into the system, which is why machine learning has become such a crucial part of Anki’s efforts. “A lot of situations where you invoke machine learning,” says Michael Wagner, a CMU robotics researcher who amazingly is not involved with Anki, “are because you don’t really understand what the system should do. How should it prefer to drive over rough terrain versus smooth terrain? You don’t know. So you throw machine learning at it.” Lots of testing, lots of training, and the system figures out how to react by itself. Many of what Anki’s dealing with are standard robotics challenges, but no one’s ever solved them for this kind of product. This robot doesn’t have to be perfectly efficient, like an assembly-line worker. This robot has to be fun.