Tech

Inside Google's AI-first approach to its new hardware

Google’s AI-first mantra is bearing fruit, with of latest crop of Made By Google hardware benefiting from the guiding hand of machine learning.

Google’s continued shift from a mobile-first to an AI-first approach to computing has seen the technology giant rethink all its major projects, says chief executive Sundar Pichai.


Google unveils two new phones

Google launches the second generation of its Pixel smartphone along with new voice-enabled home speakers, a new laptop, earbuds and a tiny camera. Vision courtesy Google.

“It is not just about applying machine learning in our products but it’s radically rethinking how computing should work,” Pichai says. “People should be able to interact with computing in a natural and seamless way.”

“In an AI-first world, I believe that computers should adapt to the way people live their lives, rather than people having to adapt to computers.”

This philosophy shines through in the latest batch of Google devices: from the Pixel Buds wireless earbuds which leverage machine learning to offer real-time translations in 40 languages, to the Google Clips smart action camera which automatically looks for perfect photo opportunities using AI.

Meanwhile the talkative Google Assistant is coming to more devices, including the Pixelbook hybrid notebook and two new smart speakers to sit alongside the original Google Home speaker.

Machine learning is also helping Google’s engineers design new hardware and overcome old challenges in new ways, according to Google VP of Product Management Mario Queiroz.

Google’s new Pixel 2 and Pixel XL 2 flagship smartphones capture portrait photos with soft backgrounds using a single lens, whereas many smartphone rivals require two lenses to gain an accurate sense of depth.

Rather than retro-fit AI to existing hardware to solve this challenge, Google’s engineers designed new hardware to leverage the strengths of AI. The new Pixel smartphones rely on a camera sensor featuring dual pixels, using the tiny difference in viewing angle between adjacent pixels to calculate a sense of depth.

“We apply a lot of math and machine learning to use that tens of microns of difference in perspective from each pixel to generate the beautiful portrait shots from both the front camera and from the back camera”, Queiroz says.

“That really was about our researchers saying ‘what information do we need in order to create beautiful photos through our machine learning work’, and then defining the hardware based on that. It really is an example of AI-first and we’re going to continue to do that in a lot of different areas.”

Machine learning was also used to redesign the lenses in the new Daydream VR viewer, which now offers users a wider field of view and broader sweet spot, making it easier for users to get the viewer to rest comfortably against their face.

Advances courtesy of machine learning are also allowing Google to branch into new product areas and offer something different in order to stand out in a saturated market, such as the Google Clips smart camera.

While it looks like yet another action camera to rival the likes of GoPro, the Clips is designed to capture candid family photos rather than daredevil feats. Google’s tiny camera relies on a combination of machine learning, face recognition and image processing to automatically capture photos, allowing the family photographer to remain in the action rather than behind the camera.

Without the advantage of AI, Google would not have bothered entering the crowded camera market, says Google Senior VP of Hardware Rick Osterloh.

“With Google Clips we wanted something that we felt could really make the user’s experience better through the application of our technology,” Osterloh says.

“We had a team of researchers that had been building on-device image processing machine learning, and we thought the camera would be a great way to showcase that technology and bring it to life.”

Of the hardware announced this week at Google’s event, the two Pixel phones, Google Home Mini, Pixel Buds and new Daydream VR viewer are coming to Australian shores soon. The Pixelbook, Clips Camera and Google Home Max will be exclusive to the US for now.

The author travelled to San Francisco as a guest of Google.

Show More

Related Articles