Google Lookout, the accessibility app that aids the blind and visually impaired identify objects through the use of their phone camera, has been upgraded with some new features and improvements. Originally released in 2019 in the US for first-party Pixel phones, Google Lookout was created to help those with limited visual capacity navigate the world through the use of AI. When pointed at objects or animals, it would attempt to identify them.
The new update brings with it two notable features: Food Label and Scan Document.
“With Food Label, you can quickly identify packaged foods by pointing your phone’s camera at the label,” a Google blog post explains. “Lookout will guide you to position the food product so that it can be properly identified through its packaging or barcode. This can be particularly helpful if you’re putting away groceries and want to make sure you’re handling the right items that might feel the same to your touch. For example, Food Label would be able to distinguish between a can of corn and a can of green beans.”
Note, however, that Food Label isn’t available in all countries yet, and is still in Beta.
For many, this will prove monumental.
“Food labels can be challenging for anyone with a visual impairment, as they are often designed to be eye-catching rather than easy to read,” the UK’s Royal National Institute of Blind People (RNIB) told the BBC.
The other new feature, Scan Document, “makes it easier to go through letters and mail,” the blog post explains. By taking a snapshot of a document, the in-app AI will then attempt to recognize and read the whole text for you. Note however that Arabic is not supported yet.
Based on feedback from the blind and low-vision community, Google is also introducing an accessible design that is even more compatible with TalkBack, Android’s screen reader. This includes making more space for the camera view, which makes it easier to frame the object in question. In addition, navigation between modes has been made to require fewer taps and less time by making it possible to scroll between them at the bottom of the screen.
Of the originally existing features, there were three.
One of those is Quick Read, which is a more streamlined version of Scan Document meant for shorter pieces of text such as mail.
Another existing feature is Currency mode, which identifies banknotes and their value. This currently works on USD only, however.
Explore mode describes the user’s surroundings, including objects and text.
“We designed Lookout to work in situations where people might typically have to ask for help—like learning about a new space for the first time, reading text and documents, and completing daily routines such as cooking, cleaning and shopping,” Google explains in a 2019 blog post.
Google recommends the visually impaired and blind to hang their phone from a lanyard around their neck or placing it in a shirt front pocket.