Unilever builds on Zappar AQR roll-out as it trials Be My Eyes tech for blind and low vision shoppers

Unilever has partnered with Be My Eyes to introduce AI assisted cooking for blind and low vision shoppers.

This builds on the addition of on-pack Accessible QR (AQR), developed by computer vision specialists Zappar, to Unilever’s Persil and Colman’s products in the UK last year and is part of the company’s global connected pack strategy, which includes using new digital experiences and technology to evolve and differentiate the way shoppers interact with and use its products.

Unilever is initially trialling the Be My Eyes technology in the UK through its Colman’s Singapore Noodles Meal Maker, with the intention of rolling it out across more brands in 2024.

Scanning the AQR code on the front of pack, shoppers can now be directed to the Be My Eyes app for free and connected to either a volunteer to read and describe cooking instructions or to a virtual AI chatbot which uses Chat GPT-4 capabilities to address recipe or cooking questions.

This is the first time the Be My Eyes AI technology has been integrated with a food product, offering an AI assisted cooking experience at home.

Unilever is working with Be My Eyes to train its volunteers and provide specific Colman’s product information and data to the Chat GPT-4 AI bot.

Unilever’s customer support team will also be integrated into the Be My Eyes app experience, allowing shoppers the option to be automatically connected to the Colman’s careline team for further support if needed.

Rachana Dongre, Senior Digital Engagement and Strategy Lead Nutrition & Ice Cream, Unilever says: “We’ve accelerated digitising our packs to offer new opportunities for brand engagement and elevated shopping experience and now we’re also focusing on how we can also use digital experiences to make our products more accessible.”

“Zappar’s AQR codes means we can support blind and low vision shoppers to have equal access to information and integrating Be My Eyes into these codes offers a totally new way to make the full experience of our products more inclusive, from the shopping aisle right through to cooking at home.” 

“The 285 million blind and low vision consumers in the world buy the same products as everyone else,” says Be My Eyes CEO Mike Buckley,

“The difficulty in reading package ingredients and cooking instructions has long marred the cooking experience of these consumers. Unilever’s commitment to use new Zappar technology with Be My Eyes is pioneering a new era in product accessibility and disability convenience.”

Caspar Thykier, Zappar’s Co-founder and CEO, says: "Our integration with Be My Eyes is a really significant step towards making access to information a basic human right.”

“By combining Zapvision's computer vision technology with Be My Eyes' new OpenAI, ChatGPT4 Vision integration, we're excited to underpin a more conversational and assistive experience for Be My Eyes users with unambiguous, structured and accurate data.”

“This is the start of a new type of ‘product as a service’, levelling the playing field still further for food brands such as Unilever’s Colman’s products. We’re so excited for our continued partnership with Unilever and now Be My Eyes, providing further access to information for the blind.”

The new Be My Eyes service is integrated into the AQR codes Unilever began adding to its products last year, including Colman’s and Persil products in the UK.

The AQR codes work by adding a dot-dash pattern around one corner of the code, which make them detectable from further away.

They an be scanned through accessibility apps from 1.1 metres (over 7x that of normal QR scanning) to help blind consumers find their desired product in sore or at home.

Unilever uses these AQR codes to provide product information such as usage direction, recycling, and ingredient information and nutritional value, interacting with standard smartphone accessibility through the Be My Eyes app to share information via audio description or displaying information in larger text.