03/09/2019 - CereProc and LEGO collaborate to bring audio instructions to visually impaired LEGO creators
The Challenge: How to bring LEGO to those with visual impairments
Founded in 1932, LEGO has been making children’s toys for almost a hundred years, introducing the first plastic LEGO brick in 1947. Since then, it has become the world’s largest toy company with revenues exceeding £3bn in recent years. However, until recently, it was hard for those who are partially sighted to enjoy LEGO, with many instruction booklets featuring intricate instructions, often in diagrams without accompanying words.
This challenge was particularly acute for Matthew Shifrin from Massachusetts, who was born blind. Matthew had loved Duplo since his parents bought him a set when he was four years old, and it was only a year later that his parents introduced him to LEGO. However, Matthew was unable to assemble formal sets without the help of a parent – and even then, the process was incredibly time-consuming.
On Matthew’s thirteenth birthday, his friend Lilya created a set of Braille instructions to a LEGO version of a Middle Eastern castle. When Lilya passed away in 2017, Matthew approached MIT about formalising the Braille instructions and the team in turn connected him with LEGO and the Austrian Research Institute for AI.
LEGO and CereProc collaborate to bring audio guidance to LEGO bricks
In response to Matthew’s request, the team at LEGO developed LEGO instructions, providing Braille and audio instructions like “Find two bright orange roof tiles 2x2” that can be understood without necessarily needing to see the bricks. However, the LEGO team needed a voice for the audio instructions, and it made sense for this to be synthetic; hiring an actor for all the necessary permutations of bricks would be incredibly time consuming, not to mention boring. Some LEGO sets require hundreds of pages of instructions in total, and a system that could convert text to speech was a much smarter choice.
The team at the Austrian Research Institute for AI had worked with Text-to-Speech experts CereProc before and recommended them to LEGO. CereProc has produced a number of bespoke synthetic voices including voices for Motor Neurone Disease sufferer Peter Scott Morgan, as well as American News Anchor Jamie Dupree and the comedian Lost Voice Guy. It specialises in creating characterful synthetic voices, including a range of regional accents from around the world and emotional, singing and fantastical voices for escape rooms and computer games.
Sarah, LEGO’s synthetic voice, goes live
The LEGO team chose ‘Sarah’ to voice the application, a synthetic voice from south-east England, designed to be comprehensible to a broad variety of people from around the globe.
“Choosing Sarah as the voice for this application was an excellent decision,” says Paul Welham, CEO CereProc. “The English voice will be heard around the world, so it was important that it could be customised and wasn’t too strongly accented. The kit allows LEGO to add emotion to the voice, speed it up, slow it down, change the pitch or change the lexicon. This is particularly important for LEGO’s specific needs; for example, without a hyphen in the text-to-speech system, “minifig” sounds like “minif-ig” but if you use “mini-fig” it is much more fluid and natural.”
“This is extremely important for blind children because there aren’t a lot of places where we can say, ‘Look Mum and Dad! I built this on my own… I did this’, says Matthew Shifrin. “For blind children, we don’t have access to what sighted kids are used to. LEGO bricks enable us to learn about our environment, to see the world. It is so important because blind kids get left out of a lot of social stuff, especially in elementary school. But LEGO building is one of the things we can do.”
The software development kit from CereProc also allowed LEGO to install the voice into its application with just a few lines of code, rather than linking out to a cloud service. This means that children and adults everywhere can enjoy creating LEGO structures and settings, regardless of internet access.
“This fantastic initiative is a major step forward for people living with visual impairment,” concluded Welham. “By adopting Text-to-Speech technology in this way, LEGO will not only improve the lives of those using its products, but hopefully also inspire other companies to follow suit with characterful synthetic speech. CereProc is incredibly proud to have supported on this project helping children with vision impairment to gain independence.”
In future, LEGO hopes to add more languages to the application, allowing blind and partially-sighted people all over the globe to play with and enjoy LEGO.
“Matthew’s story demonstrates the power of LEGO play. It brings people together, helps to build confidence and sparks creativity. It has been an honour to work with Matthew, his passion and energy are truly inspiring. But most importantly his project will help visually impaired children around the world experience the same joy of building and pride of creation that all our fans feel”, says Fenella Blaize Charity, Creative Director, LEGO Group.