A collaboration between British AI company GiveVision and Austrian company Catalysts has resulted in life-changing improvements to headset technology for visually impaired people, thanks to image processing technology used on ESA space projects including Sentinel-3.
Hundreds of visually impaired people in the UK are now able to see clearly again using augmented reality headsets, thanks to expertise gained through high performance processing of satellite imagery. The latest headsets from GiveVision were significantly enhanced thanks to an introduction to Austrian company Catalysts made through ESA Space Solutions network.
Catalysts, which is now part of IT service company Cloudflight, has worked on several ESA space projects including Sentinel-3. The skills and knowledge it shared with GiveVision resulted in a new version of its low vision aids that has been described by users as “amazing” and “life-changing”.
Maisy lost most of her vision around six years ago due to a brain tumour. At the Hay Festival in the UK, she was given the opportunity to use the SightPlus headset for the first time in front of a live audience during an interview with the BBC.
SightPlus enables people who have lost the central portion of their vision, due to conditions such as macular degeneration or diabetic retinopathy, to see in near real time. It does this by processing incoming images and projecting the augmented feed onto the working part of their retinas. The solution, which can be adjusted by the user, works at multiple focal lengths and replaces magnifier-style options that only work at close distances. It also works for people with conditions affecting the whole visual field, including albinism and optic neuritis.
Turning functional into usable
By early 2018, the team at GiveVision had developed a fully working version of its augmented reality headset, based on a smartphone. The next stage was to reduce both the processing time and the load on the processor, in order to significantly reduce lag and heat output and increase the battery life.
It was at this point that GiveVision was introduced to Catalysts through ESA’s Technology Broker Network in the UK and Austria, as part of the ESA Space Solutions network.
“GiveVision had a very clear technical challenge that they wanted to overcome,” says Helen Rogerson from the Science and Technology Facilities Council, ESA Space Solutions UK broker. “That’s exactly what we aim to help resolve with the ESA Technology Broker Network. I directed them to ESA’s Space Solutions portfolio where they identified a technology offering what they needed. Our Austrian broker Brimatech then put them in touch with Catalysts.”
“The broker involvement at this stage was critical,” explains Stan Karpenko, GiveVision’s CEO.
“Our users need to receive the augmented video feed as if they’re looking at it with their own eyes, so there needs to be no lag. We were struggling to do this and to reduce the load on the smartphone’s chips. Catalysts, who we found through the Space Solutions programme, optimised the code so well that we now run our advanced vision processing algorithms at a fraction of the speed.”
From space to sight
The development by Catalysts was funded by ESA’s Technology Transfer and Patent Office through its proof of concepts call.
“We analysed and optimised the core algorithms using know-how taken directly from our work on space projects, including the CAWA project,” notes Alessio Montuoro at Catalysts.
CAWA is the Advanced Clouds, Aerosols and Water Vapour Products for Sentinel-3/OCLI. Led by Catalysts, the CAWA consortium is developing and improving algorithms related to atmospheric measurements. The company’s skills and know-how relating to high performance image processing are also used in Earth observation (EO) projects covering crop and wetland monitoring.
“With something like SightPlus, a high number of frames per second gives the image smoothness, low latency helps avoid nausea, and consistency in processing removes variability in the image quality,” explains Montuoro.
“Despite the relatively short project time, we managed to reduce the image processing times from 250-450ms per image to around 50ms, which is far more usable. Our work also cut down the heat that’s generated, which makes the headset more comfortable to wear, and has extended the battery life.”
Providing privacy and independence
Over 400 people in the UK are now living with the enhanced SightPlus device. People using earlier versions can now see more, and see it more clearly. Others for whom the original version was too weak can now use SightPlus: this will benefit many potential users whose sight will continue to degenerate, because the upgraded version will provide a solution for longer.
Globally, there are over 191 million people suffering from moderate and severe untreatable sight loss. The substantial performance enhancements from Catalysts’ work will support GiveVision’s aim of developing a headset that looks as close as possible to a pair of spectacles, rather than being based on a smartphone. This design approach augments the natural field of view without blocking any peripheral vision, and will be lighter and more comfortable; all of which should help to significantly increase the rate of user adoption of SightPlus.
About the Technology Transfer and Patent Office
The TTPO's funding for demonstration activities aim at proving the relevance of transferring a given technology or know-how into the non-space context, reducing the technical risk and confirming the market opportunity. An annual open call invites the submission of proposals from industry for Feasibility Studies, Proof of Concepts and Demonstrators. These have been designed as a funnel of activities, de-risking the activities on a step-by-step basis. Please see here for more information.
About ESA Space Solutions
ESA Space Solutions is the go-to-place for great business ideas involving space in all areas of society and economy. Our mission is to support entrepreneurs in Europe in the development of business using satellite applications and space technology to improve everyday life. Funding typically ranges from 50KEuro to 2MEuro and supports everything from early stage incubation programs, feasibility studies to large-scale demonstration projects.