ZEISS team develops scanner for photorealistic 3D models of frames in unprecedented quality

It's a fact that people also buy glasses online. However, in quick virtual fittings of off-the-shelf frames, it can happen that details are missing and the quality is inadequate. Not so with the digital frames used in ZEISS Virtual Try-on. Armin Eichert, physicist and project manager for the frame scan at ZEISS, explains how the scanner was developed and what makes ZEISS digital frames so special.

It all started in 2017 with a ZEISS venture, an internal start-up, aimed at producing high-quality, photorealistic 3D scans. As the specific use case the eyeglass frame scanning was identified. Therefore, the ZEISS Vision Care business group was where this venture then evolved. Everyone really pulled together and worked on the scanner and software as if it was a start-up within ZEISS. This was done in collaboration with scientists from the ZEISS Innovation Hub @ KIT, as well as the design department within our R&D division and software developers from ZEISS Vision Technology Solutions in Bangalore. The core team consists of physicists and computer vision scientists, i.e. specialized computer scientists. Right from the start, our ambition was to develop the best solution – to get as close to reality as possible. However, it is not so easy to scan frames, so we really tried out a lot of things and tested various scanning layouts and processes in order to achieve our quality standards. In addition, the scanner system had to be scalable.

The digital frames are used in ZEISS Virtual Try-On. This is the digital frame try-on that can be used on-site at the specialist retailer, from home or on the go. The digital frames are superimposed onto the user's own 3D avatar. All with a quality that allows you to order glasses, even progressives, from home.

A 3D scan is always a snapshot, so everything has to be just right at the moment when it is captured. Frames are not suitable for a simple scan because the material is either partially transparent or highly reflective. As a result, establishing suitable lighting conditions in the scanner is a complex task. Uniform lighting in the scanner is absolutely essential and this ultimately presented the greatest challenge. Moreover, there is huge variance between frames and their materials, all of which behave differently. Another challenge, for example, was to create suitable environmental conditions for the frame digitization so that the same high-quality capture could be maintained across all frame model types. This was achieved through a great deal of optical know-how, which was tested in various prototypes. Digital frames were demonstrated to customers for the first time at ZEISS Convention 2019.

They were enthusiastic even back then. Two more scanner prototypes were created in collaboration with the design department within our research and development division. Using the final scanning process, we have so far generated over 5,500 high-quality, precise, and true-to-reality 3D frame models. To offer digital frame try-on as a specialist retailer, ZEISS VISUFIT 1000 Platform and the associated "Virtual Try-on" module are required as a basis. This is currently available in more than 30 countries.

The details are obviously a trade secret. What we can say, however, is that cameras with ZEISS lenses are used in the process. We perform a variety of measurements on the frames using these cameras and other hardware. The results of these measurements are then automatically transferred to a largely photorealistic 3D model using specially developed software.

No, it is not fully automated at the moment. Once the first near-photorealistic version of the 3D model has been created, the human quality check starts. Experienced graphic designers that form a worldwide network look at the model, improve it, retouch where necessary and ensure that it actually is very close to reality.

Definitely the degree of detail and precision. We have carried out comparisons and can say that we deliver the highest quality in the market in the field of virtual frames. Specifically, this means that the smallest details, engravings, or surface reliefs, structural changes of the material can be viewed using the 3D model. The reflection of the ambient light on the material is reproduced 1:1. Of course, it is still different from physically holding the frames in your hands. But our models come very close to reality. Another point illustrates the emphasis we place on accuracy: after all, the scanner was developed for use in the industrial environment, where more powerful movements can occur. The housing design meets the highest stability demands in order to keep the cameras absolutely stable in their capturing position. This is because even the smallest changes to the mechanical setup had to be re-calibrated.

The scanner we have developed is currently optimized for eyeglass frame scanning exclusively and is designed precisely for this purpose. However, if developed, the technology has the potential to produce photorealistic scans of other objects also.

Using pure in-house know-how, we have created a true key technology that can be scaled. This means we have developed something that can be replicated. We are able to scale hardware, software and the processes, basically the entire system, and thus make it available at other locations, too. We are planning additional scanners at various locations so that we can act quickly worldwide. My goal is to populate our frame catalog with thousands of frames per month in the future.

Yes, it was indeed a very fast development, but we had and still have the management's commitment and support and developed the crucial knowledge on our own. It's also really fun to be a pioneer and to work on a new task like this with the team. There was never any silo thinking on this project. Ultimately, our high-quality frame models help our customers in ophthalmology to compete successfully on the market. This has been and continues to be a great experience for all of us.

Since the photorealistic 3D frame models are used in ZEISS Virtual Try-on and ZEISS exclusively, their advantages also apply. The following can be specifically highlighted at this point:

• Digital frames as part of ZEISS Virtual Try-on (@Home) enable eye care professionals to implement an omnichannel offering. They can set themselves apart from the online competition by means of the unrivaled quality of these digital frames, which can be tried on at a specialist retailer or at home/on the go. They also expand their frame portfolio without having to increase their stock.
• Frame manufacturers can rest assured that ZEISS digitally represents their frames in the highest quality. Frame manufacturers continue to tap into potential new customer groups that are adding certain brands to their portfolio for the first time as digital frames.
• ZEISS enables consumers to purchase high-quality eyewear online with the help of digital frames by means of ZEISS Virtual Try-on and ZEISS Virtual Try-on @Home, still connected to their ECP.