Visual Inspection

Implemented for GlobalFoundries

16 June 2023
Production process of mirrors in lithography optics

Predictive maintenance for automated guided vehicles

Unnecessary production downtimes caused by mechanical faults in automation technology can be avoided through visual inspection of automated transport systems. In semiconductor manufacturing, optical sensors ensure even more precise defect analysis of rail vehicles in the clean room.

Visual Inspection Project Team

Visual Inspection Project Team

ZEISS Digital Innovation tasks

GlobalFoundries Fab 1 in Dresden is the largest semiconductor factory in Europe. Overhead Transportation Vehicles (OHV) transport the wafers in the clean room via the Automated Material Handling System located in production. As part of the Digital Product Factory, the Smart Systems Hub worked with various hardware and software partners to develop a solution that uses acoustic sensor technology and a newly created cloud infrastructure to detect anomalies in the chassis of these vehicles, thereby minimizing downtime and optimizing maintenance cycles.

To avoid possible failures of the transport systems, the project is being further enhanced by a team from ZEISS Digital Innovation. The team is developing software integrated into the cloud environment for this purpose, which uses cameras to identify further deviations and faults on the vehicles. With the help of this visual inspection, which analyzes camera images in real time on an industrial edge PC, important information such as the degree of wear and position deviations of the guide wheels or contamination can be detected.

This information is used to calculate an individual “health score” for each vehicle and a dashboard is used to automatically derive recommended actions for timely maintenance. All analysis data will be transferred to the existing Data Collection Kit in order to extend the machine learning algorithm accordingly..

Benefits for the customer

Defective OHVs can damage the rail system in the factory, causing unnecessary as well as expensive repairs. The implementation of the sensor-to-cloud infrastructure as a predictive maintenance solution safeguards factory availability. Thanks to visual inspection, possible anomalies on the chassis of the transport vehicles can be reliably and automatically detected, such as lost circlips, bent wheel axles or defective bearings.

Based on both the acoustic and visual sensor data, the OHVs can be serviced in good time. In addition, during predictive maintenance, the system continues to run as normal, without this affecting it.

Many are working on Industry 4.0 solutions, but there is still great potential in Maintenance 4.0. To implement true predictive and condition-based maintenance strategies, there is often a lack of human perception. If we digitalize the senses, we can be one step ahead. This means that these capabilities, in combination with AI components, can be available on a non-stop and flexible basis to monitor our system on a condition-based basis. The technologies we are developing with ZEISS Digital Innovation will have a future-oriented impact on the high requirements for fail-safety in fully automated semiconductor manufacturing.

Lars Fienhold

Principal Analyst Factory Automation, GlobalFoundries

Checking the demo vehicle in the test laboratory

Checking the demo vehicle in the test laboratory


Because of the high transport volume and the complex automation technology, regular inspection of the individual vehicles is very costly and resource-intensive. After the monitoring system had already been improved using position and acoustic sensor technology, this was due to be extended to include visual inspection. In the process, the system is to be seamlessly integrated into the existing hardware and software infrastructure.

A particular challenge is to detect even the smallest defects during normal operation. The limited space between rails and ceiling, the high vehicle speed and short transport cycles, as well as the special requirements for systems in the cleanroom have to be taken into account. As defects in production are a rare occurrence, very little data is available for machine learning algorithms.

Schematic overview of the Health Scores

The Health Scores result from the acoustic signals, the camera monitoring and the existing database information (click to enlarge)


In addition to the software component, the selection of suitable hardware components is also crucial. A conventional model-based computer vision approach combines geometric, photometric and statistical information. Image processing and fault detection run on an industrial edge PC connected to AWS cloud services. Communication with the cloud platform tracks the vehicles and enables connection to additional sensor data. That’s how an overall health value can be calculated for each vehicle.

Our Services

  • Planning and integration of the camera solution/lighting geometry
  • Development of the analysis software and integration into the existing cloud architecture
  • Development of the scoring model in the area of visual inspection
  • Development of the algorithms and elaboration of the error model
  • Computer vision system solution with transmission of analysis data

Technical environment

AWS, C#, C+, Python, OpenCV

About our customers

GlobalFoundries (GF) is a global foundry company for semiconductor manufacturing with around 13,000 employees providing production and technology services to its globally active customers. GF operates production facilities in Germany, Singapore and the US. These factories are supported by a global network in research and development, design enablement and customer services in Europe, America and Asia.

GF Fab 1 in Dresden manufactures innovative semiconductor products for its customers in 22nm, 28nm, 40nm and 55nm technologies and is the largest semiconductor plant in Europe with a clean room area of 52,000m². GF’s expertise in semiconductor manufacturing and its ongoing commitment to research and development have helped to establish Saxony as the leading micro- and nanoelectronics center in Europe. Currently, around 70,500 people work in 2,500 high-tech companies in the region.

Share this article