Best Practices for Implementing Deep Learning for Quality Inspection

To compete in today’s market the ability to reliably manufacture defect-free, high-quality products is critical. However, high volume manufacturing coupled with the trend for increased personalization and with that larger numbers of product variants put a significant strain on traditional quality inspection methods.

Manual inspection and classical computer vision are currently dominating quality inspection. While both do play an important role – and will continue to do so – artificial intelligence, especially deep learning has the potential to dramatically change how the quality of products is ensured especially in high volume manufacturing, e.g. of consumer packaged goods. Manual inspection, including sampling inspection, is not a feasible solution for high volume manufacturing, it is simply too time-consuming, expensive and likely to create a bottleneck.

Classical computer vision is currently the “go to” technology used by most companies and while this approach works well for many applications, e.g. detecting a specific missing part, it falls short in cases where defects can be highly variable, e.g. product surface defects.

In these use cases – were defect variability is high and abstraction and generalization are required to detect all possible defects – deep learning algorithms excel.

Classical Computer Vision vs. Deep Learning

Computer vision relies on descriptive analysis where a mathematical model is developed to describe the object, e.g. a product to be inspected for defects. This requires collecting data about the product, forming a hypothesis about patterns in the data, writing the code to perform the analysis, and then validating the hypothesis by comparing the predicted with the real outcomes. This process can be very time consuming, for example in cases where different product or product variants have different acceptable error thresholds. In such cases, significant customization is required to avoid costly false positives or false negatives.

Computer vision approaches are not only time consuming but also prone to issues, e.g. neglecting to include an important variable, that can make extensive retuning of the model necessary. However, computer vision approaches have their sweet-spot applications, specifically:

• Detecting the presence of absence of a defined part, e.g. a cap on a beer bottle
• Detecting orientation, e.g. left vs right parts
• Performing measurements on the image, e.g. size of an object

Deep learning (DL) takes a fundamentally different approach: it operates more like the human brain and learns patterns by “seeing” a large number of examples. This enables DL models to correctly detect defects that have a much higher degree of variability, e.g.

• Surfaces scratches, smudges or inclusion of varying sizes, locations, shapes and appearances
• Detection of product variants, e.g. different product models or pass/fail requirements of different customers, without the need to rewrite code
• Detecting missing parts, they are not specifically programmed to detect

Additionally, deep learning is better suited to inspect products in environments that are subject to variability, e.g. different lighting conditions, presence of reflections, various degrees of contamination or different material surfaces.

 

Computer Vision

• Characteristics of a good unit are defined
• Specifications are hard-coded
• Each unit is measured against the specs and a pass/fail decision is made
• Recognizes only defects it has been programmed to detect e.g., in this case screws with different shank length would not be identified  as that parameter is not defined
• Changes of product specs require rewriting of the code

 

 

 

Deep Learning

• The model is trained to learn how OK units look
• Based on that learning it will make pass/fail decisions
• Further training with more examples of each defect enables the model to categorize defects.
• Changes of product specs require retraining of model, little to no coding required.

Applications of Deep Learning for Quality Inspection

With the help of DL models even already low scrap rates can be further reduced saving cost as well minimizing the risk that defective products get delivered to the customer. In addition to better pass/fail detection for more variable products and in more variable environments, DL models can also be trained to recognize defect types and classify defective units into distinct defect categories.

This information:

• Enables understanding the frequency of different types of defects
• Serves as the basis for further analysis into why and how certain defects occur with the goal to address the root cause and minimize the issues that lead to defects
• Allows for tiering of products into different quality categories for different customer types.

Detecting and categorizing even rare and ultra-rare defects is no challenge for DL models that have been properly trained. Since these defects, by definition, occur infrequently it will take some time until enough examples are collected for the models to learn. Meanwhile, the model will continue to identify defective products with accuracies generally in excess of 99.9%.

Best Practices for Training Deep Learning Models for Manufacturing

At Accella AI we have developed specialized AI models and processes that allow for high accuracy quality inspection including anomaly (pass/fail) and defect categorization for programmable logic controllers (PLCs) with cycle times down to 10 milliseconds.

For training these models we have developed a process that significantly shortens time to implementation while making sure even rare defects are not missed or miscategorized.

The critical part of the process is to conduct anomaly detection and defect categorization sequentially rather than concurrently using a five-step process:

• Anomaly training – given the large number of good units modern manufacturing plants produce, the AI model can be trained quickly to identify a good unit by showing it 1000s of pictures. At this point no categorization of defects is attempted, the model will simply flag “not good” units and collect data, e.g. images.
• Model deployment – the AI model is then deployed and data about defective units are collected on an ongoing basis as part of routine operation. Over time examples of defective units accumulate in large enough numbers to be able to use clustering networks to categorize the defects.
• Expert review – in this step a quality expert reviews the collected data, identifies the defects, develops defect categories, assigns names to them, and  – importantly – sorts samples into their respective buckets.
• Training the model to detect defects – in this final step, based on the accumulated defect data, expert review and defect categorization – the model is trained to not just detect anomalies but to also categorized them.
• Continued training and retraining – continued collection of data improves the model further and enough data about rare effects are collected to categorize them as well. Edge cases are reviewed by shop floor personnel and if the model made a mistake, it is fed the updated information and will learn the correct categorization for future cases. This way the model continues to improves over time.

While at first glance it seems logical to try and achieve both anomaly detection and defect categorization in one step, our experience shows our five-step, iterative process to be vastly superior. It enables faster deployment and more accurate defect categorization. The optimization of manufacturing processes has led to the overwhelming majority of products being good while defects are rare which leads to unbalanced classes of data to train the algorithm.

Doing the easy step (pass/fail) first and then using routine operation to enrich the data sets required for accurate defect categorization shortens time to deployment by accelerating the build-up of robust image training libraries and increases the accuracy of defect categorization. Synthetic data ca be used to supplement data generated on the shop floor. The fact that AI models are easy to retrain and do not require extensive rewriting of code makes this approach feasible.

Summary

Deep learning algorithms hold enormous potential for quality inspection in high-volume manufacturing scenarios, both for fail/pass and defect categorization. Specialized methods are required to accommodate high volume manufacturing with cycle times in the milliseconds and customized processes are needed to quickly deploy the models on the shop floor. Unlike computer vision algorithms, AI models can be assessed for accuracy and quickly retrained making it easy to implement the models using an iterative process that leverages routine operation to improve the performance on an ongoing basis.