Anomaly detection in video streams in manufacturing plants using CNNs

Industry 4.0 is gaining traction at a tremendous pace. Corporations reinvent themselves as machine vision replaces human eyes to assist automation and improve efficiency. For manufacturing companies, a vital cog in the assembly wheel is detecting anomalies which could indicate potential holes that need to be plugged immediately. In this post, we will look at the case of a mobile phone manufacturing plant and show how to build a pattern detection framework which reads streams of videos captured from cameras typically located at every stage of the mobile phone assembly process.

Consider a stage in the assembly line, let’s call it the battery sealing stage, in which the mobile phone battery is placed and sealed inside the phone by a human who is tasked with the job. We will train a CNN on the captured video frames to learn the different actions in the battery sealing stage. The CNN will then be able to automatically detect the actions involved in the battery sealing stage on unseen video streams. We will also see how to improve this using more advanced models like LSTMs and temporal convolution networks in future posts. The source code is available here.

Approach

Our machine learning workflow will involve the following steps:

1. Acquiring data and preprocessing it
2. Annotating the video frames with the actions
3. Fine-tuning a pre-trained CNN on the video frames
4. Detection and evaluation of the retrained CNN on new videos

(To be updated…)