Development of a multimaterial and multidefect detection and anomaly prediction system based on machine vision, artificial intelligence and IoT (ERA-Net MANUNET)
MULTIAI will address one of the more ambitious challenges in the industry 4.0: the zero defect manufacturing, by applying cuttingedge information and communication technologies in the development of a multimaterial
and multidefect detection and anomaly prediction system. Main innovation regarding current stateoftheart is that the system to be developed within this project will allow the realtime, simultaneous analysis of multiple
types of defects in different materials, without the need to have different inspection stations for each type of defect. This system will be scalable and customizable for the quality control of any part manufactured in a production line. The application of artificial intelligence will allow establishing a correlation between the variables of the production process and the anomalies that the manufactured parts could present. The use of
IoT technologies for monitoring and realtime sensoring will be essential to guarantee obtaining adequate datasets in terms of size, quality and labeling which allow feeding both the defect detection and the anomaly prediction modules, and thus result in a robust and reliable system.
Duration of the contract: 24 months
Most common currently existing business cases on defect detection are described below:
- There is a company whose quality control processes are based on manual checking techniques of the parts manufactured in its production line.
- The company wants to automate its quality control process, because of scalability, consistency and flexibility reasons.
- After analysing commercial solutions, it is found that existing systems are able to evaluate a specific defect in a specific station (for example, detection of pores larger than a certain diameter) and they are only effective with previously identified cases.
In summary, main limitation of current systems is that the industrial company needs to acquire several quality control devices and integrate them in its production line in order to be able to detect multiple types of product defects.
Moreover, they present other limitations that will be addressed within this project as well, such as balancing real time processing speed requirements (since the system has to be integrated in an industrial production line) and processing extremely large amounts of data, or lack of specialised narrow domain specific datasets, needed to build a robust defect detection system.
Being aware of this opportunity, the five entities than comprise the project consortium will work together in the development of a system that will allow the multi-material quality control of a production line, including defect detection in parts and the anomaly prediction based on the process variables.
- Design and development of a new system based on machine vision and deep learning for the real time, multidefect and multimaterial defect detection and anomaly prediction in a single station. The system will incorporate image analysis techniques capable of detecting that a defect (whatever) occurs in a manufactured part and predict the appearance of anomalies based on the production process variables.
- Development of a semisupervised labelling procedure, which will allow the generation of knowledge on the new anomalies detected by the system, so it will have the capacity to incrementally and flexibly learn (active learning) to categorise defects and thus meet the specific needs of each productive process in terms of quality control.
- Deployment of AI models in physically located (at-the-edge) computing hardware in the production line to reduce response time and allow processing speeds appropriate to strict industrial requirements (that is, guaranteeing a fixed response time and reducing latency to a minimum).
- Medium/long term storage (local, cloud or mixed) of the information collected for its subsequent analysis.
- System validation in two very different use cases (both from the point of view of the part material and of the defects and anomalies that they could present): plastic microinjected parts with micro details and metal parts production and processing.
WP1. Project management
Task 1.1: Project technical management (M1M24).
Task 1.2: Quality Management (M1M24).
D1.1: Reports on the activities of each WP (every 6 months)
D1.2: Quality management plan.
D1.3: Risk and contingency plan.
WP2. Use cases definition & requirement analysis
Task 2.1: Analysis and characterisation of both use cases (M2M4).
Task 2.2: Functional scope and requirements catalogue (M3M5).
D2.1: Use cases definition document.
D2.2: Requirement analysis report.
WP3. Sensoring and realtime
monitoring by means of IoT + computer vision
Task 3.1: Identification of process variables likely to affect the parts quality (M4M5).
Task 3.2: Analysis of sensing alternatives for the measurement of variables (M5M6).
Task 3.3: Definition of system architecture (M5M8).
Task 3.4: Data collection and monitoring software design (M8M9).
Task 3.5: Prototype platform for data collection and monitoring development (M8M10).
Task 3.6: Communications, sensors, cameras and gateway set up (M6M12).
D3.1: Report for each use case containing process variables and the way to sense them.
D3.2: Document with generic system architecture handling both use cases.
D3.3: Prototype software platform and hardware components set up on both use cases sites to start data collection.
WP4. Data collection
Task 4.1: Data collection for use case #1 process variables (M7M12).
Task 4.2: Data collection for use case #2 process variables (M7M12).
Task 4.3: Data collection for both use cases defects in parts (M7M12).
D4.1: Structured database containing the process parameters of the 2 use cases in connection with the quality parameters of the parts.
D4.2: Defect library with all parts images.
WP5. Machine visionbased
Task 5.1: Functional analysis of expected defects for each material/use case (M9M10).
Task 5.2: Characterisation strategy design for each defect (M10M11).
Task 5.3: Scalable and customisable image analysis pipeline architecture design (M10M11).
Task 5.4: Data collection module development (M12M13).
Task 5.5: Defect detection models development (M12M17).
Task 5.6: Defect detection models training and fine-tuning (M16M18).
Task 5.7: Validation at lab scale using the datasets obtained in WP4 (M18M20).
D5.1: Functional design and usecases
expert knowledge catalogue (document).
D5.2: Report on machine vision-based system development and validation results (document).
WP6. AI-based anomaly prediction system
Task 6.1: Scope definition (M9M11).
Task 6.2: Data extraction, preprocessing
and enriching (M10M14).
Task 6.3: Data exploring analysis and patterns extraction (M14M18).
Task 6.4: Modelling (M16M18).
Task 6.5: Validation at lab scale using
D6.1: Scope definition report on the AI-based anomaly prediction system.
D6.2: Design and development of the data processing technologies.
D6.3: Pattern extraction based on the exploration of the data packets.
D6.4: Data modelling analysis.
D6.5: Validation of the prediction system at lab scale.
WP7. Integration and validation
Task 7.1: Lab-to-site handover (M16).
Task 7.2: Deployment and configuration (M16M17).
Task 7.3: Systems tuning (M17M18).
Task 7.4: Supervised testing short period (M17M19).
Task 7.5: Unsupervised testing longer period (M20M24).
Task 7.6: Results collection and market oriented conclusions (M23M24).
D7.1: 2 pilot lines based on the 2 use cases of the project with associated reports and documentation.
D7.2: A versatile solution applicable to multiple sectors and manufacturing processes.
D7.3: A deployment strategy for other materials and processing techniques.
WP8. Dissemination and exploitation
Description of work:
Task 8.1: Dissemination and communication (M1M24).
Task 8.2: Market analysis and definition of business models (M1M24).
D8.1: Dissemination and communication report for the first year.
D8.2: Final dissemination and communication report.
D8.3: Market study and business plan.
Phases and activities
Phase 1 Defining the use cases and the technical and functional requirements; analysis of sensors and data collection infrastructure 31/12/2021
A1.1 Definition of use cases – correlation WP2-T2.1
During this activity, the requirements and specifications of the identified use case were analyzed and the solution that will be developed within the project was presented.
The specification requirements have been defined and will be used to finalize the solution to be developed within the project, to monitor the progress of technical work and to perform validation.
The development of intelligent control systems based on affordable hardware is an important challenge that meets the needs of SMEs. On the corporate side, there is a desire to bring together the maximum amount of computational processing “as a service” in a common virtualized IT infrastructure, rather than owning independently and decentralized managed processing units
A1.2 Definition of the technical and functional requirements – correlation WP2 -T2.2
In this activity, the technical and functional requirements for detection, communications, IoT, Industry 4.0 and AI processing technologies were analyzed. A list of the identified technical challenges was also presented.
A1.3 Current state of the art analysis for designing the data collection mechanism – correlation WP3 -T3.1, T3.2, T3.3
In this activity, an analysis of the current state of knowledge for detection technologies, communications, IoT, Industry 4.0 and AI processing was performed.
A1.4 Realization of the hardware-software solution for data collection – correlation WP3 – T3.4, T3.5, T3.6
This activity described the data collection component taking into consideration the type of communications, sensors, cameras and gateway configuration. The data collector and monitor software tool specifications were identified.
A1.5 Data collection inside of the use cases – WP4 correlation
In this activity, the methods for collecting the identified data related to the use case were described. The process variables that can affect the quality of the parts have been identified.
Phase 2 – Design of the solution for the identification of defects and definition of the artificial intelligence algorithms for the prediction of anomalies; integration, validation, dissemination and market orientation
Activity 2.1 Definition of the architecture of the image analysis solution for defect detection
- This activity described the design, development and validation of an image-based approach for simultaneous and real-time analysis of multiple types of defects (related to surface, morphology and appearance), identified in materials and manufacturing processes of selected use cases. The use case implemented at Petal, technical and non-functional requirements and possible defects are described.
Activity 2.2 Design and implementation of the solution for data processing for defects identification (linked to T5.4, T5.5, T5.6)
- In this activity, the artificial vision system configuration was described. The algorithms used for data measurement and inspection, the graphic interface as well as the tests that were performed to validate the solution were described.
Activity 2.3 Modeling, optimization, testing and validation of the anomaly prediction system in the production process (linked to T5.7, T6.1, T6.2, T6.3, T6.4, T6.5)
- This activity includes the definition and development of the AI-based anomaly prediction system, considering the technical challenges involved in the use of sensor networks, together with image processing technologies required for extremely large amounts of data.
Activity 2.4 Integration of functional modules, configuration, optimization and solution testing (linked to T7.1, T7.2, T7.3, T7.4, T7.5, T7.6)
- In this activity, the following aspects were presented: the transfer of the laboratory elements to the two use cases, the definition of the extension criteria to other applications, the validation of the generalization capacity of the algorithms, the identification of a “low-end cost” solution for implementation in other sectors.
Activity 2.5 Market analysis and definition of business models (linked to T8.2)
- Within this activity, the potential exploitable results of the MULTI-AI project were presented, as well as the market exploitation strategy of the BEIA Consult company in order to promote the results of the project
This work was supported by a grant of the Romanian National Authority for Scientific Research and Innovation. CCCDI – UEFISCDI, project number ERANET-MANUNET-MULTIIA-1, within PNCDI III