KR20240026561A - System for part defect inspection through image analysis based on artificial intelligence - Google Patents

Abstract

A component defect inspection system through artificial intelligence-based image analysis is disclosed, and the component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present invention captures an analysis target image including the surface area of the target component. A photographing module that receives the image to be analyzed, and is generated through learning based on learning data including a learning image photographing the surface area of each of a plurality of parts and label information about the defective area appearing on the learning image. An analysis module that uses an artificial intelligence-based analysis model to output defect information including information on the presence or absence of defects and information on defect types for the target part according to the surface characteristics of the target part reflected in the analysis target image; and It may include a display module that displays the image to be analyzed and the defect information.

Description

Part defect inspection system through artificial intelligence-based image analysis {SYSTEM FOR PART DEFECT INSPECTION THROUGH IMAGE ANALYSIS BASED ON ARTIFICIAL INTELLIGENCE}

This institute is concerned with a component defect inspection system through artificial intelligence-based image analysis.

Existing vision inspection systems require individual definitions for defects that can be identified on images, and there is the inconvenience of manually setting defect characteristic values, as well as the need for printed circuit boards (PCBs) or liquid There were limitations in its use mainly in fields where the surface shape is standardized, such as crystal display panels.

In particular, the conventional vision inspection system has the inconvenience of having to set up the equipment again manually (Rule-Based) every time the inspection environment and inspection standards change, and has the problem of low accuracy in various types of defects or more precise and detailed parts. holding it

Moreover, in the case of cylindrical metal products such as motor housings, the shapes of the inner and outer surfaces are irregular and unstructured, making it difficult to manually set the characteristic value of the defect. As a result, defects in cylindrical products or parts are difficult to set. Most inspections are carried out by visual inspection.

In this regard, it is possible to consider introducing an artificial intelligence-based defect detection algorithm for cylindrical products or parts. However, in the case of the conventional artificial intelligence algorithm-based inspection system, the defect detection rate is as low as 80% and the accuracy performance is poor. Not only was there a need for improvement, but there was also a problem with the defect inspection items being applicable only to some surface cracks and missing bonds.

The technology behind this application is disclosed in Korean Patent Publication No. 10-2108956.

The purpose of this application is to solve the problems of the prior art described above, and to accurately detect defect information on parts with atypical shapes, such as cylindrical shapes, through artificial intelligence-based image analysis of analysis target images taken of the parts. The purpose is to provide a component defect inspection system through artificial intelligence-based image analysis.

However, the technical challenges sought to be achieved by the embodiments of the present application are not limited to the technical challenges described above, and other technical challenges may exist.

As a technical means for achieving the above-described technical task, a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application includes a photographing module that captures an image to be analyzed including the surface area of the target component; Artificial intelligence-based technology generated through learning based on receiving the analysis target image and learning data including learning images taken of the surface area of each of a plurality of parts and label information about defective areas appearing on the learning images An analysis module that outputs defect information including information on the presence or absence of defects and information on defect types for the target part according to surface characteristics of the target part reflected in the analysis target image using an analysis model, and the analysis target image and a display module that displays the defect information.

Additionally, each of the target component and the plurality of components may be a cylindrical component.

Additionally, the learning image and the analysis target image may be photographed to include an inner surface shape and an outer surface shape of the cylindrical part.

Additionally, the photographing module may be a three-dimensional optical camera that photographs the shape of the inner surface including the shape of the inner wall of the component corresponding to the direction in which the cylindrical shape extends.

Additionally, the three-dimensional optical camera may capture the image to be analyzed based on a shooting direction in which the target component is viewed from the cylindrical opening area.

In addition, the part defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application includes a lighting module that irradiates light toward the target part so that the brightness corresponding to the inner surface shape is more than a preset brightness. It can be included.

Additionally, the defect type may include at least one of scratches, cracks, damage, omission, poor assembly, and contamination with foreign matter.

Additionally, the label information may be displayed overlaid on the learning image to include location information of the defective area corresponding to one of the defect types, standard information of the defective area, and information about the defect type.

In addition, the part defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application includes an inspection table module on which the target part is seated to rotate the target part around the direction of extension of the cylindrical shape. can do.

Additionally, the imaging module may acquire a plurality of analysis target images according to rotation of the target component.

Additionally, the target component may be a motor housing component.

Meanwhile, the part defect inspection method through artificial intelligence-based image analysis according to an embodiment of the present application includes learning images taken of the surface area of each of a plurality of parts and label information about the defect area appearing on the learning images. Collecting learning data including, based on the learning data, when an analysis target image taken of a target part is input, the presence or absence of defects in the target part is determined by using the surface characteristics of the target part reflected in the analysis target image. Learning an artificial intelligence-based analysis model that outputs defect information including information about and defect type, receiving the analysis target image, and using the analysis model to identify the target part. Steps to derive defect information can be performed.

Additionally, the plurality of parts and the target part may each be a cylindrical part.

Additionally, the learning image and the analysis target image may be captured to include an inner surface shape and an outer surface shape of the cylindrical part.

Additionally, the shape of the inner surface may include the shape of the inner wall of the component corresponding to the direction in which the cylindrical shape extends.

Additionally, in the step of collecting the learning data, the learning image may be obtained by photographing each of the plurality of parts in a shooting direction looking at the cylindrical opening area.

Additionally, in the step of receiving the analysis target image, the analysis target image may be obtained by capturing the target part in a shooting direction looking at the cylindrical opening area.

Additionally, the defect type may include at least one of scratches, cracks, damage, omission, poor assembly, and contamination with foreign matter.

Additionally, the label information may be displayed overlaid on the learning image to include location information of the defective area corresponding to one of the defect types, standard information of the defective area, and information about the defect type.

In addition, the step of receiving the image to be analyzed may include obtaining the image to be analyzed captured in a state in which light by illumination is irradiated toward the target part such that the brightness corresponding to the inner surface shape is more than a preset brightness. You can.

In addition, the step of receiving the analysis target image may acquire a plurality of analysis target images according to rotation of the target component about the cylindrical extension direction as an axis.

Additionally, collecting the learning data may include applying data augmentation including at least one of rotation transformation, perspective distortion transformation, brightness transformation, contrast transformation, and inversion transformation to each of the learning images.

Additionally, the target component may be a motor housing component.

In addition, the method of inspecting defects in parts through artificial intelligence-based image analysis according to an embodiment of the present application may include providing a defect management interface including the analysis target image and the defect information through a user terminal. .

Meanwhile, the device for inspecting defects in parts through artificial intelligence-based image analysis according to an embodiment of the present application includes learning images taken of the surface areas of each of a plurality of parts and label information about the defective areas appearing on the learning images. A collection unit that collects learning data including, when an analysis target image of a target part is input based on the learning data, defects in the target part are detected using the surface characteristics of the target part reflected in the analysis target image. A learning unit that trains an artificial intelligence-based analysis model that outputs defect information including information on presence or absence and information on defect types, and receives the analysis target image and uses the analysis model to correspond to the target part. It may include a defect analysis unit that derives the defect information.

Additionally, the collection unit may acquire the learning image in which each of the plurality of parts is photographed in a shooting direction looking toward the cylindrical opening area.

In addition, the defect analysis unit may acquire the image of the analysis object captured in the direction in which the target component is viewed from the cylindrical opening area.

In addition, the device for inspecting defects in parts through artificial intelligence-based image analysis according to an embodiment of the present application includes at least one of rotation transformation, perspective distortion transformation, brightness transformation, contrast transformation, and inversion transformation for each of the learning images. It may include a preprocessor that applies data augmentation.

The above-described means of solving the problem are merely illustrative and should not be construed as intended to limit the present application. In addition to the exemplary embodiments described above, additional embodiments may be present in the drawings and detailed description of the invention.

According to the above-described means of solving the problem of this institute, artificial intelligence can precisely detect defect information on parts with atypical shapes, such as cylindrical shapes, through artificial intelligence-based image analysis of analysis target images taken of the parts. A component defect inspection system can be provided through video analysis.

However, the effects that can be obtained herein are not limited to the effects described above, and other effects may exist.

1 is a schematic diagram of a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.
Figure 2 is a diagram showing an example of the implementation of a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.
Figure 3 is a conceptual diagram illustrating control items by user type and system functions by server type using a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.
FIGS. 4A and 4B are diagrams for explaining the inspection table module of the component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.
FIGS. 5A and 5B are diagrams for explaining a lighting module of a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.
Figure 6 is a diagram showing an example of a visualization result displayed by overlapping the analyzed defect information on the analysis target image.
7A to 7D are diagrams illustrating a defect information display interface displayed through a display module of a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.
Figure 8 is a schematic configuration diagram of a device for inspecting defects in parts through artificial intelligence-based image analysis according to an embodiment of the present application.
Figure 9 is an operation flowchart of a method for inspecting defects in parts through artificial intelligence-based image analysis according to an embodiment of the present application.

Below, with reference to the attached drawings, embodiments of the present application will be described in detail so that those skilled in the art can easily implement them. However, the present application may be implemented in various different forms and is not limited to the embodiments described herein. In order to clearly explain the present application in the drawings, parts that are not related to the description are omitted, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout this specification, when a part is said to be “connected” to another part, this means not only “directly connected” but also “electrically connected” or “indirectly connected” with another element in between. "Includes cases where it is.

Throughout this specification, when a member is said to be located “on”, “above”, “at the top”, “below”, “at the bottom”, or “at the bottom” of another member, this means that a member is located on another member. This includes not only cases where they are in contact, but also cases where another member exists between two members.

Throughout the specification of the present application, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components unless specifically stated to the contrary.

This institute is concerned with a component defect inspection system through artificial intelligence-based image analysis.

Figure 1 is a schematic configuration diagram of a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application, and Figure 2 is a component defect inspection through artificial intelligence-based image analysis according to an embodiment of the present application. This is a drawing showing an example of system implementation.

Referring to FIGS. 1 and 2, the component defect inspection system 10 (hereinafter referred to as the ‘inspection system 10’) through artificial intelligence-based image analysis according to an embodiment of the present application is an implementation of the present application. Part defect inspection device 100 through artificial intelligence-based image analysis according to an example (hereinafter referred to as 'defect inspection device 100'), imaging module 200, user terminal 300, and administrator terminal 400 ) may include. Additionally, referring to FIGS. 4A to 5B described later, the inspection system 10 may include an inspection table module 500 and a lighting module 600.

In addition, referring to FIG. 1, the defect inspection device 100 includes a learning server 102 for generating (learning) an artificial intelligence-based analysis model described in detail below and an analysis model built by the learning server 102. A main server 101 that performs defect detection and classification using may be provided, but is not limited to this. As another example, according to the implementation of the present application, the defect inspection device 100 may be a single server device that performs the functions of the main server 101 and the learning server 102 in an integrated manner. For reference, the defect inspection device 100 disclosed herein may be referred to as an analysis module 100. In other words, in the description of the embodiments of the present application, reference numeral 100 may refer to both 'defect inspection device' and 'analysis module'.

The defect inspection device 100, the photographing module 200, the user terminal 300, the administrator terminal 400, the inspection table module 500, and the lighting module 600 may communicate with each other through the network 20. The network 20 refers to a connection structure that allows information exchange between nodes such as terminals and servers. Examples of such networks 20 include the 3rd Generation Partnership Project (3GPP) network, Long Term Evolution) network, 5G network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area) Network), wifi network, Bluetooth network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc., but are not limited thereto.

The user terminal 300 and the administrator terminal 400 include, for example, a smartphone, a SmartPad, a tablet PC, etc., as well as a personal communication system (PCS), a global system for mobile communication (GSM), and a personal communication system (PDC). Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) It can be any type of wireless communication device, such as a terminal.

Hereinafter, specific functions and operations of the defect inspection device 100 will be described with reference to FIGS. 3 to 7D.

Figure 3 is a conceptual diagram illustrating control items by user type and system functions by server type using a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.

Referring to FIG. 3, the inspection system 10 disclosed herein is a subject (such as an inspector in FIG. 3) who performs image-based defect inspection on a predetermined component (target component), a predetermined component, or a predetermined component. Subject (site manager in FIG. 3) who performs on-site management of mounted equipment, facilities, etc., learns the artificial intelligence-based analysis model mounted on the defect inspection device 100, manages and distributes the server for the learned analysis model. It may be implemented to provide functions for various users, including the entity that manages things (server and model manager in FIG. 3), but is not limited to this.

In this regard, in the description of the embodiment of the present application, the user terminal 300 refers to a terminal owned by a user such as an inspector who is the subject of defect inspection, and the administrator terminal 400 refers to a field manager, server and model manager, etc. It may be mutually distinguished as referring to terminals owned by users, but is not limited to this. As another example, according to the implementation of the present application, the user terminal 300 and the administrator terminal 400 refer to one user's terminal, but the functions that can be performed through the terminal may be differentiated from each other.

In addition, referring to FIG. 3, the defect inspection device 100 includes a learning server 102 for generating (learning) an artificial intelligence-based analysis model and defect detection and analysis using an analysis model built by the learning server 102. A main server 101 that performs classification may be provided.

Specifically, the learning server 102 preprocesses and sets parameters for learning data for learning an artificial intelligence-based analysis model through a 'model training system', trains the analysis model, and provides information on the output results of the analysis model. Analysis and update (update, optimization) of the analysis model based on it are performed, and the main server 101 performs 'defect detection and classification' using the analysis model built by the learning server 102, and uses user inspection software. The results and information on defect inspection are displayed on at least one of the user terminal 300 and the administrator terminal 400, and the results and information on the defect inspection are displayed on at least one of the user terminal 300 and the administrator terminal 400, and various parts are inspected by at least one of the user terminal 300 and the administrator terminal 400. Statistical information on comprehensive defect inspection information can be visualized and displayed. In addition, the main server 101 provides user authentication, authority management (authorization), server management, account management, database management, image/ Functions such as log/model storage can be performed.

Hereinafter, the process by which the defect inspection device 100 generates an artificial intelligence-based analysis model will be described in detail. Meanwhile, the artificial intelligence-based analysis model creation process may be performed by the above-described learning server 102 of the defect inspection device 100.

The defect inspection apparatus 100 may collect learning data including learning images obtained by photographing the surface areas of each of a plurality of components and label information about defect areas appearing on the learning images.

More specifically, the label information given to the learning image that the defect inspection device 100 collects as learning data is the location information of the defect area corresponding to one of the plurality of defect types to be detected through the analysis model, and the defect area. It can be displayed overlaid on the learning image to include information about the specification and defect type. For example, the location information and standard information of the defect area are displayed in the form of a bounding box (BBox) surrounding the defect area, and the information about the defect type includes text-based information about the pre-classified defect type. It could be.

In this regard, defect types that can be detected by the defect inspection device 100 disclosed herein may include, but are not limited to, at least one of scratches, cracks, damage, omission, poor assembly, and foreign matter contamination. no. More specifically, the defect inspection device 100 targets the motor housing part, which is a cylindrical part, as an analysis object, and analyzes defect types for the motor housing part such as cover assembly defect, caulking defect, cover/clinching bond contamination, inner bond contamination, Defect types including various categories such as clinching wheel, exterior damage, magnet damage, magnet crack, missing metal bush, metal bush damage, chip defect, etc. may be assigned as label information.

In other words, the defect inspection device 100 disclosed herein collects learning images for a plurality of parts, each of which has a cylindrical shape, as learning data, learns (builds) an artificial intelligence-based analysis model, and uses the analysis model. Thus, it can be operated to detect defects in target parts that are cylindrical parts (for example, motor housing parts mounted on automobiles, etc.).

In other words, the learning image for a plurality of parts input to the defect inspection device 100 and the analysis target image for the target part may be an image taken to include the inner surface shape and outer surface shape of the cylindrical part of the object. there is. More specifically, the shape of the inner surface of the part reflected in the learning image and the analysis target image may include the shape of the inner wall of the part corresponding to the extension direction of the cylindrical shape.

In this regard, the defect inspection device 100 uses an imaging module 200 including a stereoscopic optical camera 210 capable of photographing the shape of the inner surface, including the shape of the inner wall of the component, corresponding to the direction of extension of the cylindrical shape. It may be receiving a learning image and an image to be analyzed.

For reference, the cameras used in conventional video analysis-based defect detection solutions mainly use general camera modules that capture only a flat angle of view, called pinhole lenses, and are mainly used for OLED/LCD panel inspection, solar panel defect inspection, secondary battery, It is used for surface inspection of semiconductors (PCB, wafer, etc.). In contrast, the inspection system 10 disclosed herein is equipped with a 360-degree inside view inspection lens designed to photograph the bottom surface of the opening area (hole) of the cylindrical part and the vertical wall surface corresponding to the extension direction of the cylinder. A stereoscopic optical camera 210, which is an optical camera, can be used. The three-dimensional optical camera 210 has the advantage of being able to comprehensively capture high-resolution images of the inner and outer surfaces of cylindrical components.

In addition, the inspection system 10 disclosed herein utilizes the stereoscopic optical camera 210 as the imaging module 200, thereby not only eliminating the need to insert the optical camera directly into the cylindrical hole of the part, but also providing a very high depth of field. This has the advantage of being able to photograph products of various shapes and dimensions with a single lens. In addition, the inspection system 10 disclosed herein is capable of optimizing surface defect detection performance by effectively combining direct lighting and indirect lighting through a lighting module 600, which will be described later, to find even fine and hard-to-see defects. There is.

Meanwhile, referring to FIG. 1, the inspection system 10 disclosed herein is additionally provided with a vision camera 220 that acquires an image of the outer surface of a cylindrical part or an additional image of the entire outer surface of the part. It can be done, and according to the implementation of the present application, the learning image used for learning the analysis model and the analysis target image taken for the target part that is the target of analysis are a first type image acquired by the stereoscopic optical camera 210 and a vision camera. It may include a second type image obtained by 220.

In other words, the defect inspection device 100 acquires a first type image taken of each of the plurality of parts in the shooting direction looking at the cylindrical opening area as a learning image from the stereoscopic optical camera 210, and sets the target part as a cylindrical opening area. A first type image taken in the shooting direction looking at the opening area of the shape is acquired from the stereoscopic optical camera 210 as an image to be analyzed, and, if necessary, the external shape of the part photographed using the vision camera 220. It may operate to additionally acquire a second type image.

Additionally, the defect inspection apparatus 100 may perform preprocessing to apply data augmentation including at least one of rotation transformation, perspective distortion transformation, brightness transformation, contrast transformation, and inversion transformation to each of the collected learning images. In this regard, the defect inspection device 100 performs a preprocessing process to generate various types of defect data from a small number of learning data, considering that the number of samples of learning data including defective areas may not be sufficient compared to good product data. Through this, a sufficient number and variety of defect data can be secured, and based on this, an analysis model that can perform defect detection with high accuracy can be trained.

According to an embodiment of the present application, the defect inspection apparatus 100 applies a data augmentation algorithm based on competitive adversarial networks (GAN) to perform rotation transformation, perspective distortion transformation, and The amount of training data can be increased by performing image transformations such as brightness transformation, contrast transformation, and inversion transformation.

In addition, when an image to be analyzed is input based on preprocessed learning data, the defect inspection device 100 uses the surface characteristics (outer surface characteristics and inner characteristics) of the target part to provide information on the presence or absence of defects in the target part. And an artificial intelligence-based analysis model that outputs defect information including information about the defect type can be trained.

In this regard, according to an embodiment of the present application, the defect inspection device 100 is based on a multi-object detection algorithm to precisely obtain defect information from images of parts having an atypical shape. An analysis model can be trained. In addition, the defect inspection device 100 uses multiple deep neural networks such as the conventional Faster R-CNN (Regions with CNN features), YOLO algorithm, and SSD (Single Shot Detector) to detect defects in images to be analyzed that are captured in real time. You can build an analysis model that combines and expands models.

Additionally, after learning of the artificial intelligence-based analysis model is completed, the defect inspection apparatus 100 may receive an image to be analyzed and derive defect information corresponding to the target part using the analysis model.

FIGS. 4A and 4B are diagrams for explaining the inspection table module of the component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.

Referring to FIGS. 4A and 4B , the defect inspection apparatus 100 may acquire a plurality of analysis target images taken according to the rotation of the target component around the cylindrical extension direction. In this regard, the inspection system 10 may be provided with an inspection table module 500 on which the target component is mounted so as to rotate the target component around the cylindrical extension direction as the axis.

More specifically, referring to FIG. 4A, the inspection table module 500 allows the target part to be seated on the upper surface and rotates the seated target part around the direction of extension of the cylindrical shape, orthogonal to the axial direction of the target part. The seating portion 520 rotating in the plane direction and the direction toward the target part supported by the seating portion 520 at the top of the seating portion 520 (in other words, the shooting direction looking toward the cylindrical opening area of the target component) ) may include a camera fixing part 510 that supports and adjusts the three-dimensional optical camera 210 to adjust the shooting position and shooting direction of the three-dimensional optical camera 210.

In addition, referring to Figure 4b, the camera fixing unit 510 has a function of adjusting the left and right spacing of the camera fixing plate, a function of adjusting the fixing position of the stereoscopic optical camera 210 up, down, left and right, and a function of controlling the shooting of the stereoscopic optical camera 210. It may be provided as a stand structure that has a function to adjust direction (angle).

FIGS. 5A and 5B are diagrams for explaining a lighting module of a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.

Referring to FIGS. 5A and 5B, the defect inspection device 100 captures the image to be analyzed while light is irradiated toward the target component such that the brightness corresponding to the internal surface shape is equal to or higher than a preset brightness. It can be obtained.

Additionally, in this regard, the inspection system 10 may be provided with an illumination module 600 that irradiates light toward the target component such that the brightness corresponding to the inner surface shape of the target component is equal to or higher than a preset brightness.

In addition, referring to FIG. 5A, the lighting module 600 according to an embodiment of the present application is disposed in an axial direction of the cylindrical shape of the target component (for example, from 12 o'clock to 6 o'clock in FIG. 5A) and in an oblique direction. It may include a plurality of light source modules 601a and 601b that irradiate light to the internal surface of the target component so that the brightness is higher than a predetermined level. For reference, the inclination angle of the light source modules 601a and 601b with respect to the axial direction may be determined to be about 60 degrees, but is not limited to this, and is based on the volume of the target component and the height (depth) of the target component in the axial direction. , the inclination angle of each light source module may be variably applied in consideration of the external illumination of the inspection environment, the height of the stereoscopic optical camera 210, the number of light source modules, etc.

In addition, referring to Figure 5b, the lighting module 600 may be provided with a gripper 610 that holds the light source modules 601a and 601b and adjusts the placement height, inclination angle, etc. of the light source modules 601a and 601b. You can.

Meanwhile, according to an embodiment of the present application, the defect inspection device 100 preemptively checks whether the shape of the inner surface of the target component can be easily identified from the received analysis target image and determines whether the shape of the inner surface is determined by brightness, etc. If it is determined that it cannot be easily identified, the brightness of each light source module (601a, 601b), the placement height of each light source module (601a, 601b), and the inclination angle of each light source module (601a, 601b) are adjusted, and then the analysis target is determined. It may operate to transmit a control signal requesting to retake an image to the lighting module 600 and the photographing module 200. In this regard, whether the shape of the inner surface of the target part can be easily identified from the image to be analyzed can be determined by detecting the bottom surface (lower surface) of the target part in the received image to be analyzed.

Figure 6 is a diagram showing an example of a visualization result displayed by overlapping the analyzed defect information on the analysis target image.

Referring to FIG. 6, the defect analysis device 100 can display information on the specification of the defect area and defect type derived for the image to be analyzed through an artificial intelligence-based analysis model, overlaid on the image to be analyzed. . In addition, the defect analysis device 100 converts information about the type of defect displayed on the image to be analyzed into a defect name classified for the corresponding defect area (e.g., text such as 'crack' or 'fracture' in FIG. 6). and quantified reliability (probability) information (for example, values such as '0.73', '0.30', and '0.60' in FIG. 6) can be displayed side by side.

Additionally, the defect analysis apparatus 100 may display a defect management interface including an analysis target image and defect information through a display module (not shown). For example, the defect analysis device 100 may transmit a defect management interface created to at least one of the user terminal 300 and the manager terminal 400.

7A to 7D are diagrams illustrating a defect information display interface displayed through a display module of a component defect inspection system through artificial intelligence-based image analysis according to an embodiment of the present application.

Referring to FIGS. 7A and 7B, the defect inspection device 100 is an interface that displays inspection status information (number of defective parts, defective rate, inspection goal/performance, defect type distribution, etc.) for a plurality of parts on a time and daily basis. can be provided.

In addition, referring to FIGS. 7A and 7B, the defect inspection device 100 provides summary information of inspection results on a daily, weekly, or monthly basis (inspection date, inspection quantity, defective quantity, defective rate, quantity by defect type, inspector information). etc.) can be provided to display an interface that displays a table (diagram).

In addition, referring to FIG. 7C, the defect inspection device 100 displays the original image at the time of shooting the image to be analyzed (the first type image and the second type image) from which it is determined that a defect exists (in other words, defect information is derived). An interface for displaying images can be provided.

In addition, referring to FIG. 7D, the defect inspection device 100 inputs a selection period for an inquiry authorized by at least one of the user terminal 300 and the manager terminal 400 (for example, '2022-02-01 ~ 2022-02-01' in FIG. 7D). Based on '2022-02-18', etc.), we provide an interface that secondary processes and displays statistical information (defect rate trend compared to daily production, analysis graph by defect type, etc.) on the production history of parts and defect detection details for the relevant period. You can.

Figure 8 is a schematic configuration diagram of a device for inspecting defects in parts through artificial intelligence-based image analysis according to an embodiment of the present application.

Referring to FIG. 8 , the defect inspection device 100 may include a collection unit 110, a pre-processing unit 120, a learning unit 130, and a defect analysis unit 140.

The collection unit 110 may collect learning data including learning images obtained by photographing the surface areas of each of a plurality of components and label information about defective areas appearing on the learning images.

Specifically, the collection unit 110 may acquire learning images in which each of a plurality of cylindrical parts is photographed in a shooting direction looking toward the cylindrical opening area.

The preprocessor 120 may apply data augmentation including at least one of rotation transformation, perspective distortion transformation, brightness transformation, contrast transformation, and inversion transformation to each of the collected training images.

When an analysis target image of a target part is input based on learning data for which preprocessing has been performed on the learning image, the learning unit 130 uses the surface characteristics of the target part reflected in the analysis target image to detect defects in the target part. An artificial intelligence-based analysis model that outputs defect information including information about presence or absence and information about defect type can be trained.

The defect analysis unit 140 may receive an image to be analyzed by photographing the target part. Specifically, the defect analysis unit 140 may acquire an analysis target image taken of a target component, which is a cylindrical component, in a shooting direction looking toward the cylindrical opening area.

In addition, according to an embodiment of the present application, the defect analysis unit 140 analyzes images taken while light is irradiated toward the target component such that the brightness corresponding to the inner surface shape of the target component is equal to or higher than a preset brightness. The target image can be acquired.

In addition, according to an embodiment of the present application, the defect analysis unit 140 may acquire a plurality of analysis target images according to the rotation of the target component around the cylindrical extension direction of the target component.

In addition, the defect analysis unit 140 can derive defect information corresponding to the target part from the image to be analyzed using a learned (constructed) artificial intelligence-based analysis model.

In addition, although not shown in the drawing, the defect inspection device 100 provides a defect management interface including an image to be analyzed and defect information derived by the defect analysis unit 140 among the user terminal 300 and the manager terminal 400. It may include a visualization unit (not shown) provided through at least one.

Below, we will briefly look at the operation flow of the present application based on the details described above.

Figure 9 is an operation flowchart of a method for inspecting defects in parts through artificial intelligence-based image analysis according to an embodiment of the present application.

The method of inspecting defects in parts through artificial intelligence-based image analysis shown in FIG. 9 can be performed by the defect analysis device 100 described above. Therefore, even if the content is omitted below, the content described with respect to the defect analysis device 100 can be equally applied to the explanation of the method of inspecting defects in parts through artificial intelligence-based image analysis.

Referring to FIG. 9, in step S11, the collection unit 110 may collect training data including training images taken of the surface areas of each of a plurality of parts and label information about defective areas appearing on the training images. .

Specifically, in step S11, the collection unit 110 may acquire a learning image in which each of the plurality of cylindrical parts is photographed in a shooting direction looking toward the cylindrical opening area.

Next, in step S12, the preprocessor 120 may apply data augmentation including at least one of rotation transformation, perspective distortion transformation, brightness transformation, contrast transformation, and inversion transformation to each of the learning images collected in step S11. .

Next, in step S13, the learning unit 130 uses the surface characteristics of the target part reflected in the analysis target image when the analysis target image of the target part is input based on the learning data for which preprocessing has been performed on the learning image. Thus, an artificial intelligence-based analysis model that outputs defect information including information on the presence or absence of defects and information on defect types for the target part can be learned.

Next, in step S14, the defect analysis unit 140 may receive an image to be analyzed by photographing the target part.

Specifically, in step S14, the defect analysis unit 140 may acquire an analysis target image taken of the target part, which is a cylindrical part, in a shooting direction looking toward the cylindrical opening area.

In addition, according to an embodiment of the present application, in step S14, the defect analysis unit 140 is irradiated with light by illumination toward the target component such that the brightness corresponding to the inner surface shape of the target component is equal to or higher than a preset brightness. Captured images subject to analysis can be obtained.

In addition, according to an embodiment of the present application, in step S14, the defect analysis unit 140 may acquire a plurality of analysis target images according to the rotation of the target component around the cylindrical extension direction of the target component.

Next, in step S15, the defect analysis unit 140 may derive defect information corresponding to the target part from the image to be analyzed using the learned (constructed) artificial intelligence-based analysis model.

In addition, although not shown in the drawings, the method of inspecting defects in parts through artificial intelligence-based image analysis according to an embodiment of the present application provides a defect management interface including the image to be analyzed and defect information derived through step S15 through a user terminal ( It may include providing information through at least one of 300) and the manager terminal 400.

In the above description, steps S11 to S15 may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present disclosure. Additionally, some steps may be omitted or the order between steps may be changed as needed.

The method for inspecting defects in parts through artificial intelligence-based image analysis according to an embodiment of the present application may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and constructed for the present invention or may be known and usable by those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Additionally, the above-described method of inspecting parts for defects through image analysis based on artificial intelligence may also be implemented in the form of a computer program or application executed by a computer stored in a recording medium.

The description of the present application described above is for illustrative purposes, and those skilled in the art will understand that the present application can be easily modified into other specific forms without changing its technical idea or essential features. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as unitary may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present application is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present application.

10: Part defect inspection system through artificial intelligence-based image analysis
100: Part defect inspection device and analysis module through artificial intelligence-based image analysis
110: Collection department
120: Preprocessing unit
130: Learning Department
140: Defect analysis unit
200: Shooting module
210: Stereoscopic optical camera
220: Vision camera
300: User terminal
400: Administrator terminal
500: Inspection stand module
600: Lighting module
20: Network

Claims (10)

In a component defect inspection system using artificial intelligence-based image analysis,
an imaging module that captures an image to be analyzed including the surface area of the target component;
Artificial intelligence-based technology generated through learning based on receiving the analysis target image and learning data including learning images taken of the surface area of each of a plurality of parts and label information about defective areas appearing on the learning images an analysis module that outputs defect information including information on the presence or absence of defects and information on defect types for the target part according to surface characteristics of the target part reflected in the analysis target image using an analysis model; and
A display module that displays the analysis target image and the defect information,
Including an inspection system. According to paragraph 1,
Each of the target component and the plurality of components is a cylindrical component,
The inspection system wherein the learning image and the image to be analyzed are photographed to include an inner surface shape and an outer surface shape of the cylindrical part. According to paragraph 2,
The shooting module is,
An inspection system, which is a stereoscopic optical camera that photographs the shape of the inner surface, including the shape of the inner wall of the part corresponding to the direction of extension of the cylindrical shape. According to paragraph 3,
The stereoscopic optical camera,
An inspection system, wherein the image to be analyzed is photographed based on a photographing direction in which the object component is viewed from the cylindrical opening area. According to paragraph 3,
A lighting module that irradiates light toward the target component so that the brightness corresponding to the internal surface shape is greater than or equal to a preset brightness,
An inspection system further comprising: According to paragraph 1,
The defect type includes at least one of scratches, cracks, damage, omission, poor assembly, and foreign matter contamination. According to clause 6,
The label information is displayed overlaid on the learning image to include location information of the defect area corresponding to one of the defect types, standard information of the defect area, and information about the defect type. According to paragraph 2,
An inspection table module on which the target part is seated so as to rotate the target part around the direction of extension of the cylindrical shape,
An inspection system further comprising: According to clause 8,
The imaging module is configured to acquire a plurality of analysis target images according to rotation of the target component. According to paragraph 1,
An inspection system, characterized in that the target part is a motor housing part.

Images