KR20210128357A - Apparatus and method for inspecting surface quality of appearance based on artificial intelligence - Google Patents

Abstract

The present invention relates to a device and method for inspecting surface quality based on artificial intelligence that can accurately and uniformly derive an inspection result and can represent a quantitative numerical value. According to one embodiment of the present invention, the method for inspecting surface quality comprises: a step of receiving image data of an object; a step of pre-processing the input image data; a step of detecting a defect in the object using a deep learning study model based on the pre-processed image data; and a step of storing process information of the object in which defect is detected in a first storage unit when the defect is detected and storing process information of the object in which no defect is detected in a second storage unit when no defect is detected. In the pre-processing step, the pre-processing is performed by changing resolution and reflection components of the input image data.

Description

AI-based exterior surface quality inspection device and method

The present invention relates to an artificial intelligence-based exterior surface quality inspection apparatus and method, and more particularly, to an artificial intelligence-based exterior surface quality inspection apparatus and method for quantitatively measuring exterior surface quality inspection.

In the case of a manufacturing industry that mass-produces products having a specific shape surface by various methods such as injection, die-casting, and press, it is necessary to perform a surface quality inspection of the product appearance to confirm the presence or absence of defects in the manufactured product. Even if there is only a small defect in a part of the product, it can have a large adverse effect on the performance and reliability of the product, as well as lead to product recall, resulting in economic loss. It is one of the very important tasks.

However, the conventional product appearance surface quality inspection in industrial sites mainly relies on the visual inspection method to determine whether there is a defect by the naked eye of the field worker, so the work speed is slow, the productivity is very low, and the judgment standard is vague, so the operator's feeling However, there is a problem that the quality of the product is adversely affected because the result of the judgment varies depending on the operator who inspects it, and the accuracy is lowered because it depends only on the know-how.

Therefore, there is a need to develop a technology that can automate the exterior surface quality inspection to uniformly derive the inspection result and display it as a quantitative numerical value.

Korean Patent Application Laid-Open No. 10-2011-0040965 (published on April 20, 2011)

The present invention has been proposed to solve the above problems, and by inputting image data of the produced product into a deep learning learning model to determine whether there is a defect in the product, the inspection result is derived accurately and uniformly It is to provide an artificial intelligence-based exterior surface quality inspection method that can and can be expressed as a quantitative numerical value.

In addition, the present invention is to provide an artificial intelligence-based exterior surface quality inspection method capable of reducing costs such as working time and labor cost by automating exterior surface quality inspection based on artificial intelligence rather than an operator.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

An artificial intelligence-based exterior surface quality inspection method according to an embodiment of the present invention for solving the above-described problems, the method comprising: receiving image data of an object; pre-processing the input image data; detecting a defect in the object using a deep learning learning model based on the pre-processed image data; and storing process information of the object in which the defect is detected in the first storage unit when the defect is detected, and storing the process information of the object in which the defect is not detected in the second storage unit when the defect is not detected. The pre-processing may include pre-processing by changing the resolution and reflection components of the input image data.

In addition, an artificial intelligence-based exterior surface quality inspection apparatus according to an embodiment of the present invention includes: an input unit for receiving image data of an object; a pre-processing unit for pre-processing the input image data; a detection unit for detecting a defect in the object using a deep learning learning model based on the pre-processed image data; a storage unit including a first storage unit for storing process information of an object in which a defect is detected and a second storage unit for storing process information of an object in which a defect is not detected; And when image data is input, the input image data is pre-processed, a defect of the object is detected using a deep learning learning model based on the pre-processed image data, and the corresponding process information is stored in the first storage unit according to whether the defect is detected. Alternatively, a control unit for controlling the storage to be stored in the second storage unit may be included, wherein the pre-processing unit pre-processes the input image data by changing a resolution and a reflection component.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention, by inputting the image data of the produced product into the deep learning learning model to determine the presence or absence of defects in the product, the inspection result can be accurately and uniformly derived and expressed as a quantitative numerical value. let it be

In addition, according to the present invention, it is possible to reduce costs such as working time and labor cost by automating the external surface quality inspection based on artificial intelligence rather than the operator.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram showing the configuration of an external surface quality inspection apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating a method for inspecting an exterior surface quality according to an embodiment of the present invention.
3 is a diagram illustrating an example to which each method that can be applied to change the resolution of image data is applied according to an embodiment of the present invention.
4 is a diagram illustrating a LAB color space.
5 is a diagram illustrating an example of changing a reflection component based on a LAB color space according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. A spatially relative term should be understood as a term that includes different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component may be placed “above” of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, "part" or "module" is not meant to be limited to software or hardware. A “part” or “module” may be configured to reside on an addressable storage medium and may be configured to reproduce one or more processors. Thus, by way of example, “part” or “module” refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or additional components and “parts” or “modules”. can be further separated.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of an apparatus for inspecting exterior surface quality according to an embodiment of the present invention. In this case, the apparatus for inspecting exterior surface quality (hereinafter, referred to as an 'inspection apparatus') 100 may be a server.

Referring to FIG. 1 , the test apparatus 100 includes an input unit 110 , a preprocessor 120 , a detection unit 130 , a storage unit 140 , and a control unit 150 .

The input unit 110 receives image data of an object to be tested for quality. Here, the image data refers to a two-dimensional or three-dimensional image obtained by photographing an object, and the size of the image data is not limited. Specifically, for the quality inspection of the external surface of the product, the image data may be an image obtained by photographing the surface of the crown of an object, which is a product to be inspected. For example, the image data may be static image data that is one frame, or dynamic image data (ie, moving image data) in which a plurality of frames are continuous.

To this end, the examination apparatus 100 may be connected to a photographing apparatus (eg, a camera) by wire or wirelessly. For example, the imaging device is a test device 100 and Bluetooth (bluetooth) communication, BLE (Bluetooth Low Energy) communication, near field communication unit (Near Field Communication unit), WLAN (Wi-Fi) communication, Zigbee (Zigbee) communication, infrared (IrDA, infrared Data Association) communication, WFD (Wi-Fi Direct) communication, UWB (ultra wideband) communication, Ant+ communication WIFI communication method can be used to communicate, but is not limited thereto.

The pre-processing unit 120 performs pre-processing to input the image data input through the input unit 110 into the deep learning learning model for detection. Specifically, the preprocessor 120 changes the resolution of the input image data to a preset standard, and converts the color channel of the image data whose resolution is changed. Thereafter, at least one frame indicating the state of the reflection component is extracted from the channel-converted image data, and the color component is inverted and reflected in the original image data. Here, the resolution represents the precision (sharpness) of the image data, and the reflection component means the light applied to the image data.

The detection unit 130 detects a defect in the object by inputting the image data pre-processed through the pre-processing unit 120 to the deep learning learning model. At this time, the deep learning learning model for defect detection may be built based on an object detection algorithm, preferably any of You Only Look Once (YOLO) and Single Shot Detector (SSD) capable of real-time object recognition. may contain one.

The storage unit 140 may form a plurality of storage spaces to classify and store each piece of information according to whether a defect is detected in the object. As shown in FIG. 1 , the first storage unit 141 and the second storage unit 142 may be included. Here, the first storage unit 141 stores information about an object in which a defect is not detected, and includes, for example, original image data of the object, pre-processed image data, and information on the quality inspection process. can do. In addition, the second storage unit 142 stores information on the object in which the defect is detected, for example, original data, pre-processed image data, information on the quality inspection process process, defect information, etc. of the object. may include Data stored in the second storage unit 142 may be used to learn a deep learning learning model for defect detection.

When the image data is input, the controller 150 pre-processes the input image data, detects a defect in the object using a deep learning model based on the pre-processed image data, and information on the object according to whether the defect is detected. is controlled to be stored in the first storage unit or the second storage unit.

Specifically, when a defect is detected, the controller 150 stores information on the corresponding object in the first storage unit, and stores information on the corresponding object in the second storage unit when no defect is detected. In the latter case, the control unit 150 may control to generate and output a notification signal for notifying the operator of defect detection. In this case, the notification signal may be visually provided to the operator through the screen of the display device, or may guide the operator to detect the defect through the voice output unit, but is not limited thereto.

2 is a flowchart illustrating a method for inspecting an exterior surface quality according to an embodiment of the present invention.

First, when image data for the object is obtained by photographing the object using at least one camera, the input unit 110 receives the image data (S201), and inputs the input image data to the deep learning learning model. The pre-processing unit 120 performs pre-processing on the input image data to be suitable for the above (S203).

Thereafter, the detection unit 130 detects a defect in the object using a deep learning learning model based on the preprocessed image data (S205).

As a result of the detection, when a defect is not detected, the controller 150 stores information on the corresponding object in the first storage unit 141 ( S207 ), and when a defect is detected, the controller 150 stores the information on the object. information is stored in the second storage unit 142 (S209). Here, the information on the corresponding object may be process information of the corresponding object, and other information may be additionally included.

On the other hand, after step S209, the control unit 150 may output a notification indicating the occurrence of a defect and transmit it to a terminal of a previously registered operator (S211).

The above-described exterior surface quality inspection method according to an embodiment of the present invention may be implemented as a program (or application) and stored in a medium to be executed in combination with a computer, which is hardware.

3 is a diagram illustrating an example to which each method that can be applied to change the resolution of image data is applied according to an embodiment of the present invention.

As described above, the pre-processing unit 120 of the inspection apparatus 100 according to an embodiment of the present invention changes the resolution and reflection components of the image data through pre-processing. In this case, the pre-processing unit 120 adjusts the resolution. Various interpolation methods can be applied to change it.

Specifically, the preprocessor 120 is configured to change the resolution of the input image data by a neighbor interpolation method (Nearest-neighbor interpolation), a bilinear interpolation method (Bilinear interpolation), a bit bilinear interpolation method (Linear exact interpolation), a bicubic interpolation method ( Bicubic interpolation), spline interpolation (cubic spline interpolation), area interpolation (Area interpolation), and can be performed according to various algorithms such as Lanczos interpolation. Here, the neighbor interpolation method refers to the value of the closest pixel, and the bilinear interpolation method calculates a pixel value on real coordinates using four pixel values. In addition, the bit bilinear interpolation method is a method of linearly calculating according to the linear distance in order to estimate the value located between the end points when the values are given. In addition, it is a method of increasing the size using a total of 16 pixels by taking the nearest point, and the Lanczos interpolation method is a method of interpolating using neighboring pixels corresponding to 8X8.

For example, if the resolution of the input image data is higher than the preset standard, the preprocessor 120 adjusts the resolution of the input image data according to the preset standard based on the area interpolation method. Meanwhile, when the resolution of the input image data is lower than the preset standard, the preprocessor 120 increases the resolution of the input image data according to the preset resolution based on the bilinear interpolation method. Here, the preset standard may be changed by a user, an administrator, or the like, and may be changed and set as necessary.

(a), (b), (c), (d), (e) and (f) of FIG. 3 each show a neighbor interpolation method, a bilinear interpolation method, a bit bilinear interpolation method, a bicubic interpolation method, among the various algorithms described above. The image data preprocessed by the area interpolation method and the Lanczos interpolation method are shown.

On the other hand, in the process of carrying out the present invention, light may be used to photograph an object. In this case, an error may occur in detecting a defect due to the effect of lighting on the object, so in order to detect the defect more precisely, Removes the reflective component, i.e., removes the light to reduce its effect.

To this end, the inspection apparatus 100 according to an embodiment of the present invention may change the reflection component of the image data by using the LAB color space. In other words, after changing the resolution of the input image data, the preprocessor 120 converts the changed channel of the image data from the RGB (Red, Green, Blue) channel to the LAB (Lightness, A, B) channel.

4 is a diagram illustrating a LAB color space, and FIG. 5 is a diagram illustrating an example of changing a reflection component based on the LAB color space according to an embodiment of the present invention.

Referring to FIG. 4 , the Lab color space displays colors by using the brightness and color information of an image. The b value represents color information of blue-yellow. That is, by adjusting the L value of the input image data or, in more detail, by reflecting the image with the reduced influence of light on the original image data, it is possible to obtain image data whose reflection component, ie, brightness, is adjusted.

Meanwhile, referring to FIG. 5 , when original image data is input as shown in (a), the resolution is changed using any one of the various algorithms described above, and the image channel of the image data whose resolution is changed as shown in (b) is selected. Converts from RGB (Red, Green, Blue) channels to LAB (Lightness, A, B) channels.

Then, as shown in (c), at least one frame indicating the state of light applied to the image data in which the channel has been previously converted is extracted and inverted by using a specific filter on the L (Lightess) channel corresponding to the light information in the reflection component. . Next, as shown in (d), image data from which the reflection component is removed (reduced) may be obtained by reflecting at least one previously inverted frame on the original image data. Here, as the specific filter, a histogram equalization filter such as Contrast Limited Adaptive Histogram Equalization (CLAHE), a Gaussian filter, or the like may be used.

In this case, the frame corresponding to the at least one inverted frame may be searched for in the original image data, the searched frame may be deleted, and then reflected by replacing the at least one inverted frame. On the other hand, the position of the reflection component in the searched frame is determined, the position corresponding to the reflection component in the searched frame is determined in the inverted at least one frame, and the reflection component in the at least one frame is inverted to the position of the reflection component in the searched frame. Crop and overwrite the partial image data corresponding to . That is, various methods may be applied to the reflection method, and the method is not limited thereto.

The program is a computer such as C, C++, JAVA, machine language, etc. that a processor (CPU) of the computer can read through the device interface of the computer in order for the computer to read the program and execute the methods implemented as a program It may include code (Code) coded in the language. Such code may include functional code related to functions defining functions necessary for executing the methods, etc. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers that the computer can access or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected by a network, and a computer readable code may be stored in a distributed manner.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software model executed by hardware, or by a combination thereof. Software models may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: Appearance surface quality inspection device
110: input unit 120: pre-processing unit
130: detection unit 140: storage unit
141: first storage 142: second storage
150: control unit

Claims (10)

A method for inspecting the quality of an exterior surface in an exterior surface quality inspection apparatus, the method comprising:
receiving image data of an object;
pre-processing the input image data;
detecting a defect in the object using a deep learning learning model based on the pre-processed image data; and
When a defect is detected, storing the process information of the object in which the defect is detected in the first storage unit, and if the defect is not detected, storing the process information of the object in which the defect is not detected in the second storage unit,
The pre-processing is characterized in that the pre-processing is performed by changing the resolution and reflection components of the input image data,
Appearance surface quality inspection method.
According to claim 1,
The pre-processing step is
changing the resolution of the input image data to a preset standard;
converting a color channel of the changed image data;
extracting at least one frame indicating a state of a reflection component from the converted image data; and
inverting the color components of the extracted at least one frame; and
It characterized in that it comprises the step of reflecting the inverted at least one frame to the original image data,
AI-based exterior surface quality inspection method.
3. The method of claim 2,
The resolution of the input image data is,
Neighbor interpolation, bilinear interpolation, bit bilinear interpolation, bicubic interpolation, domain interpolation, characterized by changing using any one of Lanczos interpolation method,
AI-based exterior surface quality inspection method.
4. The method of claim 3,
The adjusting step is
If the resolution of the input image data is higher than the preset standard, lowering the resolution of the input image data to the preset standard based on the area interpolation method;
When the resolution of the input image data is lower than the preset standard, it is characterized in that the resolution of the input image data is increased to the preset standard based on the bilinear interpolation method,
AI-based exterior surface quality inspection method.
According to claim 1,
The deep learning learning model is
Characterized in that it includes any one of YOLO (You Only Look Once) and SSD (Single Shot Detector) capable of real-time object recognition,
AI-based exterior surface quality inspection method.
3. The method of claim 2,
The step of converting the color channel is converting the image channel of the changed image data from an RGB (Red, Green, Blue) channel to a LAB (Lightness, A, B) channel,
The extracting of the at least one frame may include extracting at least one frame indicating the state of light applied to the converted image data by using a specific filter in the L (Lightness) channel corresponding to the light information in the reflection component. characterized in that
AI-based exterior surface quality inspection method.
an input unit for receiving image data of an object;
a pre-processing unit for pre-processing the input image data;
a detection unit for detecting a defect in the object using a deep learning learning model based on the pre-processed image data;
a storage unit including a first storage unit for storing process information of an object in which a defect is detected and a second storage unit for storing process information of an object in which a defect is not detected; and
When image data is input, the input image data is pre-processed, a defect of the object is detected using a deep learning model based on the pre-processed image data, and the corresponding process information is stored in the first storage unit or depending on whether the defect is detected or not. It includes a control unit for controlling to be stored in the second storage unit,
The pre-processing unit is characterized in that the pre-processing by changing the resolution and reflection component of the input image data,
Artificial intelligence-based exterior surface quality inspection device.
8. The method of claim 7,
The preprocessor is
By changing the resolution of the input image data to a preset standard, converting the color channel of the changed image data, extracting at least one frame indicating the state of the reflection component from the converted image data, and inverting the color component characterized in that it is reflected in the original image data,
Artificial intelligence-based exterior surface quality inspection device.
11. The method of claim 10,
The resolution of the input image data is changed using any one of neighbor interpolation, bilinear interpolation, bit bilinear interpolation, bicubic interpolation, region interpolation, and Lanczos interpolation,
If the resolution of the input image data is higher than the preset standard, the pre-processing unit adjusts the resolution of the input image data to the preset standard based on the area interpolation method, and the resolution of the input image data is higher than the preset standard. If it is lower than the preset standard, increasing the resolution of the input image data to the preset resolution based on the bilinear interpolation method,
The deep learning learning model, characterized in that it includes any one of YOLO (You Only Look Once) and SSD (Single Shot Detector) capable of real-time object recognition,
Artificial intelligence-based exterior surface quality inspection device.
11. The method of claim 10,
The preprocessor is
When converting the color channel, the image channel of the changed image data is converted from an RGB (Red, Green, Blue) channel to a LAB (Lightness, A, B) channel,
When extracting the at least one frame, extracting at least one frame indicating the state of light applied to the converted image data by using a specific filter on the L (Lightness) channel corresponding to the information of light in the reflection component characterized in that
Artificial intelligence-based exterior surface quality inspection device.

Images