KR20220163694A - System and method for quality management of parts based on distribution of combined values based on deep learning - Google Patents

Abstract

A system and method for managing quality of components based on defect value distribution based on deep learning are disclosed. In the present invention, the system for managing quality of components based on defect value distribution based on deep learning comprises: a camera which takes images of pre-selected good products and pre-selected defective products among the components; and a computing device which receives a first image of the good product and a second image of the defective product from the camera, generates a first difference image based on the first image and the second image, receives a masking input for the first difference image according to a field worker's subjective quality discrimination criterion, and generates a second difference image by masking the first difference image based on the masking input, wherein the computing device performs deep learning of a deep neural network by defining the first image and the second image as input values to the deep neural network and defining the second difference image as an output value to the deep neural network, calculates a size of a remaining image element of the second difference image as a defect value, calculates an inspection reference value through distribution of the defect values, and classifies the components into good products and defective products based on the inspection reference value to manage quality.

Description

System and method for quality management of parts based on deep learning based defect value distribution {SYSTEM AND METHOD FOR QUALITY MANAGEMENT OF PARTS BASED ON DISTRIBUTION OF COMBINED VALUES BASED ON DEEP LEARNING}

The present invention relates to a method and system for quality control of parts based on deep learning-based defect value distribution. More specifically, it relates to a system and method capable of learning a correlation between an input image and an output image through a deep neural network and managing the quality of a part by utilizing the learned neural network.

In general, parts constituting a mechanical device may be classified into quality grades based on their shape, color, material, material, strength, and the like. Through this, when a part is shipped, a difference in price may occur depending on the quality state or grade. To this end, various inspections are conducted before shipment of the part to classify whether the part is a good product or a defective product.

Looking at KR100907058B1, which is a prior art, techniques for extracting a separate current and the like to determine the quality of arc welding and determining the quality of arc welding through this are disclosed.

As such, the prior art has a problem in that the quality of the target can be determined only when a separate sensor or the like is additionally provided. Recently, with the rapid development of deep learning technology and vision recognition algorithms, the need for a system and method capable of measuring the quality of parts only with a camera without a separate sensor and managing the quality through this has been raised.

KR100907058B1 (Title of invention: Apparatus and method for judging arc welding quality)

An object of the present invention is to provide a system and method capable of managing quality of parts by measuring quality only with part images using a vision recognition algorithm.

In addition, an object of the present invention is to introduce an optimized application filter to more accurately generate a difference image during vision recognition, and to generate an image through a camera including a stain-resistant lens that is resistant to contaminants and salt, thereby creating a clear image in any environment. It is to provide a system and method for quality control of parts based on a deep learning-based defect value distribution that generates and performs a more accurate vision recognition algorithm.

In order to solve the above-mentioned problems, the present invention is a quality control system for parts based on a deep learning-based defect value distribution, a camera for photographing pre-selected good products and pre-selected defective products among the parts, and photographing the good products. A first image and a second image of the defective product are received from the camera, a first difference image is generated based on the first image and the second image, and the quality of the defective product is determined according to subjective criteria of a field worker. A computing device that receives a masking input for a first difference image and generates a second difference image by masking the first difference image based on the masking input, wherein the computing device includes the first image and the first difference image. A second image is defined as an input value to the deep neural network, the second difference image is defined as an output value of the deep neural network, the deep neural network is deep learning trained, and the size of the remaining image elements of the second difference image is a defect value. , and an inspection reference value is calculated through the distribution of the defect values, and quality can be managed by classifying the parts into good products and defective products based on the inspection reference values.

In addition, the computing device may generate a first enlarged image obtained by enlarging the first difference image, and may receive a masking input for the first enlarged image according to a field worker's subjective criterion for determining a good product.

Also, the computing device may receive a region of interest from a field worker and generate the first enlarged image around the region of interest.

The camera may include a camera module that acquires the first image, and the camera module may include a housing including a through hole on a side wall and a lens installed in the through hole.

The computing device generates at least one corrected image by applying an image correction filter to at least one of the first image and the second image, and the first difference based on the at least one corrected image. image can be created.

In addition, in order to solve the above-mentioned problems, the present invention is a quality control method of parts based on a deep learning-based defect value distribution, in which a first image of a pre-selected non-defective product among the parts and a pre-selected among the parts Receiving a second image of a defective product; Generating a first difference image based on the first image and the second image; Receiving a user input for a region of interest among the first difference images; Generating a first enlarged image by enlarging the region of interest in the first difference image, image elements that are determined to be defective among image elements constituting the first enlarged image according to subjective criteria for determining good quality by field workers. Receiving a masking input from a worker by selecting a mask, processing remaining image elements other than the image element selected by the masking input in the first enlarged image, leaving only elements that the field worker considers important in determining defects. generating a second difference image, deep learning a correlation between input and output data by a second deep neural network by taking the first image and the second image as inputs and outputting the second difference image; Calculating the size of the remaining image elements of the second difference image as a defect value and calculating an inspection reference value through the defect value distribution of the second difference image, and classifying the inspection target part based on the inspection reference value. can

In addition, in order to solve the above-mentioned problems, the present invention is a non-transitory computer-readable medium storing instructions, which, when executed by a processor, cause the processor to perform the above-described method, a non-transitory computer-readable medium It can be a possible medium.

The present invention has an effect of recognizing the distribution of deep learning-based defect values, measuring the quality of parts only with a camera without having a separate sensor through this, and managing the quality of parts through this.

In addition, the present invention has an effect of helping to manage parts by measuring the dimensions of parts while managing quality.

In addition, the present invention creates a clear image in any environment by introducing an optimized application filter to more accurately create a difference image during vision recognition and to create an image through a camera including a stain-resistant lens that is resistant to contaminants and salt. Thus, a more accurate vision recognition algorithm can be performed.

Effects obtainable according to the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 shows a part quality control system based on deep learning-based defect value distribution according to the present invention.
2 is a diagram illustrating a computing device according to the present invention.
3 shows a memory according to the present invention as a functional element.
4 to 6 show an embodiment of a part quality control system based on deep learning-based defect value distribution according to the present invention.
7 is a graph showing inspection reference values according to the present invention.
8 is a view showing a camera module included in a camera according to the present invention.
9 is a diagram showing an example of an image correction filter according to the present invention.
10 is a diagram showing a three-dimensional HSV graph according to the present invention.
11 is a diagram illustrating an example of an HSV value of one color.
12 shows a method for quality control of parts based on deep learning-based defect value distribution according to the present invention.
13 shows steps S140 and S150 according to the present invention.
The accompanying drawings, which are included as part of the detailed description to aid understanding of the present specification, provide examples of the present specification and describe technical features of the present specification together with the detailed description.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted.

In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted.

In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Hereinafter, based on the above description, a detailed description of a part quality control system based on a deep learning-based defect value distribution according to a preferred embodiment of the present specification is as follows.

1 shows a part quality control system based on deep learning-based defect value distribution according to the present invention.

According to FIG. 1 , a system for quality control of parts based on deep learning based defect value distribution according to the present invention may include a camera 100 and a computing device 200 . The camera 100 according to the present invention can take pictures of pre-selected good products and pre-selected defective products among parts. Through photography, the camera 100 may generate a first image of a good product and a second image of a defective product.

The computing device 200 according to the present invention may receive the first image and the second image from the camera 100 and generate a first difference image based on the first image and the second image.

To generate the first difference image, the computing device 200 generates at least one corrected image by applying an image correction filter to at least one of the first image and the second image, and generates the at least one corrected image. A first difference image may be generated based on the image. Since it is necessary to extract commonalities and differences for each image of parts, the efficiency of vision recognition can be increased through an image correction filter. A detailed description of this will be given later.

In addition, the computing device 200 receives a masking input for the first difference image according to the field worker's subjective criterion for determining a good product, and generates a second difference image by masking the first difference image based on the masking input. can

In addition, the computing device 200 may be connected to the user input unit 300, and the user input unit 300 may receive masking and/or information about a region of interest from a user or a field worker. The computing device 200 may receive masking and/or information about a region of interest through the user input unit 300 .

In addition, the computing device 200 defines the first image and the second image as input values to the deep neural network, defines the second difference image as an output value of the deep neural network, performs deep learning training on the deep neural network, and defines the second difference image as an output value of the deep neural network. Quality can be managed by calculating the size of the remaining image elements as a defect value, calculating an inspection reference value through the distribution of the defect values, and classifying parts into good and defective products based on the inspection reference value.

In addition, the computing device 200 may generate a first enlarged image obtained by enlarging the first difference image, and may receive a masking input for the first enlarged image according to a field worker's subjective criterion for determining a good product. In this case, the computing device 200 may receive a region of interest from a field worker and generate a first enlarged image around the region of interest.

In addition, the computing device 200 may be a subject that performs a quality control method of a part based on a deep learning-based defect value distribution according to another preferred embodiment of the present invention, which will be described later.

2 is a diagram illustrating a computing device according to the present invention.

According to FIG. 2 , a computing device 200 according to the present invention may include a processor 210 , a memory 220 and a communication module 230 .

The processor 210 may execute commands stored in the memory 220 to control other components. The processor 210 may execute instructions stored in the memory 220 .

The processor 210 is a component capable of performing calculations and controlling other devices. Mainly, it may mean a central processing unit (CPU), an application processor (AP), a graphics processing unit (GPU), and the like. Also, the CPU, AP, or GPU may include one or more cores therein, and the CPU, AP, or GPU may operate using an operating voltage and a clock signal. However, while a CPU or AP consists of a few cores optimized for serial processing, a GPU may consist of thousands of smaller and more efficient cores designed for parallel processing.

The processor 210 may provide or process appropriate information or functions to a user by processing signals, data, information, etc. input or output through the components described above or by running an application program stored in the memory 220.

The memory 220 stores data supporting various functions of the server 200 . The memory 220 may store a plurality of application programs (application programs or applications) running in the server 200, data for operation of the server 200, and commands. At least some of these application programs may be downloaded from the external server 200 through wireless communication. Also, the application program may be stored in the memory 220, installed in the server 200, and driven by the processor 210 to perform the operation (or function) of the server 200.

The memory 220 may be a flash memory type, a hard disk type, a solid state disk type, a silicon disk drive type, or a multimedia card micro type. ), card-type memory (eg SD or XD memory, etc.), RAM (random access memory; RAM), SRAM (static random access memory), ROM (read-only memory; ROM), EEPROM (electrically erasable programmable read -only memory), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. Also, the memory 220 may include a web storage performing a storage function on the Internet.

The communication module 230 may be a component for communicating with an external server or the camera 100 . In the case of the communication module 230, information is transmitted and received with a base station or an external server or camera 100 having a communication function through an antenna. The communication module 230 may include a modulation unit, a demodulation unit, a signal processing unit, and the like.

Wireless communication may refer to communication using a wireless communication network using a communication facility previously installed by telecommunication companies and a frequency of the communication facility. At this time, the communication module 230 is CDMA (code division multiple access), FDMA (frequency division multiple access), TDMA (time division multiple access), OFDMA (orthogonal frequency division multiple access), SC-FDMA (single carrier frequency division multiple access), and the like, as well as the communication module 230 can also be used for 3rd generation partnership project (3GPP) long term evolution (LTE). In addition, not only 5G communication currently commercialized, but also 6G communication scheduled for commercialization in the future may be used. However, the present specification may utilize a pre-installed communication network without being bound by such a wireless communication method.

In addition, as a short range communication technology, Bluetooth, BLE (Bluetooth Low Energy), Beacon, RFID (Radio Frequency Identification), NFC (Near Field Communication), infrared communication (Infrared Data Association; IrDA) ), UWB (Ultra Wideband), ZigBee, etc. may be used.

3 shows a memory according to the present invention as a functional element.

According to FIG. 3, the memory 210 according to the present invention includes a region of interest receiving module 211, a masking input receiving module 212, a first difference image generating module 213, an enlarged image generating module 214, a second It may include a difference image generation module 215 , a component size measurement module 216 , and a resolution correction module 217 .

The region of interest receiving module 211 may receive information input by a field worker about a portion desired to be enlarged and viewed in the first difference image through a user input unit. The ROI receiving module 211 may generate an ROI based on the received information and transmit the generated ROI to the first enlarged image generating module 214 . The region of interest may be selected based on a difference between the first image and the second image.

The masking input receiving module 212 may receive a masking input for a specific image element according to a subjective non-good quality criterion in the first difference image through a user input unit. The masking input receiving module 212 may transmit the received input to the second difference image generating module 215 .

The first difference image generation module 213 may generate a first difference image based on the above-described first image and second image. The first difference image is an image excluding a common part between the first image and the second image, and may refer to an image in which only differences between the first image and the second image are left.

The magnified image generation module 214 may generate an enlarged image centered on the region of interest in the first difference image. The first enlarged image may enlarge the first difference image at a preset magnification.

The second difference image generation module 215 receives a masking input for the first difference image according to the field worker's subjective good quality criterion, masks the first difference image based on the masking input, and generates a second difference image. can do.

The part size measuring module 216 may capture a reference object having a reference size in an image together, and measure the size of the part based on a length difference between the reference object and the image. As such, the computing device 200 according to the present invention can help quality management by automatically measuring the size of a part.

The resolution correction module 217 may be configured to increase the resolution when the resolution is reduced when generating an enlarged image so that a field worker can easily input masking. The resolution correction module 217 may utilize a resolution upscaling algorithm.

In addition, the memory 210 according to the present invention may further include a masking input learning module 218 . The masking input learning module 218 may be a module capable of performing a similar masking input by learning a DB in which a masking input of a field worker is stored in advance and a DB and performing regression analysis. The masking input learning module 218 may perform masking input instead of a field learner by learning the subjective good quality criterion of a field worker according to an embodiment of the present invention in advance.

4 to 6 show an embodiment of a part quality control system based on deep learning-based defect value distribution according to the present invention.

According to FIG. 4 , the system for quality control of parts based on the distribution of defect values based on deep learning according to the present invention may generate a first difference image by subtracting a second image from a first image. The first difference image may include feature elements included in the second image. The system for quality control of parts based on the distribution of defect values based on deep learning according to the present invention may select elements determined to be defective by a worker among characteristic elements of the first difference image through masking.

According to FIG. 5 , the system for quality control of parts based on the distribution of defect values based on deep learning according to the present invention may generate a second difference image in which only elements determined to be defective selected through masking are left in the first difference image. The second difference image may refer to an image in which elements other than elements determined to be defective in the first difference image are masked and covered.

According to FIG. 5, the system for quality control of parts based on the deep learning-based defect value distribution according to the present invention generates a first enlarged image, receives a masking input to the generated first enlarged image, and based on this, a second A difference image can be created.

According to FIG. 6 , the deep neural network of the present invention means a system or network that builds one or more layers in one or more computers and performs a judgment based on a plurality of data. For example, a deep neural network can be implemented with a set of layers including a convolutional pooling layer, a locally connected layer, and a fully-connected layer. For example, the overall structure of the deep neural network may be a convolutional neural network (ie, convolutional neural network; CNN) structure in which a local access layer is followed by a convolutional pooling layer, and a fully connected layer is followed by a local access layer. In addition, the deep neural network may be formed, for example, in a recurrent neural network (RNN) structure in which nodes of each layer include edges pointing to the nodes and are connected recursively. The deep neural network may include various criterion (ie, the first image, the second image, and the second difference image), and a new criterion may be added by analyzing an input image. However, the structure of the deep neural network according to the embodiment of the present invention is not limited thereto, and may be formed with various structures of neural networks.

In an embodiment of the present invention, a good product sorting system including a deep neural network may be implemented in a single computer or may be implemented through a network by connecting a plurality of computers.

That is, the deep neural network of the present invention can learn the correlation between the first image and the second image and the second difference image as an output.

7 is a graph showing inspection reference values according to the present invention.

According to FIG. 7 , defect values of the second difference image may have various distributions. For example, if the distribution of defect values of good products is mostly distributed between 0 and 10 based on the defect value size, and the distribution of defect values of defective products is mostly distributed over 90 based on the defect value size, the distribution of defect values shows the difference between good and defective products. The defect size criterion of 50, which is the median value of the boundary value, may be used as the learned inspection criterion value.

That is, the part quality control system based on the deep learning-based defect value distribution according to the present invention receives the defect values of the first image and the second image, compares the received defect values with the inspection reference value, and distinguishes the parts from good products. It can be classified as defective.

8 is a view showing a camera module included in a camera according to the present invention.

According to FIG. 8 , the camera 100 according to the present invention may include a camera module 1320. The camera module 1320 may generate a video image from an optical image. The camera module 1320 may include a housing 1324 including a through hole on a side wall, a lens 1321 installed in the through hole, and a driving unit 1323 that drives the lens 1321 . The through hole may have a size corresponding to the diameter of the lens 1321 . The lens 1321 may be inserted into the through hole.

The driving unit 1323 may be configured to control the lens 1321 to move forward or backward. The lens 1321 and the driving unit 1323 are connected in a conventionally known manner, and the lens 1321 may be controlled by the driving unit 1323 in a conventionally known manner.

In order to obtain various video images, the lens 1321 needs to be exposed to the outside of the camera module 1320 or the housing 1324 .

In particular, since the camera according to the present invention needs to be directly installed in the outer area, a lens with strong contamination resistance is required. Accordingly, the present invention proposes a coating layer for coating a lens to solve this problem.

Preferably, the lens 1321 may include an acrylic compound represented by Formula 1 below on its surface; It may be coated with a coating composition containing an organic solvent, inorganic particles and a dispersant.

[Formula 1]

here,

n and m are the same as or different from each other, each independently represent an integer of 1 to 100, and L ₁ is a biphenylene group.

When the lens 1321 is coated with the coating composition, it can exhibit excellent water repellency and stain resistance, so even if the lens 1321 installed in the external environment is exposed to a polluted environment and salt for a long time, it is possible to collect clear and clear external images. .

The inorganic particles may be selected from the group consisting of silica, alumina, and mixtures thereof. The average diameter of the inorganic particles is 70 to 100 μm, but is not limited to the above examples. After forming the coating layer 1322 on the surface of the lens 1321, the inorganic particles may improve physical strength and maintain viscosity within a certain range to increase moldability.

The organic solvent is selected from the group consisting of methyl ethyl ketone (MEK), toluene, and mixtures thereof, and preferably methyl ethyl ketone may be used, but is not limited to the above examples.

As the dispersing agent, a polyester-based dispersing agent may be used, and specifically, TEGO-Disperse 670 (manufacturer : EVONIK) can be used, but it is not limited to the above examples, and all dispersants obvious to those skilled in the art can be used without limitation.

The coating composition may further include a stabilizer as other additives, and the stabilizer may include a UV absorber, an antioxidant, and the like, but is not limited to the above examples and may be used without limitation.

The coating composition for forming the coating layer 1322 may more specifically include an acrylic compound represented by Chemical Formula 1; organic solvents, inorganic particles and dispersants.

The coating composition may include 40 to 60 parts by weight of the acrylic compound represented by Formula 1, 20 to 40 parts by weight of inorganic particles, and 5 to 15 parts by weight of a dispersant, based on 100 parts by weight of the organic solvent. In the case of the above range, a synergistic effect is expressed to the extent that the water repellency effect due to the interaction of each component is of critical significance, and when it is out of the above range, the synergistic effect is rapidly reduced or almost nonexistent.

More preferably, the viscosity of the coating composition is 1500 to 1800 cP, and when the viscosity is less than 1500 cP, when applied to the surface of the lens 1321, there is a problem that it is not easy to form the coating layer 1322 because it flows down, and the coating composition exceeds 1800 cP. In this case, there is a problem in that it is not easy to form a uniform coating layer 1322.

[Preparation Example 1: Preparation of coating layer]

1. Preparation of coating composition

A coating composition was prepared by mixing methyl ethyl ketone with an acrylic compound represented by Formula 1, inorganic particles, and a dispersant:

[Formula 1]

here,

n and m are the same as or different from each other, each independently represent an integer of 1 to 100, and L ₁ is a biphenylene group.

A more specific composition of the antistatic composition is shown in Table 1 below.

TX1 TX2 TX3 TX4 TX5 organic solvent 100 100 100 100 100 acrylic compound 30 40 50 60 70 inorganic particles 10 20 30 40 50 dispersant One 5 10 15 20

(unit weight parts)

2. Preparation of coating layer

A coating layer 1322 was formed by applying the coating composition of DX1 to DX5 on one surface of the lens 1321 and curing the coating composition.

[Experimental example]

1. Evaluation of surface appearance

Due to the difference in viscosity of the coating composition, after the coating layer 1322 was prepared, a sensory evaluation was performed on whether a uniform surface was formed. Evaluation was conducted on whether or not a uniform coating layer 1322 was formed, and the evaluation was conducted according to the following criteria.

○: uniform coating layer formation

×: Formation of non-uniform coating layer

TX1 TX2 TX3 TX4 TX5 sensory evaluation Х ○ ○ ○ Х

When forming the coating layer 1322, if the viscosity is less than a certain amount, flow occurs on the surface of the lens 1321, and it is difficult to form a uniform coating layer 1322 after the curing process in many cases. Accordingly, a problem of lowering the production yield may occur. In addition, even when the viscosity is too high, it is difficult to uniformly apply the composition, and it is impossible to form a uniform coating layer 1322.

2. Measurement of water repellency angle

After forming the coating layer 1322 on the surface of the lens 1321, the results of measuring the water repellency angle are shown in Table 3 below.

Advancing contact angle (") Static contact angle (“) receding contact angle (“) TX1 117.1±2.9 112.1±4.1 < 10 TX2 132.4±1.5 131.5±2.7 141.7±3.4 TX3 138.9±3.0 138.9±2.7 139.8±3.7 TX4 136.9±2.0 135.6±2.6 140.4±3.4 TX5 116.9±0.7 115.4±3.0 < 10

As shown in Table 3, after the coating layer 1322 was formed using the coating compositions of TX1 to TX5, the result of measuring the contact angle was confirmed. TX1 and TX5 measured receding contact angles less than 10 degrees. That is, it was confirmed that a phenomenon in which water droplets are pinned occurs when the coating composition is out of the optimal range for preparing the coating composition. On the other hand, it was confirmed that the pinning phenomenon did not occur in TX2 to 4, indicating that excellent waterproofing effect could be exhibited.

3. Fouling resistance evaluation

A lens 1321 having a coating layer 1322 according to the above embodiment formed outside the facility was attached to a model camera, and exposed to an external (outdoor) environment for 4 days. As the comparative example (Con), the same lens 1321 without the coating layer 1322 was used, and the model camera was attached to the same position in each example.

After that, the degree of contamination and corrosion of the lens 1321 before and after the experiment was evaluated as related, and for objective comparison, the result was compared with a comparative example in which the coating layer 1322 was not formed, and the result was evaluated by an index of 1 to 10, and the following Table 5 shows. In the index below, the lower the number, the better the stain resistance.

Con TX1 TX2 TX3 TX4 TX5 stain resistance 10 7 3 3 3 8

(Unit: index)

Referring to Table 4, in the case of forming the coating layer 1322 on the lens 1321, even if the lens 1321 is exposed to the outside while installing the camera in the external environment, high fouling resistance can be easily analyzed for a long period of time. It can be seen that image data can be collected. In particular, in the case of TX2 to TX4, it can be confirmed that the contamination resistance by the coating layer 1322 is very excellent.

9 is a diagram showing an example of an image correction filter according to the present invention.

According to Figure 9, the type and function of the filter is shown. That is, the CNN algorithm may be a learning algorithm using a plurality of layers. In addition, the CNN algorithm can automatically learn a filter that maximizes image classification accuracy, and by adding a new layer called a convolutional layer and a polling layer before the fully connected layer, after applying the filtering technique to the original image, the filtered image A classification operation can be performed on . The CNN algorithm is configured to apply a filtering technique to the original image by adding a new layer, called a convolutional layer and a pooling layer, before the fully-connected layer, and then perform a classification operation on the filtered image. can

At this time, the operation expression for the image filter using the CNN algorithm is as shown in Equation 1 below.

[Equation 1]

(only,

: 행렬로 표현된 필터링된 이미지의 i번째 행, j번째 열의 픽셀,

: pixels in the i-th row and j-th column of the filtered image expressed as a matrix,

: 필터,

: filter,

: 이미지,

: image,

: 필터의 높이 (행의 수),

: height of the filter (number of rows),

: 필터의 너비 (열의 수)이다. )

: The width of the filter (number of columns). )

Preferably, the operation expression for the image filter using the CNN algorithm is as shown in Equation 2 below.

[Equation 2]

(only,

: 행렬로 표현된 필터링된 이미지의 i번째 행, j번째 열의 픽셀,

: pixels in the i-th row and j-th column of the filtered image expressed as a matrix,

: 응용 필터

: Application filter

: 이미지,

: image,

: 응용 필터의 높이 (행의 수),

: height of application filter (number of rows),

: 응용 필터의 너비 (열의 수)이다.)

: The width (number of columns) of the application filter.)

는 응용 필터로서 제1 차이 이미지를 생성하기 위하여 제1 이미지 및/또는 제2 이미지에 적용되는 필터일 수 있다. 특히, 부품의 경우 (1) 색상의 차이와 (2) 형태의 차이를 기초로 품질을 매길 수 있어, 색상 및 형태를 효과적으로 인지하기 위한 응용 필터가 필요할 수 있다. 이러한 필요성을 충족하기 위하여 응용 필터

는 아래의 수학식 3에 의하여 연산될 수 있다. Preferably,

may be an application filter applied to the first image and/or the second image to generate the first difference image. Particularly, in the case of parts, quality can be determined based on (1) difference in color and (2) difference in shape, so an application filter for effectively recognizing color and shape may be required. Application filters to meet these needs

Can be calculated by Equation 3 below.

[Equation 3]

: 필터,

: 계수,

: 응용 필터)(only,

: filter,

: Coefficient,

: application filter)

는 부품의 종류에 따라 나뉘어 제1 이미지 및/또는 제2 이미지에 적용될 수 있다. 이때, 각

에 따른 필터는 도 9에 따른 엣지 인식 필터(Edge detection), 샤픈 필터(sharpen) 및 박스 블러 필터(Box blur) 중 어느 하나의 행렬일 수 있다. Preferably, the application filter according to the present invention

may be divided according to the type of part and applied to the first image and/or the second image. At this time, each

The filter according to may be a matrix of any one of an edge detection filter according to FIG. 9, a sharpen filter, and a box blur filter.

를 구하는 연산식은 아래의 수학식 4와 같다. 이때,

는 필터의 효율을 높이기 위하여 사용되는 하나의 상수로서 해석될 수 있고, 그 단위는 무시될 수 있다. Preferably,

The operation expression for obtaining is as shown in Equation 4 below. At this time,

can be interpreted as a constant used to increase the efficiency of the filter, and its unit can be ignored.

[Equation 4]

However, the diameter (diameter) of the lens of the camera used to take the image is in mm, the f-value of the lens is the aperture value (F number) of the lens, and the HSV average value is the size of the color coordinates according to the image of the part through the HSV graph It may mean a value obtained by averaging values converted into values. Details of the HSV value are as follows.

10 is a diagram showing a three-dimensional HSV graph according to the present invention, and FIG. 11 is a diagram showing an example of HSV values of one color.

10 and 11, the HSV graph according to the present invention may mean a color space in which perceptual characteristics are reflected. H (Hue, 0 to 360°) may mean hue, S (Saturation, 0 to 100%) may mean saturation, and V (Value, 0 to 100%) may mean lightness. . Hue, saturation, and lightness values can be referred to as HSV values, which can be easily extracted using graphic tools such as Adobe illustrator cc 2019.

The HSV average value according to the present invention can be obtained through the HSV three-dimensional coordinates of FIG. 10 . That is, the average HSV value according to the present invention may be calculated based on the HSV value obtained through the graphic tool as shown in FIG. 11 . The distance value of the HSV coordinates measured with the origin coordinates on the HSV 3-dimensional coordinates as a reference point may constitute the above-described HSV average value. That is, the image according to the present invention includes an image of a part, and the HSV color coordinates of the image of the part may be distributed in a certain area. Therefore, the HSV average value according to the present invention may mean a distance value from the origin coordinates calculated based on average coordinates using the average of HSV coordinates for colors of a certain region.

However, in the case of a part, there may be a large difference in color for each part, so in the present invention, an average HSV value can be obtained for each divided area of the part, and different application filters can be applied to each divided area.

[Experimental Example]

The results of analyzing the accuracy of the first difference image generated when the applied filter F' of the present invention is applied to the image generated using the CNN algorithm according to the present invention are shown in Table 1 below. At this time, the accuracy indicates the correspondence between the difference elements shown in the first difference image and the difference elements directly selected by the expert as a numerical value.

no filter applied

적용filter

apply filter

apply accuracy 49 62 84

(unit: %)

는 도 9의 엣지 인식 필터(Edge detection)로 실험되었다. Table 5 above shows, for each case, the accuracy of numerically expressing the correspondence between the difference elements of the first difference image according to the present invention and the difference elements directly selected by the expert. Table 5 summarizes the results performed based on 100 samples for a total of one type of part, and the accuracy is a rounded value. At this time, the general filter

was experimented with the edge detection filter in FIG. 9.

를 적용하는 것이 더 높은 정확도를 나타내었다. 또한, 일반 필터

를 적용하는 것보다, 본 발명에 따른 응용 필터

를 적용하는 것이 현저히 높은 정확도를 나타내었다. As can be seen in Table 5, the case of creating a first difference image without applying a filter is more than a normal filter.

, showed higher accuracy. Also, the general filter

Rather than applying the applied filter according to the present invention

, showed significantly higher accuracy.

Hereinafter, based on the above description, a detailed description of a method for quality control of a part based on a deep learning-based defect value distribution according to another preferred embodiment of the present specification is as follows.

12 shows a method for quality control of parts based on deep learning-based defect value distribution according to the present invention.

According to FIG. 12, the quality control method of a part based on the deep learning-based defect value distribution according to the present invention includes the steps of receiving a first image of a pre-selected good product and a second image of a pre-selected defective product ( S110), generating a first difference image based on the first image and the second image (S120), receiving a user input for a region of interest among the first difference images (S130), interest in the first difference image Generating a first enlarged image by enlarging the area (S140), selecting an image element judged to be defective among image elements constituting the first enlarged image according to the field worker's subjective quality discrimination criteria from the operator, and masking the operator Receiving an input (S150), masking the remaining image elements except for the image element selected by the masking input in the first enlarged image, leaving only the elements that the field worker considers important in determining the defect. A masked second difference image Generating (S160), taking the first image and the second image as inputs and outputting a second difference image so that the second deep neural network learns the correlation between the input and output data by deep learning (S170), the second difference Calculating the size of the remaining image elements of the image as a defect value and calculating an inspection reference value through the defect value distribution of the second difference image (S180), and classifying agricultural products to be inspected based on the inspection reference value (S180). can do.

The method for controlling quality of parts based on the distribution of defect values based on deep learning according to the present invention includes all processes performed by the computing device of the system for managing quality of parts based on the distribution of deep learning based defect values according to the above-described embodiment. can include

13 shows steps S140 and S150 according to the present invention.

According to FIG. 13 , steps S140 and S150 according to the present invention include receiving information on a region of interest from a field worker (S151), enlarging a first difference image based on the region of interest (S152), and enlarging the image It may include up-scaling the resolution of (S153) and receiving a masking input from a field worker based on the up-scaled enlarged image (S154).

Each of the above-described steps can be clearly understood as the contents described in the system according to an embodiment of the present invention, and can be supported thereby.

The above-described present invention can be modeled as a computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. , and also includes those modeled in the form of carrier waves (eg, transmission over the Internet). Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of this specification should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of this specification are included in the scope of this specification.

Any or other embodiments of the present invention described above are not mutually exclusive or distinct. Certain or other embodiments of the present invention described above may be used in combination or combination of respective configurations or functions.

The above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: camera
200: computing device

Claims (7)

In the quality control system of parts based on deep learning-based defect value distribution,
a camera for photographing pre-selected good products and pre-selected defective products among the parts; and
A first image of the non-defective product and a second image of the defective product are received from the camera, a first difference image is generated based on the first image and the second image, and subjective determination of the non-defective product by field workers A computing device that receives a masking input for the first difference image according to a criterion and generates a second difference image by masking the first difference image based on the masking input,
The computing device,
The first image and the second image are defined as input values to the deep neural network, the second difference image is defined as an output value of the deep neural network, the deep neural network is subjected to deep learning training, and the residual image of the second difference image is defined. Calculating the size of the element as a defect value, calculating an inspection reference value through the distribution of the defect value, and classifying the parts into good and defective products based on the inspection reference value to manage quality,
Parts quality management system based on deep learning based defect value distribution. According to claim 1,
The computing device,
Generating a first enlarged image obtained by enlarging the first difference image, and receiving a masking input for the first enlarged image according to a subjective good product discrimination criterion of a field worker,
Parts quality management system based on deep learning based defect value distribution. According to claim 2,
The computing device,
Receiving a region of interest from a field worker and generating the first enlarged image around the region of interest,
Parts quality management system based on deep learning based defect value distribution. According to claim 1,
the camera,
Including a camera module for acquiring the first image,
The camera module,
A housing including a through hole in a side wall; and
To include; a lens installed in the through hole,
Parts quality management system based on deep learning based defect value distribution. According to claim 1,
The computing device,
Generating at least one corrected image by applying an image correction filter to at least one of the first image and the second image, and generating the first difference image based on the at least one corrected image. sign,
Parts quality management system based on deep learning based defect value distribution. In the quality control method of parts based on deep learning-based defect value distribution,
Receiving a first image in which good products selected in advance among the parts are photographed and a second image in which defective products selected in advance among the parts are photographed;
generating a first difference image based on the first image and the second image;
receiving a user input for a region of interest in the first difference image;
generating a first enlarged image obtained by enlarging the ROI from the first difference image;
receiving an operator's masking input by selecting an image element determined to be defective from among image elements constituting the first enlarged image according to subjective good product discrimination criteria of the operator;
Generating a masked second difference image in which only elements that a field worker considers important in determining defects are left by masking image elements other than the image elements selected by the masking input in the first enlarged image;
Deep learning a correlation between input and output data by a second deep neural network by using the first image and the second image as inputs and outputting the second difference image;
calculating a size of a remaining image element of the second difference image as a defect value and calculating an inspection reference value through a defect value distribution of the second difference image; and
Classifying parts to be inspected based on the inspection reference value; Including,
A part quality control method based on deep learning-based defect value distribution. A non-transitory computer readable medium storing instructions,
The instructions, when executed by a processor, cause the processor to perform the method of claim 6.

Images