CN113632140A - Automatic learning method and system for product inspection - Google Patents
Automatic learning method and system for product inspection Download PDFInfo
- Publication number
- CN113632140A CN113632140A CN202080024047.5A CN202080024047A CN113632140A CN 113632140 A CN113632140 A CN 113632140A CN 202080024047 A CN202080024047 A CN 202080024047A CN 113632140 A CN113632140 A CN 113632140A
- Authority
- CN
- China
- Prior art keywords
- learning
- product
- neural network
- artificial neural
- learning data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 101
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000013528 artificial neural network Methods 0.000 claims abstract description 95
- 238000012549 training Methods 0.000 claims abstract description 84
- 230000002950 deficient Effects 0.000 claims abstract description 62
- 230000007547 defect Effects 0.000 claims abstract description 49
- 238000012795 verification Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 7
- 238000004519 manufacturing process Methods 0.000 description 27
- 230000015654 memory Effects 0.000 description 26
- 238000004891 communication Methods 0.000 description 22
- 238000003860 storage Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 9
- 230000008859 change Effects 0.000 description 5
- 238000010801 machine learning Methods 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0008—Industrial image inspection checking presence/absence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
An automatic learning method and system for product inspection are provided, the automatic learning system for product inspection may include: a training system to generate learning data used to learn an artificial neural network that examines defects of a product; and a checking system learning the artificial neural network based on the generated learning data and checking whether the product is defective using the learned artificial neural network.
Description
Technical Field
Embodiments disclosed herein relate to an automatic learning method and system for product inspection, and more particularly, to an automatic learning method and system that can quickly and efficiently learn to perform product inspection using artificial intelligence on a production line.
The research is carried out according to the research results of development projects of global software professional enterprises of scientific and technological information communication departments and information communication planning and evaluation institutes (IITP-2019-0-01423-.
Background
As computing technology has evolved, the application of machine learning has increased. In particular, in recent years, in machine learning, deep learning techniques represented by artificial neural networks have been rapidly developed, and application cases to various industrial fields have been increasing. Even in the field of manufacturing, artificial neural networks have innovatively changed the manufacturing site.
Although the existing Rule-based (Rule) test method cannot detect atypical defects occurring on a production line, the artificial neural network can extract atypical defects as well as human beings. In the past, humans directly make a bad judgment on atypical bad on a production line of a product, but now, there are more and more examples of judging whether a product is bad or not and applying it by using machine vision applying an artificial neural network.
In addition, in order to identify whether or not it is bad, a learning process of an artificial neural network is required. That is, generally, an image of a defective product that may occur in a production process is acquired, and an artificial neural network is learned in a supervised learning manner using the acquired image of the defective product.
However, since the defective rate of products in the production process is generally very low, there are problems in that it is difficult to acquire images of defective products for learning the artificial neural network, and it takes a long time to learn the artificial neural network.
In connection with this, korean patent laid-open publication No. 10-2000-0087346, which is a prior art document, relates to an internet artificial intelligence learning and management method. It is described that a teacher and a learner are registered through the internet, the teacher can create and use questions and test questions desired to be used, and the learner learns by extracting questions from a question database and accurately evaluating the learning result thereof, but can not rapidly perform artificial intelligence learning.
Therefore, a technique for solving the above-described problems is required.
On the other hand, the foregoing background art is the technical information which the inventors have possessed for the derivation of the present invention or which they have learned in the course of deriving the present invention, and is not necessarily the publicly known art disclosed before applying the present invention.
Disclosure of Invention
Technical problem to be solved
An object of the embodiments disclosed in the present specification is to provide an automatic learning method and system for product inspection.
An object of the embodiments disclosed in the present specification is to provide an automatic learning method and system for product inspection.
An object of the embodiments disclosed in the present specification is to provide an automatic learning method and system for performing product inspection by combining an unsupervised learning method and a supervised learning method.
An object of an embodiment disclosed in the present specification is to provide an automatic learning method and system for product inspection that accumulates images of unclassified products other than normal products while recognizing the normal products by preferentially performing unsupervised learning based on the images of the normal products.
An object of the embodiments disclosed in the present specification is to provide an automatic learning method and system for product inspection, which accurately inspects whether a product is defective or not by performing supervised learning on a photographed image of an unclassified product.
An object of an embodiment disclosed in the present specification is to provide an automatic learning method and system for product inspection, which generates a learning image for identifying a defective product based on a photographed image of an unclassified product and learns.
Means for solving the problems
As a technical means for solving the above technical problem, according to an embodiment, in an automatic learning system for product inspection, may include: a training system to generate learning data used to learn an artificial neural network that examines defects of a product; and a checking system learning the artificial neural network based on the generated learning data and checking whether the product is defective using the learned artificial neural network.
According to another embodiment, in a method for an automatic learning system to perform automatic learning for product verification, the following steps may be included: generating learning data, the learning data being used to learn an artificial neural network that examines defects of a product; learning the artificial neural network based on the generated learning data; and verifying whether the product is defective using the learned artificial neural network.
According to still another embodiment, in a computer-readable recording medium recorded with a program for executing an automatic learning method, the automatic learning method may include the steps of: generating learning data, the learning data being used to learn an artificial neural network that examines defects of a product; learning the artificial neural network based on the generated learning data; and verifying whether the product is defective using the learned artificial neural network.
According to another embodiment, in a computer program executed by an automatic learning system and stored in a medium to perform an automatic learning method, the automatic learning method may include the steps of: generating learning data, the learning data being used to learn an artificial neural network that examines defects of a product; learning the artificial neural network based on the generated learning data; and verifying whether the product is defective using the learned artificial neural network.
Effects of the invention
According to any one of the above-described problem solving means, an automatic learning method and system for product inspection can be provided.
According to any one of the above-described problem solving means, it is possible to propose an automatic learning method and system that performs product inspection by combining an unsupervised learning method and a supervised learning method.
According to any of the above-described problem solving means, it is possible to propose an automatic learning method and system which can be applied to a production line quickly by shortening the learning time by performing unsupervised learning based on an image of a normal product.
According to any of the above-described problem solving means, it is possible to provide an automatic learning method and system for accumulating captured images of an unclassified product determined as an abnormal product and generating learning data while performing product inspection using machine vision.
According to any of the above-described problem solving means, it is possible to propose an automatic learning method and system for accurately checking whether or not there is a failure by performing supervised learning of an artificial neural network using captured images of unclassified products accumulated by the unsupervised learning.
According to any of the above-described problem solving means, it is possible to provide an automatic learning method and system which can apply product inspection using machine vision to an initial production line in which learning data for defective products is insufficient by unsupervised learning using an artificial neural network using a captured image of a normal product.
According to any of the above-described problem solving means, it is possible to provide an automatic learning method and system for accurately checking various forms of product defects by generating a learning image based on an image of a product and learning by a supervised learning method.
Effects that can be obtained by the embodiments of the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.
Drawings
Fig. 1 is a block diagram showing an automatic learning system of an embodiment.
Fig. 2 is a block diagram illustrating an automatic learning system of an embodiment.
Fig. 3 is a flowchart for explaining an automatic learning method of an embodiment.
Fig. 4 to 5 are exemplary diagrams for explaining an automatic learning method of an embodiment.
Detailed Description
Hereinafter, various embodiments are described in detail with reference to the accompanying drawings. The embodiments described below may also be modified and implemented in various different forms. In order to more clearly describe the features of the embodiments, detailed descriptions of matters known to those of ordinary skill in the art to which the following embodiments belong will be omitted. Moreover, in the drawings, portions irrelevant to the description of the embodiments are omitted, and like reference numerals are given to like portions throughout the specification.
Throughout the specification, when it is described that one component is "connected" to another component, it includes not only the case of "directly connected" but also the case of "connected with another component interposed therebetween". Also, when it is described that one component "includes" another component, it is meant that other components may be included, not excluded, unless otherwise specified.
Embodiments are described in detail with reference to the following drawings.
Before proceeding with the description, however, the meanings of the terms used below are defined.
An "artificial neural network" is an information processing technology for performing complicated control by relating input and output to each other in detail by engineering simulation of an advanced information processing mechanism of a biological nervous system, and is a network in which a plurality of three types of neuron (nerve cell) models are connected, each of the three types of neuron (nerve cell) models being composed of an input layer for transmitting a signal to a switch, a sensor, or the like, a hidden layer for prioritizing input and output based on the information and adjusting the correlation, and an output layer for calculating and outputting a required control amount based on the hidden layer.
The automatic learning system 10 described below may use a photographed image acquired by being incorporated in or connected to an inspection apparatus for inspecting a product using machine vision, for example, to learn an artificial neural network or to inspect or measure whether a product is defective.
Here, machine vision means that industry automation is realized by a camera (visual recognition), a CPU, and SW, and an existing method of judging with the naked eye of a human is replaced to inspect or measure an object.
Further, the Supervised Learning method (Supervised Learning) is a method of Machine Learning (Machine Learning) performed by using Learning Data (Training Data) including a value required to be output as a result.
An Unsupervised Learning method (Unsupervised Learning) is a Learning method for finding out how data is structured, and unlike Supervised Learning (Supervised Learning) or Reinforcement Learning (Reinforcement Learning), a target value to be output as a result is not given to Learning data.
The "unclassified product" refers to a product classified as not being a normal product according to an artificial neural network based on unsupervised learning of a captured image of the normal product, and the "defective product" refers to a product having a defect in an actual product among the unclassified products.
In addition to the above-defined terms, terms to be described will be separately described below.
Fig. 1 is a block diagram of an automatic learning system 10 for illustrating an embodiment.
The automatic learning system 10 may learn the artificial neural network in an unsupervised learning method by using a photographed image obtained by photographing a normal product, and may check whether it is a normal product by photographing a product under production with a camera through machine vision and inputting the acquired photographed image to the artificial neural network. Also, the automatic learning system 10 may analyze a photographed image of a test product, and may generate new learning data to learn the artificial neural network in a supervised learning method. Thereafter, the automatic learning system 10 may accumulate the captured images acquired through the inspection of the product, and may repeatedly learn the artificial neural network using the learning data generated based on the accumulated captured images.
Such an automatic learning system 10 may include: a checking system 11 for checking whether the product is defective or not by performing learning based on learning data of an image obtained by photographing the product; and a training system 12 for providing learning data required for checking whether the product is defective or not.
Further, the checking system 11 and the training system 12 constituting the automatic learning system 10 are connected to a remote server through a network (N), respectively, or are implemented as computers connectable to another terminal and server. Here, the computer may include, for example, a notebook computer equipped with a WEB Browser (WEB Browser), a desktop computer (desktop), a laptop computer (laptop), and the like.
First, the inspection system 11 may inspect whether the product is normal using an artificial neural network based on a photographed image of the product photographed by a machine vision camera. At this time, according to an embodiment, the inspection system 11 may include an artificial neural network or may be connected with an artificial neural network implemented in a physically spaced third server. In the following, the description will be made assuming that an artificial neural network is implemented in the inspection system 11.
Further, the training system 12 may accumulate the photographed images of the product inspected by the inspection system 11, may generate learning data for learning the artificial neural network based on the accumulated photographed images, and periodically supply the generated learning data to the artificial neural network, thereby enabling learning for improving the performance of the artificial neural network to be periodically performed.
Fig. 2 is a block diagram showing the structure of the automatic learning system 10 of an embodiment. Referring to fig. 2, an automatic learning system 10 according to an embodiment may include a verification system 11 and a training system 12.
First, the inspection system 11 may include a product photographing part 111, an inspection control part 112, an inspection communication part 113, and an inspection memory 114.
The product photographing part 111 of the inspection system 11 may include: an input part for photographing a product in production with a camera through machine vision; and an output section for displaying information such as the execution result of the operation or the state of the inspection system 11. For example, the product photographing part 111 may include: a camera which is arranged on a production line of the product and shoots the product; and a display panel (display panel) that displays a screen.
In particular, the input section may include a device capable of receiving various types of user input, such as a keyboard, physical buttons, a touch screen, a camera or microphone, or the like. In particular, the camera for machine vision is provided on a production line in which products are produced, and can photograph the products moving along the production line in real time.
In addition, the output section may include a display panel, a speaker, or the like. However, not limited thereto, the product photographing section 111 may include a structure supporting various inputs and outputs.
Further, the inspection control section 112 of the inspection system 11 controls the overall operation of the inspection system 11, and may include a processor such as a CPU. Further, the inspection control part 112 is implemented by an artificial neural network or uses an artificial neural network implemented in a third server, so that whether a product is normal or defective can be inspected from the photographed image of the product acquired by the product photographing part 111, and for this reason, other configurations included in the inspection system 11 can also be controlled.
Such an inspection control section 112 can learn the artificial neural network by using the captured image acquired by the product capturing section 111 or the captured image in the learning data acquired from the training system 12.
At this time, according to the embodiment, the inspection control part 112 may inspect the product using at least one artificial neural network, and each artificial neural network may be classified according to a learning method of the learning data.
First, the inspection control section 112 can perform learning on an artificial neural network that performs unsupervised learning using a captured image of a normal product as learning data.
For example, the inspection control part 112 may input a photographed image of a product photographed by a machine vision camera to an artificial neural network that performs unsupervised learning, and classify the input photographed images according to the similarity, so that a group of photographed images, of which the number of photographed images is large, among the classified groups may be determined as a photographed image of a normal product. Further, the inspection control section 112 may learn the artificial neural network based on the captured image of the normal product.
Alternatively, for example, the verification control section 112 may acquire captured images stored in advance from the training system 12 as learning data, classify the captured images included in the acquired learning data according to the degree of similarity, and thereby may learn the artificial neural network based on the captured images of the captured images determined to be normal products.
Thereafter, the inspection control part 112 may inspect whether or not the product is normal based on a photographed image obtained by photographing the product produced on the production line by the machine vision camera using the learned artificial neural network.
For example, the inspection control part 112 may inspect whether or not the product is normal by using a photographed image obtained by photographing the product, and may classify the product inspected on the production line into a normal product and an unclassified product as an abnormal product.
The inspection control unit 112 may provide the training system 12 described later with a captured image obtained by capturing an image of the inspected product and store the captured image.
For example, the inspection control section 112 may supply the photographed image determined as a normal product by the inspection and the photographed image determined as an unclassified product to the training system 12, respectively, so that the supplied photographed images are stored and accumulated in the training system 12.
Further, the verification control section 112 may learn an artificial neural network in which supervised learning is performed based on learning data supplied from the training system 12.
For example, the inspection control section 112 may acquire the generated learning data from the training system 12, and may perform supervised learning on the artificial neural network based on the captured image of the normal product and the captured image of the unclassified product in the acquired learning data.
Thereafter, the inspection control part 112 may inspect whether the product is defective by using the supervised learning artificial neural network, and may identify a defective product among the inspected products.
For example, the inspection control section 112 may inspect whether or not the product has a defect based on a captured image obtained by capturing an image of the product under production according to the result of the learning, and when the defect exists and thus is determined to be a defective product, it is possible to identify what type the defect of the defective product belongs to.
In this way, while whether a product is normal or not is preferentially checked using the artificial neural network through unsupervised learning, learning data for supervised learning is accumulated, so that the applicable time of machine vision on a production line can be shortened, and supervised learning is performed on the artificial neural network using shot images accumulated through inspection of the product, so that the defect inspection accuracy of the product can be improved.
Further, the inspection control section 112 may re-supply the photographed image of the inspected product to the training system 12 by using the supervised learning artificial neural network.
Thereafter, the verification control portion 112 may acquire learning data for learning the artificial neural network from the training system 12 at a certain cycle, and learn the artificial neural network each time the learning data is acquired.
For example, the inspection control section 112 may acquire, from the training system 12, a photographed image of a defective product corresponding to a type of defect that may occur in the product as learning data, and additionally learn an artificial neural network by using the acquired learning data.
Further, the inspection control section 112 may recognize whether there is a new defect that may occur in the product or may recognize the type of the defect by using the artificial neural network for additional learning, re-supply the photographed image obtained by photographing the inspected product to the training system 12, and may repeat the process of acquiring the learning data from the training system 12 for learning.
For example, the inspection control section 112 may continuously provide the captured images of the inspected product to the training system 12, and may periodically acquire additional learning data from the training system 12 to be able to inspect a defect that cannot be inspected with the existing learning.
Thus, the inspection control section 112 learns the artificial neural network by using the learning data periodically supplied from the training system 12, so that the inspection accuracy of the product can be improved.
In this manner, the inspection control section 112 performs additional learning by periodically receiving the accumulated learning data from the training system 12, so that it is possible to accurately inspect whether or not a product is defective despite the change in the photographed image due to external factors such as a change in the product to be inspected or the environment in which the product is photographed, a change in the position of the product on the production line, and the like.
The verification communication unit 113 may perform wired or wireless communication with the training system 12 described later or another system via a network. To this end, the inspection communication part 113 may include a communication module supporting at least one of various wired or wireless communication methods. For example, the communication module may be implemented in the form of a chipset (chipset).
The Wireless Communication supported by the verification Communication unit 113 may be, for example, Wireless Fidelity (Wi-Fi), Wi-Fi direct, Bluetooth (Bluetooth), Ultra Wide Band (UWB), Near Field Communication (NFC), or the like. The wired communication supported by the communication unit 130 may be, for example, a USB or a High Definition Multimedia Interface (HDMI).
Various types of data, such as files, applications, programs, and the like, may be provided and stored in the verification memory 114. The verification control unit 112 may access and use the data stored in the verification memory 114, or may store new data in the verification memory 114.
Further, the inspection memory 114 may store learning data acquired through the inspection communication section 113, and may store a photographed image of the product acquired by the inspection control section 112 through the product photographing section 111.
On the other hand, the training system 12 may include a training control section 121, a training communication section 122, and a training memory 123.
First, the training control section 121 of the training system 12 controls the overall operation of the training system 12, and may include a processor such as a CPU. The training control unit 121 of the training system 12 acquires the captured image received by the training communication unit 122, and may control other structures included in the training system 12 to generate the learning data.
For example, the training control unit 121 may execute a program stored in the training memory 123, read a file stored in the training memory 123, or store a new file in the training memory 123.
Such a training control section 121 may acquire a photographed image about the product after the inspection from the inspection system 11, and may analyze whether the product is defective or not based on the acquired photographed image.
For example, the training control part 121 may acquire a photographed image of a normal product and a photographed image of an unclassified product from the inspection system 11 which inspects whether or not it is normal by using an artificial neural network through unsupervised learning, and may analyze a defect pattern of the product by analyzing the acquired photographed images.
Before that, according to the embodiment, the training control part 121 may learn the artificial neural network by providing the inspection system 11 with the photographed image of the normal product stored in advance.
For example, the training control section 121 may provide the inspection system 11 with a photographed image of a normal product and make it learn so that the inspection system 11 can determine whether or not the produced product is defective.
Further, according to the embodiment, the training control part 121 may acquire a photographed image of the product after the inspection from the inspection system 11, and may generate the learning data based on the acquired photographed image.
For this reason, the training control part 121 may recognize the photographed image of the unclassified product determined to be abnormal from among the photographed images acquired from the inspection system 11, and may analyze the defect pattern from the recognized photographed image.
Further, the training control section 121 may generate a defective image having a new defect based on the analyzed defect pattern.
For example, the training control section 121 artificially changes the pixel values of the captured image of a normal product from the inspection system 11 based on the analyzed defect pattern, so that a defective image having a new defect can be generated.
Or, for example, the training control section 121 may generate a new defective image by recognizing a portion where a difference in pixel value occurs between a first captured image obtained by the inspection system 11 capturing a defective product and a second captured image obtained by capturing a normal product, and may randomly apply the difference in pixel value to the second captured image.
For example, the training control section 121 may generate a new defective image by applying a mask that is an image that is made up of binary pixel values stored in advance in a captured image of a normal product in captured images acquired from the inspection system 11 and that represents a defect.
Further, the training control section 121 may supply the learning data including the generated poor image to the inspection system 11, so that the artificial neural network of the inspection system 11 may be learned based on the supplied learning data.
For example, the training control section 121 may generate a new defective image by analyzing the captured image acquired from the inspection system 11, and may provide the learning data including the generated defective image to the inspection system 11 every time a preset condition is satisfied (for example, a certain period of time elapses or the capacity of the stored captured image reaches a preset capacity).
At this time, the training control part 121 may analyze the accuracy of the learning data by the verification system 11, and generate different learning data according to the analysis result.
For example, the training control section 121 may compare and analyze the defective image supplied to the inspection system 11 as the learning data and the photographed image of the defective product acquired from the inspection system 11, and may compare a portion representing a defect on the photographed image of the actual defective product and a defective portion on the defective image and calculate the similarity, and when the similarity is smaller than a preset value, may change the mask image generating the defective image.
Thus, a defective image to which a defect that may occur in an actual defective product is applied can be artificially generated.
Further, the training communication section 122 may perform wired or wireless communication with the above-described inspection system 11 or other systems via a network. To this end, the training communicator 122 may include a communication module supporting at least one of various wired or wireless communication methods. For example, the communication module may be implemented in the form of a chipset (chipset).
Various types of data, such as files, applications, programs, and the like, may be provided and stored in training memory 123. The training control unit 121 may access and use the captured image as data stored in the training memory 123, or may store a new captured image in the training memory 123.
Fig. 3 is a flowchart for explaining an automatic learning method of an embodiment.
The automatic learning method according to the embodiment shown in fig. 3 includes the steps of the chronological processing in the automatic learning system 10 shown in fig. 2. Therefore, even if the following is omitted, the contents described above with respect to the automatic learning apparatus 10 shown in fig. 2 can be applied to the automatic learning method according to the embodiment shown in fig. 3.
Fig. 4 to 5 will be described below with reference to fig. 3. Fig. 4 to 5 are exemplary diagrams for explaining an automatic learning method according to an embodiment, and are exemplary diagrams showing a method of generating learning data in the automatic learning system 10.
First, the automatic learning system 10 may generate learning data of an artificial neural network used to learn defects of an inspection product (S3001).
That is, the automatic learning system 10 can preferentially generate learning data composed of captured images of normal products for unsupervised learning of the artificial neural network.
For example, the automatic learning system 10 may generate learning data based on a shot image of a normal product stored in advance, or may generate learning data based on a shot image of a product shot on a production line.
Further, the automatic learning system 10 may perform unsupervised learning of the artificial neural network based on the learning data generated at step S3001 (S3002).
According to an embodiment, the inspection system 11 may perform unsupervised learning of the artificial neural network based on the photographed image of the product on the production line photographed by the machine vision camera in step S3001.
For example, the inspection system 11 may input a photographed image obtained by photographing a product produced in a production line to the artificial neural network, and may determine a photographed image of a group to which a plurality of photographed images belong among groups formed by classifying the photographed images as the photographed image of a normal product to learn the artificial neural network.
According to another embodiment, the verification system 11 may acquire learning data including images of normal products from the training system 12 and learn.
For example, the inspection system 11 may perform unsupervised learning by inputting captured images of normal products previously stored in the training system 12 into an artificial neural network.
Further, the automatic learning system 10 may check whether the product is normal or not by using an artificial neural network through unsupervised learning (S3003).
That is, the inspection system 11 can classify the products into normal products and unsorted products by inspecting whether the products on the production line are defective.
For example, the inspection system 11 inspects the products on the production line by using the artificial neural network subjected to unsupervised learning in step S3002. That is, the inspection system 11 can inspect whether a product is normal or not by comparing a photographed image obtained by photographing a product on a production line with a photographed image of a learned normal product, and can classify into a normal product and an unclassified product.
Further, the automatic learning system 10 may generate learning data based on the photographed image of the product verified in step S3003 (S3004).
That is, the inspection system 11 may provide the photographed image of the product inspected in step S3003 to the training system 12.
For example, the inspection system 11 may provide the photographed images of the respective products determined to be the normal product and the unclassified product by step S3003 to the training system 12.
Further, the training system 12 may analyze the defect pattern of the product based on the captured image acquired from the inspection system 11.
For example, the training system 12 may store the photographed image acquired from the inspection system 11, may recognize a difference between the photographed image of an unclassified product and the photographed image of a normal product by using the stored photographed images, and may analyze a defect pattern that generates the difference recognized on the photographed images.
Further, training system 12 may generate new learning data based on the analyzed defect patterns.
According to one embodiment, training system 12 may generate a bad image as an image including new defects by randomly incorporating the analyzed defect pattern into a captured image of a normal product.
For example, the training system 12 may randomly change the pixel area of the captured image of a normal product according to a defect pattern that analyzes the captured image of an unclassified product, so that a bad image including a new defect may be generated.
Fig. 4 is an exemplary diagram of generating a new defective image by using a defective region pattern. Referring to fig. 4, the training system 12 may identify a defect pattern 402 by analyzing a first photographed image 400 of a defective product acquired from the inspection system 11 and a second photographed image 401 of a normal product, and may generate a defective image 404 including a new defect 403 by randomly combining the identified defect pattern 402 with the second photographed image 401.
According to another embodiment, training system 12 may generate a new bad image by randomly changing the pixel values of the captured image.
For example, the training system 12 may generate a bad image including a defect by changing a part of pixel values of the captured image to a preset value.
Fig. 5 is an exemplary diagram of generating a defective image by changing the value of an arbitrary pixel of a captured image. Referring to fig. 5, training system 12 may generate a bad image 502 that includes a defect 501 by randomly changing pixels of a captured image 500 of a normal product.
Thereafter, the automatic learning system 10 may perform supervised learning of the artificial neural network by using the learning data generated in step S3004 (S3005).
For example, the training system 12 may supply the generated learning data to the inspection system 11 every time the capacity of the captured image acquired from the inspection system 11 exceeds a preset capacity or supply the generated learning data to the inspection system 11 every preset period.
Further, the verification system 11 may perform supervised learning of an artificial neural network based on the learning data acquired from the training system 12.
For example, the inspection system 11 may classify normal images and defective images of normal products based on the learning data acquired from the training system 12, and may input the normal images and the defective images to an artificial neural network, respectively, to perform supervised learning. At this time, the inspection system 11 may classify the defective image according to the type of the defect of the product to perform supervised learning.
Further, the automatic learning system 10 may check whether the product is defective using the artificial neural network supervised learning by step S3005 (S3006).
For example, while the automatic learning system 10 simultaneously uses the unsupervised and supervised artificial neural networks to check whether a product is defective, the type of defect can be specifically checked for a defective product in which a defect exists.
Thereafter, the automatic learning system 10 may automatically repeat steps S3004 to S3006 periodically, whereby it is possible to automatically generate learning data when checking the products on the production line without requiring a manager to input the learning data for checking the products, and to improve the accuracy of detecting the products having defects generated on the production line by periodically learning the generated learning data.
The term "section" used in the above embodiments refers to a hardware component such as software or a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and the "section" may perform a certain role. However, the section is not limited to software or hardware. The "parts" may be configured to reside on the addressable storage medium and may be configured to run one or more processors. Therefore, the "section" includes, as an example: components such as software components, object-oriented software components, class components, and task components; proceeding; a function; an attribute; carrying out a procedure; a sub-routine; a program specific code segment; a driver; firmware; a microcode; a circuit; data; a database; a data structure; table; an array; and a variable.
The functions provided in the components and the "parts" may be combined into a smaller number of constituent elements and "parts" or separated from additional components and "parts".
Furthermore, the components and "—" may also be implemented as one or more central processors in a running device or secure multimedia card.
The automatic learning method according to the embodiment illustrated by fig. 3 may also be implemented in the form of a computer-readable medium storing instructions and data executable by a computer. At this time, the instructions and data may be stored in the form of program code, which, when executed by a processor, may generate predetermined program modules to perform predetermined operations. Also, computer readable media can be any available media that can be accessed by the computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may be computer recording media including volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. For example, the computer recording medium may be a magnetic storage medium such as a Hard Disk Drive (HDD) and a Solid State Disk (SSD); optical recording media such as Compact Discs (CDs), Digital Video Discs (DVDs), and blu-ray discs; or a memory included in a server accessible through a network.
In addition, the automatic learning method according to the embodiment illustrated by fig. 3 may also be implemented as a computer program (or computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions that are processed by the processor and may be implemented in a High-level Programming Language (High-level Programming Language), an Object-oriented Programming Language (Object-oriented Programming Language), an assembly Language, or a machine Language. Further, the computer program may be recorded on a tangible computer-readable recording medium (e.g., a memory, a hard disk, a magnetic/optical medium, or a Solid-State Drive (SSD), etc.).
Therefore, the automatic learning method according to the embodiment explained by fig. 3 may be realized by executing a computer program as described above by a computing device. The computing device may also include at least a portion of a processor, memory, a storage device, a high-speed interface connected to the memory and the high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. Each of these components is interconnected using various buses, and may be mounted on a common motherboard or in any other suitable manner.
Among other things, the processor may process instructions in the computing device, which may include instructions stored in the memory or storage device to display graphical information for providing a Graphical User Interface (GUI) on an external input or output device, such as a display connected to a high speed Interface. For example, as another embodiment, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and memory modalities. Also, the processor may be implemented as a chipset of chips that include multiple separate analog and/or digital processors.
Also, the memory stores information in the computing device. As an example, the memory may be configured as a volatile memory unit or a collection thereof. As another example, the memory may be configured as a non-volatile memory unit or collection thereof. Also, the memory may be another form of computer-readable medium, such as a magnetic or optical disk.
Furthermore, the storage device may provide a large amount of storage space for the computing device. The Storage device may be a computer-readable medium or a component containing such a medium, and may also include, for example, a device in a Storage Area Network (SAN) or other component, and may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory, or other similar semiconductor Storage device or device array.
The above embodiments are for illustration purposes, and those skilled in the art to which the embodiments belong will appreciate that the embodiments can be easily modified into other specific forms without changing the technical idea or essential features of the embodiments. It is therefore to be understood that the above embodiments are illustrative and not restrictive in all respects. For example, each component described as singular can be implemented in distributed form, and components described as distributed can also be implemented in combined form.
The scope protected by the present specification is indicated by the appended claims rather than the detailed description, and should be understood to include the meaning and scope of the claims and all changes or modifications derived from the equivalent concepts thereof.
Claims (12)
1. An automatic learning system for product inspection, wherein,
the automatic learning system includes:
a training system for generating learning data, said learning data being used for learning an artificial neural network for inspecting defects of a product, an
A checking system for learning the artificial neural network based on the generated learning data and checking whether the product is defective using the learned artificial neural network.
2. The automatic learning system of claim 1,
the checking system is used for checking the state of the system,
performing unsupervised learning on the artificial neural network based on a photographed image of a normal product used as the learning data, and verifying whether the product is defective using the unsupervised learned artificial neural network.
3. The automatic learning system of claim 2,
the training system is used for training the training system,
acquiring a photographed image of the inspected product from the inspection system, generating the learning data based on the acquired photographed image and periodically providing the generated learning data to the artificial neural network.
4. The automatic learning system of claim 3,
the checking system is used for checking the state of the system,
performing supervised learning on the artificial neural network based on the photographed image of the normal product and the photographed image of the unclassified product in the provided learning data, and verifying whether the product is defective using the supervised-learned artificial neural network.
5. The automatic learning system of claim 4,
the training system is used for training the training system,
patterns of defects of defective products in the inspected products are analyzed, and learning data is generated based on the analyzed patterns of defects.
6. An automatic learning method for product inspection using an automatic learning system including an inspection system and a training system, wherein,
the automatic learning method comprises the following steps:
generating learning data with a training system, the learning data being used to learn an artificial neural network that examines defects of a product,
learning the artificial neural network based on the generated learning data with a checking system, and
using the learned artificial neural network to verify whether the product is defective with a verification system.
7. The automatic learning method of claim 6,
the step of learning the artificial neural network includes the steps of:
performing unsupervised learning on the artificial neural network based on a photographed image of a normal product used as the learning data;
a step of checking whether said product is defective, comprising the steps of:
verifying whether the product is defective by using an artificial neural network through unsupervised learning.
8. The automatic learning method of claim 7,
the step of generating the learning data includes the steps of:
acquiring a photographic image of the inspected product from the inspection system, an
Generating the learning data based on the acquired photographed image and periodically providing the generated learning data to the artificial neural network.
9. The automatic learning method of claim 8,
the step of learning the artificial neural network includes the steps of:
performing supervised learning on the artificial neural network based on the photographed image of the normal product and the photographed image of the unclassified product in the provided learning data,
a step of checking whether said product is defective, comprising the steps of:
verifying whether the product is defective by using an artificial neural network through supervised learning.
10. The automatic learning method of claim 9,
the step of generating the learning data includes the steps of:
analyzing the inspected product for patterns of defects of defective products, an
Learning data is generated based on the analyzed pattern of defects.
11. A computer-readable recording medium in which a program for executing the automatic learning method according to claim 6 is recorded.
12. A computer program, wherein the execution is performed by an automatic learning system, and stored in a medium to execute the automatic learning method according to claim 6.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR10-2019-0071680 | 2019-06-17 | ||
| KR1020190071680A KR102271418B1 (en) | 2019-06-17 | 2019-06-17 | Method and system for auto training for product inspection |
| PCT/KR2020/007844 WO2020256407A1 (en) | 2019-06-17 | 2020-06-17 | Automatic learning method and system for product inspection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113632140A true CN113632140A (en) | 2021-11-09 |
Family
ID=74040232
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202080024047.5A Pending CN113632140A (en) | 2019-06-17 | 2020-06-17 | Automatic learning method and system for product inspection |
Country Status (3)
| Country | Link |
|---|---|
| KR (1) | KR102271418B1 (en) |
| CN (1) | CN113632140A (en) |
| WO (1) | WO2020256407A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102516482B1 (en) * | 2021-05-26 | 2023-04-04 | (주)프로젝트노아 | Defect detection device |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020147536A1 (en) * | 2001-04-10 | 2002-10-10 | Mitsubishi Denki Kabushiki Kaisha | Abnormality detecting apparatus for automatic speed changer |
| CN103400040A (en) * | 2013-07-31 | 2013-11-20 | 中国人民解放军国防科学技术大学 | Fault diagnosis and prediction method utilizing multistep time domain difference value learning |
| CN109492767A (en) * | 2018-11-09 | 2019-03-19 | 济南浪潮高新科技投资发展有限公司 | A kind of method for detecting abnormality applied to unsupervised field based on self-encoding encoder |
| CN109584221A (en) * | 2018-11-16 | 2019-04-05 | 聚时科技(上海)有限公司 | A kind of abnormal image detection method generating confrontation network based on supervised |
| CN109635921A (en) * | 2018-11-19 | 2019-04-16 | 扬州梯子教育科技发展有限公司 | A kind of unsupervised abnormality recognition method based on deep learning |
| CN109741292A (en) * | 2017-10-27 | 2019-05-10 | 罗伯特·博世有限公司 | The method for detecting abnormal image in the first image data set with confrontation self-encoding encoder |
| WO2019107614A1 (en) * | 2017-11-30 | 2019-06-06 | 전자부품연구원 | Machine vision-based quality inspection method and system utilizing deep learning in manufacturing process |
| US20190180290A1 (en) * | 2017-12-08 | 2019-06-13 | Accenture Global Solutions Limited | Procurement fraud detection system |
| JP2019175462A (en) * | 2018-03-29 | 2019-10-10 | コリア インスティテュート オブ オーシャン サイエンス テクノロジー | System for automatically identifying abnormal navigation state of ship with unsupervised learning, and method for the same |
| US20200012896A1 (en) * | 2018-07-04 | 2020-01-09 | Kwangwoon University Industry-Academic Collaboration Foundation | Apparatus and method of data generation for object detection based on generative adversarial networks |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9542626B2 (en) * | 2013-09-06 | 2017-01-10 | Toyota Jidosha Kabushiki Kaisha | Augmenting layer-based object detection with deep convolutional neural networks |
| KR20160103398A (en) * | 2015-02-24 | 2016-09-01 | 삼성전자주식회사 | Method and apparatus for measuring the quality of the image |
-
2019
- 2019-06-17 KR KR1020190071680A patent/KR102271418B1/en active IP Right Grant
-
2020
- 2020-06-17 CN CN202080024047.5A patent/CN113632140A/en active Pending
- 2020-06-17 WO PCT/KR2020/007844 patent/WO2020256407A1/en active Application Filing
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020147536A1 (en) * | 2001-04-10 | 2002-10-10 | Mitsubishi Denki Kabushiki Kaisha | Abnormality detecting apparatus for automatic speed changer |
| CN103400040A (en) * | 2013-07-31 | 2013-11-20 | 中国人民解放军国防科学技术大学 | Fault diagnosis and prediction method utilizing multistep time domain difference value learning |
| CN109741292A (en) * | 2017-10-27 | 2019-05-10 | 罗伯特·博世有限公司 | The method for detecting abnormal image in the first image data set with confrontation self-encoding encoder |
| WO2019107614A1 (en) * | 2017-11-30 | 2019-06-06 | 전자부품연구원 | Machine vision-based quality inspection method and system utilizing deep learning in manufacturing process |
| US20190180290A1 (en) * | 2017-12-08 | 2019-06-13 | Accenture Global Solutions Limited | Procurement fraud detection system |
| JP2019175462A (en) * | 2018-03-29 | 2019-10-10 | コリア インスティテュート オブ オーシャン サイエンス テクノロジー | System for automatically identifying abnormal navigation state of ship with unsupervised learning, and method for the same |
| US20200012896A1 (en) * | 2018-07-04 | 2020-01-09 | Kwangwoon University Industry-Academic Collaboration Foundation | Apparatus and method of data generation for object detection based on generative adversarial networks |
| CN109492767A (en) * | 2018-11-09 | 2019-03-19 | 济南浪潮高新科技投资发展有限公司 | A kind of method for detecting abnormality applied to unsupervised field based on self-encoding encoder |
| CN109584221A (en) * | 2018-11-16 | 2019-04-05 | 聚时科技(上海)有限公司 | A kind of abnormal image detection method generating confrontation network based on supervised |
| CN109635921A (en) * | 2018-11-19 | 2019-04-16 | 扬州梯子教育科技发展有限公司 | A kind of unsupervised abnormality recognition method based on deep learning |
Non-Patent Citations (3)
| Title |
|---|
| GAO HUANG ET AL.: "Semi-Supervised and Unsupervised extreme learning machines", 《IEEE TRANSACTIONS ON CYBERNETICS》, vol. 44, no. 12, XP011564449, DOI: 10.1109/TCYB.2014.2307349 * |
| HAHN-MING LEE ET.AL: "Supervised Extended ART： A Fast Neural Network Classifier Trained by Combining Supervised and Unsupervised Learning", 《APPLIED INTELLIGENCE》 * |
| JING TIAN ET AL.: "Anomaly Detection Using Self-Organizing Maps-Based K-Nearest Neighbor Algorithm", 《EUROPEAN CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2014》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| KR102271418B1 (en) | 2021-07-01 |
| KR20200143957A (en) | 2020-12-28 |
| WO2020256407A1 (en) | 2020-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Clapham et al. | Automated facial recognition for wildlife that lack unique markings: A deep learning approach for brown bears | |
| CN114631115A (en) | Method and system for quality inspection of new products using deep learning | |
| CN107832774A (en) | A kind of page exception detection method and device | |
| JP6216024B1 (en) | Trained model generation method and signal data discrimination device | |
| KR20200087297A (en) | Defect inspection method and apparatus using image segmentation based on artificial neural network | |
| KR102141302B1 (en) | Object detection method based 0n deep learning regression model and image processing apparatus | |
| KR102359090B1 (en) | Method and System for Real-time Abnormal Insider Event Detection on Enterprise Resource Planning System | |
| US20210365714A1 (en) | Detecting Artificial Multimedia Content Using Deep Network Response Analysis | |
| JP2021143884A (en) | Inspection device, inspection method, program, learning device, learning method, and trained dataset | |
| CN110275820B (en) | Page compatibility testing method, system and equipment | |
| CN113632140A (en) | Automatic learning method and system for product inspection | |
| KR102427690B1 (en) | Apparatus and method for classification based on deep learning | |
| KR20210056974A (en) | Method and apparatus for identifying object using multi- cameras | |
| Heizmann et al. | Artificial intelligence with neural networks in optical measurement and inspection systems | |
| KR102273334B1 (en) | Method and apparatus for learning neural network using multi channel image | |
| Watanabe et al. | Layout cross-platform and cross-browser incompatibilities detection using classification of DOM elements | |
| KR102252590B1 (en) | Method and apparatus for setting test environment based on machine learning environment | |
| RU2774624C1 (en) | Method and system for determining synthetic changes of faces in video | |
| Cheah | Mobile application for fruits freshness/ripeness detection | |
| US11068752B2 (en) | Inspection systems and methods including image classification module | |
| CN112347893B (en) | Model training method and device for video behavior recognition and computer equipment | |
| Hagerup | Automatic verification of UI tests | |
| Raut et al. | Automated Suspicious Activity Detection from Surveillance Videos | |
| RACHANA et al. | Change Detection in Remote Sensing Images Using Ssim | |
| Saranya et al. | Fire Detection Using CNN & SVM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |