WO2018105028A1 - 検査装置及び検査方法 - Google Patents
検査装置及び検査方法 Download PDFInfo
- Publication number
- WO2018105028A1 WO2018105028A1 PCT/JP2016/086138 JP2016086138W WO2018105028A1 WO 2018105028 A1 WO2018105028 A1 WO 2018105028A1 JP 2016086138 W JP2016086138 W JP 2016086138W WO 2018105028 A1 WO2018105028 A1 WO 2018105028A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- unit
- inspected
- image data
- inspection apparatus
- Prior art date
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 197
- 238000000034 method Methods 0.000 title description 97
- 238000004458 analytical method Methods 0.000 claims abstract description 113
- 230000006835 compression Effects 0.000 claims abstract description 103
- 238000007906 compression Methods 0.000 claims abstract description 103
- 230000005856 abnormality Effects 0.000 claims abstract description 68
- 238000003860 storage Methods 0.000 claims description 76
- 238000000513 principal component analysis Methods 0.000 claims description 38
- 230000002159 abnormal effect Effects 0.000 claims description 33
- 238000013528 artificial neural network Methods 0.000 claims description 32
- 238000012937 correction Methods 0.000 claims description 31
- 238000001914 filtration Methods 0.000 claims description 4
- 239000002131 composite material Substances 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 abstract description 7
- 238000011084 recovery Methods 0.000 abstract 1
- 239000013598 vector Substances 0.000 description 40
- 230000008569 process Effects 0.000 description 35
- 238000012545 processing Methods 0.000 description 27
- 230000007547 defect Effects 0.000 description 26
- 238000010586 diagram Methods 0.000 description 18
- 238000012360 testing method Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 15
- 230000000694 effects Effects 0.000 description 13
- 230000002040 relaxant effect Effects 0.000 description 8
- 230000004913 activation Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 4
- 239000000758 substrate Substances 0.000 description 4
- 241000282326 Felis catus Species 0.000 description 3
- 230000001186 cumulative effect Effects 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 210000000225 synapse Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/956—Inspecting patterns on the surface of objects
- G01N21/95684—Patterns showing highly reflecting parts, e.g. metallic elements
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/956—Inspecting patterns on the surface of objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8896—Circuits specially adapted for system specific signal conditioning
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/956—Inspecting patterns on the surface of objects
- G01N2021/95638—Inspecting patterns on the surface of objects for PCB's
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/12—Bounding box
Definitions
- the present invention relates to an inspection apparatus and an inspection method for inspecting the presence or absence of an abnormality such as a defect, misplacement, or defect of an object from acquired object data.
- Taking a picture of an object with a camera and automatically inspecting the machine for the presence or absence of an abnormality from the acquired image data means that, for example, visual inspection or visual inspection performed in the manufacturing process of industrial products is automated or labor-saving. This is an important technology.
- the inspection apparatus of Patent Document 1 inspecting whether there is an abnormality such as a defect, misplacement, or defect of an object, for example, in the inspection apparatus of Patent Document 1, a plurality of image data obtained by photographing an object is stored, and the plurality of image data The range of the luminance value for determining that there is no abnormality in the object for each pixel having the same coordinates is calculated and set as a reference for the inspection for the presence or absence of abnormality of the object.
- the inspection apparatus determines whether or not the luminance value of the image data obtained by photographing the object to be inspected for each pixel having the same coordinates is within the range of luminance values determined to be normal for the set object. By determining, the presence or absence of an abnormality such as a defect, misplacement, or defect of the object was inspected.
- the present invention has been made in order to solve the above-described problems, and more reliably fixes the object and the camera and each pixel of the image data acquired by photographing the object than the conventional inspection apparatus. It is an object of the present invention to provide an inspection apparatus and an inspection method for inspecting whether there is an abnormality such as a defect, misplacement, or defect of an object while relaxing restrictions such as highly accurate alignment.
- the inspection apparatus calculates a parameter representing the property of data of an object that does not include an abnormality by performing dimension compression that reduces the data dimension of the data of the object that does not include an abnormality, and
- An analysis unit that performs dimensional compression of data using parameters, a restoration unit that generates restoration data obtained by restoring the data of an object to be inspected that has been dimensionally compressed by the analysis unit, and a difference between the data of the object to be inspected and the restored data
- a determination unit that outputs a determination result indicating whether or not an object to be inspected based on the size is abnormal, and an output unit that outputs the determination result output by the determination unit.
- the data representing the property of the object data not including the abnormality is calculated by performing dimension compression to reduce the data dimension of the object data not including the abnormality, and the object data to be inspected is calculated.
- Dimensionally compressed using parameters dimensionally compressed restoration object data is generated, and the object to be inspected is abnormal based on the difference between the data of the object to be inspected and the restored data
- the object and the camera are more securely fixed and the pixel data of the image data acquired by photographing the object is more accurately aligned than the conventional inspection apparatus. It is possible to inspect whether there is an abnormality such as a defect, misplacement, or defect of an object while relaxing the restrictions.
- FIG. 1 is a functional block diagram of an inspection system including an inspection device according to Embodiment 1 of the present invention. It is a hardware block diagram of the inspection apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the operation
- a printed circuit board is used as an object to inspect whether there is any abnormality on the board.
- This is an example of a case where a part of a substrate that is an object to be inspected has a defect. It is the result of threshold processing. It is an example of a two-dimensional mask for limiting an inspection object area.
- It is an example of the display content which an input / output part instruct
- FIG. 1 is a functional block diagram of an inspection system including an inspection apparatus 1 according to Embodiment 1 of the present invention.
- the inspection system includes an inspection device 1 for inspecting the object 3, a camera 2 for photographing the object 3, and an input / output device 4 for inputting inspection contents and outputting inspection results.
- the inspection apparatus 1 receives image data of the object 3 photographed by the camera 2 as input data, performs analysis, and transmits the result to the input / output apparatus 4.
- the inspection apparatus 1 includes a control unit 10 that controls each unit, an input unit 11 that receives image data, an analysis unit 12a that analyzes image data input from the input unit 11, and a storage unit that records the analysis results. 13a, a determination unit 14a that outputs a determination result indicating whether or not the object 3 is abnormal from the analyzed result and the acquired image data, and an input / output unit 15 that outputs the determination result output by the determination unit 14a. ing.
- the control unit 10 controls each unit by transmitting and receiving instructions to and from the input unit 11, the analysis unit 12a, the storage unit 13a, the determination unit 14a, and the input / output unit 15.
- the input unit 11 receives the image data of the object 3 from the camera 2.
- the image data is an example of data of the object 3 and may be data indicating a waveform, a solid, or the like without being limited to an image.
- the input image data is assumed to be digital data, but may be analog data.
- the analysis unit 12a switches and executes two different operation modes based on the instruction transmitted from the control unit 10.
- the two operation modes are a learning mode and an inspection mode.
- the learning mode one or more pieces of image data of a normal object 3 that does not include an abnormality are used, and the image data of the normal object 3 that does not include an abnormality is subjected to dimensional compression that reduces the dimension of the image data of the object 3.
- the inspection apparatus 1 does not inspect whether the object 3 is abnormal.
- the inspection for the presence or absence of abnormality is performed in an inspection mode that is performed after the learning mode is completed.
- dimensional compression similar to the dimensional compression performed when the image data of the object 3 to be inspected is calculated as a parameter representing the data characteristics of the normal object 3 learned in the learning mode is performed.
- the storage unit 13a stores the learning result based on an instruction from the control unit 10, reads the learning result, and transmits the learning result to the analysis unit 12a.
- the learning result read out here is a learning result corresponding to the dimension compression method used in the learning mode.
- the determination unit 14a restores the dimensionally compressed image data of the object 3 to be inspected by a method similar to the method used in dimension compression, and restores the restored data that is the restored image data and the image data of the object 3 to be inspected. Based on the magnitude of the difference, a determination result indicating whether or not the object 3 to be inspected is abnormal is output to the input / output unit 15.
- the determination unit 14a is an example of a unit that combines the restoration unit and the determination unit.
- the input / output unit 15 outputs information representing the progress of learning and the like from the input / output device 4 to the outside based on an instruction from the control unit 10.
- the input / output unit 15 outputs the determination result received from the determination unit 14 a to the outside from the input / output device 4 based on an instruction from the control unit 10.
- the operator confirms the input / output device 4 outside, but it is not always necessary to do so, and a signal is output to an external control device or the like without intervening the worker. You may make it do.
- the input / output unit 15 is an example of an output unit. In the first embodiment, the output unit is further provided with an input unit.
- the camera 2 acquires the image data of the object 3 by photographing the object 3 and storing it in the image data.
- the camera 2 transmits image data of the object 3 to the inspection apparatus 1.
- the camera 2 is an example, and the present invention is not limited to this as long as the data of the object 3 can be acquired.
- the input / output device 4 inputs the inspection content of the inspection device 1 and outputs the inspection result output by the inspection device 1.
- the input / output device 4 may be configured with, for example, a display, a speaker, a keyboard, a mouse, and the like.
- the display is an example of a display unit.
- FIG. 2 is a hardware configuration diagram of the inspection apparatus 1 according to Embodiment 1 of the present invention. The configuration of the inspection apparatus 1 according to the first embodiment of the present invention will be described with reference to FIG.
- the inspection apparatus 1 is configured by a computer.
- a computer constituting the inspection apparatus 1 reads and executes a program of the bus 104, an input / output interface 100 that transmits and receives data, a memory 102, a storage medium 103 that stores programs, learning data, and the like, and a storage medium 103 loaded in the memory 102.
- Hardware such as the processor 101 is provided.
- the input / output interface 100 is referred to as an input / output IF 100.
- the bus 104 is a signal path for electrically connecting the devices and exchanging data.
- the input / output IF 100 transmits and receives data.
- the input / output IF 100 receives the activation signal and the setting signal of the inspection device 1 from the input / output device 4 and transmits them to the control unit 10.
- the input / output IF 100 receives an instruction signal from the control unit 10 to the analysis unit 12a
- the input / output IF 100 transmits the instruction signal to the analysis unit 12a.
- the input unit 11 and the input / output unit 15 are realized by the input / output IF 100.
- the memory 102 functions as a work area for loading the program stored in the storage medium 103.
- the memory 102 is, for example, a RAM (Random Access Memory).
- the storage medium 103 stores a program for realizing the functions of a learning mode program and an inspection mode program.
- the storage medium 103 stores learning data and the like.
- the storage medium 103 is, for example, a ROM (Read Only Memory), a flash memory, or an HDD (Hard Disk Drive).
- the storage medium 103 also stores an OS (Operating System).
- the storage unit 13a is realized by the storage medium 103.
- the processor 101 is connected to other devices via the bus 104 and controls these other devices and each unit.
- the processor 101 reads and executes the program in the storage medium 103 loaded in the memory 102.
- the processor 101 loads at least a part of the OS stored in the storage medium 103 into the memory 102, and executes the program while executing the OS.
- the processor 101 is an integrated circuit (IC) that performs processing.
- the processor 101 is, for example, a CPU (Central Processing Unit).
- the control unit 10, the analysis unit 12a, and the determination unit 14a are realized by reading and executing the program of the storage medium 103 loaded into the memory 102 by the processor 101.
- information, data, signal values, variable values, and the like indicating the results of each device are stored in the memory 102, the storage medium 103, or a register or cache memory in the processor 101.
- the memory 102 and the storage medium 103 may be the same device without dividing the device.
- the program may be stored in a portable recording medium such as a magnetic disk, a flexible disk, an optical disk, a compact disk, or a DVD (Digital Versatile Disc).
- a portable recording medium such as a magnetic disk, a flexible disk, an optical disk, a compact disk, or a DVD (Digital Versatile Disc).
- FIG. 3 is a flowchart showing the operation in the learning mode of the inspection apparatus 1 according to Embodiment 1 of the present invention. The operation in the learning mode of the inspection apparatus 1 will be described below using FIG.
- step S10 the control unit 10 receives the activation signal and the setting signal from the input / output device 4 via the input / output unit 15. Then, an instruction is given to the input unit 11 based on the setting signal.
- the input unit 11 receives normal image data of the object 3 from the camera 2. At this time, the timing of receiving normal image data may be determined in advance, for example, 30 times per second, or may be determined based on an instruction from the control unit 10.
- the control unit 10 gives an instruction to start processing in the learning mode to the analysis unit 12a.
- the analysis unit 12 a switches to the learning mode by reading from the memory 102 a program corresponding to the learning mode of the storage medium 103 loaded in the memory 102 and executing it by the processor 101.
- the analysis unit 12a receives normal image data of the object 3 captured by the camera 2 from the input unit 11.
- the analysis unit 12a further determines whether to receive normal image data or to finish receiving normal image data.
- the end determination of normal image data reception may be determined by the analysis unit 12 a or may be determined based on an instruction from the control unit 10.
- the analysis unit 12a determines, for example, it may be considered that the reception of normal image data is terminated when the number of received normal image data reaches a predetermined number.
- the number of sheets designated in advance is, for example, 100 sheets, 1000 sheets, or the like.
- the control unit 10 receives a normal image data reception end instruction from the input / output device 4 via the input / output unit 15, and transmits it to the analysis unit 12a. It is possible to do.
- step S12 the analysis unit 12a returns to reception of normal image data based on the determination result in step S11, or proceeds to the next step.
- step S12 is No, and the process returns to step S10 again. If it is determined that the normal image data reception has been completed, step S12: Yes, and the process proceeds to the next step.
- step S13 the analysis unit 12a performs dimension compression using the received normal image data.
- dimensional compression refers to converting high-dimensional data such as image data and three-dimensional solid data into low-dimensional data.
- the analysis unit 12a performs learning using normal image data in the learning mode, and obtains a data conversion method that is optimal for normal image data.
- Dimensional compression techniques include principal component analysis, linear discriminant analysis, canonical correlation analysis, discrete cosine transform, random projection, neural network self-encoder, and the like. Of these, principal component analysis is one of the most commonly used linear dimension compression techniques. Below, the case where a principal component analysis is used is demonstrated.
- Principal component analysis is a technique for obtaining a low-dimensional space indicating the characteristics of distribution from a large number of normal image data for learning distributed in a multi-dimensional space. This low-dimensional space is called a subspace. When the pixel values of a large number of normal image data received in step S10 are plotted in the space as they are, they are often distributed in a very low-dimensional subspace.
- FIG. 4 is a conceptual diagram of principal component analysis.
- normal image data distributed in a three-dimensional space is shown as a collection confined in a plane.
- a two-dimensional plane as shown in FIG.
- a dimension does not have a physical meaning but is an element included in data. That is, the number of dimensions is the number of elements included in the data.
- 1 pixel 1 dimension.
- image data of 10 pixels vertically and 10 pixels horizontally this image data is 10 ⁇ 10 and has 100 pixels, so “100-dimensional data” It turns out that. That is, it is expressed as one point in the 100-dimensional space.
- FIG. 4 is a schematic diagram illustrated in three dimensions so that the principal component analysis can be visualized. Each ellipse in FIG. 4 corresponds to one piece of image data.
- the image data is image data of three pixels.
- image data actually distributed in a three-dimensional space is shown in a two-dimensional subspace, that is, a state in which dimension compression is performed is shown.
- N be the total number of normal image data received in step S10
- K be the total number of pixels per normal image data.
- the value of N is 100, 1000
- the value of K is 1024 if the size of normal image data is 32 pixels ⁇ 32 pixels
- 409600 if the size of normal image data is 640 pixels ⁇ 640 pixels.
- T is a transposed vector.
- Equation (2) the average vector M and the variance-covariance matrix S are obtained by Equation (2) and Equation (3).
- the first principal component which is a straight line that passes through the average point and has the largest spread
- a straight line of the second principal component in the direction of the second largest spread orthogonal to the first principal component and passing through the average is obtained.
- the direction of large spread through the average is the same as the problem of obtaining the eigenvector of the variance-covariance matrix S.
- the eigenvalue ⁇ j and the eigenvector u j satisfying the equation (4) are obtained using the calculated variance-covariance matrix S.
- j is the number of dimensions.
- d-dimensional principal components (u 1 , u 2 ,..., U d ) are obtained.
- the larger eigenvalue ⁇ j indicates the main component in the principal component analysis. Sort out important parameters of principal component analysis by extracting main components. However, d ⁇ K, and d is generally much smaller than K.
- u j is called the j-th principal component.
- the principal components are orthogonal to each other and are sometimes called bases.
- the values of the d-dimensional principal components (u 1 , u 2 ,..., u d ) are an example of parameters representing the data properties of the object 3.
- the original normal image data before dimension compression can be expressed by a linear combination of the principal components.
- normal image data which was originally K dimension can be compressed to d dimension.
- the value of d is an important parameter that affects the performance of the inspection apparatus 1 according to the first embodiment.
- the value of d By appropriately setting the value of d, it is possible to extract only important components that appear in common in the normal image data received in step S10.
- unnecessary components such as variations among objects 3 of the same type, variations in image data due to different shooting timings, camera noise, and the like can be excluded.
- the value of d is determined based on an instruction from the control unit 10, for example.
- the control unit 10 receives the value d from the input / output device 4 via the input / output unit 15 and transmits it to the analysis unit 12a.
- the analysis unit 12a may read and use the value of d stored in the memory 102 or the storage medium 103.
- the value of d stored in advance may be, for example, about 1/10 of the total number K of pixels of the image data, or about 1/5.
- the analysis unit 12a adaptively determines the value of d based on the nature of normal image data for learning. In this case, it is effective to use the cumulative contribution rate P calculated by Expression (5).
- the cumulative contribution rate P is an index representing how much the information of the original normal image data before dimensional compression can be expressed by using components up to d dimension, and is an analysis unit 12a.
- the threshold for the cumulative contribution rate P is determined based on an instruction from the control unit 10, for example.
- the control unit 10 receives the threshold value from the input / output device 4 via the input / output unit 15 and transmits it to the analysis unit 12a.
- the analysis unit 12a may read and use the threshold value stored in the memory 102 or the storage medium 103.
- the threshold value stored in advance may be 80, 100, or the like.
- the analysis unit 12 a uses the d-dimensional principal component (u 1 , u as a parameter indicating the property of the data of the object 3 as a result of learning normal image data.
- u 2 ,..., u d ) are transmitted to the storage unit 13a.
- the storage unit 13 a stores a parameter representing the data property of the object 3 as a learning result output from the analysis unit 12 a in the storage medium 103 based on an instruction from the control unit 10.
- the storage unit 13a stores the parameter representing the data property of the object 3 as the learning result output from the analysis unit 12a in the storage medium 103 based on an instruction from the control unit 10, but the storage location is
- the memory 102 may be used.
- the control unit 10 gives an instruction to start processing to the input / output unit 15 after the processing of the analysis unit 12a is completed. Based on an instruction from the control unit 10, the input / output unit 15 outputs information indicating the progress of learning and the like from the input / output device 4 to the outside.
- the worker confirms the input / output device 4 outside, but it is not always necessary to do so, and a signal is output to the external control device without any operator intervening. You may do it.
- FIG. 5 is a part of a flowchart showing an operation in the inspection mode of the inspection apparatus 1 according to the first embodiment of the present invention. The operation in the inspection mode of the inspection apparatus 1 will be described below with reference to FIG.
- step S ⁇ b> 20 the control unit 10 receives the activation signal and the setting signal from the input / output device 4 via the input / output unit 15. Then, an instruction is given to the input unit 11 based on the setting signal.
- the input unit 11 receives image data for inspecting the object 3 from the camera 2. At this time, the timing of receiving the image data may be determined in advance, for example, 30 times per second, or may be determined based on an instruction from the control unit 10.
- the control unit 10 gives an instruction to start processing in the inspection mode to the analysis unit 12a.
- the analysis unit 12 a switches to the inspection mode by reading the program corresponding to the inspection mode of the storage medium 103 loaded in the memory 102 from the memory 102 and executing it by the processor 101.
- the analysis unit 12a receives the image data of the object 3 captured by the camera 2 from the input unit 11.
- the image data to be inspected by the object 3 is an example of the data of the object to be inspected.
- the analysis unit 12a determines whether to further receive image data or to finish receiving image data.
- the end determination of the image data reception may be determined by the analysis unit 12 a or may be determined based on an instruction from the control unit 10.
- the analysis unit 12a determines, for example, it is conceivable that the reception of the image data is terminated when the number of received image data reaches a predetermined number.
- the number of sheets designated in advance is, for example, 1 or 10 sheets.
- the control unit 10 receives an image data reception end instruction from the input / output device 4 via the input / output unit 15, and transmits it to the analysis unit 12a. It is possible.
- step S22 the analysis unit 12a returns to the reception of the image data based on the determination result in step S21 or proceeds to the next step.
- step S22 No, and the process returns to S20 again. If it is determined that image data acquisition has been completed, step S22: Yes, and the process proceeds to the next step.
- step S23 the analysis unit 12a transmits a read request to the control unit 10 in order to read the result learned in the learning mode.
- the storage unit 13a reads a necessary learning result from the storage medium 103 based on an instruction from the control unit 10, and inputs the learning result to the analysis unit 12a.
- the learning result read out here is a learning result corresponding to the dimension compression method step S13 used in the learning mode. That is, in the first embodiment, since principal component analysis is used as an example, learning in which the values of d-dimensional principal components (u 1 , u 2 ,..., U d ), which are vectors representing the principal components, are read out. Result.
- step S24 the analysis unit 12a performs dimensional compression of the received image data to be inspected based on the read learning result.
- the dimension compression method is a dimension compression method corresponding to the dimension compression technique step S13 used in the learning mode. Since the values of the d-dimensional principal components (u 1 , u 2 ,..., U d ), which are vectors representing the principal components in the learning mode, are obtained, the image data to be inspected is projected onto the d-dimensional vector. Perform dimension compression.
- the analysis unit 12a transmits to the determination unit 14a image data to be inspected and a vector that is a result of dimensional compression of the image data to be inspected.
- A is the next process, and details will be described later.
- FIG. 6 is a part of a flowchart showing an operation in the inspection mode of the inspection apparatus 1 according to Embodiment 1 of the present invention. The continuation of the operation of the inspection apparatus 1 in the inspection mode will be described below with reference to FIG.
- step S30 as a continuation of the process A after step S24, the control unit 10 gives an instruction to start processing to the determination unit 14a after the processing of the analysis unit 12a is completed.
- the determination unit 14 a reads a program stored in the storage medium 103 loaded into the memory 102 from the memory 102 and executes the program on the processor 101.
- the determination unit 14a first restores, as image data, a vector that is a result of dimensional compression of the image data to be inspected received from the analysis unit 12a.
- the method for restoring the image data is the same as the method used in the dimensional compression in step S24. That is, in the first embodiment, restoration is performed using principal component analysis.
- the vector that is the result of dimensional compression of the received image data to be inspected is shown in a low-dimensional subspace as shown in FIG.
- the image data is restored by projecting onto the same dimensional space.
- step S31 the determination unit 14a calculates a difference between the restored image data and the image data to be inspected. At this time, the difference is calculated for each pixel.
- the difference may be an absolute value difference.
- the restored image data is referred to as restored data.
- FIG. 7 shows an example in which the printed circuit board is the object 3 and the substrate is inspected for any abnormality.
- FIG. 7 shows the image data to be inspected on the left, the restored data on the center, and the data representing the difference between the image data to be inspected and the restored data on the right.
- the difference is smaller as it is black, and the difference is larger as it is white.
- the substrate that is the object 3 to be inspected is normal, it is possible to restore image data that is substantially the same as the image data to be inspected even if dimension compression is performed.
- the learning mode the method of efficiently showing the characteristics of the normal image data is learned. If the object 3 to be inspected is normal in the inspection mode, the image is very similar to the normal image data used for learning. This is because the data becomes image data to be inspected.
- FIG. 8 shows an example in which a part of the substrate that is the object 3 to be inspected has a defect.
- FIG. 8 shows the image data to be inspected on the left, the restored data on the center, and the data representing the difference between the image data to be inspected and the restored data.
- the difference is smaller as it is black, and the difference is larger as it is white.
- the normal part is correctly restored in the restored data, the missing part cannot be restored correctly because it is restored based on the normal image data used in the learning mode.
- step S ⁇ b> 32 the determination unit 14 a outputs, to the input / output unit 15, a determination result indicating whether there is an abnormality in the image data to be inspected based on the difference between the restored data and the image data to be inspected.
- the determination unit 14a performs threshold processing on the difference between the restored data and the image data to be inspected, and sets the value of a pixel whose difference value is less than the threshold to 0 and the value of a pixel where the difference value is greater than or equal to the threshold.
- 0 and 1 may be interchanged or other values may be used.
- the threshold value may be determined based on an instruction from the control unit 10, for example.
- the control unit 10 receives the threshold value from the input / output device 4 via the input / output unit 15 and transmits it to the analysis unit 12a.
- the determination unit 14 a may read and use the threshold value stored in the memory 102 or the storage medium 103.
- the threshold value stored in advance is 100, 200, or the like, for example.
- the determination unit 14a may adaptively determine the threshold value according to the distribution of difference values.
- a set of pixels having a pixel value equal to or greater than the threshold when a certain threshold is determined is class 1
- a set of other pixels is class 2
- the inter-class variance and within the class are determined from the pixel values of class 1 and class 2.
- the variance is obtained, and the threshold value is determined so that the degree of separation calculated from these values is maximized.
- FIG. 9 shows the result of threshold processing. Assume that the result shown in FIG. 9 is obtained by the threshold processing. However, in FIG. 9, the area indicated by black is an area where the area is less than the threshold, and the area indicated by white is an area where the area is greater than or equal to the threshold.
- the determination unit 14a obtains a rectangle circumscribing the white area as indicated by a broken line in FIG. 9, and transmits to the input / output unit 15 that an abnormality exists at the position.
- a rectangle circumscribing the white area is referred to as a bounding box.
- the information to be transmitted includes the upper left coordinates, the vertical width, the horizontal width, etc. of the bounding box.
- the determination unit 14 a may transmit all of the positions of pixels having an abnormality to the input / output unit 15 without transmitting the bounding box to the input / output unit 15.
- the determination unit 14 a may transmit the calculated difference image data to the input / output unit 15.
- a condition may be set for the position or size of the bounding box, and the bounding box that does not satisfy the condition may be ignored. By doing so, it is possible to prevent erroneous detection outside the target area in the image data or prevent erroneous detection due to noise.
- the condition for the bounding box may be determined based on an instruction from the control unit 10, for example.
- the control unit 10 receives a condition from the input / output device 4 via the input / output unit 15 and transmits the condition to the analysis unit 12a.
- the determination unit 14a may read and use the conditions stored in the memory 102 or the storage medium 103.
- the conditions stored in advance in this case are, for example, that the vertical width of the bounding box is 3 pixels or more, the horizontal width of the bounding box is 3 pixels or more, and in the two-dimensional mask for limiting the inspection target region. There are some things.
- FIG. 10 is an example of a two-dimensional mask for limiting the inspection target area.
- FIG. 10 shows image data to be inspected on the left side, and a two-dimensional mask on the right side.
- the determination unit 14a applies the two-dimensional mask in the right figure to the image data to be inspected in the left figure.
- the area indicated in white is the inspection target area
- the area indicated in black is the bounding box. Therefore, when the two-dimensional mask in the right figure is applied to the image data to be inspected in the left figure, even if there is an abnormality in the area in the left figure corresponding to the bounding box shown in black in the two-dimensional mask in the right figure It will be ignored.
- the control unit 10 gives an instruction to start processing to the input / output unit 15 after the processing of the determination unit 14 a is completed.
- the input / output unit 15 outputs the determination result received from the determination unit 14 a to the outside from the input / output device 4 based on an instruction from the control unit 10.
- the worker confirms the input / output device 4 outside, but it is not always necessary to do so, and a signal is output to an external control device or the like without interposing the worker in particular. You may make it do.
- FIG. 11 is an example of display contents that the input / output unit 15 instructs the input / output device 4 when the input / output device 4 includes a display in its configuration.
- FIG. 11 shows a case where no abnormality is detected by the determination unit 14a.
- the image data to be inspected is simply displayed as it is, and a message for notifying that there is no abnormality is displayed.
- An example of the message for notifying that there is no abnormality is the OK mark shown in the upper left of FIG. Instead of an OK mark, a mark such as no abnormality, normal, or a mark may be used.
- FIG. 12 shows another example of display contents that the input / output unit 15 instructs the input / output device 4 when the input / output device 4 includes a display.
- FIG. 12 shows a case where two abnormalities are detected by the determination unit 14a.
- a detected abnormal point is superimposed on the image data to be inspected with a dotted line, and a message for notifying that an abnormality has been detected is displayed.
- An example of the message for notifying that an abnormality has been detected is the NG mark shown in the upper left of FIG. Instead of the NG mark, a mark such as abnormal, abnormal, or x mark may be used.
- the abnormal part is specified based on the bounding box received from the determination unit 14a.
- the determination of the abnormal part based on the bounding box may be performed by either the determination unit 14a or the input / output unit 15. Note that the bounding box may or may not be displayed.
- FIG. 13 shows still another example of display contents that the input / output unit 15 instructs the input / output device 4 when the input / output device 4 includes a display in its configuration.
- FIG. 13 shows a case where two abnormalities are detected by the determination unit 14a as in FIG.
- the difference composite image data obtained by combining the image data to be inspected on the left side and the difference image data calculated by the determination unit 14a with the image data to be inspected on the right side. indicate.
- about difference synthetic image data it has shown that a difference is so small that it is black, and a difference is so large that it is white.
- the portion that is conspicuous in white in the difference composite image data on the right side of FIG. 13 appears to be remarkably conspicuous as a portion where the difference from the normal state is large in the image data to be inspected, the abnormality is inspected. It is possible to easily grasp the location that the operator should pay attention to.
- either the determination unit 14a or the input / output unit 15 may identify the abnormal part based on the bounding box.
- the display method in the input / output device 4 shown in FIGS. 11, 12, and 13 is merely an example, and in actuality, a combination of these or a display method different from these may be used.
- the input / output device 4 may be configured by a speaker instead of a display. In this case, it may be possible to output information to the outside by voice, music, or the like.
- the object 3 and the camera 2 are securely fixed by repeating the learning mode process and the inspection mode process as described above until there is a trigger for the end of the process such as turning off the power or performing an end operation.
- a trigger for the end of the process such as turning off the power or performing an end operation.
- the learning mode process and the inspection mode process are repeated, the learning mode process may be performed only once without being repeated. Similarly, the inspection mode process may be performed only once without being repeated.
- the inspection apparatus 1 included in the inspection system of the first embodiment expresses the property of the data of the object 3 that does not include an abnormality by dimensionally compressing the data of the object 3 that does not include the abnormality.
- the parameter is calculated, the data of the object 3 to be inspected is dimensionally compressed using the parameters, the restored data of the dimension-compressed object 3 to be inspected is generated, and the data and the restored data of the object 3 to be inspected are generated. Since the determination result indicating whether or not the object 3 to be inspected is abnormal is output to the input / output unit 15 based on the difference between the object 3 and the camera 2, the object 3 and the camera 2 are more securely fixed than the conventional inspection apparatus. In addition, it is possible to inspect whether there is an abnormality such as a defect, misplacement, or defect of the object 3 while relaxing the restriction that highly accurate alignment is performed for each pixel of image data obtained by photographing the object 3.
- the inspection apparatus 1 included in the inspection system of the first embodiment extracts only parameters representing the properties of the data of the object 3 that appear in common in normal image data by dimensional compression using principal component analysis. Efficient features can be obtained. In this process, unnecessary information such as variations among objects 3 of the same type, variations in image data due to different shooting timings, camera noise, and the like are discarded, so that the data size is reduced and the storage medium 103 is required. Storage capacity can be reduced.
- the inspection apparatus 1 included in the inspection system according to the first embodiment learns and inspects the image data of the target object 3 in the normal state from the normal image data, the user, the developer, etc. about the abnormal state. There is no need to define it. For this reason, there is a leak in the definition of the abnormal state, and the abnormality is not overlooked, and can be applied universally to any abnormality.
- the inspection device 1, the camera 2, and the input / output device 4 are separated.
- the camera 2, the input / output device 4, or the camera 2 and the input / output device 4 are separated. It is good also as a structure which included both in the inspection apparatus 1. FIG. Even in the inspection system configured as described above, the effect of the first embodiment described above can be obtained.
- the determination unit 14a restores the image data of the object 3 to be inspected, which has been dimensionally compressed, by a method similar to the method used in the dimension compression, and is restored image data. Based on the difference between the data and the image data of the object 3 to be inspected, a determination result indicating whether or not the object 3 to be inspected is abnormal is output to the input / output unit 15.
- the determination unit 14a is an example of a unit that combines the restoration unit and the determination unit.
- a restoration unit that restores the image data of the object 3 to be inspected subjected to dimension compression by a method similar to the method used in the dimension compression may be provided in the inspection apparatus 1 independently, or the analysis unit 12a may include the restoration unit. You may have the function of. Even in the inspection system configured as described above, the effect of the first embodiment described above can be obtained.
- the analysis unit 12a uses principal component analysis for dimensional compression of the object 3, the storage unit 13a stores the result of the principal component analysis, and the determination unit 14a
- restoration was performed using component analysis
- the dimension compression method may be changed for each type of the object 3.
- the analysis unit 12 a uses principal component analysis for dimensional compression of the first type of object 3
- the storage unit 13 a stores the result of the principal component analysis
- the determination unit 14 a Is restored using principal component analysis
- the analysis unit 12a uses linear discriminant analysis for dimensional compression of the second type of object 3, and the storage unit 13a performs linear discrimination.
- the analysis result may be stored, and the determination unit 14a may perform restoration using linear discriminant analysis.
- the number of types of the target object 3 may be any number, and the combination of the type of the target object 3 and the dimension compression method is arbitrary. However, in the case of the same type of object 3, the same dimensional compression method is used. Even in the inspection system configured as described above, the effect of the first embodiment described above can be obtained.
- the analysis unit 12a uses principal component analysis for dimensional compression of the object 3, the storage unit 13a stores the result of the principal component analysis, and the determination unit 14a restores using principal component analysis. Went.
- Principal component analysis is a representative example of a linear dimensional compression technique.
- the analysis unit 12b, the storage unit 13b, and the determination unit 14b according to the second embodiment use a self-encoder that uses a neural network for dimensional compression.
- a self-encoder using a neural network is known as a technique capable of nonlinear dimensional compression. For this reason, since non-linear dimensional compression is possible compared with principal component analysis which is a linear dimensional compression method, it is possible to acquire features more efficiently. The rest is the same as in the first embodiment.
- FIG. 14 is a functional block diagram of an inspection system including the inspection apparatus 200 according to Embodiment 2 of the present invention.
- the configurations and operations already described are denoted by the same reference numerals, and redundant description is omitted.
- an analysis unit 12b, a storage unit 13b, and a determination unit 14b are added as a configuration of the functional block diagram instead of the analysis unit 12a, the storage unit 13a, and the determination unit 14a of FIG. 1 of the first embodiment.
- the analysis unit 12b performs dimensional compression using a self-encoder using a neural network instead of the principal component analysis used in the dimensional compression in the analysis unit 12a of the first embodiment.
- the rest is the same as the analysis unit 12a.
- the storage unit 13b stores the learning result of the principal component analysis stored in the storage unit 13a of the first embodiment and reads out the learning result of the principal component analysis and inputs it to the analysis unit 12a.
- the learning result of the self-encoder using the memory and the neural network is read out and input to the analysis unit 12b.
- the rest is the same as the storage unit 13a.
- the determination unit 14b performs restoration using a self-encoder using a neural network instead of the principal component analysis used in the restoration in the determination unit 14b of the first embodiment. Other than that is the same as the determination part 14a.
- the hardware configuration diagram of the inspection apparatus 200 according to the second embodiment of the present invention is the same as FIG. 2 of the first embodiment.
- the analysis unit 12b has the same hardware configuration as the analysis unit 12a
- the storage unit 13b has the same hardware configuration as the storage unit 13a
- the determination unit 14b has the same hardware configuration as the determination unit 14a.
- FIG. 15 is a flowchart showing the operation in the learning mode of the inspection apparatus 200 according to Embodiment 2 of the present invention. The operation in the learning mode of the inspection apparatus 200 will be described below using FIG.
- Step S40, step S41, and step S42 are the same as step S10, step S11, and step S12 of the first embodiment.
- the analysis unit 12b performs processing instead of the analysis unit 12a.
- step S43 the analysis unit 12b performs dimensional compression using the received normal image data.
- dimensional compression refers to converting high-dimensional data such as image data and three-dimensional stereoscopic data into low-dimensional data, as in the first embodiment.
- the analysis unit 12b performs learning using normal image data in the learning mode, and obtains a data conversion method that is optimal for normal image data. Below, the case where the self-encoder by a neural network is used as a dimension compression method is demonstrated.
- a neural network is a computer that mimics the mechanism of the human brain in which neurons connected in a network form through synapses perform learning, pattern recognition, etc. with the strength of the current flowing therethrough.
- the simplest model is the perceptron. be called.
- FIG. 16 is a diagram in which neurons are modeled as nodes with multiple inputs and one output.
- the perceptron is a diagram in which a neuron is modeled as a node with multiple inputs and one output.
- the total number of pixels per normal image data received in step S40 is K, and the number of dimensions is j.
- K is 1024 if the size of normal image data is 32 pixels ⁇ 32 pixels, and 409600 if the size of 640 pixels ⁇ 640 pixels.
- FIG. 16 shows the calculation for one piece of image data. If there are 1000 pieces of image data, the same calculation is performed 1000 times.
- discriminant function 2 classes take values 1 or 0 It becomes.
- a class c1 that collects dog image data and a class c2 that collects cat image data are prepared in advance, and each image data includes dog image data. Label that it is.
- image data of a dog that is not labeled as input data is put into the discriminant function, image data that is not labeled as input data from the class c1 dog image data and the class c2 cat image data is class.
- the image data is c1 dog image data or class c2 cat image data, and dog image data that is not labeled as input data is determined to be dog image data.
- the threshold logic function z (u) in the perceptron can be replaced with various other functions more generally called activation functions. For example, a sigmoid function, ReLU, etc. can be mentioned.
- a plurality of data belonging to class c1 and data belonging to class c2 are prepared in advance.
- the data belonging to the class c1 and the data belonging to the class c2 prepared in advance are described as learning data.
- the value of the weight vector w is an example of a parameter that represents the nature of the data of the object 3.
- FIG. 17 is an example of an hourglass neural network.
- the self-encoder is an hourglass type in which the number of nodes in the hidden layer between the input layer to which data is input and the output layer to be output is smaller than the number of nodes in the input layer and the output layer.
- It is a neural network.
- the left end of the network shown in FIG. 17 is the input layer
- the right end of the network shown in FIG. 17 is the output layer
- the middle of the network shown in FIG. 17 is the hidden layer.
- the total number of pixels per normal image data received in step S40 is K
- the number of dimensions is j.
- the value of K is 1024 if the size of normal image data is 32 pixels ⁇ 32 pixels, and 409600 if the size of 640 pixels ⁇ 640 pixels.
- the input data is a normal image data vector x that is a collection of pixels x j of normal image data, and a collection of weights w j is a weight vector w.
- the vector of the hidden layer be a j .
- the output data is a vector y of image data that is a collection of pixels y j that is substantially the same as a vector x of normal image data that is a collection of pixels x j of normal image data, and a collection of weights v j is a weight vector v. It becomes.
- FIG. 17 shows the calculation for one piece of image data. If there are 1000 pieces of image data, the same calculation is performed 1000 times.
- the self-encoder learns the weight of each node so that the input data and the output data substantially match the learning data. By doing so, it is known that features with reduced dimensions can be obtained while preserving input data information in the hidden layer as much as possible.
- step S43 the normal image data acquired in step S40 is used as input data, and a self-encoder that substantially matches the input data and output data is learned, so that it is common to normal image data. Only the important components that appear can be extracted in the hidden layer. In other words, a self-encoder is learned such that normal image data input as input data and image data obtained by restoring input data output through a hidden layer match with important components that appear in common in normal image data. . On the other hand, unnecessary components such as variations between objects of the same type, image variations due to different shooting timings, camera noise, and the like can be excluded.
- FIG. 18 is an example showing a state when the total number of hidden layers of the self-encoder is changed.
- the hidden layer of the self-encoder shown in FIG. 17 is one layer, the total number can be changed freely. It is easy to change the total number, and as shown in FIG. 18A, first, the hidden layer shown in FIG. Thereafter, as shown in (2) of FIG. 18, a network that compresses the dimension of the input data is obtained by removing the output layer of the self-encoder and leaving the input layer and the hidden layer. Next, as shown in (3) of FIG. 18, by further learning a self-encoder having a weight vector different from that of (1) of FIG. 18 using the dimension-compressed data as input data. Dimension can be compressed.
- the number of layers of the self-encoder is an important parameter that affects the performance of the inspection apparatus 200 according to the second embodiment.
- the number of layers By appropriately setting the number of layers, it is possible to extract only important components that appear in common in the normal image data acquired in S40.
- unnecessary components such as variations among objects 3 of the same type, variations in image data due to different shooting timings, camera noise, and the like can be excluded.
- the number of layers may be determined based on an instruction from the control unit 10.
- the control unit 10 receives the number of layers from the input / output device 4 via the input / output unit 15 and transmits it to the analysis unit 12b.
- the analysis unit 12b may read and use the value of the number of layers stored in the memory 102 or the storage medium 103.
- the number of layers stored in advance is about 5, for example, about 10.
- the analysis unit 12b transmits the value of the weight vector of each node to the storage unit 13b as a result of learning the normal image data.
- the storage unit 13 b stores the learning result output from the analysis unit 12 b in the storage medium 103 based on an instruction from the control unit 10.
- the storage unit 13b stores the learning result output from the analysis unit 12b in the storage medium 103 based on an instruction from the control unit 10, the storage location may be the memory 102.
- the control unit 10 gives an instruction to start processing to the input / output unit 15 after the processing of the analysis unit 12a is completed. Based on an instruction from the control unit 10, the input / output unit 15 outputs information indicating the progress of learning and the like from the input / output device 4 to the outside.
- the worker confirms the input / output device 4 outside, but it is not always necessary to do so, and a signal is output to the external control device without any operator intervening. You may do it.
- FIG. 19 is a part of a flowchart showing an operation in the inspection mode of the inspection apparatus 200 according to Embodiment 2 of the present invention. The operation in the inspection mode of the inspection apparatus 200 will be described below using FIG.
- Step S50, step S51, and step S52 are the same as step S20, step S21, and step S22 of the first embodiment.
- the analysis unit 12b performs processing instead of the analysis unit 12a.
- step S53 the analysis unit 12b transmits a read request to the control unit 10 in order to read the result learned in the learning mode.
- the storage unit 13b reads a necessary learning result from the storage medium 103 based on an instruction from the control unit 10, and inputs the learning result to the analysis unit 12b.
- the learning result read out here is a learning result corresponding to the dimension compression method step S43 used in the learning mode. That is, in the second embodiment, since a self-encoder using a neural network is used as an example, a learning result in which the weight vector value of each node is read out is obtained.
- step S54 the analysis unit 12b performs dimensional compression of the received image data to be inspected based on the read learning result.
- the dimension compression method is a dimension compression method corresponding to the dimension compression method step S43 used in the learning mode. Since the value of the weight vector of each node is obtained in the learning mode, the dimension compression can be performed by inputting the image data to be inspected to the neural network using the same weight.
- the analysis unit 12b transmits the image data to be inspected and a vector that is a result of dimensional compression of the image data to be inspected to the determination unit 14b.
- the vector that is the result of dimension compression of the image data to be inspected is output data that is the value of the output layer.
- B is the next process, and details will be described later.
- FIG. 20 is a part of a flowchart showing the operation in the inspection mode of the inspection apparatus 200 according to Embodiment 2 of the present invention. The continuation of the operation in the inspection mode of the inspection apparatus 200 will be described below with reference to FIG.
- step S60 as a continuation of the processing of B after step S54, the control unit 10 gives an instruction to start processing to the determination unit 14b after the processing of the analysis unit 12b is completed.
- the determination unit 14 b reads a program in the storage medium 103 loaded into the memory 102 from the memory 102 and executes the program on the processor 101.
- the determination unit 14b restores, as image data, a vector that is a result of dimensional compression of image data to be inspected received from the analysis unit 12b.
- the method for restoring the image data is the same as the method used in the dimension compression in step S54. That is, in the second embodiment, restoration is performed using a self-encoder based on a neural network.
- the output of the neural network is a one-dimensional vector
- the image data is restored by rearranging it into a two-dimensional array.
- the output of the neural network may be two-dimensional image data. In this case, it is not necessary to restore the image data.
- Step S61 and step S62 are the same as step S31 and step S32 of the first embodiment.
- the analysis unit 12b performs processing instead of the analysis unit 12a, the storage unit 13b instead of the storage unit 13a, and the determination unit 14b instead of the determination unit 14a.
- the object 3 and the camera 2 are securely fixed by repeating the learning mode process and the inspection mode process as described above until there is a trigger for the end of the process such as turning off the power or performing an end operation.
- a trigger for the end of the process such as turning off the power or performing an end operation.
- the learning mode process and the inspection mode process are repeated, the learning mode process may be performed only once without being repeated. Similarly, the inspection mode process may be performed only once without being repeated.
- the inspection apparatus 200 included in the inspection system according to the second embodiment expresses the property of the data of the object 3 that does not include an abnormality by dimensionally compressing the data of the object 3 that does not include the abnormality.
- the parameter is calculated, the data of the object 3 to be inspected is dimensionally compressed using the parameters, the restored data of the dimension-compressed object 3 to be inspected is generated, and the data and the restored data of the object 3 to be inspected are generated. Since the determination result indicating whether or not the object 3 to be inspected is abnormal is output to the input / output unit 15 based on the difference between the object 3 and the camera 2, the object 3 and the camera 2 are more securely fixed than the conventional inspection apparatus. In addition, it is possible to inspect whether there is an abnormality such as a defect, misplacement, or defect of the object 3 while relaxing the restriction that highly accurate alignment is performed for each pixel of image data obtained by photographing the object 3.
- an abnormality such as a defect, misplacement, or defect of the object 3
- the inspection apparatus 200 included in the inspection system of the second embodiment has an efficient feature in which only parameters appearing in normal image data are extracted by dimensional compression using a self-encoder based on a neural network. Obtainable. In this process, unnecessary information such as variations among objects 3 of the same type, variations in image data due to different shooting timings, camera noise, and the like are discarded, so that the data size is reduced and the storage medium 103 is required. Storage capacity can be reduced.
- the inspection apparatus 200 included in the inspection system according to the second embodiment learns the image data of the target object 3 in the normal state from the normal image data and performs the inspection, the user, the developer, etc. about the abnormal state. There is no need to define it. For this reason, there is a leak in the definition of the abnormal state, and the abnormality is not overlooked, and can be applied universally to any abnormality.
- the inspection apparatus 200 included in the inspection system of the second embodiment has a more efficient feature because the self-encoder can perform non-linear dimensional compression compared to principal component analysis, which is a linear dimensional compression method. Can be acquired.
- the inspection device 200, the camera 2, and the input / output device 4 are separated from each other. Both may be included in the inspection apparatus 200. Even in the inspection system configured as described above, the effect of the second embodiment described above can be obtained.
- the determination unit 14b restores the image data of the object 3 to be inspected that has been dimensionally compressed by the same method as that used in the dimension compression, and is restored image data. Based on the difference between the data and the image data of the object 3 to be inspected, a determination result indicating whether or not the object 3 to be inspected is abnormal is output to the input / output unit 15.
- the determination unit 14b is an example of a unit that combines the restoration unit and the determination unit.
- a restoration unit that restores the image data of the object 3 to be inspected subjected to dimension compression by a method similar to the method used in dimension compression may be provided independently in the inspection apparatus 200, or the analysis unit 12b may include a restoration unit. You may have the function of. Even in the inspection system configured as described above, the effect of the second embodiment described above can be obtained.
- the analysis unit 12b uses a self-encoder using a neural network for dimensional compression of the object 3, and the storage unit 13b stores the result of the self-encoder using the neural network. Then, the determination unit 14b performs the restoration using the self-encoder based on the neural network. However, when there are a plurality of types in the target object 3, the dimensional compression method is changed for each type of the target object 3. Good. For example, in the first type object 3, the analysis unit 12 b uses a neural network self-encoder for dimensional compression of the first type object 3, and the storage unit 13 b uses a neural network self-encoder.
- the determination unit 14b performs restoration using a self-encoder based on a neural network.
- the analysis unit 12b is linear in the dimensional compression of the second type of object 3.
- the storage unit 13b may store the result of linear discriminant analysis, and the determination unit 14b may perform restoration using linear discriminant analysis.
- the number of types of the target object 3 may be any number, and the combination of the type of the target object 3 and the dimension compression method is arbitrary. However, in the case of the same type of object 3, the same dimensional compression method is used. Even in the inspection system configured as described above, the effect of the second embodiment described above can be obtained.
- the analysis unit 12b uses a self-encoder based on a neural network for dimensional compression of the object 3, but the self-encoder is a dimensional compression method based on a neural network. Since it is the simplest method among them, a more complicated method may be used. Even in the inspection system configured as described above, in addition to the effect of the above-described second embodiment, an effect that the performance can be further improved can be obtained. Examples of more complicated methods include a method using a convolutional neural network (Convolutional Neural Network), a General Adversal Network, and the like. In the inspection apparatus 1 according to the second embodiment, any of these various dimensional compression methods may be used. Steps S43 and S44 in FIG. 15, steps S53 and S54 in FIG. The processing method of the self-encoder by the neural network in step S60 of 20 may be changed to a convolutional neural network, a generalized advisory network, or the like.
- Convolutional Neural Network Convolutional Neural Network
- General Adversal Network General Adversal
- Embodiment 3 the data representing the property of the object 3 that does not include the abnormality is calculated by dimension-compressing the data of the object 3 that does not include the abnormality, and the data of the object 3 to be inspected is used as the parameter.
- the object 3 to be inspected based on the magnitude of the difference between the data of the object 3 to be inspected and the restored data is generated by generating dimensionally compressed data and restoring data obtained by restoring the data of the object 3 to be inspected that has been dimensionally compressed.
- the determination result indicating whether or not is output to the input / output unit 15.
- a filter that reduces an error between the restored data and the image data to be inspected is designed, and correction for filtering the restored data using the designed filter is performed.
- the unit 16 is newly added, and includes a determination unit 14c instead of the determination unit 14a. For this reason, even when a significant error occurs between the restored data and the image data to be inspected due to, for example, lack of normal image data for learning used in the learning mode, the correction unit 16 reduces both errors. Thus, the determination accuracy in the subsequent determination unit 14c can be increased.
- the rest is the same as in the first embodiment.
- FIG. 21 is a functional block diagram of an inspection system including the inspection apparatus 300 according to Embodiment 3 of the present invention.
- the configurations and operations already described are denoted by the same reference numerals, and redundant description is omitted.
- a determination unit 14c is added instead of the determination unit 14a in FIG. 1 of the first embodiment, and a correction unit 16 is newly added as a configuration of the functional block diagram.
- the correcting unit 16 restores the image data of the inspection object 3 subjected to dimension compression by a method similar to the method used in the dimension compression, and designs a filter that corrects an error between the restored data and the image data to be inspected.
- the restoration data is filtered using the designed filter to generate correction data.
- the correction data is restored data obtained by filtering the restored data using a filter that corrects an error between the restored data and the data of the object to be inspected.
- the correction unit 16 is an example of a unit that combines the restoration unit and the correction unit.
- the determination unit 14c indicates whether or not the object 3 to be inspected is abnormal based on the difference between the correction data that is the image data restored and corrected by the correction unit 16 and the image data of the object 3 to be inspected.
- the determination result is output to the input / output unit 15.
- the determination unit 14c is different from 14a including the restoration unit, and the restoration is performed by the correction unit 16. Therefore, the determination unit 14c does not perform restoration. Other than that is the same as the determination part 14a.
- the hardware configuration diagram of the inspection apparatus 300 according to the third embodiment of the present invention is the same as FIG. 2 of the first embodiment.
- the hardware configuration of the determination unit 14c is the same as that of the determination unit 14a.
- the correction unit 16 is realized by reading and executing the program of the storage medium 103 loaded into the memory 102 by the processor 101.
- the flowchart showing the operation in the learning mode of the inspection apparatus 300 according to the third embodiment of the present invention is the same as FIG. 3 of the first embodiment.
- Part of the flowchart showing the operation in the inspection mode of the inspection apparatus 300 according to the third embodiment of the present invention is the same as FIG. 5 of the first embodiment.
- FIG. 22 is a part of a flowchart showing an operation in the inspection mode of the inspection apparatus 300 according to the third embodiment of the present invention. The continuation of the operation of the inspection apparatus 300 in the inspection mode will be described below with reference to FIG.
- step S70 as a continuation of the process A after step S24, the control unit 10 gives an instruction to start processing to the correction unit 16 after the processing of the analysis unit 12a is completed.
- the correction unit 16 reads the program of the storage medium 103 loaded into the memory 102 based on an instruction from the control unit 10 and executes the program on the processor 101.
- the correction unit 16 receives a vector which is a result of dimensional compression of the image data to be inspected received from the analysis unit 12a and an image data to be inspected, and images the vector which is a result of dimensional compression of the received image data to be inspected Restore as data.
- the method for restoring the image data is the same as the method performed by the determination unit 14a in step S30 of FIG. 6 of the first embodiment.
- step S71 the correction unit 16 designs a filter that reduces an error between the restored data and the image data to be inspected.
- the correction unit 16 reduces both errors.
- the determination accuracy in the subsequent determination unit 14c can be increased.
- Equation (9) N represents the Fourier transform of noise.
- the filter K w represented by the formula (10) is called a Wiener filter.
- step S72 the correcting unit 16 transmits the image data to be inspected after performing correction by filtering using a filter K w relative to the restored data and correction data to the determining unit 14c.
- C is the next processing, and details will be described later.
- FIG. 23 is a part of a flowchart showing an operation in the inspection mode of the inspection apparatus 300 according to the third embodiment of the present invention. The continuation of the operation of the inspection apparatus 300 in the inspection mode will be described below with reference to FIG.
- step S80 as a continuation of the process of C after step S72, the control unit 10 gives an instruction to start the process to the determination unit 14c.
- the determination unit 14 c reads the program of the storage medium 103 loaded into the memory 102 based on an instruction from the control unit 10 from the memory 102 and executes the program on the processor 101.
- the determination unit 14c first receives image data and correction data to be inspected from the correction unit 16.
- the determination unit 14c calculates a difference between the correction data and the image data to be inspected. Except for the above, the determination unit 14c performs the same processing as the determination unit 14a in step S31 of FIG.
- Step S81 is the same as step S32 in the first embodiment.
- the determination unit 14c performs processing instead of the determination unit a.
- the object 3 and the camera 2 are securely fixed by repeating the learning mode process and the inspection mode process as described above until there is a trigger for the end of the process such as turning off the power or performing an end operation.
- a trigger for the end of the process such as turning off the power or performing an end operation.
- the learning mode process and the inspection mode process are repeated, the learning mode process may be performed only once without being repeated. Similarly, the inspection mode process may be performed only once without being repeated.
- the inspection apparatus 300 included in the inspection system according to the third embodiment expresses the property of the data of the object 3 that does not include an abnormality by dimensionally compressing the data of the object 3 that does not include an abnormality.
- the parameter is calculated, the data of the object 3 to be inspected is dimensionally compressed using the parameters, the restored data of the dimension-compressed object 3 to be inspected is generated, and the data and the restored data of the object 3 to be inspected are generated. Since the determination result indicating whether or not the object 3 to be inspected is abnormal is output to the input / output unit 15 based on the difference between the object 3 and the camera 2, the object 3 and the camera 2 are more securely fixed than the conventional inspection apparatus. In addition, it is possible to inspect whether there is an abnormality such as a defect, misplacement, or defect of the object 3 while relaxing the restriction that highly accurate alignment is performed for each pixel of image data obtained by photographing the object 3.
- an abnormality such as a defect, misplacement, or defect of the object 3
- the inspection apparatus 300 included in the inspection system of the third embodiment extracts only parameters representing the data properties of the object 3 that appear in common with normal image data by dimensional compression using principal component analysis. Efficient features can be obtained. In this process, unnecessary information such as variations among objects 3 of the same type, variations in image data due to different shooting timings, camera noise, and the like are discarded, so that the data size is reduced and the storage medium 103 is required. Storage capacity can be reduced.
- the inspection apparatus 300 included in the inspection system of the third embodiment learns the image data of the target object 3 in the normal state from the normal image data and performs the inspection, the user, the developer, etc. about the abnormal state There is no need to define it. For this reason, there is a leak in the definition of the abnormal state, and the abnormality is not overlooked, and can be applied universally to any abnormality.
- the inspection apparatus 300 included in the inspection system of the third embodiment designs a filter that reduces an error between the restored data restored from the image data of the dimensionally compressed object 3 to be inspected and the image data to be inspected.
- restoration data using the designed filter is newly added, and the determination part 14c is provided instead of the determination part 14a. For this reason, even when a significant error occurs between the restored data and the image data to be inspected due to, for example, lack of normal image data for learning used in the learning mode, the correction unit 16 reduces both errors. Thus, the determination accuracy in the subsequent determination unit 14c can be increased.
- the inspection device 300, the camera 2, and the input / output device 4 are separated.
- the camera 2, the input / output device 4, or the camera 2 and the input / output device 4 are separated. It is good also as a structure which included both in the inspection apparatus 300. FIG. Even with the inspection system configured as described above, the effects of the third embodiment described above can be obtained.
- the correction unit 16 designs a filter that corrects an error between the restored data and the image data to be inspected, and filters the restored data using the designed filter.
- the correction unit 16 is an example of a unit that combines the restoration unit and the correction unit.
- a restoration unit that restores the image data of the object 3 to be inspected subjected to dimension compression by a method similar to the method used in the dimension compression may be provided independently in the inspection apparatus 300, or the analysis unit 12a may include the restoration unit. You may have the function of. Even with the inspection system configured as described above, the effects of the third embodiment described above can be obtained.
- the analysis unit 12a uses principal component analysis for dimensional compression of the object 3, the storage unit 13a stores the result of the principal component analysis, and the correction unit 16
- restoration was performed using component analysis
- the dimension compression method may be changed for each type of the object 3.
- the analysis unit 12 a uses principal component analysis for dimensional compression of the first type of object 3
- the storage unit 13 a stores the result of the principal component analysis, and the correction unit 16. Is restored using principal component analysis, and in the second type of object 3, the analysis unit 12a uses linear discriminant analysis for dimensional compression of the second type of object 3, and the storage unit 13a performs linear discrimination.
- the analysis result may be stored, and the correction unit 16 may perform restoration using linear discriminant analysis.
- the number of types of the target object 3 may be any number, and the combination of the type of the target object 3 and the dimension compression method is arbitrary. However, in the case of the same type of object 3, the same dimensional compression method is used. Even with the inspection system configured as described above, the effects of the third embodiment described above can be obtained.
- the analysis unit 12a uses principal component analysis for dimensional compression of the object 3, but a self-encoder by a neural network or a convolutional neural network (General Neural Network), General A more complicated method may be used than a self-encoder using a neural network such as Adversial Network. Even in the inspection system configured as described above, in addition to the effect of the third embodiment described above, an effect that the performance can be further improved can be obtained. In the inspection apparatus 300 according to the third embodiment, any of these various dimensional compression techniques may be used.
- the inspection system provided with the inspection apparatus described in the above-described embodiment is merely an example, and can be appropriately combined and configured, and is not limited to the configuration of the embodiment alone.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Biochemistry (AREA)
- Analytical Chemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Chemical & Material Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Processing (AREA)
Abstract
Description
図1は、本発明の実施の形態1に係る検査装置1を含む検査システムの機能ブロック図である。
実施の形態1では、解析部12aは、対象物3の次元圧縮に主成分分析を用い、記憶部13aは、主成分分析の結果を記憶し、判定部14aは、主成分分析を用いて復元を行った。主成分分析は線形な次元圧縮手法の代表例である。実施の形態2に係る解析部12b、記憶部13b及び判定部14bは、図14~図20に示すように、次元圧縮にニューラルネットワークを用いた自己符号化器を用いる。ニューラルネットワークを用いた自己符号化器は、非線形な次元圧縮が可能な手法として知られている。このため、線形な次元圧縮手法である主成分分析と比べ、非線形な次元圧縮が可能であるため、さらに効率的に特徴を獲得することが可能である。それ以外は、実施の形態1と同様である。
実施の形態1では、異常を含まない対象物3のデータを次元圧縮することにより異常を含まない対象物3のデータの性質を表すパラメータを算出し、検査する対象物3のデータをパラメータを用いて次元圧縮し、次元圧縮した検査する対象物3のデータを復元した復元データを生成し、検査する対象物3のデータと復元データとの差分の大小に基づき検査する対象物3が異常であるか否かを示す判定結果を入出力部15に出力していた。実施の形態3では、図21~図23に示すように、復元データと検査する画像データとの間の誤差を小さくするようなフィルタを設計及びその設計したフィルタを用いて復元データをフィルタリングする補正部16が新たに加えられ、判定部14aの代わりに判定部14cを備える。このため、例えば学習モードにおいて使用した学習用の正常画像データの不足等により、復元データと検査する画像データとの間に著しい誤差が生じてしまう場合でも補正部16において両者の誤差を低減することで、続く判定部14cにおける判定精度を高めることが可能となる。それ以外は、実施の形態1と同様である。
4 入出力装置、 10 制御部、 11 入力部、
12a, 12b 解析部、 13a, 13b 記憶部、
14a, 14b, 14c 判定部、 15 入出力部、
16 補正部、 100 入出力IF、 101 プロセッサ、
102 メモリ、 103 記憶媒体、 104 バス。
Claims (15)
- 異常を含まない対象物のデータをデータの次元を減らす次元圧縮をすることにより前記異常を含まない対象物のデータの性質を表すパラメータを算出し、検査する対象物のデータを前記パラメータを用いて次元圧縮する解析部と、
前記解析部で次元圧縮した前記検査する対象物のデータを復元した復元データを生成する復元部と、
前記検査する対象物のデータと前記復元データとの差分の大小に基づき前記検査する対象物が異常であるか否かを示す判定結果を出力する判定部と、
前記判定部が出力した前記判定結果を出力する出力部と
を備えた検査装置。 - 前記解析部が算出した前記パラメータを記憶する記憶部とを備え、
前記解析部は、前記検査する対象物のデータを前記記憶部に記憶した前記パラメータを用いて次元圧縮する
請求項1に記載の検査装置。 - 前記復元部が復元した復元データと前記検査する対象物のデータとの誤差を補正するフィルタを用いて前記復元データをフィルタリングして補正した復元データを生成する補正部とを備え、
前記判定部は、前記検査する対象物のデータと補正した前記復元データとの差分の大小に基づき前記検査する対象物が異常であるか否かを示す判定結果を出力する
請求項1~2のいずれか1項に記載の検査装置。 - 前記フィルタは、ウィーナーフィルタである
請求項3に記載の検査装置。 - 前記判定部は、前記差分が閾値以上のとき前記検査する対象物が異常であると示す判定結果を出力する
請求項1~4のいずれか1項に記載の検査装置。 - 前記異常を含まない対象物のデータは、異常を含まない対象物の画像データであり、
前記検査する対象物のデータは、検査する対象物の画像データである
請求項1~5のいずれか1項に記載の検査装置。 - 前記次元圧縮は、主成分分析を用いる
請求項1~6のいずれか1項に記載の検査装置。 - 前記次元圧縮は、ニューラルネットワークを用いた自己符号化器を用いる
請求項1~6のいずれか1項に記載の検査装置。 - 前記復元部は、前記パラメータを用いて前記復元データを生成する
請求項8記載の検査装置。 - 前記判定部は、前記検査する対象物が異常であると示す判定結果を出力した前記差分が閾値以上の領域に外接する矩形であるバウンディングボックスを算出し、
前記出力部は、前記判定部が算出した前記バウンディングボックスを前記判定結果として出力する
請求項5に記載の検査装置。 - 前記出力部は、前記バウンディングボックスに基づき特定された前記検査する対象物の画像データの異常個所を点線で前記検査する対象物の画像データに重畳表示させるように出力する
請求項10に記載の検査装置。 - 前記出力部は、前記検査する対象物の画像データに対して前記差分の画像データを合成した差分合成画像データを出力する
請求項10に記載の検査装置。 - 前記出力部は、前記判定部が前記検査する対象物が異常であると示す判定結果を出力する場合、異常が検出されたことを通知するためのメッセージを表示させるように出力する
請求項1~12のいずれか1項に記載の検査装置。 - 前記出力部が出力した前記判定結果を表示する表示部とを備えた
請求項1~13のいずれか1項に記載の検査装置。 - 異常を含まない対象物のデータをデータの次元を減らす次元圧縮をすることにより前記異常を含まない対象物のデータの性質を表すパラメータを算出し、検査する対象物のデータを記憶媒体に記憶された前記パラメータを用いて次元圧縮するステップと、
次元圧縮した前記検査する対象物のデータを復元した復元データを生成するステップと、
前記検査する対象物のデータと前記復元データとの差分の大小に基づき前記検査する対象物が異常であるか否かを示す判定結果を出力するステップと、
前記判定結果を出力するステップと
を備えた検査方法。
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/343,306 US11645744B2 (en) | 2016-12-06 | 2016-12-06 | Inspection device and inspection method |
KR1020197015539A KR102011628B1 (ko) | 2016-12-06 | 2016-12-06 | 검사 장치 및 검사 방법 |
CN201680091308.9A CN110036279B (zh) | 2016-12-06 | 2016-12-06 | 检查装置和检查方法 |
JP2017523001A JP6241576B1 (ja) | 2016-12-06 | 2016-12-06 | 検査装置及び検査方法 |
DE112016007498.0T DE112016007498B4 (de) | 2016-12-06 | 2016-12-06 | Untersuchungseinrichtung und untersuchungsverfahren |
PCT/JP2016/086138 WO2018105028A1 (ja) | 2016-12-06 | 2016-12-06 | 検査装置及び検査方法 |
TW105143296A TWI638157B (zh) | 2016-12-06 | 2016-12-27 | Inspection device and inspection method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2016/086138 WO2018105028A1 (ja) | 2016-12-06 | 2016-12-06 | 検査装置及び検査方法 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018105028A1 true WO2018105028A1 (ja) | 2018-06-14 |
Family
ID=60570330
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2016/086138 WO2018105028A1 (ja) | 2016-12-06 | 2016-12-06 | 検査装置及び検査方法 |
Country Status (7)
Country | Link |
---|---|
US (1) | US11645744B2 (ja) |
JP (1) | JP6241576B1 (ja) |
KR (1) | KR102011628B1 (ja) |
CN (1) | CN110036279B (ja) |
DE (1) | DE112016007498B4 (ja) |
TW (1) | TWI638157B (ja) |
WO (1) | WO2018105028A1 (ja) |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2018211891A1 (ja) * | 2017-05-18 | 2018-11-22 | 住友電気工業株式会社 | 異変検出装置及び異変検出方法 |
KR20200026784A (ko) * | 2018-08-31 | 2020-03-11 | 도시바 미쓰비시덴키 산교시스템 가부시키가이샤 | 제조 프로세스 감시 장치 |
WO2020110224A1 (ja) * | 2018-11-28 | 2020-06-04 | Eizo株式会社 | 情報処理方法及びコンピュータプログラム |
JP2020139905A (ja) * | 2019-03-01 | 2020-09-03 | セイコーエプソン株式会社 | 検査装置、検査方法、およびプログラム |
JP2020160616A (ja) * | 2019-03-25 | 2020-10-01 | ブラザー工業株式会社 | 生成装置、コンピュータプログラム、生成方法 |
WO2020202734A1 (ja) * | 2019-03-29 | 2020-10-08 | 株式会社ワコム | ペン状態検出回路、システム及び方法 |
JP2020190791A (ja) * | 2019-05-20 | 2020-11-26 | 株式会社アルム | 画像処理装置、画像処理システム、および画像処理プログラム |
JP2020188839A (ja) * | 2019-05-20 | 2020-11-26 | 株式会社アルム | 画像処理装置、画像処理システム、および画像処理プログラム |
CN112082653A (zh) * | 2019-06-12 | 2020-12-15 | 株式会社日立制作所 | 自动检查系统以及无线分机 |
JP2021176070A (ja) * | 2020-05-02 | 2021-11-04 | Arithmer Inc | 画像検査装置、画像検査方法及びプログラム |
WO2021220437A1 (ja) * | 2020-04-28 | 2021-11-04 | 三菱電機株式会社 | 外観検査装置および外観検査方法 |
WO2022201968A1 (ja) | 2021-03-26 | 2022-09-29 | コニカミノルタ株式会社 | 情報処理装置、制御プログラム、および制御方法 |
WO2022264824A1 (ja) * | 2021-06-16 | 2022-12-22 | 株式会社日立製作所 | 異常判定用計算機及び異常判定方法 |
Families Citing this family (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102171491B1 (ko) * | 2017-12-21 | 2020-10-29 | 동의대학교 산학협력단 | 딥러닝을 이용한 양품 선별 방법 |
JP2021042955A (ja) * | 2017-12-27 | 2021-03-18 | キユーピー株式会社 | 食品検査装置、食品検査方法及び食品検査装置の食品再構成ニューラルネットワークの学習方法 |
US20190205751A1 (en) * | 2017-12-29 | 2019-07-04 | University Of Southern California | Method for prioritizing candidate objects |
WO2019146538A1 (ja) | 2018-01-29 | 2019-08-01 | 日本電気株式会社 | 画像処理装置、画像処理方法及び記録媒体 |
DE112018007171T5 (de) | 2018-03-29 | 2021-01-28 | Mitsubishi Electric Corporation | Abnormalitätsinspektionsvorrichtung und abnormalitätsinspektionsverfahren |
JP2019178941A (ja) * | 2018-03-30 | 2019-10-17 | リコーエレメックス株式会社 | 検査システムおよび検査方法 |
JP6424309B1 (ja) * | 2018-08-15 | 2018-11-14 | 裕樹 有光 | 採寸値に基づく3次元モデルを生成するプログラム及び装置 |
JP2021009441A (ja) * | 2019-06-28 | 2021-01-28 | ルネサスエレクトロニクス株式会社 | 異常検知システム及び異常検知プログラム |
JP6653929B1 (ja) * | 2019-07-18 | 2020-02-26 | Jeインターナショナル株式会社 | 自動判別処理装置、自動判別処理方法、検査システム、プログラム、および記録媒体 |
JP2021089219A (ja) * | 2019-12-05 | 2021-06-10 | 東洋製罐グループホールディングス株式会社 | 画像検査システム、及び画像検査方法 |
KR20220146666A (ko) * | 2020-04-27 | 2022-11-01 | 미쓰비시덴키 가부시키가이샤 | 화상 검사 장치 및 화상 검사 방법 |
KR102437115B1 (ko) * | 2020-06-04 | 2022-08-29 | 한국생산기술연구원 | 제품 구조 예측 기술을 이용한 딥러닝 기반 결함 검사 장치 및 방법 |
TWI803353B (zh) * | 2022-04-19 | 2023-05-21 | 南亞科技股份有限公司 | 晶圓檢測方法 |
JP7386287B1 (ja) | 2022-05-09 | 2023-11-24 | Jfeテクノリサーチ株式会社 | 検査方法及び検査装置 |
CN114782445B (zh) * | 2022-06-22 | 2022-10-11 | 深圳思谋信息科技有限公司 | 对象缺陷检测方法、装置、计算机设备和存储介质 |
TWI819698B (zh) * | 2022-07-14 | 2023-10-21 | 友達光電股份有限公司 | 判斷瑕疵的方法及電子裝置 |
CN117315368B (zh) * | 2023-10-23 | 2024-04-23 | 龙坤(无锡)智慧科技有限公司 | 一种大规模数据中心智能运维巡检方法 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH06281592A (ja) * | 1993-03-29 | 1994-10-07 | Sumitomo Metal Ind Ltd | 表面検査方法 |
JP2000346627A (ja) * | 1999-06-07 | 2000-12-15 | Toray Eng Co Ltd | 検査装置 |
JP2006322766A (ja) * | 2005-05-18 | 2006-11-30 | Hitachi High-Technologies Corp | パターン欠陥検査装置および方法 |
JP2009032273A (ja) * | 2008-08-29 | 2009-02-12 | Olympus Corp | 分類装置 |
JP2011138388A (ja) * | 2009-12-28 | 2011-07-14 | Canon Inc | データ補正装置及び方法 |
JP2016091359A (ja) * | 2014-11-06 | 2016-05-23 | 株式会社リコー | 情報処理システム、情報処理装置、情報処理方法、及びプログラム |
Family Cites Families (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA1209693A (en) * | 1983-06-02 | 1986-08-12 | Craig C. Todd | Helical scan recording and playback systems |
JPH0695728B2 (ja) * | 1984-05-29 | 1994-11-24 | 株式会社リコー | 画像データ圧縮再生装置 |
JPS60251771A (ja) * | 1984-05-29 | 1985-12-12 | Ricoh Co Ltd | 画像デ−タ圧縮再生方法及び装置 |
CA1288516C (en) * | 1987-07-31 | 1991-09-03 | Leendert M. Bijnagte | Apparatus and method for communicating textual and image information between a host computer and a remote display terminal |
CN1062963C (zh) * | 1990-04-12 | 2001-03-07 | 多尔拜实验特许公司 | 用于产生高质量声音信号的解码器和编码器 |
AUPN002594A0 (en) * | 1994-12-13 | 1995-01-12 | Dunn, Alexander Atkinson | Improvements in software for the protection of executable programs, libraries and their data |
CA2595719A1 (en) * | 1997-07-15 | 1999-01-28 | Silverbrook Research Pty. Limited | Image transformation means including user interface |
EP1296351A4 (en) * | 2000-06-27 | 2009-09-23 | Ebara Corp | INVESTIGATION DEVICE FOR LOADED PARTICLE RAYS AND METHOD FOR PRODUCING A COMPONENT ELEVATED WITH THIS INSPECTION DEVICE |
KR100425447B1 (ko) * | 2001-05-10 | 2004-03-30 | 삼성전자주식회사 | 명도 보정 및 선택적 결함 검출 방법 및 이를 기록한 기록매체 |
JP3904419B2 (ja) * | 2001-09-13 | 2007-04-11 | 株式会社日立製作所 | 検査装置および検査システム |
EP1451561A2 (en) * | 2001-11-28 | 2004-09-01 | Applied Materials, Inc. | Method for detecting defects in substrates |
JP2005156516A (ja) | 2003-11-05 | 2005-06-16 | Hitachi Ltd | パターン欠陥検査方法及びその装置 |
JP4550559B2 (ja) | 2004-11-24 | 2010-09-22 | シャープ株式会社 | 外観検査装置、外観検査方法およびコンピュータを外観検査装置として機能させるためのプログラム |
US7570796B2 (en) * | 2005-11-18 | 2009-08-04 | Kla-Tencor Technologies Corp. | Methods and systems for utilizing design data in combination with inspection data |
CN101796399A (zh) * | 2007-09-05 | 2010-08-04 | 株式会社尼康 | 观察装置和观察方法以及检查装置和检查方法 |
JP5301955B2 (ja) * | 2008-11-13 | 2013-09-25 | オリンパス株式会社 | 欠陥修正装置 |
JP5247480B2 (ja) * | 2009-01-13 | 2013-07-24 | キヤノン株式会社 | オブジェクト識別装置及びオブジェクト識別方法 |
JP4742193B2 (ja) * | 2009-04-28 | 2011-08-10 | Necソフト株式会社 | 年齢推定装置、年齢推定方法及びプログラム |
JP5506274B2 (ja) * | 2009-07-31 | 2014-05-28 | 富士フイルム株式会社 | 画像処理装置及び方法、データ処理装置及び方法、並びにプログラム |
JP5695924B2 (ja) * | 2010-02-01 | 2015-04-08 | 株式会社ニューフレアテクノロジー | 欠陥推定装置および欠陥推定方法並びに検査装置および検査方法 |
JP5804834B2 (ja) * | 2011-08-03 | 2015-11-04 | 株式会社オービット | 検査プログラム、同プログラムを格納した記録媒体及び検査装置 |
DE102012101242A1 (de) * | 2012-02-16 | 2013-08-22 | Hseb Dresden Gmbh | Inspektionsverfahren |
CN102881029A (zh) | 2012-07-19 | 2013-01-16 | 中国人民解放军第二炮兵工程大学 | 一种热波图像序列的压缩和重构方法 |
JP5944850B2 (ja) * | 2013-03-11 | 2016-07-05 | 株式会社日立ハイテクノロジーズ | 欠陥検査方法及びこれを用いた装置 |
DE112013004507T5 (de) * | 2013-03-28 | 2015-12-31 | Fujifilm Corporation | Bildverarbeitungsvorrichtung, Bilderfassungsvorrichtung, Bildverarbeitungsverfahren, Programm und Aufzeichnungsmedium |
JP6389651B2 (ja) * | 2013-09-10 | 2018-09-12 | Juki株式会社 | 検査方法、実装方法、及び実装装置 |
US9710728B2 (en) * | 2014-10-28 | 2017-07-18 | Kla-Tencor Corporation | Image based signal response metrology |
JP2016139467A (ja) * | 2015-01-26 | 2016-08-04 | 株式会社日立ハイテクノロジーズ | 試料観察方法および試料観察装置 |
US10012599B2 (en) * | 2015-04-03 | 2018-07-03 | Kla-Tencor Corp. | Optical die to database inspection |
TWI571701B (zh) * | 2015-04-30 | 2017-02-21 | 力晶科技股份有限公司 | 偵測微影熱點的方法 |
CN105426902B (zh) * | 2015-09-16 | 2019-06-25 | 西安印钞有限公司 | 基于形变复原技术的钞券质量检测系统及方法 |
US10416087B2 (en) * | 2016-01-01 | 2019-09-17 | Kla-Tencor Corporation | Systems and methods for defect detection using image reconstruction |
US9915625B2 (en) * | 2016-01-04 | 2018-03-13 | Kla-Tencor Corp. | Optical die to database inspection |
US10346740B2 (en) * | 2016-06-01 | 2019-07-09 | Kla-Tencor Corp. | Systems and methods incorporating a neural network and a forward physical model for semiconductor applications |
CN106663316A (zh) * | 2016-08-30 | 2017-05-10 | 深圳大学 | 一种基于块稀疏压缩感知的红外图像重构方法及其系统 |
US10031997B1 (en) * | 2016-11-29 | 2018-07-24 | Taiwan Semiconductor Manufacturing Co., Ltd. | Forecasting wafer defects using frequency domain analysis |
-
2016
- 2016-12-06 WO PCT/JP2016/086138 patent/WO2018105028A1/ja active Application Filing
- 2016-12-06 JP JP2017523001A patent/JP6241576B1/ja active Active
- 2016-12-06 US US16/343,306 patent/US11645744B2/en active Active
- 2016-12-06 KR KR1020197015539A patent/KR102011628B1/ko active IP Right Grant
- 2016-12-06 DE DE112016007498.0T patent/DE112016007498B4/de active Active
- 2016-12-06 CN CN201680091308.9A patent/CN110036279B/zh active Active
- 2016-12-27 TW TW105143296A patent/TWI638157B/zh active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH06281592A (ja) * | 1993-03-29 | 1994-10-07 | Sumitomo Metal Ind Ltd | 表面検査方法 |
JP2000346627A (ja) * | 1999-06-07 | 2000-12-15 | Toray Eng Co Ltd | 検査装置 |
JP2006322766A (ja) * | 2005-05-18 | 2006-11-30 | Hitachi High-Technologies Corp | パターン欠陥検査装置および方法 |
JP2009032273A (ja) * | 2008-08-29 | 2009-02-12 | Olympus Corp | 分類装置 |
JP2011138388A (ja) * | 2009-12-28 | 2011-07-14 | Canon Inc | データ補正装置及び方法 |
JP2016091359A (ja) * | 2014-11-06 | 2016-05-23 | 株式会社リコー | 情報処理システム、情報処理装置、情報処理方法、及びプログラム |
Cited By (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2018211891A1 (ja) * | 2017-05-18 | 2018-11-22 | 住友電気工業株式会社 | 異変検出装置及び異変検出方法 |
KR20200026784A (ko) * | 2018-08-31 | 2020-03-11 | 도시바 미쓰비시덴키 산교시스템 가부시키가이샤 | 제조 프로세스 감시 장치 |
KR102194352B1 (ko) | 2018-08-31 | 2020-12-23 | 도시바 미쓰비시덴키 산교시스템 가부시키가이샤 | 제조 프로세스 감시 장치 |
KR102647255B1 (ko) | 2018-11-28 | 2024-03-12 | 에이조 가부시키가이샤 | 정보 처리 방법 및 기록 매체에 기록된 컴퓨터 프로그램 |
JPWO2020110629A1 (ja) * | 2018-11-28 | 2021-10-21 | Eizo株式会社 | 情報処理方法及びコンピュータプログラム |
WO2020110629A1 (ja) * | 2018-11-28 | 2020-06-04 | Eizo株式会社 | 情報処理方法及びコンピュータプログラム |
US11776114B2 (en) | 2018-11-28 | 2023-10-03 | Eizo Corporation | Information processing method and computer program |
JP7008843B2 (ja) | 2018-11-28 | 2022-01-25 | Eizo株式会社 | 情報処理方法及びコンピュータプログラム |
WO2020110224A1 (ja) * | 2018-11-28 | 2020-06-04 | Eizo株式会社 | 情報処理方法及びコンピュータプログラム |
KR20210084576A (ko) * | 2018-11-28 | 2021-07-07 | 에이조 가부시키가이샤 | 정보 처리 방법 및 기록 매체에 기록된 컴퓨터 프로그램 |
TWI735967B (zh) * | 2018-11-28 | 2021-08-11 | 日商Eizo股份有限公司 | 信息處理方法及計算機程序 |
JP2020139905A (ja) * | 2019-03-01 | 2020-09-03 | セイコーエプソン株式会社 | 検査装置、検査方法、およびプログラム |
US11562479B2 (en) | 2019-03-01 | 2023-01-24 | Seiko Epson Corporation | Inspection apparatus, inspection method, and non-volatile storage medium |
JP2020160616A (ja) * | 2019-03-25 | 2020-10-01 | ブラザー工業株式会社 | 生成装置、コンピュータプログラム、生成方法 |
WO2020202352A1 (ja) * | 2019-03-29 | 2020-10-08 | 株式会社ワコム | ペン状態検出回路及びペン状態検出方法 |
US11604521B2 (en) | 2019-03-29 | 2023-03-14 | Wacom Co., Ltd. | Pen state detection circuit, system, and method |
WO2020202734A1 (ja) * | 2019-03-29 | 2020-10-08 | 株式会社ワコム | ペン状態検出回路、システム及び方法 |
US11841999B2 (en) | 2019-03-29 | 2023-12-12 | Wacom Co., Ltd. | Pen state detection circuit, system, and method |
JPWO2020202734A1 (ja) * | 2019-03-29 | 2021-11-25 | 株式会社ワコム | ペン状態検出回路、システム及び方法 |
JP7277567B2 (ja) | 2019-03-29 | 2023-05-19 | 株式会社ワコム | ペン状態検出回路、システム及び方法 |
JP2020188839A (ja) * | 2019-05-20 | 2020-11-26 | 株式会社アルム | 画像処理装置、画像処理システム、および画像処理プログラム |
JP2020190791A (ja) * | 2019-05-20 | 2020-11-26 | 株式会社アルム | 画像処理装置、画像処理システム、および画像処理プログラム |
CN112082653A (zh) * | 2019-06-12 | 2020-12-15 | 株式会社日立制作所 | 自动检查系统以及无线分机 |
JP7270842B2 (ja) | 2020-04-28 | 2023-05-10 | 三菱電機株式会社 | 外観検査装置および外観検査方法 |
JPWO2021220437A1 (ja) * | 2020-04-28 | 2021-11-04 | ||
WO2021220437A1 (ja) * | 2020-04-28 | 2021-11-04 | 三菱電機株式会社 | 外観検査装置および外観検査方法 |
JP2021176070A (ja) * | 2020-05-02 | 2021-11-04 | Arithmer Inc | 画像検査装置、画像検査方法及びプログラム |
JP6989860B2 (ja) | 2020-05-02 | 2022-01-12 | Arithmer株式会社 | 画像検査装置、画像検査方法及びプログラム |
WO2022201968A1 (ja) | 2021-03-26 | 2022-09-29 | コニカミノルタ株式会社 | 情報処理装置、制御プログラム、および制御方法 |
WO2022264824A1 (ja) * | 2021-06-16 | 2022-12-22 | 株式会社日立製作所 | 異常判定用計算機及び異常判定方法 |
Also Published As
Publication number | Publication date |
---|---|
KR102011628B1 (ko) | 2019-08-16 |
US11645744B2 (en) | 2023-05-09 |
CN110036279A (zh) | 2019-07-19 |
DE112016007498B4 (de) | 2020-11-26 |
JPWO2018105028A1 (ja) | 2018-12-06 |
TWI638157B (zh) | 2018-10-11 |
DE112016007498T5 (de) | 2019-08-29 |
JP6241576B1 (ja) | 2017-12-06 |
KR20190065457A (ko) | 2019-06-11 |
US20210295485A1 (en) | 2021-09-23 |
TW201821788A (zh) | 2018-06-16 |
CN110036279B (zh) | 2022-03-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6241576B1 (ja) | 検査装置及び検査方法 | |
JP6547275B2 (ja) | 情報処理システム、情報処理装置、情報処理方法、及びプログラム | |
US20150363634A1 (en) | Face Hallucination Using Convolutional Neural Networks | |
KR101786754B1 (ko) | 나이 추정 장치 및 방법 | |
US8421877B2 (en) | Feature data compression device, multi-directional face detection system and detection method thereof | |
JP6347589B2 (ja) | 情報処理装置、情報処理方法及びプログラム | |
JP2011227704A (ja) | 情報処理装置および方法ならびにプログラム | |
JP5936561B2 (ja) | 画像における外観及びコンテキストに基づく物体分類 | |
US9940691B2 (en) | Information processing apparatus, control method of the same, and video camera | |
CN110879982A (zh) | 一种人群计数系统及方法 | |
CN112036381B (zh) | 视觉跟踪方法、视频监控方法及终端设备 | |
Giboulot et al. | Steganalysis into the wild: How to define a source? | |
US20220414827A1 (en) | Training apparatus, training method, and medium | |
JP5146797B2 (ja) | パターン欠陥検査のための装置、その方法及びそのプログラムを記録したコンピュータ読取り可能な記録媒体 | |
CN114419406A (zh) | 图像变化检测方法、训练方法、装置和计算机设备 | |
JP7046768B2 (ja) | 情報推定装置及び情報推定方法 | |
CN113139963A (zh) | 缺陷检测方法和装置 | |
WO2022194344A1 (en) | Learnable augmentation space for dense generative adversarial networks | |
JP2012243285A (ja) | 特徴点位置決定装置、特徴点位置決定方法及びプログラム | |
JP2004054442A (ja) | 顔検出装置、顔検出方法および顔検出プログラム | |
CN115272229A (zh) | 一种类别不平衡条件下的异常视觉图像检测方法及装置 | |
Mandal et al. | Super-resolving a single intensity/range image via non-local means and sparse representation | |
JP6147003B2 (ja) | 情報処理装置、情報処理方法及びプログラム | |
Sadjadi | Measures of effectiveness and their use in comparative image fusion analysis | |
WO2016051707A1 (ja) | 情報処理装置、情報処理方法、及び、記録媒体 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2017523001 Country of ref document: JP Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16923132 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20197015539 Country of ref document: KR Kind code of ref document: A |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16923132 Country of ref document: EP Kind code of ref document: A1 |