JPH05149728A - Visually recognizing device - Google Patents

Abstract

PURPOSE:To improve the efficiency of teaching work and recognizing accuracy by artificially editing the picture of a defective sample from a single sample picture upon starting the teaching work. CONSTITUTION:A picture processor 3 has a feature extracting section 5 and learning discriminating section 6. The section 5 extracts a feature amount by inputting picture signals from a TV camera 2 and generates a feature data string. The section 6 inputs the feature data string and performs a recognizing process by using a discriminating parameter generated in advance from a teaching picture. The picture processor 3 has picture memories 20 and 21 which respectively store sample and teaching pictures and, upon performing teaching work, a CPU 24 edits a teaching picture from the sample pictures based on picture editing information inputted from an input device 25 and draws the edited picture in the memory 21. The above-mentioned discriminating parameter is generated by using the edited teaching picture.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual recognition device used for inspecting the appearance of a product in a manufacturing or inspection line such as a factory, and in particular, the present invention images a recognition object. For the input image obtained by
The present invention relates to a visual recognition device that performs recognition processing using a determination parameter generated in advance from an image for teaching and determines whether a recognition target is good or bad.

[0002]

2. Description of the Related Art As shown in FIG. 6, a conventional visual recognition device of that type is a television camera 2 which picks up an image of an object 1 to be recognized and generates an image signal, and a predetermined recognition process which takes in the image signal. And an image processing device 3 that executes the image. The image processing apparatus 3 includes an A / D converter 4, a feature extraction unit 5, a learning determination unit 6, and an output unit 7. The A / D converter 4 converts an analog image signal from the television camera 2 into a digital amount. Image signal.

The feature extraction unit 5 extracts the feature amount of the input image and, for example, a feature data string Di including nine types of feature amounts.
(However, i = 1, 2, ..., 9). Here, the feature amount is an area, a density distribution, a total density, a perimeter, etc., and an optimum feature amount is selected according to the recognition target object 1. The learning determination unit 6 inputs the characteristic data string Di, performs a recognition process using a determination parameter generated in advance from an image for teaching, determines the quality of the recognition target object 1, and outputs the determination result. Output from section 7.

FIG. 7 shows an example of the structure of the learning judging section 6. In the figure, Ci, Si, kni, kgi are the determination parameters, where Ci is the feature amount element average, Si is the feature amount element variance, kni is the abnormal coefficient, and kgi is the normal coefficient. The learning determination unit 6 in the illustrated example includes an intermediate data calculation unit 8, a multiplication unit 9, an addition unit 10, and a determination unit 11. The intermediate data calculation unit 8 includes nine arithmetic units 12, and each arithmetic unit 12 has corresponding data of the characteristic data sequence Di, the characteristic amount element distribution Ci, and the characteristic amount element distribution Si (for example, the first arithmetic unit 1).
2 is input with D0, C0, S0), and the intermediate data Mi is calculated by the following equation.

[0005]

[Equation 1]

The multiplying unit 9 receives nine intermediate multipliers Mi output from each arithmetic unit 12 and the abnormal coefficient kni and multiplies them by nine first multipliers 13, each intermediate data Mi and the normal coefficient. and 9 second multipliers 14 for inputting and multiplying kgi. The adder 10 includes a first adder 15 for obtaining a sum of multiplication outputs of the first multipliers 13,
A second adder 16 for obtaining the sum of the multiplication outputs of the respective second multipliers 14.

The determination section 11 includes first and second comparators 17 and 18 for determining whether or not the addition output of the first and second adders 15 and 16 is zero or more, The first comparator 17 outputs an abnormality determination output when the addition output of the first adder 15 is zero or more, and outputs a normal determination output when the addition output of the second adder 16 is zero or more. Therefore, when the first comparator 17 outputs an abnormality determination output and the second comparator 18 does not output a normal determination output, it is determined that the recognition target object 1 is a defective product, and the first comparator 17 When the second comparator 18 does not output the abnormality determination output and outputs the normal determination output, it is determined that the recognition target object 1 is a good product.
When the first and second comparators 17 and 18 output contradictory judgment outputs, it is concluded that the judgment is impossible.

Each determination parameter Ci, Si, kn
i and kgi are generated by executing the following teaching procedure. First, when the non-defective sample 1a is placed in the observation field of view of the television camera 2 and imaged, this image signal is taken into the image processing device 3 and A / D converted, and the feature extraction unit 5 performs feature extraction to perform feature data. The column Di1 is generated.
This characteristic data string Di1 is fetched and held by the learning judging unit 6, and when the characteristic data string Di2 for the next non-defective sample 1a is fetched by the learning judging unit 6, the calculation unit (not shown) calculates the following equation. Is executed to calculate the feature amount element average Ci and the feature amount element variance Si,
It is set in the register.

[0009]

[Equation 2]

[0010]

[Equation 3]

The learning judging section 6 has a register in which the abnormal coefficient kni and the normal coefficient kgi are set.
At this stage, the contents of these registers are reset.

Hereinafter, the non-defective sample 1a and the defective sample 1b are sequentially picked up to obtain the characteristic data string Di and loaded into the learning determination section 6, and the learning determination section 6 is notified that the non-defective sample 1a is non-defective. Teaching judgment result,
For the defective product sample 1b, the teaching determination result indicating that the defective product is defective is input. If the teaching judgment result indicating that it is a non-defective product is input,
When a teaching determination result indicating that the product is defective is input, the abnormal coefficient kni and the normal coefficient kgi are calculated by the following equations, and are set in the register each time.

[0013]

[Equation 4]

[0014]

[Equation 5]

[0015]

[Equation 6]

[0016]

[Equation 7]

In the above equation, the intermediate data Mi is given by the above equation, and Sig (Mi) is Sig (Mi) =-1 if the intermediate data Mi is negative, and if the intermediate data Mi is positive, It is a function that Sig (Mi) = 1. Each time the abnormal coefficient kni and the normal coefficient kgi are calculated, the feature amount element average Ci and the feature amount element variance Si
Also, the value is updated by executing the calculation of the following equation, and the determination parameters Ci, Si, kni, kgi are optimized by repeating the above teaching procedure.

[0018]

[Equation 8]

[0019]

[Equation 9]

[0020]

However, in the case of the above teaching method, it is complicated because it is necessary to prepare a large number of non-defective samples 1a and defective samples 1b. In particular, it is true that many defective samples 1b are collected. It's difficult. Therefore, the teaching must be performed with a small number of defective sample 1b, and the recognition accuracy is unavoidably deteriorated. There is also a method of artificially manufacturing a large number of defective product samples 1b, but this requires a great deal of labor and time and only increases the burden on the operator for teaching.

The present invention has been made in view of the above problem, and artificially edits an image of a defective sample from a single sample image instead of using a large number of defective samples in teaching. Accordingly, it is an object of the present invention to provide a visual recognition device that realizes the efficiency of teaching work and the improvement of recognition accuracy.

[0022]

According to a first aspect of the present invention, a recognition process is performed on an input image obtained by picking up an image of an object to be recognized, using a determination parameter generated in advance from an image for teaching, and recognition is performed. In a visual recognition device for determining the quality of an object, a sample image storage means for storing a sample image obtained by imaging a sample of the recognition object, an input means for inputting image editing information, and the input Teaching image editing means for editing the teaching image from the sample image stored in the sample image storage means based on the image editing information inputted by the means.

According to a second aspect of the present invention, the input image obtained by picking up an image of the recognition target object is subjected to recognition processing using a judgment parameter generated in advance from an image for teaching, and the quality of the recognition target object is judged. In the visual recognition device, a sample image storage unit for storing a sample image obtained by imaging a sample of a recognition target, and a partial image pattern for storing a plurality of partial image patterns for generating the teaching image Teaching image editing means for editing the teaching image by synthesizing a storage means, one of the partial image patterns stored in the partial image pattern storage means and a sample image stored in the sample image storage means It is equipped with and.

According to the third aspect of the present invention, the input image obtained by picking up the image of the recognition target object is subjected to the recognition process using the judgment parameter generated in advance from the image for teaching, and the quality of the recognition target object is judged. In the visual recognition device, sample image storage means for storing a sample image obtained by imaging a sample of a recognition object, input means for inputting image editing information, and an image input by the input means Partial image pattern editing means for editing a plurality of partial image patterns for generating the teaching image based on editing information, and partial images for storing the plurality of partial image patterns edited by the partial image pattern editing means The pattern storage means, any one of the partial image patterns stored in the partial image pattern storage means, and the sample image storage means stored in the sample image storage means. By synthesizing the pull image, in which a teaching image editing means for editing the image for the teachings.

[0025]

According to the first aspect of the invention, the sample of the object to be recognized is imaged to generate the sample image, and the teaching image for the defective sample is edited from the sample image,
A judgment parameter is generated from the image.

According to the second aspect of the present invention, a defective sample is obtained by picking up a sample of the object to be recognized to generate a sample image, extracting any one of the partial image patterns from the partial image pattern storing means, and synthesizing the sample image with the partial image pattern. After the image for teaching about is created, the determination parameter is generated from the image.

According to the third aspect of the present invention, a plurality of partial image patterns for generating the teaching image are edited in advance and stored in the partial image pattern storage means. After that, a sample image of the recognition object is picked up to generate a sample image, and one of the partial image patterns is taken out from the partial image pattern storage means and is combined with the sample image to form an image for teaching about a defective sample. After the creation, the judgment parameter is generated from the image.

With the above configuration, it is not necessary to collect defective samples when creating the determination parameters, the teaching work is efficiently performed, and the recognition accuracy is improved.

[0029]

FIG. 1 shows a visual recognition device according to a first embodiment of the present invention, which comprises a television camera 2 and an image processing device 3. The image processing device 3 includes an A / D converter 4, a feature extraction unit 5, a learning determination unit 6, and an output unit 7. These configurations are the same as those of the conventional example shown in FIG. Detailed description is omitted.

The image processing apparatus 3 includes two image memories 20 and 21, one of which is a sample image obtained by picking up a non-defective sample 1a during teaching, and the other image memory 21. Each of the teaching images edited from the sample image is stored in.

FIG. 2 (1) shows a sample image 30 stored in the image memory 20, and FIG. 2 (2) shows an edited teaching image 31. In the figure, 32 is the image portion of the object, and 33 is the background image portion.

Returning to FIG. 1, the D / A converter 22 converts the image signal of the teaching image generated on the image memory 21 into an analog amount image signal. The video monitor 23 inputs the image signal and displays an image 31 for teaching.

The CPU 24 is the main body for controlling and calculating the microcomputer, and here constitutes the image editing means together with the image memory 21. The input device 25 is, for example, a mouse or a light pen, and is used to input information such as position data for image editing.

In the visual recognition device having the above-described structure, when the non-defective sample 1a is first placed in the observation field of view of the television camera 2 and imaged, this image signal is taken in by the image processing device 3 and is sent to the feature extraction section 5 as in the conventional case. Feature extraction is performed to generate a feature data string Di1. When the feature data string Di2 for the next non-defective sample 1a (the same non-defective sample may be used) is taken in by the learning determination unit 6, the feature amount element average Ci and the feature amount element variance Si are calculated as in the conventional case. The calculation is executed.

The sample image of the non-defective sample 1a is taken in and stored in the image memory 20, and when the operator inputs the image editing information using the input device 25, C
The PU 24 operates the image memory 2 based on the image editing information.
The teaching image is edited from the sample image stored in 0, and the image is written in the image memory 21. Figure 2 (2)
2 is a teaching image 31 edited from the sample image 30 of FIG. 2 (1). In the figure, 34 is a defective portion artificially generated by operating the input device 25.

When the editing is completed as described above, the teaching image stored in the image memory 21 is read by the feature extraction unit 5, the feature extraction unit 5 generates the feature data string Di and outputs it to the learning determination unit 6. To be done. At this time, the learning determination unit 6
On the other hand, when the instruction determination result indicating that the product is defective is input, the abnormal coefficient kni and the normal coefficient kgi are calculated, and the respective values of the characteristic amount element dispersion Ci and the characteristic amount element dispersion Si are updated as in the conventional case. It Similarly, the judgment parameters Ci, Si, kni, kgi are optimized by editing and teaching various kinds of teaching images, and thereafter, the recognition object 1 is recognized by using the judgment parameters. Transition.

FIG. 3 shows a visual recognition device according to a second embodiment of the present invention, which has a television camera 2 and an image processing device 3. The image processing apparatus 3 of this embodiment is also similar to the first embodiment in that the A / D converter 4, the feature extraction unit 5, the learning determination unit 6, the output unit 7, and the two image memories 20, 21,
It has a D / A converter 22, a video monitor 23, a CPU 24, and an input device 25, and further has a pattern memory 26 and a graphic memory 27 in addition to these components.

The pattern memory 26 stores a plurality of partial image patterns (hereinafter, referred to as "defective patterns") regarding defective portions for generating a teaching image for each defective category.

FIG. 4 shows a plurality of types of defective patterns 40A, 4A.
0B and 40C are shown. Category A has a plurality of defective patterns regarding product scratches, category B has a defective pattern regarding product holes, and category C has a defective pattern regarding product defects. Are included respectively. In the figure, the shaded portion 4 in each defective pattern 40A
1A indicates a scratch, and a shaded portion 4 in each defective pattern 40B.
1B is a hole, and the shaded portion 4 is in each defective pattern 40C.
1C indicates a chip, respectively. Each of these defective patterns 4
0A, 40B, and 40C are stored in the storage area for each category.

In the case of this embodiment, the CPU 24, together with the image memory 21 and the graphic memory 27,
Image editing means for editing the teaching image by synthesizing the sample image and any defective pattern, and partial image pattern editing means for editing the defective pattern and storing it in the pattern memory 26. To do. In addition to the position data for image editing, the input device 25 is used for inputting various information such as the rotation angle of the defective pattern, the magnification, the density and color of the defective portion when the images are combined.

In the second embodiment having the above-mentioned structure, prior to teaching, when the image editing information relating to the defective portion is input using the input device 25, the CPU 24 edits the defective pattern based on the image editing information and writes the pattern memory. 26. By repeating the same operation for each category, the pattern memory 26 stores a plurality of defective patterns for each category.

Next, when creating an image for teaching, when the worker uses the input device 25 to specify a necessary defective pattern in the pattern memory 26, the CPU 24
Reads out the corresponding defective pattern from the pattern memory 26 and transfers it to the graphic memory 27.

On the other hand, the non-defective sample 1a is used as the TV camera 2
The sample image captured by the
The CPU 24 reads the sample image from the image memory 20 and transfers it to the graphic memory 27, and displays a composite image of the defective pattern and the sample image on the video monitor 23.

Next, when the worker inputs information such as the position, rotation angle, magnification, density and color of the defective pattern using the input device 25 while looking at the composite image displayed on the video monitor 23, the CPU 24 inputs the information. Edit the image for teaching by synthesizing the sample image and the defective pattern based on the information,
The image is written in the image memory 21 and displayed on the video monitor 3.

FIG. 5 shows this editing work. In FIG. 5, 30 is a sample image 3 stored in the image memory 20.
0 indicates 40, 40A indicates a defective pattern regarding a scratch read from the pattern memory 26, and 31 indicates an edited teaching image 31. In the figure, 32 is an image portion of the object, and 33 is a background image portion.

[0046]

As described above, according to the present invention, by capturing a sample of a recognition object to generate a sample image and then inputting image editing information, or by using a partial image pattern generated in advance, Since the image for teaching is edited to generate the judgment parameter, it is not necessary to collect the defective product sample, the teaching work can be performed efficiently, and the recognition accuracy is improved. Play.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of a visual recognition device according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a sample image and a teaching image.

FIG. 3 is a block diagram showing a configuration example of a visual recognition device according to a second embodiment of the present invention.

FIG. 4 is an explanatory diagram showing a specific example of a defective pattern stored in a pattern memory.

FIG. 5 is an explanatory diagram showing a process of generating a teaching image by synthesizing a sample image and a defective pattern.

FIG. 6 is a block diagram showing a configuration of a conventional visual recognition device.

FIG. 7 is a block diagram showing a configuration of a learning determination unit.

[Explanation of symbols]

 2 TV camera 3 Image processing device 20, 21 Image memory 24 CPU 25 Input device 26 Pattern memory 27 Graphic memory

Claims (3)

[Claims] 1. A visual recognition device for determining whether a recognition target is good or bad by performing recognition processing on an input image obtained by imaging a recognition target using a determination parameter generated in advance from a teaching image. , A sample image storage means for storing a sample image obtained by imaging a sample of a recognition object, an input means for inputting image editing information, and based on the image editing information input from the input means, A visual recognition device comprising: a teaching image editing means for editing the teaching image from the sample image stored in the sample image storage means. 2. A visual recognition device for performing a recognition process on an input image obtained by picking up an image of a recognition target object by using a judgment parameter generated in advance from an image for teaching, and judging whether the recognition target object is good or bad. A sample image storage means for storing a sample image obtained by imaging a sample of a recognition object; a partial image pattern storage means for storing a plurality of partial image patterns for generating the teaching image; And a teaching image editing means for editing the teaching image by synthesizing any of the partial image patterns stored in the partial image pattern storage means and the sample image stored in the sample image storage means. Visual recognition device. 3. A visual recognition apparatus for performing a recognition process on an input image obtained by picking up an image of a recognition target object by using a determination parameter generated in advance from an image for teaching, and determining whether the recognition target object is good or bad. A sample image storage means for storing a sample image obtained by imaging a sample of a recognition object, an input means for inputting image editing information, and based on the image editing information input from the input means, Partial image pattern editing means for editing a plurality of partial image patterns for generating the image for teaching, partial image pattern storage means for storing a plurality of partial image patterns edited by the partial image pattern editing means, One of the partial image patterns stored in the partial image pattern storage means and the sample image stored in the sample image storage means Form, the visual recognition device comprising a teaching image editing means for editing the image for the teachings.