JPH0410173A - Picture monitoring method - Google Patents

Abstract

PURPOSE:To enable monitoring by the ambigurous and abstract discrimination basis of the inspection by an outward appearance, etc., by dividing a monitoring area set in the inside of the image pickup picture of an image pickup means into partial areas while overlapping important areas. CONSTITUTION:The quality of the overlapped area is discriminated by a majority among the results of the respectively discriminated result partial areas L1, L2 and L3. Thus, for example, when the position of an oblong hole 11 is deviated and gets out of the area L1, the probability of the 'good' discrimination is high to be done even when the probability of the over-error judgement is high to be done by the influence of noise through the oblong hole 11 is in the area 11. For example, when the oblong hole 11 is deviated upward considerably to be discriminated 'not good' in the area L1 and L3, the upward deviation of the position of the hole is discriminated by the result of the discrimination.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is useful for monitoring whether IC elements, etc. are correctly mounted on a flat substrate, whether holes are correctly opened, etc., in the manufacturing and assembly process, for example. This invention relates to an image monitoring method.

[Prior Art] Conventionally, quality determination of length, pattern, etc., i.e., determination of whether the length of the inspected object (object to be monitored) is as specified or whether the shape of the inspected object is as specified, has conventionally been carried out using the is imaged with a TV camera, etc., and the characteristic area where the length and shape appear is specified in a window, the inspection area is narrowly specified, and the number of pixels in that area where the above length and shape appear is calculated. A physical quantity is measured with high precision by counting, etc., and quality determination is performed based on whether the number of pixels is within a permissible value or not.

By the way, in the appearance inspection after processing in the manufacturing and assembly process, etc., for example, in drilling, monitoring whether the hole is correctly drilled after processing in the process, etc. is not important, but the accuracy of the hole position is not important. In many cases, it is sufficient if the door is open. In addition, in the appearance inspection before processing in the manufacturing and assembly process, etc., there may be cases where it is desired to vaguely monitor whether there is large dust or the like attached to the processing surface.

[Problem to be solved] Therefore, in this type of visual inspection, it is not necessary to measure physical quantities with high precision as described above, and because it is desired to inspect the external appearance of the object to be inspected as a whole, it is necessary to measure the physical quantity in a narrow range as described above. Even if you specify and inspect only that part, it is not possible to monitor the entire object to be inspected. On the other hand, if the entire object to be inspected is inspected as one inspection area, for example, there may be dust attached to a certain part, or there may be scratches in another part, and these will appear as different signs at the inspection stage. Therefore, these factors cancel each other out, and there are cases where the object is erroneously determined as normal.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an image monitoring method that can perform monitoring based on ambiguous abstract judgment criteria such as visual inspection, and even in this case, important feature areas can be weighted and monitored. There is a particular thing.

[Means for Solving the Problems] In order to achieve the above object, the present invention captures an image of the monitored object using an imaging means and compares it with a predetermined reference image to determine the quality of the monitored object. In an image monitoring method that performs The results of each pass/fail judgment in the area are compared, and if a difference in judgment is detected, the pass/fail judgment of the overlapping area is performed by a majority vote of the results of each pass/fail judgment in the overlapping partial areas.

[Example] Hereinafter, an example of the present invention will be described in detail based on the drawings.

First, the overall configuration of the image monitoring device will be explained. Third
In the figure, a monitoring target 1 is in the process of drilling a hole in a manufacturing line or the like (not shown), and a television camera 3 images the surface on which the hole is being drilled under illumination 2 . Imaging is performed for both a preliminary image before the hole is drilled and a post-hole image after the hole is drilled.

Input control signals for completion of machining preparation and completion of machining in the processing machine for hole drilling are inputted to the main body 7 of the apparatus through an input line 5. Operating conditions and the like of the device main body 7 are inputted using the keypad 8. Every time an input control signal is input through the input line 5, the device body 7 executes preliminary monitoring or post-monitoring, and outputs the results as an output control signal through the output line 6. The device main body 7 includes a CRT capable of displaying images captured by the television camera 3 and image signals obtained by binary or multi-value encoding of the images.
7a is the setting.

Next, the principle of the present invention will be explained using FIG. 1, which is a first embodiment of a partial area setting method.

In Figures 1(b) to (e), the entire screen of the television camera 3 (horizontal pixel number LH, vertical pixel number LV) is taken as the monitoring area S, and each partial area for post-monitoring is given in units of scanning lines. . By machining, the monitoring area is divided into four equal parts, and a long hole 11 is opened in the horizontal direction in the second region from the top of the two equal parts.See figure (a), before machining. The monitoring area S is in a planar state), and as shown in Figure (b), the partial area Ll is the second area from the top of the above-mentioned equally divided four parts that covers the elongated hole 11, and as shown in Figure (C). The partial area L2 shown in FIG.
3 is an area that covers the second and third areas from the top of the above-mentioned equally divided four areas, and further covers the part slightly above the upper end of Ll, and the partial area L4 shown in FIG. This is the fourth region from the top of the four divisions.

The part where the elongated hole 11 is made is an important feature area, and is more important than monitoring whether there is dirt or not.
Therefore, the three characteristic regions LI, L2, and L3 are inspected redundantly.

The quality of the overlapping area is determined by Ll, L2. This is done by majority vote of the determination results of each L3. As a result, for example, if the position of the elongated hole 11 shifts and deviates from the area L1, even if the elongated hole 11 is within the area L1, but the rate of incorrect determination is high due to the influence of noise, etc. , the probability of a good judgment being made is increased.

Also, for example, if the long hole 11 is largely rubbed upward, the area Ll
If a defective determination is made in region L3, it is determined based on the determination result that the hole position has shifted upward. Furthermore,
If the position of the long hole 11 does not shift, but there is a change in the region Ll, such as when only half of the hole in the long hole 11 is opened, the change can be expressed in the three regions Ll, L2, and L3. It will be done.

It should be noted that before processing, the monitoring area S is in a flat state and there are no particularly important feature areas, and in advance monitoring,
The partial area is divided into four equal parts as described above, and each part is divided into four parts,
It is assumed that partial areas do not overlap.

Next, the overall block configuration of the image monitoring device will be explained using FIG. 4.

A program for operating the CPU 22 is stored in the ROM 21 .

The RAM 23 stores pre and post reference image signals obtained by capturing images of the pre-normal state and post-normal state of the monitored object 1 with the television camera 3 and performing binary or multi-value processing, and the positions of partial areas in pre- and post-monitoring.

Data such as allowable values for determining the quality of each partial area and positions of overlapping partial areas in post-monitoring are stored. Conditions such as the positions of partial areas in pre- and post-monitoring, tolerance values for determining the quality of each partial area, positions of overlapping partial areas in post-monitoring, timing, etc. can be determined using the keypad while looking at the CRT7a, etc. Input from 8 etc. and R via I/O port 29
It is stored in AM23.

Based on the commands in the ROM 21 and the data in the RAM 23, the CPU 22 performs quality determination of each partial area in pre- and post-monitoring, majority decision on overlapping areas in post-monitoring, and the like.

The image signal captured by the television camera 3 is sent to the A/D conversion circuit 3
0 and stored in the frame memory 31, and the contents of the frame memory 31 are displayed on the CRT 7a via the display circuit 32. Note that two frame memories 31 are prepared, each containing a preliminary image.

After the fact, the image is memorized.

Next, the operational flow of this method will be explained using FIG. 5.

First, an operator displays the captured object 1 on a CRT.
7a, set pre- and post-monitoring conditions, such as setting a partial area in a window using the keypad 8 (102). Note that the partial areas in the prior image and the partial areas in the subsequent image do not have to be the same; in this example, the partial areas in the prior image are divided into four equal parts as described above, and there is no overlapping area. .

Next, an unprocessed image of the normal monitoring object is taken with the television camera 3 and the image is displayed with the CRT 7a (103), the feature values of each partial area are calculated, and these quality judgment reference values are used. and set tolerance values, etc. (104)
.

Next, normal hole processing is performed on the above-mentioned normal monitoring object, the image after this processing is captured by the television camera 3, and the same operation as above is performed in the partial area in the post-monitoring (103, 104). .

Next, the process moves to monitoring, and when a processing preparation completion signal is received via the input line 5 (105), an image of the surface of the object 1 to be drilled is imaged, and a pre-inspection image is input (106).
, compares the feature amount with the reference image stored in the RAM 23 for each partial area, and determines whether it is good or bad depending on whether it is within the tolerance value, and when all partial areas are judged as good, the monitored object Assuming that the preliminary monitoring result of 1 is good (107)
, the signal is transmitted to the processing machine etc. through the output line 6 (10
8) Hole drilling is performed. If it is determined that there is an abnormality, the retry or hole drilling will be canceled.

If processing is performed because the preliminary monitoring results are good,
After receiving a signal indicating the end of processing (109), the processed image is input (110).

The details of subsequent flows 111 and 112 are shown in FIG. 6, and will be explained along this flow diagram. First, the feature amount of each partial region is extracted (204). Note that the feature amount is calculated by taking a difference image between the prior image and the subsequent image. As a result, only the parts that have changed between before and after processing are extracted.For example, small particles that are attached or do not affect the quality judgment are not extracted because they are still attached after processing, and are not extracted during post-monitoring. It is possible to eliminate the influence on pass/fail judgment. Next, in each partial region, the feature amount consisting of this difference image is compared with the data of the reference post-image stored in the RAM 23 to determine whether or not it is within the tolerance value, and a pass/fail judgment is made (205). , this result is temporarily stored in the RAM 23.

When the quality judgment of all partial areas is completed (206),
The CPU 22 verifies whether or not the overlapping areas are determined to be the same, using the data temporarily stored in the RAM 23 (
209). If all of the partial areas that make up the overlapping area do not have the same determination (210), a majority vote is taken. If it is determined to be good by majority vote, the overlapping portion is judged to be good (211). A majority decision is made for all overlapping areas (212), and even if all partial areas are judged to be good or there is a partial area that is judged to be bad, that part overlaps with other partial areas, and If the majority decision determines that the overlapping portion is good, the post-monitoring result is determined to be good, and the final result is output (213.112).

Furthermore, if a final result of an abnormality is obtained by majority decision, if possible, only the area where the decision was incorrect is re-examined and the final result is determined.

In this embodiment, partial regions are set by overlapping important feature regions, and if the judgment results of each partial region are not the same, the decision is made by majority vote. Therefore, for example, slight positional deviation of the feature regions is acceptable. In cases where only the presence of a feature region is important, the correct decision is made by majority decision. This can also be achieved by increasing the area of a partial region in which the characteristic region exists, and determining whether the characteristic region is good or bad using this partial region. That is, in this case, since the area of the partial region is large, even if the characteristic region is misaligned, the characteristic region can still exist within this partial region, and a correct determination can be made. In addition, the wider the area, the greater the influence of noise, etc., so if the probability of error in the judgment results increases, inspect the feature region or existing partial region multiple times, and determine whether it is acceptable or not by majority vote. It is also possible to perform discrimination to obtain more accurate results. For example, in the first embodiment shown in FIG. 1, the upper end of the partial region including the feature region is made the same as the upper end of the feature region L3, and the lower end is made the same as the lower end of the feature region L2. Since this increases the area of the partial area, this partial area is inspected, for example, three times.
It is also conceivable to judge whether this partial area is good or bad based on the majority vote. However, according to the experimental results, when the feature regions are overlapped and different partial regions are set and the decision is made by majority vote as in the present invention, the percentage of incorrect decisions is The result was that the rate of incorrect determinations was lower than that when only the partial area was inspected multiple times and the determination was made by majority vote. This is probably because, in the present invention, the overlapping partial areas are displaced from each other, so noise is dispersed and the rate of erroneous determination is reduced.

Note that in the above embodiment, the area L2. For L3, a predetermined amount of extension is provided in the first and second, second and third areas of the four equal divisions, respectively, or areas L2L3 are provided as the first and second, second and third areas of the four equal divisions, respectively. often,
In this case, erroneous determination due to the influence of noise can be further prevented by determining whether the misalignment cannot be covered or by making a majority decision in this case as well.

In addition, in the above embodiment, the partial area was set in units of scanning lines, but various methods of setting the partial area can be considered.
As an example, as shown in FIG. 2(a), the monitoring area S
When divided into four equal rectangles, if there is a characteristic region consisting of a round hole 16 in the upper left part, the partial region L21 is divided into four equal parts, as shown in (b) to (e) of the same figure. The partial area L22 includes the upper left part and the lower left part of the four equal parts, and the area is slightly expanded to the right of these parts, and the partial area L23 includes the upper left part and the upper right part of the above four equal parts. The partial area L24 is a region that is slightly expanded to the lower side of the area L22 and the right end line of the area L22.
It may also be an area surrounded by the lower end line of 24. Furthermore, when the monitoring area S is divided into four equal rectangles, if there is a feature region consisting of the round hole 1B in the upper left part, the partial area L21 is the upper left part of the above four equal parts, and the partial area L22 is the four equal parts. The upper left part and the lower left part of the division may be the upper left part and the lower left part, the partial area L23 may be the upper left part and the upper right part of the above-mentioned quarter division, and the partial area L24 may be the lower right part of the above-mentioned quarter division. Note that the shape of the partial region is expressed by the number 9, the number of overlapping regions.

The number, arrangement, etc. of partial regions in the overlapping region can be freely selected, and these depend on the shape of the feature region included in the object to be monitored and the importance of monitoring that feature region.

Further, it is determined based on the degree of disturbance (noise) mixed in when image processing the monitored object. For particularly important regions, the size of the partial region is made smaller to make it easier to quantify and to increase the degree of overlap.

Further, in the above embodiment, the post-monitoring process was performed using the difference image between the pre-image and the post-image, but it is also possible to perform the post-monitoring process by comparing the post-image with a reference post-image.

Furthermore, although the present invention is not applied to advance monitoring in the above embodiments, the present invention can also be applied to advance monitoring when a characteristic region exists in the advance image.

Further, in the embodiment described above, the entire screen of the television camera 3 is used as the monitoring area, but a part of the screen may be surrounded by a window or the like to be used as the monitoring area, and the present invention may be applied to this part.

In addition, in the present invention, the process of calculating the feature amount in each partial region and determining the quality of each partial region 1 When there are multiple overlapping regions, the processing can be performed by simultaneously performing majority judgment at the multiple locations, etc. It is possible to increase the speed.

[Effects] According to the present invention, monitoring can be performed using ambiguous abstract criteria such as visual inspection, and even in this case, important feature regions can be weighted and monitored.

[Brief explanation of the drawing]

FIG. 1 shows the region dividing method of the first embodiment, and FIG. 1(a) is a plan view showing how partial regions overlap on the screen of the imaging means, and FIG. 1(b) to (d) 2 is a plan view showing the position of the partial areas, FIG. 2 is a plan view showing the area dividing method of the second embodiment, and FIG. 2(a) is a plan view showing how the partial areas overlap on the screen of the imaging means , same figure (b) ~
(d) is a plan view showing the partial area Saga, FIG. 3 is an external perspective view showing how an object to be monitored is monitored by the image monitoring device, FIG. 4 is a system block diagram of the image monitoring device, and FIG.
The figure is a system flow diagram of the entire image monitoring apparatus, and FIG. 6 is a specific flow diagram of the monitoring process in the system flow diagram of FIG. 5. 1...Monitored object, 3...TV camera, Ll-L4, L21-L24...partial area, S.
...Monitoring area. that's all

Claims (1)

[Scope of Claims] An image monitoring method for determining the quality of the monitored object by capturing an image of the monitored object using an imaging means and comparing it with a predetermined reference image, comprising: Divide the monitoring area set in , into partial areas by overlapping important areas, perform a pass/fail judgment for each partial area, compare the results of each pass/fail judgment in the overlapping partial areas, and check for incorrect judgments. An image monitoring method characterized in that, when the overlapping partial area is detected, the overlapping area is judged to be good or bad based on a majority vote of the results of each pass/fail judgment in the overlapping partial area.