JPH046404A - Image monitoring method - Google Patents

Abstract

PURPOSE:To make it possible to perform monitoring based on ambiguous, abstract judging references by dividing the monitoring region in a screen wherein images are picked up into a plurality of block regions, and judging the pass or fail for every block region. CONSTITUTION:The image of the surface of a monitored object 1 where a hole is formed is picked up with a TV camera 3. Input control signals for the finish of the machining preparation and the end of the machining in a hole drilling machine are inputted into a main body 7 of an apparatus through an input circuit 5. At every time when the input control signal is inputted through the circuit 5, the main body 7 executes the pre-monitoring and the post-monitoring. The results are outputted through an output circuit 6 as a control signal. When the entire image which is picked up with the camera 3 is made to be the monitored region, the entire image is divided into block regions. When a part of the image is marked and made to be the monitoring region, the marked monitoring region is divided into block regions. The monitoring region is covered with the block regions. The monitoring is performed as follows. The image is compared with a reference image at every block region. The pass or fail is judged based on whether the value in each block region of the image under inspection in each block is within an allowance range or not.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention monitors whether IC elements, etc. are correctly mounted on a flat substrate, whether holes are opened correctly, etc., in the manufacturing and assembly process, etc., for example. The present invention relates to an image monitoring method for.

[Prior Art] To determine the quality of length, pattern, etc., for example, to determine whether the length of the object to be inspected (object to be monitored) is as specified or whether the shape of the object is as specified, Take an image with a camera, etc., specify in a window the characteristic region where the length, shape, etc. appear, specify a narrow inspection area, and count the number of pixels in that area where the above length and shape appear. The physical quantity is measured with high precision using the method, and quality determination is performed based on whether the number of pixels is within an allowable value or not.

By the way, in the appearance inspection after processing in the manufacturing and assembly process, etc., for example, in drilling, monitoring whether the hole is correctly drilled after processing in the process, etc. is not important, but the accuracy of the hole position is not important. In many cases, it is sufficient that the hole be opened near a predetermined position. In addition, in the appearance inspection before processing in the manufacturing and assembly process, etc., there may be cases where it is desired to vaguely monitor whether there is any large dust or the like attached to the processed surface.

[Problem to be solved] Therefore, in this type of visual inspection, it is not necessary to measure physical quantities with high precision as described above, and because it is desired to inspect the external appearance of the object to be inspected as a whole, it is necessary to measure the physical quantity in a narrow range as described above. Even if you specify and inspect only that part, it is not possible to monitor the entire object to be inspected. On the other hand, if the entire object to be inspected is inspected as one inspection area, for example, there may be dust attached to a certain part, or there may be a flaw in another part, and these will appear as different signs at the inspection stage. Therefore, these factors cancel each other out, and there are cases where the object is erroneously determined as normal.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an image monitoring method that allows monitoring based on vague and abstract criteria such as visual inspection.

[Means for Solving the Problems] In order to achieve the above object, the present invention provides an image for determining the quality of the monitored object by comparing an image of the monitored object captured by an imaging means with a reference image. In the monitoring method, the monitoring area in the screen imaged by the imaging means is divided into a plurality of block areas, and a quality determination is made for each block area.

In addition, a permissible range is set for each reference image in each block area, and a pass/fail judgment is made for each block area based on whether the image of the monitored object captured by the imaging means is within the permissible range. It is preferable to do so.

Furthermore, the allowable range of the reference image in each block area is
It is preferable to weight each monitor according to its importance.

[Example] Hereinafter, an example of the present invention will be described in detail based on the drawings.

First, the overall configuration of the image monitoring device will be explained. 1st
In the figure, an object to be monitored 1 is, for example, in the process of drilling a hole in a production line (not shown), and under illumination 2, a TV camera 3 images the surface to be drilled. Imaging is performed for both a preliminary image before hole drilling and a post-hole image after hole drilling. Input control signals for completion of machining preparation and completion of machining in the processing machine for hole drilling are inputted to the main body 7 of the apparatus through an input line 5. Monitoring conditions for the device main body 7 and the like are input using the keypad 8. Every time an input control signal is input through the input line 5, the device body 7 executes preliminary monitoring or post-monitoring, and outputs the results as an output control signal through the output line 6. The main body 7 of the apparatus is provided with a CRT 7a capable of displaying images captured by the television camera 3, image signals obtained by binary or multi-value encoding, and the like.

Next, the principle will be explained using FIG. 2.

If the entire screen imaged by the TV camera 3 is to be the monitoring area, the entire screen is divided into blocks, and if a part of the entire screen imaged by the TV camera 3 is to be partitioned to be the monitoring area, the partitioned monitoring area is blocked. Divide into areas and cover the monitoring area. There are various methods of division, as shown in Figure 2.
The monitoring area S is the entire imaged screen (the number of horizontal pixels is LH and the number of vertical pixels is LV), which is divided into four equal parts in the horizontal direction and four equal parts in the vertical direction, and the 16 blocks obtained by these divisions are divided into four equal parts. The block area is U.

Generally, in order to automatically divide into block areas and monitor each block area based on the division, the dividing method is to equally divide the monitoring area S into 2 degrees (denoted as -M) in the horizontal direction. Divide the vertical direction equally into 2 degrees (-N),
It is preferable that the MXN blocks obtained by these divisions be the block area U. The reason why 21.20 is used here is because computers calculate in binary notation, so by multiplying the number by an exponent of 2, there will be no remainder. Index 1. n is operationally set depending on the shape of the monitored object 1, etc.

Monitoring is then performed for each block area. Monitoring is performed by comparing with a reference image, a tolerance range is set for each block area of the reference image, and a pass/fail judgment is made based on whether the values in each block area of the inspection image are within the tolerance range. .

The allowable range for each block area is weighted according to the importance of monitoring that block area. For example, in order to make the monitoring level of the block area where the image of the hole made by processing is reflected higher than the monitoring level of the block area where the image of the hole etc. is not visible but only the attached dust etc. The allowable range for block areas where images of holes and the like are visible is narrower than the allowable range for block areas where important images are not visible. Weighting of the permissible range can be done, for example, by setting the monitoring level to 1. ...j,...,J
The lower the number, the higher the monitoring level)
, when the unit tolerance is dQ, a certain block area U
If the reference value of Qp1 is the monitoring level j, then the upper limit of the allowable range of that block area U is QpH-Qp+jXdQ
, the lower limit value QpL-Qp-jXdQ.

Hereinafter, in this embodiment, a description will be given assuming that the monitoring area consists of M x N block areas partitioned by equally divided vertical lines and horizontal lines, and that each block area is designated with a monitoring level j.

Next, the overall block configuration of the image monitoring method will be explained using FIG.

A program for operating the CPU 22 is stored in the ROM 21 .

The RAM 23 stores pre and post reference image signals obtained by capturing the pre-normal state and post-normal state of the monitored object 1 with the television camera 3 and performing binary or multi-value processing, and the positions of each block area in pre- and post-monitoring. , unit tolerance, monitoring level j1 of each block area, and other data are stored.

The method of dividing into block areas, the number of two divisions, etc. may be different before and after, and in this case, both the before and after information is stored. The monitoring conditions are determined by the operator
T? While looking at a etc., input from keypad 8 etc., and press 11.
The data is stored in the RAM 23 via the 0 port 29.

Based on the commands in the ROM 21 and the data in the RAM 23, the CPU 22 determines the quality of each block area during preliminary and post-monitoring, and determines the quality of the entire monitoring area based on these results.

The image signal captured by the television camera 3 is sent to the A/D conversion circuit 3
0 and stored in the frame memory 31, and the contents of the frame memory 31 are displayed on the CRT 7a via the display circuit 32. Note that two frame memories 31 are prepared, each containing a preliminary image.

Post-images are stored.

Next, the flow of this method will be explained using FIGS. 4 and 5.

The image monitoring process as a whole consists of two monitoring operations: pre-monitoring and post-monitoring. The former type of advance monitoring monitors the state of the monitored object 1 before an event such as opening a hole, determines whether it is in a normal preparation state, and immediately cancels the event if an abnormality is detected. Also, post-event monitoring determines whether the state after the event is normal or not, and if an abnormality is detected, immediately cancels the next execution.

First, the operator displays the captured object 1 on the CRT7.
While viewing the image using a, the keypad 8 is used to set monitoring conditions, such as how to divide the area into block areas and the number of divisions, beforehand and afterward (102).

Next, an unprocessed image of the normal object to be monitored is taken by the television camera 3, and the image is taken by the CRT 7a.
3) Calculate the feature amount in each block area U, and determine the standard value Qp for quality determination. Then, the unit tolerance value dQ and the monitoring level j in each block area
Set. This determines the permissible range ±jdQ in each block area (104).

Next, an event such as normal hole machining is performed on the above-mentioned normal monitoring object, an image after this event is captured by the television camera 3, and the same operation as above is performed in each block area in the post-event monitoring. 103, 104).

Next, the process moves to monitoring, and when a signal indicating the completion of event preparation is received via the input line 5 (105), an image of the surface of the monitored object 1 to which the event will be applied is captured, and a pre-inspection image is input (106, 202). ), feature quantity q for each block region
is calculated (204), compared with the reference value of the reference preliminary image stored in the RAM 23, and judged as good or bad (205) depending on whether it is within the tolerance range ±jdQ (205).
6゜207), when the judgment is good in all block areas, the pre-monitoring result of the monitored object 1 is considered to be good (210), and the signal is transmitted to the processing machine etc. through the line 6 (108), and the hole drilling is performed by processing etc. event will be held. If any block area is determined to be abnormal, the retry or drilling of that block area will be canceled.

When monitoring the entire monitoring area, each block area has a different monitoring level, important block areas are monitored more strictly, and less important areas are monitored relatively loosely, so monitoring that meets the purpose of overall monitoring is possible. will be held.

If the pre-monitoring result is good and the event is held, a signal indicating the end of the event is received (109) and an image after the event is input (110).

Details of subsequent flows 111 and 112 are shown in FIG. 5, and will be explained based on this. A feature quantity q is calculated for each block area (204).

Note that the feature amount calculation in this case is performed by taking a difference image between the prior image and the subsequent image. As a result, only the parts that have changed between before and after processing are extracted. For example, small particles that were attached before processing or did not affect the pass/fail judgment are still attached after processing and are not extracted. It is possible to eliminate the influence on quality judgment in monitoring. Next, in each block area, the feature amount consisting of this difference image is compared with the reference value of the reference post-image stored in RAPv123, and depending on whether it is within the tolerance range ±jdQ (205
), determine whether it is good or bad (206°207),
When all block areas are judged to be good, the pre-monitoring result of the monitored object 1 is determined to be good (210), and the signal is transmitted to the processing machine via the line 6 (108), and the next event is continued. If it is determined to be abnormal, the event will be retried or the next event will be canceled. In the case of post-monitoring, as in the case of pre-monitoring, the monitoring level is set for each block area, so monitoring is executed in accordance with the purpose of overall monitoring.

In the above embodiment, the post-monitoring process was performed by comparing the difference image between the pre-image and the post-image with the reference post-image, but it is also possible to perform the post-mortem monitoring process by comparing the post-image with the reference post-image.

Further, by performing calculation of feature amounts in each block area and determination of quality of each block area simultaneously in each block area, processing speed can be increased.

[Effects] According to the present invention, since the monitoring area is divided into block areas of appropriate size and covered, it is possible to detect changes exceeding a predetermined amount, and it is possible to detect changes that exceed a predetermined amount. It becomes possible to perform overall monitoring.

In addition, by setting a permissible range and determining the quality of each block area based on whether or not it falls within the permissible range, the quality determination can be quickly performed under uniform conditions.

Furthermore, by weighting the allowable range for pass/fail judgment, block areas with a high monitoring level are monitored more strictly, and block areas with a low monitoring level are monitored more loosely, making it possible to perform more effective monitoring. .

[Brief explanation of the drawing]

Fig. 1 is an external perspective view showing how an object to be monitored is monitored by an image monitoring device, Fig. 2 is a plan view showing a state in which the monitoring area is divided into multiple block areas, and Fig. 3 is a system block diagram. , FIG. 4 is an overall system flowchart of image monitoring, and FIG. 5 is a specific flowchart of monitoring processing in the system flowchart of FIG. 4. 1...Monitored object, 3...TV camera, S...Monitoring area, U...Block area. Figure 2. N=M=4. n=m=2 Viewing hood figure 3 Tato part body figure

Claims (3)

[Claims] (1) In an image monitoring method in which the quality of the monitored object is determined by comparing the image of the monitored object captured by the imaging means with a reference image, a plurality of monitoring areas within the screen imaged by the imaging means are An image monitoring method characterized in that the image is divided into block areas and a pass/fail judgment is made for each block area. (2) Set an allowable range for each reference image in each of the block areas, and determine whether or not the image of the monitored object captured by the imaging means is within the allowable range for each block area. 2. The image monitoring method according to claim 1, wherein the quality determination is made for each region. (3) The image monitoring method according to claim 2, wherein the permissible range of the reference image in each block area is weighted according to the importance of monitoring.