JPS61187637A - Apparatus for inspecting appearance - Google Patents

Abstract

PURPOSE:To enable the detection of a flaw with high reliability reducing the brightness difference between a normal part and a flaw part and hardly receiving the effect of illumination irregularity, by providing an illumination means, a means for generating a multi-value picture element data group, an operation means and a judge means. CONSTITUTION:An image processing apparatus 11, a TV camera 12, a control calculator 11 and a half mirror 16 are provided. The light from a spot source 17 is passed through a diffusion plate 15 to be converted to diffused light which, in turn, irradiates an object 10. A mirror 16 is provided at the intermediate position between the diffused plate 15 and the object 10 and illumination light passes through the mirror 16 while the reflected light from the object 10 is refracted by the mirror 16 to be incident to the image pick-up lens 13 of the camera 12. Further, the image signal from the camera 12 is supplied to the apparatus 11 where the index showing the non-uniformity degree of the brightness of the object 10 is calculated to be sent to the calculator 14. This calculator 14 issues an order such as a processing start order to the apparatus 11 and receives the data calculated by the apparatus 11 to perform the synthetic processing for judging a flaw and sends out a quality judge result to a sequencer 20.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Application of the Invention] The present invention relates to an apparatus for automatically inspecting the appearance of an object, and more particularly to an apparatus for determining acceptability based on variations in brightness and darkness of the surface. Such a device can be used, for example, for visual inspection of LSI packages.

[Background of the invention]

One of the conventional devices for inspecting the appearance of objects is the Fuji Electric V
O2, 56, 49, pp. 591-595.

“Application of Fuji Electric Video Senna “Multi-Window””
It is described in. This device captures two images of the surface of an object.
The number of pixels in the 1" white area is counted, and pass/fail is determined based on the size of the pixel. If there is any non-uniformity, it is not easy to obtain accurate judgments.

[Purpose of the invention]

The purpose of the present invention is to provide automatic visual inspection that can reliably detect defects with a small difference in brightness from normal parts, such as slight irregularities, small holes, scratches, etc., and is less susceptible to the effects of uneven illumination. The goal is to provide equipment.

[Summary of the invention]

In order to deal with minute differences in brightness between normal parts and defective parts, the present invention represents bright and dark images on the surface of an object using multilevel image data, and also uses appropriate methods to reduce the effects of uneven illumination. A wide area is set and the non-uniformity of the distribution of brightness and darkness within this area is investigated. As an indicator of the non-uniformity of the distribution of brightness and darkness, the difference between the maximum and minimum values, the difference between the mode and the minimum value obtained by measuring the frequency distribution over a relatively wide area, or the difference between the mode and the minimum value, or a combination thereof. is used. Other indicators include a relatively narrow area centered around each pixel (for example, 3
3 pixels) is used. Furthermore, as another indicator, a narrow area centered on each pixel (for example, 3 x 3 pixels), targeting a medium-sized area.
brightness pattern and surrounding area (for example, within a range of 5 x 5 pixels)
The degree of mismatch between the brightness pattern and the brightness pattern of the low phase area (for example, the sum of the absolute values of the differences in brightness of corresponding pixels) can be used. Further, a combination of some or all of these indicators may be used. When this index of brightness non-uniformity exceeds a predetermined value, it is determined that there is a defect.
.

The influence of the inclination angle of the object surface is alleviated by using a diffuser plate in the illumination section.

[Embodiments of the invention]

FIG. 2 conceptually represents an entire example of a visual inspection system to which the present invention is applied. The object to be inspected 10 is fed through the inspection station by means of a feed device 19 .

In the inspection station, a lighting device is attached to the support stand 18, and this lighting device allows the light from the point light source 17 to
The light is configured to pass through the diffuser plate 15 to become diffused light, and to irradiate the object 10 . The diffuser plate 15 may be made of ground glass, for example. A half mirror 16 is provided at an intermediate position between the diffuser plate 15 and the object 10, and the illumination light passes through this, while
The reflected light from the object 10 is bent here and enters the imaging lens 13 of the TV camera 12 provided on the side. A prism may be used in place of the no-f mirror.

A video signal from the TV camera 12 is supplied to the image processing device 11. The image processing device 11 processes this video signal, calculates several indices representing the degree of non-uniformity of brightness on the surface of the object 10, and sends the results to the control computer 14. The control computer 14 issues commands to the image processing device 11 to start processing and other commands, receives data calculated by the image processing device 11, performs comprehensive processing for defect determination, and outputs a pass/fail determination result 21. It is sent to a sequencer (not shown). The sequencer itself is a normal one, and the necessary information 20 is exchanged with the control computer 14.
While transmitting and receiving data, the entire inspection system including the feeding device 19 is controlled in an integrated manner.

FIG. 3 schematically shows the illumination and imaging optical system in FIG. 2. However, for the sake of simplicity, the point light source and half mirror are omitted, and the position of the imaging system is shown assuming that there is no half mirror. It is assumed that specular reflection occurs on the surface 300 of the object. The diffuser plate 15 allows reflected light to enter the imaging lens even if the object surface 300 is somewhat inclined with respect to a plane perpendicular to the optical axis.
It is provided. The larger the diffuser plate is, and the closer it is to the object, the smaller the influence of the tilt angle on the brightness of the object surface. On the other hand, surface defects such as dents and bulges are detected as dark areas corresponding to the slope of the surface in those areas. The amount of light that is reflected from the sloped portion corresponding to such a defect and enters the imaging lens must be kept sufficiently small. Therefore, there are limits to the dimensions of the diffuser plate and its distance from the object.

FIG. 3 shows the incident light from the end of the diffuser plate 15 to the object surface 3.
This shows a critical state in which the light reflected by 00 can barely enter the imaging lens 13.

The distance between the diffuser plate and the object surface is tl, the distance between the diffuser plate and the imaging lens is t2, the radius of the diffuser plate is rIN, the distance between the end of the diffuser plate and the object is ``2'', and the distance between the diffuser plate and the imaging lens is ``2''. If the distance between the optical axes is x1, the inclination angle of the object surface with respect to the plane perpendicular to the optical axis is θ, and the angle between the incident light and the reflected light with respect to the normal line is α, then the following equation holds true.

rl=r2+x tan (ex 〇)=rz/1x tan (a-θ)=x/(zt +tz) Therefore, -tan-' ((x/tl)/(1+lz)
/1t)):l ・” (1) In particular, if x = O, the distribution of the inclination angle θ of the object surface can be known in advance, so the diffusion plate 15 can be determined by equation (1) or (2). The size and position can be determined, and the inclination angle of the detectable defect can also be calculated. Note that if the object surface has no inclination, the diffuser plate may be omitted.

FIG. 1 schematically shows the internal configuration of the image processing device 11 in FIG. 2. As shown in FIG. The video signal from the TV right camera 2 is converted into multivalued digital data by the analog-to-digital converter 200, and once stored in the image memory 201, the video signal is sent to the frequency distribution generation section 202 and the maximum difference filtering section 2.
03 and correlation value filtering section 204, respectively. The frequency distribution generation unit 202 divides the object surface into a plurality of relatively wide regions (for example, several tens of pixels by several tens of pixels), and calculates the frequency of brightness values for each region, that is, the frequency of pixels at each brightness level. The number is examined and the maximum (brightest level) value, minimum (darkest level) value, mode value (level value with the largest number of pixels), etc. are determined. The maximum difference filtering unit 203 calculates, for each pixel, the difference between the maximum value and the minimum value in a relatively narrow area (3×3 pixels in this example) centered on the pixel. The correlation value filtering unit 204 divides each pixel into a relatively narrow area (
For example, the degree of mismatch in the brightness and darkness pattern between a region (e.g., 3×3 pixels) and each of a plurality of regions similar to it in its vicinity (e.g., 5×5 pixels) is calculated, and its maximum value is determined. .

These calculation results are comprehensively judged by the control computer 14, and a pass/fail judgment result 21 is sent out.

The address generation circuit 205 specifies a write address for each pixel data from the analog-to-digital converter 200 and a read address for processing by the processing units 202 to 204 to the image memory 201, and , supplies clock signals to each part.

FIG. 4 shows an outline of the configuration of the frequency distribution generation section 202.

Generation of the frequency distribution is performed for each of a plurality of relatively wide regions (several tens of pixels by several tens of pixels) obtained by dividing the surface of the object. The frequency distribution has a predetermined area (Xs≦X≦
A group of pixel data forming a bright and dark pattern of
It is obtained by counting the number h (v l) of pixel data having l.

Each pixel data 410 in a predetermined area sequentially read out from the image memory 201 is given to the integration memo IJ 400 as its addressing information. For example, if pixel data consists of 8 bits, the data value will range from 0 to 255, and the integration memory 400 will have addresses 0 to 255 correspondingly. The contents of the address specified by the value of input pixel data 410 are read out at output 411 and sent to adder 401, where @t 11j is added under the control of increment control signal 412. The output 413 of adder 401 is written to the original address, thus the number of pixel data having each value is accumulated to the address corresponding to that value. Increment control signal 412 that indicates whether an addition of 1" is to be performed or not.
is the output from mask memory 402. The mask memory 402 holds a binary mask pattern having ``1'' at the position where 1'' is to be added in the adder 401, and this mask pattern is set by the control computer 14 via the data line 414. Ru. The mask memory read address 415 is set by the mask memory address setting circuit 4.
Specified from 03 onwards. The mask pattern is image memo IJ
The "1" area is determined to cover a slightly smaller area than the entire image area in 201, and its relative position with respect to the entire image area is determined by the mask memory address setting circuit from the control computer 14. 403. Such a mask mechanism is provided to determine an appropriate area to be inspected in response to the positional shift of the object. The mask memory address setting circuit 403 is It also receives a clock signal 417 from the address generation circuit 205 to maintain synchronization with the readout operation of the image memory 201. 'From the frequency distribution obtained as described above, the maximum and minimum values of the pixel values are determined. The mode value (the largest number of pixel values) is determined. In this embodiment, the maximum value and minimum value are determined by internal processing of the control computer 14. For this determination, The control computer 14 reads the contents of the integration memory 400 via the data line 418 and performs the processing shown in FIG. 5. For example,
It is assumed that the pixel values are distributed between 0 and 255. First, in order to determine the maximum value, starting from the address (address 255) corresponding to the upper limit value (v=255), the contents h (
v) sequentially and check whether it is °ゝOpp. In this process, the first address holding a value other than "0" corresponds to the maximum value. Then, the lower limit value (V=O)
It jumps to the address corresponding to (address O) and performs the same search as above in the direction of increasing addresses from there. In this process, the first address that holds a value other than '0' corresponds to the minimum value. Through the above steps, even though it is a sequential process, it is possible to determine all the maximum and minimum values in a short time. However, a special logic circuit for determining the maximum value and minimum value may be provided inside the frequency distribution generation unit 202.

The mode is determined by a mode detection circuit 404 provided inside the frequency distribution generation section 202.

FIG. 6 shows the configuration of the mode detection circuit 404.

Counter 0 (1200) corresponding to pixel value 0~255
- Counters 255 (1255) are prepared, and all of these counters are activated by the initialization signal 1272 from the control computer 14 at the start of operation.
It is initialized to n. The decoder 1260 is an image memo! Each pixel data 410 from 7201 is decoded, and the decoded output 1271 is given as an increment enable signal to one of the counters 255 corresponding to the pixel value. The counter that receives this signal is incremented by application of an increment control signal 412 from the mask memory 402. Zero value detection circuit 1262 monitors the count values of counter 0 to counter 255, and if there is a counter with a count value of 0'', it decrements all the counters in response to control signal 1270. The count value of counter 255 is always equal to or less than It-1j''. After all pixel data in the four predetermined value quadrants are read out from the pixel memory, the processing/
The coder 1261 determines the mode by encoding the number of the counter having the maximum count value, that is, the count value of "par-1", and sends the output 420 to the control computer 14.
send to

In this way, the mode is determined at the same time as the generation of the frequency distribution is completed. Since the mode can be regarded as the average brightness of the surface of the object, using the difference between the mode and the minimum value as an index for determining defects means that areas that are abnormally dark compared to the average brightness can be detected. It means to detect it as a defect. An average value may be used as a substitute for the mode, but since calculating the average value requires a considerable amount of time, it is advantageous to use the above mechanism.

The control computer 14 calculates the difference between the maximum value and the minimum value and the difference between the mode and the minimum value using the maximum value and minimum value determined by itself and the mode value from the mode detection circuit. Then, it is checked whether each of these differences exceeds a predetermined limit value, and if at least one of them exceeds the predetermined limit value, it is determined that there is a defect.

The maximum difference filtering unit 203 processes a relatively narrow area in the vicinity of each pixel. In this embodiment, this area is a 3×3 pixel portion centered on the pixel of interest. The maximum difference filtering unit 203 determines the maximum value and minimum value of all pixel data belonging to this area,
Next, the difference between the maximum value and the minimum value is calculated, and it is checked whether this difference is larger than a predetermined value. If this difference is larger than a predetermined value, it indicates that there is a defect. According to the maximum difference filtering process, local changes in brightness and darkness are emphasized, so minute defects can be detected.

FIG. 7 shows an outline of the configuration of the maximum difference filtering section 203. Nine pixel data 510 to 518 belonging to a square area of 3×3 pixels are read out in parallel from the image memory 201 and sent to the maximum value detection circuit 500 and the minimum value detection circuit 501.
is supplied to both parties. The maximum value detection circuit 500 selects the maximum value in the input pixel data group and sends it to the output 520, and the minimum value detection circuit 501 selects the minimum value in the input pixel data group and sends it to the output 521. do. The subtractor 502 calculates the difference between the maximum value and the minimum value, and this difference 523 is compared with a predetermined limit value 524 by the comparator 504. This limit value 524 is transmitted from the control computer 14 to the maximum difference limit value register 503 via a data line 527.
is stored in. If this difference is larger than the limit value,
Information 525 indicating this is set in the result register 505, and is further transmitted to the control computer 1 via a data line 526.
Sent to 4.

As shown in FIG. 8, the maximum value detection circuit 500 consists of a plurality of comparison and selection circuits connected in a tree shape, and each comparison and selection circuit selects and outputs the larger one of the two. , As a result, the maximum value of input colors 10 to 518 becomes the final output 520. The structures of these comparison and selection circuits are all the same, and a comparison and selection circuit 600 is shown in FIG. 9 as a representative. That is, the comparison circuit 700 compares the input data 510 and 511, controls the selector 701 by a signal 710 indicating which one is larger, and sends out the larger data to the output 610.

, the minimum value detection circuit 501 has the same structure as in FIGS. 8 and 9, except that the comparator controls the selector so that the smaller one of the input data is sent out.

Next, the correlation value filtering section 204 will be explained. The correlation value filtering referred to here is the following process. A relatively narrow region R of mxn pixels centered on the pixel of interest is cut out as the region of interest, and at the same time, a region R centered on each pixel in the vicinity of the pixel of interest (range of MXN pixels centered on the pixel of interest) is cut out as the region of interest. A region R+ (i=
1 to MXN-1) are extracted as neighboring regions. Then, the sum S1 of the absolute values of the differences between the brightness values jP and s P+ N=1 to mXn) of the pixels located at the respective corresponding positions in the area hole and R+ is calculated. That is, equation (3) is calculated for all of the neighboring regions Ri, and the maximum value among them becomes the correlation value filtering output. The correlation value filtering output thus obtained represents the overall degree of mismatch between regions R and R+. If this filtering output exceeds a predetermined limit value, it is determined that there are defects. Such processing integrates changes in brightness over a certain range of regions, and is therefore effective in detecting defective areas where brightness gradually changes from normal areas.

FIG. 10 schematically shows how to cut out each region in the case of m=n=M=N=3o. The total area to be processed is (m+M/2) pixels x (n+N/2) pixels, and in this case is an area 1000 of 5×5 pixels. The center of this area is the pixel of interest, so the pixel data group 820 in the central 3x3 pixel area 1010 (shaded area) is
The pattern of the hole of interest becomes the data, and the 3X of other positions
Pixel data groups 821 to 3 pixel areas 1011 to 1018
828 is compared with the pixel data group 820 as pattern data of the neighboring region R+.

FIG. 11 shows an outline of the configuration of the correlation value filtering section 204 for the region group cut out as shown in FIG. 10.

The pixel data group 820 in FIG.
7, and the other pixel data group 8
21-828 are supplied to the other inputs of correlators 800-807, respectively. These correlators each perform the calculation of equation (3) on the two supplied pixel data groups, and send the respective calculated values 830 to 837 to the maximum value detection circuit 808. The maximum value detection circuit 808 selects the maximum value among these calculated values and stores it in the maximum value register 809. The correlation value limit value register 810 holds the limit value for quality determination supplied from the control computer 1.4. Comparison circuit 811 compares the contents of maximum value register 809 and the contents of correlation value limit value register 810, and if the contents of maximum value register 809 is larger, sets information indicating that there is a defect in result register 812. This information is sent to the control computer 4 via a data line 840.

Correlators 800 to 807 all have the same structure, and correlator 800 is shown in FIG. 12 as a representative. The pixel data group 820 is data 96 of 3×3=9 pixels.
0 to 968, each of which is subtracted by a subtracter 90.
This is one input from 0 to 908. The pixel data group 821 consists of pixel data 950 to 958 corresponding to the pixels of data 960 to 968, respectively, and these data are as follows.
Each becomes the other input of subtracters 900-908.

The difference outputs from these subtractors are sent to absolute value circuits 910-9.
18, and are summed by adders 920 to 927,
A summation output 830 is obtained.

Judgment results based on the maximum value, minimum value, and mode obtained from the frequency distribution, judgment results based on maximum difference filtering processing, and judgment results based on correlation value filtering processing can be determined as appropriate depending on the inspection purpose and the properties of the target object. The final pass/fail is determined. For example, if any one judgment barrier indicates a failure, it may be determined to fail immediately, or when at least two judgment results indicate failure (it may be determined to fail. It is also possible to use the result obtained by multiplying the result by an appropriate weight and summing the result.

Further, in the above embodiment, the maximum difference filtering section and the correlation value filtering section have a structure for completely parallel processing, but if the processing speed is not strictly required, the hardware may be configured as a parallel/serial processing structure. can be reduced.

〔Effect of the invention〕

According to the present invention, there is little difference in brightness between the normal area and the defective area, and even when there is some uneven illumination,
It is possible to detect defects with high reliability and also meet the demands for high-speed processing.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of an image processing device according to an embodiment of the present invention, FIG. 2 is a schematic diagram of an external appearance inspection system, FIG. 3 is a schematic diagram of an illumination and imaging optical system, and FIG. 4 is a diagram similar to that of FIG. 1. A block diagram of the frequency distribution generator in
Figure 5 is a flowchart of the process of determining the maximum and minimum values from the frequency distribution, Figure 6 is a block diagram of the mode detection circuit in Figure 4, and Figure 7 is a block diagram of the maximum difference filtering section in Figure 1. 8 is a block diagram of the maximum value detection circuit in FIG. 7, FIG. 9 is a block diagram of the comparison and selection circuit in FIG. 8, and FIG. 10 is a schematic diagram showing area selection for correlation value filtering processing. , FIG. 11 is a block diagram of the correlation value filtering section in FIG. 1, and FIG. 12 is a block diagram of the correlator in FIG. 11. 10...Target, 11...Image processing device, 12...
・TV camera, 14... Control computer, 15... Diffusion plate, 16... Half mirror 117... Point light source, 2
01... Image memory, 202... Frequency distribution generation unit,
203... Maximum difference filtering section, 204... Correlation value filtering section, 205... Address generation circuit.

Claims (1)

[Claims] 1. Illumination means for applying epi-illumination to the surface of an object, means for generating a group of multivalued pixel data corresponding to brightness and darkness images of the surface of the object, and a method using the group of multivalued pixel data. Appearance comprising: calculation means for calculating data representing the degree of non-uniformity of brightness within each predetermined partial region of the surface of the object; and determination means for determining the presence or absence of a defect based on the degree of non-uniformity. Inspection equipment. 2. The visual inspection apparatus according to claim 1, wherein the calculation means includes distribution determining means for examining the frequency distribution of brightness and darkness in each area and calculating data representing the width of the distribution. 3. An appearance inspection device according to claim 2, wherein the distribution determining means calculates the difference between at least one of the maximum value or the mode and the minimum value as data representing the distribution. 4. The visual inspection apparatus according to claim 1, wherein the calculation means includes maximum difference filtering means for calculating the difference between the maximum value and the minimum brightness value within a certain area including the vicinity of each pixel. 5. In claim 1, the calculation means calculates a correlation value for each pixel to calculate the maximum value of the degree of mismatch of brightness and darkness patterns between a certain region including its neighborhood and a plurality of similar regions in its neighborhood. Appearance inspection device including filtering means. 6. In claim 5, the correlation value filtering means is an appearance inspection device that calculates the sum of the absolute values of the differences in brightness of each pair of pixels as an amount representing the degree of mismatch of brightness patterns between regions. . 7. The visual inspection apparatus according to claim 1, wherein the calculation means includes a plurality of means for calculating different types of data representing the degree of non-uniformity of brightness. 8. In claim 7, the calculation means includes distribution calculation means for examining the frequency distribution of brightness and darkness in each region and calculating data representing the width of the distribution, and a fixed region including the vicinity of each pixel. Maximum difference filtering means that calculates the difference between the maximum and minimum brightness values within the area, and the degree of mismatch in brightness and darkness patterns between a certain area including the vicinity of each pixel and a plurality of similar areas in the vicinity of each pixel. A visual inspection device including at least two correlation value filtering means for calculating a maximum value of . 9. An appearance inspection apparatus according to each of claims 1 to 8, wherein the determining means includes means for determining whether the calculation result of the calculating means exceeds a predetermined value. 10. In each of claims 1 to 9,
The illumination means includes a light source, a light diffusion means provided between the light source and the object, and an imaging optical path separation means provided between the light diffusion means and the object.