JP2016038315A - Measuring device, substrate inspection equipment, and method for controlling the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide technology capable of accurately measuring a three-dimensional shape of a solder surface irrespective of height of glossiness (specularity) of the solder surface.SOLUTION: A measuring device has: an image acquisition part which acquires a first image picked up in a state constituted so that beams in a plurality of colors are irradiated at incident angles different from one another, and color information according to their inclination angles appears on a surface of solder, and a second image picked up in a state constituted by projecting pattern light and so that phase information of a pattern according to height of the pattern light appears on the surface of the solder; a color reliability calculation part which calculates reliability of the color information in the first image for respective points on the surface of the solder; a phase reliability calculation part which calculates reliability of the phase information in the second image for the respective points on the surface of the solder; and a solder shape measurement part which generates a three-dimensional shape of the solder by calculating three-dimensional information of the respective points on the surface of the solder by using information with higher reliability of the color information and the phase information.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for inspecting the state of solder bonding of components mounted on a printed circuit board, and more particularly to a technique for measuring a three-dimensional shape of solder.

  In a surface mounting line of a printed circuit board, a board inspection apparatus (also called an appearance inspection apparatus) for inspecting the quality of solder joint after reflow is widely used. In the board inspection apparatus, various indicators related to the shape of the solder are measured from an image obtained by photographing the board, and the bonding state of the solder to the electrode or land (pad) is inspected based on the measured value. At this time, since it is necessary to inspect the three-dimensional shape of the solder using an image that is two-dimensional information, various processing techniques have been proposed.

  As one of them, a color highlight illumination method is known. The color highlight illumination method irradiates the substrate with multiple colors of light at different angles of incidence, and the color information (the color of the light source in the specular reflection direction as seen from the camera) is applied to the solder surface according to the tilt angle. This is a method of capturing the three-dimensional shape of the solder surface as two-dimensional hue information by performing imaging in a state where it appears. This method has been found to be extremely effective in capturing the shape of solder fillets in normal solder inspection. In addition, a method has been proposed in which the inclination angle (gradient) of the solder surface is obtained from the color information of the image and the three-dimensional shape of the solder surface is restored by integrating the gradient (see, for example, Patent Documents 1 and 2). However, in recent surface mounting, there is a tendency that solder with a large amount of flux is used, and there is a problem that the gloss (mirror finish) of the solder surface is partially lowered due to the influence of the flux. In the part where the glossiness of the solder surface is lowered, the illumination light is diffusely reflected or diffusely reflected, and a color close to the diffused light (white light) is observed, or a color different from the actual inclination angle is observed. . When such an image partially including erroneous color information is used, the inclination angle (gradient) of the solder surface is misrecognized, leading to a reduction in the reconstruction accuracy of the three-dimensional shape.

  On the other hand, a technique called a phase shift method is also known (see, for example, Patent Documents 2, 3, and 4). The phase shift method is a method for restoring the three-dimensional shape of the object surface by analyzing pattern distortion (phase change) when pattern light is projected onto the object surface. However, this method is effective for diffused objects (parts, electrodes, solder paste, etc.) and solder with low gloss, but when the gloss of the solder surface is high, the phase of the pattern light can be analyzed appropriately. There is a problem that the measurement error of the three-dimensional shape (height information) of the solder surface becomes large.

  Thus, each of the conventional proposed methods has advantages and disadvantages, and there is no method that can accurately measure both high-gloss (specularity) solder and low-solder.

  Patent Document 2 proposes a method that combines the advantages of the color highlight illumination method and the phase shift method. However, the technique is only to position the three-dimensional shape of the solder surface restored by the color highlight illumination method to the height obtained by the phase shift method. In the case of a decrease in the three-dimensional shape, it is inevitable that the restoration accuracy of the three-dimensional shape will decrease.

JP 2010-71844 A Special table 2013-543591 gazette JP 2012-145484 A JP 2013-221861 A

  The present invention has been made in view of the above circumstances, and regardless of the level of gloss (mirror surface) of the solder surface, or a portion with high and low gloss (mirror surface) in the solder surface. Even if it is, it aims at providing the technique which can measure the three-dimensional shape of the solder surface with high precision.

  In order to achieve the above object, in the present invention, the color information and phase information of the solder are obtained by different methods, and the three-dimensional shape of the solder is generated using the information with higher reliability of the color information and the phase information. The configuration is adopted.

  Specifically, the measuring device according to the present invention is a measuring device that measures a three-dimensional shape of solder, irradiates light of a plurality of colors at different incident angles, and depending on the inclination angle of the surface of the solder. The first image was captured with the color information appearing, and the pattern light was projected on the surface of the solder so that the pattern phase information corresponding to the height appeared on the solder surface. An image acquisition unit for acquiring a second image; a color reliability calculation unit for determining reliability of color information in the first image for each point on the surface of the solder; and each point on the surface of the solder The phase reliability calculation unit for obtaining the reliability of the phase information in the second image, the three-dimensional information of each point on the surface of the solder, the color information in the first image and the second image Using information with higher reliability of phase information By Mel, a measuring apparatus characterized by having a solder shape measurement unit for generating a solder three-dimensional shape.

  In the present invention, two types of images (first image and second image) captured by different illumination methods are used. In the first image, the inclination angle of the solder surface appears as color information, and in the second image, the height of the solder surface appears as phase information. Since it is possible to generate the three-dimensional shape of the solder using only one of the information, the present invention obtains the reliability of the color information and the reliability of the phase information, and obtains the information with the higher reliability. It is used to generate the three-dimensional shape of solder. In principle, the first image illumination method can be expected to be more accurate as the gloss of the solder is higher, and the second image illumination method can be expected to be more accurate in principle as the solder is less glossy. Therefore, if the one with higher reliability (one that can be expected to be accurate) of the two methods as in the present invention is used preferentially, regardless of the glossiness (mirror surface) of the solder surface, Even if a portion with high glossiness (specularity) and a portion with low glossiness are mixed in the solder surface, the three-dimensional shape of the solder surface can be accurately measured.

  It is good to further have a result output part which displays on a display device an image showing the three-dimensional shape of solder obtained by the solder shape measurement part. By viewing such a display image, the user can accurately and intuitively grasp the three-dimensional shape of the solder to be measured. This display image can be used for solder fillet shape inspection, teaching of a board inspection apparatus (setting of inspection program, determination standard, etc.) and the like.

  The color reliability calculation unit may determine the reliability of the color information of the point on the surface of the solder corresponding to the pixel based on the saturation or brightness of the pixel of the first image. This is because in the first image illumination method, the higher the glossiness of the solder surface (closer to specular reflection), the more saturated or lighter color is observed. Therefore, the reliability of the color information can be set so as to have a positive correlation with the high saturation or brightness. The reliability may be a continuous value, or may be binary or multivalued.

  The second image is a plurality of images picked up by changing the phase of the pattern light, and the phase reliability calculation unit is based on the amount of change in luminance of the same pixel between the plurality of images. The reliability of the phase information of the points on the solder surface corresponding to the pixels may be obtained. In the second image illumination method, the lower the gloss of the solder surface (closer to diffuse reflection), the clearer the change in the phase of the pattern light, and the greater the change in luminance of the same pixel. It is. Therefore, the reliability of the phase information can be set so as to have a positive correlation with the amount of change in luminance of the same pixel. The reliability may be a continuous value, or may be binary or multivalued.

  A color information analysis unit that generates a multi-valued image in which each pixel value represents a different tilt angle range by performing multi-value processing for dividing the first image into regions for each hue; and the second image A phase information analysis unit that obtains height information of the surface of the solder corresponding to each pixel from the phase information of the plurality of pixels, and the solder shape measurement unit is more phase information than color information among the pixels of the multi-valued image. The pixel value of the pixel having higher reliability is corrected based on the inclination angle calculated from the height information of the pixel and its neighboring pixels obtained by the phase information analysis unit, and the corrected multi-value image The three-dimensional shape of the solder may be generated using Noise is removed by performing multi-value processing on the first image. Therefore, the three-dimensional shape of the solder can be easily and accurately restored by joining the inclination angles (gradients) based on the multi-valued image. However, if a part of the surface of the solder has a reduced gloss due to the influence of flux, etc., the color information for that part may not be accurate (indicating an incorrect tilt angle). Yes, the accuracy of reconstruction of the 3D shape is lowered. Therefore, for the pixel whose reliability of the color information is decreased due to the decrease in glossiness and the reliability of the phase information is increased, the color of the multi-valued image is corrected based on the inclination angle calculated from the phase information. As a result, the color information (inclination angle information) of the part with reduced glossiness can be supplemented with phase information, and even if the part with high glossiness and the part with low glossiness are mixed, the three-dimensional shape of the entire solder surface Can be restored accurately.

  A color information analysis unit that generates a multi-valued image in which each pixel value represents a different tilt angle range by performing multi-value processing for dividing the first image into regions for each hue; and the second image A phase information analysis unit that obtains the height information of the surface of the solder corresponding to each pixel from the phase information of the pixel, and the solder shape measurement unit includes the height information of each pixel obtained by the phase information analysis unit Among these, the height information of the pixel whose color information is more reliable than the phase information is corrected based on the information on the inclination angle of the pixel obtained from the multi-valued image, and the height of each pixel after the correction is corrected. It is also preferable to generate a three-dimensional shape of the solder using the depth information. If the phase information of the second image is used, the height information of the solder surface can be obtained. However, although the accuracy of height information can be expected when the glossiness of the solder surface is low, the phase information may not be accurate in a portion with high glossiness, and the accuracy of restoring the three-dimensional shape is lowered. Therefore, for pixels whose phase information reliability has decreased due to high glossiness, height correction is performed based on inclination angle information obtained from color information with high reliability. As a result, the phase information (height information) of the part with high glossiness can be complemented with color information, and even if the part with high glossiness and the part with low glossiness are mixed, the three-dimensional shape of the entire solder surface can be obtained. It becomes possible to restore accurately.

  The present invention can be understood as a measuring device including at least a part of the above means or functions. Moreover, this invention can also be grasped | ascertained as a board | substrate inspection apparatus which test | inspects the joining state of solder using the three-dimensional shape of the solder obtained with the said measuring device. Further, the present invention can be understood as a control method for a measurement apparatus or a substrate inspection apparatus, a computer program for causing a computer to execute each step of the method, or a computer-readable storage medium that stores the program in a non-temporary manner. You can also Each of the above configurations and processes can be combined with each other as long as no technical contradiction occurs.

According to the present invention, the solder surface can be used regardless of whether the gloss (mirror finish) of the solder surface is high or low, or even if the solder surface has both high and low gloss (mirror finish) portions. The three-dimensional shape can be accurately measured.

The schematic diagram which shows the hardware constitutions of a board | substrate inspection apparatus. The block diagram of the function in connection with an inspection process. The example of CH image imaged by irradiating R, G, B light. The example of the phase image imaged by projecting pattern light. The flowchart which shows the flow of an inspection process. The flowchart which shows the flow of a solder shape measurement process. The example of the image for demonstrating the procedure of solder shape measurement typically.

  Preferred embodiments for carrying out the present invention will be exemplarily described in detail below with reference to the drawings. However, the dimensions, materials, shapes, relative arrangements, and the like of the components described in the following embodiments are not intended to limit the scope of the present invention only to those unless otherwise specified.

<First Embodiment>
(Configuration of board inspection equipment)
With reference to FIG. 1, the whole structure of the board | substrate inspection apparatus which comprises the measuring device which concerns on embodiment of this invention is demonstrated. FIG. 1 is a schematic diagram showing a hardware configuration of a substrate inspection apparatus. This board inspection apparatus 1 is preferably used for board appearance inspection (for example, inspection of solder joint state after reflow) in a surface mounting line.

  The substrate inspection apparatus 1 includes a stage 10, a measurement unit 11, a control device 12, an information processing device 13, and a display device 14 as main components. The measurement unit 11 includes a camera (image sensor) 110, an illumination device 111, and a projection device (projector) 112.

  The stage 10 is a mechanism for holding the substrate 15 and aligning the component 150 or the solder 151 to be inspected with the measurement position of the camera 110. As shown in FIG. 1, when the X axis and the Y axis are taken in parallel to the stage 10 and the Z axis is taken perpendicular to the stage 10, the stage 10 can translate at least two axes in the X direction and the Y direction. The camera 110 is arranged so that the optical axis is parallel to the Z axis, and images the substrate 15 on the stage 10 from above. Image data captured by the camera 110 is taken into the information processing apparatus 13.

  The illumination devices 111 (111R, 111G, 111B) are illumination means that irradiate the substrate 15 with illumination lights RL, GL, BL of different colors (wavelengths). FIG. 1 schematically shows an XZ cross section of the illuminating device 111. Actually, the illuminating device 111 has an annular shape so that light of the same color can be illuminated from all directions (all directions around the Z axis). Or it has a dome shape. The projection device 112 is a pattern projection unit that projects pattern light PL having a predetermined pattern onto the substrate 15. Projection device 112 projects pattern light PL through an opening provided in the middle of illumination device 111. The number of the projection devices 112 may be one, but a plurality of projection devices 112 may be provided in order to eliminate the blind spot of the pattern light PL. In the present embodiment, the two projectors 112 are arranged in different directions (diagonal positions). The illumination device 111 and the projection device 112 are both illumination systems used when the substrate 110 is photographed by the camera 110, but the illumination device 111 is used for the purpose of measuring the surface shape of the solder by the color highlight illumination method. The projection device 112 is used for the purpose of measuring the surface shape of the solder by the phase shift method.

The control device 12 is a control means for controlling the operation of the substrate inspection apparatus 1 and is responsible for stage 10 movement control, lighting device 111 lighting control, projection device 112 lighting control and pattern change, camera 110 imaging control, and the like. ing.

  The information processing apparatus 13 acquires various measurement values related to the component 150 and the solder 151 using the image data captured from the camera 110, and the state of solder bonding to the electrode of the component 150 and the land (pad) on the board It is a device having a function of inspecting. The display device 14 is a device that displays measurement values and inspection results obtained by the information processing device 13. The information processing device 13 can be configured by, for example, a general-purpose computer having a CPU (Central Processing Unit), a memory, an auxiliary storage device (such as a hard disk drive), and an input device (such as a keyboard, mouse, and touch panel). In FIG. 1, the control device 12, the information processing device 13, and the display device 14 are illustrated as separate blocks, but these may be configured as separate devices or may be configured as a single device. Good.

(Functional configuration)
FIG. 2 is a block diagram illustrating a configuration of functions related to solder shape measurement and inspection processing provided by the information processing device 13. As functions related to the solder shape measurement processing, the image acquisition unit 20, the color information analysis unit 21, the color reliability calculation unit 22, the phase information analysis unit 23, the phase reliability calculation unit 24, the solder shape measurement unit 25, and the like are provided. As functions related to the inspection process, the inspection unit 26, the inspection program storage unit 27, the result output unit 28, and the like are provided. These functions are realized by the CPU of the information processing device 13 reading and executing a program stored in the auxiliary storage device. However, all or some of the functions may be configured by a circuit such as an ASIC or FPGA.

  The image acquisition unit 20 has a function of capturing image data from the camera 110. The color information analysis unit 21 is a function of analyzing color information from an image captured by the color highlight illumination method, and the color reliability calculation unit 22 is a function of obtaining the reliability of the color information of this image. The phase information analysis unit 23 has a function of analyzing pattern phase information from an image captured by the phase shift method, and the phase reliability calculation unit 24 has a function of obtaining reliability of the phase information of the image. . The solder shape measuring unit 25 has a function of generating a three-dimensional shape of solder based on the color information obtained by the color information analyzing unit 21 and the phase information obtained by the phase information analyzing unit 23. Details of these functions will be described later.

  The inspection unit 26 measures (calculates) various indexes related to the shape of the solder fillet based on the three-dimensional shape data of the solder surface obtained by the solder shape measuring unit 25, and uses these measured values to determine the solder joint. This is a function for checking the state. The inspection program storage unit 27 has a function of storing an inspection program that defines inspection items and conditions in the inspection unit 26. In the inspection program, for example, the position and size of the land to be inspected, the size of the part, the type of index to be measured, the determination reference value for each index (threshold value or range for determining non-defective product and defective product), etc. are defined. ing. The result output unit 28 is a function for outputting the three-dimensional shape of the solder obtained by the solder shape measuring unit 25 and the measurement value and the inspection result obtained by the inspection unit 26 on the screen.

(Measurement of color information according to the inclination angle of the solder surface)
In this embodiment, a so-called color highlight illumination method is used to measure the inclination angle of the solder surface. The color highlight illumination method irradiates a substrate with light of a plurality of colors (wavelengths) at different angles of incidence, and color information on the solder surface according to the tilt angle (of a light source in the regular reflection direction when viewed from the camera) This is a method of capturing the three-dimensional shape of the solder surface as two-dimensional hue information by performing imaging in a state where the color) appears.

With reference to FIG. 1, the structure of the illuminating device 111 used for the color highlight illumination method is demonstrated. The illumination device 111 has a structure in which three annular light sources, a red light source 111R, a green light source 111G, and a blue light source 111B, are arranged concentrically around the optical axis of the camera 110. Each of the light sources 111R, 111G, and 111B is adjusted in elevation angle and direction so that the incident angle with respect to the substrate increases in the order of red light RL, green light GL, and blue light BL. Such an illuminating device 111 can be formed, for example, by arranging LEDs of R, G, and B colors in an annular shape outside a dome-shaped diffusion plate.

  FIGS. 3A to 3C show examples of image data (hereinafter referred to as color highlight (CH) images) obtained when the camera 110 is photographed with the illumination device 111 turned on. 3 (a) shows a state where the solder fillet is good, FIG. 3 (b) shows a state where there is little solder, and FIG. 3 (c) shows a state where the solder is excessive. The figure which looked at the solder from the direction parallel to a board | substrate is shown.

  As shown in these drawings, in the CH image, color information corresponding to the normal direction (inclination angle) appears in the portion of the solder 31 that is a specular object. For example, in the case of FIG. 3A, since the inclination of the solder 31 becomes gentler as the distance from the component electrode 30, a hue change of B → G → R appears in the area of the solder 31. Further, as can be seen from FIG. 1, the part where the specular reflection direction with respect to the optical axis of the camera 110 is close to 90 degrees (the part below the blue light source 111B) and the part close to 0 degrees (the opening part where the camera 110 is disposed). Since there is no light source, black appears in the portion where the inclination of the solder 31 is close to vertical and the portion close to horizontal. As shown in FIG. 3A to FIG. 3C, the shape, width, order of appearance, and the like of the R, G, B, and black color regions vary depending on the surface shape of the solder 31. Further, since diffuse reflection is dominant on the surface of the component main body and the electrode 30, the same color of the object itself as when illuminated with white light appears instead of the light source color such as R, G, B. Accordingly, only the solder region is extracted from the CH image, and the inverse problem is solved based on the shape, width, and order of each of the R, G, B, and black regions, thereby restoring the three-dimensional shape of the solder 31. It is possible.

(Measurement of phase information according to the height of the solder surface)
In the present embodiment, a so-called phase shift method is used to measure (measure) the height of the solder surface.

  The phase shift method is one of methods for measuring three-dimensional information (height information) on an object surface by analyzing the distortion of the pattern when pattern light is projected onto the object surface. Specifically, photographing is performed with the camera 110 in a state where a predetermined pattern (for example, a striped pattern whose luminance changes in a sine wave shape) is projected onto the substrate using the projection device 112. Then, as shown in FIG. 4, pattern distortion corresponding to the unevenness appears on the object surface. At this time, specular reflection is dominant on the solder surface with high glossiness, so the pattern cannot be observed correctly, but the diffuse reflection component increases at the portion where the glossiness is reduced due to the influence of flux, etc. Is possible. By repeating this process a plurality of times (for example, four times) while changing the phase of the luminance change of the pattern light, a plurality of images having different luminance characteristics (hereinafter referred to as phase images) are obtained as shown in FIG. It is done. Since the brightness (luminance) of the same pixel in each image should change in the same cycle as the change in the stripe pattern, the phase of each pixel can be obtained by applying a sine wave to the change in brightness of each pixel. I understand. The distance (height) from the reference position can be calculated by obtaining a phase difference with respect to the phase of a predetermined reference position (table surface, substrate surface, etc.).

  In the present embodiment, the phase shift method is used, but other methods may be used as long as the height information of the diffused object can be obtained. Examples of methods for obtaining height information by projecting striped or grid pattern light onto an object and analyzing the image of the pattern distortion include a light cutting method, a fringe analysis method, and a spatial coding method.

(Solder shape measurement and inspection process)
Next, the flow of solder shape measurement and inspection processing performed in the board inspection apparatus 1 will be described with reference to FIGS. 5, 6, and 7. 5 and 6 are flowcharts showing the flow of processing, and FIG. 7 is an example of an image for schematically explaining the procedure of solder shape measurement.

  First, the control device 12 controls the stage 10 according to the inspection program, and moves the solder to be inspected to the measurement position (the visual field of the camera 110) (step S500). Then, the control device 12 turns on the illumination device 111 (step S501), and performs imaging with the camera 110 in a state where the red light RL, the green light GL, and the blue light BL are irradiated (step S502). The obtained image data (CH image in FIG. 3) is taken into the information processing apparatus 13 by the image acquisition unit 20. After the illumination device 111 is turned off, the control device 12 projects pattern light from the projection device 112 (step S503), and the camera 110 captures an image (step S504). When the phase shift method is used, the processes in steps S503 and S504 are executed a plurality of times while changing the phase of the pattern light. In the present embodiment, four imaging operations are performed while changing the phase of the pattern light by 90 degrees, and four pieces of image data are acquired. The obtained plural pieces of image data (phase image in FIG. 4) are taken into the information processing apparatus 13 by the image acquisition unit 20. In the present embodiment, the photographing with the illumination device 111 is performed first, but the photographing with the projection device 112 may be performed first. In addition, when another inspection object exists outside the field of view of the camera 110, the processes in steps S500 to S504 may be repeatedly executed.

  The subsequent processing is performed by the information processing apparatus 13. Details of the process of measuring the three-dimensional shape of the solder surface (step S505) are shown in the flow of FIG.

  First, processing for a CH image will be described. The color information analysis unit 21 extracts a solder region image (hereinafter, also referred to as “first solder region image” or simply “first image”) from the CH image (step S600). For example, the color information analysis unit 21 specifies the land area in the CH image by acquiring information on the position and size of the land on the substrate from the inspection program, and also identifies the component (diffuse object) area from the CH image. The region obtained by recognizing and excluding the component region from the land region can be extracted as the first solder region image. Reference numeral 70 in FIG. 7 is an example of a first solder region image.

  Next, the color information analysis unit 21 applies multi-value processing to the first solder region image 70 (step S601). Multi-value processing is processing for segmenting an image into regions for each hue. Since only the red light RL, the green light GL, and the blue light BL are emitted from the illumination device 111, if the solder surface is an ideal mirror surface, as shown in FIG. The area should be separated cleanly. In practice, however, diffuse reflection components and irregular reflection components due to minute irregularities (micro facets) on the solder surface generate color mixing and noise, so the boundary between the R, G, and B hues is often ambiguous. The multi-value process is a pre-process for removing such color mixture and noise between hues.

For example, after the color information analysis unit 21 converts the first solder region image 70 into an image in the HSV color space,
・ Pixel with lightness (V) smaller than predetermined value → Black pixel ・ Pixel with hue (H) 0 ° ± 60 degrees → R pixel ・ Pixel with hue (H) 120 degrees ± 60 degrees → B pixel ・ Hue (H ) Is converted into a quaternary value such as a pixel of 240 ° ± 60 ° → G pixel, and a noise removal process such as expansion / contraction is performed to generate a multi-value image 71. The multi-valued image 71 obtained in this way has a black area (a part where the inclination angle is almost horizontal), a red area (a part where the inclination angle is small), a green area (an inclination angle) in order from the side where the inclination of the solder surface is gentle. Is an intermediate portion), a blue region (a portion with a large inclination angle), and a black region (a portion with a substantially vertical inclination angle). In addition, when the surface of the land (pad) on a board | substrate is contained in the 1st solder area | region image 70, it divides | segments into six area | regions including the area | region of a land. The correspondence relationship between each color region and the inclination angle range of the solder surface is geometrically determined from the positional relationship among the camera 110, the solder, and the light sources 111R, 111G, and 111B.

Subsequently, the color reliability calculation unit 22 calculates the reliability of the color information of each pixel (step S602). In this embodiment, after converting the first solder region image 70 into an image in the HSV color space,
-Pixels whose saturation (S) is equal to or higher than the threshold T1 → reliability = 1 (high)
-Pixels with saturation (S) smaller than threshold T1 → reliability = 0 (low)
As described above, the reliability is determined based on the saturation. This is because the stronger the specular reflection component, the higher the saturation. Reference numeral 72 in FIG. 7 is an example of a color information reliability map (black pixels have reliability = 1 (high) and white pixels have reliability = 0 (low)). It can be seen that a portion with high reliability (a portion with high glossiness) and a portion with low reliability (a portion with low glossiness) are mixed in the surface of the solder.

  Next, processing for the phase image will be described. The phase information analysis unit 23 extracts a solder region image (hereinafter, also referred to as “second solder region image” or simply “second image”) from each of the four phase images (step S603). 73 in FIG. 7 is an example of a second solder region image. And the phase information analysis part 23 calculates | requires the height of each pixel by analyzing the phase of the luminance change of the same pixel of the four 2nd solder area | region images 67 (step S604). 7 in FIG. 7 is a height map that expresses the height (Z position) of each pixel of the second solder region image 67 by a pixel value.

Subsequently, the phase reliability calculation unit 24 calculates the reliability of the phase information of each pixel (step S605). In the present embodiment, the amount of change in luminance of the same pixel between the four second solder region images 73 (difference between the minimum luminance and the maximum luminance) is calculated,
-Pixels whose luminance change is equal to or greater than the threshold value T2 → Reliability = 1 (high)
-Pixel whose luminance change amount is smaller than threshold T2 → Reliability = 0 (low)
Thus, the reliability of the phase information is determined. This is because the amount of change in luminance increases as the pattern light becomes brighter. Reference numeral 75 in FIG. 7 is an example of a phase information reliability map (black pixels have reliability = 1 (high) and white pixels have reliability = 0 (low)). It can be seen that a highly reliable portion (low glossy portion) and a low reliability portion (high glossy portion) are mixed in the solder surface.

  In the present embodiment, the phase image processing is described after the CH image processing, but the order of the two processings may be either first or in parallel. The threshold values T1 and T2 may be set as appropriate based on experiments using sample images. As the threshold value T1, the same threshold value may be used for all hues, or the threshold value may be changed according to the hue. For example, if irregular reflection occurs due to the influence of flux, the light source color (red in this embodiment) that enters from above the solder tends to be observed by the camera, so the threshold value for the reddish hue is set to other values. You may make it larger than the threshold value with respect to a hue.

Next, the solder shape measuring unit 25 refers to the color information reliability map 72 and the phase information reliability map 75 to compare the color information reliability and the phase information reliability of each pixel. If a pixel whose phase information reliability is higher than the color information reliability is detected, the solder shape measuring unit 25 determines from the height map 74 that pixel and its neighboring pixels (for example, adjacent pixels). The respective height information is acquired, the amount of change in height (that is, the gradient) at the pixel is calculated, and the inclination angle is obtained. As described above, since the correspondence relationship between the tilt angle and the hue is known, the tilt angle obtained from the height map 74 can be converted into a color (R, G, B, or black). Then, the solder shape measuring unit 25 replaces the pixel value (color) of the pixel in the multi-value image 71 with the color obtained from the height map 74. By performing the above processing for all the pixels of the multi-valued image 71, the value (color) of the pixel having the higher reliability of the phase information than the color information among the pixels of the multi-valued image 71 is based on the phase information. Correction is performed (step S606). Reference numeral 76 in FIG. 7 indicates a multi-valued image after correction.

  Thereafter, the solder shape measuring unit 25 estimates the inclination angle (gradient) of each pixel (that is, each point on the solder surface) from the corrected multi-value image 76, and stitches them together to obtain the three-dimensional shape of the solder. Is restored (step S607). The data of the three-dimensional shape calculated in this way is stored, for example, in the form of a height map (77 in FIG. 7).

  Returning to the flow of FIG. 5, the inspection unit 26 uses the three-dimensional shape data of the solder obtained in step S505 to inspect the solder joint state (step S506). At this time, based on the three-dimensional shape data, the shape of the solder joint to the part or land can be captured three-dimensionally and accurately, so that a higher-accuracy inspection than before can be performed.

  Finally, the result output unit 28 displays an image representing the three-dimensional shape of the solder obtained in step S505 and the inspection result obtained in step S506 on the display device (step S507). At this time, various images, maps (see FIG. 7), measurement values, and the like obtained in the process described above may be displayed on the display device. Further, the inspection program may be corrected (teaching) on this display screen.

(Advantages of this embodiment)
According to the configuration of the present embodiment described above, two types of images (first solder region image and second solder region image) captured by different illumination methods such as the color highlight illumination method and the phase shift method are used. Then, the reliability of the color information and the reliability of the phase information are obtained, and the information with the higher reliability is used for generating the three-dimensional shape of the solder. In principle, the color highlight illumination method can be expected to be more accurate as the gloss of the solder is higher, and the phase shift method can be expected to be more accurate in principle as the gloss of the solder is lower. Therefore, if one of the two methods having higher reliability (one that can be expected to be accurate) is preferentially used as in this embodiment, the solder surface can be used regardless of the glossiness (mirror surface) of the solder surface. It is possible to accurately measure the three-dimensional shape.

  More specifically, if a part of the surface of the solder has a reduced gloss due to the influence of flux, etc., the color information of that part can be inaccurate (indicating an incorrect tilt angle). And lowers the accuracy of reconstruction of the three-dimensional shape. Therefore, due to the decrease in glossiness, the reliability of color information decreases, and conversely, for pixels with high reliability of phase information, the color of the multilevel image is corrected based on the tilt angle calculated from the phase information. . As a result, the color information (inclination angle information) of the part with reduced glossiness can be supplemented with phase information, and even if the part with high glossiness and the part with low glossiness are mixed, the three-dimensional shape of the entire solder surface Can be restored accurately.

  In addition, the information obtained by different illumination methods is aggregated into one measurement value called the height of the solder surface, which makes it easier to design inspection logic and judgment criteria using that measurement value. There are also advantages.

<Other embodiments>
The description of the above embodiment is merely illustrative of the present invention, and the present invention is not limited to the above specific form. The present invention can be variously modified within the scope of its technical idea.

For example, in the above-described embodiment, a configuration is used in which a multi-valued image obtained from color information is corrected based on phase information. Conversely, a configuration in which a height map obtained from phase information is corrected based on color information is also used. Good. In this case, the processing flow of FIGS. 5 and 6 is generally the same, but the content of the correction processing in step S606 may be changed as follows. That is, the solder shape measuring unit 25 refers to the reliability map 72 of the color information and the reliability map 75 of the phase information, and compares the reliability of the color information and the reliability of the phase information of each pixel. If a pixel whose color information reliability is higher than the phase information reliability is detected, the solder shape measuring unit 25 obtains the inclination angle of the pixel and its neighboring pixels from the multi-valued image 71. . Then, the solder shape measuring unit 25 corrects the height information of the pixel and its neighboring pixels in the height map 74 based on the information on the inclination angle obtained from the multi-value image 71. For example, the inclination angle (gradient) may be accumulated on the basis of the height of a pixel with high reliability in the height map 74. By performing the above processing for all the pixels of the height map 74, the value (height) of the pixels having higher reliability of color information than the phase information among the pixels of the height map 74 is included in the color information. Based on correction. The height map after this correction becomes the data of the three-dimensional shape of the solder. Even in such a process, the three-dimensional shape of the entire solder surface can be accurately restored regardless of the level of gloss, as in the above embodiment.

  In the above embodiment, three color light sources of R, G, and B are used as the color highlight illumination method. However, the number of light source colors may be more than three colors, and the arrangement order of the light sources is also arbitrary. is there. In the above embodiment, the phase shift method is used as a method for obtaining the phase information. However, other illumination methods may be used as long as the method is effective when the glossiness is low.

  In the above embodiment, the reliability of the color information and the reliability of the phase information are both binary. However, the reliability can be expressed by a multivalue or a continuous value. The method for obtaining the index used as the reliability is not limited to that in the above embodiment, and any index may be used as long as it is correlated with the level of gloss (mirror surface) of the solder surface.

1: substrate inspection device 10: stage, 11: measurement unit, 12: control device, 13: information processing device, 14: display device 20: image acquisition unit, 21: color information analysis unit, 22: color reliability calculation unit, 23: Phase information analysis unit, 24: Phase reliability calculation unit, 25: Solder shape measurement unit, 26: Inspection unit, 27: Inspection program storage unit, 28: Result output unit 110: Camera, 111: Illumination device, 111B: Blue light source, 111G: green light source, 111R: red light source, 112: projection device RL: red light, BL: blue light, GL: green light, PL: pattern light

Claims (9)

A measuring device for measuring the three-dimensional shape of solder,
A plurality of colors of light are irradiated at different incident angles, and a first image and pattern light, which are captured in a state where color information corresponding to the inclination angle appears on the surface of the solder, are projected onto the solder. An image acquisition unit that acquires a second image captured in a state in which phase information of a pattern corresponding to the height appears on the surface of
For each point on the surface of the solder, a color reliability calculation unit for determining the reliability of color information in the first image;
For each point on the surface of the solder, a phase reliability calculation unit for determining the reliability of the phase information in the second image,
By obtaining the three-dimensional information of each point on the surface of the solder using the color information in the first image and the phase information in the second image, which has higher reliability, the three-dimensional information of the solder A solder shape measuring unit for generating a shape;
A measuring apparatus comprising: The measurement apparatus according to claim 1, further comprising a result output unit that displays an image representing a three-dimensional shape of the solder obtained by the solder shape measurement unit on a display device. The color reliability calculation unit obtains the reliability of color information of a point on the surface of the solder corresponding to the pixel based on the saturation or brightness of the pixel of the first image. Or the measuring apparatus of 2. The second image is a plurality of images picked up by changing the phase of the pattern light,
The phase reliability calculation unit obtains the reliability of phase information of a point on the surface of the solder corresponding to the pixel based on the amount of change in luminance of the same pixel between the plurality of images. The measuring device according to any one of claims 1 to 3. A color information analysis unit that generates a multi-value image in which each pixel value represents a different inclination angle range by performing multi-value processing for dividing the first image into regions for each hue;
A phase information analysis unit that obtains the height information of the surface of the solder corresponding to each pixel from the phase information of the second image,
The solder shape measuring unit is
Of the pixels of the multi-valued image, the pixel value of the pixel having higher reliability of the phase information than the color information is calculated from the height information of the pixel obtained by the phase information analysis unit and its neighboring pixels. Correct based on the tilt angle,
The measuring apparatus according to claim 1, wherein a three-dimensional shape of the solder is generated using the corrected multi-valued image. A color information analysis unit that generates a multi-value image in which each pixel value represents a different inclination angle range by performing multi-value processing for dividing the first image into regions for each hue;
A phase information analysis unit that obtains the height information of the surface of the solder corresponding to each pixel from the phase information of the second image,
The solder shape measuring unit is
Among the height information of each pixel obtained by the phase information analysis unit, the height information of the pixel having higher reliability of color information than the phase information is obtained from the inclination angle of the pixel obtained from the multi-valued image. Corrected based on information,
5. The measuring device according to claim 1, wherein the three-dimensional shape of the solder is generated using the height information of each pixel after the correction. The measuring device according to any one of claims 1 to 6,
Using the data of the three-dimensional shape of the solder acquired by the measuring device, an inspection unit for inspecting the bonding state of the solder to the component or the board,
A board inspection apparatus comprising: A control method for a measuring device for measuring the three-dimensional shape of solder,
A plurality of colors of light are irradiated at different incident angles, and a first image and pattern light, which are captured in a state where color information corresponding to the inclination angle appears on the surface of the solder, are projected onto the solder. Acquiring a second image captured in a state in which phase information of a pattern corresponding to its height appears on the surface of
For each point on the surface of the solder, determining the reliability of the color information in the first image;
Determining the reliability of the phase information in the second image for each point on the surface of the solder;
By obtaining the three-dimensional information of each point on the surface of the solder using the color information in the first image and the phase information in the second image, which has higher reliability, the three-dimensional information of the solder Generating a shape; and
A control method for a measuring apparatus, comprising:   A program for causing a computer to execute each step of the method for controlling a measuring apparatus according to claim 8.

Images