WO2021153057A1

WO2021153057A1 - Three-dimensional shape measurement device, three-dimensional shape measurement method, and program

Info

Publication number: WO2021153057A1
Application number: PCT/JP2020/046679
Authority: WO
Inventors: 心平藤井; 貴茂田中
Original assignee: オムロン株式会社
Priority date: 2020-01-28
Filing date: 2020-12-15
Publication date: 2021-08-05
Also published as: DE112020006612T5; CN114867985A; JP7459525B2; JP2021117157A

Abstract

This three-dimensional shape measurement device comprises: a projection means which projects, to an object to be measured, pattern light from a plurality of directions in which angles around a vertical axis with respect to the subject to be measured are different; an image-capturing means which captures an image of the subject to be measured; and a measurement means which measures a three-dimensional shape of the subject to be measured on the basis of the image captured by the image-capturing means, wherein the image-capturing means acquires a plurality of pattern-projected images which are a plurality of images obtained by projecting the pattern light from the plurality of directions, and in which reflection modes such as the intensity of the pattern light are different at a portion inclined within the subject to be measured, and the measurement means measures the direction of the inclination on the basis of the differences between the images with a prescribed feature amount at each pixel forming the plurality of pattern-projected images.

Description

3D shape measuring device, 3D shape measuring method and program

The present invention relates to a three-dimensional shape measuring device, a three-dimensional shape measuring method, and a program.

Conventionally, in the technical field of inspecting the solder joint state of parts mounted on a printed circuit board, a method of measuring a three-dimensional shape by a so-called phase shift method has been known. The phase shift method obtains (photographs) a plurality of images in which pattern light is projected onto the surface of an object by changing the phase, and analyzes the distortion of the pattern in the plurality of images to obtain a three-dimensional shape of the surface of the object. It is one of the restoration methods. When the printed circuit board is measured by using such a phase shift method, there is a problem that the light irradiated to a member having a high mirror surface such as solder is positively reflected, which adversely affects the measurement accuracy.

On the other hand, for example, Patent Document 1 discloses a substrate inspection apparatus that combines the above-mentioned phase shift method with three-dimensional shape measurement of a mirrored object by a so-called color highlighting method. The color highlighting method irradiates the substrate with light of a plurality of colors (wavelengths) at different incident angles, and the color characteristics (in the normal reflection direction when viewed from the camera) correspond to the normal direction of the solder surface. This is a method of capturing the three-dimensional shape of the solder surface as two-dimensional hue information by performing imaging with the color of the light source appearing. As a result, the accuracy of the three-dimensional shape measurement can be improved by measuring the mirror surface portion of the printed circuit board by color highlighting and measuring the diffused object such as resin by the phase shift method.

Japanese Unexamined Patent Publication No. 2016-11857

By the way, even with the inspection device described in Patent Document 1, there is a problem that it is difficult to determine the "direction" of the inclination of the surface of the measurement target. That is, in the inspection of the printed circuit board, the inclination of the solder surface is either piled up with respect to the electrode (so-called wet) or low with respect to the electrode (so-called non-wet state). It becomes difficult to determine accurately.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique for accurately discriminating the direction of inclination included in a measurement target in three-dimensional shape measurement.

In order to achieve the above object, the present invention adopts the following configuration.

The three-dimensional shape measuring device according to the present invention includes a projection means for projecting pattern light onto a measurement target from a plurality of directions having different angles around the vertical axis with respect to the measurement target, and a photographing means for photographing the measurement target. A three-dimensional shape measuring device including a measuring means for measuring a three-dimensional shape of a measurement target based on an image taken by the photographing means, wherein the photographing means has pattern light from the plurality of directions. Is a plurality of projected images, and a plurality of pattern projected images having different modes of reflection including the intensity of the pattern light at a portion having an inclination in the measurement target are acquired, and the measuring means obtains the plurality of patterns. It is characterized in that the direction of the inclination is measured based on the difference between each image of a predetermined feature amount in each pixel constituting the projected image.

Note that "measurement" here includes measurement by calculation (the same applies hereinafter). Further, the "pattern" is, for example, a striped pattern in which the change in brightness shows periodicity, and the phase can be changed in time. Further, the "slope" here means not only the slope defined by a straight line with respect to the horizontal plane but also the slope in a broad sense defined by a curve. Further, the slope including the slope is also understood to include a slope in a broad sense including not only a flat surface but also a curved surface. Hereinafter, the same meaning will be used in the present specification.

It should be noted that "a plurality of pattern projection images having different reflection modes including the intensity of pattern light" does not mean a plurality of images in which the same pattern is irradiated from the same direction but only in different phases. It shows a plurality of images in which the pattern light is reflected in different modes because the pattern light is irradiated from a plurality of different directions.

With such a configuration, it is possible to measure the direction of the inclination included in the measurement target, so that the three-dimensional shape of the inclined portion can be measured with high accuracy. The predetermined feature amount may be, for example, the brightness of each pixel.

Further, the projection means projects the pattern light onto the measurement target from at least a plurality of different positions facing the measurement target on one circumference centered on the measurement target. , The pattern light may be projected from the plurality of directions. With such a configuration, it is possible to suppress the occurrence of a shadow portion that is not irradiated with the pattern light in the measurement target.

Further, the projection means is rotatably arranged in the circumferential direction around the measurement target, and is directed to the measurement target from a plurality of different positions on one circumference centered on the measurement target. By projecting the pattern light, the pattern light may be projected from the plurality of directions.

According to such a configuration, the pattern light can be irradiated to the measurement target from a desired position on the circumference, and the pattern light is irradiated from the optimum direction regardless of the orientation and shape of the predetermined portion of the measurement target. can do. Further, a plurality of the projection means are arranged at a plurality of different positions on one circumference centering on the measurement target, and each of the projection means projects the pattern light onto the measurement target, whereby the pattern light is projected from the plurality of directions. May be projected.

Further, the three-dimensional shape measuring device further includes an illuminating means for irradiating the measurement object with illumination light having a different wavelength from a plurality of different angles between the vertical direction and the horizontal direction. The photographing means further acquires an illumination light irradiation image of the measurement target irradiated with the illumination light, and the measurement means reflects an aspect including either the intensity or the wavelength of the illumination light in the illumination light irradiation image. Therefore, the degree of the inclination may be further measured.

With such a configuration, it is possible to perform three-dimensional shape measurement by the color highlighting method, and it is possible to accurately measure the degree of inclination included in the measurement target. Further, the accuracy of the measurement can be improved by combining the three-dimensional shape measurement by the color highlighting method and the phase shift method.

Further, the three-dimensional shape measuring device further includes an image display means for displaying the illumination light irradiation image acquired by the photographing means and a special image created from the plurality of pattern projection images. The special image has a different color and the direction of the inclination obtained based on the difference between the presence / absence and degree of the inclination obtained from the illumination light irradiation image and the feature amount of each pixel constituting the plurality of pattern projection images. / Or the image may be displayed and separated so as to be identifiable by the pattern.

According to such a configuration, the user can recognize the three-dimensional shape of the measurement target measured by the three-dimensional measuring device by checking the two-dimensional image. It is also possible to adjust the settings of the three-dimensional measuring device by comparing the image with the actual measurement target.

Further, the three-dimensional shape measuring method according to the present invention includes a projection step of projecting pattern light from a plurality of directions having different angles around the vertical axis with respect to the measurement target, and photographing the measurement target. It is a three-dimensional shape measuring method including a shooting step and a measurement step of measuring the three-dimensional shape of the measurement target based on the image shot in the shooting step. In the shooting step, the plurality of directions are described. A plurality of images in which pattern light is projected from the above, and a plurality of pattern projection images having different modes of reflection including the intensity of the pattern light at a portion having an inclination in the measurement target are acquired, and in the measurement step, the said The direction of the inclination is measured based on the difference between the images having a predetermined feature amount in each pixel constituting the plurality of pattern projection images.

Further, in the projection step, the pattern light is projected from the plurality of directions by projecting the pattern light onto the measurement target from at least a plurality of different positions facing the measurement target. You may.

Further, the three-dimensional shape measuring method further includes an illumination step of irradiating the measurement target with illumination light having a different wavelength from a plurality of different angles between the vertical direction and the horizontal direction. In the photographing step, the illumination light irradiation image of the measurement target irradiated with the illumination light is further acquired, and in the measurement step, the mode of reflection including either the intensity or the wavelength of the illumination light in the illumination light irradiation image. From the above, the degree of inclination may be further measured.

Further, the three-dimensional shape measuring method further includes an image display step of displaying the illumination light irradiation image acquired in the shooting step and a special image created from the plurality of pattern projection images. The special image has a different color and the direction of the inclination obtained based on the difference between the presence / absence and degree of the inclination obtained from the illumination light irradiation image and the feature amount of each pixel constituting the plurality of pattern projection images. / Or the image may be displayed and separated so as to be identifiable by the pattern.

Further, the present invention can be regarded as a program for causing the three-dimensional shape measuring device to execute the above method, and as a computer-readable recording medium in which such a program is recorded non-temporarily.

Further, each of the above configurations and processes can be combined with each other to construct the present invention as long as no technical contradiction occurs.

According to the present invention, it is possible to provide a technique for accurately discriminating the direction of inclination included in a measurement target in three-dimensional shape measurement.

FIG. 1 is a schematic view showing a configuration of a three-dimensional shape measuring device according to an application example of the present invention. FIG. 2 is a flowchart showing a flow of three-dimensional shape measurement processing of the three-dimensional shape measuring device according to the application example of the present invention. FIG. 3 is a schematic view showing a hardware configuration of the substrate inspection apparatus according to the first embodiment. FIG. 4 is a block diagram illustrating a function of the information processing apparatus according to the first embodiment. FIG. 5 is a plan view illustrating the configuration of the lighting device according to the first embodiment. FIG. 6A is a side view for explaining the soldered portion of the substrate to be measured. FIG. 6B is a diagram illustrating a color highlight image of the soldered portion of the substrate. FIG. 6C is a diagram illustrating a color highlight correction image of the soldered portion of the substrate. FIG. 7 is a flowchart showing the flow of the substrate inspection process of the substrate inspection apparatus according to the first embodiment.

Hereinafter, examples of the present invention will be described with reference to the drawings.

<Application example>
(Configuration of application example)
The present invention can be applied to, for example, a three-dimensional shape measuring device as shown in FIG. FIG. 1 is a schematic view showing a configuration of a three-dimensional shape measuring device 9 according to this application example. The three-dimensional shape measuring device 9 is a device for measuring the three-dimensional shape of the measurement target O, and as shown in FIG. 1, the main configurations are

projectors

91a and 91b as projection means, a camera 92 as a photographing means, and a measuring means. The information processing device 93 (for example, a computer) is provided. The measurement target O includes a three-dimensional portion OP having an inclination.

The

projectors

91a and 91b are means for projecting a pattern onto a measurement target. Here, the pattern is, for example, a striped pattern in which the change in brightness shows periodicity, and the phase can be changed in time. In this application example, the pattern projected from the projector 91a is referred to as pattern a, and the pattern projected from the projector 91b is referred to as pattern b. The

projectors

91a and 91b are arranged so as to have a constant inclination angle with respect to the measurement target O, respectively.

The camera 92 is a means for photographing the measurement target O in a state where the pattern is projected and outputting a digital image. In the following, the image taken by the photographing means is also referred to as an observation image. The camera 92 includes, for example, an optical system and an image sensor. As shown in FIG. 1, the camera 92 is arranged so as to shoot the measurement target O from directly above the measurement target O. The projector 91a and the projector 91b are arranged at positions facing each other along the circumferential direction with the camera 92 as the center.

The information processing device 93 has functions such as control of

projectors

91a and 91b, a camera 92 and a transport mechanism, processing of an image captured from the camera 92, and three-dimensional shape measurement, and corresponds to the measuring means in the present invention. .. The information processing device 93 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a non-volatile storage device (for example, a hard disk drive, a flash memory, etc.), an input device (for example, a keyboard, a mouse, a touch panel, etc.), and a display. It can be configured by a computer equipped with a device (eg, a liquid crystal display, etc.).

When measuring the three-dimensional shape of the object O to be measured by the three-dimensional shape measuring device 9 having the above configuration, a plurality of images are captured by a camera while changing the phase of the pattern projected from each projector onto the object O to be measured. The three-dimensional shape of the measurement target O is measured by the information processing apparatus 93 processing the captured image by, for example, the phase shift method or the like.

(Function of information processing device)
Subsequently, the functions related to the three-dimensional shape measurement of the information processing apparatus 93 will be described. The information processing device 93 is a functional module related to three-dimensional shape measurement, which includes an image acquisition unit 931, a height data calculation unit 932, a feature amount extraction unit 933, a feature amount comparison unit 934, a composite data creation unit 935, and a three-dimensional shape measurement. It has a part 936.

The image acquisition unit 931 is a function of capturing a plurality of observation images used for three-dimensional shape measurement from the camera 92. For example, a projector can display four images in which the phases of patterns projected on the measurement object O differ by 1/4 π. It is acquired by each of the projection pattern of 91a and the projection pattern of the projector 91b. In this application example, the projection pattern of the projector 91a is referred to as a pattern a, and the observed image obtained by capturing the pattern a is referred to as an observed image a. Further, the projection pattern of the projector 91b is referred to as a pattern b, and the observation image obtained by capturing the pattern b is referred to as an observation image b.

The height data calculation unit 932 is a function of calculating the height data of the measurement object O based on the acquired plurality of observed images. For example, based on the two-dimensional phase difference of the pixels representing the position of one point on the surface of the object O to be measured between the four acquired observation images, the height of the point is determined by the observation image a and the observation image b. Ask for each. In this application example, the height data calculated from the observation image a will be described as height data a, and the height data calculated from the observation image b will be described as height data b.

The feature amount extraction unit 933 extracts the feature amount (for example, the brightness value) possessed by each pixel of the observed image from each acquired observation image. In this application example, the feature amount data extracted from the observation image a will be described as the feature amount data a, and the feature amount data extracted from the observation image b will be described as the feature amount data b.

The feature amount comparison unit 934 compares the feature amount data a extracted by the feature amount extraction unit 933 with the feature amount data b. More specifically, in each observation image, the value of the feature amount of the pixels representing the same location of the measurement target O is compared, and the pixel with the larger value is either the observation image a or the observation image b. Identify if it is a pixel. Here, when the feature amount is luminance, a brighter image, that is, an image in which more light enters the camera 92 is specified. Then, when there is a portion having an inclination in the measurement target O, the pixels of the image on the side where the pattern light is irradiated from the direction facing the inclination become brighter, so that the direction of the inclination is changed from here. It becomes possible to identify. In this way, the feature amount comparison unit 934 generates data related to the identification of the inclination direction (hereinafter, inclination direction data).

The composite data creation unit 935 creates composite data for three-dimensional shape measurement from the height data a and b and the tilt direction data. Specifically, the profiles of the height data a and b are combined by a predetermined method, averaged, etc. to generate the combined height data, which is corrected by the inclination direction data. To create profile data of three-dimensional shape. As a result, it is possible to obtain profile data for three-dimensional shape measurement in which the orientation of the inclined portion of the measurement target O is specified.

The three-dimensional shape measurement unit 936 measures the three-dimensional shape of the measurement target O based on the profile data created by the composite data creation unit 935.

(Flow of 3D shape measurement processing)
Next, the procedure of three-dimensional shape measurement in this application example will be described with reference to FIG. First, the information processing device 93 controls the projector 91a and projects the pattern a onto the measurement target O from the first direction on the circumference centered on the measurement target O (step S901). Next, the information processing apparatus 93 controls the camera 92 to take a picture of the measurement target O in a state where the pattern a light is irradiated from the first direction, and acquires the observation image a (step S902).

Next, the information processing device 93 controls the projector 91b and projects the pattern b onto the measurement target O from the second direction facing the first direction with the measurement target O in between (step S903). Next, the information processing device 93 controls the camera 92 to take a picture of the measurement target O in a state where the pattern b is projected from the second direction, and acquires the observation image b (step S904).

Next, the information processing apparatus 93 calculates the height data a and b from the acquired observation images a and b (step S905), and further extracts the feature amount possessed by the pixels of each image (step S906). The information processing apparatus 93 subsequently compares the extracted feature amounts and creates tilt direction data using pixels having a larger feature amount (step S907). Then, synthetic data for measuring the three-dimensional shape of the measurement target O is generated from the obtained height data a and b and the tilt direction (step S908), and the measurement target including the tilt portion OP is generated based on the composite data. The three-dimensional shape measurement of O is performed (step S909), and a series of processes is completed. The measurement result may be displayed on a display device (not shown). Further, the feature amount extraction does not necessarily have to be performed from the observed images a and b for calculating the height data a and b, and the image data for feature amount extraction may be separately acquired.

With the configuration of the three-dimensional shape measuring device 9 according to the present application example as described above, when there is a portion OP having an inclination in the measurement target O, the height is increased based on the three-dimensional shape profile data in which the direction of the inclination is specified. Accurate three-dimensional shape measurement becomes possible.

<Embodiment 1>
Next, the substrate inspection apparatus 1 which is another example of the embodiment for carrying out the present invention will be described. However, unless otherwise specified, the dimensions, materials, shapes, relative arrangements, and the like of the components described in this embodiment are not intended to limit the scope of the present invention to those.

(Hardware configuration of board inspection equipment)
The overall configuration of the substrate inspection apparatus according to the embodiment of the present invention will be described with reference to FIG. FIG. 3 is a schematic view showing the hardware configuration of the substrate inspection device. This substrate inspection device 1 is used for substrate appearance inspection (for example, inspection of solder joint state after reflow) in a surface mounting line.

The board inspection device 1 mainly includes a stage 10, a measurement unit 11, a control device 12, an information processing device 13, and a display device 14. The measurement unit 11 includes a camera 110, a lighting device 111, and a pattern projection device (projector) 112.

The stage 10 is a mechanism for holding the substrate K and aligning the component KB and the solder KH to be inspected with the measurement position of the camera 110. As shown in FIG. 3, when the X-axis and the Y-axis are taken parallel to the stage 10 and the Z-axis is taken perpendicular to the stage 10, the stage 10 can translate at least two axes in the X direction and the Y direction. The camera 110 is arranged so that the optical axis is parallel to the Z axis, and images the substrate K on the stage 10 from vertically above. The image data captured by the camera 110 is taken into the information processing device 13.

The lighting device 111 (111R, 111G, 111B) is a lighting means for irradiating the substrate K with illumination light of a different color (wavelength). FIG. 3 schematically shows an XZ cross section of the illuminating device 111. In reality, the illuminating device 111 is annular so that light of the same color can be illuminated from all directions (all directions around the Z axis). Or it has a dome shape. The

projectors

112a and 112b are pattern projection means for projecting pattern light having a predetermined pattern on the substrate K. The projector 112 projects pattern light through an opening provided in the middle of the lighting device 111. In the present embodiment, the two projectors 112 are arranged at diagonal positions with the substrate K in between, but more projectors may be provided. Both the lighting device 111 and the projector 112 are lighting systems used when the substrate K is photographed by the camera 110, but the lighting device 111 is used for the purpose of measuring the surface shape of a mirror object such as solder, and the projector 112 is It is used for the purpose of measuring the surface shape of diffused objects such as parts.

The control device 12 is a control means for controlling the operation of the substrate inspection device 1, and is a control means for controlling the movement of the stage 10, lighting and dimming control of the lighting device 111, lighting control and pattern change of the projectors 112a and 112, and imaging of the camera 110. It is in charge of control.

The information processing device 13 is a device having a function of acquiring various measured values related to the component KB and the solder KH and inspecting the state of the solder joint of the component KB by using the image data captured from the camera 110. The display device 14 is a device that displays the measured values and inspection results obtained by the information processing device 13. The information processing device 13 can be composed of, for example, a general-purpose computer having a CPU, RAM, a non-volatile storage device, and an input device. Although the control device 12, the information processing device 13, and the display device 14 are shown in separate blocks in FIG. 3, they may be configured by separate devices or may be configured by a single device. good.

(Functional configuration)
FIG. 4 is a block diagram showing a configuration of a functional module related to inspection processing provided by the information processing apparatus 13. These functional modules are realized by the CPU of the information processing device 13 reading and executing a program stored in the auxiliary storage device. However, all or part of the functions may be configured by a circuit such as an ASIC or FPGA.

The image acquisition unit 131 is a function module that acquires image data from the camera 110. The solder shape measuring unit 132 is a functional module that restores the three-dimensional shape of a mirrored object part such as solder from two-dimensional image data, and the part shape measuring unit 133 is a diffused object part such as a part from two-dimensional image data. It is a functional module that restores the three-dimensional shape of. The image data and the restoration algorithm used in each restoration process will be described later.

The inspection unit 134 measures various indexes related to the shapes of the solder KH and the component KB based on the three-dimensional shape data obtained by the solder shape measurement unit 132 and the component shape measurement unit 133, and uses these measured values. It is a functional module that inspects the state of solder joints. The inspection program storage unit 135 is a functional module that stores an inspection program that defines inspection items and conditions in the inspection unit 134. In the inspection program, for example, the position and size of the land to be inspected, the size of the part, the type of the index to be measured, the judgment reference value for each index (threshold value and range for judging non-defective product and defective product), etc. are defined. ing. The output processing unit 136 is a functional module that externally outputs the measured values and inspection results obtained by the inspection unit 134, the three-dimensional shape of the parts KB and the solder KH, and the like to the display device 14 and the like.

Hereinafter, the flow of the inspection process of the information processing apparatus 13 will be described after explaining the method of restoring the three-dimensional shape of the solder KH (mirror object) and the method of restoring the three-dimensional shape of the component KB (diffusing object).

(Three-dimensional shape measurement of solder)
An image obtained by the so-called color highlighting method is used to measure the three-dimensional shape of the solder KH. The color highlighting method irradiates a substrate with light of a plurality of colors (that is, wavelengths) at different incident angles, and causes the solder surface to have color characteristics according to its normal direction (that is, a normal reflection direction when viewed from a camera). This is a method of capturing the three-dimensional shape of the solder surface as two-dimensional hue information by taking a picture with the color of the light source in the above appearing. Extracting only the region where the light source colors of R, G, and B appear from the image, and restoring the three-dimensional solder shape based on the shape, width, and order of each region of R, G, and B. Can be done. Since a known method can be used for the restoration of the three-dimensional shape, detailed description thereof will be omitted here.

First, the configuration of the lighting device 111 used for the color highlighting method will be described with reference to FIG. FIG. 5 is a schematic plan view schematically showing the arrangement relationship of the

light sources

111R, 111G, and 111B of the lighting device 111. The lighting device 111 has a structure in which three annular light sources, a red light source 111R, a green light source 111G, and a blue light source 111B, are arranged concentrically around the optical axis of the camera 110. The elevation angles and orientations of the

light sources

111R, 111G, and 111B are adjusted so that the incident angles with respect to the substrate K increase in the order of red light, green light, and blue light. Such a lighting device 111 can be formed, for example, by arranging LEDs of each color R, G, and B in an annular shape on the outside of a dome-shaped diffuser plate.

When the substrate K is photographed by the camera 110 with the lighting device 111 turned on, a color feature corresponding to the normal direction (tilt angle) appears in the portion of the solder KH which is a mirror object. For example, when the inclination of the solder KH becomes gentler as the distance from the component electrodes increases, a change in hue of B → G → R appears in the region of the solder KH. The shape, width, appearance order, etc. of the regions of each of R, G, and B change depending on the surface shape of the solder KH.

With reference to FIG. 6, an image of a solder portion that can be obtained when a substrate is photographed by the camera 110 with the lighting device 111 according to the present embodiment turned on will be described. FIG. 6A is a side view of an electrode portion of a component KB on a substrate K and a solder portion (hereinafter, referred to as a soldered portion) joined to the electrode portion. As shown in FIG. 6A, the solder KH of the case is not sufficiently familiar with the electrode extending from the component KB, and the contact area is small (that is, it is in a non-wetting state).

FIG. 6B is an image of the soldered portion (hereinafter referred to as a color highlight image) when the lighting device 111 is turned on and photographed by the camera 110. FIG. 6B shows regions of each color of R, G, and B. From the image shown in FIG. 6B, it can be inferred that there is a slope (a slope defined by) at the joint between the solder and the electrode, but which direction this slope is facing (that is, whether it is wet). It is not possible to determine whether it is in a non-wet state). Since diffuse reflection is dominant on the surface of the component KB body and the electrode, the color of the object itself as when illuminated with white light appears instead of the light source color such as R, G, and B.

(Three-dimensional shape measurement of parts)
On the other hand, the phase shift method is used to measure the three-dimensional shape of the component KB, which is a diffusion object. The phase shift method is one of the methods for restoring the three-dimensional shape of the object surface by analyzing the distortion of the pattern when the pattern light is projected onto the object surface. Specifically, the projectors 112a and 112 are used to project a predetermined pattern (for example, a striped pattern in which the brightness changes in a sinusoidal shape) onto a substrate, and the camera 110 takes a picture. Then, on the surface of the substrate K, the distortion of the pattern corresponding to the unevenness appears. By repeating this process a plurality of times while changing the phase of the brightness change of the pattern light, a plurality of images having different brightness characteristics (hereinafter, referred to as a pattern analysis image) can be obtained. Since the brightness of the same pixel in each image should change in the same cycle as the change in the striped pattern, the phase of each pixel can be known by applying a sine wave to the change in the brightness of each pixel. Then, by obtaining the phase difference with respect to the phase of a predetermined reference position (table surface, substrate surface, etc.), the distance (that is, height) from the reference position can be calculated. The pattern projected by the projector 112a is referred to as a pattern a, the pattern projected by the projector 112b is referred to as a pattern b, and the images obtained by capturing each pattern are referred to as an observation image a and an observation image b.

Here, in the present embodiment, the information processing device 13 has acquired the observation image a and the observation image b, and the projectors 112a and b are provided at diagonal positions with the substrate K in between. The observed image a and the observed image b have patterns projected from opposite directions. From this, as in the case of the application example, the feature amount data is extracted from the observation image a and the observation image b and compared to specify the inclination direction of the inclined portion such as the soldered portion. , Tilt direction specific data can be obtained. Since the tilt direction specific data can be generated by the same method as in the application example, detailed description thereof will be omitted.

Then, by correcting the profile data obtained from the above-mentioned color highlight image with the inclination direction identification data, it becomes possible to determine the direction of the slope of the joint portion between the solder KH and the electrode.

FIG. 6C shows an example of a special image (hereinafter referred to as a color highlight correction image) created based on the composite data created in this way. In the color highlight correction image, the inclination in the basic direction (the direction in which more faces are facing) is displayed in the same red (R), green (G), and blue (B) as the original color highlight image. However, the inclination in the direction opposite to the basic direction is represented by magenta (M), yellow (Y), and cyan (C). The color selection here is completely arbitrary and is not limited to these six colors, but the degree of inclination is the same by using similar colors even though they have different color systems. You can intuitively understand that the direction of inclination is different.

If such a color highlight correction image can be displayed on the display device 14, for example, the user can easily grasp the state of the soldered portion by looking at the color highlight correction image.

(Flow of inspection process)
Next, the flow of the inspection process performed by the substrate inspection apparatus 1 will be described with reference to FIG. 7. FIG. 7 is a flowchart showing the flow of the inspection process.

First, the control device 12 controls the stage 10 according to the inspection program, and moves the component KB and the solder KH to be inspected to the measurement position (the field of view of the camera 110) (step S101). Then, the control device 12 turns on the lighting device 111 (step S102), and executes a shooting process by the camera 110 in a state of irradiating red light, green light, and blue light (step S103). The obtained image data (color highlight image) is taken into the information processing device 13 by the image acquisition unit 131.

Next, the control device 12 projects the pattern light from the projector 112a (step S104), and the camera 110 takes an image (step S105). Further, the pattern light is projected from the projector 112b (step S106), and the camera 110 takes an image (step S107). When the phase shift method is used, the processes of steps S104 to S107 are executed a plurality of times while changing the phase of the pattern light. The obtained plurality of image data are taken into the information processing device 13 by the image acquisition unit 131. In the present embodiment, the shooting with the lighting device 111 is executed first, but the shooting with the projector 112 may be executed first. Further, when another inspection target exists outside the field of view of the camera 110, the processes of steps S101 to S107 may be repeatedly executed.

After that, the processing is performed by the information processing device 13. The component shape measuring unit 133 restores the three-dimensional shape of the component KB from the observation images a and b obtained in steps S105 and S107 by the phase shift method (step S108). The restored three-dimensional shape data is stored, for example, in the form of image data (called a height map) in which the height (Z position) of each pixel is expressed by a pixel value.

The solder shape measuring unit 132 restores the three-dimensional shape of the solder KH (and the electrodes of the component KB) from the color highlight image obtained in step S103 (step S109). Further, the solder shape measuring unit 132 extracts the luminance value from the observed images a and b obtained by photographing the pattern light, and generates the tilt direction specifying data (step S110). The restored three-dimensional shape data is stored, for example, in the form of a height map in which the height (Z position) of each pixel in the solder KH region is expressed by a pixel value.

Then, by synthesizing these height maps and tilt direction specific data, it is possible to obtain an overall height map representing the height information of both the solder KH which is a mirror object and the component KB which is a diffusion object. Here, the overall height map is corrected by specifying the direction of the slope.

Then, the inspection unit 134 inspects the substrate K based on the overall height map and the threshold value of the inspection program (step S111). When the inspection is completed, the display device 14 displays the result of the inspection and the color highlight composite image visually representing the composite data created in step S108 (step S112), and ends the series of processes.

According to the substrate inspection apparatus of the present embodiment described above, the three-dimensional shape of the solder, which is a mirror object, and the component electrode, which is a diffusion object, are restored by a method suitable for each, so that both the solder and the component electrode are restored. It is possible to obtain highly accurate three-dimensional shape data. In addition, when restoring the three-dimensional shape of the solder, the orientation of the inclined surface at the inclined surface is specified by performing correction based on the inclined direction specifying data obtained by projecting the pattern from a plurality of directions, and the third order is obtained. Since the original shape is restored, the shape of the slope of the solder can be restored with high accuracy.

<Others>
The above-described embodiment is merely an example of the present invention, and the present invention is not limited to the above-mentioned specific embodiment. The present invention can be modified in various ways within the scope of its technical idea. For example, in the above-described first embodiment, the three-dimensional shape of the mirrored object is measured based on the color highlight image, and the three-dimensional shape of the diffused object is measured based on the pattern projection image. It is not necessary to do so, and two profile data for measuring the shape of the entire measurement target may be created based on each image, and then these data may be combined. Further, in the first embodiment, the projector is fixed, but the projector may be configured to be rotatable around the vertical axis. In such a case, the number of projectors can be one.

Further, in the above embodiment, the color highlight correction image makes it possible to identify the degree and direction of the inclination by color coding, but these can be identified not only by the color but also by the difference in the pattern such as hatching. It may be an image to be displayed, or an image in which a difference in color and a difference in pattern are combined.

<Additional notes>
One aspect of the present invention is a projection means (91a, 91b) that projects pattern light onto a measurement target (O) from a plurality of directions having different angles around the vertical axis with respect to the measurement target, and the measurement target. A three-dimensional shape measuring device comprising a photographing means (92) for photographing and a measuring means (93) for measuring the three-dimensional shape of the measurement target based on an image photographed by the photographing means.
The photographing means is a plurality of images in which pattern light is projected from the plurality of directions, and a plurality of pattern projection images having different modes of reflection including the intensity of the pattern light at a portion having an inclination in the measurement target. Acquired,
The three-dimensional shape measuring device is characterized in that the measuring means measures the direction of inclination based on the difference between each image of a predetermined feature amount in each pixel constituting the plurality of pattern projection images. Is.

Further, another aspect of the present invention includes a projection step (S901, S903) for projecting pattern light onto a measurement target from a plurality of directions having different angles around the vertical axis with respect to the measurement target, and the measurement target. This is a three-dimensional shape measuring method including a photographing step (S902 and S904) for photographing the image and a measuring step (S909) for measuring the three-dimensional shape of the measurement target based on the image photographed by the photographing means. In the shooting step, a plurality of images in which pattern light is projected from the plurality of directions, and a plurality of patterns are projected having different modes of reflection including the intensity of the pattern light at a portion having an inclination in the measurement target. A three-dimensional image is acquired, and in the measurement step, the direction of the inclination is measured based on the difference between each image of a predetermined feature amount in each pixel constituting the plurality of pattern projection images. This is a measurement method for the original shape.

1 ... Board inspection device 9 ... Three-dimensional shape measuring device 10 ... Stage 11 ...

Inspection unit

110, 92 ... Camera 111, 91 ... Lighting device 12 ...

Control device

13, 93 ・・・ Information processing device 14 ・・・ Display device K ・・・ Board O ・・・ Measurement target

Claims

A projection means that projects pattern light onto a measurement target from a plurality of directions having different angles around the vertical axis with respect to the measurement target.
A shooting means for shooting the measurement target and
A three-dimensional shape measuring device including a measuring means for measuring a three-dimensional shape of a measurement target based on an image taken by the photographing means.
The photographing means is a plurality of images in which pattern light is projected from the plurality of directions, and a plurality of pattern projection images having different modes of reflection including the intensity of the pattern light at a portion having an inclination in the measurement target. Acquired,
The measuring means measures the direction of inclination based on the difference between each image of a predetermined feature amount in each pixel constituting the plurality of pattern projection images.
A three-dimensional shape measuring device characterized by this.
The projection means projects the pattern light onto the measurement target from at least a plurality of different positions facing the measurement target on one circumference centered on the measurement target. Projecting pattern light from multiple directions,
The three-dimensional shape measuring device according to claim 1, wherein the three-dimensional shape measuring device is characterized in that.
The projection means is rotatably arranged in the circumferential direction around the measurement target, and pattern light is directed to the measurement target from a plurality of different positions on one circumference centered on the measurement target. By projecting, the pattern light is projected from the plurality of directions.
The three-dimensional shape measuring device according to claim 1 or 2, wherein the three-dimensional shape measuring device is characterized in that.
A plurality of the projection means are arranged at a plurality of different positions on one circumference centering on the measurement target, and each projects the pattern light onto the measurement target to project the pattern light from the plurality of directions. do,
The three-dimensional shape measuring device according to claim 1 or 2, wherein the three-dimensional shape measuring device is characterized in that.
Further, it has an illuminating means for irradiating the measurement target with illumination light having a different wavelength from a plurality of different angles between the vertical direction and the horizontal direction.
The photographing means further acquires an illumination light irradiation image of the measurement target irradiated with the illumination light, and further obtains the illumination light irradiation image.
The measuring means further measures the degree of inclination from the aspect of reflection including either the intensity or the wavelength of the illumination light in the illumination light irradiation image.
The three-dimensional shape measuring device according to any one of claims 1 to 4, wherein the three-dimensional shape measuring device is characterized in that.
It further has an image display means for displaying the illumination light irradiation image acquired by the photographing means and a special image created from the plurality of pattern projection images.
The special image has different colors in the direction of the inclination obtained based on the difference between the presence / absence and degree of the inclination obtained from the illumination light irradiation image and the feature amount of each pixel constituting the plurality of pattern projection images. And / or an image that is identifiable by a pattern,
The three-dimensional shape measuring device according to claim 5, wherein the three-dimensional shape measuring device is characterized in that.
A projection step of projecting pattern light onto a measurement target from a plurality of directions having different angles around the vertical axis with respect to the measurement target.
The shooting step of shooting the measurement target and
A three-dimensional shape measuring method including a measuring step for measuring a three-dimensional shape of a measurement target based on an image taken in the shooting step.
In the shooting step, a plurality of images in which pattern light is projected from the plurality of directions, and a plurality of pattern projection images having different modes of reflection including the intensity of the pattern light at a portion having an inclination in the measurement target are displayed. Acquired,
In the measurement step, the direction of the inclination is measured based on the difference between the images having a predetermined feature amount in each pixel constituting the plurality of pattern projection images.
A three-dimensional shape measurement method characterized by this.
In the projection step, the pattern light is projected from the plurality of directions by projecting the pattern light onto the measurement target from at least a plurality of different positions facing the measurement target.
The three-dimensional shape measuring method according to claim 7, wherein the three-dimensional shape measuring method is characterized in that.
It further has an illumination step of irradiating the measurement target with illumination light having a different wavelength from a plurality of different angles between the vertical direction and the horizontal direction.
In the shooting step, an illumination light irradiation image of the measurement target irradiated with the illumination light is further acquired.
In the measurement step, the degree of inclination is further measured from the aspect of reflection including either the intensity or the wavelength of the illumination light in the illumination light irradiation image.
The three-dimensional shape measuring method according to claim 7 or 8, wherein the three-dimensional shape measuring method is characterized in that.
It further has an image display step of displaying the illumination light irradiation image acquired in the shooting step and a special image created from the plurality of pattern projection images.
The special image has different colors in the direction of the inclination obtained based on the difference between the presence / absence and degree of the inclination obtained from the illumination light irradiation image and the feature amount of each pixel constituting the plurality of pattern projection images. And / or an image that is identifiable by a pattern,
The three-dimensional shape measuring method according to claim 9, wherein the three-dimensional shape measuring method is characterized in that.
A program for causing a three-dimensional shape measuring device to execute each step according to any one of claims 7 to 10.