JPWO2020039575A1 - 3D measuring device, 3D measuring method - Google Patents

Abstract

The imaging camera 31 has a high-sensitivity pixel Ph having a spectral sensitivity characteristic SP (G) having a high sensitivity and a spectral sensitivity characteristic SP having a low sensitivity with respect to light having a wavelength of λg irradiated to the solder B (object). It includes a low-sensitivity pixel Pl having (R) and SP (B). Therefore, of the surface of the solder B, the pattern light L (S) reflected in the high reflection region Ah can be converted into an appropriate pixel value V by the low-sensitivity pixel Pl, and the pattern light L (S) reflected in the low reflection region Al ( S) can be converted into an appropriate pixel value V by the high-sensitivity pixel Ph. That is, both the high reflection region Ah and the pattern light L (S) reflected by the low sensitivity pixel Pl can be converted into an appropriate pixel value V. In this way, even when the high reflection region Ah and the low sensitivity pixel Pl are mixed in the solder B, it is possible to acquire accurate pixel values V for both regions Ah and Al.

Description

The present invention relates to a technique for measuring a three-dimensional shape of an object.

Patent Document 1 describes a three-dimensional measuring device that measures a three-dimensional shape of an object such as solder printed on an electrode of a printed circuit board. This three-dimensional measuring device irradiates an object with white light and measures the three-dimensional shape of the object based on an image obtained by capturing the light reflected by the object. Further, in such a three-dimensional measuring device, an imaging camera having a plurality of pixels that output pixel values according to the intensity of the incident light can be used for imaging the light reflected by the object.

Japanese Patent No. 4256059

It should be noted that each pixel of the imaging camera has a dynamic range, and it is not possible to output an accurate pixel value for light darker or brighter than the dynamic range. Therefore, when the reflectance of the object is low, the intensity of the light irradiating the object is increased, while when the reflectance of the object is high, the intensity of the light irradiating the object is decreased. Can be considered. However, when a region with low reflectance and a region with high reflectance coexist in the object, accurate pixel values cannot be obtained for the region with high reflectance when the light intensity is high, and reflection is obtained when the light intensity is low. Accurate pixel values cannot be obtained for regions with low reflectance. Therefore, such a method was not always effective.

The present invention has been made in view of the above problems, and even when a region having low reflectance and a region having high reflectance coexist in an object, it is possible to obtain accurate pixel values for both regions. The purpose is to provide the technology.

The three-dimensional measuring device according to the present invention has a projector that irradiates an object with light having a predetermined wavelength, and a plurality of pixels on which the light reflected by the object is incident, and each of the plurality of pixels is of the incident light. It is equipped with an imaging camera that outputs pixel values according to the intensity and a control unit that calculates the shape of the three-dimensional shape of the object based on the pixel values. Includes a plurality of low-sensitivity pixels having a spectral sensitivity characteristic in which the ratio of the output to the input of a predetermined wavelength is low as compared with the spectral sensitivity characteristic of the camera.

The three-dimensional measurement method according to the present invention corresponds to a step of irradiating an object with light having a predetermined wavelength and light reflected by the object incident on a plurality of pixels according to the intensity of the incident light on the plurality of pixels. It includes a step of outputting a pixel value and a step of executing the shape calculation of a three-dimensional shape of an object based on the pixel value. It includes a plurality of low-sensitivity pixels having spectral sensitivity characteristics in which the ratio of output to input of a predetermined wavelength is relatively low.

In the present invention (three-dimensional measuring device, three-dimensional measuring method) configured in this way, the imaging camera has a ratio of an output to an input of a predetermined wavelength in comparison with the spectral sensitivity characteristics of the high-sensitivity pixel. Has a low sensitivity pixel having a low spectral sensitivity characteristic. That is, a high-sensitivity pixel having a high-sensitivity spectral sensitivity characteristic with respect to light of a predetermined wavelength irradiating an object, and a low-sensitivity pixel having a low-sensitivity spectral sensitivity characteristic as compared with a high-sensitivity out-of-image. Is provided. Therefore, among the objects, the light reflected in the region with high reflectance can be converted into an appropriate pixel value by the low-sensitivity pixel, and the light reflected in the region with low reflectance can be converted into an appropriate pixel by the high-sensitivity pixel. Can be converted to a value. That is, both the light reflected in the high reflectance region and the light reflected in the low reflectance region can be converted into appropriate pixel values. In this way, even when a region having a low reflectance and a region having a high reflectance coexist in the object, it is possible to acquire accurate pixel values for both regions.

Further, the three-dimensional measuring device may be configured so that the high-sensitivity pixels and the low-sensitivity pixels are arranged alternately. In such a configuration, since the high-sensitivity pixels and the low-sensitivity pixels are uniformly arranged while being adjacent to each other, the light reflected with a high reflectance can be accurately captured by the high-sensitivity pixels and the light reflected with a high reflectance. Can be accurately captured with low-sensitivity pixels. As a result, even when a region having a low reflectance and a region having a high reflectance coexist in the object, it is possible to acquire accurate pixel values for both regions.

Further, the three-dimensional measuring device may be configured so that the plurality of pixels include the high-sensitivity pixels and the low-sensitivity pixels in the same ratio. In such a configuration, it is possible to convert the light reflected in the high reflectance region and the light reflected in the low reflectance region into appropriate pixel values without being biased to one of them. As a result, even when a region having a low reflectance and a region having a high reflectance coexist in the object, it is possible to acquire accurate pixel values for both regions.

Further, the predetermined wavelength is a green wavelength, and the plurality of pixels are arranged by a Bayer array in which red, green and blue are arranged in a predetermined pattern, and each of the plurality of high-sensitivity pixels is a green pixel in the Bayer array. Therefore, the three-dimensional measuring device may be configured so that the plurality of low-sensitivity pixels include the same number of blue pixels and the same number of red pixels in the Bayer array. In such a configuration, the green high-sensitivity pixels and the red or blue low-sensitivity pixels are uniformly arranged adjacent to each other, so that the light reflected with low reflectance is accurately captured by the high-sensitivity pixels and highly reflected. The light reflected by the rate can be accurately captured by the low-sensitivity pixels. As a result, even when a region having a low reflectance and a region having a high reflectance coexist in the object, it is possible to acquire accurate pixel values for both regions.

Note that the pixel value obtained by converting the light reflected in the high reflectance region and incident on the high-sensitivity pixel or the light reflected in the low reflectance region and incident on the low-sensitivity pixel may be inappropriate. high. Therefore, the control unit executes a determination process for determining whether or not the pixel value output from the pixel is appropriate based on the pixel value for each of the plurality of pixels, and executes shape calculation based on the determination result in the determination process. As described above, a three-dimensional measuring device may be configured. In such a configuration, shape calculation can be performed using an appropriate pixel value while suppressing the influence of an inappropriate pixel value. Therefore, even when a region having a low reflectance and a region having a high reflectance coexist in the object, it is possible to accurately calculate the three-dimensional shape of the object.

In addition, the control unit executes interpolation based on the pixel values of the low-sensitivity pixels located within a predetermined range from the high-sensitivity pixels, or determines the high-sensitivity pixels whose pixel values are determined to be inappropriate in the determination process. For low-sensitivity pixels whose pixel values are determined to be inappropriate by processing, shape calculation is executed based on the result of interpolation by the pixel values of high-sensitivity pixels located within a predetermined range from the low-sensitivity pixels. , A three-dimensional measuring device may be configured. In such a configuration, the pixel values of the pixels determined to be inappropriate in the determination process can be interpolated by the pixel values of the pixels located within a predetermined range from the pixels, and the shape calculation can be executed based on the result. As a result, even when a region having a low reflectance and a region having a high reflectance coexist in the object, it is possible to accurately calculate the three-dimensional shape of the object.

Further, the projector irradiates the object with light having a plurality of striped patterns having a predetermined wavelength and different phases, and the control unit performs a shape calculation by the phase shift method. May be configured. In such a configuration, even when a region having a low reflectance and a region having a high reflectance are mixed in the object, the three-dimensional shape of the object can be appropriately calculated by the phase shift method.

According to the present invention, it is possible to obtain accurate pixel values for both regions even when the region having low reflectance and the region having high reflectance coexist in the object.

The block diagram which schematically exemplifies the appearance inspection apparatus which concerns on this invention. The figure which shows typically the structure of the image pickup unit. The figure which shows typically the spectral sensitivity characteristic of a pixel which an image pickup unit has. The figure which shows typically the relationship between the light reflected in each of a high reflection region and a low reflection region, and a pixel value. A flowchart showing an example of three-dimensional measurement performed by a visual inspection device. FIG. 5 is a flowchart showing an example of interpolation of unsuitable pixels executed in the three-dimensional measurement of FIG. The figure explaining the content of the operation executed by the three-dimensional measurement of FIG. The figure which shows an example of the interpolation of the unsuitable pixel.

FIG. 1 is a block diagram schematically illustrating a visual inspection apparatus according to the present invention. In the figure and the following figures, XYZ Cartesian coordinates composed of the Z direction parallel to the vertical direction, the X direction parallel to the horizontal direction, and the Y direction are appropriately shown. The visual inspection device 1 of FIG. 1 is in a state of solder B for joining a component (electronic component) to a substrate 10 (printed circuit board) by controlling a conveyor 2, an inspection head 3, and a drive mechanism 4 by a control device 100. Inspect the quality.

The transport conveyor 2 transports the substrate 10 along a predetermined transport path. Specifically, the transport conveyor 2 carries the substrate 10 before inspection to the inspection position in the visual inspection device 1 and holds the substrate 10 horizontally at the inspection position. When the inspection of the substrate 10 at the inspection position is completed, the transfer conveyor 2 carries the inspected substrate 10 out of the visual inspection apparatus 1.

The inspection head 3 has an imaging camera 31 that images the inside of the imaging field of view V31 from above, and the solder B of the substrate 10 carried into the inspection position is housed in the imaging field of view V31 and imaged by the imaging camera 31. The image pickup camera 31 has a flat plate-shaped image pickup unit 311 that captures the reflected light from the solder B. Details of the imaging unit 311 will be described later with reference to FIG. Further, the inspection head 3 has a projector 32 that projects a striped pattern light L (S) whose light intensity distribution changes in a sinusoidal shape onto an imaging field of view V31. The projector 32 has a light source such as an LED (Light Emitting Diode) and a digital micromirror device that reflects the light from the light source toward the imaging field of view V31. By adjusting the angle of each micromirror of the digital micromirror device, the projector 32 can project a plurality of types of pattern lights L (S) having different phases to the imaging field of view V31. That is, the inspection head 3 takes an image with the imaging camera 31 while changing the phase of the pattern light L (S) projected from the projector 32, and thereby, the three-dimensional shape Bs of the solder B in the imaging field of view V31 by the phase shift method. Can be measured.

Incidentally, the inspection head 3 has eight projectors 32 (in FIG. 1, two projectors 32 are represented as representatives for simplification of illustration). The eight projectors 32 are arranged so as to surround the periphery of the image pickup camera 31, and are arranged at equal pitches in a circle around the vertical direction Z. Then, each projector 32 projects the pattern light L (S) from diagonally above the image field of view V31 of the image pickup camera 31. Therefore, the pattern light L (S) can be projected onto the imaging field of view V31 from one of the plurality of projectors 32, which has an appropriate positional relationship with the solder B.

The drive mechanism 4 supports the inspection head 3 and drives the inspection head 3 in the horizontal direction and the vertical direction by a motor. By driving the drive mechanism 4, the inspection head 3 can move above the solder B to capture the solder B in the imaging field of view V31, and can measure the three-dimensional shape Bs of the solder B in the imaging field of view V31.

The control device 100 has a main control unit 110 which is a processor composed of a CPU (Central Processing Unit) and a memory, and the inspection is executed when the main control unit 110 controls the control of each unit of the device. .. Also. The control device 100 has a user interface 200 composed of input / output devices such as a display, a keyboard, and a mouse, and the user can input a command to the control device 100 via the user interface 200 or control the control device 100. You can check the inspection result by. Further, the control device 100 includes a projection control unit 120 that controls the projector 32, an image pickup control unit 130 that controls the image pickup camera 31, and a drive control unit 140 that controls the drive mechanism 4. When the transfer conveyor 2 carries the substrate 10 to the inspection position, the main control unit 110 controls the drive mechanism 4 by the drive control unit 140 to move the inspection head 3 above the solder B of the substrate 10. As a result, the solder B fits within the imaging field of view V31 of the imaging camera 31.

Subsequently, the main control unit 110 uses the imaging camera 31 to image the pattern light L (S) projected on the imaging field V31 while projecting the pattern light L (S) from the projector 32 onto the imaging field V31 including the solder B. (Pattern imaging operation). Specifically, the main control unit 110 has a storage unit 150 composed of a non-volatile memory, and reads out the projection pattern T (S) stored in the storage unit 150. Then, the main control unit 110 projects the angle of each micromirror of the digital micromirror device of the projector 32 by controlling the projection control unit 120 based on the projection pattern T (S) read from the storage unit 150. Adjust according to the pattern T (S). In this way, the pattern light L (S) having the projection pattern T (S) is projected on the imaging field of view V31. Further, the main control unit 110 controls the image pickup control unit 130 to take an image of the pattern light L (S) projected on the image pickup field of view V31 by the image pickup camera 31 and acquire the image capture image I (S). The captured image I is stored in the storage unit 150. The storage unit 150 stores four types of projection patterns T (S) having different phases by 90 degrees from each other, and the pattern imaging operation is executed four times while changing the projection pattern T (S) ( S = 1, 2, 3, 4). As a result, four types of captured images I (S) in which the pattern lights L (S) having different phases by 90 degrees are captured are acquired.

The main control unit 110 obtains the height of the imaging field of view V31 for each pixel of the imaging camera 31 from the four types of captured images I (S) thus acquired by the phase shift method. As a result, the height of the surface of the solder B can be obtained for each pixel of the image pickup camera 31.

FIG. 2 is a diagram schematically showing the configuration of the imaging unit. The image pickup unit 311 has, for example, a solid-state image sensor 312 such as a CCD (Charge Coupled Device) image sensor, and a color filter 313 superimposed on the solid-state image sensor 312. The solid-state image sensor 312 has a plurality of light receiving pixels Pi arranged at a constant array pitch ΔP in each of the X direction and the Y direction. That is, in the solid-state image sensor 312, a plurality of light receiving pixels Pi are two-dimensionally arranged. Further, the color filter 313 has a plurality of filter pixels Pf arranged at an array pitch ΔP in each of the X direction and the Y direction. That is, in the color filter 313, a plurality of filter pixels Pf are arranged two-dimensionally.

In this way, the plurality of light receiving pixels Pi and the plurality of filter pixels Pf are provided in a one-to-one correspondence relationship, and the light receiving pixels Pi and the filter pixels Pf corresponding to each other face each other. In other words, in the image pickup unit 311, the pixel Px is composed of the light receiving pixels Pi and the filter pixels Pf facing each other, and the plurality of pixels Px are arranged in the X direction and the Y direction at the arrangement pitch ΔP, respectively. Then, each pixel Px outputs a pixel value V (FIG. 3) corresponding to the intensity of the light transmitted through the filter pixel Pf and incident on the light receiving pixel Pi from the light receiving pixel Pi.

As shown in FIG. 2, in the color filter 313, a plurality of filter pixels Pf are arranged according to a Bayer array, and each filter pixel Pf is among red (R), green (G), and blue (B). Allows the transmission of light of a color according to the arrangement position, and limits the transmission of light of a color different from the color according to the arrangement position. Therefore, each pixel Px of the image pickup unit 311 has a spectral sensitivity characteristic according to the color of light that the filter pixel Pf allows transmission.

FIG. 3 is a diagram schematically showing the spectral sensitivity characteristics of the pixels of the imaging unit. In FIG. 3, in a graph showing a wavelength of light on the horizontal axis and a pixel value V on the vertical axis, a pixel Px having a filter pixel Pf that allows transmission of red (R) (hereinafter referred to as “red pixel Px”). The spectral sensitivity characteristic SP (R) of (appropriately referred to as) is indicated by a two-point chain line, and the spectrum of a pixel Px (hereinafter, appropriately referred to as “green pixel Px”) having a filter pixel Pf that allows transmission of green (G). The sensitivity characteristic SP (G) is indicated by a broken line, and the spectral sensitivity characteristic SP (B) of the pixel Px (hereinafter, appropriately referred to as “blue pixel Px”) having the filter pixel Pf that allows the transmission of blue (B) is It is indicated by a single point chain line. Further, in FIG. 3, the wavelength distribution of the pattern light L (S) projected from the projector 32 is also shown by a solid line.

That is, in the present embodiment, the pattern light L (S) has a wavelength distribution having a peak at the green wavelength λg (in other words, has a green emission spectrum). On the other hand, the green pixel Px has a spectral sensitivity characteristic SP (G) having high sensitivity to the wavelength λg of the pattern light L (S). The red pixel Px has a spectral sensitivity characteristic SP (R) having a peak at a wavelength longer than the wavelength λg, and has a lower sensitivity than the green pixel Px with respect to the wavelength λg of the pattern light L (S). The blue pixel Px has a spectral sensitivity characteristic SP (B) having a peak at a wavelength shorter than the wavelength λg, and has a lower sensitivity than the green pixel Px with respect to the wavelength λg of the pattern light L (S).

That is, as shown in FIG. 2, among the plurality of pixels Px included in the image pickup unit 311, the green pixel Px functions as a high-sensitivity pixel Ph showing high sensitivity to the wavelength λg, and the red pixel Px and the blue pixel Px Each of the pixels Px functions as a low-sensitivity pixel Pl that exhibits a sensitivity lower than that of the high-sensitivity pixel Ph with respect to the wavelength λg. Then, the high-sensitivity pixel Ph (green pixel Px) and the low-sensitivity pixel Pl (red pixel Px) are alternately arranged in the Y direction, and the high-sensitivity pixel Ph (green pixel Px) and the low-sensitivity pixel Pl (blue pixel Px) are arranged alternately. Pixels Px) are arranged alternately in the X direction. In this way, the low-sensitivity pixels Pl (red pixels Px) are adjacent to the high-sensitivity pixels Ph (green pixels Px) on both sides in the Y direction, and the low-sensitivity pixels Ph (green pixels Px) are adjacent to the high-sensitivity pixels Ph (green pixels Px). Pl (blue pixel Px) is adjacent on both sides in the X direction. In other words, the high-sensitivity pixel Ph is adjacent to the low-sensitivity pixel Ph from four directions, and the low-sensitivity pixel Pl is adjacent to the high-sensitivity pixel Ph from four directions. Here, the fact that the pixels Px are adjacent means that the two target pixels Px are arranged at the array pitch ΔP.

In such a configuration, as shown in FIG. 4, both the light reflected in the high reflectance region Ah having a high reflectance and the light reflected in the low reflection region Al having a low reflectance on the surface of the solder B are combined. , It can be converted into an accurate pixel value V by the pixel Px.

FIG. 4 is a diagram schematically showing the relationship between the light reflected in each of the high reflection region and the low reflection region and the pixel value. In the figure, the pattern light L (S) which is a sine wave is projected onto each of the high reflection region Ah and the low reflection region Al, and the light reflected by each region Ah and Al is red (R) and green (G). ), The pixel value V output by these pixels Px when detected by the blue (B) pixels Px is schematically shown. Actually, since the pixel value V output from the pixel Px cannot be a value outside the dynamic range D, the waveform of the pixel value V is crushed, but here, the waveform is shown without being crushed.

The pixel value V output by the green G pixel Px (high-sensitivity pixel Ph) that detects the pattern light L (S) reflected in the high-reflection region Ah is the dynamic range D of the pixel Px (in other words, the light-receiving pixel Pi). The upper limit of the dynamic range D) is partially exceeded. Therefore, the green pixel Px cannot convert the pattern light L (S) reflected in the high reflection region Ah into an accurate pixel value V. On the other hand, the pixel value V output by the red (R) and blue (B) pixels Px (low-sensitivity pixels Pl) that detect the pattern light L (S) reflected in the high reflection region Ah is the dynamic range of the pixels Px. It fits in D. Therefore, the red (R) or blue (B) pixel Px can convert the pattern light L (S) reflected in the high reflection region Ah into an accurate pixel value V.

The pixel value V output by the red (R) and blue (B) pixels Px (low-sensitivity pixels Pl) that detect the pattern light L (S) reflected in the low-reflection region Al is the dynamic range D of the pixels Px. The lower limit is partially exceeded. Therefore, the red (R) and blue (B) pixels Px cannot convert the pattern light L (S) reflected in the low reflection region Al into an accurate pixel value V. On the other hand, the pixel value V output by the green (G) pixel Px (high-sensitivity pixel Ph) that detects the pattern light L (S) reflected in the low reflection region Al falls within the dynamic range D of the pixel Px. Therefore, the green (G) pixel Px can convert the pattern light L (S) reflected in the low reflection region Al into an accurate pixel value V.

That is, the pattern light L (S) reflected in the high reflection region Ah can be converted into an accurate pixel value V by the red (R) and blue (B) pixels Px (low-sensitivity pixels Pl), and the low-reflection region Al. The pattern light L (S) reflected by the above can be converted into an accurate pixel value V by the green (G) pixel Px (high-sensitivity pixel Ph).

FIG. 5 is a flowchart showing an example of three-dimensional measurement executed by the visual inspection apparatus, FIG. 6 is a flowchart showing an example of interpolation of unsuitable pixels executed in the three-dimensional measurement of FIG. 5, and FIG. 7 is FIG. It is a figure explaining the content of the operation executed by the three-dimensional measurement of. 5 and 6 are executed under the control of the main control unit 110.

In step S101, the pattern imaging operation of capturing the pattern light L (S) by the imaging camera 31 while projecting the pattern light L (S) on the solder B is performed while changing the phase of the pattern light L (S) by 90 degrees. By repeating the execution, four captured images I (S) having different phases by 90 degrees are acquired (S = 1, 2, 3, 4).

In step S102, the main control unit 110 calculates a three-dimensional image showing the three-dimensional shape Bs of the solder B from these captured images I (S) based on the phase shift method. Specifically, based on Equation 1 in FIG. 7, the calculation of obtaining the angle θ from the pixel values V0 to V3 of the four captured images I (S) is executed for each of the plurality of pixels Px in three dimensions. An image is obtained.

In step S103, the main control unit 110 calculates a reliability image showing the reliability of the pixel value V of each of the plurality of pixels Px. This reliability indicates whether or not the pixel value V of the pixel Px is within the dynamic range D. That is, when the pixel value V of the pixel Px is too bright or too dark, the reliability becomes low. Specifically, based on Equation 2 in FIG. 7, the reliability is calculated by executing the calculation for obtaining the reliability from the pixel values V0 to V3 of the four captured images I (S) for each of the plurality of pixels Px. An image is obtained. If a saturated pixel value (that is, a pixel value indicating "255" when represented by 8 bits) exists among these pixel values V0 to V3, it does not depend on Equation 2 in FIG. The reliability of the pixel value V is set to "0".

In step S104, the interpolation of unsuitable pixels shown in FIG. 6 is executed. In step S201, the count value N for identifying the plurality of pixels Px is reset to zero, and in step S202, the count value N is incremented. Then, it is determined whether or not the reliability of the pixel value V of the pixel Px having the count value N is equal to or greater than the threshold value (step S203). When the reliability is equal to or higher than the threshold value (when “YES” in step S203), the process returns to step S202 and the count value N is incremented.

When the reliability is less than the threshold value (when "NO" in step S203), the pixel Px located within the array pitch ΔP from the pixel Px (inappropriate pixel) of this count value N, that is, the pixel Px adjacent to the unsuitable pixel 4 It is determined whether the pixel value V of the unsuitable pixel can be interpolated by the pixel value V of the pixel Px (step S204). Specifically, when there is a pixel Px having a reliability less than the threshold value among these four pixel Px, it is determined that interpolation is not possible, and all of the pixel values V of these four pixel Px are the threshold value. If it has the above reliability, it is judged that interpolation is possible.

If interpolation is not possible (“NO” in step S204), the process returns to step S202 and the count value N is incremented. When interpolation is possible (when "YES" in step S204), the interpolation calculation is executed, and the pixel value V of the unsuitable pixel is interpolated by the pixel value V of the four pixels Px adjacent to the unsuitable pixel. (Step S205). That is, the pixel value V0 of the four adjacent pixels Px interpolates the pixel value V0 of the target pixel Px, and the pixel values V1 to V3 are also interpolated in the same manner. Such interpolation operations can be performed using well-known interpolation methods such as linear interpolation and polynomial interpolation. Further, in step S205, the angle θ is calculated from the interpolated pixel values V (V0 to V3) based on Equation 1 in FIG. 7, and the corresponding pixel Px in the three-dimensional image calculated in step S102 (that is, step S205). It is adopted as the pixel Px) that is the object of interpolation. Then, steps S202 to S205 are executed until the count value N reaches the maximum value (until “YES” in step S206).

In the embodiment described above, the imaging camera 31 has a high-sensitivity pixel Ph having a spectral sensitivity characteristic SP (G) having high sensitivity to light having a wavelength of λg irradiated on the solder B (object). It includes low-sensitivity pixels Pl having spectral sensitivity characteristics SP (R) and SP (B) having low sensitivity. Therefore, of the surface of the solder B, the pattern light L (S) reflected in the high reflection region Ah can be converted into an appropriate pixel value V by the low-sensitivity pixel Pl, and the pattern light L (S) reflected in the low reflection region Al ( S) can be converted into an appropriate pixel value V by the high-sensitivity pixel Ph. That is, both the high reflection region Ah and the pattern light L (S) reflected by the low sensitivity pixel Pl can be converted into an appropriate pixel value V. In this way, even when the high reflection region Ah and the low sensitivity pixel Pl are mixed in the solder B, it is possible to acquire accurate pixel values V for both regions Ah and Al.

Further, the high-sensitivity pixels Ph and the low-sensitivity pixels Pl are arranged alternately. In such a configuration, since the high-sensitivity pixel Ph and the low-sensitivity pixel Pl are uniformly arranged while being adjacent to each other, the pattern light L (S) reflected in the low-reflection region Al can be accurately captured by the high-sensitivity pixel Ph. The pattern light L (S) reflected in the high reflection region Ah can be accurately captured by the low-sensitivity pixels Pl. As a result, even when the high reflection region Ah and the low sensitivity pixel Pl are mixed in the solder B, it is possible to acquire accurate pixel values V for both regions Ah and Al.

Further, the high-sensitivity pixel Ph and the low-sensitivity pixel Pl are included in the same ratio. In such a configuration, the pattern light L (S) reflected in the high reflection region Ah and the pattern light L (S) reflected in the low reflection region Al are not biased, and these are converted into appropriate pixel values V. can do. As a result, even when the high reflection region Ah and the low sensitivity pixel Pl are mixed in the solder B, it is possible to acquire accurate pixel values V for both regions Ah and Al.

Further, the wavelength λg is a green wavelength, and a plurality of pixels Px are arranged in a Bayer array. Each of the plurality of high-sensitivity pixels Ph is a green pixel Px in the Bayer array, and the plurality of low-sensitivity pixels Pl include the same number of blue pixels Px and red pixels Px in the Bayer array. In such a configuration, since the green high-sensitivity pixel Ph and the red or blue low-sensitivity pixel Pl are uniformly arranged while being adjacent to each other, the pattern light L (S) reflected in the low-reflection region Al is the high-sensitivity pixel. The pattern light L (S) reflected in the high reflection region Ah can be accurately captured by the low-sensitivity pixel Pl while being accurately captured by Ph. As a result, even when the high reflection region Ah and the low sensitivity pixel Pl are mixed in the solder B, it is possible to acquire accurate pixel values V for both regions Ah and Al.

A pixel obtained by converting the pattern light L (S) reflected in the high reflection region Ah and incident on the high-sensitivity pixel Ph and the pattern light L (S) reflected in the low reflection region Al and incident on the low-sensitivity pixel Pl. The value V is likely to be inappropriate. Therefore, the main control unit 110 executes a determination process (step S203) for determining whether or not the pixel value V output from the pixel Px is appropriate based on the pixel value V for each of the plurality of pixel Px. Then, the shape calculation is executed based on the determination result in the determination process (step S203) (step S205). In such a configuration, the shape can be calculated using the appropriate pixel value V while suppressing the influence of the inappropriate pixel value V (steps S102 and S205). Therefore, even when the high reflection region Ah and the low-sensitivity pixels Pl are mixed in the solder B, it is possible to accurately calculate the three-dimensional shape Bs of the solder B.

Further, the main control unit 110 has a low-sensitivity pixel located within the range of the arrangement pitch ΔP from the high-sensitivity pixel Ph with respect to the high-sensitivity pixel Ph in which the solder B is determined to be inappropriate in the determination process (step S203). For the low-sensitivity pixel Pl that is determined to have an inappropriate pixel value V in the determination process (step S203) or the interpolation by the pixel value V of Pl is executed, the low-sensitivity pixel Pl is within the range of the array pitch ΔP. Shape calculation (steps S102 and S205) is executed based on the result of performing interpolation by the pixel value V of the positioned high-sensitivity pixel Ph. In such a configuration, the pixel value V of the unsuitable pixel Px determined to be inappropriate in the determination process (step S203) is interpolated by the pixel value V of the pixel Px located within the range of the array pitch ΔP from the unsuitable pixel Px, and the pixel value V thereof is interpolated. Shape calculation (steps S102, S205) can be executed based on the result. Therefore, even when the high reflection region Ah and the low reflection region Al are mixed in the solder B, it is possible to accurately calculate the three-dimensional shape Bs of the solder B.

Further, the projector 32 irradiates the solder B with pattern light L (S) of four projection patterns T (S) having a wavelength λg and having different phases. Then, the main control unit 110 executes the shape calculation by the phase shift method (steps S102 and S205). In such a configuration, even when the high reflection region Ah and the low reflection region Al are mixed in the solder B, the three-dimensional shape Bs of the solder B can be appropriately calculated by the phase shift method.

As described above, in the present embodiment, the visual inspection device 1 corresponds to an example of the "three-dimensional measuring device" of the present invention, the projector 32 corresponds to an example of the "projector" of the present invention, and the imaging camera 31 corresponds to the present invention. The control device 100 corresponds to an example of the "control unit" of the present invention, the pattern light L (S) corresponds to an example of the "light" of the present invention, and the projection pattern T ( S) corresponds to an example of the "striped pattern" of the present invention, the wavelength λg corresponds to an example of the "predetermined wavelength" of the present invention, and the solder B corresponds to an example of the "object" of the present invention. The shape Bs corresponds to an example of the "three-dimensional shape" of the present invention, the pixel Px corresponds to an example of the "pixel" of the present invention, and the high-sensitivity pixel Ph corresponds to an example of the "high-sensitivity pixel" of the present invention. The low-sensitivity pixel Pl corresponds to an example of the "low-sensitivity pixel" of the present invention, the arrangement pitch ΔP corresponds to an example of the "predetermined range" of the present invention, and the spectral sensitivity characteristics SP (R), SP (B), SP (G) corresponds to an example of the "spectral sensitivity characteristic" of the present invention, pixel value V corresponds to an example of the "pixel value" of the present invention, and step S203 corresponds to an example of the "determination process" of the present invention. do.

The present invention is not limited to the above embodiment, and various modifications can be made to the above as long as it does not deviate from the gist thereof. For example, it is not always necessary to arrange a plurality of pixels Px according to a Bayer array. For example, the red (R) pixel Px may be arranged instead of the blue (B) pixel Px. In this case, the pattern light L (S) having a wavelength of red (R) may be projected from the projector 32. Alternatively, the blue (B) pixel Px may be arranged instead of the red (R) pixel Px. In this case, the pattern light L (S) having a wavelength of blue (G) may be projected from the projector 32.

Further, the ratio of the number of high-sensitivity pixels Ph to the low-sensitivity pixels Pl, the arrangement pattern, and the like can be appropriately changed.

Further, the specific method for determining whether or not interpolation is possible in step S204 is not limited to the above example. That is, two pixels forming one of a pair of two pixels Px arranged in the X direction with the unsuitable pixel sandwiched between them and a pair of two pixel Px arranged in the Y direction with the inappropriate pixel sandwiched between them. If the reliability of Px is equal to or higher than the threshold value, that is, it is valid, it may be determined that interpolation is possible. In such an example, the interpolation operation in step S205 may be performed as follows. That is, when there is only one pair for which the reliability is valid, the pixel value V of the unsuitable pixel is interpolated by the average value of the pixel values V of the two pixels Px constituting the pair. When there are two pairs with effective reliability, the one with the smaller absolute value of the difference (brightness difference) of the pixel values V of the two pixels Px constituting the pair among these two pairs. The pixel value V of the unsuitable pixel is interpolated by the average value of the pixel value V of the two pixels Px forming the pair. FIG. 8 is a diagram showing an example of interpolation of unsuitable pixels. As shown in FIG. 8, the pixel value Vn of the unsuitable pixel Pxn can be interpolated by the average value of the pixel values Vg of the two pixels Pxg sandwiching the unsuitable pixel Pxn (linear interpolation). According to these methods, even when the unsuitable pixel is located at the boundary between the high reflection region Ah and the low reflection region Al, the error of the interpolated pixel value V can be suppressed. However, the interpolation operation is not limited to the linear interpolation illustrated here, and can be executed by using other well-known interpolation methods, as described above.

Further, in the above example, interpolation is performed on the pixel value V of the pixel Px of the captured image I (S) obtained by capturing the pattern light L (S). However, interpolation may be executed for the angle θ of each pixel Px calculated from the pixel values V0 to V3 of the four captured images I (S). Alternatively, interpolation may be executed for the height of each pixel Px calculated from this angle θ. In this way, the three-dimensional shape of the solder B can be calculated while interpolating unsuitable pixels based on the pixel value V having a reliability equal to or higher than the threshold value.

Further, the method of calculating the reliability is not limited to the above example. For example, the reliability may be calculated by the method described in Japanese Patent Application Laid-Open No. 2014-119442 or Japanese Patent No. 3996560.

Further, the object of the three-dimensional measurement is not limited to the solder B.

1 ... Visual inspection device (three-dimensional measuring device)
31 ... Imaging camera 32 ... Projector 100 ... Control device (control unit)
B ... Handa (object)
Bs ... Three-dimensional shape L (S) ... Pattern light (light)
Px ... Pixel Ph ... High-sensitivity pixel Pl ... Low-sensitivity pixel ΔP ... Arrangement pitch (predetermined range)
SP (R), SP (B), SP (G) ... Spectral sensitivity characteristics T (S) ... Projection pattern (striped pattern)
V ... Pixel value λg ... Green wavelength (predetermined wavelength)
S203 ... Judgment processing

Claims (8)

A projector that irradiates an object with light of a predetermined wavelength,
An imaging camera having a plurality of pixels on which the light reflected by the object is incident, and each of the plurality of pixels outputting a pixel value according to the intensity of the incident light.
A control unit that executes shape calculation of the three-dimensional shape of the object based on the pixel value is provided.
The plurality of pixels include a plurality of high-sensitivity pixels and a plurality of low-sensitivity pixels having a spectral sensitivity characteristic in which the ratio of an output to an input having a predetermined wavelength is lower than that of the spectral sensitivity characteristic of the high-sensitivity pixel. Three-dimensional measuring device. The three-dimensional measuring device according to claim 1, wherein the high-sensitivity pixels and the low-sensitivity pixels are alternately arranged. The three-dimensional measuring device according to claim 2, wherein the plurality of pixels include the high-sensitivity pixels and the low-sensitivity pixels in the same ratio. The predetermined wavelength is a green wavelength,
The plurality of pixels are arranged by a Bayer array in which red, green, and blue are arranged in a predetermined pattern.
Each of the plurality of high-sensitivity pixels is a green pixel in the Bayer array.
The three-dimensional measuring device according to claim 3, wherein the plurality of low-sensitivity pixels include the same number of blue pixels and the same number of red pixels in the Bayer array. The control unit executes a determination process for determining whether or not the pixel value output from the pixel is appropriate based on the pixel value for each of the plurality of pixels, and the determination result in the determination process is used as the basis for the determination process. The three-dimensional measuring device according to any one of claims 1 to 4, which executes shape calculation. The control unit executes interpolation by the pixel value of the low-sensitivity pixel located within a predetermined range from the high-sensitivity pixel with respect to the high-sensitivity pixel whose pixel value is determined to be inappropriate by the determination process. Alternatively, the low-sensitivity pixel for which the pixel value is determined to be inappropriate by the determination process is interpolated by the pixel value of the high-sensitivity pixel located within the predetermined range from the low-sensitivity pixel. The three-dimensional measuring device according to claim 5, which executes the shape calculation based on the result. The projector irradiates the object with light having a plurality of striped patterns having the predetermined wavelengths and having different phases from each other.
The three-dimensional measuring device according to any one of claims 1 to 6, wherein the control unit executes the shape calculation by a phase shift method. The process of irradiating an object with light of a predetermined wavelength and
A step of incidenting light reflected by the object on a plurality of pixels and outputting a pixel value according to the intensity of the incident light on the plurality of pixels.
A step of executing the shape calculation of the three-dimensional shape of the object based on the pixel value is provided.
The plurality of pixels include a plurality of high-sensitivity pixels and a plurality of low-sensitivity pixels having a spectral sensitivity characteristic in which the ratio of an output to an input having a predetermined wavelength is lower than that of the spectral sensitivity characteristic of the high-sensitivity pixel. Three-dimensional measurement method.

Images