JP2021117909A - Three-dimensional image generation device and three-dimensional image generation method - Google Patents

Abstract

To generate a three-dimensional image of an object in a short time.SOLUTION: A three-dimensional image generation device comprises: a plurality of cameras 10, 20, 30 using image pick-up devices 11, 21, 31; and a control unit 40 which processes each image photographed by the cameras 10, 20, 30. The control unit 40 sets a plurality of voxels V in a space including the object, images the plurality of voxels V from a plurality of directions, (a) detects each brightness of each of the image pick-up devices 11, 21, 31 corresponding to one voxel V of the plurality of voxels V, (b) specifies the smallest one in the detected brightness as the minimum brightness of the one voxel V, (c) specifies the one voxel V as the specific voxel including the object when the specified minimum brightness is equal to or greater than a prescribed threshold, repeatedly executes the operations of (a)-(c) for all the voxels V, and generates the three-dimensional image of the object by connecting the specific voxels specified in (c).SELECTED DRAWING: Figure 1

Description

The present invention relates to a three-dimensional image generation device and a three-dimensional image generation method for an object using a camera.

A method of generating a three-dimensional image of an object such as a bonding wire (hereinafter referred to as a wire) connecting a pad of a semiconductor chip and a lead of a substrate has been proposed (see, for example, Patent Document 1).

In the method described in Patent Document 1, a wire is illuminated with a ring-shaped illuminator, a wire image is imaged while changing the focusing height using an optical system having a shallow focal depth, and the wire image is centered on each wire image. By detecting the appearing dark part, each XY coordinate of the wire at each focusing height is detected, the three-dimensional shape of the entire wire is detected from the data, and a three-dimensional image is generated.

Patent No. 3235009

However, the method described in Patent Document 1 has a problem that it takes a long time to generate a three-dimensional image because it is necessary to capture a plurality of images by changing the focusing height of the optical system. there were.

Therefore, an object of the present invention is to generate a three-dimensional image of an object in a short time.

The three-dimensional image generation device of the present invention includes a plurality of cameras using an image pickup element and a control unit for processing each image captured by the cameras, and the control unit includes a plurality of voxels in a space including an object. Is set, a plurality of voxels are imaged from a plurality of directions by a plurality of cameras, (a) each brightness of each imaging element of each camera corresponding to one voxel among the plurality of voxels is detected, and (b) each The smallest of the lightness detected by the camera is specified as the minimum lightness of one voxel, and (c) when the specified minimum lightness is equal to or higher than a predetermined threshold, one voxel is specified as a specific voxel including an object. Then, the operations (a) to (c) are repeatedly executed for all the plurality of voxels, and the plurality of specific voxels specified in (c) are connected to generate a three-dimensional image of the object. ..

In the three-dimensional image generation method of the present invention, a plurality of cameras using an imaging element are prepared, a plurality of voxels are set in a space including an object, and a plurality of voxels are imaged from a plurality of directions by the plurality of cameras. , (A) Detect each brightness of each imaging element of each camera corresponding to one voxel among a plurality of voxels, and (b) the smallest of each brightness detected by each camera is the minimum of one voxel. It is specified as lightness, and (c) when the specified minimum lightness is equal to or higher than a predetermined threshold, one voxel is specified as a specific voxel including an object, and the operations (a) to (c) are repeated for all of a plurality of voxels. It is characterized in that it is executed and a plurality of specific voxels specified in (c) are connected to generate a three-dimensional image of an object.

In this way, since images taken by a plurality of cameras are processed to generate a three-dimensional image, it is possible to generate a three-dimensional image without hardware operations such as changing the focusing height of the optical system. It is possible to generate a three-dimensional image of an object in a short time.

In the three-dimensional image generation method of the present invention, the object may be illuminated from above, and the plurality of cameras may be set above the object.

This makes it possible to generate a three-dimensional image by a simple method.

In the three-dimensional image generation method of the present invention, the object may be an electrode of a semiconductor element and an electrode of a substrate, or a wire connecting one electrode of the semiconductor element and another electrode of the semiconductor element.

In this way, the three-dimensional image generation of the wire can be performed in a short time.

The present invention can generate a three-dimensional image of an object in a short time.

It is a system diagram which shows the structure of the 3D image generation apparatus which executes the 3D image generation method of embodiment. It is explanatory drawing which shows the relationship between the center of each voxel and the position of the image sensor of each camera when the voxel of the surface of y = y1 shown in FIG. 1 is imaged by a plurality of cameras. It is a flowchart which shows the process of the 3D image generation method of embodiment. It is a flowchart which shows the process of specifying the specific voxel including the object in the process shown in FIG. It is a table explaining the minimum brightness of each voxel and the specific processing of a specific voxel.

Hereinafter, the three-dimensional image generation device 100 that executes the three-dimensional image generation method of the embodiment will be described with reference to the drawings. In the following description, as shown in FIG. 1, the three-dimensional image generator 100 will be described as generating a three-dimensional image of the wire 53 connecting the electrode 51 of the semiconductor element and the electrode 52 of the substrate. It is also possible to generate a three-dimensional image of the object of.

The three-dimensional image generation device 100 processes the images captured by the three cameras 10, 20, 30 using the image sensor and the cameras 10, 20, 30 to generate a three-dimensional image of the wire 53, which is an object. It includes a control unit 40 and a light source 45 that illuminates the wire 53. In the present embodiment, the number of cameras is described as three, but the number of cameras is not limited to three, and may be two or four or more.

The light source 45 is arranged above the wire 53. Further, the camera 10 is arranged above the wire 53, and the cameras 20 and 30 are arranged above the wire 53 so that the optical axes 20a and 30a are inclined with respect to the optical axis 10a of the camera 10. ing. The control unit 40 is composed of a computer including a CPU 41 that processes information internally and a memory 42.

A plurality of voxels V are set in the space including the wire 53. The voxel V is set in the entire space where the wire 53 exists. Each center coordinate of voxel V is represented by Vc (x, y, h). FIG. 1 shows nine voxels V1 to V9 set in the space including the wire 53. In each of the nine voxels V1 to V9, the center coordinates of Vc (x, y, h) are located in the plane of y1 in the y direction, the x coordinates are x1, x2, x3, and the heights are h, respectively. Are the positions of h1, h2, and h3. Further, the cross section of the wire 53 y = y1 is located in the voxel V5 centered on Vc (x2, y1, h2).

Next, with reference to FIG. 2, the relationship between the nine voxels V1 to V9 described with reference to FIG. 1 and the position of each pixel of each image sensor 11, 21, 31 of each camera 10, 20, 30. An example will be described.

The voxel V1 is at the center position Vc (x1, y1, h1) and corresponds to the pixel P11 of the image sensor 11 of the camera 10, the pixel P23 of the image sensor 21 of the camera 20, and the pixel P31 of the image sensor 31 of the camera 30. Similarly, the center position Vc (x2, y1, h1) of the voxel V2 corresponds to the pixel P12 of the image sensor 11, and corresponds to the pixel P24 of the image sensor 21 and the pixel P32 of the image sensor 31. Further, the center position Vc (x3, y1, h1) of the voxel V3 corresponds to the pixel P13 of the image sensor 11, and corresponds to the pixel P25 of the image sensor 21 and the pixel P33 of the image sensor 31. Similarly, similarly, the center position Vc (x1, y1, h2) of the voxel V4 corresponds to the pixel P11, the pixel P22, and the pixel P32, and the center position Vc (x2, y1, h2) of the voxel V5 corresponds to the pixel P12. The center position Vc (x3, y1, h2) of the voxel V6 corresponds to the pixel P23 and the pixel P33, and corresponds to the pixel P13, the pixel P24, and the pixel P34. Further, the center position Vc (x1, y1, h3) of the voxel V7 corresponds to the pixel P11, the pixel P21, and the pixel P33, and the center position Vc (x2, y1, h3) of the voxel V8 corresponds to the pixel P12, the pixel P22, and the pixel P34. The center position Vc (x3, y1, h3) of the voxel V9 corresponds to the pixel P13, the pixel P23, and the pixel P35.

Then, when the voxels V1 to V9 are imaged by the cameras 10, 20 and 30, the brightness of the voxels V1 to V9 is the brightness of the corresponding pixels of the image sensors 11 and 21 and 31 of the corresponding cameras 10, 20 and 30. Is detected as.

Next, the operation of the three-dimensional image generator 100 will be described with reference to FIGS. 3 to 5.

As shown in step S101 of FIG. 3, the CPU 41 of the control unit 40 of the three-dimensional image generation device 100 images a plurality of voxel Vs from a plurality of directions with the plurality of cameras 10, 20, and 30. Then, the image analysis surface imaged by the cameras 10, 20, and 30 in step S102 is set to the surface of y = 0, and the voxel V including the wire 53 which is the object in the surface of y = 0 is set in step S103. Specify as a specific voxel. Then, in step S105, y is increased by Δy until y becomes the maximum value of y in which the voxel V exists, and step S103 is repeatedly executed. If YES is determined in step S104, the process proceeds to step S106 of FIG. 3 to connect a plurality of specific voxels specified in step S103 to generate a three-dimensional image of the object.

Here, an example of a process for identifying a specific voxel including an object in step S103 of FIG. 3 executed by the CPU 41 of the control unit 40 will be described with reference to FIGS. 4 and 5. In the following description, a process of specifying a specific voxel including the wire 53 which is an object in the voxels V1 to V9 whose center coordinates are located on the plane of y = y1 shown in FIG. 1 will be described.

As described with reference to FIG. 3, when the voxels V1 to V9 are imaged by the cameras 10, 20 and 30, the brightness of the voxels V1 to V9 is the image sensor 11 of the corresponding cameras 10, 20 and 30. It is detected as the brightness of each of the corresponding pixels of 21 and 31. Since the voxel V including the wire 53 reflects light on the wire 53, the pixel corresponding to the voxel V including the wire 53 detects a bright brightness 1. On the other hand, since the voxel V that does not include the wire 53 does not reflect light, it detects a dark brightness of 0. However, if another voxel V including the wire 53 is present between the optical path between the voxel V and the pixel or on the extension of the optical path, the pixel detects the bright brightness 1 of the other voxel V.

As shown in step S201 of FIG. 4, the CPU 41 of the control unit 40 captures each of the cameras 10, 20, and 30 corresponding to one voxel among the plurality of voxels from the images captured by the cameras 10, 20, and 30. Each brightness of the elements 11, 1, 21 and 31 is detected.

A case where the CPU 41 detects the brightness of the voxel V1 will be described. As shown in FIGS. 2 and 5, the voxel V1 is a voxel V that does not include the wire 53. The pixels P11 and P31 of the image sensors 11 and 31 of the corresponding cameras 10 and 30 detect the dark brightness 0 of the voxel V1 because the wire 53 does not exist between the pixels P11 and P31 and the voxel V1. On the other hand, since the pixel P23 of the image sensor 21 of the corresponding camera 20 has the voxel V5 including the wire 53 between the voxel V1 and the pixel P23, the bright brightness 1 of the voxel V5 is detected instead of the dark brightness of the voxel V1. do. Therefore, as shown in FIG. 5, the CPU 41 detects the brightness of the three pixels P11, P23, and P31 corresponding to the voxel V1 as brightness 0, 1, 0, respectively.

Next, the CPU 41 proceeds to step S202 of FIG. 4, and identifies the smallest of the lightnesses detected by the cameras 10, 20, and 30 as the minimum lightness of the voxel V. In voxel V1, the detected brightness is 0, 1, 0, so the minimum brightness is specified as 0.

Then, the CPU 41 proceeds to step S203 of FIG. 4, and when the specified minimum brightness is larger than a predetermined threshold value, the voxel is specified as a specific voxel including the wire 53 which is an object. The threshold value can be a predetermined value larger than 0, and may be 1, for example. In the case of voxel V1, since the minimum brightness is 0, the CPU 41 proceeds to step S204 of FIG. 4 without specifying voxel V1 as a specific voxel, and steps for all voxels V centered on the plane of y = y1. It is determined whether or not the processes of S201 to S203 have been performed, and if NO, the process returns to step S201 and the processes of steps S201 to S203 are performed for the next voxel V.

After processing the voxel V1, the CPU 41 determines NO in step S204 of FIG. 4, returns to step S201 of FIG. 4, and performs the processing of steps S201 to S203 for voxel V2.

As shown in FIG. 5, the CPU 41 detects the brightness of the three pixels P12, P24, and P32 corresponding to the voxel V2, similarly to the voxel V1. In this case, since the voxel V5 including the wire 53 is located between the voxel V2 and the pixel P12, the pixel P12 detects the brightness 1. Therefore, the CPU 41 detects the brightness of the three pixels P12, P24, and P32 corresponding to the voxel V2 as 1,0,0. Then, the CPU 41 specifies the minimum brightness of the voxel V2 as 0 in step S202, proceeds to step S204 of FIG. 4 without specifying the voxel V2 as a specific voxel in step S203, and processes the voxel V3.

Hereinafter, similarly, as shown in FIG. 5, the CPU 41 specifies the brightness of each pixel for the voxels V3 to V4, specifies the minimum brightness as 0, and does not use the voxels V3 to V4 as the specific voxels.

The CPU 41 detects the brightness of each pixel P12, P23, and P33 corresponding to the voxel V5. Since the voxel V5 is a voxel V including the wire 53, the corresponding pixels P12, P23, and P33 of the image pickup elements 11 and 21, 31 of each of the cameras 10, 20, and 30 also detect the bright brightness 1. Therefore, the CPU 41 specifies the minimum brightness of the voxel V5 as 1, identifies the voxel V5 as the specific voxel, and proceeds to the process of the voxel V6.

As shown in FIG. 5, the CPU 41 specifies the brightness of each pixel corresponding to the voxel V6 as 0, 0, 0 in step S201, respectively. Then, the CPU 41 specifies the minimum brightness of the voxel V6 as 0 in step S202, and proceeds to the process of the voxel V7 without setting the voxel V6 as the specific voxel.

In the voxel V7, the wire 53 is located on the extension line of the optical path between the voxel V7 and the corresponding pixel P33 of the camera 30. Therefore, the CPU 41 specifies the pixels P11, P21, and P31 corresponding to the voxel V7 as 0, 0, 1, respectively. Then, the CPU 41 specifies the minimum brightness of the voxel V7 as 0, and proceeds to the processing of the voxels V8 and V9 without setting the voxel V7 as the specific voxel.

Similar to voxel V7, the CPU 41 detects the brightness of the corresponding pixels of voxels V8 and V9 as 1,0,0 and 0,1,0, respectively, specifies each minimum brightness as 0, and voxel V8, The process proceeds to step S204 of FIG. 4 without using V9 as the specific voxel, and a determination of YES in step S204 ends the process of identifying the voxel to be specified including the object shown in step S103 of FIG.

By this process, as shown in FIG. 5, the CPU 41 identifies only the voxels V5 including the wire 53 among the nine voxels V1 to V9 at the coordinate center on the plane of y = y1 as the specific voxels.

The CPU 41 changes y by Δy, executes the process of step S103 of FIG. 3 in all the spaces where the voxel V exists, determines YES in step S104 of FIG. 3, and proceeds to step S106 of FIG. By connecting specific voxels in a plane, a three-dimensional image of the wire 53 is generated.

As described above, in the three-dimensional image generation method of the embodiment, when a plurality of voxels V are set in a space including an object and a plurality of voxels V are imaged from different angles by a plurality of cameras 10, 20, 30. The brightness detected by each pixel of each of the cameras 10, 20, and 30 corresponding to the voxel V including the object becomes a bright brightness 1 due to the reflection by the object, and the minimum brightness of the voxel V becomes 1. On the other hand, at least one of the lightness detected by each pixel of each of the cameras 10, 20, and 30 corresponding to the voxel V in which the object does not exist becomes a dark lightness 0, and the minimum lightness of the voxel V becomes 0. As a result, when the minimum brightness becomes 1, the voxel V is specified as a specific voxel including the object, and the specific voxel is connected to generate a three-dimensional image of the object.

As described above, the three-dimensional image generation device 100 of the embodiment processes the images captured by the plurality of cameras 10, 20, and 30 to generate a three-dimensional image, so that the focusing height of the optical system is changed. It is possible to generate a three-dimensional image without any hardware operation such as making the wire 53, and it is possible to generate a three-dimensional image of an object such as a wire 53 in a short time.

10, 20, 30 camera, 10a, 20a, 30a optical axis, 11,21,31 image sensor, 40 control unit, 41 CPU, 42 memory, 45 light source, 51, 52 electrodes, 53 wires, 100 3D image generator ..

Claims (4)

It is a three-dimensional image generator,
Multiple cameras using an image sensor and
A control unit that processes each image captured by the camera is provided.
The control unit
Set multiple voxels in the space containing the object,
Multiple voxels are imaged from multiple directions with the plurality of cameras.
(A) Detecting each brightness of each image sensor of each camera corresponding to one voxel among a plurality of voxels,
(B) The smallest of the brightness detected by each camera is specified as the minimum brightness of the one voxel.
(C) When the specified minimum brightness is equal to or higher than a predetermined threshold value, the one voxel is specified as a specific voxel containing the object.
The operations (a) to (c) are repeatedly executed for all of the plurality of voxels, and the operations are repeatedly executed.
To generate a three-dimensional image of the object by connecting a plurality of specific voxels specified in (c).
A three-dimensional image generator characterized by. It is a three-dimensional image generation method.
Prepare multiple cameras using the image sensor,
Set multiple voxels in the space containing the object,
Multiple voxels are imaged from multiple directions with the plurality of cameras.
(A) Detecting each brightness of each image sensor of each camera corresponding to one voxel among a plurality of voxels,
(B) The smallest of the brightness detected by each camera is specified as the minimum brightness of the one voxel.
(C) When the specified minimum brightness is equal to or higher than a predetermined threshold value, the one voxel is specified as a specific voxel containing the object.
The operations (a) to (c) are repeatedly executed for all of the plurality of voxels, and the operations are repeatedly executed.
To generate a three-dimensional image of the object by connecting a plurality of specific voxels specified in (c).
A three-dimensional image generation method characterized by. The three-dimensional image generation method according to claim 2.
Illuminate the object from above
The plurality of the cameras should be set above the object.
A three-dimensional image generation method characterized by. The three-dimensional image generation method according to claim 2 or 3.
The object is a wire that connects an electrode of a semiconductor element and an electrode of a substrate, or one electrode of the semiconductor element and another electrode of the semiconductor element.
A three-dimensional image generation method characterized by.

Images