JP3986748B2 - 3D image detection device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional image detection apparatus that detects a three-dimensional shape or the like of a measurement object using a stereo image and a light propagation time measurement method.
[0002]
[Prior art]
In photogrammetry using a stereo image, a measurement object is photographed from two different viewpoints, and the distance to the measurement point is measured based on the principle of triangulation using the parallax between the two photographed images. At this time, it is necessary to search for corresponding points on the epipolar line of the two images. Conventionally, the search for corresponding points on the epipolar line is performed by, for example, performing image processing on a captured stereo image to extract feature points and obtaining a correspondence relationship between these feature points.
[0003]
On the other hand, as a survey using the light propagation time measurement method, the distance to the measurement object is obtained by irradiating the measurement object with the distance measurement light and receiving and detecting the reflected light with an image sensor such as a CCD at a predetermined timing. There is known a three-dimensional image detection apparatus that can detect each pixel.
[0004]
[Problems to be solved by the invention]
However, in photogrammetry using stereo images, it is difficult to extract feature points, for example, when the subject has no pattern and its shape is flat, or when there are few areas where two images are shared due to large parallax. It is difficult to associate the two images. On the other hand, in the distance measuring method using the above three-dimensional image detection device, the pixel value of each pixel directly corresponds to the distance to the corresponding measurement object, so that the correspondence as when using a stereo image is used. The problem does not occur. However, this method has a problem that measurement accuracy cannot always be obtained sufficiently.
[0005]
It is an object of the present invention to obtain a three-dimensional image detection apparatus for simplifying corresponding point search processing in a stereo image at high speed.
[0006]
[Means for Solving the Problems]
The three-dimensional image detection apparatus of the present invention captures a stereo image of a two-dimensional image, which is a normal visual image, and distance measuring means that can detect a three-dimensional image whose pixel value corresponds to the distance to the subject. The corresponding point in the stereo image of the two-dimensional image captured using the stereo optical system for two-dimensional image and the stereo optical system for two-dimensional image is searched based on the three-dimensional image detected by the distance measuring means. And corresponding point searching means.
[0007]
The three-dimensional image detection apparatus preferably includes a three-dimensional image stereo optical system for capturing the three-dimensional image as a stereo image. Further, at this time, the corresponding point search means preferably obtains a correspondence relationship between pixels in a stereo image of a three-dimensional image captured using the stereo optical system for a three-dimensional image. Is preferably performed based on a three-dimensional coordinate value calculated from a three-dimensional image. Thereby, the correspondence between the pixels in the stereo image of the three-dimensional image is obtained simply and at high speed, and the correspondence between the pixels in the stereo image of the two-dimensional image is obtained more simply and at high speed based on this correspondence. In addition, since the correspondence from the stereo image of the three-dimensional image to the stereo image of the two-dimensional image is required more directly, the correspondence relationship between the pixels in the stereo image of the two-dimensional image is also simplified, fast, and with high accuracy. Can be sought.
[0008]
The three-dimensional image detection apparatus preferably includes a connection unit for connecting the stereo optical system for two-dimensional images with the optical system of the camera. By connecting the stereo optical system for two-dimensional images and the optical system of the camera through this connection unit, it is possible to take a stereo image of a two-dimensional image in the camera, and the three-dimensional image detection device can be attached to and detached from an existing camera. A stereo adapter type can be used. At this time, it is preferable that the connecting portion is a screw mount attached to a screw mount provided at the opening of the lens barrel end of the camera. As a result, the stereo adapter type three-dimensional image detection apparatus can be easily mounted without changing the existing camera.
[0009]
Further, the three-dimensional image detection apparatus includes an external trigger input unit for detecting an external trigger signal output from, for example, a camera hot shoe, and the driving of the distance measuring unit is controlled based on the external trigger signal. . Alternatively, the three-dimensional image detection apparatus includes strobe light detection means for detecting strobe light as a light reception trigger signal, and driving of the distance measurement means is controlled based on the light reception trigger signal. Accordingly, it is possible to synchronize the two-dimensional image capturing operation with the camera and the three-dimensional image detecting operation with the three-dimensional image detection device without changing the existing camera.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a perspective view schematically showing a state in which a stereo adapter type three-dimensional image detection apparatus according to an embodiment of the present invention is mounted on a conventional digital camera. A stereo adapter type three-dimensional image detection apparatus used in this embodiment will be described with reference to FIG.
[0011]
The digital camera 10 is a conventional digital camera. A hot shoe 11 and a release switch 14 are provided on the top surface of the digital camera 10, and an optional connector 12, for example, USB, for connecting an interface cable is provided on the side surface. ing. A stereo adapter type three-dimensional image detection device (hereinafter referred to as a stereo adapter) 20 is attached to the tip of the lens barrel 13 of the digital camera 10. For mounting the stereo adapter 20 to the lens barrel 13, for example, a screw mount that is formed at the front end opening of the lens barrel 13 and used for attaching a filter, a hood, or the like is used.
[0012]
On the front surface of the stereo adapter 20, a pair of stereo shooting lenses is provided in two upper and lower stages. The pair of lenses 21 provided in the upper stage are two-dimensional image stereo lenses for dividing an optical axis of the digital camera into two and capturing a normal stereo image with parallax with a CCD provided in the digital camera 10. On the other hand, the pair of lenses 22 provided in the lower stage is a three-dimensional image stereo lens for capturing a stereo image of a three-dimensional image, which will be described later. On both sides of the stereo adapter 20, light source devices 23 are provided for diffusing and irradiating infrared laser light for distance measurement. In addition, a light reception trigger detection device 24 that detects the reflected light of the flashlight is provided at the center of the top surface of the stereo adapter 20. The distance between the lenses in the three-dimensional image stereo lens 22 is equal to the distance between the lenses in the two-dimensional image stereo lens 21.
[0013]
One end of the external trigger cable 18 and the interface cable 19 is connected to the rear surface of the stereo adapter 20, and the other ends of the cables 18 and 19 are connected to the hot shoe 11 and the option connector 12, respectively. In FIG. 1, only the hot shoe 11, the option connector 12 and the lens barrel 13 are shown in the digital camera 10, and the other parts are omitted.
[0014]
2, 3A, and 3B are diagrams schematically showing the structure of the optical system of the stereo adapter 20 in the present embodiment. FIG. 2 shows the structure of an optical system (two-dimensional image stereo optical system) related to the two-dimensional image stereo lens 21 (21a and 21b), and FIGS. 3A and 3B are three-dimensional image stereo lenses. The structure of the optical system regarding 22 (22a and 22b) is shown.
[0015]
The optical system of the stereo adapter 20 is symmetrical with respect to the optical axis L of the digital camera. Light incident from the two-dimensional image stereo lens 21a for photographing the left image of the two-dimensional stereo image is projected onto the right half region of the CCD 50 mounted in the digital camera 10 along the optical axis La. On the other hand, light incident from the two-dimensional image stereo lens 21b for photographing the right image of the two-dimensional stereo image is projected onto the left half region of the CCD 50 along the optical axis Lb.
[0016]
That is, the optical axis La parallel to the optical axis L in the two-dimensional image stereo lens 21a is bent by 90 ° toward the optical axis L by the reflecting mirror 25a arranged to be inclined by 45 ° with respect to this axis. The optical axis La bent by the reflection mirror 25a is bent in the direction of the CCD 50 by the reflection mirror 26a arranged to be inclined with respect to this axis. The optical axis La bent by the reflecting mirror 26 a reaches the right light receiving surface of the CCD 50 via the imaging lens 51 of the digital camera 10. On the other hand, in the stereo lens 21b for two-dimensional images, the optical axis Lb parallel to the optical axis L is bent 90 ° toward the optical axis L by the reflecting mirror 25b arranged to be inclined by 45 ° with respect to this axis. The optical axis Lb bent by the reflection mirror 25b is bent in the direction of the CCD 50 by the reflection mirror 26b arranged to be inclined with respect to this axis. The optical axis Lb bent by the reflection mirror 26 b reaches the light receiving surface on the left side of the CCD 50 via the imaging lens 51 of the digital camera 10.
[0017]
As described above, FIGS. 3A and 3B are diagrams schematically showing the structure of an optical system related to the three-dimensional image stereo lens 22 (22a and 22b) in the stereo adapter 20. FIG. (A) is a schematic diagram when the stereo adapter 20 is viewed from above, and FIG. 3 (b) is a schematic diagram when viewed from the front.
[0018]
Light incident from the three-dimensional image stereo lens 22a for photographing the left image of the three-dimensional stereo image is projected onto the left half region of the CCD 27 mounted on the bottom 28 of the stereo adapter 20 along the optical axis La ′. Is done. On the other hand, the light incident from the three-dimensional image stereo lens 22b for photographing the right image of the three-dimensional stereo image is projected onto the right half region of the CCD 27 along the optical axis Lb '.
[0019]
That is, in the stereo lens 22a for three-dimensional images, the optical axis La ′ parallel to the optical axis L of the digital camera 10 is bent by 90 ° toward the lens 30a by the reflecting mirror 29a arranged to be inclined by 45 ° with respect to this axis. Is done. The optical axis La ′ bent by the reflecting mirror 29a passes through the center of the lens 30a, and is bent 90 ° toward the CCD 27 by the reflecting mirror 31a arranged to be inclined by 45 ° with respect to the optical axis. To the light receiving surface on the left side of the CCD 27 installed on the bottom portion 28 of the CCD. On the other hand, in the stereo lens 22b for three-dimensional images, the optical axis Lb ′ parallel to the optical axis L is bent 90 ° in the direction of the lens 30b by the reflecting mirror 29b arranged to be inclined by 45 ° with respect to this axis. The optical axis Lb ′ bent by is bent by 90 ° in the direction of the CCD 27 by the reflecting mirror 31b disposed through an angle of 45 ° with respect to the optical axis through the center of the lens 30b. Thereafter, the optical axis Lb ′ reaches the light receiving surface on the right side of the CCD 27 installed on the bottom 28 of the stereo adapter 20.
[0020]
FIG. 4 is a block diagram schematically showing a circuit configuration of the stereo adapter 20 shown in FIGS. A stereo adapter type three-dimensional image detection apparatus according to this embodiment will be described with reference to FIG. Note that the configuration of the imaging optical system in FIG. 4 is schematic.
[0021]
The stereo adapter 20 of the present embodiment includes an external trigger mode for capturing a three-dimensional image using the CCD 27 using a signal from the hot shoe 11 of the digital camera 10 as a trigger, and a strobe (not shown) of the digital camera 10. There is a light reception trigger mode in which light emission is detected by a photodiode (PD) 24 and a three-dimensional image is taken using this as a trigger. FIG. 4 shows a configuration when an external trigger cable 18 is connected to the hot shoe 11.
[0022]
When the external trigger mode is set, when the release switch 14 of the digital camera 10 is pressed and a trigger signal for strobe light emission is output from the hot shoe 11, the trigger signal output from the hot shoe 11 is the hot shoe. 11 and the external trigger cable 18 connected to the connector 70, the signal is input to the external trigger input circuit 61 of the stereo adapter 20. Thereafter, a trigger signal is output from the external trigger input circuit 61 to the system control circuit 60. When a trigger signal from the external trigger input circuit 61 is input to the system control circuit 60, the system control circuit 60 controls the light emitting element control circuit 62 and the CCD drive circuit 63 to start capturing a stereo image for the three-dimensional image. To do. That is, in the light emitting device 23 including the light emitting element 23a and the illumination lens 23b, the light emitting element 23a such as an infrared laser diode emits pulsed ranging light based on the control signal of the light emitting element drive control circuit 62, Ranging light in the infrared region is irradiated to the entire subject via the illumination lens 23b. Further, in the CCD 27, based on the CCD drive signal from the CCD drive circuit 63, charge accumulation and charge read operations are performed in accordance with the light emission of the light emitting device 23.
[0023]
The charge signal read from the CCD 27, that is, the image signal is amplified by the amplifier 64 and converted from an analog signal to a digital signal by the A / D converter 65. The digital image signal is subjected to processing such as gamma correction in the imaging signal processing circuit 66 and temporarily stored in the image memory 67. Thereafter, the image signal is read from the image memory 67 and output to the digital camera 10 via the interface cable 19 connected to the system control circuit 60, the interface circuit 68, and the connector 69, and the image memory of the digital camera 10 and the memory card are output. Or the like.
[0024]
When the mode is set to the light reception trigger mode, the strobe light emitted from the strobe built in or attached to the camera body is detected by the photodiode (PD) 24 connected to the light reception trigger detection circuit 73 and used as a trigger signal. Input to the system control circuit 60. At this time, the system control circuit 60 controls the light emitting element control circuit 62 and the CCD drive circuit 63 to start capturing a stereo image for the three-dimensional image, as in the external trigger mode. In the light reception trigger mode, when the stereo adapter 20 is integrally mounted on a camera body, for example, with a digital camera not equipped with a hot shoe, or when a strobe light is required for taking a two-dimensional image, the strobe is hot. Used when the external trigger cable cannot be connected to the hot shoe, such as when attached to a shoe.
[0025]
The system control circuit 60 is also connected with a display element 71 made of liquid crystal or the like and a mode changeover switch 72 for switching between the external trigger mode and the light receiving trigger mode. The stereo adapter 20 is connected by attaching the screw mount 32 to the screw mount 15 provided in the lens barrel 13 of the digital camera 10.
[0026]
5 and 6 are flowcharts of programs executed in the stereo adapter 20 of the present embodiment. With reference to FIG. 5 and FIG. 6, the detection operation of the three-dimensional image executed in the stereo adapter type three-dimensional image detection apparatus of the present embodiment will be described.
[0027]
First, in step 101, it is determined whether or not the power switch of the stereo adapter 20 is in an on state, and step 101 is repeatedly executed until the power switch is set in an on state. If it is determined in step 101 that the power switch is on, it is determined in step 102 whether or not the mode of the connected digital camera 10 is set to the PC communication mode. The PC communication mode is one mode provided in a conventional digital camera, and is a mode for performing data communication by connecting the digital camera and the computer main body 10 via an interface cable connected to the option connector 12. It is. In step 102, communication with the system control circuit of the digital camera 10 is performed via the interface circuit 68, and it is determined whether or not the digital camera is set to the PC communication mode.
[0028]
If it is determined in step 102 that the digital camera 10 is set to the PC communication mode, data such as image data stored in the image memory 67 is transferred to the digital camera 10 via the interface circuit 68 in step 106. . In step 107, it is determined whether or not the transfer of the image data stored in the image memory 67 has been completed. When the transfer of the image data is not completed, the process returns to step 106, and the image data transfer operation is executed again. When the transfer of the image data is completed, the process proceeds to step 103.
[0029]
In step 103, it is determined whether or not the mode changeover switch 72 of the stereo adapter 20 is set to the light reception trigger mode. If it is determined that the mode switch 72 is set to the light reception trigger mode, it is determined in step 108 whether or not the strobe light is received by the photodiode 24. The strobe light reception detection operation by the photodiode 24 is repeated until the strobe light is received. That is, a standby state for receiving a light reception trigger signal by strobe light is set. When the strobe light is received by the photodiode 24, the process proceeds to step 109. At this time, in the digital camera 10, a stereo image composed of a normal image (for example, a color image) is captured by the CCD 50. This stereo image is recorded on a recording medium such as a memory provided in the digital camera 10.
[0030]
On the other hand, if it is determined in step 103 that the mode switch 72 is not set to the light reception trigger mode, the process proceeds to step 104, and it is determined whether or not the setting of the mode switch 72 is the external trigger mode. Is done. When the setting of the mode changeover switch 72 is not the external trigger mode, the process returns to step 101, and the above steps are repeated again. On the other hand, if it is determined that the setting of the mode selector switch 72 is the external trigger mode, the process proceeds to step 105.
[0031]
In step 105, it is determined whether or not an external trigger signal from the hot shoe 11 of the digital camera 10 is input to the external trigger input circuit. The determination operation in step 105 is repeated until an external trigger signal is input to the external trigger input circuit 61. If it is determined in step 105 that the external trigger signal has been input to the external trigger input circuit 61, the process proceeds to step 109. At this time, in the digital camera 10, a normal stereo image is picked up by the CCD 50 as in the case where the strobe light is received by the photodiode 24 in step 108, and is recorded on a recording medium such as a memory provided in the digital camera 10. To be recorded.
[0032]
In step 109, the light emission control using the light emitting device 23 for irradiating the distance measuring light is set to the on state. That is, driving of the light emitting element 23a by the light emitting element control circuit 62 is started. In step 200, driving of the CCD 27 in accordance with the distance measurement light irradiation timing is started, and signal charges corresponding to the distance to the subject are detected as image data for each pixel. That is, the CCD drive circuit 63 starts driving the CCD 27 to detect a three-dimensional image. In this embodiment, in order to obtain a large signal output, pulsed ranging light is repeatedly and repeatedly irradiated onto the subject, and the signal charge accumulation operation in the CCD 27 is controlled in accordance with the irradiation of the ranging light. The That is, the principle of distance measurement described later with reference to FIGS. 7 and 8 is repeatedly executed over, for example, one field period. The signal charges detected in each of the repeated distance measuring operations are integrated in a vertical transfer unit (not shown) of the CCD 27, thereby obtaining a large signal output.
[0033]
In step 201, the light emission control for the light emitting device 23 set in the on state in step 109 is set in the off state, and the distance measuring operation executed in step 200 ends. In step 202, calculation processing is performed based on the detected pixel value (image data) of the three-dimensional image, distance data to the subject is calculated for each pixel, and coordinate data of the subject corresponding to each pixel is obtained. Calculated. The distance measurement (three-dimensional image detection) operation in step 200 and the distance data and coordinate data calculation method in step 202 will be described later. The distance data and coordinate data are calculated for each of the left and right stereo images. That is, it is performed for each of the right and left regions of the imaging surface of the CCD 28 corresponding to the left and right stereo images.
[0034]
In step 203, a correspondence relationship between each pixel is obtained as pixel correspondence data between the left image and the right image captured as a stereo image in step 200. That is, the correspondence between the pixel when a certain point of the subject is projected on the right image and the pixel when the point is projected on the left image is based on the coordinate data of the subject calculated for each pixel in step 202. Desired.
[0035]
In step 204, image compression processing is performed on the three-dimensional image detected in step 200. In step 205, the three-dimensional image subjected to the image compression processing, the distance data and coordinate data calculated in step 202, and pixel correspondence data indicating the correspondence relationship of the pixels in the left and right images obtained in step 203 are stored in the image memory 67. Is remembered. Thereafter, the processing returns to step 101 again, and the above-described processing is repeatedly executed.
[0036]
Note that image data, distance data, coordinate data, pixel correspondence data, and the like stored in the image memory 67 are transferred to a recording medium such as a memory of the digital camera 10 and temporarily stored in step 106. These data stored in the memory of the digital camera 10 are then transferred to a computer together with image data of a stereo image (two-dimensional image) captured by the digital camera 10.
[0037]
In the measurement of the distance to the subject using a three-dimensional image, the accuracy is not necessarily high because the CCD 27 must be driven at a very high speed. In order to realize high-speed driving of the CCD, it is desired to reduce the stray capacity of the CCD circuit. For this purpose, it is desired to reduce the number of pixels of the CCD. For these reasons, the CCD 27 having a smaller number of pixels than the CCD 50 for detecting a two-dimensional image is used to detect a three-dimensional image. From the above, in the computer to which the data has been transferred, first, the correspondence between the pixels in the left and right images of the two-dimensional image is obtained roughly from the pixel correspondence data obtained from the three-dimensional image. Conventionally known pattern matching or the like is applied to the two-dimensional image. As a result, a more accurate correspondence between the pixels is obtained, and based on this correspondence, the coordinate value of the subject in the three-dimensional space can be calculated with higher accuracy using a conventionally known principle of stereoscopic photogrammetry. It becomes possible. A method for obtaining the correspondence between pixels in the left and right images of the two-dimensional image from the pixel correspondence data obtained from the three-dimensional image will be described later.
[0038]
Next, the principle of the distance measurement operation (three-dimensional image detection operation) executed in the stereo adapter 20 of this embodiment will be described with reference to FIGS. In FIG. 8, the horizontal axis represents time.
[0039]
The distance measuring light output from the distance measuring device B is reflected by the subject S and received by a CCD (not shown). The distance measuring light is pulsed light having a predetermined pulse width H. Therefore, the reflected light from the subject S is also pulsed light having the same pulse width H. The rising edge of the reflected light pulse is delayed by a time δ · t (where δ is a delay coefficient) from the rising edge of the ranging light pulse. Since the distance measuring light and the reflected light have traveled a distance r twice that between the distance measuring device B and the subject S, the distance r is
r = δ · t · C / 2 (1)
Is obtained. However, C is the speed of light.
[0040]
For example, when the reflected light is detected from the rising edge of the ranging light pulse and switched to the undetectable state before the reflected light pulse falls, that is, when the reflected light detection period T is provided, The received light amount A in the reflected light detection period T is a function of the distance r. That is, the received light amount A decreases as the distance r increases (the time δ · t increases).
[0041]
The three-dimensional measurement in the present embodiment is performed by detecting the received light amount A in each of a plurality of photodiodes provided in the CCD 27 using the principle described above and arranged two-dimensionally. That is, the distance information from the stereo adapter 20 to the point corresponding to each photodiode on the surface of the subject S based on the received light amount A detected in each photodiode (each pixel) is an image signal for each photodiode (pixel). Detected as (three-dimensional image), distance data representing the surface shape of the subject S is calculated for each photodiode (pixel) from this image signal.
[0042]
Next, a method for obtaining the correspondence between pixels in the left and right images of the two-dimensional image from the pixel correspondence data obtained from the three-dimensional image will be described with reference to FIGS.
[0043]
FIG. 9 schematically shows the principle of conventional distance measurement using a stereo image. Point O_L, Point O_RIs the left image (left projection plane) S_LAnd the right image (right projection plane) S_RThe half line L_L, L_RIs the viewpoint O_L, O_RAre optical axes parallel to each other. Two optical axes L_L, L_RDistance h is the viewpoint O_L, O_RCorresponds to the length of the base line connecting the lines (base line length). A point P representing an arbitrary point on the subject S is an image S_L, S_REach point P_L, Point P_RProjected on. Point P_L, Point P_RA straight line L passing through_EL, L_ERHas two viewpoints O_L, O_RAnd an epipolar plane defined by a point P on the subject and an image S_L, S_RIs an epipolar line. The position of the point P on the subject_LIs represented by a three-dimensional coordinate system XYZ with the coordinate origin as the image S_L, S_RProjection point P of point P to_L, P_REach position is a two-dimensional photographic coordinate system X_LY_L, And photographic coordinate system X_RY_R, The three-dimensional coordinate value (x, y, z) of the point P is the point P_LCoordinate value (x_L, Y_L) And point P_RCoordinate value (x_R, Y_R)Using,
x = x_L・ H / (x_L―X_R(2)
y = y_R・ H / (x_L―X_R(3)
z = f · h / (x_L―X_R(4)
It is required as follows. Note that f is a focal length, and a viewpoint O_L, O_RAnd image S_L, S_RCorresponds to the distance between. Also, the coordinate system XYZ has an optical axis L_LIs taken as the Z axis and the X axis is the viewpoint O_LViewpoint O_RThis is a left-handed coordinate system taken in the direction of. Photo coordinate system X on the left and right images_LY_L, And photographic coordinate system X_RY_RIs the optical axis L that penetrates each image._L, L_REach above, X_LAxis, X_RAxis is X axis, Y_LAxis, Y_RThe axes are taken parallel to the Y axis. In FIG. 9, the left image S corresponding to the left projection plane._LAnd the right image S corresponding to the right projection plane_RIs drawn as a separate plane, but as in the present embodiment, the left and right images S are divided into left and right projection planes by bisecting the imaging plane of one CCD._R, S_LEven if they are adjacent, there is no change.
[0044]
Thus, in stereoscopic photogrammetry, the corresponding point P on the same epipolar line in the left and right images._L, P_RTo obtain the three-dimensional coordinates of the subject. However, when the parallax between the left and right images is large, or when there are not many feature points found in the captured image, it is difficult to search for the corresponding points using only a two-dimensional image as in the past. Become.
[0045]
Next, a corresponding point search method according to the present embodiment will be described with reference to FIG. In FIG. 10, a curve S represents a cross-sectional shape of the subject corresponding to one epipolar line of a three-dimensional image captured as a stereo image. Straight line L_ELIs the epipolar line in the left image, straight line L_ERIs the epipolar line in the right image. A region surrounded by a broken line indicates a common region in the left and right images.
[0046]
Epipolar line L_EL, L_ERSince the pixel value of the arbitrary pixel above corresponds to the distance to the subject corresponding to the pixel, the distance is obtained based on the distance measurement principle described with reference to FIGS. Since the positional relationship between each pixel of the CCD and the focal point (viewpoint) is known, the three-dimensional coordinate value of the point on the subject corresponding to each pixel is obtained based on this. For example, the epipolar line L in the left image_ELAny pixel (point) P above_L(X_L, Y_L) Is determined, the three-dimensional coordinates (x, y, z) of the point P on the subject corresponding to the pixel are obtained. In the right image, the pixel P corresponding to the three-dimensional coordinates (x, y, z)_R(X_R, Y_R) To obtain an arbitrary point P in the left image_LPoint P of the right image corresponding to_RIs required. Since the correspondence between the pixels of the CCD 27 and the pixels of the CCD 50 is known from information such as the number of pixels of each CCD, the correspondence between the pixels of the left and right images obtained by the method using the above-described three-dimensional image is used. Correspondence between pixels in a two-dimensional image is required. In addition, it is possible to easily determine a region (a region surrounded by a broken line) that is photographed in common in the left and right images. Note that in the stereo image shooting for the two-dimensional image of the present embodiment, the left image is picked up by the right half light receiving surface of the CCD 50 and the right image is picked up by the left half light receiving surface. On the other hand, in the imaging of the stereo image with respect to the three-dimensional image of the present embodiment, the left image is picked up by the left half light receiving surface of the CCD 27 and the right image is picked up by the right half light receiving surface. Therefore, in the present embodiment, the right half pixel of the CCD 27 corresponds to the left half pixel of the CCD 50, and the left half pixel corresponds to the right half pixel.
[0047]
In the present embodiment, the epipolar line L in the left and right images._L, L_RAre on the same horizontal line of the CCD, so the left and right photographic coordinate systems (x_L, Y_L), (X_R, Y_R) In y_L= Y_RAnd the object of the search is x_LAnd x_RIt is only the correspondence between them.
[0048]
As described above, according to the present embodiment, the conventional stereo adapter is equipped with the three-dimensional image detection device that detects a three-dimensional image (an image in which each pixel value corresponds to the distance to the subject) as a stereo image. As a result, it is possible to speed up and simplify the process of searching for corresponding points in a conventional stereo image. Further, according to the stereo adapter type three-dimensional image detection apparatus of the present embodiment, the stereo adapter can be attached to and detached from a normal digital camera. Therefore, when a stereo photograph or a three-dimensional image is not required, the stereo adapter can be detached from the digital camera. Therefore, the digital camera can be used as a normal digital camera. On the other hand, in a stereo adapter type three-dimensional image detection apparatus, a configuration for detecting a normal two-dimensional image and mechanisms such as a finder and a release switch can be omitted.
[0049]
Furthermore, in the present embodiment, a signal from a hot shoe equipped in the digital camera or strobe light is used as a trigger signal for the distance measurement operation (three-dimensional image detection operation) in the stereo adapter. Without any special change, it is possible to detect a three-dimensional image in synchronization with the photographing of the digital camera.
[0050]
In the present embodiment, a three-dimensional image is also captured as a stereo image. As a result, the three-dimensional image and the two-dimensional image can be directly associated with each other, so that the association can be performed with higher speed and higher accuracy. However, the three-dimensional image does not have to be a stereo image. For example, the three-dimensional coordinate value of the subject calculated from one three-dimensional image is converted into a coordinate system with the left and right viewpoints of the two-dimensional image as the origin. Thus, the correspondence relationship between the pixels in the stereo image of the two-dimensional image may be roughly obtained.
[0051]
In this embodiment, the correspondence between pixels in a stereo three-dimensional image is obtained using a three-dimensional coordinate value calculated from the three-dimensional image. Pattern matching processing or the like may be performed to obtain the correspondence between pixels.
[0052]
In the present embodiment, the three-dimensional image detection apparatus is mounted on, for example, a stereo adapter that can be mounted on a normal digital camera, but may be mounted on a stereo camera dedicated to stereo photography.
[0053]
In the present embodiment, the two-dimensional image is taken by a digital camera, but the two-dimensional stereo picture may be taken by a camera using a silver salt film. At this time, if the correspondence between each pixel of the CCD for detecting a three-dimensional image and the point on the silver salt film is known, the correspondence between the left and right silver salt films can be roughly determined by the same method as in this embodiment.
[0054]
【The invention's effect】
As described above, according to the present invention, it is possible to obtain a three-dimensional image detection apparatus for simplifying the corresponding point search process in a stereo image at high speed.
[Brief description of the drawings]
FIG. 1 is a perspective view when a stereo adapter type three-dimensional image detection apparatus according to an embodiment of the present invention is mounted on a conventional digital camera.
2 is a horizontal cross-sectional view schematically showing the structure of an optical system related to photographing a stereo image in a digital camera among the imaging optical system of the stereo adapter shown in FIG.
3 is a horizontal diagram schematically showing the structure of an optical system for taking a stereo image in a three-dimensional image pickup CCD provided in the stereo adapter among the imaging optical system of the stereo adapter shown in FIG. 1; It is sectional drawing and a vertical sectional view.
FIG. 4 is a block diagram schematically illustrating a circuit configuration of the stereo adapter according to the embodiment.
FIG. 5 is a first half of a flowchart of a program executed in the stereo adapter of the present embodiment.
6 is the latter half of the flowchart shown in FIG.
FIG. 7 is a diagram for explaining the principle of distance measurement using distance measuring light.
FIG. 8 is a diagram showing light amount distribution received by ranging light, reflected light, gate pulse, and CCD.
FIG. 9 is a diagram for explaining the principle of stereoscopic photogrammetry using stereo images.
FIG. 10 is a diagram for explaining a principle when a correspondence between pixels is obtained between left and right three-dimensional images captured as a stereo image in the present embodiment.
[Explanation of symbols]
10 Digital camera
13 Lens tube
20 Stereo adapter
21 Stereo lens for 2D images
22 Stereo lens for 3D images
23 Light source device
27, 50 CCD

Claims (7)

Distance measuring means for detecting a three-dimensional image in which a pixel value corresponds to the distance to the subject by irradiating the subject with distance measuring light and receiving and detecting the reflected light with an imaging device ;
A stereo optical system for a two-dimensional image for capturing a stereo image of a two-dimensional image which is a normal visual image;
A corresponding point search means for searching for a corresponding point in a stereo image of a two-dimensional image captured using the stereo optical system for a two-dimensional image based on the three-dimensional image ;
A three-dimensional image detection apparatus comprising: a three-dimensional image stereo optical system for capturing the three-dimensional image as a stereo image . The three-dimensional image detection according to claim 1 , wherein the corresponding point search unit obtains a correspondence relationship between pixels in a stereo image of a three-dimensional image captured using the stereo optical system for the three-dimensional image. apparatus. The three-dimensional image detection apparatus according to claim 2 , wherein a correspondence relationship between pixels in the stereo image of the three-dimensional image is performed based on a three-dimensional coordinate value calculated from the three-dimensional image.   The three-dimensional image detection apparatus according to claim 1, further comprising a connection unit for connecting the stereo optical system for two-dimensional images to an optical system of a camera. The three-dimensional image detection apparatus according to claim 4 , wherein the connection portion is a screw mount attached to a screw mount provided at a lens barrel tip opening portion of a camera.   The external trigger input means for detecting the external trigger signal output from the hot shoe of a camera is provided, The drive of the said distance measurement means is controlled based on the said external trigger signal. 3D image detection apparatus.   The three-dimensional image detection according to claim 1, further comprising: a strobe light detecting means for detecting strobe light as a light receiving trigger signal, wherein the driving of the distance measuring means is controlled based on the light receiving trigger signal. apparatus.

Images