JP2001148867A - Three-dimensional image detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional image detector which emits range finding light to an object so as to detect the distance up to the object for each pixel by receiving reflected light from the object, and which matches the signal level of distance information with the signal level of reflectance information on the reflectance of the surface of the object. SOLUTION: A light pulse with a width TS is repetitively emitted to the object and the sum of signal charges with respect to the light received for a storage period TU1 among the reflected light pulses (hatched line part A1) is detected as the distance information. The same range finding light is repetitively emitted to the object, the reflected light is all received for a storage period TU2 and the sum of signal charges with respect to the received light pulses (hatched line part A2) is detected as the reflectance information. The ratio TD/TS is calculated from the ratio of the distance information to the reflectance information (hatched line parts A1, A2) detected by one pixel representing the object. The number of emission times of the range finding light emitted repetitively in the detection of the reflectance information is selected as the TD/TS that is used to detect the distance information so that the sum of the signal charges used for the distance information is selected equal to the sum of the signal charges corresponding to the reflectance information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a three-dimensional image detecting apparatus for detecting a three-dimensional shape of a subject using a light propagation time measuring method.

[0002]

2. Description of the Related Art Conventionally, as a three-dimensional measurement in a three-dimensional image input apparatus, an apparatus using a light propagation time measuring method is known. "Measurement Science and Technology" (SC
In the three-dimensional image input device described in Hristie et al., vol. 6, p1301-1308 (1995), a pulse-modulated laser beam is irradiated on an object to be measured (subject), and the reflected light is reflected by an image intensifier. Is received by the two-dimensional CCD sensor having the attached, and is converted into an electric signal. The image intensifier is shutter-controlled by a gate pulse synchronized with the pulse emission of the laser beam. Further, in the apparatus disclosed in WO 97/01111, light such as a pulse-modulated laser beam is irradiated onto an object to be measured, and the reflected light is composed of a mechanical or liquid crystal element or the like and a pulse of distance measuring light. Are received by a two-dimensional CCD sensor combined with an electro-optical shutter controlled at different timings and are converted into electric signals. According to these configurations,
Since the amount of light received by reflected light from a distant object to be measured is smaller than the amount of light received by reflected light from a near object to be measured, an output corresponding to the distance to the object to be measured is obtained for each pixel of the CCD.

[0003]

However, the distance information detected by the above-described method includes an error due to a difference in reflectance on the surface of the subject (measured object). Therefore, in order to obtain distance information with high detection accuracy, it is necessary to correct these errors. In order to correct an error due to a difference in reflectance, it is necessary to detect information on reflectance (reflectance information). However, since the signal level of the reflectance information is higher than the signal level of the distance information, if the operation range of the A / D converter is set to the signal level of the reflectance information, the quantization S / N ratio for the distance information deteriorates. Resulting in.

The present invention provides a quantization S / N for distance information.
It is an object of the present invention to obtain a three-dimensional image detection device capable of detecting highly accurate distance information without deteriorating the ratio.

[0005]

A three-dimensional image detecting apparatus according to the present invention irradiates a subject with distance measuring light, receives reflected light from the subject within a predetermined accumulation period in an image pickup unit, and generates a light beam based on the reflected light. A first charge accumulating means for accumulating one signal charge for each pixel; and a first charge accumulating means driven repeatedly N times to integrate the first signal charge accumulated in each drive for each pixel. A first signal charge integrating means for accumulating the first integrated charge, irradiating the subject with distance measuring light, and receiving the reflected light from the subject in the imaging unit, thereby obtaining the total amount of the received reflected light And a second charge accumulating means for accumulating a second signal charge corresponding to each pixel for each pixel, and driving the second charge accumulating means repeatedly M times, and accumulating the second signal charge accumulated in each drive for each pixel. The second integral charge by integrating each time The second signal charge integrating means, and the first signal charge integrating means so that the first integrated charge and the second integrated charge have substantially the same size in a pixel corresponding to the measurement area of the subject. Adjusting means for adjusting the number of repetitions or the number of repetitions in the second signal charge integrating means, wherein the first integrated charge and the second
The distance to the subject is detected for each pixel on the basis of the integrated charge.

Preferably, the number-of-repetitions adjusting means adjusts the number of repetitions based on a result of pre-ranging which roughly measures the distance to the subject in advance. At this time, preferably, the pre-ranging is performed by driving the first and second signal charge integration means with the same number of repetitions in the first and second signal charge integration means. Thereby, the number of repetitions can be adjusted more appropriately.

More preferably, the number-of-repetitions adjusting means repeats based on the first and second integrated charges in one or more sample pixels which are detected in the pre-ranging and represent pixels corresponding to the measurement area of the subject. Adjust the number of times.
At this time, for example, the number-of-repetitions adjusting means adjusts the number of repetitions so that the ratio of the first integrated charge to the second integrated charge in one sample pixel is 1: 1. This makes it possible to adjust the number of repetitions quickly and easily.

[0010] Preferably, the repetition number adjusting means is adjusted by making the number of repetitions in the first charge integration means constant and reducing the number of repetitions in the second charge integration means.

Preferably, the three-dimensional image detecting device is provided with an aperture stop for adjusting the illuminance of the image plane of the optical system, and when performing pre-ranging, the number-of-repetitions adjusting means and the first and second signals are used. The measurement is performed in a state where the aperture stop is narrowed as compared with the main ranging in which the distance to the subject is measured by driving the charge integration means.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a perspective view of a camera-type three-dimensional image detection device according to an embodiment of the present invention.

On the front of the camera body 10, a finder window 12 is provided at the upper left of the taking lens 11, and a flash 13 is provided at the upper right. A light emitting device (light source) 14 for irradiating a laser beam, which is a distance measuring light, is provided on the upper surface of the camera body 10 and directly above the taking lens 11. On the left side of the light emitting device 14, a release switch 15,
A liquid crystal display panel 16 is provided, and a mode change dial 17 and a V / D mode change switch 18 are provided on the right side. On the side surface of the camera body 10, a card insertion slot 19 for inserting a recording medium such as an IC memory card is formed, and a video output terminal 20 and an interface connector 21 are provided.

FIG. 2 is a block diagram showing a circuit configuration of the camera shown in FIG. An aperture 25 is provided in the taking lens 11. The opening of the aperture 25 is determined by the iris drive circuit 2.
Adjusted by 6. The focus adjustment operation and the zooming operation of the taking lens 11 are controlled by a lens drive circuit 27.

A CCD (image pickup unit) 28 is provided on the optical axis of the taking lens 11. A subject image is formed on the CCD 28 by the photographing lens 11, and charges corresponding to the subject image are generated. Operations such as a charge accumulation operation and a charge read operation in the CCD 28 are controlled by the CCD drive circuit 30. The charge signal, that is, the image signal, read from the CCD 28 is amplified by the amplifier 31 and the A / A
The analog signal is converted into a digital signal in the D converter 32. The digital image signal is output from the imaging signal processing circuit 3
In the image memory 3, processing such as gamma correction is performed.
4 is temporarily stored. The iris drive circuit 26, the lens drive circuit 27, and the imaging signal processing circuit 33 are controlled by a system control circuit 35.

The CCD driving circuit 30 includes a V / D switching circuit 2
9 is controlled in accordance with the pulse signal output from. V /
The pulse signal output from the D switching circuit 29 is the pulse signal output from the normal drive pulse generation circuit 24 or 3
A pulse signal generated by the D drive pulse generation circuit 22 and output through the pulse number conversion circuit 23, and selectively supplied to the CCD drive circuit 30 in accordance with the setting of the V / D mode switch 18 (FIG. 1). Is output. The pulse signal output from the normal drive pulse generation circuit 24 is for performing a normal photographing operation in the CCD 28. On the other hand, the pulse signal output from the 3D drive pulse generation circuit 22 is a pulse signal used when the CCD 28 detects distance information. The pulse number conversion circuit 23 controls the number of pulses of the pulse signal output from the 3D drive pulse generation circuit 22 based on a signal command from the system control circuit 35. The 3D drive pulse generation circuit 22, the normal drive pulse generation circuit 24, and the pulse number conversion circuit 23 are controlled by a system control circuit 35.

The image signal temporarily stored in the image memory 34 is read from the image memory 34 and supplied to the LCD drive circuit 36. The LCD drive circuit 36 operates according to the image signal, whereby an image corresponding to the image signal is displayed on the image display LCD panel 37.

If the camera is connected to a monitor device 39 provided outside the camera body 10 by a cable, the image signal read from the image memory 34 is transmitted to the TV signal encoder 3.
8. It can be transmitted to the monitor device 39 via the video output terminal 20. The system control circuit 35 is connected to the interface circuit 40, and the interface circuit 40 is connected to the interface connector 21. Therefore, if the camera is connected via an interface cable to a computer 41 provided outside the camera body 10, the image signal read from the image memory 34 can be transmitted to the computer 41. The system control circuit 35 is connected to the image recording device 43 via the recording medium control circuit 42. Therefore, the image signal read from the image memory 34 is stored in a recording medium M such as an IC memory card mounted on the image recording device 43.
Can be recorded in

The light emitting device 14 comprises a light emitting element 14a and an illumination lens 14b. The light emitting operation of the light emitting element 14a is controlled by a light emitting element control circuit 44. Light emitting element 1
Reference numeral 4a denotes a laser diode (LD), and the emitted laser light is used as distance measuring light for detecting the distance to the subject. This laser light is applied to the entire subject through the illumination lens 14b. Light reflected by the subject enters the photographing lens 11. By detecting this light with the CCD 28, distance information on the surface shape of the subject is obtained as a three-dimensional image, as described later.

The light emitting element control circuit 44 is controlled according to a pulse signal output from the pulse number conversion circuit 23. The pulse signal output to the light emitting element control circuit 44 is
Like the pulse signal output to the CCD drive circuit 30 via the V / D switching circuit 29, the 3D drive pulse generation circuit 22
Is based on the pulse signal generated in the step (1), and the number of pulses is converted by a signal command of the system control circuit 35. Thereby, the light emitting element 1
The number of light emission pulses emitted from 4a is adjusted.

The system control circuit 35 is connected to a switch group 45 including a release switch 15, a mode switching dial 17, and a V / D mode switching switch 18, and a liquid crystal display panel (display element) 16.

Next, the photographing operation in the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart of the photographing operation program.

When it is confirmed in step 101 that the release switch 15 is fully depressed, step 1 is executed.
02 is executed, video (V) mode and distance measurement (D)
It is determined which of the modes is selected. Switching between these modes is performed by operating the V / D mode switch 18.

If it is determined in step 102 that the D mode is selected, in step 103, the pre-ranging, which is a preliminary measurement of the actual ranging (step 105), sets the iris 25 to be smaller than the F-number of the actual ranging. It is performed in a state where the aperture is narrowed down (for example, the aperture is narrowed down by two steps from the F number in the actual distance measurement), and reflectance information for correcting an error caused by a difference in reflectance to a subject or a reflectance on a surface of the subject, etc. Is detected.

In step 104, based on the distance information and the reflectance information detected in step 103, the number of pulses of the distance measuring light emitted in the detection of the reflectance information in the actual distance measurement (step 105) is calculated. Is calculated. In the actual distance measurement in step 105, the distance information to the subject and the reflectance information are detected again. In the detection of the reflectance information, the distance is the distance measurement light according to the pulse number of the distance measurement light calculated in step 104. Pulse light is applied. This is because, as will be described later, the quantization S
This is to prevent the deterioration of the / N ratio, whereby the level of the signal output from the CCD 28 when detecting the distance information becomes substantially equal to the level of the signal output from the CCD 28 when detecting the reflectance information. . Then step 106
Is stored in the recording medium M,
This shooting operation program ends. Note that the number of distance measurement light pulses applied to the subject is the same in the detection of distance information and the detection of reflectance information in pre-ranging.

On the other hand, if it is determined in step 102 that the setting mode is the V mode, the control of the distance measuring light is determined to be off in step 107, and the video control of the CCD is determined to be on in step 108. . That is, image information (image data) of the subject is detected by a normal photographing operation of the CCD. In step 109, the detected image data is stored in the recording medium M, and the program of the photographing operation ends.

Next, the principle of distance measurement in the present embodiment will be described with reference to FIGS. FIG.
, The horizontal axis is time t.

The distance measuring light output from the distance measuring device B is reflected by the subject S and received by a CCD (not shown). The distance measuring light is a pulsed light having a predetermined pulse width H, and therefore, the reflected light from the subject S is also a pulsed light having the same pulse width H. The rise of the reflected light pulse is delayed by a time δ · t (δ is a delay coefficient) from the rise of the distance measuring light pulse. Since the ranging light and the reflected light have traveled twice the distance r between the distance measuring device B and the subject S, the distance r is given by r = δ · t · C / 2 (1) can get. Where C is the speed of light.

For example, a state in which reflected light can be detected from the rise of the pulse of the distance measuring light is determined, and the state is switched to an undetectable state before the reflected light pulse falls, that is, a reflected light detection period T is provided. And the received light amount A during the reflected light detection period T is a function of the distance r. That is, the light receiving amount A decreases as the distance r increases (the time δ · t increases).

In this embodiment, utilizing the above-described principle,
The distance from the camera body 10 to each point on the surface of the subject S is detected by detecting the amount of received light A at each of a plurality of photodiodes provided in the CCD 28 and arranged two-dimensionally. Data of a three-dimensional image relating to a shape is input collectively.

FIG. 6 is a diagram showing the arrangement of the photodiode 51 and the vertical transfer unit 52 provided on the CCD 28.
FIG. 7 is a cross-sectional view of the CCD 28 cut along a plane perpendicular to the substrate 53. This CCD 28 is a conventionally known interline type CCD, and uses a VOD for sweeping out unnecessary charges.
(Vertical overflow drain) method.

The photodiode 51 and the vertical transfer section 52 are formed along the surface of the n-type substrate 53. The photodiodes 51 are two-dimensionally arranged in a lattice, and the vertical transfer units 52 are provided adjacent to the photodiodes 51 arranged in a line in a predetermined direction (the vertical direction in FIG. 6). The vertical transfer section 52 has four vertical transfer electrodes 52a, 52b, 52c, and 52d for one photodiode 51. Therefore, in the vertical transfer section 52, four potential wells can be formed, and signal charges can be output from the CCD 28 by controlling the depths of these wells as conventionally known. The number of vertical transfer electrodes can be freely changed according to the purpose.

A photodiode 51 is formed in a p-type well formed on the surface of the substrate 53, and the p-type well is completely depleted by a reverse bias voltage applied between the p-type well and the n-type substrate 53. You. In this state, charges corresponding to the amount of incident light (reflected light from the subject) are accumulated in the photodiode 51. When the substrate voltage Vsub is increased to a predetermined value or more, the electric charge accumulated in the photodiode 51 is discharged to the substrate 53 side. On the other hand, when a charge transfer signal (voltage signal) is applied to the transfer gate unit 54, the charges accumulated in the photodiode 51 are transferred to the vertical transfer unit 52. That is, after the charges are swept to the substrate 53 side by the charge sweeping signal, the signal charges accumulated in the photodiode 51 are changed by the charge transfer signal to the vertical transfer unit 5.
It is transferred to the two sides. By repeating such an operation, signal charges are integrated in the vertical transfer unit 52, and a so-called electronic shutter operation is realized.

FIG. 8 is a timing chart of the distance information detecting operation executed in the pre-ranging in step 103 and the main ranging in step 105. 1, 2, and 6
The distance information detecting operation in the present embodiment will be described with reference to FIGS. In the distance information detecting operation of the present embodiment, unlike the description of the principle of the distance measurement performed with reference to FIG. 5, in order to reduce noise due to the influence of external light, the distance measurement light is reflected from the falling edge of the pulse. The timing chart is configured so that the light can be detected, and the state is switched to the undetectable state after the pulse of the reflected light has fallen, but it is not different in principle.

A charge discharge signal (pulse signal) S1 is output in synchronization with the output of the vertical synchronization signal (not shown), whereby unnecessary charges stored in the photodiode 51 are discharged in the direction of the substrate 53. Then, the accumulated charge amount in the photodiode 51 becomes zero (reference S2). After the start of output of the charge sweeping signal S1, pulse-shaped ranging light S3 having a constant pulse width is output. The period (pulse width) during which the ranging light S3 is output can be adjusted.
The adjustment is performed so that the distance measurement light S3 is turned off simultaneously with the output of the charge sweeping signal S1.

The distance measuring light S3 is reflected by the object, and
It is incident on D28. That is, although the reflected light S4 from the subject is received by the CCD 28, the charge sweeping signal S1
Is not stored in the photodiode 51 (reference S2). Charge sweep signal S1
Is stopped, the photodiode 51 starts to accumulate charges by receiving the reflected light S4, and the reflected light S4
4 and the external light generate signal charges S5. When the reflected light S4 disappears (reference S6), the photodiode 51 ends the charge accumulation based on the reflected light (reference S7).
Charge accumulation due to only external light continues (reference S8).

Thereafter, when the charge transfer signal S9 is output, the charges accumulated in the photodiode 51 are transferred to the vertical transfer section 52. This charge transfer is completed when the output of the charge transfer signal ends (reference S10). In other words, the charge accumulation in the photodiode 51 continues due to the presence of external light, but the signal charge S1 accumulated in the photodiode 51 until the output of the charge transfer signal ends.
1 is transferred to the vertical transfer unit 52. The charge S14 accumulated after the output of the charge transfer signal ends remains in the photodiode 51 as it is.

As described above, the period T from the end of the output of the charge sweeping signal S1 to the end of the output of the charge transfer signal S9.
_{During U1} , the photodiode 51 accumulates signal charges corresponding to the distance to the subject. Then, the reflected light S4
Until the end of light reception (reference S6), the charges accumulated in the photodiode 51 are transferred to the vertical transfer unit 52 as signal charges S12 (hatched portion) corresponding to the distance information of the subject,
Other signal charges S13 are caused only by external light.

After a predetermined time has elapsed from the output of the charge transfer signal S9, the charge sweeping signal S1 is output again, and after the transfer of the signal charge to the vertical transfer section 52, the photodiode 51
Unnecessary charges accumulated in the substrate 53 are swept toward the substrate 53.
That is, accumulation of signal charges in the photodiode 51 is newly started. Then, as described above, when the charge accumulation period T _U1 has elapsed, the signal charge is transferred to the vertical transfer unit 52.

The vertical transfer section 5 of such signal charges S11
2 is repeatedly executed until the next vertical synchronization signal is output. As a result, in the vertical transfer unit 52, the signal charge S11 is integrated, and the signal charge S11 integrated in a period of one field (a period sandwiched between two vertical synchronization signals) is regarded as a period in which the subject is stationary. If possible, it corresponds to distance information to the subject.

The detection operation of the signal charge S11 described above relates to one photodiode 51, and such a detection operation is performed in all the photodiodes 51. As a result of the detection operation in the period of one field, the vertical transfer unit 52 adjacent to each photodiode 51
The distance information detected by the photodiode 51 is held in each of the parts. This distance information is output from the CCD 28 by a vertical transfer operation in the vertical transfer unit 52 and a horizontal transfer operation in a horizontal transfer unit (not shown).

However, the reflected light detected by the CCD 28 is affected by the reflectance of the surface of the subject. Therefore, the distance information obtained via the reflected light includes an error caused by the reflectance. In addition, the reflected light detected by the CCD 28 includes components such as external light in addition to the reflected light from the subject, and there is an error due to this. A method for correcting these errors will be described with reference to the flowchart of the next distance measurement operation.

FIGS. 12 and 13 are flow charts of the distance measuring operation executed in the pre-distance measurement in step 103 and the main distance measurement in step 105 (see FIG. 3). The distance measurement operation in the present embodiment will be described with reference to FIGS. 9 to 11
5 is a timing chart in the operation of detecting distance correction information, reflectance information, and reflectance correction information.

In step 201, a vertical synchronizing signal is output, and ranging light control is started. That is, the light emitting device 14 is driven, and the pulse-shaped ranging light S3 is output intermittently. Next, step 202 is executed and the CCD
28 starts detection control. That is, the distance information detection operation described with reference to FIG. 8 is started, the charge sweeping signal S1 and the charge transfer signal S9 are alternately output, and the signal charge S11 of the distance information is integrated in the vertical transfer unit 52.

In step 203, it is determined whether one field period has elapsed from the start of the distance information detection operation, that is, whether a new vertical synchronizing signal has been output.
When one field period ends, integration of the signal charges S11 over one field period is completed, and the integrated signal charges are output from the CCD 28 in step 204. This integrated signal charge corresponds to the distance information and is temporarily stored in the image memory 34 in step 205. In step 206, the ranging light control is switched to the off state, and the light emitting operation of the light emitting device 14 is stopped.

In steps 207 to 210, an operation of detecting distance correction information (see FIG. 9) is performed. First, in step 207, a vertical synchronizing signal is output and the CCD
28 starts detection control. That is, the light emitting device 1
In the state where the light source is turned off without performing the light emitting operation of No. 4, the charge discharge signal S21 and the charge transfer signal S22 are output alternately. The charge accumulation time T _U1 is the same as the distance information detection operation shown in FIG. 8, but since no distance measurement light is irradiated on the subject (reference S23), there is no reflected light (reference S).
24). Therefore, no signal charge of the distance information is generated, but a disturbance component such as external light is incident on the CCD 28.
The signal charge S25 corresponding to the disturbance component is generated, and the signal charge S26 that has been accumulated in the photodiode is transferred to the vertical transfer unit by the output of the charge transfer signal S22. This signal charge S26 corresponds to distance correction information for the charge accumulation time T _U1 for correcting the influence of the disturbance component on the distance information.

In step 208, it is determined whether one field period has elapsed from the start of the operation of detecting the distance correction information, that is, whether a new vertical synchronizing signal has been output. When one field period ends, integration of the signal charge S26 over one field period is completed, and step 20 is performed.
At 9, the integrated signal charges are output from the CCD 28. This integrated signal charge corresponds to the distance correction information, and is temporarily stored in the image memory 34 in step 210.

In steps 211 to 216, an operation of detecting reflectance information (see FIG. 10) is performed. Step 21
At 1, the vertical synchronization signal is output and the ranging light control is started, and the pulsed ranging light S33 is output intermittently. In step 212, the detection control by the CCD 28 is started, and the charge discharge signal S31 and the charge transfer signal S3
5 are output alternately. By outputting the charge sweeping signal S31, the accumulated charge amount in the photodiode becomes zero (reference S32). Charge sweep signal S31
Is completed, the ranging light S33 is output and the CCD
Is reflected light S34. After the reflected light S34 has disappeared, a charge transfer signal S35 is output. That is, the operation of detecting the reflectance information is performed such that all of the reflected light S34 is received within the charge accumulation period T _U2 from the end of the output of the charge sweep signal S31 to the end of the output of the charge transfer signal S35. Controlled.

As described above, in the photodiode 51, while receiving the reflected light S34, the reflected light S34 and the signal charge S36 caused by the external light are accumulated, and the reflected light S34 is accumulated.
Is not received, the signal charge S3 caused only by external light
7, S38 is accumulated. Then, by the output of the charge transfer signal S35, the signal charges S39 stored in the photodiode up to that time are transferred to the vertical transfer unit. This signal charge S39 corresponds to the reflectance information and includes a component S'39 based on external light.

In step 213, it is determined whether one field period has elapsed from the start of the reflectance information detection operation, that is, whether a new vertical synchronizing signal has been output. When one field period ends, one of the signal charges S39 becomes one.
Integration over the field period is complete, step 214
, The integrated signal charge is output from the CCD 28. This integrated signal charge corresponds to the reflectance information, and is temporarily stored in the image memory 34 in step 215. In step 216, the ranging light control is switched to the off state, and the light emitting operation of the light emitting device 14 is stopped.

In steps 217 to 220, an operation of detecting reflectance correction information (see FIG. 11) is performed. In step 217, the vertical synchronizing signal is output and the CCD
28 starts detection control. That is, the light emitting device 1
In the state where the light source is turned off without performing the light emitting operation of No. 4, the charge sweeping signal S41 and the charge transfer signal S42 are output alternately. The charge accumulation time T _U2 is the same as that of the reflectance information detection operation shown in FIG. 10, but no reflected light exists (reference S44) because no distance measurement light is applied to the subject (reference S43). Accordingly, no signal charge of the reflectance information is generated, but a signal charge S46 corresponding to a disturbance component such as external light is generated in the CCD 28. The signal charge S46 corresponds to reflectance correction information for correcting the influence of the disturbance component on the reflectance information for the charge accumulation time T _U2 .

In step 218, it is determined whether one field period has elapsed from the start of the operation of detecting the reflectance correction information.
That is, it is determined whether a new vertical synchronization signal has been output. When one field period ends, the signal charges S47
Is completed over one field period of
At 19, the integrated signal charges are output from the CCD. This integrated signal charge corresponds to the reflectance correction information, and is temporarily stored in the image memory 34 in step 220.

In step 221, steps 201 to 2
The arithmetic processing of the distance data is performed using the distance information, the distance correction information, the reflectance information, and the reflectance correction information obtained in step 20, and the subroutine of the distance measurement operation ends.

Next, the contents of the arithmetic processing executed in step 221 will be described with reference to FIGS. It is assumed that a subject having a reflectance R is illuminated, and the subject is regarded as a secondary light source having a luminance I and is imaged on a CCD. At this time, the output Sn obtained by integrating the charge generated in the photodiode 51 during the charge storage time t is represented by: Sn = kR.I.t (2) Here, k is a proportionality constant, which varies depending on the F number, magnification, and the like of the taking lens.

[0053] When the object is illuminated with light from a light source such as a laser, the luminance I becomes were synthesized with the luminance I _B by the luminance I _S and the background light due to the light source, I = I _S + I _B · ·・ (3)

As shown in FIG. 8, the charge storage time is T _U1 , the pulse width of the distance measuring light S3 is T _S , and the pulse width of the distance information signal charge S12 is T _D, and the charge storage during one field period is performed. When the time is to be repeated N times, the output SM _{10 obtained, SM 10 = Σ (k ·} R (I S · T D + I B · T U1)) = k · N · R (I S · T D + I _B · T _U1 ) (4) The pulse width T _D can be expressed as T _D = δ · t = 2r / C (5)

As shown in FIG. 10, the charge storage time T _U2
Is sufficiently larger than the period (pulse width) T _S of the pulse-shaped ranging light S33, and the output SM ₂₀ obtained when controlled so as to include the entire unit light receiving time of the reflected light is SM ₂₀ = Σ ( _{k · R (I S · T} S + I B · T U2)) = k · N · R (I S · T S + I B · T U2) becomes (6).

Light emission is stopped as shown in FIG.
Output SM ₁₁ obtained when performing a pulsed charge accumulation at the same time width _{is, SM 11 = Σ (k ·} R · I B · T U1) = k · N · R · I B · T U1 (7) Similarly, the output SM ₂₁ obtained when performing charge accumulation as shown in Figure _{11, SM 21 = Σ (k ·} R · I B · T U2) = k · N · R · I B · T _U2 (8)

[0057] (4), (6), (7), from equation _{(8), S D = (SM 10 -SM} 11) / (SM 20 -SM 21) = T D / T S ··· (9 ) Is obtained.

[0058] disturbance component of each outside light or the like and the distance measuring light S3 are in the reflected light S4 as described above (luminance I _B due to the background light)
It is included. T _D / T _{S in} equation (9) is the distance measuring light S3
The amount of reflected light S4 from the subject when the light is irradiated is normalized by the amount of distance measuring light S3, which is the amount of distance measuring light S3 (corresponding to signal charge S11 in FIG. 8).
The disturbance component (corresponding to the signal charge S'39 in FIG. 11) is calculated from the value obtained by removing the disturbance component (corresponding to the signal charge S26 in FIG. 9) from the light amount of the reflected light S4 (corresponding to the signal charge S39 in FIG. 10). Equal to the ratio with the value removed.

Each output value SM ₁₀ , SM ₁₁ , SM of the equation (9)
₂₀ and SM ₂₁ are performed in steps 205, 210, 215 and 220.
, Are stored as distance information, distance correction information, reflectance information, and reflectance correction information. Therefore, T _D / T _S is obtained based on these pieces of information. Pulse width T _s
Is known, the distance r is obtained from the equation (5) and T _D / T _S. That is, 2r = C · T _S · (SM ₁₀ −SM ₁₁ ) / (SM ₂₀ −SM ₂₁ ) (10)

Next, the number of pulses of the distance measuring light calculated in step 104 of FIG. 3 will be described with reference to FIGS.

FIG. 14 shows the relationship between the pulse light (light receiving pulse) received by a certain pixel and the accumulation period in the pre-ranging. FIG. 14A shows a case where distance information is detected.
FIG. 14B shows the relationship between the pulse light (light receiving pulse) received by the CCD 28 and the accumulation period T _U1 .
When detecting the reflectivity information, it illustrates a relationship between the accumulation period T _U2 and received by the pulsed light (light pulse) by CCD 28. 14A and 14B, the influence of external light is omitted. Further, the accumulation period is represented by a pulse-like waveform, and the period during which the pulse is raised corresponds to the accumulation period.

In the detection of the distance information, only the hatched portion A1 which is a part of the light receiving pulse is received within the accumulation period _{TU1 and} is accumulated as signal charges (FIG. 14A). When the pulse height of the light receiving pulse is h, the signal charges stored in the detection of the distance information corresponds to the area h × T _D of the hatched part A1, which represents the distance information to the object. In the detection of the reflectance information, all of the light receiving pulses are received within the accumulation period _{TU2 and} are accumulated as signal charges. The signal charge accumulated at this time is the area h of the shaded portion (entire light receiving pulse) A2
× T _S and represents reflectance information. As described above, the signal charge accumulation operation is repeatedly performed a predetermined number of times (for example, N times) over one field period, and is integrated in the vertical transfer unit 52. The integrated signal charge is converted into a signal potential, and is converted via an amplifier 31 into an A / D converter 3.
2 is output.

SM _{10 in} equation (4) and SM _{20 in} equation (6)
Ignoring the influence of external light, SM ₁₀ = k ・ N ・ R ・_IS・ T _D = N ・ h ・ T _D ... (11) SM ₂₀ = k ・ N ・ R ・I _s T _s = N · h · T _s (12) (where h = k · R · I _s ). Therefore, the level (signal level) of the signal potential with respect to the integrated signal charge is the area (h · T _D , h) of the hatched portions A1 and A2.
T _S ) and the number of integrations N.

Since the hatched portion A2 is the entire light receiving pulse, while the hatched portion A1 is a part of the received light pulse, the signal charge of the distance information corresponding to the hatched portion A1 has the reflectance corresponding to the hatched portion A2. There are cases where the signal charge becomes significantly smaller than the signal charge of information. At this time, if the number of integrations N is constant in each detection, the signal level of the distance information output to the A / D converter 32 is significantly lower than the signal level of the reflectance information. When the input level of the A / D converter 32 is set so that the signal level of the reflectance information does not overload the A / D converter 32, the number of effective bits relative to the signal level of the distance information is relatively reduced. And the quantization S / N ratio for the distance information is degraded. For example, when the signal level of the reflectance information is 0 to V volts and the signal level of the distance information is 1 / n, that is, 0 to V / n volts, A / D conversion that performs linear quantization of P bits The operating range of the device is 0 to V according to the signal level of the distance information.
When set to volts, the reflectance information is quantized by P bits, but the distance information is quantized by P-Int (log ₂ n) bits, and the quantization S /
The N ratio deteriorates. Here, Int () is a function for converting the decimal part of the numerical value in parentheses to a truncated integer.

In order to prevent the deterioration of the quantized S / N ratio with respect to the distance information, it is necessary to match the signal level of the reflectance information output to the A / D converter 32 with the signal level of the distance information. The signal level of the signal level and reflectivity information of the distance information, since the correlated respectively to the integrated signal charges (output) SM _10, SM _20, distance information, to equalize the level of the signal of the reflectance information, the output SM ₁₀ and SM ₂₀ may be made substantially equal. Here, the outputs SM ₁₀ and SM ₂₀ are substantially equal when the output SM ₂₀ is within, for example, twice the output SM ₁₀ . In this distance measurement, this adjustment is performed by controlling the number of integrations when detecting the reflectance information.

FIG. 15 shows the relationship between the pulse light (light receiving pulse) received by a certain pixel and the accumulation period in the actual distance measurement. FIG. 15A shows that when detecting distance information, C
FIG. 15B shows the relationship between the pulse light (light receiving pulse) received by the CD 28 and the accumulation period T _U1 . FIG. 15B shows the pulse light (light receiving) received by the CCD 28 when the reflectance information is detected. (Pulse) and the accumulation period T _U2 . In this figure, as in FIG. 14, the influence of external light is omitted, and the accumulation period is represented by a pulse-like waveform.

Output to each pixel from the result of pre-ranging
SM_TenAnd output SM₂₀Ratio T with_D: T _SIs calculated. Suffered
In the measured area of the object, the output SM_Ten, SM₂₀Is an abbreviation
In the subject area to be measured
SM in one pixel (sample pixel) representing a pixel
_Ten= SM₂₀When detecting reflectance information such that
Determine the number of times. That is, when detecting reflectance information
Is the number of integrations of N ', and the number of integrations when detecting distance information
Is N and SM_TenAnd SM₂₀And the sample image in pre-ranging
Assuming that the output is a prime, N '= Int ((SM_Ten/ SM₂₀) · N) = Int ((T_D/ T_S) · N) (13) is used to calculate the number of integrations N ′. Reflectivity in actual ranging
In the detection of information, the signal charge is accumulated only N 'times calculated.
Is not performed, and as shown in FIG.
The number of integrations is smaller than when the distance information is detected in (a).
The sum of the area of the hatched portion A1 'and the area of the hatched portion A2'
The sums are equal. As a result, distance information and
When detecting the reflectance and reflectance information, the
The signal level output from the element becomes approximately equal, and the distance information
, The deterioration of the quantized S / N ratio can be prevented.

Next, the difference in the setting of the aperture (iris 25) between the pre-distance measurement and the actual distance measurement will be described with reference to FIGS.
This will be described with reference to FIG.

[0069] Since the operating range of the A / D converter 32 is set according to the signal level at the distance measurement (the signal level of the output SM _10), when detecting the reflectivity information by the pre-distance-measuring, The output SM ₂₀ may overload the A / D converter 32. Therefore, in pre-ranging, F
The number is set to be large, and the illuminance of the reflected light on the image plane of the CCD 28 is suppressed as compared with the case of the actual distance measurement.

As described above, according to the present embodiment, the pre-ranging is performed, and based on the measurement result, the number of integrations of the signal charge when the reflectance information is detected is controlled.
It is possible to detect highly accurate distance information in consideration of the reflectance information without deteriorating the quantization S / N ratio for the distance information. Also, by setting the F-number in the pre-ranging to be larger than the F-number in the main ranging, the signal level of the reflectance information in the pre-ranging may overload the A / D converter. , The operating range of the A / D converter can be set to the signal level in the main distance measurement.

In this embodiment, 1 pixel is used as the sample pixel.
Although one pixel is used, a plurality of pixels may be used as sample pixels, and the number of integrations N ′ may be calculated from the average value of T _D / T _{S over} the entire sample pixel.

[0072]

As described above, according to the present invention, it is possible to obtain a three-dimensional image detecting apparatus capable of detecting highly accurate distance information without deteriorating the quantization S / N ratio for the distance information.

[Brief description of the drawings]

FIG. 1 is a perspective view of a camera-type three-dimensional image detection device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a circuit configuration of the camera shown in FIG.

FIG. 3 is a flowchart of a shooting operation program executed in the embodiment.

FIG. 4 is a diagram for explaining the principle of distance measurement using distance measurement light.

FIG. 5 Distance measuring light, reflected light, gate pulse, and CCD
FIG. 4 is a diagram showing a distribution of the amount of light received by the light source.

FIG. 6 is a diagram illustrating an arrangement of a photodiode and a vertical transfer unit provided in a CCD.

FIG. 7 is a cross-sectional view of the CCD cut along a plane perpendicular to the substrate.

FIG. 8 is a timing chart when detecting distance information.

FIG. 9 is a timing chart when detecting distance correction information.

FIG. 10 is a timing chart when detecting reflectance information.

FIG. 11 is a timing chart when detecting reflectance correction information.

FIG. 12 is a first half of a flowchart of a program relating to a distance measuring operation.

FIG. 13 is a second half of a flowchart of a program relating to a distance measuring operation.

FIG. 14 is a diagram illustrating a relationship between a light receiving pulse and an accumulation period when detecting distance information and reflectance information in pre-ranging.

FIG. 15 is a diagram illustrating a relationship between a light receiving pulse and an accumulation period when detecting distance information and reflectance information in the main distance measurement.

[Explanation of symbols]

 14 light emitting device (light source) 23 pulse number conversion circuit 28 CCD (imaging unit)

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (reference) H04N 5/335 G02B 7/11 FF term (reference) 2F065 AA04 AA06 AA53 BB05 DD03 DD04 EE05 FF12 FF32 GG04 HH04 JJ00 JJ03 JJ18 JJ26 NN12 QQ00 QQ03 QQ14 QQ24 QQ34 QQ51 SS13 2F112 AD01 BA06 BA07 CA08 CA12 DA25 DA28 EA03 FA03 FA07 FA21 FA29 FA33 FA45 2H051 AA00 BB27 CC02 CE01 CE06 CE14 EB20 5C024 AA00 GA03 AB03 JA06 FA06 AB08 AB12

Claims (7)

[Claims] 1. A first object for irradiating a subject with distance measuring light, receiving reflected light from the subject within a predetermined accumulation period in an imaging unit, and accumulating a first signal charge generated by the pixel for each pixel. And the first charge storage means is repeatedly driven N times, and the first signal charge stored in each drive is integrated for each pixel to store a first integrated charge. A first signal charge integrating means, and a second light source corresponding to the total amount of the reflected light received by irradiating the subject with distance measuring light and receiving reflected light from the subject in the imaging unit. A second charge accumulating means for accumulating a signal charge for each of the pixels; and repeatedly driving the second charge accumulating means M times, wherein the second signal charge accumulated in each drive is stored for each of the pixels. By integrating, the second integral A second signal charge integrating means for accumulating a load, wherein the first integrated charge and the second integrated charge have substantially the same size in a pixel corresponding to a measurement area of the subject. Adjusting means for adjusting the number of repetitions in the signal charge integrating means or the number of repetitions in the second signal charge integrating means, and based on the first integrated charge and the second integrated charge. A three-dimensional image detection device, wherein a distance to a subject is detected for each of the pixels. 2. The apparatus according to claim 1, wherein the number-of-repetitions adjusting means adjusts the number of repetitions based on a result of pre-ranging that roughly measures a distance to the subject in advance.
3. The three-dimensional image detection device according to item 1. 3. The pre-ranging is performed by driving the first and second signal charge integration means with the same number of repetitions in the first and second signal charge integration means. The three-dimensional image detection device according to claim 2, wherein 4. The apparatus according to claim 1, wherein the number-of-repetitions adjusting means detects the first and second integrated charges in one or more sample pixels which are detected in the pre-ranging and represent pixels corresponding to a measurement area of the subject. 4. The three-dimensional image detecting apparatus according to claim 3, wherein the number of repetitions is adjusted by performing the adjustment. 5. The method according to claim 1, wherein the repetition number adjusting means adjusts the number of repetitions such that a ratio of the first integrated charge and the second integrated charge in one sample pixel is 1: 1. The three-dimensional image detection device according to claim 4, wherein: 6. The repetition number adjusting means makes the number of repetitions in the first charge integration means constant, and
The three-dimensional image detecting apparatus according to claim 1, wherein the adjustment is performed by reducing the number of repetitions in the charge integration means. 7. An aperture stop for adjusting an image plane illuminance of an optical system, wherein when performing the pre-ranging, the repetition number adjusting means and the first and second signal charge integrating means are provided. 3. The three-dimensional image detection apparatus according to claim 2, wherein the measurement is performed with the aperture stop being narrowed as compared with the main distance measurement in which the distance to the subject is measured by driving.