JP2000341720A - Three-dimensional image input device and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To quickly and easily execute three-dimensional(3D) image processing and to record 3D image data in a recording medium in a format capable of reducing storage capacity. SOLUTION: Picture data Ri, j, Gi, j, Bi, j (i, j=1, 2,...) concerned with the R, G and B of a target to be measured are obtained by a CCD. The target is irradiated with pulse-like range finding light and distance data Di, j up to the target corresponding to CCD pixels are obtained by utilizing the light receiving quantity of reflected light of the range finding light. The picture data Ri, j, GLi, j, Bi, j and the distance data Di, j are recorded in respectively different files on the recording medium correspondingly to respective pixels. File relational information indicating that respective files are a pair of related files is recorded in respective file management areas (header areas).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional image input device for detecting information relating to a three-dimensional shape and a two-dimensional image of an object to be measured.

[0002]

2. Description of the Related Art Conventionally, three-dimensional measurement by a three-dimensional image input device is classified into an active method in which light, radio waves, or sound is irradiated to an object to be measured and a passive method in which light or the like is not irradiated. As the active method, a light transit time measurement method, a phase difference measurement method using a modulated light wave, a triangulation method, a moire method, an interferometry method, and the like are known. Etc. are known.

[0003]

As a method of simultaneously recording the distance information to the object to be measured and the two-dimensional image information of the object to be measured obtained by these three-dimensional measurement methods, for example, an image format standard for a digital still camera is used. There is Exif (JEIDA standard). In this method, a distance to a subject is detected using an automatic focusing device or the like, and the obtained distance data is recorded in a file management area and two-dimensional image data (hereinafter, referred to as image data) is recorded in a data area. However, the distance data recorded in this format is only one distance data to the measured object,
No three-dimensional image processing could be performed using this.

In the apparatus disclosed in Japanese Patent Application Laid-Open No. H8-317425, a pair of left and right eye image data in stereoscopic viewing is mixed and recorded on a single recording medium. The related information of the pair of image data is recorded in a file management area. However, in this method, two pieces of image data must be recorded in order to obtain one piece of three-dimensional image information, and a recording capacity almost twice that of normal image data is required. Moreover, it is difficult to obtain distance data corresponding to all pixels because it is necessary to calculate a relationship such as which pixel of the image for the left eye corresponds to which pixel of the image for the right eye in calculating the distance information. It is inconvenient in three-dimensional image processing or the like that directly uses distance data.

The present invention provides a three-dimensional image input apparatus for recording high-precision three-dimensional image information with a small storage capacity and recording image data and distance data on a recording medium in a format that can easily execute three-dimensional image processing. The purpose is to get.

[0006]

A three-dimensional image input apparatus according to the present invention comprises: an image input means for obtaining image data of a subject for each pixel; and a distance for detecting a distance to a subject corresponding to the pixel and obtaining the distance data. Data detecting means, and image data and distance data corresponding to the pixels are recorded in the image data file and the distance data file, respectively, and file-related information indicating that the image data file and the distance data file are associated with each other is recorded. And a three-dimensional image recording means.

[0007] The image data, the distance data and the file related information are preferably recorded on the same recording medium. Further, the image data file preferably has a first file management area for recording information shared in the file and a first data area for recording image data, and the distance data file is also preferable. , A second file management area for recording information shared in the file and a second data area for recording distance data. In this case, preferably, the file related information is recorded in the first or second file management area.

Preferably, distance data file related information for identifying the corresponding distance data file is recorded in the first file management area, and the corresponding image data file is identified in the second file management area. Is recorded. For example, the mutual relationship is indicated by recording common file identification data in the distance data file related information and the image data file related information. In this case, the file identification data is a part of the file name including the extension, for example.

The distance data is, for example, absolute distance data based on a predetermined distance unit, and the distance unit is recorded in the file management area. Further, the distance data is, for example, relative distance data in a predetermined distance range, and distance range information on the distance range is recorded in the file management area, and the distance range information is the closest distance and the farthest distance or the closest distance and the distance of the distance range. Expressed as a range width.

The distance data is represented by, for example, difference data from the reference distance data. In this case, preferably, the reference distance data is distance data adjacent in a predetermined direction, and when there is no adjacent distance data to be a reference, the distance data is not difference data.

In the recording medium of the present invention, image data corresponding to each pixel of the subject image and distance data indicating the distance to the subject corresponding to each pixel are recorded as an image data file and a distance data file, respectively. And file related information indicating that the image data file and the distance data file are related to each other.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a perspective view of a camera provided with a three-dimensional image input device according to a first embodiment of the present invention.

On the front of the camera body 10, a finder window 12 is provided at the upper left of the taking lens 11, and a flash 13 is provided at the upper right. A light emitting device (light source) 14 for irradiating a laser beam, which is a distance measuring light, is provided on the upper surface of the camera body 10 and directly above the taking lens 11. On the left side of the light emitting device 14, a release switch 15 and a liquid crystal display panel 16 are provided, and on the right side, a mode switching dial 17 and a V / D mode switching switch 18 are provided. A card insertion slot 19 for inserting a recording medium such as an IC memory card is formed on a side surface of the camera body 10, and a video output terminal 20 and an interface connector 21 are provided.

FIG. 2 is a block diagram showing a circuit configuration of the camera shown in FIG. An aperture 25 is provided in the taking lens 11. The opening of the aperture 25 is determined by the iris drive circuit 2.
Adjusted by 6. The focus adjustment operation and the zooming operation of the taking lens 11 are controlled by a lens drive circuit 27.

An image pickup device (C
CD) 28 is provided. A subject image is formed on the CCD 28 by the photographing lens 11, and charges corresponding to the subject image are generated. Operations such as a charge accumulation operation and a charge read operation in the CCD 28 are controlled by the CCD drive circuit 30. The charge signal, that is, the image signal, read from the CCD 28 is amplified by the amplifier 31 and the A / A
The analog signal is converted into a digital signal in the D converter 32. The digital image signal is output from the imaging signal processing circuit 3
In the image memory 3, processing such as gamma correction is performed.
4 is temporarily stored. The iris drive circuit 26, the lens drive circuit 27, the CCD drive circuit 30, and the imaging signal processing circuit 33 are controlled by a system control circuit 35.

The image signal is read from the image memory 34 and supplied to the LCD drive circuit 36. The LCD drive circuit 36 operates in accordance with the image signal, and thereby the image display L
An image corresponding to the image signal is displayed on the CD panel 37.

The image signal read from the image memory 34 is sent to a TV signal encoder 38, and can be transmitted via the video output terminal 20 to a monitor device 39 provided outside the camera body 10. The system control circuit 35 is connected to the interface circuit 40, and the interface circuit 40 is connected to the interface connector 21.
It is connected to the. Therefore, the image signal read from the image memory 34 can be transmitted to the computer 41 connected to the interface connector 21. The system control circuit 35 includes a recording medium control circuit 42
Is connected to the image recording device 43 via the. Therefore, the image signal read from the image memory 34 can be recorded on a recording medium M such as an IC memory card mounted on the image recording device 43. The image signal once recorded on the recording medium M is read out from the recording medium M as necessary, and is sent to the LCD panel 37 via the system control circuit 35.
Can be displayed.

A light emitting element control circuit 44 is connected to the system control circuit 35. The light emitting device 14 is provided with a light emitting element 14a and an illumination lens 14b, and the light emitting operation of the light emitting element 14a is controlled by a light emitting element control circuit 44. The light emitting element 14a irradiates a laser beam that is distance measuring light, and the laser beam is radiated to the entire measurement object via the illumination lens 14b. Light reflected from the object to be measured enters the photographing lens 11. By detecting this light by the CCD 28, a three-dimensional image of the measured object is measured as described later. In this measurement, control of the transfer operation timing and the like in the CCD 28 is performed by the system control circuit 35 and the CCD drive circuit 30.

The system control circuit 35 is connected to a switch group 45 including a release switch 15, a mode switching dial 17, and a V / D mode switching switch 18, and a liquid crystal display panel (display element) 16.

The principle of distance measurement in the present embodiment will be described with reference to FIGS. In FIG. 4, the horizontal axis is time t.

The distance measuring light output from the distance measuring device B is reflected by the measured object S and received by a CCD (not shown). The distance measuring light is a pulsed light having a predetermined pulse width H. Therefore, the reflected light from the measured object S is also a pulsed light having the same pulse width H. The rise of the reflected light pulse is delayed by a time δ · t (δ is a delay coefficient) from the rise of the distance measuring light pulse.
Since the ranging light and the reflected light have traveled twice the distance r between the distance measuring device B and the measured object S, the distance r is r = δ · t · C / 2 (1) ). Where C is the speed of light.

For example, a state in which reflected light can be detected from the rise of the pulse of distance measuring light is determined, and the state is changed to an undetectable state before the reflected light pulse falls, that is, a reflected light detection period T is provided. And the received light amount A during the reflected light detection period T is a function of the distance r. That is, the light receiving amount A decreases as the distance r increases (the time δ · t increases).

In the present embodiment, utilizing the above-described principle,
The distance from the camera body 10 to each point on the surface of the object S to be measured is detected by detecting the amount of received light A at each of a plurality of photodiodes (photoelectric conversion elements) provided on the CCD 28 and arranged two-dimensionally. The data of the three-dimensional image relating to the surface shape of the measured object S is detected and input at a time.

FIG. 5 is a diagram showing the arrangement of the photodiode 51 and the vertical transfer unit 52 provided on the CCD 28.
FIG. 6 is a cross-sectional view of the CCD 28 cut along a plane perpendicular to the substrate 53. This CCD 28 is a conventionally known interline type CCD, and uses a VOD for sweeping out unnecessary charges.
(Vertical overflow drain) method.

The photodiode 51 and the vertical transfer section (signal charge holding section) 52 are formed along the surface of the n-type substrate 53. The photodiodes 51 are two-dimensionally arranged in a lattice, and the vertical transfer units 52 are provided adjacent to the photodiodes 51 arranged in a line in a predetermined direction (vertical direction in FIG. 5). The vertical transfer unit 52 includes four vertical transfer electrodes 52 a and 52 for one photodiode 51.
b, 52c and 52d. Therefore, in the vertical transfer section 52, four potential wells can be formed, and signal charges can be output from the CCD 28 by controlling the depths of these wells as conventionally known. The number of vertical transfer electrodes can be freely changed according to the purpose.

A photodiode 51 is formed in a p-type well formed on the surface of the substrate 53, and the p-type well is completely depleted by a reverse bias voltage applied between the p-type well and the n-type substrate 53. You. In this state, a charge corresponding to the amount of incident light (reflected light from the measured object) is accumulated in the photodiode 51. When the substrate voltage Vsub is increased to a predetermined value or more, the electric charge accumulated in the photodiode 51 is discharged to the substrate 53 side. On the other hand, when a charge transfer signal (voltage signal) is applied to the transfer gate unit 54, the charges accumulated in the photodiode 51 are transferred to the vertical transfer unit 52. That is, after the charges are discharged to the substrate 53 side by the charge discharge signal, the photodiode 5
The signal charges accumulated in 1 are transferred to the vertical transfer unit 52 by a charge transfer signal. By repeating such operations, signal charges are integrated in the vertical transfer unit 52, and an electronic shutter operation is realized.

FIG. 7 is a timing chart of a distance information detecting operation for detecting data relating to a distance to each point on the surface of the measured object. The operation of detecting distance information will be described with reference to FIGS.

A charge sweeping signal (pulse signal) S2 is output in synchronization with the output of the vertical synchronization signal S1, whereby unnecessary charges accumulated in the photodiode 51 are removed from the substrate 53.
In the direction of The light emitting device 14 is activated almost simultaneously with the end of the output of the charge sweeping signal S2, and the pulsed ranging light S3 having a constant pulse width is output. Distance measuring light S3
Is reflected by the object to be measured and enters the CCD.
That is, the reflected light S from the object to be measured by the CCD 28
4 is received. When a certain time has elapsed from the output of the distance measuring light S3, a charge transfer signal (pulse signal) S5 is output, and the charges accumulated in the photodiode 51 are transferred to the vertical transfer unit 52. Note that the charge transfer signal S
The output of No. 5 is performed before the end of the output of the ranging light.

As described above, during the period T _U1 from the end of the output of the charge sweep signal S2 to the start of the output of the charge transfer signal S5, signal charges corresponding to the distance to the object to be measured are accumulated in the photodiode 51. You. That is, the distance measuring light S3
When the period T _S during which is output and the charge accumulation period T _U1 start at substantially the same time, S4 is delayed by δ · t as compared with S3, so that the charge accumulation period T _U1 ends earlier and one of the reflected light S4 Only the portion is detected by the CCD 28, and the signal charge S6 generated by the detected light corresponds to the distance to the measured object. In other words, the reflected light S from the measured object
4, among the photodiodes 51 during the charge accumulation period T _U1 .
Is accumulated in the photodiode 51 corresponding to the light that has arrived. This signal charge S6 is transferred to the vertical transfer unit 52 by the charge transfer signal S5. Note that the output period T _S of the distance measurement light S3 may start earlier than the charge accumulation period T _U1 .

After a predetermined time has elapsed from the output of the charge transfer signal S5, the charge sweep signal S2 is output again, and after the transfer of the signal charge to the vertical transfer section 52, the photodiode 51
Unnecessary charges accumulated in the substrate 53 are swept toward the substrate 53.
That is, accumulation of signal charges in the photodiode 51 is newly started. Then, as described above, when the charge accumulation period T _U1 has elapsed, the signal charge is transferred to the vertical transfer unit 52.

The vertical transfer section 52 of such signal charges S6
Is repeatedly executed until the next vertical synchronization signal S1 is output. As a result, in the vertical transfer unit 52, the signal charge S6 is integrated, and in the signal charge S6 integrated during the period of one field (the period sandwiched between the two vertical synchronization signals S1), the measured object is stationary during that period. If it can be considered, it corresponds to the distance information to the measured object.

The operation of detecting the signal charge S6 described above is 1
One of the photodiodes 51 performs such a detection operation in all the photodiodes 51. As a result of the detection operation in one field period, the distance information detected by the photodiode 51 is held in each part of the vertical transfer unit 52 adjacent to each photodiode 51. This distance information is output from the CCD 28 by a vertical transfer operation in the vertical transfer unit 52 and a horizontal transfer operation in a horizontal transfer unit (not shown).

However, the reflected light detected by the CCD 28 is affected by the reflectance of the surface of the object to be measured.
Therefore, the distance information obtained via the reflected light includes an error due to the reflectance. In addition, CCD2
The reflected light detected by 8 includes components such as external light in addition to the reflected light from the object to be measured, and there is an error due to this. Next, a method for correcting these errors will be described with reference to FIGS. 8 to 10 and FIGS. 11, 12, and 13. FIG.

First, each mode is set by manually switching each mode switch provided on the camera. That is, by operating the V / D mode switch 18, the video (V) mode and the distance measurement (D)
One of the modes is set. By manually switching a three-dimensional image input mode (3D mode) switch 61 (see FIGS. 13A and 13B) provided on the back of the camera body 10, the three-dimensional image input mode is turned on / off. Is set. In the three-dimensional image input mode, three-dimensional distance information and two-dimensional image information that are related to each other are
An operation mode in which a pair of files is recorded on a recording medium. When the three-dimensional image input mode is not set and the V / D mode switch 18 is set to the V mode, only two-dimensional image information is detected and recorded on the recording medium M as single image data. Is done.
Also, the three-dimensional image input mode is not set, and V / D
When the mode switch 18 is set to the D mode, only three-dimensional distance information is detected and recorded on the recording medium M as single distance data.

When it is confirmed in step 101 that the release switch 15 is fully depressed, step 1 is executed.
02 is performed, and the set mode is determined. When the mode is the V mode or the three-dimensional image input mode, the process proceeds to step 125, and the distance measuring light control is set to the off state. Subsequently, at step 126, the video control of the CCD 28 is set to the ON state.

In step 127, it is determined whether or not one image has been captured. When the imaging of one image is completed, the captured image data is recorded on the recording medium M as an image data file in step 128. Step 1
At 29, it is determined whether or not the mode is the three-dimensional image input mode. When the mode is not the three-dimensional image input mode, this detection operation ends. On the other hand, in the case of the three-dimensional image input mode, the process proceeds to step 103.

In step 103, the vertical synchronizing signal S1
Is output and the ranging light control is started. That is, the light emitting device 14 is driven, and the pulse-shaped ranging light S3 is output intermittently. Then step 104 is performed,
The detection control by the CCD 28 is started. That is, FIG.
Is started, the charge sweep signal S2 and the charge transfer signal S5 are output alternately, and the signal charge S6 of the distance information is integrated in the vertical transfer unit 52.

In step 105, it is determined whether one field period has elapsed from the start of the distance information detection operation, that is, whether a new vertical synchronizing signal S1 has been output. When one field period ends, the routine proceeds to step 106, where the signal charge S6 of the distance information is output from the CCD. This signal charge S6 is temporarily stored in the image memory 34 in step 107. In step 108, the ranging light control is switched to the off state, and the light emitting operation of the light emitting device 14 stops.

In steps 109 to 112, an operation of detecting distance correction information is performed. First, in step 109,
As shown in FIG. 8, the vertical synchronizing signal S11 is output, and the detection control by the CCD 28 is started. That is, the charge sweeping signal S12 and the charge transfer signal S15 are alternately output in a state where the light source is turned off without performing the light emitting operation of the light emitting device 14. The charge accumulation time T _U1 is the same as the distance information detecting operation shown in FIG. 7, but since the object to be measured is not irradiated with the distance measurement light (reference S13), there is no reflected light (reference S14). Therefore, although no signal charge of the distance information is generated, since a disturbance component such as external light is incident on the CCD 28, the signal charge S1 corresponding to the disturbance component is generated.
6 occurs. This signal charge S16 has a charge accumulation time T _U1 for correcting the influence of a disturbance component on distance information.
Corresponding to the distance correction information.

In step 110, it is determined whether one field period has elapsed from the start of the operation of detecting the distance correction information, that is, whether a new vertical synchronizing signal S11 has been output. Step 1 when one field period ends
At 11, the signal charge S16 of the distance correction information is
28. The signal charge S16 of the distance correction information is temporarily stored in the image memory 34 in step 112.

In steps 113 to 117, an operation of detecting reflectance information is performed. In step 113, as shown in FIG. 9, the vertical synchronization signal S21 is output, and the distance measurement light control is started, and the pulse-shaped distance measurement light S23 is output intermittently. In step 114, the detection control by the CCD 28 is started, and the charge discharge signal S22 and the charge transfer signal S25 are output alternately. The operation of detecting the reflectance information is controlled such that all of the reflected light S24 is received within the charge accumulation period T _U2 from when the charge sweep signal S22 is output to when the charge transfer signal S25 is output. . That is, the pulse width T _S of the signal charge S26 accumulated in each photodiode 51 of the CCD 28 is equal to the distance measurement light S2.
3, which is the same as the pulse width T _S.

Therefore, the signal charge S26 has no relation to the distance to the object to be measured, and corresponds to reflectance information depending on the reflectance of the surface of the object to be measured.

In step 115, it is determined whether one field period has elapsed from the start of the reflectance information detection operation, that is, whether a new vertical synchronizing signal S21 has been output. When one field period ends, step 116
Then, the signal charge S26 of the reflectance information is output from the CCD 28. This signal charge S26 is temporarily stored in the image memory 34 in step 117. Step 1
At 18, the distance measuring light control is switched to the off state, and the light emitting operation of the light emitting device 14 is stopped.

In steps 119 to 122, an operation of detecting reflectance correction information is performed. In step 119, as shown in FIG. 10, the vertical synchronizing signal S31 is output, and the detection control by the CCD 28 is started. That is, the charge sweeping signal S32 and the charge transfer signal S35 are alternately output in a state where the light source is turned off without performing the light emitting operation of the light emitting device 14. The charge accumulation time T _U2 is the same as the reflectance information detection operation shown in FIG. 9, but since the object to be measured is not irradiated with the ranging light (reference S33), there is no reflected light (reference S34). Accordingly, no signal charge of the reflectance information is generated, but a signal charge S36 corresponding to a disturbance component such as external light is generated in the CCD 28. The signal charge S36 corresponds to reflectance correction information for correcting the influence of the disturbance component on the reflectance information with respect to the charge accumulation time T _U2 .

In step 120, it is determined whether or not one field period has elapsed since the start of the operation of detecting the reflectance correction information.
That is, it is determined whether a new vertical synchronization signal S31 has been output. When one field period is completed, in step 121, the signal charge S36 of the reflectance correction information becomes C
The data is output from the CD 28 and temporarily stored in the image memory 34 in step 122.

In step 123, steps 103-1
Using the distance information, the distance correction information, the reflectance information, and the reflectance correction information obtained in step S22, the arithmetic processing of the distance data is performed. In step S124, the distance data is recorded on the recording medium M as a distance data file. The detection operation ends. The data recording in step 124 and step 128 will be described later in detail.

The contents of the arithmetic processing executed in step 123 will be described with reference to FIGS. Reflectivity R
Is illuminated, and the measured object has a luminance I of 2
It is assumed that the image is formed on the CCD by being regarded as the next light source. At this time, the output Sn obtained by integrating the charge generated in the photodiode during the charge accumulation time t is represented by: Sn = kR.I.t (2) Here, k is a proportionality constant, which varies depending on the F number, magnification, and the like of the taking lens.

[0048] If the measurement subject is illuminated with light from a light source such as a laser, the luminance I becomes were synthesized with the luminance I _B by the luminance I _S and the background light due to the light source, I = I _S + I _B ... (3) can be expressed.

[0049] The pulsed charge accumulation time as shown in FIG. 7 T _U1, the pulse width T _S of the distance measuring light S3, a pulse width of the signal charge S6 in the distance information and T _D, in one field period When the charge storage time is to be repeated N times, the output S _{10 resulting, S 10 = Σ (k ·} R (I S · T D + I B · T U1)) = k · N · R (I S · T _D + I _B · T _U1 ) (4) The pulse width T _D can be expressed as _{T D = T U1 -δ · t} = T U1 -2r / C ··· (5).

As shown in FIG. 9, the pulse-shaped charge accumulation time T _U2 is controlled to be sufficiently longer than the period (pulse width) T _{S of the} distance measuring light S23 and to include the entire unit light receiving time of the reflected light. the output S ₂₀ obtained when _{the, S 20 = Σ (k ·} R (I S · T S + I B · T U2)) = k · N · R (I S · T S + I B · T U2) ·・ ・ (6)

When light emission is stopped as shown in FIG.
Output S ₁₁ obtained when performing a pulsed charge accumulation at the same time width _{is, S 11 = Σ (k ·} R · I B · T U1) = k · N · R · I B · T U1 (7) Similarly, the output S ₂₁ obtained when performing charge accumulation as shown in FIG. _{10, S 21 = Σ (k ·} R · I B · T U2) = k · N · R · I B · T _U2 (8)

From the equations (4), (6), (7), and (8), S _D = (S ₁₀ −S ₁₁ ) / (S ₂₀ −S ₂₁ ) = T _D / T _S (9) ) Is obtained.

As described above, the distance measuring light S3 and the reflected light S4 respectively include disturbance components such as external light (luminance I _B due to background light).
It is included. T _D / T _{S in} equation (9) is the distance measuring light S3
Is obtained by normalizing the amount of reflected light S4 from the object to be measured when the object is irradiated with the amount of ranging light S3,
This value is obtained by removing the disturbance component (corresponding to the signal charge S16 in FIG. 8) from the light amount of the distance measuring light S3 (corresponding to the signal charge S6 in FIG. 7), and the light amount of the reflected light S4 (corresponding to the signal charge S26 in FIG. 9).
) From the value obtained by removing the disturbance component (corresponding to the signal charge S36 in FIG. 10) from the corresponding value.

Each output value S ₁₀ , S ₁₁ , S ₂₀ , S in the equation (9)
₂₁ is stored in steps 107, 112, 117, and 122 as distance information, distance correction information, reflectance information, and reflectance correction information. Therefore, T _D / T _S is obtained based on these pieces of information. Since the pulse width T _U1 is known, the distance r is obtained from the equation (5) and T _D / T _S.

As described above, the distance information from the camera body to each point on the surface of the object to be measured is corrected based on the expressions (5) and (9), and the detection accuracy of the distance information is improved.

In the present embodiment, the influence of external light or the like has been removed from the distance information of the measured object. integrated value of the impact signal charge related (i.e. S _11, S ₂₁₎
May be omitted. As a result, correction is performed only on the reflectance of the surface of the measured object.

In this embodiment, step 10
In 5, 110, 115, and 120, signal charges are accumulated for one field period. Alternatively, charge accumulation may be performed for a plurality of field periods.

Next, with reference to FIGS. 13, 14 and 15, a method of recording each data in steps 124 and 128 will be described.

When the 3D mode switch 61 is not set to the three-dimensional image input mode (FIG. 13A)
The distance data recorded in step 124 and the image data recorded in step 128 are recorded as independent distance data files and image data files on the recording medium M, respectively.
Will be recorded. That is, the image data obtained in the V mode and the distance data obtained in the D mode are treated as irrelevant data.

On the other hand, when the 3D mode switch 61 is set to the three-dimensional image input mode (FIG. 13B),
Distance data recorded in step 124 and step 12
The image data recorded in 8 is treated as a pair of data. That is, file-related information for identifying the corresponding image data file from other files is recorded in the distance data file, and file-related information for identifying the corresponding distance data file is also recorded in the image data file. . Thus, even if image data and distance data are recorded in separate data files, the other data file can be identified from one data file, and image data and distance data corresponding to each pixel required for three-dimensional image processing are obtained. be able to.

FIG. 14 shows an example of an image data file and a distance data file created in the three-dimensional image input mode. When the three-dimensional image input mode is set, in step 128, for example, an image data file "IMAG0001.TIF" (TIFF file)
The image data is recorded in the. At this time, in step 124, for example, the distance data file "DIST0001.D
ST is recorded in the image data file “IMAG0001. In the TIF, the end of the file name excluding the extension of the paired distance data file is “0001”, which is common with the image data file, and the extension is “DST” as file-related information. Similarly, the distance data file “DIST0” is recorded.
001. In the “DST”, the end of the file name excluding the extension of the paired image data file is “0001” and the extension is “TIF” is recorded as file-related information.

The names of the files created when recording the distance data and image data in steps 124 and 128 are automatically obtained by checking the file names of the distance data files and image data files already existing on the recording medium. Attached. That is, in the three-dimensional image input mode,
Examine the file names of both the existing distance data file and image data file, select the file with the largest number at the end of the file name from these two types of files, and add 1 to the number to indicate the common number for the file. The name of the distance data file and the name of the image data file are given so as to be the last. Also, in step 124 when the three-dimensional image input mode is not set, one is added to the file having the largest number at the end of the existing distance data files to create a new file name. A new file name is obtained by adding 1 to the largest number of file names at the end of the existing image data files.

FIG. 15 schematically shows the structure of the image data file and the distance data file. These two
One file is large and the file management area (header area)
_Ah and a data area _Ad . The header area A _h of the image data file, for example, the image data is red (R), green (G), and that the luminance information about the blue (B), the order in which it is recorded, the image data including the number of bits In addition to the information related to the shooting conditions such as shooting date and time, the photographer, and shooting conditions, the above-described file-related information and the like are recorded. Distance data file in the header area A _h distance unit in the distance data in the format of the data, information and the above-described file-related information relating to the distance data such as the number of bytes is recorded.

[0064] The data area A _d of the image data file, R for each pixel to be recorded in step 128 (FIG. 12)
GB image data is recorded. FIG. 16 (a)
The arrangement of the memory of the image data corresponding to each pixel that has been written on the data area A _d is a representation schematically. In the figure, since the data arrangement on the memory is shown corresponding to the physical arrangement of the pixels, it is expressed two-dimensionally. That is, M ₀ , M ₁ , and M ₂ have RGB image data (pixel values) R _1,1 , R ₁ ,
G _1,1 and B _1,1 are recorded. Similarly, RGB image data R _1,2 , G _1,2 , B _1,2 in the second pixel from the left of the first line are recorded in M _{3 to} M ₅ . M
_{In 3n to} M _{3n + 2} , RGB image data R _2,1 , G _2,1 and B _2,1 of the first pixel on the second line are recorded. Here, n is the number of pixels in the horizontal direction.

[0065] The data area A _d of the distance data file distance data for each pixel to be recorded is recorded at step 124. FIG. 16 (b) is an arrangement of the data area A of the _d distance data recorded in the memory a representation schematically. As in the case of the image data in FIG. 16A, the data arrangement on the memory is represented in a two-dimensional manner in correspondence with the physical arrangement of the pixels. That is, D _{i, j}
(I, j = 1, 2,...) Is distance data corresponding to the j-th pixel from the left of the i-th line.

As described above, according to the first embodiment, highly accurate three-dimensional image information can be recorded without using a large storage capacity. Further, since the distance data corresponding to each pixel is directly recorded, the distance to each part of the measured object can be obtained quickly and easily, and three-dimensional image processing using the distance data can be easily performed.

In this embodiment, the file-related information is recorded in each of the image data file and the distance data file. However, the file-related information may be recorded in only one of them (for example, the image data file). Further, in the present embodiment, a part of the file name including the extension is used as the file-related information for identifying the file, but may be a part of the path name obtained by adding the directory name to the file name.

Next, a second embodiment of the present invention will be described with reference to FIG. In the second embodiment, a mechanical and electrical configuration and a distance information detecting operation are the same as those in the first embodiment. The only difference between the second embodiment and the first embodiment is the data format of the distance data file.

In the first embodiment, RGB image data and distance data are recorded for each pixel. However, in the second embodiment, several pixels (for example, four pixels) are grouped into one group, and one group includes one pixel. Record two distance data. The figure schematically shows the arrangement of the memory in a case where a total of four pixels of two horizontal pixels and two vertical pixels are grouped into one group. FIG. 17A relates to RGB image data.
This is exactly the same as the embodiment. FIG. 17A relates to the distance data, but only the distance data corresponding to the lower right pixel of the above four pixels is recorded. Here, m is an integer part of (n / 2) obtained by dividing the number n of pixels in the horizontal direction by two.

As described above, according to the second embodiment, when the distance data is not as detailed as the image data, the distance data is recorded every several pixels to reduce the capacity of the distance data file. It can be reduced compared to the form.

In this embodiment, the distance data corresponding to the lower right pixel among the four pixels is recorded as the distance information of the group. However, this may be distance data corresponding to other pixels in the group, or may be an average value of the distance data in the group.

Next, a third embodiment will be described with reference to FIG. The third embodiment differs from the first embodiment only in the data format of the distance data, similarly to the second embodiment, and the other embodiments are the same as the first embodiment.

The horizontal axis in FIG. 18 is the distance to the measured object, and the vertical axis is the distance data value corresponding to the relative distance in a predetermined distance range. For example, when the distance to the measured object corresponding to a certain pixel is L, the distance L is represented by the sum of the nearest distance L1 of the distance range 2 and the relative distance Lr from L1, and the data area of the three-dimensional image information file Has distance data D (D _min ≦ D ≦) corresponding to the relative distance Lr.
D _max ) is recorded. That is, when the distance data D is, for example, 8-bit data, D _min = 0 and D _max = 255.
And Lr can be expressed by Lr = (L2−L1) / 256 × D.

In the header area of the distance data file, in addition to the information exemplified in the first embodiment, information on the nearest distance and the furthest distance in each distance range, and information on which distance range each pixel corresponds to, is displayed. The indicated information is recorded. At this time, since the adjacent pixels usually correspond to the same distance range, it is not necessary to record the correspondence between each pixel and the distance range for each pixel. For example, when pixels continuous in the horizontal direction correspond to the same distance range over a certain section, the address at which this section starts or ends and the distance range in that section may be recorded. In addition, as information on the distance range, in addition to the closest distance and the farthest distance of the distance range,
The closest distance and its width (the difference between the farthest distance and the closest distance) may be used.

As described above, according to the third embodiment, the number of bits assigned to one distance data can be reduced by expressing the distance to the measured object by the relative distance in each distance range. The size of the distance data file can be reduced.

Next, a fourth embodiment will be described with reference to FIG. In the fourth embodiment, the distance data is represented as a difference from the reference distance data, and the rest is the same as the first embodiment.

The figure shows the distance data corresponding to the physical arrangement of the pixels. The numerical values in FIG. 19A represent distance data when the distance of each pixel to the object to be measured is expressed as an absolute or relative distance. On the other hand, FIG.
The numerical value of represents the distance data in FIG. 19A in the form of a difference with reference to distance data adjacent in a predetermined direction on the memory. However, when there is no distance data to be a reference, the distance data corresponding to the absolute or relative distance is used as it is. For example, FIG.
In the upper left distance data D ₀ (123), the distance data corresponding to the absolute or relative distance is recorded as it is because the distance data to be used as a reference is not recorded in the memory ahead of it. The difference between the distance data d _{i (i} =
, N) is the difference (D _i ) between the distance data D _i and D _i−1.
−D _i−1 ). Here, N is the total number of distance data.

In the above example, the difference between adjacent data is used. However, difference distance data may be obtained using one distance data as reference distance data. For example the D ₀ as the reference distance data, the differential distance data d _i D _i and D ₀
(D _i -D ₀ ).

As described above, according to the fourth embodiment, the third
Since the number of bits required for each distance data can be reduced as in the embodiment, the capacity of the distance data file can be reduced.

In the present embodiment, since only one area sensor was provided, the image information and the distance information had to be obtained separately. However, the incident light from the imaging lens 11 was obtained by using a half mirror or the like. The image information and the distance information may be input simultaneously by receiving the divided light and receiving the divided light by two area sensors.

In this embodiment, RG is used as image data.
Although B is used, YCbCr may be used, and black and white image data may be used instead of color image data. Alternatively, digital data may be converted into an analog signal and recorded on an analog recording medium.

[0082]

As described above, according to the present invention, high-precision three-dimensional image information can be recorded with a small storage capacity, and image data and distance data can be easily recorded on a recording medium in a format that can easily execute three-dimensional image processing. A three-dimensional image input device for recording can be obtained.

[Brief description of the drawings]

FIG. 1 is a perspective view of a camera provided with a three-dimensional image input device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a circuit configuration of the camera shown in FIG.

FIG. 3 is a diagram for explaining the principle of distance measurement using ranging light.

FIG. 4 Distance measuring light, reflected light, gate pulse, and CCD
FIG. 4 is a diagram showing a distribution of the amount of light received by the light source.

FIG. 5 is a diagram showing an arrangement of a photodiode and a vertical transfer unit provided in a CCD.

FIG. 6 is a cross-sectional view of the CCD cut along a plane perpendicular to the substrate.

FIG. 7 is a timing chart of a distance information detecting operation for detecting data relating to a distance to an object to be measured.

FIG. 8 is a timing chart of a detection operation of distance correction information.

FIG. 9 is a timing chart of an operation of detecting reflectance information.

FIG. 10 is a timing chart of an operation of detecting reflectance correction information.

FIG. 11 is a first half of a flowchart of a distance information detecting operation.

FIG. 12 is the latter half of the flowchart of the distance information detection operation.

FIG. 13 is a diagram of a three-dimensional image input mode (3D mode) switch provided on the back of the camera.

FIG. 14 is a diagram showing a conceptual structure of an image data file and a distance data file.

FIG. 15 is a diagram illustrating file-related information of an image data file and a distance data file.

FIG. 16 is a diagram showing a conceptual structure of image data and distance data recorded in a file in the first embodiment.

FIG. 17 is a diagram illustrating a conceptual structure of an image data file and a distance data file according to the second embodiment.

FIG. 18 is a diagram illustrating a relationship between a distance to an object to be measured and relative distance data in a distance range according to the third embodiment.

FIG. 19 is a diagram illustrating difference distance data according to the fourth embodiment.

[Explanation of symbols]

_Ad data area _Ah file management area (header area) M Recording medium

Continued on the front page F-term (reference) 2F065 AA06 BB05 DD02 DD07 FF12 FF32 FF41 GG04 GG12 HH04 HH18 JJ01 JJ03 JJ18 JJ26 KK02 NN12 NN17 PP22 QQ03 QQ24 UU05 4M118 AA10 AB01 AB03 BA13 CA03 GA01 BA01CB01 CH11 DA06 DB03 DB06 DC02 5C052 AA17 AB03 AB04 CC06 DD02 DD04 DD08 EE02 EE03 EE08 5C061 AA29 AB03 AB21

Claims (14)

[Claims] An image input unit that obtains image data of a subject for each pixel; a distance data detecting unit that detects a distance to the subject corresponding to the pixel and obtains the distance data; Three-dimensional image recording means for recording image data and the distance data in an image data file and a distance data file, respectively, and recording file-related information indicating that the image data file and the distance data file are related to each other; A three-dimensional image input device, comprising: 2. The three-dimensional image input device according to claim 1, wherein the image data, the distance data, and the file-related information are recorded on a same recording medium. 3. The distance data file, wherein the image data file has a first file management area for recording information shared in the file and a first data area for recording image data. 3. The three-dimensional image input device according to claim 1, further comprising a second file management area for recording information shared in the file and a second data area for recording distance data. apparatus. 4. The three-dimensional image input device according to claim 3, wherein the file-related information is recorded in the first or second file management area. 5. The distance data file related information for identifying the corresponding distance data file is recorded in the first file management area, and the corresponding image data file is stored in the second file management area. The three-dimensional image input device according to claim 4, wherein image data file related information for identification is recorded. 6. The three-dimensional image according to claim 5, wherein mutual relationship is indicated by recording common file identification data in the distance data file related information and the image data file related information. Input device. 7. The three-dimensional image input device according to claim 6, wherein the file identification data is a part of a file name including an extension. 8. The three-dimensional image input device according to claim 1, wherein the distance data is absolute distance data based on a predetermined distance unit. 9. The three-dimensional image input device according to claim 3, wherein the distance unit is recorded in the file management area. 10. The three-dimensional data according to claim 3, wherein the distance data is relative distance data in a predetermined distance range, and distance range information relating to the distance range is recorded in the file management area. Image input device. 11. The three-dimensional image input device according to claim 10, wherein the distance range information is represented by the closest distance and the farthest distance of the distance range or the width of the closest distance and the distance range. 12. The three-dimensional image input device according to claim 1, wherein the distance data is represented by difference data from reference distance data. 13. The method according to claim 12, wherein the reference distance data is the distance data adjacent in a predetermined direction, and when there is no adjacent distance data to be used as a reference, the distance data is not difference data. 3D image input device. 14. Image data corresponding to each pixel of a subject image and distance data indicating a distance to the subject corresponding to each pixel are recorded as an image data file and a distance data file, respectively. And file information indicating that the distance data file is related to each other.