JP2001119723A - Still object image detector and still object image detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate an image resulting from photographing only a target object by eliminating a mobile object passing the front and rear of the object. SOLUTION: A two-dimensional image each of pixels of which corresponds to visual sense information of an object and a three-dimensional image in pairs with this two-dimensional image and each of pixels values of a which corresponds to distance information up to the object are photographed respectively by a prescribed number of times. The distance information and the visual sense information obtained by the 1st photographing are substituted into variables D and P respectively. The pixel value ON of the three-dimensional image photographed at the N-th order is compared with the variable D. When the DN is smaller than the D and the pixel of the N-th three-dimensional image has a pixel value remoter in distance than the variable D, the variables D, P are updated with the pixel values DN, PN of the N-th three-dimensional image and the N-th two-dimensional image. After repeating this (steps 401-410) for a prescribed number of times, the two-dimensional image using the variable P for the pixel value is generated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image detecting apparatus for photographing a still image such as a digital camera, a method for detecting the image, and a three-dimensional image for detecting a three-dimensional shape of a subject using a light propagation time measuring method. It relates to a detection device.

[0002]

2. Description of the Related Art When photographing a landscape or a building using a camera or the like, in order to prevent a person or a vehicle traveling before and after a target subject from being photographed, it is usually necessary to wait for the traffic to stop. In addition, shooting is not easy. A device and a method disclosed in Japanese Patent Application Laid-Open No. 10-341395 are known as a device and a method for capturing an image of a target scenery, a building, or the like without waiting for a passing traffic or a vehicle to be interrupted. In this method, the subject is repeatedly photographed, the pixel values of each pixel are compared between the photographed images, and when the same pixel value is obtained in a predetermined number of images, the pixel value of the pixel is determined. Is repeated until the pixel values of all the pixels are determined, so that only the image corresponding to the still object is extracted from the image including the moving object, and the image composed of only the still object is synthesized.

On the other hand, as a three-dimensional image detecting device for detecting a distance to a subject for each pixel, “Measurement Scienc
e and Technology "(S. Christie et al., vol. 6, p1301-1
308, 1995) and WO 97/01111
And the like disclosed in Japanese Patent Application Laid-Open Publication No. H10-260, are known. In these three-dimensional image detection devices, the subject is irradiated with pulse-modulated laser light, and the reflected light is received by a two-dimensional CCD sensor and converted into an electric signal. At this time, by controlling a shutter operation of an electric engineering shutter composed of a mechanical or liquid crystal display combined with a two-dimensional CCD, an electric signal correlating to the distance to the subject is converted to a C signal.
It can be detected for each pixel of the CD. From this electric signal, the distance to the subject corresponding to each pixel of the CCD is detected. The detection of the distance at this time is performed by one shutter operation.

[0004]

Problems to be Solved by the Invention Japanese Patent Laid-Open No. Hei 10-3413
In the apparatus and the method disclosed in Japanese Patent Publication No. 95, when a predetermined number of images have the same pixel value in a plurality of captured images, the pixel value of the pixel is determined, and the image including the moving object is determined. An image of a stationary object is extracted. Therefore, if the image includes a monochromatic object moving at a low speed or an object that does not move for a while after the start of the shooting operation and suddenly starts moving during the shooting operation, the image of the moving object is included. It may be extracted.

The present invention is to accurately detect distance information or image information of a stationary object from an image including a moving object based on distance information to a subject detected for each pixel by a three-dimensional image detection device. It is an object.

[0006]

A still object image detecting apparatus according to the present invention comprises: a three-dimensional image detecting means for detecting a three-dimensional image in which each pixel value of an image corresponds to distance information to a subject;
By driving the three-dimensional image detecting means, the first three-dimensional image and the second three-dimensional image from the same point and the same direction are detected at a predetermined time interval, and the distance between the first and second three-dimensional images is detected. And pixel selection means for comparing the distance information corresponding to each pixel and selecting a pixel indicating the distance information at a longer distance for each pixel.

[0007] The still object image detecting device preferably includes a two-dimensional image detecting means for detecting a two-dimensional image in which each pixel value of the image corresponds to visual information of a subject, and a first three-dimensional image corresponding to the first three-dimensional image. Detecting a two-dimensional image and a second two-dimensional image corresponding to the second three-dimensional image, and extracting a pixel value of the two-dimensional image corresponding to the pixel selected by the pixel selecting means for each pixel And a two-dimensional image generating means for generating a two-dimensional image of a subject at a longer distance.

Preferably, the pixel value corresponding to the visual information is
These are three pixel values corresponding to the three color system of red, green, and blue.

Preferably, the still object image detecting apparatus generates a three-dimensional image of a subject located at a farther distance from the pixel value selected by the pixel selecting means.

Preferably, when the distance information between the first three-dimensional image and the second three-dimensional image is equal in the pixel selecting means, the pixel values from the first and second two-dimensional images in the two-dimensional image generating means are determined. The extraction is performed by comparing the magnitude of the luminance corresponding to the pixel value of each two-dimensional image.

The still object image detecting method according to the present invention comprises
A two-dimensional image and a two-dimensional image corresponding to the three-dimensional image are repeatedly photographed, and for each pixel, a two-dimensional image corresponding to a pixel of the three-dimensional image that holds the longest distance information in the repeatedly photographed image A two-dimensional image of a subject is generated using pixel values of the image.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a perspective view of a camera-type image detection device according to an embodiment of the present invention.

On the front of the camera body 10, a finder window 12 is provided at the upper left of the taking lens 11, and a flash 13 is provided at the upper right. On the upper surface of the camera body 10, a light emitting device 14 for irradiating a laser beam, which is distance measuring light, is provided just above the taking lens 11. A release switch 15 and a liquid crystal display panel 16 are provided on the left side of the light emitting device 14, and a mode switching dial 1 is provided on the right side.
7 are provided. The IC on the side of the camera body 10
A card insertion slot 19 for inserting a recording medium such as a memory card is formed, and a video output terminal 20 and an interface connector 21 are provided.

FIG. 2 is a block diagram showing a circuit configuration of the camera shown in FIG. An aperture 25 is provided in the taking lens 11. The opening of the aperture 25 is determined by the iris drive circuit 2.
Adjusted by 6. The focus adjustment operation and the zooming operation of the taking lens 11 are controlled by a lens drive circuit 27.

An image pickup device (C
CD) 28 is provided. A subject image is formed on the CCD 28 by the photographing lens 11, and charges corresponding to the subject image are generated. Operations such as a charge accumulation operation and a charge read operation in the CCD 28 are controlled by the CCD drive circuit 30. The charge signal, that is, the image signal, read from the CCD 28 is amplified by the amplifier 31 and the A / A
The analog signal is converted into a digital signal in the D converter 32. The digital image signal is output from the imaging signal processing circuit 3
In the image memory 3, processing such as gamma correction is performed.
4 is temporarily stored. The iris drive circuit 26, lens drive circuit 27, CCD drive circuit 30, and image signal processing circuit 33 are controlled by a system control circuit 35.

The image signal is read from the image memory 34 and supplied to the LCD drive circuit 36. The LCD drive circuit 36 operates in accordance with the image signal, and thereby the image display L
An image corresponding to the image signal is displayed on the CD panel 37.

If the camera is connected to a monitor device 39 provided outside the camera body 10 by a cable, the image signal read from the image memory 34 is transmitted to the TV signal encoder 3.
8. It can be transmitted to the monitor device 39 via the video output terminal 20. The system control circuit 35 is connected to the interface circuit 40, and the interface circuit 40 is connected to the interface connector 21. Therefore, if the camera is connected to the computer 41 provided outside the camera body 10 via the interface cable 41, the image signal read from the image memory 34 can be transmitted to the computer. The system control circuit 35 is connected to the image recording device 43 via the recording medium control circuit 42. Therefore, the image signal read from the image memory 34 is
The data can be recorded on a recording medium M such as an IC memory card mounted on the image recording device 43.

The light emitting device 14 includes a light emitting element 14a and an illumination lens 14b. The light emitting operation of the light emitting element 14a is controlled by a light emitting element control circuit 44. Light emitting element 1
Reference numeral 4a denotes a laser diode (LD), and the emitted laser light is used as distance measuring light for detecting the distance to the subject. This laser light is applied to the entire subject through the illumination lens 14b. The laser beam reflected by the subject enters the photographic lens 11 and is detected by the CCD 28, whereby distance information to the subject is detected.

The system control circuit 35 is connected to a switch group 45 composed of the release switch 15 and the mode switching dial 17, and a liquid crystal display panel (display element) 16.

Next, the photographing operation executed in this embodiment will be described with reference to the flowchart of FIG.

In the photographing operation of this embodiment, a two-dimensional image (image information detected by ordinary CCD video control) corresponding to the visual information of the subject and a three-dimensional image corresponding to the distance information (the distance information detecting operation). (The detected image information) is detected a predetermined number of times, and a moving object such as a traffic object is removed based on the data to generate an image of a target still object. This photographing operation is executed when the mode switching dial 17 is set to a photographing mode for removing a moving object.

When it is confirmed in step 101 that the release switch 15 is fully depressed, step 1 is executed.
02 is executed, and 1 which is an initial setting value is substituted for a variable N indicating the number of times of photographing. Next, in step 103, a first (N = 1) two-dimensional image of the subject is captured by an image information detecting operation (two-dimensional image detecting means). Step 1
In 04, the first (N = 1) three-dimensional image of the subject is detected by the distance information detecting operation (three-dimensional image detecting means).

In step 105, it is determined whether or not N = 1. If it is determined that N = 1, the process proceeds to step 107. If it is determined that N ≠ 1, the image processing for removing the image corresponding to the moving object in the two-dimensional image is performed in step 106.

In step 107, it is determined whether or not N = M. M is a predetermined number of detections (photographings) set in the main photographing operation, and when it is determined that N = M, this photographing operation ends. If it is determined that N ≠ M, 1 is added to N in step 108, and the process returns to step 103 again. Step 103 to step 108
Are repeatedly executed until it is determined in step 107 that N = M. Therefore, at the end of the main photographing operation, M sets of two-dimensional images and three-dimensional images have been detected.
The default value of M is set at the time of manufacture or shipment of the camera, and is recorded in a ROM or the like.

FIG. 4 is a flowchart of the image detecting operation executed in step 103. Step 20
At 1, normal CCD video control is turned on and an Nth two-dimensional image of the subject is captured. The image data of the captured N-th two-dimensional image is
Is temporarily stored in the image memory 34. In step 203, the normal CCD video control is set to the off state, and this processing ends.

Next, the distance information detecting operation executed in step 104 will be described. First, the basic principle of distance measurement in the present embodiment will be described with reference to FIGS. In FIG. 6, the horizontal axis is time t.

The distance measuring light output from the distance measuring device B is reflected by the subject S and received by a CCD (not shown). The distance measuring light is a pulsed light having a predetermined pulse width H, and therefore, the reflected light from the subject S is also a pulsed light having the same pulse width H. The rise of the reflected light pulse is delayed by a time δ · t (δ is a delay coefficient) from the rise of the distance measuring light pulse. Since the ranging light and the reflected light have traveled twice the distance r between the distance measuring device B and the subject S, the distance r is given by r = δ · t · C / 2 (1) can get. Where C is the speed of light.

For example, a state in which the reflected light can be detected from the rise of the pulse of the distance measuring light is determined, and the state is changed to an undetectable state before the reflected light pulse falls, that is, a reflected light detection period T is provided. And the received light amount A during the reflected light detection period T is a function of the distance r. That is, the light receiving amount A decreases as the distance r increases (the time δ · t increases).

In the present embodiment, utilizing the above-described principle,
The distance from the camera body 10 to each point on the surface of the subject S is detected by detecting the amount of received light A at each of a plurality of photodiodes provided in the CCD 28 and arranged two-dimensionally. Data of a three-dimensional image relating to a shape is input collectively.

FIG. 7 is a diagram showing the arrangement of the photodiode 51 and the vertical transfer unit 52 provided on the CCD 28.
FIG. 8 is a cross-sectional view of the CCD 28 cut along a plane perpendicular to the substrate 53. This CCD 28 is a conventionally known interline type CCD, and uses a VOD for sweeping out unnecessary charges.
(Vertical overflow drain) method.

The photodiode 51 and the vertical transfer section 52 are formed along the surface of the n-type substrate 53. The photodiodes 51 are two-dimensionally arranged in a lattice pattern, and the vertical transfer units 52 are provided adjacent to the photodiodes 51 arranged in a line in a predetermined direction (vertical direction in FIG. 7). The vertical transfer section 52 has four vertical transfer electrodes 52a, 52b, 52c, and 52d for one photodiode 51. Therefore, in the vertical transfer section 52, four potential wells can be formed, and signal charges can be output from the CCD 28 by controlling the depths of these wells as conventionally known. The number of vertical transfer electrodes can be freely changed according to the purpose.

The photodiode 51 is formed in a p-type well formed on the surface of the substrate 53, and the p-type well is completely depleted by a reverse bias voltage applied between the p-type well and the n-type substrate 53. You. In this state, charges corresponding to the amount of incident light (reflected light from the subject) are accumulated in the photodiode 51. When the substrate voltage Vsub is increased to a predetermined value or more, the electric charge accumulated in the photodiode 51 is discharged to the substrate 53 side. On the other hand, when a charge transfer signal (voltage signal) is applied to the transfer gate unit 54, the charges accumulated in the photodiode 51 are transferred to the vertical transfer unit 52. That is, after the charges are swept to the substrate 53 side by the charge sweeping signal, the signal charges accumulated in the photodiode 51 are changed by the charge transfer signal to the vertical transfer unit 5.
It is transferred to the two sides. By repeating such an operation, the signal charge is integrated in the vertical transfer unit 52, and a signal charge corresponding to the amount of light reflected from the subject (light reception amount A) is detected.

FIG. 9 is a timing chart in the distance information detecting operation in the present embodiment. The distance information detecting operation in the present embodiment will be described with reference to FIGS. 1, 2, and 7 to 9. In the distance information detection operation of the present embodiment, unlike the description of the basic principle of the distance measurement performed with reference to FIG. 6, in order to reduce noise due to the influence of external light, the distance measurement pulse starts from the falling edge of the pulse. The timing chart is configured such that the reflected light is set to a detectable state, and the state is switched to an undetectable state after the pulse of the reflected light has fallen. However, the timing chart is not different in principle.

A charge discharging signal (pulse signal) S1 is output in synchronization with the output of the vertical synchronizing signal (not shown), whereby unnecessary charges accumulated in the photodiode 51 are discharged in the direction of the substrate 53. Then, the accumulated charge amount in the photodiode 51 becomes zero (reference S2). After the start of output of the charge sweeping signal S1, pulse-shaped ranging light S3 having a constant pulse width is output. The period (pulse width) during which the ranging light S3 is output can be adjusted.
The adjustment is performed so that the distance measurement light S3 is turned off simultaneously with the output of the charge sweeping signal S1.

The distance measuring light S3 is reflected by the object,
It is incident on D28. That is, although the reflected light S4 from the subject is received by the CCD 28, the charge sweeping signal S1
Is not stored in the photodiode 51 (reference S2). Charge sweep signal S1
Is stopped, the photodiode 51 starts to accumulate charges by receiving the reflected light S4, and the reflected light S4
4 and the external light generate signal charges S5. When the reflected light S4 disappears (reference S6), the photodiode 51 ends the charge accumulation based on the reflected light (reference S7).
Charge accumulation due to only external light continues (reference S8).

Thereafter, when the charge transfer signal S9 is output, the charges stored in the photodiode 51 are transferred to the vertical transfer unit 52. This charge transfer is completed when the output of the charge transfer signal ends (reference S10). In other words, the charge accumulation in the photodiode 51 continues due to the presence of external light, but the signal charge S1 accumulated in the photodiode 51 until the output of the charge transfer signal ends.
1 is transferred to the vertical transfer unit 52. The charge S14 accumulated after the output of the charge transfer signal ends remains in the photodiode 51 as it is.

As described above, the period T from the end of the output of the charge sweeping signal S1 to the end of the output of the charge transfer signal S9.
_{During U1} , the photodiode 51 accumulates signal charges corresponding to the distance to the subject. Then, the reflected light S4
Until the end of light reception (reference S6), the charges accumulated in the photodiode 51 are transferred to the vertical transfer unit 52 as signal charges S12 (hatched portion) corresponding to the distance information of the subject,
Other signal charges S13 are caused only by external light.

After a predetermined time has elapsed from the output of the charge transfer signal S9, the charge sweeping signal S1 is output again, and after the transfer of the signal charge to the vertical transfer unit 52, the photodiode 51
Unnecessary charges accumulated in the substrate 53 are swept toward the substrate 53.
That is, accumulation of signal charges in the photodiode 51 is newly started. Then, as described above, when the charge accumulation period T _U1 has elapsed, the signal charge is transferred to the vertical transfer unit 52.

The vertical transfer section 5 of such signal charges S11
2 is repeatedly executed until the next vertical synchronization signal is output. As a result, in the vertical transfer unit 52, the signal charge S11 is integrated, and the signal charge S11 integrated in a period of one field (a period sandwiched between two vertical synchronization signals) is regarded as a period in which the subject is stationary. If possible, it corresponds to distance information to the subject. Since the signal charge S13 is smaller than the signal charge S12, the signal charge S11 can be regarded as being equal to the signal charge S12.

The detection operation of the signal charge S11 described above relates to one photodiode 51, and such a detection operation is performed in all the photodiodes 51. As a result of the detection operation in the period of one field, the vertical transfer unit 52 adjacent to each photodiode 51
The distance information detected by the photodiode 51 is held in each of the parts. This distance information is output from the CCD 28 by a vertical transfer operation in the vertical transfer unit 52 and a horizontal transfer operation in a horizontal transfer unit (not shown).

FIG. 10 is a flowchart of the distance information detecting operation executed in step 104.

In step 301, a vertical synchronizing signal is output, and distance measuring light control is started. That is, the light emitting device 14 is driven, and the pulse-shaped ranging light S3 is output intermittently. Next, step 302 is executed.
28 starts detection control. That is, the distance information detection operation described with reference to FIG. 9 is started, the charge sweeping signal S1 and the charge transfer signal S9 are alternately output, and the signal charge S11 of the distance information is integrated in the vertical transfer unit 52.

In step 303, it is determined whether one field period has elapsed from the start of the distance information detection operation, that is, whether a new vertical synchronizing signal has been output.
When one field period ends, the process proceeds to step 304,
The signal charge of the distance information integrated in the vertical transfer unit 52 is output from the CCD 28. This signal charge is temporarily stored in the image memory 34 in step 305.

In step 306, the distance measuring light control is turned off, and the light emitting operation of the light emitting device 14 is stopped. In step 307, arithmetic processing of distance data is performed.
In step 308, the calculated distance data is temporarily stored in the image memory 34 as three-dimensional image data.

Next, the contents of the arithmetic processing executed in step 307 of the distance information detecting operation will be described with reference to FIG.

It is assumed that a subject having a reflectance R is illuminated, and this subject is regarded as a secondary light source having a luminance I and is imaged on a CCD. At this time, the output S obtained by integrating the charge generated in the photodiode during the charge storage time t is obtained.
n is represented by: Sn = k · R · I · t (2) Here, k is a proportionality constant, which varies depending on the F number, magnification, and the like of the taking lens.

As shown in FIG. 9, the charge storage time is T _U1 , the pulse width of the distance measuring light S3 is T _S , and the pulse width of the distance information signal charge S12 is T _D, and the charge storage during one field period is performed. When the time is to be repeated N times, the output SM _{10 obtained, SM 10 = Σk · R ·} I · T D = k · N · R · I · T D ··· (3) and composed. The pulse width T _D can be expressed as T _D = δ · t = 2r / C (4) At this time, the distance r to the subject can be expressed as follows: r = C · SM ₁₀ / (2 · k · N · R · I) (5) Therefore, if the proportional constant k, the reflectance R, and the luminance I are obtained in advance, the distance r can be obtained.

Next, the image processing executed in step 106 (FIG. 3) will be described with reference to FIGS. FIGS. 11A to 11C show first, second, and third two-dimensional images obtained by driving the image information detecting means (step 103) three times.

In each of the captured images shown in FIGS. 11 (a) to 11 (c), a house O ₁ and a tree O ₂ which are objects to be photographed, and a dog O ₃ other than the object to be photographed are photographed. Have been. Dog O ₃ is moving through the screen from right to left,
First, second, third two-dimensional images, because they are captured at predetermined time intervals, the position being imaged dog O ₃ for each image is different. That is, in FIG. 11 (a), on the right side of the screen, and in FIG.
In the figure, an image of the dog O ₃ is captured on the left side of the screen. The house O ₁ and the tree O ₂ which are the shooting objects are still objects,
The position does not change in each of the first to third two-dimensional images. FIG. 12 shows two-dimensional images generated based on first to third two-dimensional images (FIGS. 11A to 11C) and three-dimensional images corresponding to the two-dimensional images.
Dog O ₃ which is a moving object other than the house O ₁ and the tree O ₂ which are stationary objects
Has been removed from the image.

A moving object behind a stationary object is not photographed, and the photographed moving object is always located in front of a stationary object such as a landscape. Therefore, the pixel value indicating the farthest distance for each pixel among the plurality of three-dimensional images corresponds to the distance to the stationary object. That is, by selecting the pixel value of the two-dimensional image corresponding to the pixel holding the farthest distance for each pixel, it is possible to generate a two-dimensional image of only the stationary object from which the image of the moving object has been removed.

For example, the three points Q ₀ , Q ₁ , and Q ₂ shown in each of FIGS.
Pixel points corresponding to the same pixel in the three two-dimensional images shown in FIGS. In the three images, the distance to the subject corresponding to the pixel point Q ₀ is the longest in the second and third two-dimensional images (FIGS. 11B and 11B).
A (c)), and corresponds to the point on the home O _1. The first and third two-dimensional image of the distance to the object corresponding to the pixel point Q ₁ is the most distant (FIG. 11 (a), the FIG. 11 (c)) is, corresponding to points on the wooden O ₂ ing. The pixel point Q ₂
The first and second two-dimensional images (FIGS. 11 (a) and 11 (b)) have the longest distances to the subject corresponding to the point corresponding to the point at infinity.

The comparison of the distance for each pixel can be performed based on the pixel value of the three-dimensional image paired with each two-dimensional image. Therefore, for pixel point Q ₀ , the pixel value of the second or third two-dimensional image, for pixel point Q ₁ , the pixel value of the first or third two-dimensional image, and for pixel point Q ₂ , by employing one or pixel values of the second two-dimensional images, it is possible to generate an image dogs O ₃ is removed from the image is a moving object as shown in FIG. 12. In the present embodiment, when there are a plurality of options for the pixel values of the two-dimensional image to be adopted, such as the pixel points Q ₀ , Q ₁ , and Q ₂ , the pixel value of each two-dimensional image having the larger luminance is determined. Has adopted. This is because the shooting conditions change during the shooting operation,
This is so that even when the pixel values of the two-dimensional image of the same subject are different, the unevenness of the pixel values adopted for each pixel is reduced.

FIG. 13 is a flowchart of the image processing routine executed in step 106.

The set (X, Y) of variables X and Y in FIG.
This corresponds to the two-dimensional arrangement of pixels (photodiodes) in D28. That is, X corresponds to the arrangement in the horizontal direction, and Y corresponds to the arrangement in the vertical direction. For example, X =
When 3, Y = 5, the set of variables X, Y (3, 5) represents the pixels of the CCD 28 located at the third pixel from the left and the fifth pixel from the top. The variables D (X, Y) and P (X, Y) are the pixels (X, X) in the 3D image and the 2D image, respectively.
Y) represents the pixel value. That is, the variable D (X, Y)
Corresponds to the distance information to the subject, and the variables P (X,
Y) corresponds to, for example, three pixel values related to red (R), green (G), and blue (B), and these three pixel values are represented by one variable.

When the first three-dimensional image and the first two-dimensional image are photographed (steps 104 and 103), the variables D (X, Y) and P (X, Y) Are respectively assigned. That is, when the image processing routine is started, each pixel value corresponding to the image shown in FIG.
(X, Y).

In step 401, the variables X and Y are initialized, and each of the variables X and Y is set to 1.

In step 402, the pixel value DN (X, Y) of the pixel (X, Y) in the N-th three-dimensional image is compared with the pixel value D (X, Y). If it is determined that the pixel value DN (X, Y) is smaller than the pixel value D (X, Y), the value of D (X, Y) is changed to DN (X, Y) in step 406.
, And at the same time, the value of P (X, Y) is rewritten to the pixel value PN (X, Y) of the N-th two-dimensional image.
That is, D (X, Y) and P (X, Y) hold the pixel values of the three-dimensional image and the two-dimensional image that are farther away from the subject, respectively. In step 406, D
When the values of (X, Y) and P (X, Y) are rewritten, the process proceeds to step 407.

On the other hand, at step 402,
Pixel value DN (X, Y) of the pixel (X, Y) in the three-dimensional image
If it is determined that the value of (Y) is equal to or greater than the value of the pixel value D (X, Y), in step 403, it is determined whether or not the value of DN (X, Y) is equal to the value of D (X, Y). Is determined. D
If it is determined that (X, Y) ≠ DN (X, Y), the process proceeds to step 407. D (X, Y) = DN (X,
If determined as Y), at step 404, P (X,
Y) and the brightness B (P (X, Y)) of PN (X, Y) and B
The value of (PN (X, Y)) is compared. When the luminance B (PN (X, Y)) of the N-th two-dimensional image is equal to or less than B (P (X, Y)), the process proceeds to step 407. Also, the N-th 2
The brightness B (PN (X, Y)) of the two-dimensional image is B (P (X, Y
Y)), then in step 405 P
When the value of (X, Y) is the pixel value PN (X, X,
Y). That is, D (X, Y) = DN
In the case of (X, Y), a pixel value with higher luminance is P
(X, Y). Note that B (P (X, Y)) is a function for calculating the brightness of the RGB pixel values held in the variable P (X, Y).

In step 407, it is determined whether X is equal to the number H of pixels in the horizontal direction. When it is determined that X ≠ H, 1 is added to X in step 408, and the process returns to step 402. Step 402 to step 4
Step 08 is repeatedly executed until it is determined in step 407 that X = H. That is, by repeating Steps 402 to 408, the pixel values of one horizontal line are compared with each other, and the values of D (X, Y) and P (X, Y) of one horizontal line are set. Is done.

On the other hand, if it is determined in step 407 that X = H, it is determined in step 409 whether or not Y is equal to the number V of pixels in the vertical direction. When it is determined that Y ≠ V, X is set to 1, 1 is added to Y, and the process returns to step 402. Steps 403 to 410 are repeatedly executed until it is determined in step 409 that Y = V. That is, step 403
Steps 410 to 410 are repeatedly performed to compare the pixel values corresponding to each pixel from the top horizontal line to the bottom horizontal line in order, and D (X,
Y) and P (X, Y) are set. Step 409
If it is determined that Y = V, the image processing subroutine ends.

The image processing routine is repeatedly executed during the photographing operation shown in FIG. 3, and at the end of the photographing operation, a two-dimensional image of only a stationary object is generated from the M two-dimensional images, and a variable P ( X, Y). Variable D
(X, Y) holds distance information corresponding to a stationary object. These variables P (X, Y) and variables D (X, Y)
Is recorded on the recording medium M or the like at the end of the photographing operation.

As described above, according to the present embodiment, since the image of the still object is detected based on the distance information, if the still object to be photographed is photographed even once during the photographing operation, the still image is surely formed. Can be detected.

In the photographing operation of this embodiment, the photographing is repeated a predetermined number of times. However, the photographing may be repeated while the release switch 15 is pressed.

In the present embodiment, the two-dimensional image and the three-dimensional image to be paired are sequentially photographed.
And a half mirror may be used to simultaneously capture images.

[0065]

As described above, according to the present invention, based on the distance information to the subject detected for each pixel by the three-dimensional image detection device, the distance information of the stationary object or the distance information of the stationary object from the image including the moving object is obtained. Image information can be accurately detected.

[Brief description of the drawings]

FIG. 1 is a perspective view of a camera-type distance measuring apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a circuit configuration of the camera shown in FIG.

FIG. 3 is a flowchart of a shooting operation according to the embodiment.

FIG. 4 is a flowchart of an image information detecting operation.

FIG. 5 is a diagram for explaining the principle of distance measurement using ranging light.

FIG. 6: distance measuring light, reflected light, gate pulse, and CCD
FIG. 4 is a diagram showing a distribution of the amount of light received by the light source.

FIG. 7 is a diagram illustrating an arrangement of a photodiode and a vertical transfer unit provided in a CCD.

FIG. 8 is a cross-sectional view of the CCD cut along a plane perpendicular to the substrate.

FIG. 9 is a timing chart of a distance information detecting operation for detecting data relating to a distance to a subject.

FIG. 10 is a flowchart of a distance information detecting operation.

FIG. 11 is a diagram showing first, second, and third two-dimensional images.

12 is a two-dimensional image in which only a stationary object as a target object generated by performing image processing on the first, second, and third two-dimensional images of FIG. 11 is recorded.

FIG. 13 is a flowchart of an image processing routine.

[Explanation of symbols]

 14 Light emitting device 28 CCD

Claims (6)

[Claims] 1. A three-dimensional image detecting means for detecting a three-dimensional image in which each pixel value of an image corresponds to distance information to a subject; and driving the three-dimensional image detecting means to detect a three-dimensional image from the same point and from the same direction. The first three-dimensional image and the second three-dimensional image are detected at predetermined time intervals, and distance information corresponding to each pixel is compared between the first and second three-dimensional images. A still object image detecting device, comprising: a pixel selecting unit that selects a pixel indicating distance information of the distance for each pixel. 2. A two-dimensional image detecting means for detecting a two-dimensional image in which each pixel value of the image corresponds to visual information of a subject; a first two-dimensional image corresponding to the first three-dimensional image;
By detecting a second two-dimensional image corresponding to the second three-dimensional image and extracting a pixel value of a two-dimensional image corresponding to the pixel selected by the pixel selecting unit for each pixel, Generate a two-dimensional image of a distant subject 2
2. The apparatus according to claim 1, further comprising: a two-dimensional image generating unit.
3. The still object image detecting device according to 1. 3. A pixel value corresponding to the visual information is red,
3. The still object image detecting device according to claim 2, wherein the three pixel values correspond to three color systems of green and blue. 4. The still object image detecting apparatus according to claim 1, wherein a three-dimensional image of a subject at a farther distance is generated from the pixel value selected by the pixel selecting unit. 5. The method according to claim 1, wherein the first three
When the distance information between the two-dimensional image and the second three-dimensional image is the same, the first and second two-dimensional image generating means may be used.
3. The still object image detecting device according to claim 2, wherein the extraction of the pixel value from the two-dimensional image is performed by comparing the magnitude of the luminance corresponding to the pixel value of each of the two-dimensional images. 6. The three-dimensional image according to claim 1, wherein the three-dimensional image
3. A two-dimensional image according to claim 2, which corresponds to a three-dimensional image corresponding to a three-dimensional image that holds the distance information of the longest distance among the repeatedly captured images. A two-dimensional image of a subject is generated using pixel values of the two-dimensional image.