JP2008175538A - Photographing device, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable accurate distance measurement when distance measurement is performed by a TOF (Time Of Flight) method. <P>SOLUTION: An optical distance calculation part 52 irradiates a subject with light, detects light reflected by the subject through a photographing system 20, and calculates an optical subject distance of the subject based on the time for the photographing system 20 to detect the reflected light after the emission of the light. Light emitting parts 41A and 41B irradiate the subject with a plurality of lights of different wavelengths when this optical subject distance is calculated. The photographing system 20 receives reflected lights generated by reflection of the plurality of lights by the subject for every different wavelength. An intensity acquisition part 50 acquires intensity of the received reflected light. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a photographing apparatus and method having a function of calculating a subject distance from a photographing means for obtaining an image by photographing, and a program for causing a computer to execute the photographing method.

  In a photographing device that captures an image by photographing a subject, the distance from the photographing device to the subject is measured by measuring the time until ranging light such as near infrared rays emitted toward the subject is reflected by the subject and returns. A distance image is created by measuring a distance. A method of measuring the distance to the subject (ranging) using the reflection of light in this manner is called a TOF (Time Of Flight) method. When the TOF method is used, the reflected light from the subject is not reflected. If it cannot be obtained, ranging cannot be performed. Even if the reflected light is obtained, the influence of noise increases if the amount of light is small, so accurate distance measurement cannot be performed. For example, when the subject is at a position far from the photographing apparatus, a sufficient amount of reflected light cannot be obtained, and thus accurate distance measurement cannot be performed.

  In such a case, if the detection period of the reflected light is lengthened, a sufficient amount of reflected light necessary for distance measurement can be received. However, if the detection period of the reflected light is lengthened, the light detection element that receives the reflected light is likely to be saturated when the subject is close to the photographing apparatus. When the light detection element is saturated, the received light amount of the reflected light does not correspond to the output signal of the light detection element, so that accurate distance measurement cannot be performed.

For this reason, a plurality of detection periods for integrating the signal charges generated by the light detection element are set, and the detected charge amount is within a range not exceeding the charge amount allowed for the light detection element from the set detection periods. There has been proposed a method of calculating a distance to a subject with as little influence as possible by selecting a maximum detection period and performing distance measurement using the detected period (see Patent Document 1).
JP 2006-84430 A

  However, when distance measurement is performed by the TOF method, if the ambient light included in the shooting environment includes a lot of wavelength components of light irradiated to the subject for distance measurement, the ambient light is included as noise in the reflected light. Therefore, accurate distance measurement cannot be performed.

  The present invention has been made in view of the above circumstances, and an object thereof is to enable accurate distance measurement when distance measurement is performed in the TOF method.

An imaging apparatus according to the present invention irradiates a subject with light, detects reflected light of the light from the subject, and detects the optical of the subject based on a time from when the light is emitted until the reflected light is detected. In an imaging apparatus provided with an optical distance calculation means for calculating a target object distance,
Light irradiation means for irradiating the subject with a plurality of lights having different wavelengths;
A light receiving means for receiving the reflected light of the plurality of lights by the subject for each of the different wavelengths or for each predetermined wavelength in a band in which the different wavelengths and the different wavelengths overlap;
Intensity acquisition means for acquiring the intensity of the received reflected light is provided.

  Here, the “optical distance calculation means” is a means for performing distance measurement by the TOF method.

In the photographing apparatus according to the present invention, the light receiving means is a means for obtaining a reflected light image that is an image of the received reflected light,
The intensity acquisition means is a means for acquiring a histogram of luminance of the reflected light image as the intensity of the reflected light,
The apparatus may further include wavelength determining means for determining light having a wavelength obtained from a reflected light image in which an area having a predetermined luminance or less in the histogram is equal to or less than a predetermined threshold as light for irradiating the subject.

Further, in the photographing apparatus according to the present invention, when there is no reflected light image in which an area equal to or smaller than the predetermined luminance in the histogram is equal to or smaller than a predetermined threshold value, the intensity acquisition unit is provided for each of the reflected light images. As a means to calculate the luminance histogram,
The wavelength determining unit may be a unit that determines, as light to irradiate the subject, light having a wavelength at which a reflected light image having the smallest area equal to or less than a predetermined luminance in the plurality of histograms is obtained.

The imaging method according to the present invention irradiates a subject with light, detects reflected light of the light from the subject, and detects the optical of the subject based on the time from when the light is emitted until the reflected light is detected. In the photographing method in the photographing apparatus provided with the optical distance calculating means for calculating the target object distance,
Irradiating the subject with a plurality of lights having different wavelengths;
The reflected light from the subject of the plurality of lights is received for each different wavelength, or for each predetermined wavelength in a band where the different wavelengths and the different wavelengths overlap,
The intensity of the received reflected light is acquired.

  In addition, you may provide as a program for making a computer perform the imaging | photography method by this invention.

  According to the present invention, the subject is irradiated with a plurality of lights having different wavelengths, and the reflected light from the subject of the plurality of lights is received for each different wavelength, or for each predetermined wavelength in a band where different wavelengths and different wavelengths overlap, The intensity of the received reflected light is acquired. Therefore, based on the intensity of the acquired reflected light, it is possible to determine the light that irradiates the subject with light having a wavelength that is not affected by the environmental light as much as possible. For this reason, when performing distance measurement by the TOF method, the influence of ambient light can be reduced, and as a result, the noise of the reflected light can be reduced and the subject distance can be accurately calculated.

  In particular, a histogram of the brightness of the reflected light image is generated, and the reflected light image in which the area below the predetermined brightness in the histogram is equal to or less than the predetermined threshold is determined as the light for irradiating the subject with light of the wavelength. Thus, it is possible to determine the light to be irradiated on the subject by a simple calculation.

  In addition, when there is no reflected light image in which the area less than or equal to the predetermined luminance in the histogram is less than or equal to the predetermined threshold, a histogram of the luminance of the plurality of reflected light images is generated, and the area less than or equal to the predetermined luminance in the histogram By determining the light having the wavelength with the smallest reflected light image as the light for irradiating the subject, the light for irradiating the subject can be determined by a simple calculation.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a digital camera to which the photographing apparatus according to the first embodiment of the present invention is applied. A digital camera 1 shown in FIG. 1 includes an operation system 10 such as an operation mode switch, up / down / left / right buttons, a release button, and a power switch, and an operation system control unit that is an interface part for transmitting operation contents of these switches to the CPU 42 12.

  As an optical system, a lens 21 is provided. The lens 21 includes a plurality of functional lenses such as a focus lens for focusing on a subject and a zoom lens for realizing a zoom function, and the position of each functional lens is adjusted by a lens driving unit 22.

  The diaphragm 23 is driven by a diaphragm driving unit 24. The aperture drive unit 24 adjusts the aperture diameter.

  The shutter 25 is a mechanical shutter and is driven by a shutter driving unit 26. The shutter driving unit 26 controls opening / closing of the shutter 25 in accordance with a signal generated by pressing the release button.

  An image sensor 27 is provided behind the optical system. The image sensor 27 is a distance image CMOS sensor that can detect both distance and image, and has a photocathode in which pixels including a plurality of light receiving elements are two-dimensionally arranged and passes through an optical system. The subject light thus formed forms an image on the photoelectric surface and is photoelectrically converted. In front of the photocathode, a microlens array for condensing light on each pixel and a color filter array in which filters of R, G, and B colors are regularly arranged are arranged. The image sensor 27 outputs the electric charge accumulated for each pixel as a serial analog image signal for each line in synchronization with the vertical transfer clock and the horizontal transfer clock supplied from the image sensor control unit 28. The time for accumulating charges in each pixel, that is, the exposure time is controlled by the image sensor control unit 28.

  Here, the exposure time for each pixel can be controlled by changing the electronic shutter speed for each pixel. For example, when it is desired to shorten the exposure time, the shutter speed is increased. When it is desired to increase the exposure time, the shutter speed may be decreased. The gain of the image sensor 27 is adjusted by the image sensor control unit 28 so that an analog image signal having a predetermined size can be obtained.

  The image sensor 27 captures reflected light from the subject of the ranging light emitted from the light emitting units 41A and 41B, and acquires an imaging signal of the reflected light.

  A filter switching mechanism 60 for changing the wavelength of light detected by the image sensor 27 is provided between the diaphragm 23 and the shutter 25. FIG. 2 is a diagram showing the configuration of the filter switching mechanism 60. As shown in FIG. 2, the filter switching mechanism 60 is provided with two types of band-pass filters 62 </ b> A and 62 </ b> B on a disc 61 that can rotate around a point O, and by rotating the disc 61 around a point O. The band-pass filters 62A and 62B located on the optical axis of the image sensor 27 can be switched. Note that the rotation of the disk 61 is performed by a filter driving unit 64 controlled by the CPU 42.

  Here, in the present embodiment, the bandpass filters 62A and 62B each have a characteristic of transmitting light having a wavelength emitted from the light emitting units 41A and 41B described later. The wavelength of light emitted from the light emitting units 41A and 41B will be described later.

  In the present embodiment, the lens 21, the diaphragm 23, the shutter 25, the image sensor 27, and the filter switching mechanism 60 constitute the imaging system 20.

  An analog image signal captured from the image sensor 27 during distance measurement of the subject is input to the analog signal processing unit 30. The analog signal processing unit 30 converts a correlated double sampling circuit (CDS) that removes noise of the analog imaging signal, an auto gain controller (AGC) that adjusts the gain of the analog imaging signal, and converts the analog imaging signal into a digital signal. It consists of an A / D converter (ADC). The image data converted into the digital signal is RAW image data having R, G, and B density values for each pixel.

  The image input controller 31 writes the RAW image data input from the analog signal processing unit 30 into the frame memory 33.

  The timing generator 32 generates a timing signal. By supplying this timing signal to the shutter drive unit 26, the image sensor control unit 28, and the analog signal processing unit 30, the release button is operated, the shutter 25 is opened and closed, The capturing of the charge of the image sensor 27 and the processing of the analog signal processing unit 30 are synchronized.

  The frame memory 33 is a working memory that is used when various kinds of processing described later are performed on the image data.

  The media control unit 34 controls the writing and reading of the image file by accessing the recording medium 35.

  The display control unit 36 is for displaying an image of image data acquired by photographing on the liquid crystal monitor 37 and for displaying an image of image data stored in the recording medium 35 on the liquid crystal monitor 37.

  The image processing unit 38 performs image quality correction processing such as gradation correction, sharpness correction, and color correction on the image data, RAW image data is Y data that is a luminance signal, Cb data that is a blue color difference signal, and a red color difference signal. YC processing for converting into YC data consisting of Cr data is performed.

  The compression / decompression processing unit 39 performs a compression process on the image data of the image subjected to the correction / conversion process by the image processing unit 38 in a compression format such as JPEG, and generates an image file. In addition, when reproducing an image, the compression / decompression processing unit 39 reads the compressed image file from the recording medium 35 and performs an expansion process. The decompressed image data is output to the liquid crystal monitor 37.

  The internal memory 40 stores various constants set in the digital camera 1, programs executed by the CPU 42, and the like.

  The light emitting units 41A and 41b are controlled by the CPU 42 to emit distance measuring light toward the subject. The distance measuring light emitted toward the subject is reflected by the subject, and the reflected light is detected by the image sensor 27. The light emitting portions 41A and 41b can be configured by arranging a large number of LEDs in a planar shape or by combining a semiconductor laser and a lens.

  FIG. 3A is a diagram showing an emission spectrum of light emitted from the light emitting unit 41A, and FIG. 3B is a diagram showing an emission spectrum of light emitted from the light emitting unit 41B. As shown in FIG. 3A, the light emitting unit 41A emits light having a central wavelength of λ1 = 780 nm, and as shown in FIG. 3B, the light emitting unit 41B emits light having a central wavelength of λ2 = 850 nm. To do.

  For this reason, the transmission characteristics of the bandpass filters 62A and 62B of the filter switching mechanism 60 described above have transmission characteristics with transmittance peaks at 780 nm and 850 nm, respectively, as shown in FIGS. 4 (a) and 4 (b). Is used.

  The CPU 42 controls each part of the main body of the digital camera 1 in accordance with signals from the operation system 10 and various processing units. Further, the imaging system 20 is controlled so as to adjust the focus position by AF processing. Further, the imaging system 20 is controlled so that an appropriate exposure can be obtained by the AE process.

  In addition, the CPU 42 alternately emits light having center wavelengths λ1 and λ2 from the light emitting units 41A and 41B, and switches the bandpass filters 62A and 62B of the filter switching mechanism 60 to cause the imaging system 20 to perform photographing. That is, when the light having the center wavelength λ1 is emitted from the light emitting unit 41A, the filter switching mechanism 60 is switched so that the bandpass filter 62A is positioned on the optical axis of the imaging system 20, and the light having the center wavelength λ2 is emitted from the light emitting unit 41B. Is switched, the filter switching mechanism 60 is switched so that the bandpass filter 62B is positioned on the optical axis of the imaging system 20. Then, the imaging system 20 is caused to perform imaging with light of each wavelength.

  The intensity acquisition unit 50 creates a luminance histogram of an image acquired by the imaging system 20 by shooting. For the luminance of the image, Y data obtained by the image processing unit 38 may be used. Note that the luminance histogram corresponds to the intensity of reflected light received by the imaging system 20.

  The wavelength determining unit 51 determines the wavelength of ranging light used when the optical distance calculating unit 52 described later performs distance measurement. Specifically, the intensity acquisition unit 50 first creates a histogram H1 of the luminance of an image obtained from light having one of the center wavelengths (λ1). FIG. 5 is a diagram showing an example of a luminance histogram. Then, the wavelength determination unit 51 calculates an area A1 of the histogram equal to or lower than the predetermined threshold value Th1 of the histogram H1, and determines whether or not the calculated area A1 is equal to or lower than the predetermined threshold value Th2. . When the calculated area A1 is equal to or smaller than the threshold Th1, the light having the center wavelength λ1 obtained from the histogram H1 is determined as the distance measuring light.

  On the other hand, if the calculated area A1 exceeds the threshold value Th2, a histogram H2 of the brightness of the image obtained from the optical center of the other center wavelength λ2 is created, and similarly, the threshold value Th1 of the predetermined brightness or less. An area A2 of the histogram is calculated, and it is determined whether or not the calculated area A2 is equal to or less than a predetermined threshold value Th2.

  When the calculated area A2 is equal to or smaller than the threshold value Th1, the light having the center wavelength λ2 obtained from the histogram H2 is determined as the distance measuring light. When the calculated area A2 exceeds the threshold value Th2, the areas A1 and A2 are compared, and the light having the wavelength from which the image having the histogram with the smaller area is obtained is determined as the distance measuring light. For example, in the histograms H1 and H2 shown in FIG. 5, since the area A2 is smaller than the area below the threshold Th1, the light having the center wavelength λ2 obtained from the histogram H2 is determined as the distance measuring light.

  Here, when light having the same wavelength range as that of the light emitted from the light emitting units 41A and 41B is included in the environmental light in the photographing environment, the area below the threshold Th1 in the histogram increases. The wavelength determination unit 51 can reduce the influence of the ambient light during the distance measurement by determining the distance measurement light as described above.

  When the release button is pressed, the optical distance calculation unit 52 emits ranging light toward the subject from the light emitting units 41A and 41B, and then each pixel of the image sensor 27 reflects the reflected light from the subject of the ranging light. Is measured to calculate the optical object distance Lo. For example, as described in Patent Document 1, the intensity of the light emitted from the light emitting units 41A and 41B is modulated such that the intensity of the light periodically changes at a constant period, and the reflected light of the intensity-modulated light is converted into the imaging element 27 The phase difference of the reflected light with respect to the distance measuring light is detected from the analog imaging signal obtained by receiving the light, and after the distance measuring light is emitted using the detected phase difference, the reflected light of the distance measuring light is detected. The optical subject distance Lo is calculated from the calculated time and the speed of light.

  The calculated subject distance Lo is associated with each pixel of the image sensor 27 and is recorded on the recording medium 35 as a distance image.

  Next, processing performed in the first embodiment will be described. FIG. 6 is a flowchart showing processing performed in the first embodiment. When a shooting instruction is input from the operation system 10, the CPU 42 starts processing, and sets the light emitting unit that emits light as the first light emitting unit (here, 41A) (step ST1). As a result, the light emitting unit 41A emits light (step ST2), and the imaging system 20 captures an image to acquire an image (step ST3).

  Next, the intensity acquisition unit 50 creates a histogram of the luminance of the image acquired by photographing (step ST4), and the wavelength determination unit 51 determines that the area where the luminance in the histogram is equal to or less than the predetermined threshold Th1 is the threshold Th2. It is determined whether or not the following is true (step ST5). If step ST5 is affirmed, the distance measuring light is determined as the light emitted by the currently set light emitting unit 41A (step ST6).

  If step ST5 is negative, it is determined whether or not the determination of step ST5 has been made for all the light emitting units (step ST7). If step ST7 is negative, the light emitting unit that emits light is changed to the next light emitting unit ( Here, 41B) is set (step ST8), the process returns to step ST2, and the processes after step ST2 are repeated.

  If step ST7 is affirmed, the areas of all the histograms where the luminance is equal to or less than the predetermined threshold value Th1 are compared, and the distance measurement light is the light used to capture the image that yielded the histogram with the smallest area. The light emitted from the light emitting unit (minimum area acquisition light emitting unit) is determined to be emitted (step ST9).

  Subsequent to steps ST6 and ST9, the CPU 42 starts monitoring whether or not the release button has been pressed (step ST10). When step ST10 is affirmed, the optical distance calculation unit 52 emits the determined light. The distance measuring light is emitted from the light emitting unit to measure the distance by the TOF method (step ST11), the optical object distance Lo is acquired (step ST12), and the process is terminated. The acquired subject distance Lo is recorded on the recording medium 35 as a distance image in association with each pixel.

  As described above, in the first embodiment, since distance measurement by the TOF method can be performed using distance measurement light that is less influenced by the environmental light, the influence of the environmental light is reduced when performing the distance measurement by the TOF method. As a result, the noise of the reflected light can be reduced and the subject distance can be accurately calculated.

  Next, a second embodiment of the present invention will be described. FIG. 7 is a schematic block diagram showing the configuration of a digital camera to which the photographing apparatus according to the second embodiment of the present invention is applied. Note that in the second embodiment, the same reference numerals are assigned to the same components as those in the first embodiment, and detailed description thereof is omitted. The digital camera 1A according to the second embodiment is different from the first embodiment in that a filter switching mechanism 60 is provided on the front surface of the light emitting units 41A and 41B. As described above, even if the filter switching mechanism 60 is provided on the front surface of the light emitting units 41A and 41B, the wavelength of light used for distance measurement by the TOF method can be determined as in the first embodiment.

  In the first embodiment, the light emitting units 41A and 41B that emit light having the center wavelengths of λ1 = 780 nm and λ2 = 850 nm are used. However, when two lights are emitted simultaneously, as shown in FIG. , Light having an intensity also in a wavelength region between 780 nm and 850 nm can be obtained. Therefore, as shown in FIG. 9, two types of light emitting units 41A and 41B are used by providing the filter switching mechanism 60 with a bandpass filter 62C that transmits a wavelength region between 780 nm and 850 nm (for example, λ3 = 820 nm). Thus, it is possible to determine whether or not to use light of three types of wavelengths for ranging.

  At this time, light is emitted only from the light emitting unit 41A, and the bandpass filter 62A is positioned on the optical axis of the imaging system 20, so that photographing is performed with light having a center wavelength of λ1 = 780 nm. Further, by emitting light only from the light emitting unit 41B and positioning the band pass filter 62B on the optical axis of the imaging system 20, photographing is performed with light having a center wavelength of λ2 = 850 nm. Further, light is emitted from both the light emitting units 41A and 41B, and the bandpass filter 62C is positioned on the optical axis of the imaging system 20, so that photographing is performed with light having a center wavelength of λ3 = 820 nm. Then, similarly to the first embodiment, light having a wavelength used for distance measurement is determined from the luminance histograms of the three images acquired by photographing.

  Moreover, in the said 1st and 2nd embodiment, although the two light emission parts 41A and 41B are used, you may make it use three or more light emission parts.

  In the first and second embodiments, two light emitting units 41A and 41B are provided, but only one light emitting unit capable of changing the wavelength of emitted light is used, and one light emitting unit is used. Light having different wavelengths may be emitted to determine light having a wavelength used for distance measurement.

  Although the digital camera according to the embodiment of the present invention has been described above, the computer functions as a unit corresponding to the intensity acquisition unit 50, the wavelength determination unit 51, and the optical distance calculation unit 52 as shown in FIG. A program for performing a simple process is also one embodiment of the present invention. A computer-readable recording medium in which such a program is recorded is also one embodiment of the present invention.

1 is a schematic block diagram showing the configuration of a digital camera to which a photographing apparatus according to a first embodiment of the present invention is applied. Diagram showing the configuration of the filter switching mechanism The figure which shows the emission spectrum of the light which a light emission part emits Diagram showing transmission characteristics of bandpass filter Diagram showing luminance histogram The flowchart which shows the process performed in 1st Embodiment. Schematic block diagram showing the configuration of a digital camera to which a photographing apparatus according to a second embodiment of the present invention is applied. Diagram for explaining superposition of two wavelength ranges The figure which shows the other structure of a filter switching mechanism

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,1A Digital camera 10 Operation system 20 Imaging system 21 Lens 23 Aperture 25 Shutter 27 Image sensor 38 Image processing part 40 Internal memory 41A, 41B Light emission part 42 CPU
DESCRIPTION OF SYMBOLS 50 Intensity acquisition part 51 Wavelength determination part 52 Optical distance calculation part 60 Filter switching mechanism

Claims (5)

Optics for irradiating a subject with light, detecting reflected light of the light from the subject, and calculating an optical subject distance of the subject based on a time from when the light is emitted until the reflected light is detected In an imaging device equipped with a target distance calculation means
Light irradiation means for irradiating the subject with a plurality of lights having different wavelengths;
A light receiving means for receiving the reflected light of the plurality of lights by the subject for each of the different wavelengths or for each predetermined wavelength in a band in which the different wavelengths and the different wavelengths overlap;
An imaging apparatus comprising: an intensity acquisition unit that acquires the intensity of the received reflected light. The light receiving means is means for acquiring a reflected light image that is an image of the received reflected light,
The intensity acquisition means is means for acquiring a histogram of luminance of the reflected light image as the intensity of the reflected light,
The apparatus further comprises wavelength determining means for determining light having a wavelength obtained from a reflected light image in which an area having a predetermined luminance or less in the histogram is equal to or less than a predetermined threshold as light for irradiating the subject. Item 1. The photographing apparatus according to Item 1. When there is no reflected light image in which the area equal to or smaller than the predetermined luminance in the histogram is equal to or smaller than a predetermined threshold, the intensity acquisition means is means for calculating a luminance histogram for each of the reflected light images;
3. The wavelength determining unit is a unit that determines, as light to irradiate the subject, light having a wavelength at which a reflected light image having the smallest area not exceeding a predetermined luminance in the plurality of histograms is obtained. The imaging device described. Optics for irradiating a subject with light, detecting reflected light of the light from the subject, and calculating an optical subject distance of the subject based on a time from when the light is emitted until the reflected light is detected In the photographing method in the photographing apparatus provided with the target distance calculating means,
Irradiating the subject with a plurality of lights having different wavelengths,
The reflected light from the subject of the plurality of lights is received for each different wavelength, or for each predetermined wavelength in a band where the different wavelengths and the different wavelengths overlap,
An imaging method comprising obtaining the intensity of the received reflected light. Optics for irradiating a subject with light, detecting reflected light of the light from the subject, and calculating an optical subject distance of the subject based on a time from when the light is emitted until the reflected light is detected In a program for causing a computer to execute a photographing method in a photographing apparatus provided with a target distance calculating means,
Irradiating the subject with a plurality of lights having different wavelengths;
Receiving the reflected light of the plurality of lights from the subject for each different wavelength, or for each predetermined wavelength in a band where the different wavelengths and the different wavelengths overlap;
A program for obtaining the intensity of the received reflected light.

Images