JP2008219370A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of simultaneously outputting a subject image and distance information up to a target subject during photographing and further eliminating an influence by infrared light in natural light. <P>SOLUTION: A CPU 8 controls a lens 1, a diaphragm 2, a CMOS sensor 3, AFE 4, multiplex 5, AGC 6, TG 7, camera signal processing circuit 9, a brightness level detecting part 12 and a memory 14 for camera data. A signal of the camera signal processing circuit 9 is outputted to a video signal processing circuit 10 and a monitor 11. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus such as a video camera or a digital still camera.

  In recent years, from the interest in automobile safety, calculate the distance from your vehicle to the vehicle in front, demand for driving with a safe driving distance, obstacles on the road in advance, The demand for a device that promotes avoidance has increased, and observation of a road with an imaging device has started from within a car.

  In addition, in order to realize a high-definition image and improve the accuracy of focusing and adjusting the amount of strobe light emission, there is an increasing demand for an imaging apparatus that can output image information and distance information simultaneously.

  Furthermore, in order to reduce the size of such an image pickup apparatus, there is a demand for a single image pickup element that can use incident light more effectively in place of a conventional color filter instead of a mosaic image pickup element (Bayer array).

  In view of this situation, each pixel unit of the solid-state image sensor improves the utilization efficiency of incident light by stacking electromagnetic wave absorbing films that absorb light of the three primary colors RGB and light of each wavelength. The technique to make is proposed (for example, refer patent document 1). Thereby, high resolution can be realized without increasing the size of the image sensor.

There has also been proposed an imaging apparatus that simultaneously acquires a high-resolution image and distance information to a subject using two imaging elements (see, for example, Patent Document 2).
JP 2003-234460 A Japanese Patent Laid-Open No. 11-41521

  As described above, in the method of realizing a high resolution by multilayering the electromagnetic wave absorbing films that absorb the respective wavelengths of RGB and RGB, which are the three primary colors of light for each pixel unit, the RGB wavelength of the light is set for each pixel unit. Can be absorbed. With this structure, the light use efficiency is improved by a factor of three compared to a mosaic color filter type solid-state imaging device, so that high resolution can be realized.

  However, in this method, since the electromagnetic wave absorbing film corresponding to the three primary colors RGB of light has a laminated structure, it is possible to achieve high resolution, but the image pickup device outputs only image information. The distance information to the subject cannot be output at the same time.

  Further, in the method of simultaneously acquiring high resolution image quality and distance information using two image sensors, the image sensor for image and the image sensor for distance information acquisition pass through a half mirror that separates incident light from one lens. Light is incident. Therefore, an image and distance information can be acquired with one optical device such as a lens.

  However, since the incident light is separated from one optical device via the half mirror, the mechanical attachment accuracy of the half mirror is required so that the incident angle of the light to the two image sensors does not deviate.

  In addition, since two imaging elements and a half mirror are used, there is a problem that it is difficult to reduce the size and power consumption of the imaging apparatus, and further increase the cost.

  An object of the present invention is to provide an imaging apparatus that can simultaneously output a subject image and distance information to the target subject during shooting, and can eliminate the influence of infrared light in natural light. is there.

  In order to achieve the above object, the imaging apparatus according to claim 1 includes first to third electromagnetic wave absorbing films that can absorb R, G, and B, which are the three primary colors of light, in the visible light, and the first electromagnetic wave absorbing film. A multilayer laminated type imaging device having a fourth electromagnetic wave absorbing film capable of absorbing electromagnetic waves corresponding to near-infrared light wavelength from electromagnetic waves not absorbed by the third electromagnetic wave absorbing film, and an LED installed in the vicinity of the imaging device In the imaging device, the subject image is calculated from the charges absorbed by the first to third electromagnetic wave absorbing films, and the reflected light from the LED through the subject is reflected by the fourth electromagnetic wave absorbing film. And calculating means for calculating distance information to the subject from the time difference between the irradiation light output timing and the arrival reflected light absorption timing, and output means for simultaneously outputting the subject image and the distance information corresponding to the subject image Characterized in that it comprises a.

  According to the present invention, it is possible to simultaneously output a subject image and distance information to the target subject during shooting, and further, it is possible to eliminate the influence of infrared light in natural light.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block configuration diagram of an imaging apparatus according to an embodiment of the present invention.

  The imaging apparatus includes an electromagnetic wave corresponding to a near-infrared light wavelength from first to third electromagnetic wave absorbing films that can absorb R, G, and B, which are the three primary colors of light, and an electromagnetic wave that is not absorbed by the electromagnetic wave absorbing film. A multilayer stacked type imaging device having a fourth electromagnetic wave absorbing film capable of absorbing. Moreover, it has LED installed in the image pick-up element vicinity. Each unit will be described later.

  In FIG. 1, the imaging apparatus includes the following components centering on a CPU (calculation unit, output unit, comparison unit, irradiation light emission control unit, synthesis unit, and recording unit component) 8. Hereinafter, the configuration will be described together with the operation.

  An optical signal (visible light) that has passed through the lens 1 is input to the CMOS sensor 3 via the diaphragm 2. The CMOS sensor 3 is driven by a timing pulse output from a TG (timing generator) 7 and converts an incident optical signal into an electric signal by photoelectric conversion and outputs the electric signal.

  The TG 7 is controlled by the CPU 8. For example, the TG 7 outputs a 3V amplitude timing pulse in synchronization with the horizontal synchronizing signal HD at a constant cycle.

  The output signal of the CMOS sensor 3 is composed of “S” indicating a signal level and “N” indicating a noise level, and “S−N” indicates a value obtained by converting incident light into an electric signal.

  The output signal of the CMOS sensor 3 is input to an AFE 4 composed of a CDS that performs correlated double sampling for calculating the difference value of “S−N” and an A / D converter that converts an analog signal into a digital signal. After being converted into a digital signal, it is input to the multiplex 5.

  The multiplex 5 synthesizes the charges converted from the electromagnetic waves absorbed by each layer in the CMOS sensor 3, and the output signal is input to the camera signal processing circuit 9 via the AGC 6.

  The camera signal processing circuit 9 performs YC separation, video signal processing, color signal processing, and gamma correction to generate an image.

  The output of the camera signal processing circuit 9 is input to the video signal processing circuit 10, and the video signal processing circuit 10 performs video signal processing and outputs a signal. The monitor 11 displays an image based on the signal output from the video signal processing circuit 10.

  Further, the camera signal processing circuit 9 outputs an evaluation value for performing exposure correction of the imaging apparatus to the luminance level detection unit 12.

  The luminance level detection unit 12 calculates a luminance level detection value from the evaluation value and outputs it to the CPU 8.

  The CPU 8 calculates a difference value of the exposure correction amount from the output signal from the luminance level detection unit 12 with reference to a previously provided exposure correction table, and if there is a difference, the diaphragm 2, the CMOS sensor 3, and the AGC 6 The control signal is output to the exposure compensation.

  A memory 13 is connected to the CPU 8. Further, a camera data memory 14 is connected to the camera signal processing circuit 9.

  Next, the charge readout configuration from the multilayer laminated film included in the CMOS sensor 3 will be described with reference to FIGS. 2, 3, and 4.

  FIG. 2 is a diagram showing the electromagnetic wave wavelength input to the CMOS sensor in FIG. 1 and the transmittance of each electromagnetic wave absorbing film. FIG. 3 is a configuration diagram of each electromagnetic wave absorption film and electrode of the CMOS sensor in FIG. FIG. 4 is a configuration diagram of the AFE in FIG.

  In FIG. 2, the input signal wavelength is an electromagnetic wave wavelength of 400 nm or more due to, for example, ultraviolet (UV) cut characteristics provided in an optical LPF (Low Pass Filter) inserted between the lens 1 and the CMOS sensor 3.

  Next, in the electromagnetic wave absorbing film having a multilayer structure that constitutes the CMOS sensor 3, the first electromagnetic wave absorbing film is the first electromagnetic wave absorbing film, and each of R, G, and B, which are the three primary colors of light, in visible light. It is formed of an organic film that can absorb water. Specifically, the first electromagnetic wave absorbing film is formed of an organic film that absorbs electromagnetic waves having an electromagnetic wave wavelength of 400 nm to 500 nm of incident light that is visible light.

  In FIG. 3, the first electromagnetic wave absorbing film is connected to an electrode 300 for converting electromagnetic waves absorbed by the absorbing film into electric charges and reading the converted electric charges.

  The electric charge read from the electrode 300 is output to the AFE 4 as DATA1 through the AMP 304.

  In FIG. 4, DATA1 is subjected to correlated double sampling and A / D conversion by CDS 400 inside AFE 4 and A / D 404.

  On the other hand, electromagnetic waves other than the electromagnetic wave absorbed by the first electromagnetic wave absorbing film pass through the first electromagnetic wave absorbing film and permeate the second electromagnetic wave absorbing film.

  The second electromagnetic wave absorbing film is formed of an organic film that can absorb R, G, and B, which are the three primary colors of light, in visible light. Specifically, the second electromagnetic wave absorbing film is formed of an organic film that absorbs an electromagnetic wave having an electromagnetic wave wavelength of 500 nm to 600 nm of incident light.

  Connected to the second electromagnetic wave absorbing film is an electrode 301 for converting electromagnetic waves absorbed by the absorbing film into electric charges and reading out the converted electric charges.

  The electric charge read from the electrode 301 is output to the AFE 4 as DATA 2 via the AMP 305.

  DATA2 is subjected to correlated double sampling and A / D conversion by CDS 401 inside AFE4 and A / D405.

  On the other hand, electromagnetic waves other than the electromagnetic wave absorbed by the second electromagnetic wave absorbing film pass through the second electromagnetic wave absorbing film and permeate the third electromagnetic wave absorbing film.

  The third electromagnetic wave absorbing film is formed of an organic film that can absorb R, G, and B, which are the three primary colors of light, in visible light. Specifically, the third electromagnetic wave absorbing film is formed of an organic film that absorbs an electromagnetic wave having an electromagnetic wave wavelength of 600 nm to 700 nm of incident light.

  The third electromagnetic wave absorbing film is connected to an electrode 302 for converting electromagnetic waves absorbed by the absorbing film into electric charges and reading out the converted electric charges.

  The electric charge read from the electrode 302 is output to the AFE 4 as DATA 3 via the AMP 306.

  DATA3 is subjected to correlated double sampling and A / D conversion by CDS 402 inside AFE and A / D 406.

  Next, a mechanism for calculating distance information to a subject using an LED arranged near the image sensor will be described with reference to FIGS. 3, 4, and 5.

  FIG. 5 is an explanatory diagram of a distance calculation method for calculating a distance using irradiation light and reflected light from an LED in the imaging apparatus of FIG.

  In FIG. 5, while photographing an object with the imaging apparatus 500, infrared light having a wavelength of 800 nm to 1200 nm is emitted from the LED 504 constituting the imaging unit 502 to the photographing subject 501 (irradiation light 505).

  The irradiation light 505 is applied to a person or object projected in the photographic subject 501, and when the person or object is at a distance that the irradiation light 505 reaches, the irradiation light 505 is reflected by the person or object. (Reflected light 506).

  The reflected light 506 is incident on the multi-layer image sensor 503 as an optical signal of the photographic subject 501.

  The reflected light 506 incident on the multilayer stacked type image pickup device 503 is a signal having a wavelength of 800 nm to 1200 nm, and therefore passes without being absorbed by the first to third electromagnetic wave absorbing films included in the image pickup device 503. It penetrates into the fourth electromagnetic wave absorbing film disposed in the lowermost layer.

  The fourth electromagnetic wave absorbing film is formed of an organic film that can absorb an electromagnetic wave corresponding to the near-infrared light wavelength from an electromagnetic wave that is not absorbed by the first to third electromagnetic wave absorbing films. Specifically, since it is formed of an organic film that absorbs electromagnetic waves having an electromagnetic wave wavelength of 800 nm to 1200 nm of incident light, the reflected light 506 is absorbed by the absorbing film and converted into electric charges.

  The converted charge is output to the AFE 4 as DATA 4 through the charge reading electrode 303.

  DATA 4 is subjected to correlated double sampling and A / D conversion by CDS 403 and A / D 407 inside AFE 4, and is output to multiplex 5 via AMP 307.

  Next, a method of calculating the distance to the subject with the light emitted from the LED and the reflected light from the subject will be described with reference to FIGS.

  FIG. 6 shows, for example, pixel information of an imaging element included in the imaging apparatus, and all pixels have address information.

  FIG. 7 shows the LED irradiation timing for acquiring distance information and the distance calculation method, and the LED irradiation starts within the V ranking period in the video signal.

  The pulse emitted from the LED is incident on each pixel of the image sensor 503 through the subject, and the arrival time of the reflected pulse is detected in each pixel.

  Here, when detecting the arrival time of the reflected pulse, the time difference is calculated by comparing the rising edge portion of the irradiation pulse with the rising edge portion of the reflected pulse.

  In FIG. 7, the reflected pulse first reaches the sensor 23 pixel and the sensor 33 pixel, and the time difference Φ1 is calculated by comparing the pulse with the irradiation pulse.

  Next, the reflected pulse arrives at the sensor 13 pixel, the sensor 22 pixel, and the sensor 32 pixel, and the time difference Φ2 is calculated by comparing the pulse with the irradiation pulse.

  Next, the reflected pulse reaches the sensor 12 pixel, and the pulse and the irradiation pulse are compared to calculate the time difference Φ3.

  Finally, since the distance to the subject is very far, the reflected pulse does not reach the sensor 11 pixel, the sensor 21 pixel, and the sensor 31 pixel.

  As described above, the arrival time of the reflected pulse is detected in all the pixels included in the image sensor 503, and the distance information to the subject is calculated in all the pixels from the arrival time.

  Here, when calculating the distance information from the detected arrival time, the flight time is converted from the arrival time of the reflected light, and the flight speed is multiplied by the speed of light.

  The readout frequency of the pixels included in the image sensor is as follows.

F = 1 / (X / A * C) (1)
C: Speed of light X: Distance detection accuracy A: Number of pixels of the image sensor The distance between all the pixels of the image sensor 503 is driven by driving the image sensor at the pixel readout frequency calculated by the equation (1). Information can be calculated.

  As described above, the subject image is output as charges from the first to third electromagnetic wave absorption films included in the image sensor 503, and the distance information to the subject is output as charges from the fourth electromagnetic wave absorption film included in the image sensor 503. Is done.

  Each output charge is converted into a digital signal via the AFE 4 and then input to the camera signal processing circuit 9 via the multiplex 5 and the AGC 6.

  The camera signal processing circuit 9 combines the subject image and the distance information to the subject, outputs them to the camera data memory 14 and records them as RAW data.

  According to the present embodiment, it is possible to simultaneously output the subject image and the distance information to the subject without affecting the subject image at the time of photographing by one solid-state image sensing device and the LED arranged near the image sensing device. A simple imaging device can be realized.

  Furthermore, since the subject image and the distance information to the subject can be acquired at the same time, it is possible to perform focusing for each distance using each distance information during subject photographing.

  In addition, since the distance information to the subject can be accurately calculated, it is possible to adjust the flash emission amount corresponding to the distance.

  In the above-described embodiment, the LED irradiation timing is generated in the video V blanking period using the multilayer stacked image sensor and the LED arranged near the image sensor, and the subject image and the distance information to the subject are simultaneously displayed. Explained the output technology.

  Hereinafter, means for generating the irradiation timing of the distance information acquisition LED within the video signal period will be described with reference to FIG.

  FIG. 8 shows an image generation, distance information generation, and LED irradiation timing when, for example, 300 frames are applied to a reading frame of the imaging element when an imaging element capable of high-speed reading is employed.

  The image sensor 503 has a charge storage for reading out 300 frames into 1 V (field) and a readout electrode structure. Charges corresponding to 300 frames of image information are output from the readout electrodes connected to the first to fourth electromagnetic wave absorbing films of the image sensor 503.

  The charges read from the first to fourth electromagnetic wave absorbing films are input to the camera signal processing circuit 9 via the AFE 4, the multiplex 5, and the AGC 6.

  The camera signal processing circuit 9 first generates an image from the input 300-frame signal using “1” to “60” frame data.

  Next, distance information is generated using “61” to “300” frame data not used for image generation.

  Here, in order not to affect the subject image, a signal is emitted from the LED 504 to the subject in the head of a frame not used for image generation, that is, in the “61” frame.

  The method for calculating the distance information at each pixel of the image sensor based on the LED irradiation signal reflected through the subject is the same as in the above embodiment.

  Further, in order to improve the accuracy of detecting the LED irradiation signal reflected through the subject by each pixel of the image sensor, 240 frames that are not used for image generation are divided into two, and the signal irradiation operation from the LED is “61”. This is done in the frame and the “181” frame. In this way, the distance information calculation operation may be performed twice.

  As a result, the subject image and the distance information can be acquired simultaneously without affecting the subject image at the time of shooting, and further, the erroneous detection of the signal is prevented by providing the timing of emitting the signal from the LED twice. The accuracy of calculating the distance information to the subject can be improved.

  The imaging apparatus according to the present embodiment calculates a subject image from the charges absorbed by the first to third electromagnetic wave absorbing films, and the irradiation light from the LED 504 converts the reflected light through the subject to the fourth electromagnetic wave absorbing film. To absorb. Then, distance information to the subject is calculated from the time difference between the irradiation light output timing and the arrival reflected light absorption timing. Also, the subject image and distance information corresponding to the subject image are output simultaneously.

  Further, in the imaging device of the present embodiment, a pulse signal is superimposed on the irradiation light from the LED 504, and the pulse signal is output from the fourth electromagnetic wave absorption film as reflected light through the subject. The edge portion of the pulse signal superimposed on the irradiation light is compared. Then, distance information to the subject is calculated from the comparison result.

  In the imaging device of this embodiment, the first electromagnetic wave absorption film has a characteristic of absorbing an electromagnetic wave of 400 nm to 500 nm, and the second electromagnetic wave absorption film has a characteristic of absorbing an electromagnetic wave of 500 nm to 600 nm. The third electromagnetic wave absorbing film has a characteristic of absorbing electromagnetic waves of 600 nm to 700 nm, and the fourth electromagnetic wave absorbing film has an absorption characteristic equivalent to the wavelength of the near infrared light LED.

  In the imaging device according to the present embodiment, the first to fourth electromagnetic wave absorption films include a readout electrode and a gain amplifier capable of gain control connected to the readout electrode.

  In the imaging apparatus of the present embodiment, the emission timing of the irradiation light from the LED 504 disposed adjacent to the imaging element 503 is within the video V blanking period.

  In the imaging apparatus according to the present embodiment, the CPU 8 serves as irradiation light emission control means for irradiating irradiation light from the LED 504 in a frame portion other than the frame used for image generation when the image sensor 503 is capable of high-speed frame reading. Function.

  In the imaging apparatus according to the present embodiment, the irradiation light emission control means performs LED emission control twice or more in a frame other than the frame used for image generation to prevent a malfunction in detecting distance information.

  In the imaging apparatus of the present embodiment, the CPU 8 combines a subject image output from the image sensor 503 and distance information to the subject, and a subject having the distance information synthesized by the combining unit. It has a function of recording means for recording an image as image data.

  In addition, the imaging device of this embodiment has address information in all the pixels included in the imaging element 503.

  In addition, the imaging apparatus according to the present embodiment performs strobe light emission amount adjustment and focusing using the image signal output from the imaging element 503 and distance information.

  The intended object can be achieved by the imaging apparatus having the above configuration.

It is a block block diagram of the imaging device which concerns on embodiment of this invention. It is a figure showing the electromagnetic wave wavelength input into the CMOS sensor in FIG. 1, and the transmittance | permeability of each electromagnetic wave absorption film. It is a block diagram of each electromagnetic wave absorption film and electrode of the CMOS sensor in FIG. It is a block diagram of AFE in FIG. FIG. 3 is an explanatory diagram of a distance calculation method for calculating a distance using irradiation light and reflected light from an LED in the imaging apparatus of FIG. 1. It is a figure showing the positional information on all the pixels of the image pick-up element with which the imaging device of FIG. 1 is provided. 2 is a timing chart showing the timing of irradiating an LED within a video V blanking period and the distance information calculation method for each pixel in the imaging apparatus of FIG. 1. 2 is a timing chart illustrating a method for calculating distance information to a subject while preventing influence on a subject image in the imaging apparatus of FIG. 1.

Explanation of symbols

1 Lens 2 Aperture 3 CMOS sensor 4 AFE
5 Multiplex 6 AGC
7 TG
8 CPU
DESCRIPTION OF SYMBOLS 9 Camera signal processing circuit 10 Video signal processing circuit 11 Monitor 12 Brightness level detection part 13 Memory 14 Memory for camera data 300 Electrode 301 Electrode 302 Electrode 303 Electrode 304 AMP
305 AMP
306 AMP
307 AMP
400 CDS
401 CDS
402 CDS
403 CDS
404 A / D
405 A / D
406 A / D
407 A / D
500 Imaging Device 501 Shooting Subject 502 Imaging Unit 503 Imaging Element 504 LED
505 Irradiation light 506 Reflected light

Claims (10)

In visible light, near-infrared light is obtained from first to third electromagnetic wave absorbing films that can absorb R, G, and B, which are the three primary colors of light, and electromagnetic waves that are not absorbed by the first to third electromagnetic wave absorbing films. In an imaging device having a multilayer laminated type imaging device having a fourth electromagnetic wave absorbing film capable of absorbing electromagnetic waves corresponding to the wavelength, and an LED installed in the vicinity of the imaging device,
A subject image is calculated from the charges absorbed by the first to third electromagnetic wave absorption films, and the reflected light from the LED is absorbed by the fourth electromagnetic wave absorption film by the fourth electromagnetic wave absorption film. A calculation means for calculating distance information to the subject from a time difference between the output timing and the arrival reflected light absorption timing;
Output means for simultaneously outputting a subject image and distance information corresponding to the subject image;
An imaging apparatus comprising: A pulse signal is superimposed on the irradiation light from the LED, and the edge of the pulse signal superimposed on the irradiation light and the edge portion of the signal output from the fourth electromagnetic wave absorbing film as the reflected light through the subject. Comparing means for comparing parts,
The imaging apparatus according to claim 1, wherein the calculation unit calculates distance information to a subject from a comparison result of the comparison unit.   The first electromagnetic wave absorbing film has a characteristic of absorbing an electromagnetic wave of 400 nm to 500 nm, the second electromagnetic wave absorbing film has a characteristic of absorbing an electromagnetic wave of 500 nm to 600 nm, and the third electromagnetic wave absorbing film is 2. The imaging apparatus according to claim 1, wherein the imaging device has a characteristic of absorbing electromagnetic waves of 600 nm to 700 nm, and further, the fourth electromagnetic wave absorbing film has absorption characteristics of wavelengths of near-infrared light LEDs.   The imaging apparatus according to claim 1, wherein each of the first to fourth electromagnetic wave absorbing films includes a readout electrode and a gain amplifier capable of gain control connected to the readout electrode.   The imaging apparatus according to claim 1, wherein an emission timing of irradiation light from the LED arranged adjacent to the imaging element is within a video V blanking period.   2. The imaging apparatus according to claim 1, further comprising irradiation light emission control means for irradiating irradiation light from the LED in a frame portion other than a frame used for image generation when the imaging element is capable of high-speed frame reading. .   The imaging apparatus according to claim 6, wherein the irradiation light emission control unit performs LED light emission control twice or more in a frame other than a frame used for image generation to prevent a malfunction in detecting distance information. A synthesizing unit that synthesizes the subject image output simultaneously from the image sensor and the distance information to the subject;
Recording means for recording the subject image having the distance information synthesized by the synthesizing means as image data;
The imaging apparatus according to claim 1, further comprising:   The image pickup apparatus according to claim 1, wherein all pixels included in the image pickup element have address information.   The image pickup apparatus according to claim 1, wherein strobe light emission amount adjustment and focusing are performed using an image signal output from the image pickup device and distance information.

Images