JP5161752B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging device, and more particularly to an imaging device for capturing an image with reduced influence of ambient light.

The present applicant irradiates a target area with modulated light obtained by high-speed modulation of infrared light emitted from a light emitting diode, distinguishes the reflected light component of the modulated light received by the image sensor from the reflected light component of ambient light, and modulates the light. An imaging apparatus that generates an image (light modulation image) having only a reflected light component as a pixel value has already been proposed (see Patent Document 1). By using such an imaging apparatus, an image with reduced influence of ambient light can be taken.
JP 2006-121617 A

  By the way, the imaging device used in the conventional example of Patent Document 1 is slightly different in structure from a general solid-state imaging device (CCD image sensor, CMOS image sensor, etc.) used in a digital camera or the like. Specifically, a plurality of control electrodes are provided for one light receiving unit, and the sensitivity of the light receiving unit can be adjusted by changing the number of control electrodes to which the control voltage is applied. Therefore, the image sensor in Patent Document 1 has a relatively low resolution when compared with the same size because the number of pixels is smaller than that of a general image sensor.

  On the other hand, using a general imaging device, an image when modulated light is irradiated and an image when modulated light is not irradiated are alternately captured, and a light modulated image is generated from the difference between the pixel values of both images In this case, the time lag for capturing two types of images is limited by the frame rate of the image sensor, and the subject image may be blurred when a fast-moving subject is imaged.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an imaging apparatus that can reduce the influence of ambient light, achieve both miniaturization and high resolution, and suppress blurring of a subject image. It is in.

  In order to achieve the above object, the first aspect of the present invention provides an imaging device having a large number of light receiving portions that perform photoelectric conversion, a light irradiation unit that irradiates light in a target space, and a light irradiation unit that emits light in the target space. A difference image between the first image captured by the image sensor when irradiating the light and the second image captured by the image sensor when the light irradiating means does not irradiate the target space is generated. A difference image generating means, wherein the difference image generating means includes a first image formed by a charge converted by a part of the light receiving portions of the plurality of light receiving portions and the one of the plurality of light receiving portions. A difference image from the second image formed by the charges converted by the remaining light receiving portions excluding the light receiving portions is generated.

  According to the first aspect of the present invention, a general image pickup device can be used as the image pickup device instead of an image pickup device having a special structure as in the conventional example. In addition, since the photoelectric conversion is performed by a part of the light receiving units to capture the first image and the photoelectric conversion is performed by the remaining light receiving units to capture the second image, the first image is obtained. Since the imaging interval between the first image and the second image is shorter than the frame rate of the image sensor, blurring of the subject image can be suppressed.

  According to a second aspect of the present invention, in the first aspect of the invention, the difference image generating means includes the part of the plurality of light receiving units arranged side by side along the vertical direction and the horizontal direction. A first image formed by the charge converted by the light receiving unit, and a second image formed by the charge converted by the remaining light receiving unit sandwiched between the partial light receiving units adjacent to each other. A difference image from the image is generated.

  According to the second aspect of the present invention, the first image and the second image can be captured with the same image quality.

  According to a third aspect of the present invention, in the first or second aspect of the present invention, the light irradiating means irradiates the target space with light in synchronization with the time when the partial light receiving units perform photoelectric conversion, and the remaining light receiving units. Is characterized in that no light is irradiated during the period of photoelectric conversion.

  According to the present invention, it is possible to reduce the influence of ambient light, achieve both miniaturization and high resolution, and suppress blurring of the subject image.

  As shown in FIG. 1, the imaging apparatus according to the present embodiment includes an imaging element 1 that images a target space, a light irradiation unit 2 that irradiates light to the target space, and a light irradiation unit 2 that irradiates light to the target space. The difference which produces | generates the difference image of the 1st image imaged with the image pick-up element 1, and the 2nd image imaged with the image pick-up element 1 when the light irradiation part 2 is not irradiating light to object space The image generation means 3 and the control part 4 which controls the image pick-up element 1 and the light irradiation part 2 are provided.

  The light irradiation unit 2 includes a light emitting diode that emits infrared light, and an LED drive circuit that emits light by supplying direct current power to the light emitting diode (none of which is shown), and the controller 4 drives the LED. The operation of the circuit is controlled.

  The image sensor 1 is composed of a general-purpose two-dimensional CMOS image sensor having a global shutter function, for example. FIG. 2A shows an equivalent circuit of one pixel of the image sensor 1, and a light receiving unit (photodiode) 10 that performs photoelectric conversion, a reset switch 11 that discharges (resets) accumulated charge in the light receiving unit 10, An amplifier 12 that amplifies charges (electrons) photoelectrically converted by the light receiving unit 10, a global shutter unit 13 provided at a subsequent stage of the amplifier 12, and a charge for transferring the charges amplified by the amplifier 12 to the signal line 15 in the vertical column. A horizontal transfer switch 14 is provided. Note that the charges transferred to the signal lines 15 in the vertical column are read out through the vertical transfer switch 16.

  Further, the global shutter unit 13 stores in the storage element 13a, a storage element (capacitor) 13a for storing the charge of the light receiving unit 10, a shutter switch 13b that opens and closes a charge transfer path from the light receiving unit 10 to the storage element 13a. And an amplifier 13c that outputs pixel information (pixel value) corresponding to the charged charges to the signal line 15. That is, when the shutter switch 13b is opened and closed while the reset switch 11 and the horizontal transfer switch 14 are opened (turned off), the charge photoelectrically converted by the light receiving unit 10 during the period when the shutter switch 13b is closed (turned on). Is stored in the storage element 13a via the amplifier 12 and the shutter switch 13b. When the horizontal transfer switch 14 is closed (turned on) after the shutter switch 13b is opened (turned off), the shutter switch 13b is turned on (exposure time). Accordingly, the pixel information having the charge amount corresponding to the signal information 15 is transferred to the signal line 15. However, since the two-dimensional CMOS image sensor having such a global shutter function is well known in the art, illustration and description of the detailed configuration are omitted.

  The control unit 4 includes a microcomputer as a main component, controls the light irradiation unit 2 to periodically irradiate light (infrared light) on the target space, and the light irradiation unit 2 irradiates light as described later. The image sensor 1 is caused to capture images (first image and second image) during a period during which the light irradiation unit 2 does not irradiate light (a light non-irradiation period).

  The difference image generation unit 3 includes a first image storage unit 30 that stores a first image captured by the image sensor 1, and a second image storage unit 31 that stores a second image captured by the image sensor 1. The first interpolation processing unit 32 that performs interpolation processing (described later) on the first image, the second interpolation processing unit 33 that similarly executes interpolation processing on the second image, and the first interpolation processing A difference processing unit that calculates a difference between pixel values of each pixel in the first image interpolated by the unit 32 and the second image interpolated by the second interpolation processing unit 33, and generates a difference image. ing. The units 30 to 34 of the difference image generating unit 3 can be realized by causing a microcomputer to execute a dedicated program or using a general-purpose image processing LSI.

  Next, the operation of the imaging apparatus according to the present embodiment will be described. Here, the image pickup device 1 is provided with a large number of light receiving units 10 and reset switches 11, amplifiers 12, global shutter units 13, and amplifiers 14 corresponding to the respective light receiving units 10 arranged vertically and horizontally. In the following description, FIG. As shown in (b), a group of a plurality of light receiving units 10A arranged in a line along the vertical and horizontal directions is defined as a first light receiving unit group, and is sandwiched between light receiving units 10A belonging to the first light receiving unit group. A group of the plurality of light receiving units 10B is defined as a second light receiving unit group. That is, the light receiving units 10A belonging to the first light receiving unit group and the light receiving units 10B belonging to the second light receiving unit group are arranged in a checkered pattern.

  First, the control unit 4 closes the reset switch 11 to reset the light receiving unit 10A belonging to the first light receiving unit group, and then closes the shutter switch 13 for a predetermined exposure time so as to belong to the first light receiving unit group. The charge photoelectrically converted by 10A is stored in the storage element 13a of the global shutter unit 13. Further, the control unit 4 controls the light irradiation unit 2 to irradiate the target space with light (infrared light) in synchronization with the exposure time of the light receiving unit 10A (see FIGS. 3A and 3C).

  Subsequently, the control unit 4 closes the reset switch 11 to reset the light receiving unit B belonging to the second light receiving unit group, and then closes the shutter switch 13 for a predetermined exposure time to thereby receive the light received in the second light receiving unit group. The charge photoelectrically converted by the unit 10B is stored in the storage element 13a of the global shutter unit 13. At this time, the control unit 4 does not irradiate the target space with light (infrared light) during the exposure time of the light receiving unit 10B by controlling the light irradiation unit 2 (see FIGS. 3B and 3C).

  Then, the control unit 4 performs switching control of the horizontal transfer switch 14 and the vertical transfer switch 16 inserted in the signal line 15 of the vertical column, thereby forming the pixel information read from the light receiving unit 10A of the first light receiving unit group. The image (first image) to be stored is stored in the first image storage unit 30 of the difference image generating means 3, and the image (second image) formed by the pixel information read from the light receiving unit 10B of the second light receiving unit group Are stored in the second image storage unit 31 of the difference image generating means 3.

  The first interpolation processing unit 32 calculates the pixel value of the pixel corresponding to the light receiving unit 10B belonging to the second light receiving unit group in the first image stored in the first image storage unit 30, for example, Interpolation is performed with the average value of the pixel values of a total of four pixels (pixels corresponding to the light receiving unit 10A) adjacent to the pixels in the vertical and horizontal directions. Then, the first interpolation processing unit 32 outputs the first image obtained by performing the above interpolation on the pixels corresponding to all the light receiving units 10B belonging to the second light receiving unit group to the difference processing unit 34. To do.

  Similarly, the second interpolation processing unit 33 calculates the pixel value of the pixel corresponding to the light receiving unit 10A belonging to the first light receiving unit group among the second images stored in the second image storage unit 31, for example, Interpolation is performed with the average value of the pixel values of a total of four pixels (pixels corresponding to the light receiving unit 10B) adjacent to the pixel in the vertical and horizontal directions. Then, the second interpolation processing unit 33 outputs the second image obtained by performing the above interpolation on the pixels corresponding to all the light receiving units 10A belonging to the first light receiving unit group to the difference processing unit 34. To do.

  The difference processing unit 34 calculates the pixel value of each pixel of the first image output from the first interpolation processing unit 32 and the pixel value of each pixel of the second image output from the second interpolation processing unit 33. A difference is obtained, and a difference image is generated using the difference as the pixel value of the pixel. Here, the first image is an image including both the reflected light component of the infrared light and the reflected light component of the ambient light emitted from the light irradiation unit 2, and the second image includes only the reflected light component of the ambient light. Therefore, the difference image between the two images includes only the reflected light component of the infrared light irradiated from the light irradiation unit 2 and is an image in which the influence of ambient light is reduced.

  Here, when the first image and the second image are captured by all the light receiving units 10 without grouping the light receiving units 10 as described above, the first image is captured as shown in FIG. Since a time lag Tx ′ corresponding to the frame rate occurs between the imaging time T1 and the imaging time T2 for capturing the second image, when a fast-moving subject (for example, a moving car or the like) is captured. In some cases, the difference image may be blurred.

  However, in the imaging apparatus according to the present embodiment, the plurality of light receiving units 10 are divided into two groups, the first image is captured by one group (first light receiving unit group), and the other group (second light receiving unit). Group), the time lag Tx (<< Tx ′) between the first image and the second image is very short (see FIGS. 3A and 3B), and the movement is fast. There is an advantage that the difference image is less likely to be shaken when the subject is imaged.

  Note that the imaging apparatus according to the present embodiment is suitable for use in, for example, a face image input device or a monitoring camera in a face authentication apparatus, particularly for applications in which an object is imaged outdoors in sunlight.

It is a block diagram which shows embodiment of this invention. (A) is a principal part block diagram of the image pick-up element same as the above, (b) is explanatory drawing explaining arrangement | positioning of a light-receiving part. It is a timing chart for operation | movement description same as the above. It is operation | movement explanatory drawing same as the above.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up element 2 Light irradiation part (light irradiation means)
3 Difference image generation means

Claims (3)

An image pickup device having a large number of light-receiving portions that perform photoelectric conversion, a light irradiation unit that irradiates light to the target space, and a first image picked up by the image pickup device when the light irradiation unit irradiates light to the target space A difference image generation means for generating a difference image between the image and the second image captured by the image sensor when the light irradiation means is not irradiating the target space with light,
The difference image generating means includes a first image formed by charges converted by some of the light receiving units of the plurality of light receiving units, and the rest of the plurality of light receiving units excluding the part of the light receiving units. An image pickup apparatus that generates a difference image from a second image formed by charges converted by a light receiving unit.   The difference image generating means is a first formed by charges converted by the partial light receiving units arranged in a row in the vertical direction and the horizontal direction among a plurality of light receiving units arranged in parallel in the vertical and horizontal directions. A difference image between the image and a second image formed by charges converted by the remaining light receiving units sandwiched between the partial light receiving units adjacent to each other is generated. Item 2. The imaging device according to Item 1.   The light irradiating means irradiates the target space with light in synchronization with the time when the partial light receiving units perform photoelectric conversion, and does not irradiate light during the period during which the remaining light receiving units perform photoelectric conversion. The imaging device according to claim 1 or 2.

Images