JP2003169259A - Imaging unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging unit employing a CMOS image sensor capable of photographing a photographing image without deterioration in the resolution of a low luminance part and a high luminance part of an object whose luminance distribution is extended over a wide range. <P>SOLUTION: The CMOS image sensor is driven for long time exposure and short time exposure to acquire two kinds of photographing images, and a memory once stores either of them. In the case of reading the other photographing image, the imaging image signal is read from the memory and mixing the two imaging image signals obtains an imaging signal with a wide dynamic range. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus for recording an image having a wide dynamic range using a CMOS image sensor.

[0002]

2. Description of the Related Art Conventionally, in an image pickup device for converting an optical image into an electric signal for recording, a CCD image sensor is often used in a moving image pickup device. Recently, from the viewpoint of power saving, a CMOS image sensor that consumes less power than a CCD image sensor is increasingly used in mobile devices and the like. When shooting with an image pickup device using these image pickup devices, the brightness of the subject is measured from the image pickup signal, and the exposure time of the image pickup device is adjusted so that the signal level of the subject is appropriate. The point was optimized.

[0003]

When the brightness distribution of the object to be imaged is wide, the brightness of the object is measured from the image pickup signal so that the signal level of the object becomes appropriate. When the technique of optimizing the operating point of the image pickup device by adjusting the exposure time is used, the following problem occurs. For example, if the exposure time is adjusted so that the signal level of the high-luminance portion of the subject becomes appropriate, the signal of the low-luminance portion will be destroyed, and the resolution of the low-luminance portion will not be obtained. At this time, the signal read from the image pickup device also becomes weak, so that the S / N also deteriorates. Conversely, if the exposure time is adjusted so that the signal level in the low-brightness part becomes appropriate, the signal in the high-brightness part will be saturated and the image in that part will be in a single white color, making it difficult to distinguish the subject. . Therefore, it is an object of the present invention to provide an imaging device using a CMOS image sensor capable of capturing a captured image in which the resolution of a low-luminance portion and a high-luminance portion of a subject having a wide luminance distribution is not degraded. To do.

[0004]

[Means for Solving the Problems] In order to solve the above problems,
The present invention relates to a CMOS image sensor that converts an optical image into electrical information, a CDS / A / D conversion unit that samples electrical information read from the CMOS image sensor and converts the electrical information into a digital imaging signal, and a digital imaging signal. RGB processing means for processing and converting into image data expressed in RGB, memory means for recording RGB image data, RGB image data stored in the memory means, and RGB newly generated by the RGB processing means. An image pickup device is configured by a Y / C processing / mixing unit that synthesizes image data to generate new RGB image data and generates a luminance signal and a color difference signal from the new RGB image data. The following processing is performed using. First,
The CMOS image sensor performs a process of capturing image data acquired by short-time exposure and image data acquired by long-time exposure. Then, one of the image data is read from the CMOS image sensor and stored in the field memory means as first image data. Then, the other image data having an exposure time different from that of the first image data is read out as second image data from the CMOS image sensor, and at the same time, the first image data is read out to the field memory means, and the Y / C processing / mixing means slices each of the first image data and the second image data having different exposure times at a certain appropriate signal level, and the high-luminance side and the long time of the stepwise short-time exposure image data By connecting the low-luminance sides of the exposure image data, an image signal with a wide dynamic range having high resolution is generated in both the high-luminance portion and the low-luminance portion. Then, the generated image signal of the wide dynamic range is recorded by the recording means.

In the present invention, since the CMOS image sensor is used, it is possible to obtain signals with different exposure times by driving different from CCD.
Between the readout of the first photodiode of the CMOS image sensor and the readout of the next second photodiode, the readout processing of the photodiode of the coordinates offset by a certain amount from the coordinates of the first photodiode is performed. By doing so, the CMOS image sensor is driven at a normal double speed to generate two types of image data having different exposure times. Alternatively, the photodiode of the CMOS image sensor is provided with two independent signal reading ports, and two reading processes are performed so that the time is shifted from the two signal reading ports to an arbitrary photodiode. The configuration is such that image data for time exposure and image data for short exposure are generated.

[0006]

FIG. 1 is a block diagram showing the configuration of a wide dynamic range CMOS image sensor imaging device as an example of an embodiment of the present invention.
In the figure, 11 is a CM for converting an optical image into electrical information.
OS image sensor, 12 is timing generation means, 13
Is a CDS (double sampling) / A / D conversion means for sampling electrical information read from the CMOS image sensor 11 and converting it into a digital image pickup signal, 14
Is an RGB processing means for processing a digital image pickup signal and converting it into RGB image data; 15 is a field memory means;
Reference numeral 16 synthesizes the RGB image data stored in the field memory 15 and the RGB image data newly generated by the RGB processing means 14 to generate new RGB image data, and a luminance signal Y is generated from the new RGB image data. Y / C processing mixing means for generating a color difference signal C, and 17 for a luminance signal Y
And 18 is an image signal recording means for recording the color difference signal C, and 18 is a display system signal processing means for converting the luminance signal Y and the color difference signal C into a signal format that can be used by the monitoring device. 2 shows an example of the configuration of the CMOS image sensor 11, 21 is a read gate drive control means, 22 is a reset gate drive control means, 23 is a photodiode, 2
Reference numeral 4 indicates a read gate switch, and 25 indicates a reset gate switch. The operation of this embodiment will be described below with reference to FIGS. 1 and 2.

First, the CMOS image sensor 11
At, the optical image is converted into an electrical signal to generate an image pickup signal. The image pickup signal is obtained by short-time exposure of the CMOS image sensor and by long-time exposure. Here, a procedure of converting an optical image into an electrical signal and reading it as an image pickup signal in the CMOS image sensor 11 will be briefly described. The conversion of the optical image into an electric signal is performed by the photodiode 23 arranged on the grid of m rows × n columns of the CMOS image sensor 11. The photodiode 23 arranged on the grid is connected to a read gate switch 24 opened and closed by the read gate drive control means 21 and 22 and a reset gate switch 25 opened and closed by the reset gate drive control means 21 and 22. . The read gate switch 24 and the reset gate switch 25 are independently opened / closed by the two gate opening / closing means of the read gate drive control means 21 and the reset gate drive control means 22.
The output of the read gate switch is connected to the second-stage gate switch provided for each row, and the output of the second-stage gate switch is connected to the signal output terminal.

Now, when an optical image is formed on the plane formed by the photodiodes 23, each photodiode 2
3 is excited by the light energy of the optical image and charges. Here, since the amount of electric charge to be charged is almost proportional to the amount of light energy applied to each photodiode, the time when each photodiode is exposed to the optical image, that is, the exposure time is all equal. At this time, the distribution of the charges charged in the photodiodes arranged in the CMOS image sensor reflects the brightness distribution of the optical image formed on the CMOS image sensor. In this state, when the read gate switch in the 1st row and 1st column and the gate switch in the 2nd stage provided in the 1st row are opened (hereinafter, this operation is referred to as read access), the charge accumulated in the photodiode is The light energy discharged through the signal output terminal and applied to the photodiode in the first row and the first column is expressed as a voltage change. On the contrary, when the reset gate switch in the 1st row and 1st column is opened (hereinafter, this operation is referred to as reset access), the charge accumulated in the photodiode is discharged to the outside and the charge accumulation is reset. The read access is performed by scanning the read access in the order of, for example, the 1st row and 1st column, the 1st row and 2nd column ... 1st row and nth column, the 2nd row and 1st column ... i row, kth column, and the mth row and nth column. Then, the image pickup signal of the subject is acquired. At this time, if this scanning operation is repeatedly performed at a constant cycle (for example, a field cycle), if attention is paid to a certain arbitrary photodiode, light energy is generated by the photodiode from one read to the next read. The charge is accumulated, and the scanning cycle is the exposure time. Further, when the reset access scanning operation is performed prior to the above read access scanning operation, the period corresponding to the shaded portion of the exposure time shown in FIG. 5 (a diagram showing read access timing and reset access timing in an arbitrary photodiode) Since the charge accumulated in the photodiode is discharged through the read gate switch, the charge accumulated during the delay time from the reset access timing to the read access timing (hereinafter this time is referred to as offset time) is It will be read out as a signal. By adjusting this offset time, an image pickup signal taken with an exposure time shorter than the scanning cycle is acquired. Further, here, as shown in FIG. 6 (a diagram showing read access timing and reset access timing when the reset access in FIG. 5 is paused every other field), when the reset access corresponding to the read access in the field B is paused, The exposure time of the imaging signal read in the field A is short in the field B and shorter than the field B.
The exposure time of the image pickup signal read in (2) becomes long (more than the field period). As a result, the imaging signal read from the CMOS image sensor switches between long-time exposure and short-time exposure every other field, and a short-time exposure imaging signal and a long-time exposure imaging signal are obtained.

Both the short-time exposure image pickup signal and the long-time exposure image pickup signal are converted into digital image pickup signals after being sampled by the CDS / A / D conversion means 13. The digital image pickup signal is separated into RGB components by the RGB processing means 14, and an RGB digital signal is generated. Here, the RGB digital signals are alternately switched for each field between a long-time exposure imaged image and a short-time exposure imaged image, and one of them is switched to the field memory means 1.
Store in 5. The other RGB digital signal is Y /
It is supplied to the C treatment mixing means 16. At the same time, the RGB digital signals stored in the previous field in the field memory means are read out and supplied also to the Y / C processing mixing means 16. The Y / C processing mixing means 16 slices these two RGB signals having different exposure times at a certain arbitrary signal level to obtain a low brightness side of the RGB digital signal of the long exposure and a high RGB digital signal of the short exposure. Connect the brightness side. FIG. 7 is a diagram showing how the long-exposure image signal and the short-exposure image signal are combined, and it is assumed that the luminance distribution of the subject on a certain line is in the state shown in FIG. 7A. At this time, it is assumed that the signal level distribution of the short-time exposure image signal is B as shown in FIG. 7, and the signal level distribution of the long-time exposure image signal is as shown in C of FIG.
At this time, it can be seen that the long-exposure image signal is saturated in the high-luminance portion of the subject and does not faithfully represent the subject's luminance distribution. Therefore, the image signal of the long-time exposure is sliced at a level slightly lower than the saturation level, and the signal component corresponding to the portion higher than the saturation level is cut out from the image signal of the short-time exposure and multiplied by an arbitrary coefficient. An image signal that faithfully represents the luminance distribution of the original subject is added to the time exposure signal to synthesize it. As a result, a mixed image signal of long-time exposure and short-time exposure as shown in D of FIG. 7 is obtained. The RGB signals thus obtained are further converted into a luminance signal Y and a color difference signal C by the Y / C processing mixing means 16.
Then, the luminance signal Y and the color difference signal C are supplied to the image signal recording means 17 and the display system signal processing means 18, and the image signal recording means 17 records the captured image, and the display system signal processing means 18 to the monitoring means. Converted to a signal format suitable for delivery. The above is the operation of the wide dynamic range CMOS image sensor imaging device shown in FIG.

Next, in the wide dynamic range CMOS image sensor image pickup device shown in FIG. 1, a method for obtaining long-time exposure image pickup data and short-time exposure image pickup data by driving the CMOS image sensor at double speed will be described. I do. FIG. 8 shows a signal read pattern when normal reading is performed and a signal read pattern when double speed reading is performed. As shown in FIG. 8, charges are read from the CMOS image sensor at a normal double speed and at a double speed, and the read pattern is located at a position where the read access is performed in the row direction from the normal read position in the row direction and in the column direction. By reading out the charges by offsetting the b columns, as shown in FIG. 9 (a diagram showing read access timing focusing on an arbitrary photodiode at the time of double-speed reading), an arbitrary photodiode is read. Therefore, the accumulated charges having different exposure times are read out. As a result, it is possible to acquire two image data having different exposure times in one field period.

FIG. 3 shows an example of the configuration of a wide dynamic range image pickup device using a two-line readout type CMOS image sensor. In the figure, 31 is a two-line readout type CMOS image sensor, and 32 is a CDS / A /
D means 2 and 33 represent the RGB processing means 2, and the rest is the same as the configuration of the wide dynamic range imaging device using the CMOS image sensor of FIG. Further, FIG. 4 shows a configuration of the two-line readout type CMOS image sensor of FIG. The difference from the CMOS image sensor of FIG. 2 is that the reset charge discharging unit is changed to the signal output unit 2. After making this change, the newly installed 22 is C
The DS / A / D units 2 and 23 use the RGB processing unit 2 to convert the signal read from the signal output unit 2 into an RGB signal, and obtain R from the image pickup signal read from the signal output unit 1.
By performing the process of connecting GB signals, as shown in the example of FIG.
The same effect as when the CMOS image sensor is read at double speed can be obtained.

[0012]

According to the present invention, it is possible to obtain both long-time exposure image data and short-time exposure image data from a CMOS image sensor, and by properly connecting them, the CMOS image sensor originally has it. It is possible to capture a captured image having a luminance distribution that is equal to or greater than the existing dynamic range.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of the configuration of a wide dynamic range CMOS image sensor imaging device.

2 is a configuration example of the CMOS image sensor in FIG.

FIG. 3 is a block diagram showing an example of a configuration of a wide dynamic range imaging device using a 2-line readout type CMOS image sensor.

4 is a configuration example of the two-line read-out type CMOS image sensor of FIG.

FIG. 5 is a diagram showing read access timing and reset access timing in an arbitrary photodiode.

FIG. 6 is a diagram showing read access timing and reset access timing when the reset access in FIG. 5 is suspended every other field.

FIG. 7 is a diagram showing how the long-time exposure image signal and the short-time exposure image signal are combined.

FIG. 8 is a diagram comparing read patterns of normal read and double speed read.

FIG. 9 is a diagram showing read access timing focusing on an arbitrary photodiode during double speed reading.

[Explanation of symbols]

11 CMOS image sensor 12 Timing generation means 13 CDS / A / D conversion means 14 RGB processing means 15 Field memory means 16 Y / C treatment mixing means 17 Image signal recording means 18 Display system signal processing means 21 Read Gate Drive Control Means 22 Reset gate drive control means 23 Photodiode 24 reed gate switch 24 Reset gate switch 31 2-line readout type CMOS image sensor 32 CDS / A / D means 2 33 RGB processing means 2

─────────────────────────────────────────────────── ─── Continued Front Page (51) Int.Cl. ⁷ Identification Code FI Theme Coat (Reference) H04N 9/64 H01L 27/14 D (72) Inventor Toshiro Kinugasa 292 Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa F-Term (Reference) 4M118 AA02 AB01 BA14 CA02 DD09 DD12 FA06 GC08 5C024 AX01 CX43 CX47 GY31 GZ41 HX23 HX28 HX55 5C065 BB48 CC01 DD15 GG18 GG21 GG27 GG44 5C066 A0701A11A11A11A11A11A11A11A11

Claims (3)

[Claims] 1. A CMOS image sensor for converting an optical image into electrical information, and a CD for sampling electrical information read from the CMOS image sensor and converting it into a digital image pickup signal.
S / A / D conversion means, RGB processing means for processing the digital image pickup signal and converting it into image data expressed in RGB, and memory means for storing the RGB image data output from the RGB processing means, RGB image data and the RGB stored in the memory means
The RGB image data newly generated by the processing means is combined to generate new RGB image data, and a new RG is generated.
Y / C for generating a luminance signal and a color difference signal from B image data
In the image pickup apparatus including a processing / mixing unit, the CMOS image sensor includes long-time exposure image data obtained by performing long-time exposure, and short-time exposure obtained by performing short-time exposure by the CMOS image sensor. Image data is generated, the memory means stores the image data of either long-time exposure or short-time exposure, and the YC processing / mixing means compares the image data stored in the memory means with the exposure time. Of the image data stored in the memory means in accordance with the generation timings of the different image data, thereby synthesizing the short-time exposure image data and the long-time exposure image data. . 2. Between the readout of the first photodiode of the CMOS image sensor and the readout of the next second photodiode, a coordinate offset by a certain amount from the coordinate of the first photodiode is used. The image pickup apparatus according to claim 1, wherein two types of image data having different exposure times are generated by driving the CMOS image sensor at a normal double speed by performing a reading process of the photodiode. 3. A photodiode of the CMOS image sensor is provided with two independent signal reading ports, and two reading processes are performed so that the time is shifted from the two signal reading ports to an arbitrary photodiode. The image pickup apparatus according to claim 1, wherein the image data for long exposure and the image data for short exposure are thereby generated.