JPH10136269A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the image pickup device that extends a dynamic range of an image pickup element and provides an output. SOLUTION: An image classification section 27 that reads an image from a CCD 23 classifies the image into that of even and odd number of lines. Edge detection sections 37, 39 detect an edge strength of an image of even and odd number of lines, a strength comparison section 41 compares detected edge strength values for each picture element, and a selection section 43 selects the image classified by an image classification section 27 for each of picture elements having a higher edge strength. In this case, an exposure time calculation section 45 calculates an exposure time so that a mean density of the image of even and odd number of lines is a proper mean density stored in advance, a vertical mobile section 13 controls the image-pickup charges reciprocally to be moved in the vertical direction alternately as an exposure process of the CCD 23 and controls an exposure timing of the CCD 23 in response to the exposure time calculated by the exposure time calculation section 45.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image pickup apparatus for expanding a dynamic range of a CCD camera for output.

[0002]

2. Description of the Related Art Conventionally, in an image pickup apparatus for expanding and outputting a dynamic range, an image picked up at an exposure time T1 is stored, and then an image is picked up at an exposure time T2. The image at the exposure time T1 and the image at the exposure time T2 are compared, and a clearer image is selected as one image.

For example, as shown in FIG. 8, when an image of a stopped vehicle is taken, the front side of the vehicle is in the shade and the rear side is in the sun, or an appropriate exposure state is obtained from either the sun or the sun. In this case, the other image is underexposed or overexposed. There is an advantage that an image of a proper exposure can be obtained as a whole by selecting and combining images of a proper exposure portion from these two images.

[0004]

However, in the conventional image pickup apparatus, one image reading from the camera takes a long time. Therefore, when the object to be imaged or the camera itself is moving, it takes a long time to read the image. There is a difference in the captured image between the image at the exposure time T1 and the image at the exposure time T2. As a result, even when comparing images at the same position on the images, there is a portion where different subjects are compared due to the time difference. As a result, in the output image, there is a problem that a subject is deformed or an image that is not properly exposed is selected. For example, when the vehicle is moving forward in the above example, the vehicle is deformed and output as shown in FIG. The present invention has been made in view of the above, and an object of the present invention is to provide an imaging apparatus capable of expanding and outputting a dynamic range of an imaging device.

[0005]

According to the first aspect of the present invention,
In order to solve the above-mentioned problem, an image pickup element capable of transmitting an image by transmitting light every other line and moving a charge generated at the time of image pickup in a sub-scanning direction, reading an image from the image pickup element and reading an image of even and odd lines Image classification means, two edge detection means for detecting the edge intensity of the image of the even-numbered and odd-numbered lines, intensity comparison means for comparing the detected edge intensity value for each pixel, Selecting means for selecting an image classified by the image classifying means for each of the larger pixels; and setting the exposure time so that the average density value of the even-numbered and odd-numbered line images becomes a suitable average density value stored in advance. Exposure time calculation means to calculate, and control so as to alternately reciprocate the charge imaged as an exposure step of the image sensor in the vertical direction,
The gist of the present invention is to include vertical movement control means for controlling the exposure timing of the image sensor according to the exposure time calculated by the exposure time calculation means.

According to a second aspect of the present invention, in order to solve the above-mentioned problem, the exposure time calculating means sets one of two images of even and odd lines so that a dark portion of a subject is appropriately exposed. The gist of the present invention is to calculate an exposure time so that a bright portion of the other subject is appropriately exposed.

According to a third aspect of the present invention, in order to solve the above-mentioned problem, one exposure step in the vertical movement control means includes exposing a transmission portion of the image sensor to a first exposure time for a first exposure time. After moving the charge of the entire surface of the image sensor downward by one line, and moving the charge of the light-transmitting portion of the image sensor to the light-shielding portion, and moving the charge of the light-shielding portion to the light-transmitting portion, The gist of the present invention is to include a second step of exposing in this state for a second exposure time and a third step of moving the electric charges on the entire surface of the image sensor upward line by line.

According to a fourth aspect of the present invention, in order to solve the above problem, the exposure time calculating means sets the number of repetitions of the exposure process in the vertical movement control means.

According to a fifth aspect of the present invention, in order to solve the above-mentioned problem, the image pickup device is disposed on a front surface of the image pickup device and blocks light every other horizontal line with respect to the image pickup device. The gist of the present invention is to have a stripe composed of a stripe pattern.

[0010]

According to the first aspect of the present invention, there is provided an image pickup device capable of transmitting an image by transmitting light every other line and moving an electric charge generated during the image pickup in the sub-scanning direction.
The image is read out from the image pickup device and is classified by the image classification means into images of even and odd lines. Next, the edge intensities of the images of the even and odd lines are detected by two edge detecting means, and the detected edge intensity values are compared for each pixel by the intensity comparing means. The image classified by the image classification means is selected by the selection means. At this time, the exposure time is calculated by the exposure time calculation means so that the average density value of the images of the even and odd lines becomes a pre-stored appropriate average density value. As the control is performed such that the imaged electric charges are alternately reciprocated in the vertical direction, and the exposure timing of the image sensor is controlled in accordance with the exposure time calculated by the exposure time calculation means. The average density value of the image of the odd-numbered line can be set to an appropriate average density value stored in advance, and as a result, the dynamic range of the image sensor can be enlarged and output.

For example, when a camera using an image pickup device is installed in a vehicle to photograph a road, even when there is a tunnel in front of the vehicle, a tunnel portion that is underexposed during the exposure time T1 and a sunlit portion that is an appropriate exposure portion. On the other hand, in the exposure time T2, an overexposed sunshine portion and an appropriately exposed portion of a shaded portion can be imaged, so that an appropriately exposed portion having a high contrast can be selected, and images of an appropriate exposure can be obtained inside and outside the tunnel. Can be.

Further, when the vehicle is photographed by the same camera, even when a part of the road and the vehicle is shadowed by the building, the vehicle which is the underexposed building portion and the appropriately exposed portion at the exposure time T1. Can be imaged, while at the exposure time T2, the sunlit portion that is overexposed and the shaded portion that is the appropriate exposure portion can be imaged, so that the appropriate exposure portion with high contrast can be selected, and the image of the appropriate exposure of the entire vehicle and roads Can be obtained.

According to the second aspect of the present invention, one of the two images of the even-numbered lines and the odd-numbered lines is appropriately exposed such that a dark portion of the subject is appropriately exposed and a bright portion of the other subject is appropriately exposed. By calculating the exposure time so that the exposure is performed, the dynamic range of the image sensor can be expanded and output.

According to the present invention described in claim 3, 1
The first exposure step includes a first step of exposing a transmission portion of the image sensor over a first exposure time, and a step of moving electric charges of the entire surface of the image sensor downward line by line, and then, A second step of moving the charge of the light-shielded portion to the light-shielding portion and moving the charge of the light-shielded portion to the light-transmitting portion, and then exposing for a second exposure time in this state; And the third step of moving the line upward by one line, the average density value of the images of the even and odd lines can be set to an appropriate average density value stored in advance, and as a result, the dynamic range of the image sensor Can be enlarged and output.

According to the present invention, by setting the number of repetitions of the exposure step in the vertical movement control means, the average density value of the image of the even and odd lines is stored in the appropriate average. The density value can be set, and as a result, the dynamic range of the image sensor can be expanded and output.

According to the fifth aspect of the present invention, a stripe composed of a stripe pattern for blocking light at every other horizontal line with respect to the image sensor is arranged on the front surface of the image sensor. The light can be transmitted every other line for imaging.

[0017]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a system configuration of an imaging device 1 according to one embodiment of the present invention. In FIG. 1, a clock generation unit 11 generates a clock signal that is a reference for the operation of each unit. The vertical movement control unit 13 uses C
A vertical movement signal for controlling the charge of the CD 23 to reciprocate alternately in a direction perpendicular to the stripe 21 is generated.

The read control unit 15 generates a read signal for reading the charge after image pickup from the CCD 23.

The CCD drive unit 17 includes a vertical movement control unit 13
The vertical movement signal generated by the read control unit 15 and the readout signal generated by the readout control unit 15 are collectively given to the CCD 23 and
3 is driven. When the captured image passing through the lens 19 is projected on the CCD 23 on which the stripe 21 is arranged, this C
The stripe 21 arranged on the front of the CD 23 is shown in FIG.
As shown in FIG. 7, the light-shielding portion 21a blocks light and the light-transmitting portion 21b transmits light. Since the stripe 21 is a fixed stripe pattern, it can be easily formed using, for example, a photographic film. The feature of this stripe 21 is that the stripe interval is C
The light-shielding portion 21a for blocking light is arranged on an odd-numbered line while the light-transmitting portion 21b for transmitting light is arranged on an even-numbered line so as to match the width of one line of the CD 23. In this manner, by arranging the stripes 21 made of a stripe pattern for blocking light every other horizontal line with respect to the CCD 23 on the front surface of the CCD 23,
Light can be transmitted through the CCD 23 every other line to capture an image.

Returning to FIG. 1, the image signal output from the CCD 23 is amplified by the amplifier 25 in accordance with the read signal output from the CCD
7. Output via the image selection unit 35. The image classification unit 27 includes an A / D conversion unit 29, an image storage unit 31, and a storage control unit 33. The A / D conversion unit 29
The image signal amplified by the amplifier 25 is subjected to A / D conversion and quantized to output image data. The image storage unit 31
The image data after the A / D conversion is sequentially stored for one line, and the image data for one line is output. The storage control unit 33 outputs a control signal for causing the image storage unit 31 to sequentially store the image data after the A / D conversion for one line and to output the image data one line before.

The image selection unit 35 includes edge detection units 37 and 3
9, an intensity comparison unit 41 and a selection unit 43.
The edge detectors 37 and 39 detect edges representing points of change in brightness of input image data, and detect contours of the captured object. The intensity comparison unit 41 compares the intensities of the edges output from the two edge detection units 37 and 39, and outputs a selection signal for selecting the one with the larger edge intensity. The selection unit 43 selects the image data 29d immediately after A / D conversion or the image data 31d output from the image storage unit 31 and delayed by one line according to the selection signal output from the intensity comparison unit 41. The output of the imaging device 1 is used. The exposure time calculation unit 45 calculates the average density value of each imaging surface based on the two image data input from the image classification unit 27, and adjusts the value so as to approach the appropriate average density value stored in advance. Control the exposure time.

Next, the operation of the image pickup apparatus 1 will be described with reference to the timing chart shown in FIG. Note that the imaging device 1
Is automatically controlled internally so as to periodically repeat imaging with the CCD 23, reading of image data from the CCD 23, calculation of exposure time, and the like. First, at the time of imaging, as shown in FIG.
Prior to imaging at 23, the charges on the entire surface of the CCD 23 are cleared. Next, at time t1, the CCD 23 is exposed for a predetermined time T1. As a result, the stripe 21
An electric charge corresponding to the amount of incident light is accumulated in the image sensor corresponding to the transmission portion 21b.

After the exposure, at time t2, the vertical movement control unit 13 moves the electric charge on the entire surface of the CCD 23 downward by one line, and moves the electric charge of the transmissive portion 21b of the CCD 23 to the light-shielding portion 21a of the stripe 21. The charge of the portion 21a is moved to the transmission portion 21b. Next, at time t3, exposure for a certain time T2 is performed in this state,
After this exposure, at time t4, the charges on the entire surface of the CCD 23 are moved upward one line at a time, returned to the initial position, and the first exposure process is completed. Similarly, after time t5,
The repetition of the exposure for a certain time T1, the electric charge transfer in the vertical direction downward, the exposure for the certain time T2, the electric charge transfer in the vertical direction, and the like are repeated by a certain number n.

The exposure time T1 is a minimum value which can be achieved by the physical characteristics of the CCD 23. The exposure time T1 and the number of exposures n are calculated by the exposure time calculation unit 45.

The exposure time calculation section 45 calculates the average values I1a and I2a of the densities of the captured images corresponding to the previous exposure times T1 and T2, and obtains the appropriate average density values I1s and I2s obtained in advance. , The total exposure times T1a and T2a are calculated. That is, the average values I1a and I2a of the densities correspond to the appropriate average density I1.
s, I2s, the previous total exposure time T1
In this case, the exposure time is made longer than the previous total exposure times T1a and T2a by Δt.

Here, from the exposure time T1a, the number of times of imaging n is obtained by n = T1a / T1, and the exposure time T2 is obtained by T2 = T2a / n. In addition, the appropriate average density value I
1s and I2s are obtained from I1s = (Imax-Imin) .times.3 / 4 + Imin I2s = (Imax-Imin) .times.1 / 4 + Imin, based on the density upper limit Imax and the lower limit Imin.

After image pickup, as shown in FIG.
D23, the charges of the lines L1 to Li are sequentially read out, and the A / D converter 2
In step 9, A / D conversion is performed and quantization is performed to output image data. The image classification unit 27 classifies the image data into even-line image data and odd-line image data. See CCD2
3. In the case of reading sequentially from above, first, the image data of the first line is read, written to the image storage unit 31, and stored. Next, when reading the image data of the second line from the CCD 23, the image data at the same horizontal position is read from the image storage unit 31 and read. In this way, the odd-numbered and even-numbered one-line image data are simultaneously output to the image selecting unit 35.

Next, as shown in FIG. 3B, the image selection section 35 selects an image having a high contrast from the two input image data, that is, selects an image having a larger edge intensity. As one of the methods, differential processing is performed on two input image data to compare their intensities for each pixel, and an input pixel having a higher intensity is selected and output according to the comparison result. I do. Details will be described with reference to a circuit configuration example of the image selection unit 35 shown in FIG. 4 and a timing chart shown in FIG. In FIG. 4, the edge detection unit 37 includes a latch 51 that delays one clock of image data in synchronization with a clock signal output from the clock generation unit 11, and an image data 29d output from the A / D conversion unit 29. A subtracter 53 for subtracting the image data delayed by one clock from (input 1).
Similarly, the edge detection unit 39 includes a latch 55 that delays one clock of image data in synchronization with a clock signal output from the clock generation unit 11, and an image data 31d (input 2) output from the image storage unit 31. ) Is subtracted from the image data delayed by one clock.

An intensity comparator 41 generates an absolute value for the subtraction result of the subtractor 53, and an absolute value generator for generating the absolute value of the subtraction result of the subtractor 57. 61 and the output value from the absolute value device 59,
And a subtractor 63 for subtracting the output value from 1 and outputting the sign bit of the subtraction result. The selection unit 43
On the other hand, image data 29 output from the A / D converter 29
d (input 1) and the image storage unit 3
The selector 65 receives the image data 31d (input 2) output from 1 and selects the input 1 or the input 2 according to the result of the subtraction by the subtractor 63 in the intensity comparison unit 41, and the selector 65 selects the input 1 or the input 2. A latch 67 for outputting the image data delayed by one clock. In this configuration, when the sign bit output from the subtracter 63 is positive, the selector 43 outputs the image data 29d (input 1) output from the A / D converter 29.
On the other hand, when the sign bit output from the subtractor 63 is negative, the selector 43 outputs image data 31d (input 2) output from the image storage unit 31.

That is, the image selecting section 35 compares the edge strength of the image data at the same bit position for each pixel, and selects and outputs the image data having the larger edge strength value. For example, in a cycle j shown in FIG. 5, two j-th pixel data d1j and d2j input to the edge detectors 37 and 39 and j-1 from the latches 51 and 55 are output.
Subtractors 53 and 57 determine difference values between the pixel data d1j-1 and d2j-1. Absolute value units 59 and 61 take the absolute values. Subtractor 63 calculates the difference value between the absolute values. A certain sign bit is obtained as a comparison result.

For example, when | d1j-d1j-1 | <| d2j-d2j-1 |, the result is negative (sign bit = 1), the pixel data of input 2 is selected by the selector 65, and the next rising edge of the clock Is latched by the latch 67 and output. Similarly, in the cycle j + 1, for example, when | d1j + 1−d1j |> | d2j + 1−d2j |, d1j + 1 is selected and output from the latch 67.

As a result, for example, when a camera using the image pickup device 1 is installed in a vehicle to photograph a road, FIG.
As shown in the figure, even when there is a tunnel in front of the vehicle,
As shown in FIG. 7B, an underexposed tunnel portion and a sunshine portion, which is a proper exposure portion, can be imaged at the exposure time T1, whereas, as shown in FIG. Since the sunshine portion and the shaded portion, which is the appropriate exposure portion, can be imaged, and the appropriate exposure portion having high contrast is selected by the image selection unit 35, as shown in FIG.
Images with proper exposure can be obtained inside and outside the tunnel.
In addition, when photographing a vehicle ahead in the same camera, FIG.
As shown in FIG. 7A, even when a part of the road and the vehicle is shadowed by the building, as shown in FIG. 7B, the shadow portion of the underexposed building and the properly exposed portion at the exposure time T1. A part of a certain vehicle can be imaged. On the other hand, as shown in FIG. 7C, a sunlit portion that is overexposed and a shaded portion that is an appropriate exposure portion can be imaged at the exposure time T2, and the image selection unit 35 has high contrast. Since an appropriate exposure part is selected, FIG.
As shown in (d), an image of the vehicle and the entire road with proper exposure can be obtained.

As described above, the CCD 23 which allows light to pass through every other line to capture an image by the CCD 23 and which can move the charge generated at the time of imaging in the sub-scanning direction is prepared.
The image is read from the CD 23 and is classified by the image classifying unit 27 into images of even and odd lines. Next, the edge intensities of the images of the even-numbered and odd-numbered lines are determined by the edge detection units 37 and 39.
The intensity comparison unit 41 compares the detected edge intensity values for each pixel, and the selection unit 43 selects an image classified by the image classification unit 27 for each pixel having the larger edge intensity value. At this time, the exposure time is calculated by the exposure time calculation unit 45 so that the average density value of the images of the even and odd lines becomes an appropriate average density value stored in advance. As a process, the control is performed such that the imaged charges are alternately reciprocated in the vertical direction, and the exposure timing of the CCD 23 is controlled in accordance with the exposure time calculated by the exposure time calculation unit 45. In addition, the average density value of the image of the odd-numbered line can be set to an appropriate average density value stored in advance, and as a result, the dynamic range of the CCD 23 can be enlarged and output.

For example, when a camera using an image pickup device is installed in a vehicle to photograph a road, even if there is a tunnel in front of the vehicle, a tunnel portion that is underexposed during the exposure time T1 and a sunlit portion that is an appropriate exposure portion. On the other hand, in the exposure time T2, an overexposed sunshine portion and an appropriately exposed portion of a shaded portion can be imaged, so that an appropriately exposed portion having a high contrast can be selected, and images of an appropriate exposure can be obtained inside and outside the tunnel. Can be.

When the vehicle is photographed in front of the vehicle by the same camera, even if the road and a part of the vehicle are in the shadow of the building, the vehicle which is the underexposed building and the appropriately exposed part in the exposure time T1. Can be imaged, while at the exposure time T2, the sunlit portion that is overexposed and the shaded portion that is the appropriate exposure portion can be imaged, so that the appropriate exposure portion with high contrast can be selected, and the image of the appropriate exposure of the entire vehicle and roads Can be obtained.

The exposure time calculating section 45 sets one of the two images of the even and odd lines so that the dark portion of the subject is properly exposed and the bright portion of the other subject is properly exposed. By calculating the exposure time, the dynamic range of the image sensor can be expanded and output.

One exposure step in the vertical movement control unit 13 includes exposing the transmission portion of the CCD 23 for an exposure time T1 and then moving the charge on the entire surface of the CCD 23 line by line downward. After the electric charges in the transmitting portion of the CCD 23 are moved to the light-shielding portion, the electric charges in the light-shielding portion are moved to the transmitting portion, and then the exposure is performed for an exposure time T2 in this state. Next, by moving the charge on the entire surface of the CCD 23 upward by one line, the average density value of the images of the even and odd lines can be set to a suitable average density value stored in advance. The dynamic range can be expanded and output.

Further, by setting the number of repetitions of the exposure process in the vertical movement control unit 13, the average density value of the images of the even and odd lines can be set to the appropriate average density value stored in advance. , The dynamic range of the CCD 23 can be enlarged and output. Further, according to the present invention, it is possible to eliminate the difference between the imaging timings of the two captured images as in the related art.

[Brief description of the drawings]

FIG. 1 is a diagram showing a system configuration of an imaging device 1 according to an embodiment of the present invention.

FIG. 2 is a diagram showing a state of a stripe 21 arranged on a front surface of a CCD 23.

FIG. 3 is a timing chart for explaining the operation of the imaging apparatus 1.

FIG. 4 is a diagram illustrating a circuit configuration example of an image selection unit 35;

FIG. 5 is a timing chart for explaining an operation of an image selection unit 35;

FIG. 6 is an example showing that the dynamic range of the CCD camera is expanded.

FIG. 7 is an example showing that the dynamic range of the CCD camera is expanded.

FIG. 8 is an example for explaining the operation of a conventional CCD camera.

FIG. 9 is an example showing an output result when a conventional CCD camera is used.

[Explanation of symbols]

 Reference Signs List 11 clock generation unit 13 vertical movement control unit 15 readout control unit 17 CCD drive unit 13 vertical movement control unit 19 lens 21 stripe 23 CCD 25 amplifier 27 image classification unit 29 A / D conversion unit 31 image storage unit 33 storage control unit 35 image Selection units 37, 39 Edge detection unit 41 Intensity comparison unit 43 Selection unit 45 Exposure time calculation unit

Claims (5)

[Claims] An image pickup device capable of transmitting an image by transmitting light every other line and moving an electric charge generated at the time of image pickup in a sub-scanning direction, reading an image from the image pickup device and classifying the image into images of even and odd lines. Image classification means, two edge detection means for detecting edge intensities of the images of the even and odd lines, intensity comparison means for comparing detected edge intensity values for each pixel, Selecting means for selecting an image classified by the image classifying means for each pixel, and calculating an exposure time so that the average density value of the even-numbered and odd-numbered line images becomes an appropriate average density value stored in advance. Exposure time calculation means, and controls the electric charge imaged as an exposure step of the image sensor to alternately reciprocate in the vertical direction, and calculates the charge by the exposure time calculation means. A vertical movement control means for controlling the exposure timing of the image sensor in accordance with the determined exposure time. 2. The exposure time calculation means according to claim 1, wherein one of the two images of the even and odd lines is set so that a dark portion of the subject is appropriately exposed and a bright portion of the other subject is appropriately exposed. The imaging apparatus according to claim 1, wherein the exposure time is calculated. 3. A single exposure step in the vertical movement control means includes: a first step of exposing a transmission portion of the imaging element to light over a first exposure time; After moving the line downward, the charges in the transmitting portion of the image sensor are moved to the light-shielding portion, and the charges in the light-shielding portion are moved to the transmitting portion. In this state, the light is exposed for a second exposure time. 2. The imaging apparatus according to claim 1, further comprising: a second step of causing the electric charges on the entire surface of the imaging element to move upward one line at a time. 3. 4. The imaging apparatus according to claim 1, wherein the exposure time calculation unit sets the number of repetitions of the exposure step in the vertical movement control unit. 5. The image pickup device according to claim 1, wherein the image pickup device has a stripe disposed in front of the image pickup device and having a stripe pattern for blocking light every other horizontal line with respect to the image pickup device. Item 2. The imaging device according to Item 1.