JP2004135074A - Image pickup device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To simultaneously perform short time exposure and long time exposure by using a pair of cameras which are horizontally arranged right and left to an object and to obtain an image of which the cotour line is clear. <P>SOLUTION: An image pickup device is provided with one camera which is arranged in the same direction as the other camera that images the front object at prescribed shutter time and outputs a first image signal and which images the front object and outputs a second image signal, an image deviation correction means correcting a deviation between the first image and the second image, which are outputted from both cameras, and an image synthesis means overlapping and outputting the first image and the second image which are corrected by the image deviation correction means. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an imaging apparatus using an electronic camera (CCD camera) having a dynamic range substantially increased.
[0002]
[Prior art]
As one example of this type of conventional apparatus, one CCD camera alternately and continuously captures images of the same object with both short-time exposure S1 and long-time exposure L1 (see FIG. 4). The dynamic range is expanded by synthesizing the image obtained by S1 and the image obtained by long-time exposure L1 to obtain one image (for example, see Patent Document 1).
[0003]
However, in such a conventional apparatus, an area (shaded area) illuminated by sunlight is exposed while a CCD camera is exposed to image the imaging area 1 as shown in FIG. 5, for example. When moving from the portion indicated by the symbol A1 to the portion indicated by the symbol A2, that is, when the area is illuminated by the sun rays during the imaging of one image and moves from the portion indicated by the symbol A1 to the portion indicated by the symbol A2 Has a problem that the portion A3 illuminated by the sunlight is band-shaped, so that a particularly wide dynamic range cannot be obtained.
Particularly, during the exposure, the illuminated portions are separated as indicated by reference numerals A1 and A2 in FIG. 5, and the image captured by the short-time exposure S1 while the portion indicated by the reference numeral A1 is illuminated; Since the image taken by the long-time exposure L1 while the portion shown is illuminated is significantly different from the image taken, the image quality of the image obtained by combining the two images deteriorates, and the dynamic range does not widen. There was a problem.
[0004]
More specifically, in a case where the vehicle travels in a sunlight-blind sunlight, as shown in FIG. 5, the image pickup area denoted by reference numeral 1, that is, the area indicating a part of the occupant in the car, has strong sunlight. Moves in the direction of the arrow from the hatched portion indicated by reference numeral A1 to the hatched portion indicated by reference numeral A2.
For example, in the case where a car travels at 100 km / h during a sunshine day, assuming that the shutter speed of the CCD camera, that is, the exposure time L is 10 ms, the portion where the strong sunlight falls during the exposure time L A1 and A2 move by 28 cm, and the portions A1 and A2 where strong sunlight shines in the image picked up by the CCD camera, that is, oval white spots as shown in FIG. A portion (hatched portion) is imaged, and an oval white portion (hatched portion) as shown in FIG. 6 (B) is imaged by the long-time exposure S2, and they cannot be complemented because they do not overlap with each other. There is a possibility that a problem may occur in that signal processing such as image synthesis for the purpose of extending the dynamic range cannot be performed.
[0005]
As a method for solving this problem, an exposure time setting means for setting an exposure time having a relationship of first exposure time> second exposure time >>...> nth exposure time; The time arrangement is such that the first exposure time, the second exposure time,..., The nth exposure time, the (n−1) th exposure time,..., The second exposure time, and the first exposure time. An exposure time state arranging means for setting an exposure state of the exposure time setting means; and a means for obtaining a composite image from images exposed and photographed by the (2n-1) exposure states. However, when the subject moves because it is located at a temporally intermediate position in terms of the imaging order, the longest exposure time is placed at the beginning and end of the (2n-1) types of exposure times, and in the center Because the shortest exposure time is set, a balanced image Can Rukoto (see Patent Document 2).
[0006]
[Patent Document 1]
Japanese Patent No. 3040379 [Patent Document 2]
JP-A-6-225217
[Problems to be solved by the invention]
However, the image obtained by this method is intended to be natural as a photograph, so that even if the surroundings are blurred and sacrificed, the effect is achieved by clearing the center, so that the subject is clear. If a contour line is desired, it still cannot compensate for the dynamic range of long-time exposure.
[0008]
Therefore, the present invention uses a pair of cameras horizontally arranged on the left and right toward the object to detect the position of the object in the vehicle interior, performs short-time exposure and long-time exposure simultaneously, and makes the outline clear. The purpose is to obtain an image.
[0009]
Specifically, one CCD camera performs short-time exposure, for example, the other CCD camera performs one long-time exposure during a continuous line, for example, 4 to 6 times. Short-time exposure is continuously performed the same number of times, and one CCD camera performs long-time exposure once.
[0010]
Then, the dynamic range of the selected image is expanded, and the image of the contour of the object such as the occupant is clearly obtained.
[0011]
[Means for Solving the Problems]
An imaging device according to the present invention is arranged such that one camera that images a front object at a predetermined shutter time and outputs a first image signal and is oriented in the same direction as the one camera, Image shift correction that corrects a shift between the other camera that outputs the second image signal by capturing the front object with a short shutter time and that outputs the second image signal. And an image synthesizing means for superimposing and synthesizing the first image and the second image corrected by the image shift correcting means, and outputting the synthesized image.
[0012]
In addition, the shutter operation of the other camera is performed a plurality of times while the one camera is operated once, and the same pixels are compared with each other among the pixels constituting the plurality of images obtained thereby, and the brightest image is obtained. By selecting the brightness data of the pixels and forming them into one image, it is possible to obtain a synthesized image that is not affected by the sunshine or the like moving at a higher speed, and the dynamic range can be further improved.
[0013]
BEST MODE FOR CARRYING OUT THE INVENTION
According to the invention described below, when one CCD camera is performing short-time exposure, the other CCD camera performs long-time exposure, and when the other CCD camera is performing short-time exposure, one CCD camera performs long-time exposure. I do. In addition, it is characterized in that the short-time exposure during the long-time exposure is performed several times continuously.
[0014]
Embodiment 1 FIG.
Next, an embodiment will be described below with reference to FIGS.
Reference numeral 10 denotes a subject to be imaged, for example, an object sitting on a passenger seat in a vehicle, which is a human here. Numerals 11 and 12 denote left and right CCD cameras which are attached to the mounting portion of the rearview mirror of the vehicle toward the passenger seat and are set in the same imaging direction. When S3) is at the high level, the left CCD camera 11 performs short-time exposure, and the right CCD camera 12 performs long-time exposure. Further, the image data picked up by the left and right CCD cameras 11 and 12 are swept out and stored in the first left image storage means 14 or the first right image storage means 15 respectively connected thereto. After that, when the sweeping out and the storing are completed, it is reset at the end timing.
[0015]
The timing generating unit 13 outputs a plurality of pulses P1 for short-time exposure (see narrow pulses P1 to P4 in S1 and S2 in FIG. 2) continuously from the output line S1, and then outputs the pulse L1 for long-time exposure to the left. A plurality of pulses P2 for short-time exposure are supplied to the CCD camera 11, and after a pulse L2 for long-time exposure (see pulses L1 to L4 having a wide width in FIGS. To the right CCD camera 12.
The pulses P1 to P4 are the same pulse, and the pulses L1 to L4 are also the same pulse.
[0016]
In addition, the timing generation unit 13 sends control signals (left side and left side) indicated by S3 in FIG. (A signal indicating the selection status of the right CCD cameras 11 and 12).
[0017]
The modes of the pulses output to the output lines S1 and S2 are different in phase by 180 degrees as shown in S1 and S2 in FIG. That is, when pulses P1 and P2 for short-time exposure are output to the output line S1, a pulse L2 for long-time exposure is output to the output line S2, and a pulse for long-time exposure is output to the output line S1. When L1 is output, pulses P2 and P4 for short-time exposure are output to the output line S2.
The first left image storage means 14 and the first right image storage means 15 temporarily store image data when supplied from the corresponding one of the left CCD camera 11 and the right CCD camera 12, respectively. Are supplied to the first and second image synthesizing means 16 and 17 as appropriate.
[0018]
When the control signal supplied from the output line S3 of the timing generating unit 13 is at a high level, the first image synthesizing unit 16 synthesizes an image obtained by short-time exposure supplied from the first left image storage unit 14. It is supplied to the dynamic range expanding means 19. When the control signal is at the low level, the image obtained by the long-time exposure supplied from the first left image storage unit 14 is supplied to the dynamic range expansion unit 19 without processing.
[0019]
On the other hand, when the control signal supplied from the output line S3 of the timing generating unit 13 is at a low level, the second image synthesizing unit 17 synthesizes an image obtained by the short-time exposure supplied from the first right image storage unit 15. And supplies it to the image shifting means 18. When the control signal is at a high level, the image obtained by the long-time exposure supplied from the first right image storage unit 15 is supplied to the image shift unit 18 without processing.
[0020]
The combination of the images in the first and second image combining means 16 and 17 will be described below based on the diagram schematically shown in FIG. Note that the first and second image synthesizing units 16 and 17 have substantially the same function with respect to image synthesizing, so the first image synthesizing unit 16 will be described as an example.
[0021]
In synthesizing the four image data (see FIGS. 3A to 3D) supplied from the first left image storage unit 14, the first image synthesizing unit 16 may generate four images by simple addition or averaging. If the same part of the image data is bright, only the bright part becomes bright and the other part becomes dark. Therefore, the same pixel of each of the four images (see FIGS. 3A to 3D) is used. When the brightness, in other words, the image data is considered as XY coordinates, the brightnesses of the same coordinates are compared, and the image is synthesized in such a manner that the larger (brighter) value indicating the brightness is adopted.
[0022]
Specifically, this will be described with reference to schematic diagrams (A) to (D) shown in FIG.
For example, assuming that a value indicating the brightness of each pixel of the four images supplied from the first left image storage unit 14 (the larger the numerical value, the brighter) is given in the figure, the The brightness of the XY coordinates is as follows.
The brightness of the coordinates (1, 1) is "6" for the image data A, "4" for the image data B, "6" for the image data C, and "6" for the image data D. The brightness of the data E is “6”.
The brightness of the coordinates (2, 1) is “8” for the image data A, “8” for the image data B, “8” for the image data C, and “6” for the image data D. The brightness of the data E is “8”.
Thereafter, a composite image in the short-time exposure is similarly formed.
[0023]
Next, the image shifting means indicated by reference numeral 18 calculates the amount of shift of the image data of the second image synthesizing means 17 in the left-right direction with respect to the first image synthesizing means 16 from both distance detecting means 24 described later. An instruction based on the distance calculated using the principle of triangulation from the deviation amount of the image data of the CCD cameras 11 and 12, that is, in accordance with the distance calculated by the distance detecting means 24 described later based on the previous image data. Correct according to the amount of deviation and make them coincide.
[0024]
The dynamic range enlarging unit 19 superimposes and synthesizes the first image synthesizing unit 16 with the image data from the second image synthesizing unit 17 supplied via the image shifting unit 18.
In this image synthesis, a value indicating the brightness of each pixel constituting the image data supplied from both the first image synthesis means 16 and the image shift means 18 is simply averaged for each pixel, and the synthesized image is synthesized. And outputs the synthesized image data to an external circuit (not shown), for example, an arithmetic circuit for extracting a contour of a human head or the like from the image data.
[0025]
Reference numeral 20 denotes a switch, and when the control signal (see S3 in FIG. 2) supplied from the timing generator 13 is at a high level, the contacts are connected as indicated by a broken line to output the output from the dynamic range expander 19. It is supplied to and stored in the second left image storage means 22.
If it is at the low level, the contacts are connected as indicated by a solid line, and the output from the dynamic range expansion means 19 is supplied to the second right image storage means 21 for storage.
[0026]
Reference numeral 23 denotes an image shift return unit which inputs image data supplied from the second right image storage unit 21 as a control signal, which is a signal indicating the amount of image shift supplied from the distance detection unit 24, and outputs a control signal. In accordance with the switching timing from the supply state to the supply state, the image data is returned by the amount of displacement shifted by the image shifting means 18 to the distance detection means 24.
[0027]
Numeral 24 denotes a distance detecting means, which is the latest image data picked up by the left CCD camera 11 supplied from the second left image storing means 22 and the latest image data picked up by the right CCD camera 12 supplied from the image shift returning means 23. Based on the image data of, the amount of shift of both image data in the left and right direction, that is, the distance to the imaged target object is calculated, and output to the external circuit (not shown), and the calculated amount of shift is calculated. The image data is supplied to the image shift unit 18 for the next image synthesis, and is supplied to the image shift return unit 23 to restore the shifted image data.
[0028]
Next, the operation of the above configuration will be described below.
(At time T1 in FIG. 2)
Four short pulses P1 are continuously output to the output line S1 of the timing generation circuit 13, and the left CCD camera 11 performs four short exposures based on the short pulses P1 and stores the image data for four times in the first left image. It is stored in the means 14.
[0029]
Further, since one long pulse L1 is output to the signal line S2 from the timing generation means 13 to the right CCD camera 12, the right CCD camera 12 performs long-time exposure based thereon, and the image data of the long-time exposure is It is stored in the first right image storage means 15.
[0030]
At this time, since the high-level control signal is supplied to the signal line S3 from the timing generating unit 13, the first image synthesizing unit 16 executes the four short-time exposures stored in the first left image storing unit 14. The same coordinates are compared for each pixel of the image data created in step, and the brightest pixel among the compared pixels having the same coordinates is determined to be the true brightness of the pixel at that coordinate, and the short-time exposure is performed. The second image synthesizing unit 17 supplies one image data stored in the first right image storing unit 15 to the image shifting unit 18 as it is, and converts the image data into the distance detecting unit. The shift is performed in the left-right direction according to the shift amount supplied from 24.
[0031]
As a result, the dynamic range enlarging unit 19 receives the image data from the first image synthesizing unit 16 and the image shifting unit 18, and the image data supplied from each of the first image synthesizing unit 16 and the image shifting unit 18. Are combined and supplied to the switch means 20 and to the external circuit (not shown).
[0032]
At this time, since the high-level control signal from the timing generation means 13 is supplied to the switch means 20, the contacts of the switch means 20 are connected as shown by the solid lines. The combined image data is stored in the second right image storage means 21 and is shifted before the image data is matched with the image data from the left CCD camera 11 by the image shift return means 23 before the shift operation is performed. The image data is returned to the image data which is not present and supplied to the distance detecting means 24. The distance detecting means 24 calculates the distance to the object 10 based on the amount of deviation from the synthesized image, The signal is output to the return means 23 and to the external circuit (not shown).
[0033]
(At time T2 in FIG. 2)
Contrary to (in the section T1 in FIG. 2), four short pulses P2 and P4 are continuously output to the output line S2 of the timing generation circuit 13, and the signal of the timing generation means 13 is output to the right CCD camera 12. Since one long pulse L1 is output to the line S1, the left CCD camera 11 performs long-time exposure based on the output, and the right CCD camera 12 performs four short-time exposures continuously. As a result, the dynamic range expanding unit 19 combines the image data supplied from the first image combining unit 16 with the image data supplied from the image shifting unit 18 and supplies the combined image data to the switch unit 20. At this time, the contacts of the switch means 20 are connected as shown by the broken lines, and are stored in the second left image storage means 22.
[0034]
As a result, the distance detection unit 24 determines whether the image stored in the second right image storage unit 21 is in the original state before shifting the image and the image data stored in the second left image storage unit 22 in the left-right direction. Is calculated and supplied to the image shifter 18 and the image shifter 23 as the next shift, and the distance to the object 10 is calculated based on the shift and output to an external circuit (not shown). I do.
[0035]
【The invention's effect】
According to the imaging apparatus of the present invention, since the short-time exposure and the long-time exposure are performed simultaneously and the short-time exposure is always performed during the long-time exposure, it is possible to respond to a fast change in sunlight. That is, it is possible to clearly capture the outline of the obtained image.
Further, when performing the short-time exposure and the long-time exposure simultaneously, there is obtained an effect that one camera does not require a plurality of image pickup devices.
In addition, the quality of the captured image can be improved by performing the short-time exposure a plurality of times. Furthermore, when synthesizing a plurality of images by short-time exposure, the degree of white spots in the image can be reduced by selecting the pixel having the brightness of the brightest pixel and synthesizing it into one image. The effect that the range can be improved is obtained.
[Brief description of the drawings]
FIG. 1 is a circuit block diagram showing a first embodiment of the present invention.
FIG. 2 is a timing chart for explaining a circuit block in FIG. 1;
FIG. 3 is an explanatory diagram illustrating a method of synthesizing an image, which is a main part of the present invention.
FIG. 4 is a timing chart for explaining a conventional example.
FIG. 5 is an explanatory diagram for explaining a conventional problem.
FIG. 6 is an explanatory diagram for explaining a conventional problem.
[Explanation of symbols]
10 subject (object)
11 Left CCD camera 12 Right CCD camera 13 Timing generation means 14 First left image storage means 15 First right image storage means 16 First image synthesis means 17 Second image synthesis means 18 Image shift means 19 Dynamic range enlargement means 20 Switch means 21 second right image storage means 22 second left image storage means 23 image shift return means 24 distance detection means

Claims (3)

One camera that images a front object at a predetermined shutter time and outputs a first image signal;
The other camera arranged in the same direction as the one camera and imaging the front object with a shorter shutter time than the one camera and outputting a second image signal;
Image shift correcting means for correcting a shift between the first image and the second image output from both cameras;
An image pickup apparatus comprising: an image synthesizing unit that superimposes and synthesizes the first image and the second image corrected by the image shift correcting unit and outputs the synthesized image. 2. The imaging apparatus according to claim 1, wherein the shutter operation of the other camera is performed a plurality of times while the shutter operation of the one camera is performed once. The plurality of images captured by the other camera are formed by comparing the same pixels among the pixels constituting each image with each other, selecting the brightness data of the brightest pixel, and forming one image. The imaging device according to claim 2, wherein:

Images