JP2003174582A - Camera system and automatic exposure control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform optimum AE (automatic exposure control) to pickup a high luminance object and a low luminance object. <P>SOLUTION: As a method for outputting picked-up video images of a wide dynamic range, two or the image pickup for the high luminance object and the image pickup for the low luminance object are performed and two video images are mixed and outputted. Then, an exposure processing part 6B of a DSP 6 is provided with a switch 7C switching a signal route for long time exposure and short time exposure, a long time exposure signal processing part 7A performing a signal processing at the long time exposure, a short time exposure signal processing part 7B performing the signal processing at the short time exposure and a mixing part (MIX) 7D mixing signals of the respective exposure processing parts 7A and 7B. Then, by OPDs (optical detection circuits) 18A and 18B provided in the respective exposure signal processing parts 7A and 7B respectively, a luminance distribution is obtained independently at the long time exposure and at the short time exposure respectively and the optimum exposure control is performed. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention uses a solid-state image pickup device having an electronic shutter function to perform image pickup for a high-luminance subject and image pickup for a low-luminance subject, and obtain two types of video signals obtained by each image pickup. The present invention relates to a camera system and an automatic exposure control method for synthesizing and outputting an imaged image with a wide dynamic range.

[0002]

2. Description of the Related Art Conventionally, a camera system having an AE (automatic exposure control) function using, for example, a CCD image pickup device has been provided. By the way, in AE (automatic exposure control) in this type of camera system, a value obtained by integrating the luminance component of an image is acquired by an OPD (optical detection circuit), and the luminance integral value at the proper exposure is set to a reference luminance integral value (hereinafter, Reference: referred to as ref). Then, the mechanical shutter (iris), the electronic shutter, the gain of the AGC (automatic gain control), etc. are controlled so that the current integrated luminance value (Intg) becomes equal to the reference, and the system gain is increased / decreased to achieve the proper exposure. To get

On the other hand, as a method for outputting a wide dynamic range picked-up image, there is a camera system that picks up images for a high-luminance subject and two times for a low-luminance subject and synthesizes and outputs the two images. Proposed. As a conventional AE (automatic exposure control) in such a camera system, the reference of the AE for high-luminance subject imaging is set to a lower level, and the A
It was handled by an algorithm that sets the reference of E to a higher level.

[0004]

However, as in the above-mentioned conventional example, in the method of responding to the high-luminance subject imaging AE and the low-luminance subject imaging AE by only changing the reference, for example, the backlight condition or the over-forward light condition is used. When imaging is performed in a state or the like, the captured video may not converge at an appropriate convergence position for high brightness and low brightness.

Therefore, an object of the present invention is to perform optimum AE (automatic exposure control) for high-luminance subject imaging and low-luminance subject imaging, and to obtain a high-luminance captured image and a low-luminance captured image. It is an object of the present invention to provide a camera system and an automatic exposure control method capable of converging light at an appropriate convergence position.

[0006]

In order to achieve the above object, the present invention obtains an image for a high-luminance subject and an image for a low-luminance subject by a solid-state image pickup device having an electronic shutter function. In a camera system for synthesizing the two types of image signals thus generated and outputting a captured image with a wide dynamic range, exposure control during short-time exposure for performing imaging for the high-luminance subject and imaging for the low-luminance subject It is characterized by having an automatic exposure control means for independently performing the exposure control at the time of long-time exposure for performing.

Further, according to the present invention, a solid-state image pickup device having an electronic shutter function is used to perform image pickup for a high-luminance subject and image pickup for a low-luminance subject, and synthesize two types of video signals obtained by each image pickup. In an automatic exposure control method for a camera system that outputs an imaged image with a wide dynamic range, exposure control during short-time exposure for performing imaging for the high-luminance subject and long time for performing imaging for the low-luminance subject It is characterized by having an automatic exposure control step for independently performing exposure control at the time of exposure.

Since the camera system of the present invention is provided with the automatic exposure control means for independently performing the exposure control at the time of short-time exposure and the exposure control at the time of long-time exposure, imaging for a high-luminance subject and low-luminance Optimal automatic exposure control can be performed with the image pickup for the subject, and the image pickup image for high brightness and the image pickup image for low brightness can be converged at appropriate convergence positions. Therefore, for example, even when the image is captured in the backlight condition or the over-illumination condition, the high-intensity imaged image and the low-intensity imaged image can be obtained by appropriate exposure control, and the high dynamic range image capturing can be effectively performed. Becomes

Further, in the automatic exposure control method of the present invention, the automatic exposure control step for independently performing the exposure control for the short-time exposure and the exposure control for the long-time exposure is provided. Optimal automatic exposure control can be performed by the image pickup and the image pickup for the low-luminance subject, and the image pickup image for high luminance and the image pickup image for low luminance can be converged at appropriate convergence positions. Therefore, for example, even when the image is captured in the backlight condition or the over-illumination condition, the high-intensity imaged image and the low-intensity imaged image can be obtained by appropriate exposure control, and the high dynamic range image capturing can be effectively performed. Becomes

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of a camera system and an automatic exposure control method according to the present invention will be described below. FIG. 1 is a block diagram showing a configuration example of a camera system according to an embodiment of the present invention. This camera system includes a lens 1 that collects imaging light and a C that captures an image of a subject.
CD image pickup unit 2 and CDS for removing noise of image pickup signal
(Correlated double sampling) circuit 3, AGC (automatic gain control) circuit 4 for controlling the gain of the image pickup signal, and A / D converter 5 for converting an analog signal into digital data.
And a DSP (digital signal processor) 6 for signal processing, and a D for converting digital data into an analog signal
An A / A converter 8, a TG (timing generator) 9 that generates a necessary timing signal and supplies it to each unit,
Mechanical iris (aperture mechanism) as a mechanical shutter mechanism 1
1, a drive motor 12 for mechanical iris, and a microcomputer (μ-COM) 10 for controlling the entire system
And has an internal bus 13.

Further, the DSP 6 is provided with an exposure processing unit 6B for performing signal processing for AE (automatic exposure control) before the signal processing unit 6A for performing original signal processing.
The exposure processing unit 6B includes a switch 7C that switches a signal path between long-time exposure and short-time exposure, a long-time exposure signal processing unit 7A that performs signal processing during long-time exposure, and a signal during short-time exposure. A short-time exposure signal processing unit 7B that performs processing and a synthesis unit (MI that synthesizes the signals of the exposure processing units 7A and 7B).
X) 7D. Further, the long-time exposure signal processing unit 7A and the short-time exposure signal processing unit 7B are provided with OPDs (optical detection circuits) 18A and 18B, respectively, so that the brightness distribution (histogram) is independently output. Has become. Further, the microcomputer 10 includes an internal bus interface (I / F) 14, a CPU 15, and an RO.
M16 and RAM17 are provided.

The operation of the camera system according to this embodiment will be described below. The present camera system, for example, as shown in FIG. 2, can be used in a situation with a wide dynamic range such as when a person 22 is photographed by a camera 23 against a background of a window 21 of a building 20 exposed to direct sunlight. It is possible to perform effective exposure control in imaging. In this camera system, the captured image is the lens 1
, And is photoelectrically converted into an electric signal by the CCD image pickup unit 2. Then, it is sampled through the CDS circuit 3 and AG
The C circuit 4 controls the gain so that the gain is appropriate, and the A / D converter 5 converts the gain into a digital signal.

This digital signal is processed by the DSP 6, but in this example, an exposure processing section 6B is provided in front of the signal processing section 6A to enable imaging of a wide dynamic range. Imaging (short-time exposure imaging)
And low-luminance subject image pickup (long-time exposure image pickup), and two images are combined and output. Explaining in the example of FIG. 2, the high-brightness image pickup (short-time exposure image pickup) is
For example, the image capturing outside the window in FIG. 2 corresponds to the image capturing for the low brightness subject (long-time exposure image capturing) corresponds to the image capturing inside the room in FIG. 2, for example. Then, an image as shown in FIG. 3A is captured by the high-luminance subject image pickup (short-time exposure image pickup), and an image as shown in FIG. 3B is obtained by the low-luminance subject image pickup (long-time exposure image pickup). Is imaged.

Therefore, by combining these images, it is possible to obtain an image having a wide dynamic range as shown in FIG. 3 (C). The switch 7C in the DSP 6 sends the low-luminance subject image pickup signal to the long-time exposure signal processing unit 7A for processing, and the high-luminance subject image pickup signal to the short-time exposure processing unit 7B for processing. Execute by switching in time division. Then, the low-luminance object captured image created by the long-time exposure signal processing and the high-luminance object captured image created by the short-time exposure signal processing are combined by the combining unit 7D, and a wide dynamic range captured image is output. Then, the signal processing unit 6A in the DSP 6 performs signal processing, the D / A converter 8 converts the signal into an analog video signal, and outputs the analog video signal to the subsequent stage.

The software for executing such an algorithm is executed by the microcomputer 10, and the program on the ROM 16 is stored in the CPU 15
Read the necessary commands and data by the I / F 14
Also, each process is executed by supplying each unit via the internal bus 12.

Next, the optimization control of the exposure amount using the OPD (optical detection circuit) 18A, 18B, which is a feature of this system, will be described. FIG. 4 is an explanatory diagram showing the relationship between the subject brightness and the system output value, and FIG. 5 is an explanatory diagram showing the convergence point of the long / short exposure control, where the vertical axis is the output level,
The horizontal axis represents the subject brightness level (absolute value). 6A and 6B are explanatory diagrams showing the specifications of the OPD used in this example, FIG. 6A shows a histogram, and FIG. 6B shows a circuit configuration. FIG. 7 is an explanatory diagram showing an example of a histogram at the long-time exposure control convergence point, and FIG. 8 is an explanatory diagram showing an example of a histogram at the short-time exposure control convergence point.
9 is a flowchart showing an example of long-time exposure control operation, and FIG. 10 is a flowchart showing an example of short-time exposure control operation.

The automatic exposure control method of this system is a DSP.
6 long exposure OPD18A and short exposure OPD1
8B provides an algorithm for performing long-time exposure control and short-time exposure control with good performance in a system capable of obtaining exposure information from different imaging states. First, the appropriate exposure on the short-time exposure side is adjusted to the outdoor reference (high brightness portion) as shown in FIG. This is the dotted line part (high brightness part) in the signal (2) in FIG. 4 when exposure control is performed with the standard exposure reference value.
Equivalent to the output limit of the system,
It is output as white crushed. Therefore, in the short-time exposure control, the system gain is reduced to reduce the signal (3).
You should get output like

On the other hand, the proper exposure on the long-time exposure side is
As shown in FIG. 3B, the indoor reference (low-luminance portion) is set. This may be the signal (2) in FIG. 4 as it is, but in order to clearly output a darker portion, the system gain should be increased to obtain an output signal like the signal (3). As a result, as shown in FIG. 5, appropriate long-time exposure signal (4) and short-time exposure signal (5) are obtained,
By combining them, it is possible to output a captured image with a wide dynamic range within the output limit value of the conventional system.

In this system, in order to enable these appropriate exposure controls, as shown in FIG. 6 (A),
Area a separated by thresholds Th1 and Th2 on the luminance distribution
Intensity integral value Intg of each of 1, a2, and a3
1, Intg2, Intg3, and the number of pixels Cn of each
OPDs (optical detection circuits) 18A and 18B capable of detecting t1, Cnt2, and Cnt3 were prepared. As shown in FIG. 6B, each of the OPDs 18A and 18B receives two luminance signals (Y output) and compares them with thresholds Th1 and Th2. , 32, and Σ circuits 33, 34, 35 for integrating the comparison outputs from the respective Σ circuits 33, 34, 35.
From the output value of Intg1, Intg2, In
It has subtracters 36, 37 and 38 for calculating tg3 and the numbers of pixels Cnt1, Cnt2 and Cnt3.

As shown in FIG. 7, the example of the histogram at the time of proper exposure in the long-time exposure control is such that the histogram of the low-luminance portion below the threshold value is uniform and the high-luminance portion is saturated. If a value obtained by integrating the brightness of the entire screen is used as an evaluation value in such a histogram, a problem occurs. Therefore, in order to obtain an evaluation value with weighting on the integrated value of the low brightness part, the following formula (1) or (2) is used. To use. Long-time exposure control evaluation value = k × (Intg1 / Cnt1) + (1-k) × (Intg2 / Cnt2) Equation (1) Or long-time exposure control evaluation value = (k × Intg1 + (1-k ) * (Intg2) / (k * Cnt1 + (1-k) * (Cnt2) ... Formula (2) However, the variable k in this case is 0.5 <k <= 1.

On the other hand, since the short-time exposure control has a histogram as shown in FIG. 8 in the convergent state, the following formula (3) or (4) is used to obtain an evaluation value which weights the integrated value of the high luminance part. To use. Short-time exposure control evaluation value = k × (Intg1 / Cnt1) + (1-k) × (Intg2 / Cnt2) Equation (3) Or short-time exposure control evaluation value = (k × Intg1 + (1-k ) * (Intg2) / (k * Cnt1 + (1-k) * (Cnt2) ... Formula (4) However, the variable k in this case is 0 <= k <0.5.
By using the above as the evaluation value, appropriate control can be performed in each of the long-time exposure control and the short-time exposure control. The evaluation value calculation coefficient as described above can be changed according to the emphasis on the high luminance component and the emphasis on the low luminance component.

Next, the procedure of the long-time exposure control and the short-time exposure control will be described with reference to the flowcharts of FIGS. First, in the long-time exposure control shown in FIG. 9, the threshold Th is set in step S1, and the integrated luminance values IntgL1, In from the long-time exposure OPD 18A are set in step S2.
It receives tgL2 and the pixel numbers CntL1 and CntL2.
Then, the evaluation value calculation described above is performed (step S3),
The error amount with respect to the optimum exposure amount is calculated (step S
4). Further, based on this error amount, the mechanical iris 11, the AGC circuit 4, and the electronic shutter of the CCD image pickup unit 2 are controlled to perform exposure control (step S5). In addition,
The electronic shutter does not function as the shutter operation during long-time exposure, and can be performed only by the mechanical iris 11.

Next, in the short-time exposure control shown in FIG. 10, the threshold Th is set in step S11, and the threshold Th is set in step S11.
Intensity integrated value Int from OPD 18B for short-time exposure at 12
gS1, IntgS2 and the number of pixels CntS1, CntS2
To receive. Then, the evaluation value calculation described above is performed (step S13), and the error amount with respect to the optimum exposure amount is calculated (step S14). Further, the electronic shutter of the CCD image pickup unit 2 is controlled based on this error amount to perform exposure control (step S15). That is, during short-time exposure, exposure control is performed only by the electronic shutter. In this way, by differentiating the combination of control elements used for exposure control during long-time exposure and during short-time exposure, optimal exposure control can be performed for each.

By using the algorithm as described above, even when the image is captured in the backlight or over-illumination state, which has conventionally been inconvenient, the high-luminance and low-luminance imaged images are converged at appropriate convergence positions. Therefore, it is possible to perform imaging with a high dynamic range. The embodiment described above is a preferred specific example of the present invention,
Although various technically preferable limitations are given, the present invention is not limited to such specific examples, and various modifications can be made without departing from the gist of the present invention.

[0025]

As described above, according to the camera system of the present invention, the automatic exposure control means for independently performing the exposure control during the short-time exposure and the exposure control during the long-time exposure is provided. Optimum automatic exposure control can be performed for high-luminance and low-luminance subjects, and high-luminance and low-luminance images can be converged at appropriate convergence positions. Becomes Therefore, appropriate long-time exposure control and short-time exposure control are possible in imaging in all situations, and even when imaging is performed in backlight conditions, over-forward lighting conditions, etc. A low-luminance imaged image can be obtained, and high dynamic range shooting can be effectively performed.

Further, according to the automatic exposure control method of the present invention, the automatic exposure control step for independently performing the exposure control during the short-time exposure and the exposure control during the long-time exposure is provided. It is possible to perform optimal automatic exposure control for the image pickup for low brightness and the image pickup for low brightness subject, and it is possible to converge the imaged video for high brightness and the imaged video for low brightness at appropriate convergence positions. Therefore, appropriate long-time exposure control and short-time exposure control are possible in imaging in all situations, and even when imaging is performed in backlight conditions, over-forward lighting conditions, etc. A low-luminance imaged image can be obtained, and high dynamic range shooting can be effectively performed.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a camera system according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an example of a situation at the time of shooting by the camera system shown in FIG.

FIG. 3 is an explanatory diagram showing a state in which images taken by long-time exposure and short-time exposure are combined by the camera system shown in FIG.

FIG. 4 is an explanatory diagram showing a relationship between subject brightness and a system output value.

FIG. 5 is an explanatory diagram showing a convergence point of long / short exposure control.

6 is an explanatory diagram showing specifications of OPD used in the camera system shown in FIG. 1. FIG.

FIG. 7 is an explanatory diagram showing an example of a histogram at a long-time exposure control convergence point.

FIG. 8 is an explanatory diagram showing an example of a histogram at a short-time exposure control convergence point.

9 is a flowchart showing an example of a long-time exposure control operation of the camera system shown in FIG.

10 is a flowchart showing an example of a short-time exposure control operation of the camera system shown in FIG.

[Explanation of symbols]

1 ... Lens, 2 ... CCD image pickup unit, 3 ... CDS circuit, 4 ... AGC circuit, 5 ... A / D converter, 6 ...
... DSP, 6A ... signal processing unit, 6B ... exposure processing unit,
7A ... Long exposure signal processing unit, 7B ... Short exposure signal processing unit, 7C ... Switch, 7D ... Synthesis unit, 8 ...
D / A converter, 9 ... TG, 10 ... Microcomputer, 11 ... Mechanical iris, 12 ... Drive motor, 13 ... Internal bus, 14 ... Internal bus interface, 15 ... CPU, 16 ... ROM , 17 ... RA
M, 18A, 18B ... OPD.

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 2H002 CC01 DB25 GA06                 5C022 AB03 AB17 AC42 AC52 AC69                       AC74

Claims (16)

[Claims] 1. A solid-state image pickup device having an electronic shutter function performs image pickup for a high-luminance subject and image pickup for a low-luminance subject, and synthesizes two types of video signals obtained by each image pickup to obtain a wide dynamic range. In the camera system that outputs the imaged image, the exposure control during short-time exposure for performing the image pickup for the high-luminance subject and the exposure control during the long-time exposure for performing the image pickup for the low-luminance subject are independent. A camera system having automatic exposure control means for performing the above. 2. The automatic exposure control means includes short-time exposure control means for performing exposure control during short-time exposure and long-time exposure control means for performing exposure control during long-time exposure. The camera system according to claim 1. 3. The automatic exposure control means is means for performing automatic exposure control by controlling a plurality of types of control elements for controlling the exposure amount and the gain of the image pickup signal in the solid-state image pickup element, and the exposure during the short time exposure. 2. The camera system according to claim 1, wherein the control and the exposure control at the time of long-time exposure are performed by control by different combinations of the plurality of control elements. 4. The camera system according to claim 3, wherein the plurality of control elements include a mechanical shutter unit, an electronic shutter unit, and an automatic gain control unit. 5. The automatic exposure control means controls the exposure by the mechanical shutter means, the electronic shutter means, and the automatic gain control means during long-time exposure, and controls the exposure by the electronic shutter means during short-time exposure. The camera system according to claim 4, which is characterized in that. 6. An optical detection means for detecting the integrated luminance value and the number of pixels of each of a high-luminance portion and a low-luminance portion partitioned by a predetermined threshold on a histogram showing the luminance distribution of a photographed image, The automatic exposure control means separately obtains the evaluation value for long-time exposure control and the evaluation value for short-time exposure control using the optical detection means, and performs long-time exposure control and short-time exposure control based on each evaluation value. The camera system according to claim 1, wherein: 7. The property of exposure control convergence is changed by changing an evaluation value calculation coefficient between the evaluation value for long-time exposure control and the evaluation value for short-time exposure control. Camera system. 8. The camera system according to claim 7, wherein the evaluation value calculation coefficient is changed in accordance with emphasis on high luminance components and emphasis on low luminance components. 9. A solid-state image pickup device having an electronic shutter function is used to pick up an image of a high-luminance subject and an image of a low-luminance subject, and synthesize two types of video signals obtained by each image pickup to obtain a wide dynamic range. In the automatic exposure control method of the camera system for outputting the imaged image of, the exposure control at the time of short-time exposure for performing the image pickup for the high-luminance subject and the time for the long-time exposure for performing the image pickup for the low-luminance subject An automatic exposure control method comprising an automatic exposure control step of performing exposure control independently. 10. The automatic exposure control step includes a short-time exposure control step for performing exposure control during short-time exposure, and a long-time exposure control step for performing exposure control during long-time exposure. The automatic exposure control method according to claim 9. 11. The automatic exposure control step is a step of performing automatic exposure control by controlling a plurality of types of control elements for controlling an exposure amount and a gain of an image pickup signal in a solid-state image pickup element. 10. The automatic exposure control method according to claim 9, wherein the control and the exposure control at the time of long-time exposure are performed by control by different combinations of the plurality of control elements. 12. The automatic exposure control method according to claim 11, wherein the plurality of control elements include a mechanical shutter unit, an electronic shutter unit, and an automatic gain control unit. 13. The automatic exposure control step comprises the mechanical shutter means, electronic shutter means, and
13. The automatic exposure control method according to claim 12, wherein exposure control is performed by the automatic gain control means, and exposure control is performed by the electronic shutter means during short-time exposure. 14. The optical detection means for detecting the luminance integrated value and the number of pixels of each of the high-luminance portion and the low-luminance portion partitioned by a predetermined threshold on the histogram showing the luminance distribution of the photographed image. In the automatic exposure control step, the long-time exposure control evaluation value and the short-time exposure control evaluation value are separately obtained using the optical detection means, and the long-time exposure control is performed based on each evaluation value. 14. The automatic exposure control method according to claim 13, wherein short-time exposure control is performed. 15. The property of exposure control convergence is changed by changing an evaluation value calculation coefficient between the evaluation value for long-time exposure control and the evaluation value for short-time exposure control. Automatic exposure control method. 16. The automatic exposure control method according to claim 15, wherein the evaluation value calculation coefficient is changed in accordance with emphasis on high luminance components and emphasis on low luminance components.