JP2007288805A - Camera system and automatic exposure control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform optimum AE (automatic exposure control) to pickup a high luminance object and a low luminance object. <P>SOLUTION: As a method for outputting picked-up video of a wide dynamic range, two of the image pickup for the high luminance object and the image pickup for the low luminance object are performed and two video are mixed and output. Then, an exposure processing part 6B of a DSP 6 is provided with a switch 7C switching a signal route for long time exposure and short time exposure, a long time exposure signal processing part 7A performing a signal processing at the long time exposure, a short time exposure signal processing part 7B performing the signal processing at the short time exposure and a mixing part (MIX) 7D mixing signals of the respective exposure processing parts 7A and 7B. Then, by OPDs (optical detection circuits) 18A and 18B provided in the respective exposure signal processing parts 7A and 7B respectively, a luminance distribution is obtained independently at the long time exposure and at the short time exposure respectively and the optimum exposure control is performed. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention performs imaging for a high-luminance subject and imaging for a low-luminance subject by using a solid-state imaging device having an electronic shutter function, and synthesizes two types of video signals obtained by the respective imaging so as to have a wide dynamic range. The present invention relates to a camera system that outputs a captured image and an automatic exposure control method.

Conventionally, for example, a camera system with an AE (automatic exposure control) function using a CCD image sensor has been provided.
By the way, AE (automatic exposure control) in this type of camera system obtains a value obtained by integrating luminance components of an image by an OPD (optical detection circuit), and calculates a luminance integrated value at the time of proper exposure as a reference luminance integrated value (hereinafter, referred to as “integrated luminance”). Reference: ref).
Then, the mechanical shutter (iris), electronic shutter, AGC (automatic gain control) gain, etc. are controlled so that the current luminance integral value (Intg) is equal to the reference, and the system gain is increased or decreased to achieve proper exposure. To be obtained.

On the other hand, as a method for outputting captured images with a wide dynamic range, a camera system has been proposed in which imaging is performed twice for high-luminance subjects and imaging for low-luminance subjects, and the two images are combined and output. Yes.
As a conventional AE (automatic exposure control) in such a camera system, the reference for the high-luminance subject imaging AE is set lower and the reference for the low-luminance subject imaging AE is set higher. It was supported by an algorithm.

  However, as in the above-described conventional example, in the method corresponding to the high-intensity subject imaging AE and the low-intensity subject imaging AE only by changing the reference, for example, when imaging is performed in a backlight state or an over-order light state. In other words, the picked-up image does not converge at an appropriate convergence position for high luminance and low luminance.

  Accordingly, an object of the present invention is to perform optimum AE (automatic exposure control) for high-brightness subject imaging and low-brightness subject imaging. It is an object of the present invention to provide a camera system and an automatic exposure control method that can converge at a convergence position.

  In order to achieve the above object, the present invention performs imaging for a high-luminance subject and imaging for a low-luminance subject using a solid-state imaging device having an electronic shutter function, and synthesizes two types of video signals obtained by each imaging. In a camera system that outputs a wide dynamic range imaged image, exposure control during short exposure for performing imaging for the high-brightness subject and long-time exposure for performing imaging for the low-brightness subject It has automatic exposure control means for performing exposure control independently.

  In addition, the present invention performs imaging for a high-brightness subject and imaging for a low-brightness subject using a solid-state imaging device having an electronic shutter function, and synthesizes two types of video signals obtained by the respective imaging to provide a wide dynamic range. In the automatic exposure control method of the camera system that outputs the captured image of the above, the exposure control at the time of short exposure for performing the imaging for the high brightness subject and the long exposure at the time of performing the imaging for the low brightness subject An automatic exposure control step for performing exposure control independently is provided.

The camera system of the present invention is provided with automatic exposure control means for independently performing exposure control at the time of short exposure and exposure control at the time of long exposure. Optimal automatic exposure control can be performed with imaging, and a high-brightness captured image and a low-brightness captured image can be converged at an appropriate convergence position.
Therefore, for example, even in the case of imaging in a backlighting state or over-ordering state, it is possible to obtain a high-brightness imaged image and a low-brightness imaged image by appropriate exposure control, enabling high dynamic range shooting to be performed effectively. It becomes.

Further, in the automatic exposure control method of the present invention, an automatic exposure control step for independently performing exposure control during short-time exposure and exposure control during long-time exposure is provided. Optimal automatic exposure control can be performed with the imaging for the luminance object, and the high-luminance imaging image and the low-luminance imaging image can be converged at an appropriate convergence position.
Therefore, for example, even in the case of imaging in a backlighting state or over-ordering state, it is possible to obtain a high-brightness imaged image and a low-brightness imaged image by appropriate exposure control, enabling high dynamic range shooting to be performed effectively. It becomes.

Embodiments of a camera system and an automatic exposure control method according to the present invention will be described below.
FIG. 1 is a block diagram illustrating a configuration example of a camera system according to an embodiment of the present invention.
This camera system controls a lens 1 that collects imaging light, a CCD imaging unit 2 that images a subject, a CDS (correlated double sampling) circuit 3 that removes noise from the imaging signal, and the gain of the imaging signal. AGC (automatic gain control) circuit 4 that performs conversion, an A / D converter 5 that converts an analog signal into digital data, a DSP (digital signal processor) 6 that performs signal processing, and a D / A that converts digital data into an analog signal Controls the converter 8, a TG (timing generator) 9 that generates necessary timing signals and supplies them to each part, a mechanical iris (aperture mechanism) 11 as a mechanical shutter mechanism, a drive motor 12 for the mechanical iris, and the entire system A microcomputer (μ-COM) 10 and an internal bus 13.

Further, the DSP 6 is provided with an exposure processing unit 6B for performing signal processing for AE (automatic exposure control) in front of the signal processing unit 6A for performing original signal processing, and the exposure processing unit 6B includes A switch 7C that switches a signal path between long exposure and short exposure, a long exposure signal processing unit 7A that performs signal processing during long exposure, and a short exposure signal that performs signal processing during short exposure. It has a processing unit 7B and a combining unit (MIX) 7D that combines the signals of the exposure processing units 7A and 7B.
Further, the long exposure signal processing unit 7A and the short exposure signal processing unit 7B are provided with OPDs (optical detection circuits) 18A and 18B, respectively, so that luminance distribution (histogram) is output independently. It has become.
The microcomputer 10 is provided with an internal bus interface (I / F) 14, a CPU 15, a ROM 16, and a RAM 17.

The operation of the camera system according to this embodiment will be described below.
For example, as shown in FIG. 2, the present camera system is used in a situation with a wide dynamic range, such as when a person 22 is photographed with a camera 23 against a window 21 of a building 20 where direct sunlight enters. It is possible to perform effective exposure control in imaging.
In this camera system, the picked-up image passes through the lens 1 and is photoelectrically converted to a CCD image pickup unit 2 to be an electric signal. Then, the signal is sampled through the CDS circuit 3 and controlled to have an appropriate gain by the AGC circuit 4, and converted into a digital signal by the A / D converter 5.

This digital signal is signal-processed by the DSP 6, but in this example, an exposure processing unit 6B is provided in front of the signal processing unit 6A to enable imaging with a high dynamic range in order to enable imaging with a wide dynamic range. Two exposures (time exposure imaging) and low-luminance subject imaging (long exposure imaging) are performed, and two images are combined and output.
Referring to the example of FIG. 2, imaging for high-luminance subjects (short-time exposure imaging) corresponds to imaging outside the window in FIG. 2, for example, and imaging for low-luminance subjects (long-time exposure imaging) is, for example, This corresponds to indoor imaging in FIG.
Then, an image as shown in FIG. 3A is captured by high-luminance subject imaging (short-time exposure imaging), and an image as shown in FIG. 3B is captured by low-luminance subject imaging (long-time exposure imaging). Is imaged.

Therefore, by synthesizing these images, it is possible to obtain an image having a wide dynamic range as shown in FIG.
This is a switch 7C in the DSP 6, so that the low-brightness subject imaging signal is sent to the long-time exposure signal processing unit 7A for processing, and the high-brightness subject imaging signal is sent to the short-time exposure processing unit 7B for processing. Execute by switching in time division.
Then, the synthesis unit 7D combines the low-brightness subject captured video created by the long exposure signal processing and the high brightness subject captured video created by the short exposure signal processing, and outputs a wide dynamic range captured video.
Thereafter, the signal processing unit 6A in the DSP 6 performs signal processing, the D / A converter 8 converts it into an analog video signal, and outputs it to the subsequent stage.

  The software for executing such an algorithm is executed by the microcomputer 10, and the CPU 15 reads the program on the ROM 16, and sends necessary commands and data to each unit via the I / F 14 and the internal bus 12. Each process is performed by supplying.

Next, exposure amount optimization control using OPD (optical detection circuits) 18A and 18B, which is a feature of this system, will be described.
4 is an explanatory diagram showing the relationship between the subject brightness and the system output value, and FIG. 5 is an explanatory diagram showing the convergence point of the long / short-time exposure control. The vertical axis represents the output level, and the horizontal axis represents the subject brightness level (absolute Value).
FIG. 6 is an explanatory diagram showing the specifications of the OPD used in this example. FIG. 6A shows a histogram, and FIG. 6B shows a circuit configuration.
FIG. 7 is an explanatory diagram showing an example of a histogram at a long-time exposure control convergence point, and FIG. 8 is an explanatory diagram showing an example of a histogram at a short-time exposure control convergence point.
FIG. 9 is a flowchart showing an example of the long-time exposure control operation, and FIG. 10 is a flowchart showing an example of the short-time exposure control operation.

The automatic exposure control method of this system is a system in which exposure information can be obtained from different imaging states by the long exposure OPD 18A and the short exposure OPD 18B of the DSP 6, and long exposure control and short exposure control It provides an algorithm that performs with good performance.
First, as shown in FIG. 3A, the appropriate exposure on the short-time exposure side is adjusted to the outdoor standard (high luminance portion).
This corresponds to the dotted line portion (high luminance portion) in the signal (2) when exposure control is performed with the standard exposure reference value in FIG. 4, and the output limit value of the system is exceeded as it is, and it is output as whiteout. End up.
Therefore, in the short-time exposure control, the system gain should be lowered to obtain an output like the signal (3).

On the other hand, as shown in FIG. 3B, the appropriate exposure on the long exposure side is adjusted to the indoor reference (low luminance portion).
This may be the signal (2) in FIG. 4, but the system gain should be increased to obtain an output signal such as the signal (3) in order to clearly output a darker portion.
As a result, as shown in FIG. 5, appropriate long exposure signal (4) and short exposure signal (5) are obtained, and when they are combined, a wide dynamic range is achieved within the output limit value range of the conventional system. A captured image can be output.

In this system, in order to enable such appropriate exposure control, as shown in FIG. 6A, in the luminance distribution, the regions a1, a2, and a3 divided by the threshold values Th1 and Th2, respectively. OPDs (optical detection circuits) 18A and 18B capable of detecting the luminance integral values Intg1, Intg2, and Intg3 and the respective pixel numbers Cnt1, Cnt2, and Cnt3 are prepared.
As shown in FIG. 6B, each OPD 18A, 18B has two comparators 31, 32 that receive a luminance signal (Y output) and compare it with thresholds Th1, Th2, a luminance signal (Y output), and each comparator 31. , 32 for integrating the comparison outputs from each of them, and the luminance integrated values Intg1, Intg2, Intg3 and the number of pixels Cnt1, Cnt2, Cnt3 are calculated from the output values of the Σcircuits 33, 34, 35 respectively. Subtractors 36, 37, and 38 are provided.

As shown in FIG. 7, the histogram example at the time of proper exposure in the long exposure control is such that the histogram of the lower luminance part than the threshold is uniform and the high luminance part is saturated. When a value obtained by integrating the luminance of the entire screen at the time of such a histogram is used as an evaluation value, a problem occurs. Therefore, in order to obtain an evaluation value with a weight on the integrated value of the low luminance part, the following equation (1) or (2) is obtained. Use.
Long-time exposure control evaluation value = k × (Intg1 / Cnt1) + (1−k) × (Intg2 / Cnt2) (1)
Or
Long-time exposure control evaluation value = (k × Intg1 + (1−k) × (Intg2) / (k × Cnt1 + (1−k) × (Cnt2)) (2)
However, the variable k in this case is 0.5 <k ≦ 1.

On the other hand, in the short-time exposure control, since the histogram as shown in FIG. 8 is obtained in the converged state, the following equation (3) or (4) is used to obtain an evaluation value weighted on the integral value of the high luminance part.
Short-time exposure control evaluation value = k × (Intg1 / Cnt1) + (1−k) × (Intg2 / Cnt2) (3)
Or
Short-time exposure control evaluation value = (k × Intg1 + (1−k) × (Intg2) / (k × Cnt1 + (1−k) × (Cnt2)) (4)
However, the variable k in this case is 0 ≦ k <0.5.
By using the above as evaluation values, appropriate control can be performed in each of the long-time exposure control and the short-time exposure control.
Note that the evaluation value calculation coefficient as described above can be changed according to high-luminance component emphasis and low-luminance component emphasis.

Next, the procedure of long-time exposure control and short-time exposure control will be described with reference to the flowcharts of FIGS.
First, in the long-time exposure control shown in FIG. 9, the threshold value Th is set in step S1, and the luminance integrated values IntgL1 and IntgL2 and the pixel numbers CntL1 and CntL2 are received from the long-time exposure OPD 18A in step S2.
Then, the above-described evaluation value calculation is performed (step S3), and an error amount with respect to the optimum exposure amount is calculated (step S4).
Further, based on the error amount, the mechanical iris 11, the AGC circuit 4, and the electronic shutter of the CCD image pickup unit 2 are controlled to perform exposure control (step S5). The shutter operation during long exposure can be performed only with the mechanical iris 11 without functioning the electronic shutter.

Next, in the short-time exposure control shown in FIG. 10, the threshold value Th is set in step S11, and the luminance integrated values IntgS1 and IntgS2 and the pixel numbers CntS1 and CntS2 are received from the short-time exposure OPD 18B in step S12.
Then, the above-described evaluation value calculation is performed (step S13), and an error amount with respect to the optimum exposure amount is calculated (step S14).
Further, based on this error amount, the electronic shutter of the CCD image pickup unit 2 is controlled to perform exposure control (step S15). That is, during short exposure, exposure control is performed only with an electronic shutter.
In this way, by varying the combination of control elements used for exposure control during long exposure and short exposure, optimal exposure control can be performed for each.

By using the algorithm as described above, it is possible to converge high-luminance and low-luminance captured images at appropriate convergence positions even in the case where imaging has been performed in a conventional backlight or over-order lighting state. It is possible to perform imaging with a high dynamic range.
The embodiment described above is a preferable specific example of the present invention, and various technically preferable limitations are given, but the present invention is not limited to such a specific example. Various modifications can be made without departing from the spirit of the present invention.

As described above, according to the camera system of the present invention, the automatic exposure control means for independently performing the exposure control at the time of short exposure and the exposure control at the time of long time exposure is provided. Optimal automatic exposure control can be performed by imaging and imaging for a low-luminance subject, and the high-luminance imaging image and the low-luminance imaging image can be converged at an appropriate convergence position.
Therefore, it is possible to perform appropriate long exposure control and short exposure control with imaging in all situations. A low-luminance captured image can be obtained, and high dynamic range imaging can be performed effectively.

In addition, according to the automatic exposure control method of the present invention, an automatic exposure control step for independently performing exposure control during short-time exposure and exposure control during long-time exposure is provided. In addition, it is possible to perform optimal automatic exposure control with imaging for a low-luminance subject, and it is possible to converge a high-luminance imaging image and a low-luminance imaging image at an appropriate convergence position.
Therefore, it is possible to perform appropriate long exposure control and short exposure control with imaging in all situations. A low-luminance captured image can be obtained, and high dynamic range imaging can be performed effectively.

It is a block diagram which shows the structure of the camera system by embodiment of this invention. It is explanatory drawing which shows an example of the situation at the time of imaging | photography with the camera system shown in FIG. It is explanatory drawing which shows a mode that the image | video image | photographed by the long exposure and the short exposure with the camera system shown in FIG. 1 is synthesize | combined. It is explanatory drawing which shows the relationship between a subject brightness | luminance and a system output value. It is explanatory drawing which shows the convergence point of long and short time exposure control. It is explanatory drawing which shows the specification of OPD used with the camera system shown in FIG. It is explanatory drawing which shows the example of a histogram in a long time exposure control convergence point. It is explanatory drawing which shows the example of a histogram in a short time exposure control convergence point. 3 is a flowchart showing an example of a long-time exposure control operation of the camera system shown in FIG. 1. 3 is a flowchart showing an example of a short time exposure control operation of the camera system shown in FIG. 1.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Lens, 2 ... CCD imaging part, 3 ... CDS circuit, 4 ... AGC circuit, 5 ... A / D converter, 6 ... DSP, 6A ... Signal processing part, 6B ... Exposure processing part , 7A: long exposure signal processing unit, 7B: short exposure signal processing unit, 7C ... switch, 7D ... synthesis unit, 8 ... D / A converter, 9 ... TG, 10 ... microcomputer 11 ... Mechanical iris, 12 ... Drive motor, 13 ... Internal bus, 14 ... Internal bus interface, 15 ... CPU, 16 ... ROM, 17 ... RAM, 18A, 18B ... OPD.

Claims (16)

A solid-state image sensor with an electronic shutter function performs imaging for high-luminance subjects and imaging for low-luminance subjects, and synthesizes two types of video signals obtained by each imaging to output a wide dynamic range captured image In the camera system that
Automatic exposure control means for independently performing exposure control during short-time exposure for performing imaging for the high-luminance subject and exposure control during long-time exposure for performing imaging for the low-luminance subject;
A camera system characterized by that.   2. The automatic exposure control means includes short-time exposure control means for performing exposure control during short-time exposure and long-time exposure control means for performing exposure control during long-time exposure. Camera system.   The automatic exposure control means is means for performing automatic exposure control by controlling a plurality of types of control elements for controlling the exposure amount and the gain of the imaging signal in the solid-state imaging device. 2. The camera system according to claim 1, wherein the exposure control at the time is performed by control based on different combinations of the plurality of control elements.   4. The camera system according to claim 3, wherein the plurality of control elements include mechanical shutter means, electronic shutter means, and automatic gain control means.   The automatic exposure control means performs exposure control by the mechanical shutter means, electronic shutter means, and automatic gain control means during long-time exposure, and performs exposure control by the electronic shutter means during short-time exposure. 4. The camera system according to 4.   On the histogram showing the luminance distribution of the photographed video, it has optical detection means for detecting the respective luminance integral values and the number of pixels in the high luminance part and the low luminance part partitioned by a predetermined threshold, and the automatic exposure control means The optical detection means is used to separately obtain a long-time exposure control evaluation value and a short-time exposure control evaluation value, and perform long-time exposure control and short-time exposure control based on each evaluation value. The camera system according to claim 1.   7. The camera system according to claim 6, wherein a property of exposure control convergence is changed by changing an evaluation value calculation coefficient between the long-time exposure control evaluation value and the short-time exposure control evaluation value.   8. The camera system according to claim 7, wherein the evaluation value calculation coefficient is changed according to emphasis on a high luminance component and emphasis on a low luminance component. A solid-state image sensor with an electronic shutter function performs imaging for high-luminance subjects and imaging for low-luminance subjects, and synthesizes two types of video signals obtained by each imaging to output a wide dynamic range captured image In an automatic exposure control method for a camera system,
An automatic exposure control step for independently performing exposure control during short-time exposure for performing imaging for the high-luminance subject and exposure control during long-time exposure for performing imaging for the low-luminance subject;
And an automatic exposure control method.   10. The automatic exposure control step includes a short exposure control step for performing exposure control during short exposure and a long exposure control step for performing exposure control during long exposure. Automatic exposure control method.   The automatic exposure control step is a step of performing automatic exposure control by controlling a plurality of types of control elements that control the exposure amount and the gain of the imaging signal in the solid-state imaging device, and the exposure control and long-time exposure during the short-time exposure. The automatic exposure control method according to claim 9, wherein the exposure control at the time is performed by control based on different combinations of the plurality of control elements.   12. The automatic exposure control method according to claim 11, wherein the plurality of control elements include mechanical shutter means, electronic shutter means, and automatic gain control means.   The automatic exposure control step is characterized in that exposure control is performed by the mechanical shutter means, electronic shutter means, and automatic gain control means during long-time exposure, and exposure control is performed by the electronic shutter means during short-time exposure. 12. The automatic exposure control method according to 12.   The camera system has optical detection means for detecting the respective integrated luminance values and the number of pixels in the high luminance portion and the low luminance portion partitioned by a predetermined threshold on the histogram showing the luminance distribution of the captured video, The automatic exposure control step obtains a long-time exposure control evaluation value and a short-time exposure control evaluation value separately using the optical detection means, and the long-time exposure control and the short-time exposure control based on each evaluation value. The automatic exposure control method according to claim 13, wherein:   15. The automatic exposure control method according to claim 14, wherein a property of exposure control convergence is changed by changing an evaluation value calculation coefficient between the evaluation value for long-time exposure control and the evaluation value for short-time exposure control. .   16. The automatic exposure control method according to claim 15, wherein the evaluation value calculation coefficient is changed according to emphasis on high luminance components and emphasis on low luminance components.

Images