JP2002057939A - Image pickup device and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device that picks up a natural image corresponding to an image pickup subject to be photographed. SOLUTION: When it is discriminated that a target dynamic range is higher than a subject dynamic range, the opening/closing operation of an aperture 2 is controlled so as to match a mean luminance of an image with a desired luminance by a user. When it is discriminated that the subject dynamic range is higher than the target dynamic range, the opening/closing operation of the aperture 2 is controlled so as to match a difference between a maximum luminance of the target dynamic range and a maximum luminance of the subject dynamic range with a difference between a minimum luminance of the target dynamic range and a minimum luminance of the subject dynamic range. Thus, exposure control in response to the subject dynamic range can be conducted to allow the image pickup device to pick up a natural image corresponding to the image pickup subject.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus and a method therefor, and is suitably applied to, for example, a video camera for monitoring.

[0002]

2. Description of the Related Art Conventionally, a video which realizes a so-called wide dynamic range for generating an image having a wide dynamic range (an image expressed from a bright region to a dark region) by combining two types of images captured with different exposure amounts. There is a camera.

This video camera uses a solid-state image pickup device (CCD: Charge Coupled Device) as an image pickup device and uses an electronic shutter which is a function of the CCD as a method of picking up two types of images having different exposure amounts. A method of capturing two types of images in a time-sharing manner by changing the time is adopted.

In this method, charge is accumulated and read out in an arbitrary one-field period in the same manner as in normal imaging, and then charge accumulation and read-out are performed again using a vertical blanking period. Two types of images having different exposure times during a field period, that is, a long-time exposure image and a short-time exposure image are obtained.

In this way, the video camera obtains a long-time exposure image with high reproducibility in a dark portion of a subject and a short-time exposure image with high reproducibility in a bright portion of the subject. By combining the time-exposure image and the short-time exposure image, a synthesized image having a wide dynamic range is generated.

[0006]

Generally, in a video camera, the amount of incident light is adjusted by a diaphragm mechanism to reach a CCD in order to generate a natural image corresponding to a subject to be imaged. Exposure control for controlling the amount of light is performed.

At this time, in a video camera that realizes the above-described wide dynamic range, exposure control is performed based only on an exposure state obtained from a long-time exposure image out of a captured long-time exposure image and a short-time exposure image. Although a method has been considered, this method is still insufficient in obtaining a natural image corresponding to the subject to be imaged.

The present invention has been made in view of the above points, and has as its object to propose an imaging apparatus and a method thereof capable of generating a natural composite image corresponding to an imaging target.

[0009]

According to the present invention, a target dynamic range set in advance as a range of luminance values at which gradation can be recognized in accordance with a user's input operation, and an object to which the subject is actually placed are described. The subject dynamic range indicating the range of the brightness value of the captured image is compared, and when it is determined that the target dynamic range is larger than the subject dynamic range, the average brightness value of the image is adjusted to match the brightness value desired by the user. While controlling the opening and closing operation of the aperture, if it is determined that the subject dynamic range is larger than the target dynamic range, the difference between the maximum luminance value of the target dynamic range and the maximum luminance value of the subject dynamic range is determined. Match the difference between the minimum luminance value of the range and the minimum luminance value of the dynamic range of the subject. By controlling the opening and closing operation of the throttle way, it is possible to perform exposure control in accordance with the subject dynamic range.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings.

In FIG. 1, reference numeral 1 denotes the overall configuration of a video camera for surveillance, and incident light L1 incident from the subject side is incident on a diaphragm mechanism 2, and the diaphragm mechanism 2
After the light amount is adjusted by the
D: Charge Coupled Device) 3.

The CCD 3 photoelectrically converts the received incident light L1 to capture two images having different exposure amounts in a time-division manner. That is, CCD3
In any one field period, charge accumulation and readout are performed in the same manner as in normal imaging, and then charge accumulation and readout are performed again using the vertical blanking period, so that the exposure time within one field period is reduced. Two different images are generated.

In this way, the CCD 3 generates the long-time exposure image S1A and the short-time exposure image S1B, and sends them to the signal processing circuit 4. The signal processing circuit 4 combines the long-time exposure image S1A and the short-time exposure image S1B while sending the long-time exposure image S1A and the short-time exposure image S1B to the detection unit 5 as they are, thereby obtaining a signal having a wide dynamic range. A composite image S2 is generated and sent to the signal output circuit 6.

At this time, the signal processing circuit 6 integrates, for example, a coefficient corresponding to the ratio of the captured exposure amount to each image as a method of synthesizing the long-time exposure image S1A and the short-time exposure image S1B. A method of generating a composite image S2 by switching and outputting each image by threshold processing is adopted.

The signal output circuit 6 performs a predetermined image processing on the composite image S2 to execute NTSC (National T
The signal is converted into a television signal S3 of an elevision system (elevation system), which is output to an external monitor (not shown) and displayed.

By the way, as shown in FIG.
The long exposure image S1A and the short exposure image S1B supplied from the signal processing circuit 4 are input to the signal separation circuit 10. The signal separation circuit 10 separates the luminance signals from the long-time exposure image S1A and the short-time exposure image S1B to obtain a long-time exposure luminance signal S10A and a short-time exposure luminance signal S10B.
0A is sent to the bottom value detection circuit 11A and the integration circuit 12A, and the short-time exposure luminance signal S10B is sent to the peak value detection circuit 11B and the integration circuit 12B.

The bottom value detecting circuit 11A detects the minimum value of the luminance values of the pixels in one field period of the supplied long-time exposure luminance signal S10A and calculates this as the long-time exposure bottom value S11A. To the unit 14.

On the other hand, the peak value detection circuit 11B detects the maximum value of the luminance value of each pixel during one field period of the supplied short-time exposure luminance signal S10B,
This is sent to the arithmetic unit 14 as the short-time exposure peak value S11B.

The integrating circuit 12A integrates (adds) the luminance value of each pixel of the supplied long-time exposure luminance signal S10A for one field period, and obtains the resulting long-time exposure luminance signal integrated value S12A. It is sent to the operation unit 14.

On the other hand, the integrating circuit 12B integrates the luminance value of each pixel of the supplied short-time exposure luminance signal S10B for one field period, and calculates the resulting short-time exposure luminance signal integrated value S12B in the arithmetic unit 14 To send to.

By the way, in this video camera 1,
Various parameters set by the user operating the input unit 15 are input to the calculation unit 14 (FIG. 1). The parameters include the following.

That is, in the long-time exposure luminance signal S10A obtained by the detection unit 5, the blackout level indicating the minimum value of the luminance values at which the gradation of the image can be visually recognized is determined. Obtained short-time exposure luminance signal S10
B. The overexposure level indicating the maximum value among the luminance values at which the gradation of the image in B can be visually recognized, and the short-time exposure image S1 with respect to the exposure time of the long-time exposure image S1A
There is an exposure ratio indicating the ratio of the exposure time of B (that is, the value obtained by dividing the exposure time of the long exposure image S1A by the exposure time of the short exposure image S1B).

The operation unit 14 includes, as shown in FIG.
The blackout level S15A set in 5 is input to the dynamic range operation circuits 16A and 17A, the whiteout level S15B is input to the multiplier 18A and the dynamic range operation circuit 17B, and the exposure ratio S16 is further multiplied by the multipliers 18A and 18B. Input to 19.

The calculating section 14 inputs the long-time exposure bottom value S11A supplied from the detecting section 5 to the dynamic range calculating circuits 16B and 17A, and outputs the short-time exposure peak value S11B to the multiplier 18B and the dynamic range calculating circuit 17B. To enter.

Further, the arithmetic section 14 inputs the long-time exposure luminance signal integrated value S12A supplied from the detection section 5 to the adder 20, and multiplies the short-time exposure luminance signal integrated value S12B by the multiplier 1
Enter 9

The multiplier 18A is provided with a whiteout level S15B.
And the exposure ratio S16, and sends the multiplication result to the dynamic range calculation circuit 16A. The dynamic range calculation circuit 16A sets the exposure ratio S to the whiteout level S15B.
After dividing the value multiplied by 16 by the blackout level S15A, a logarithmic operation is performed to calculate a target dynamic range when performing exposure control, and this is sent to the exposure control unit 25 as a target dynamic range S20A. I do.

The multiplier 18B calculates a short-time exposure peak value S1.
1B is multiplied by the exposure ratio S16, and the multiplication result is sent to the dynamic range calculation circuit 16B. The dynamic range calculation circuit 16B calculates a value obtained by multiplying the short-time exposure peak value S11B by the exposure ratio S16 to the long-time exposure bottom value S11.
After dividing by A, the logarithmic operation is performed to calculate the dynamic range of the actually captured subject, and the calculated dynamic range is set as the subject dynamic range S20B.
5

After dividing the long-time exposure bottom value S11A by the blackout level S15A, the dynamic range calculation circuit 17A performs logarithmic calculation to obtain a luminance value at which the gradation of an image can be visually recognized. A dynamic range representing a range difference between the minimum value among the minimum values and the minimum value among the luminance values of the actually picked-up subject is calculated, and the calculated dynamic range is transmitted to the exposure control unit 25 as a blackout error amount S21A.

The dynamic range calculation circuit 17B divides the short-time exposure peak value S11B by the overexposure level S15B and then performs logarithmic calculation to obtain a luminance value at which the gradation of an image can be visually recognized. A dynamic range representing a range difference between the maximum value among the maximum values and the maximum value among the luminance values of the actually captured subject is calculated, and the calculated dynamic range is transmitted to the exposure control unit 25 as a whiteout error amount S21B.

The multiplier 19 multiplies the short-time exposure luminance signal integrated value S12B by the exposure ratio S16, and sends the multiplication result to the adder 20. The adder 20 adds the integrated value of the long-time exposure luminance signal S12A and the value obtained by multiplying the integrated value of the short-time exposure luminance signal S12B by the exposure ratio S16, and sends the addition result to the divider 26.

By the way, the number of times of signal detection (ie, the total number of pixels) per field in the detector 5 is input to the multiplier 27.
Multiplies the number of signal detections per field of the detection unit 5 by 2, and sends the multiplication result to the divider 26.

The divider 26 calculates the sum of the integrated value of the long-time exposure luminance signal S12A and the value obtained by multiplying the integrated value of the short-time exposure luminance signal S12B by the exposure ratio S16. Is divided by twice to calculate the average value of the brightness values of the subject, and sends this to the exposure control unit 25 as the subject average value S22.

When the target dynamic range S20A and the subject dynamic range S20B are supplied from the arithmetic section 14, the exposure control section 25 performs an exposure control procedure R shown in FIG.
Execute T1. That is, in FIG.
Enters the exposure control procedure RT1, proceeds to step SP1, and determines whether the target dynamic range S20A is larger than the subject dynamic range S20B.

As a result, if a positive result is obtained in step SP1, this indicates that the dynamic range of the actually photographed subject is smaller than the dynamic range of the image in which the gradation can be visually recognized. At this time, the exposure control unit 25 proceeds to step SP2, and inputs the subject average value S22 supplied from the arithmetic unit 14 to the input unit 15 by the user in the same manner as a normal video camera that is not a video camera realizing a wide dynamic range. Exposure drive information S26 for driving the aperture mechanism 2 is generated to match the convergence target value S25 set according to the operation, that is, the luminance value desired by the user, and is sent to the exposure drive unit 30. .

On the other hand, if a negative result is obtained in step SP1, this means that the dynamic range of the actually picked-up object is larger than the dynamic range of the image in which the gradation can be visually recognized, that is, the image picked up. This means that it is impossible to obtain an exposure that allows the gradation to be recognized over the entire image of the subject, and at this time, the exposure control unit 25 determines in step SP3
Then, the difference between the blackout error amount S21A and the whiteout error amount S21B supplied from the calculation unit 14 is calculated.

Then, the exposure control section 25 proceeds to step S
Moving to P4, the aperture mechanism unit 2 is adjusted so that the difference between the black-out condition error amount S21A and the white-out condition error amount S21B becomes zero.
Is generated, and is transmitted to the exposure driving unit 30. Thus, the exposure control unit 25
Performs exposure control so that the long-time exposure bottom value S11A and the short-time exposure peak value S11B do not greatly deviate from the black underexposure level S15A and the white underexposure level S15B, respectively. The white area is evenly generated. In this way, the exposure control unit 25 performs the exposure control for each field, and when the exposure control of the current field is performed, returns to step SP1 and repeats the above-described processing, thereby exposing the next field. Execute control.

The exposure drive unit 30 includes a detection unit 5, an operation unit 14,
And an exposure control circuit block 31 together with the exposure control unit 25. When the exposure control unit 25 receives the exposure drive information S26, the exposure control unit 25 generates an exposure drive signal S27 corresponding to the exposure drive information S26, and outputs the exposure drive signal S27 to the aperture mechanism. Send it to unit 2. The aperture mechanism unit 2 controls the amount of light reaching the CCD 3 by controlling the opening and closing operation of the aperture and adjusting the amount of incident light.

In the above configuration, the video camera 1
Generates a synthesized image S2 having a wide dynamic range by synthesizing a long-time exposure image S1A and a short-time exposure image S1B imaged with different exposure amounts, and sends the image to an external monitor for display.

In the exposure control circuit block 31,
A blackout level S1 indicating a minimum luminance value at which gradation can be recognized, which is set according to a user's input operation on the input unit 15
5A, an overexposure level S15B indicating the maximum luminance value at which the gradation can be recognized, and an exposure ratio S16 indicating the exposure time ratio of the long-time exposure image S1A and the short-time exposure image S1B. By calculating a target dynamic range as a target, it is possible to set a target dynamic range S20A in which any gradation can be recognized according to the characteristics of the CCD 3 and the signal processing circuit 4.

The exposure control circuit block 31 includes a long-time exposure bottom value S11A indicating the minimum luminance value of each pixel constituting the long-time exposure image S1A and a short-time exposure image S1A.
Based on the short-time exposure peak value S11B indicating the maximum luminance value among the pixels constituting 1B and the exposure ratio S16, the subject dynamic range S20
Calculate B.

The exposure control circuit block 31 also provides a blackout error indicating a range difference between the minimum luminance value at which the gradation can be recognized and the minimum luminance value of the actual subject based on the long exposure bottom value S11A and the blackout level S15A. In addition to calculating the amount S21A, based on the short-time exposure peak value S11B and the overexposure level S15B, the overexposure error amount S21B indicating the range difference between the maximum luminance value at which the gradation can be recognized and the actual maximum luminance value of the subject is calculated. I do.

The exposure control circuit block 31 further includes a long-time exposure luminance signal integrated value S12A obtained by adding the luminance values of the respective pixels of the long-time exposure image S1A and a short-time value obtained by adding the luminance values of the respective pixels of the short-time exposure image S1B. Time exposure luminance signal integrated value S
Based on 12B and the exposure ratio S16, a subject average value S22 indicating the average value of the luminance values of the actually imaged subject is calculated.

The exposure control circuit block 31 compares the target dynamic range S20A with the subject dynamic range S20B. As a result, if it is determined that the target dynamic range S20A is larger than the subject dynamic range S20B, the subject average value S22 Is adjusted to match the convergence target value (that is, the luminance value desired by the user), but if it is determined that the subject dynamic range S20B is larger than the target dynamic range S20A, the blackout error amount S2
Exposure control is performed so that 1A and the whiteout error amount S21B match (that is, a blackout area and a whiteout area occur evenly).

As described above, the video camera 1 can execute the exposure control according to the dynamic range of the actually picked-up subject, and therefore can generate a natural composite image S2 corresponding to the picked-up subject. .

According to the above configuration, when it is determined that the target dynamic range S20A is larger than the subject dynamic range S20B, the exposure control is performed so that the subject average value S22 matches the luminance value desired by the user. On the other hand, when it is determined that the subject dynamic range S20B is larger than the target dynamic range S20A, the exposure control is performed so that the black-out area and the white-out area are equally generated, so that the captured object Exposure control according to the dynamic range can be performed, and thus a natural synthesized image S2 corresponding to the subject to be imaged can be generated.

In the above-described embodiment, when it is determined that the target dynamic range S20A is smaller than the subject dynamic range S20B, the difference between the blackout error amount S21A and the whiteout error amount S21B is zero.
We described the case where exposure control was performed so that
The present invention is not limited to this, and detects a pixel having a luminance value less than the blackout level S15A from the long-time exposure luminance signal S10A, counts the number of the pixels, and detects the whiteout level S15B or more from the short-time exposure luminance signal S10B. A pixel having a luminance value may be detected, the number of pixels may be counted, and exposure control may be performed so that the sum of these count values is minimized. In this case, the sum of the count values represents the area of a region in which gradation cannot be recognized, such as a blackened region or a whitened region in an image, and the sum of the count values is minimized. By performing the exposure control, it is possible to obtain an image in which the area where the gradation can be recognized is maximized.

In the above-described embodiment, the case where the present invention is applied to the video camera 1 for surveillance has been described. However, the present invention is not limited to this.
The present invention can be widely applied to various other imaging apparatuses such as a home video camera.

[0048]

As described above, according to the present invention, a target dynamic range set in advance as a range of luminance values that can be recognized in gradation according to a user's input operation, and a target dynamic range of an image in which a subject is actually captured. The subject dynamic range indicating the range of brightness values is compared, and if it is determined that the target dynamic range is larger than the subject dynamic range, the aperture opening / closing operation is performed so that the average brightness value of the image matches the brightness value desired by the user. If it is determined that the subject dynamic range is larger than the target dynamic range, the difference between the maximum luminance value of the target dynamic range and the maximum luminance value of the subject dynamic range and the minimum luminance value of the target dynamic range are controlled. Aperture opening / closing operation to match the difference between the minimum brightness value of the subject dynamic range By Rukoto, it can set the exposure control in accordance with the subject dynamic range may thus capture a natural image corresponding to the object to be imaged.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of an imaging device according to the present invention.

FIG. 2 is a block diagram illustrating a configuration of a detection unit.

FIG. 3 is a block diagram illustrating a configuration of a calculation unit.

FIG. 4 is a flowchart illustrating an exposure control procedure.

[Explanation of symbols]

1 ... video camera, 2 ... aperture mechanism, 3 ... CC
D, 4 ... signal processing circuit, 5 ... detection unit, 10 ... signal separation circuit, 11A ... bottom value detection circuit, 11B ... peak value detection circuit, 12A, 12B ... integration circuit, 14 ...
... Calculation unit, 15 ... Input unit, 16A, 16B, 17A,
17B: dynamic range calculation circuit, 18A, 18
B, 19, 20 ... adder, 26 ... divider, 25 ...
Exposure control unit, 27 Multiplier, 30 Exposure drive unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification code FI Theme coat ゛ (Reference) G06T 3/00 300 G06T 3/00 300 H04N 5/225 H04N 5/225 C 5/335 5/335 QF Terms (reference) 2H002 CC21 DB02 DB19 DB24 DB25 DB26 DB27 EB09 5B047 AA07 BB06 BC04 CA17 CB30 5B057 AA19 BA02 BA11 CE08 DC22 DC36 5C022 AA01 AB05 AB12 AB17 AC31 AC42 AC56 AC69 5C024 BX04 CX47 CX54 DX01 GYX H20X

Claims (8)

[Claims] An aperture for adjusting the amount of incident light to be incident; an imaging means for receiving the incident light to capture an image; and a brightness value range in which gradation can be recognized according to a user's input operation. The set target dynamic range is compared with a subject dynamic range indicating a range of luminance values of the image in which the subject is actually imaged, and when it is determined that the target dynamic range is larger than the subject dynamic range, While controlling the opening and closing operation of the aperture so that the average luminance value of the image matches the luminance value desired by the user, if the subject dynamic range is determined to be larger than the target dynamic range, the target dynamic range is determined. The difference between the maximum luminance value of the range and the maximum luminance value of the subject dynamic range, and the target dynamic range Imaging apparatus characterized by comprising a exposure control means for controlling the aperture of the opening and closing operation so as to match the difference between the minimum luminance value and the minimum luminance value of the subject dynamic range. 2. A combined image obtained by combining a first image captured at a first exposure time and a second image captured at a second exposure time shorter than the first exposure time. The image pickup apparatus according to claim 1, further comprising an image synthesizing unit that generates the image data. 3. A stop for adjusting the amount of incident light incident thereon, an imaging means for receiving the incident light and capturing an image, and a luminance value range which can be recognized in gradation according to a user's input operation. The set target dynamic range is compared with a subject dynamic range indicating a range of luminance values of the image in which the subject is actually imaged, and when it is determined that the target dynamic range is larger than the subject dynamic range, While controlling the opening / closing operation of the aperture so that the average luminance value of the image matches the luminance value desired by the user, if the subject dynamic range is determined to be larger than the target dynamic range, the image is captured. A pixel having a luminance value equal to or greater than the maximum luminance value of the target dynamic range in the image, And an exposure control means for controlling the opening / closing operation of the aperture so as to count pixels having a luminance value less than the minimum luminance value and to minimize the sum of these count values. . 4. A combined image obtained by combining a first image captured at a first exposure time and a second image captured at a second exposure time shorter than the first exposure time. The image pickup apparatus according to claim 3, further comprising an image synthesizing unit that generates the image. 5. A target dynamic range preset as a range of luminance values that can be recognized in gradation according to a user's input operation, and a subject dynamic range indicating a range of luminance values of an image of an actually captured subject. In comparison, when it is determined that the target dynamic range is larger than the subject dynamic range, the aperture opening / closing operation is controlled so that the average luminance value of the image matches the luminance value desired by the user. When it is determined that the subject dynamic range is larger than the target dynamic range, the difference between the maximum luminance value of the target dynamic range and the maximum luminance value of the subject dynamic range, the minimum luminance value of the target dynamic range, Opening and closing the aperture so that the difference from the minimum luminance value of the subject dynamic range matches Imaging method and controlling the work. 6. A combined image obtained by combining a first image captured at a first exposure time and a second image captured at a second exposure time shorter than the first exposure time. The imaging method according to claim 5, wherein is generated. 7. A target dynamic range preset as a range of luminance values at which gradation can be recognized in accordance with a user's input operation, and a subject dynamic range indicating a range of luminance values of an image of an actually captured subject. In comparison, when it is determined that the target dynamic range is larger than the subject dynamic range, the aperture opening / closing operation is controlled so that the average luminance value of the image matches the luminance value desired by the user. If it is determined that the subject dynamic range is larger than the target dynamic range, a pixel having a luminance value equal to or greater than the maximum luminance value of the target dynamic range in the captured image and a minimum luminance value of the target dynamic range Pixels having a luminance value less than or equal to each other, and minimizing the sum of these count values Imaging method characterized by controlling the opening and closing operation of the urchin the aperture. 8. A combined image obtained by combining a first image captured at a first exposure time and a second image captured at a second exposure time shorter than the first exposure time. The imaging method according to claim 7, wherein is generated.