JPH11258654A - Shutter speed controller of camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To secure appropriate exposure in accordance with the luminance of a subject being a processed object and to improve accuracy in image processing by controlling the shutter speed of a camera so that section length on a low luminance side to the 1st index value of an integrated value obtained by adding the frequency of the histogram of luminance distribution may be equal to the section length on a high luminance side to a 2nd index value. SOLUTION: The histogram for controlling a shutter is formed by dividing the luminance distribution in a sample image into every specified luminance gradation, allotting the index value of i=0 to (n) to each section and setting the index value as the axis of abscissa and the frequency of each index value as the axis of ordinate. By using the section length D' to the section index value D on the low luminance side of the historgram and the section length B' to the section index value B on the high luminance side, a shutter control speed value (t) is found by t=t0+K.(D'-B') assuming that the present shutter speed control value is t0 and a control gain is K, and the shutter speed is controlled so that the section length D' may be equal to the section length B'.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera shutter speed control device capable of obtaining a proper exposure without being affected by the luminance distribution of a captured image.

[0002]

2. Description of the Related Art In recent years, in mobile objects such as automobiles and small helicopters, a technology for processing a captured image by mounting a camera and performing ambient environment recognition and self-position recognition processing has been actively employed.

In the technique of performing recognition processing using this captured image, a factor that greatly affects the processing result is a shutter speed (exposure time) of a camera. Conventionally, a method of controlling the shutter speed is shown in FIG. As shown in
The method of controlling the shutter speed by obtaining the average luminance of the image, or the distribution (A) of the lower luminance side of the image as shown in FIG.
A method is known in which the shutter speed is controlled based on the difference in brightness in an image such that the distribution on the high-luminance side (the frequency on A ') is equal to the distribution on the high-luminance side.

[0004]

However, in the method of controlling the shutter speed using the average luminance of an image, when an object having an extremely large difference in luminance is displayed on a screen, both light and dark cannot be properly exposed. In some cases, the method of controlling the shutter speed by using the difference between the light and dark ends effectively works on an image with a wide dynamic range. May not be.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and has as its object to provide a camera shutter speed control device that can always secure an appropriate exposure without being affected by the luminance distribution of a captured image. And

[0006]

According to the first aspect of the present invention,
Means for sampling a captured image of the camera, creating a luminance histogram by segmenting the luminance distribution in the captured image,
When the integrated value obtained by sequentially adding the frequencies of the histogram in the lower luminance side exceeds the first set value, the section index value is set to the first set value.
Means for obtaining as a second index value a section index value obtained when an integrated value obtained by sequentially adding the frequencies of the histogram on the higher luminance side exceeds a second set value; and Means for controlling the shutter speed of the camera so that the section length on the low luminance side up to the first index value is equal to the section length on the high luminance side up to the second index value. It is characterized by the following.

According to a second aspect of the present invention, in the first aspect of the present invention, the section length on the low luminance side up to the first index value and the section length on the high luminance side up to the second index value are set. A difference from the section length is obtained, and a value obtained by multiplying the difference by a control gain is added to a current shutter speed control value to obtain a shutter speed control value at the next imaging timing.

That is, in the camera shutter speed control apparatus according to the present invention, a histogram is created by dividing the luminance distribution of a sampled captured image by an index value, and the frequency of the histogram is added in order from the lower luminance side. When the section index value obtained when the value exceeds the first set value is obtained as the first index value, and the integrated value obtained by sequentially adding the frequencies of the histogram on the side of higher luminance exceeds the second set value Is obtained as a second index value. Then, the section length on the low luminance side up to the first index value is equal to the section length on the high luminance side up to the second index value,
Controls the shutter speed of the camera. In this case, the shutter speed control value is a value obtained by multiplying the difference between the section length on the low luminance side up to the first index value and the section length on the high luminance side up to the second index value by a control gain. Is added to the current shutter speed control value.

[0009]

Embodiments of the present invention will be described below with reference to the drawings. 1 to 3 relate to an embodiment of the present invention. FIG. 1 is a block diagram of a stereo image processing system, FIG. 2 is an explanatory diagram showing a histogram on a lower luminance side, and FIG. 3 is a histogram on a higher luminance side. FIG.

FIG. 1 shows a stereo image processing system for processing an image taken by a stereo camera 1 which is synchronized with each other and is composed of two cameras having variable shutter speeds and incorporating an image sensor such as a charge-coupled device (CCD). Is mounted on a mobile object such as a car or a helicopter, and is used as a part of a system for recognizing the surrounding environment and the self-position.

The stereo camera 1 includes a main camera that captures a reference image when one of the cameras performs stereo processing,
The other camera is provided with a predetermined base line length on a camera stay (not shown) as a sub camera that captures a comparative image in stereo processing. The two images captured by the stereo camera 1 are stored in a gain control amplifier (GCA) 2
The signal is input to an A / D converter 3 via an analog I / F (interface) 2 having a, and is converted into a digital image having a predetermined luminance gradation. Further, the image digitally converted by the A / D converter 3 is input to an image processing unit 10 having various functions for digital processing such as image correction / conversion and camera control.

Various functions for digital signal processing in the image processing section 10 can be realized by, for example, logic design of a gate using an FPGA. The functions of the image processing unit 10 include the GCA 2a and the A
D / A for generating control voltage for / D converter 3
The D / A controller 11 for controlling the converter 4 and the gain and offset of the image signal for correcting the mutual image signal variation caused by the difference in the output characteristics of the image sensors of the two cameras constituting the stereo camera 1. Look-up table (LUT; ROM) for changing the amount
12) image data corrected by the LUT 12, shading correction data from a shading correction memory 5 including a ROM and a RAM for storing shading correction data for correcting luminance distortion of each pixel. , A log (Log) conversion table (comprising a ROM) 14 for logarithmically converting the light and dark portions of the image to adjust the sensitivity, an address controller 15, and a shading correction memory 5. Controller 16 that controls the transfer of shading correction data to the camera, shutter speed of stereo camera 1 (exposure time of electronic shutter to CCD)
There are a camera controller 17 for performing control and the like, an external I / F 18 for externally rewriting various parameters inside the FPGA, and reading them from the outside.

The image corrected and converted by the image processing unit 10 is stored in an image memory 20 for storing a main image and a sub image, and a shutter control original image memory 21 for storing image data for camera shutter control. A distance image is generated by stereo matching between the main image and the sub image in the stereo matching circuit 22 and stored in the distance image memory 23.

The image data stored in the shutter control original image memory 21 and the distance image memory 23 is read into a recognition processing unit 30 which performs various recognition processes. The recognition processing unit 30 realizes high-speed processing by, for example, a multiprocessor configuration in which a plurality of RISC processors are operated in parallel, controls the shutter speed of the stereo camera 1, and converts the distance image into three-dimensional information. For example, a process of recognizing a surrounding environment and a self-position of a moving body such as an automobile or a helicopter is performed.

In the above-described stereo processing system, when the system is started by turning on the power, hardware components are initialized, and variable values such as a specification constant value of the stereo camera 1 and various processing variable values are initialized. At the same time, the shading controller 16 transfers the shading correction data of the shading correction memory 7 from the ROM to the RA.
M.

Next, a horizontal synchronizing signal and a vertical synchronizing signal based on the system clock are supplied to each camera of the stereo camera 1 as external synchronizing signals, and are written from the recognition processing unit 30 to the camera controller 17 via the external I / F 18. A shutter trigger signal is output to each camera of the stereo camera 1 according to the shutter speed control value.

An image picked up by the stereo camera 1 is input to an analog I / F 2, and the analog I / F 2
G according to the control voltage from D / A converter 4
The gain of the image signal is adjusted by the CA 2 a, and the variation in the gain of the two cameras is adjusted and output to the A / D converter 3.

The A / D converter 3 adjusts the offset difference between the image signals of the two cameras by the reference voltage from the D / A controller 4 to convert the analog image into a predetermined luminance gradation (for example, a gray scale of 256 gradations). ) To a digital image. The gain control voltage of the GCA 2a and the reference voltage of the A / D converter 4 are supplied from the recognition processing unit 30 via the external I / F 18 to the D / F.
This is a voltage value written to the A controller 11.

The digital image data whose gain / offset has been matched for each camera from the analog I / F 2 through the A / D converter 3 is converted into an L signal in the image processing unit 10.
The address is given to the UT 12 as address data, and the LUT 12
And the shading correction data from the shading correction memory 7 transferred by the instruction from the shading controller 16 are multiplied by the multiplier 14. As a result, the gain and the offset are more strictly matched for each camera, and the decrease in brightness due to the shading phenomenon occurring in the optical system of each camera is corrected.

Next, the image data output from the multiplier 13 is logarithmically converted by the log conversion table 14 to remove the influence of noise components proportional to brightness, and the main memory and sub-image data are stored in the image memory 20. Stored. The image data stored in the image memory 20 is
For example, it is data for every four horizontal lines, and at the same time, image data for one horizontal line is stored in the shutter control original image memory 21.

The original image data stored in the image memory 20 is transferred to the stereo matching circuit 22. The stereo matching circuit 22 calculates a city block distance for each predetermined small area (for example, an area of 4 × 4 pixels) between the main image and the sub-image, finds a mutual correlation therebetween, specifies a corresponding area, and A pixel shift (= parallax) generated according to the distance to the object is obtained. Then, three-dimensional image information (distance image) in which the perspective information to the target obtained from the deviation amount is quantified is generated and written into the distance image memory 23. Note that the processing for generating a distance image from a captured image of a stereo camera is described in Japanese Patent Application Laid-Open No. H11-11 / 1999.
No. 4099 discloses this in detail.

In the recognition processing unit 30, the distance image memory 23
Reads the distance image data from the camera and performs various recognition processes, reads the original image data of the shutter control original image memory 21 as a sample image, and creates a histogram of the luminance in the sample image to be optimal for stereo matching and recognition processing. A shutter speed control value that can obtain a proper image is determined.

The luminance histogram can be created by software processing in the recognition processing section 30, but may be created by a dedicated circuit. The use of the histogram processing circuit (a processing circuit in which a plurality of self-adding tone / frequency modules are connected in parallel) according to -247320 makes it possible to greatly improve the processing speed.

In the histogram for shutter control, the luminance distribution in the sample image is divided into sections at predetermined luminance gradations, and index values i = 0 to n are assigned to each section.
This index value is created on the horizontal axis, and the frequency for each index value is created on the vertical axis. For example, when an analog captured image is converted into a digital image with 256 luminance levels, a histogram is created by dividing the image into 16 levels, and the histogram distribution on the dark side as shown in FIG. The shutter speed is controlled based on the distribution of the histogram on the bright side as shown in FIG.

Specifically, as shown in the following equation (1),
A section index value (first index value) D on the horizontal axis when an integrated value (sum of histograms) Hdark obtained by integrating the frequency h [i] from the lower luminance side exceeds a first set value is obtained. ,below
As shown in equation (2), the section index value (second index value) on the horizontal axis when the sum Hbright of the histogram obtained by integrating the frequency h [i] from the side with higher luminance exceeds the second set value Find B.
Here, equation (1) is the sum of i = 0 to D, and equation (2) is i = n to
This is the sum up to B, where D ≦ n and 0 ≦ B.

Hdark = Σh [i] (1) Hbright = Σh [i] (2) Then, the section index value D on the lower luminance side of the histogram
The section length D ′ (D ′ = D; the magnitude of the vector indicated by the arrow in FIG. 2) and the section length B ′ (B ′ = n−B) up to the section index value B on the higher luminance side ; The current shutter speed control value is t0 and the control gain is K, the shutter speed control value t is obtained by the following equation (3), and the section length D The shutter speed is controlled so that 'and the section length B' become equal.

T = t0 + K · (D′−B ′) (3) In this case, the first set value and the second set value may be usually the same value. The ratio of each other can be changed accordingly. That is, by controlling the shutter speed so that the area of the histogram on the dark side is equal to the area of the histogram on the bright side by setting the first set value and the second set value to be the same value, Even in the case of an image of uniform brightness where the brightness of the subject to be processed is distributed between dark and bright areas, the dark and bright areas are less likely to fall into the dead zone as with the conventional shutter speed control based on contrast. Shutter speed (exposure) can be adjusted to an appropriate shutter speed according to the brightness of the subject by average photometry, thereby preventing mismatching in stereo matching and erroneous recognition in recognition processing. Can be.

On the other hand, in the case of a dark subject or a bright subject, the shutter speed is controlled by the conventional average photometry so that the entire image has an intermediate brightness, and stereo matching with the subject is not performed. Although there was a problem that accurate distance information could not be obtained due to being appropriate and erroneous recognition occurred, in the case of a dark subject, the first set value was made larger than the second set value, and in the case of a bright subject By setting the second set value larger than the first set value, weighted average photometry is performed.
Accurate distance information can be obtained by appropriate stereo matching as an appropriate shutter speed (exposure) according to the luminance of the subject, and the reliability of recognition processing can be improved.

[0029]

As described above, according to the present invention, a histogram is created by dividing the luminance distribution of a sampled picked-up image into sections, and the integrated value obtained by sequentially adding the frequencies of the histogram in the lower luminance side is the first value. 1 is obtained as a first index value, and an integrated value obtained by sequentially adding the frequencies of the histogram in a higher luminance side exceeds a second set value. The value is obtained as a second index value, and the shutter speed of the camera is set so that the section length on the low luminance side up to the first index value is equal to the section length on the high luminance side up to the second index value. To control, it is possible to secure an appropriate exposure according to the brightness of the subject to be processed without being affected by the brightness distribution of the captured image,
Excellent effects are obtained, such as the accuracy in image processing can be improved.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a stereo image processing system.

FIG. 2 is an explanatory diagram showing a histogram on a lower luminance side;

FIG. 3 is an explanatory diagram showing a histogram on a higher luminance side;

FIG. 4 is an explanatory diagram of conventional shutter speed control based on image luminance averaging.

FIG. 5 is an explanatory diagram of a conventional shutter speed control based on a difference in brightness between images.

[Description of Signs] 2, 3 Camera 10 Image processing unit 17 Camera controller 30 Recognition processing unit Hdark, Hbright Histogram sum (integrated value) D ... First index value B ... Second index value

Claims (2)

[Claims] 1. A means for sampling a captured image of a camera, segmenting a luminance distribution in the captured image to create a luminance histogram, and an integrated value obtained by sequentially adding the frequencies of the histogram in a lower luminance side. The section index value when the set value exceeds 1 is set to the first
Means for calculating the index value of the histogram, and the section index value when the integrated value obtained by sequentially adding the frequencies of the histogram in the higher luminance side exceeds the second set value is set to the second value.
Means for determining the index value of the camera, and the camera of the camera such that the section length on the low-luminance side up to the first index value is equal to the section length on the high-luminance side up to the second index value. Means for controlling a shutter speed. 2. A difference between a section length on the low luminance side up to the first index value and a section length on the high luminance side up to the second index value is obtained, and a control gain is calculated based on the difference. 2. The camera shutter speed control device according to claim 1, wherein the multiplied value is added to a current shutter speed control value to obtain a shutter speed control value at the next imaging timing.