JP2012010282A - Imaging device, exposure control method, and exposure control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve visibility of a following vehicle in the evening and at night.SOLUTION: An image including a visible area component and a near-infrared component of a subject is generated in an imaging element 14. The image of the visible area component and near-infrared component obtained by imaging is divided into a plurality of evaluation areas in an evaluation value calculation part 24 and an average value (evaluation value) in each evaluation area is calculated. A high brightness area detection part 28 determines that the evaluation area is a high brightness area when evaluation values of the visible area component and near-infrared component in the evaluation area exceed a predetermined level. A light source determination part 32 determines the kind of a light source contained in the picked-up image based on results of comparing evaluation values of the visible area component with the near-infrared component in the high brightness area. An exposure control part 30 controls the electronic shutter speed of the imaging element 14 and the gain value in an AGC circuit 18 according to the kind of the light source determined by the light source determination part 32.

Description

  The present invention relates to an imaging apparatus, an exposure control method, and an exposure control program used for an in-vehicle video camera or the like.

  In recent years, in-vehicle camera devices have been widely used for the purpose of supporting safe driving of automobiles. For example, the driver can easily identify the presence or absence of the following vehicle by displaying the images of the side and rear of the vehicle captured by the in-vehicle camera on the display installed in the driver's seat.

  In such an in-vehicle camera device, the average value of the luminance level of the image obtained by imaging or the peak value of the luminance level is calculated, and the electronic value of the image sensor is adjusted so that the average value or the peak value approaches the set value. It is preferable to control the shutter speed and the gain to the video signal output from the image sensor.

  Here, if a high-luminance object such as the sun or a headlight of a succeeding vehicle enters the imaging range of the imaging device, the main subject is underexposed or halation occurs in the image obtained by imaging. The image obtained by imaging becomes unclear. In such a case, it becomes difficult for the driver of the vehicle to determine the presence or absence of the following vehicle, and the purpose of assisting safe driving is lost. Such a tendency is particularly noticeable when the sun and headlights appear in a situation where the main subject is dark, such as in the evening or at night.

  Therefore, in the automatic exposure control apparatus described in Patent Document 1, an image obtained by imaging is analyzed to detect a region corresponding to the peak luminance, and a representative luminance value corresponding to the area of the peak luminance region is calculated. An invention is disclosed in which exposure control is performed so that the calculated representative luminance value follows the target value. As a result, for example, when the area of the high-brightness area is large, the bright part of the screen becomes appropriate brightness, and when the area of the high-brightness area is small, the dark part of the screen becomes bright. By performing each exposure control, it is possible to perform appropriate exposure control even in a scene with excessive light and a scene with backlight.

Japanese Patent No. 3477936

  In the exposure control apparatus described in the above-mentioned patent document, exposure control is performed according to the area of a high-luminance portion (bright portion) in an image obtained by imaging. For this reason, for example, when the following vehicle is away from the host vehicle in the evening or at night, it is determined that the area of the headlight in the image obtained by imaging is small (the area of the high luminance region is small), and the average luminance The exposure is controlled so as to increase. In this case, the luminance of the area of the headlight part becomes excessively high, and an image that is crushed with halation is output.

  In this case, for a driver who has seen an image including a white area, what is the number of the following vehicles, what is the type of the following vehicle (whether it is a two-wheeled vehicle or a four-wheeled vehicle), and the following vehicle is the own vehicle? It has been difficult to identify how far away it is from the camera, making it difficult to use as a vehicle-mounted camera.

  The present invention has been made in view of the above background, and an object thereof is to provide an imaging apparatus, an exposure control method, and an exposure control program that can improve the visibility of a rear image in the evening or at night.

  The imaging apparatus of the present invention captures a subject and generates an image including a visible region component and a near-infrared region component of the subject as a captured image, and the visible region component and the near red color from the captured image. High luminance that determines whether or not a high luminance region is included in the captured image based on an evaluation value calculation unit that calculates an evaluation value of an outer region component and an evaluation value of the visible region component and / or the near infrared region component When a high-luminance area is included in the area detection unit and the captured image, the evaluation value of the near-infrared region component and the visible area component in the high-luminance area is compared, and the type of light source included in the captured image And an exposure control unit that controls the imaging conditions of the imaging unit according to the type of the light source determined by the light source determination unit.

  With this configuration, since exposure control is performed according to the type of light source included in the captured image, the presence or type of the following vehicle can be easily identified even in a relatively dark subject such as evening or night.

  In the imaging apparatus of the present invention, the evaluation value calculation unit divides the captured image into a plurality of evaluation regions, and calculates an average value of the visible region component and the near infrared region component in each evaluation region as an evaluation value. The high-intensity region detection unit determines that the evaluation region is a high-intensity region when the evaluation value of the visible region component and / or the near-infrared region component exceeds a predetermined level, and the light source determination unit Then, the evaluation value of the near-infrared region component and the visible region component in the evaluation region determined to be a high luminance region is compared to determine the type of light source included in the captured image.

  With this configuration, since the presence / absence and type of the light source are determined using only the evaluation values of the visible region component and the near infrared region component in each evaluation region, the size of the memory required for the determination can be reduced. Further, since the presence / absence of the high luminance region is determined from the evaluation value in the evaluation region of a certain size, it is possible to eliminate the possibility of erroneously determining noise or the like as the high luminance region.

  The imaging apparatus of the present invention includes a full-screen average value calculation unit that calculates an average value of the visible region component and the near infrared region component in the entire captured image, and the exposure control unit includes the light source determination unit, When it is determined that a specific light source is not included in the captured image, the average value of the visible region component and / or the near infrared region component calculated by the full screen average value calculation unit approaches a predetermined level. The imaging condition in the imaging unit is controlled.

  With this configuration, appropriate exposure control can be performed even when a specific light source is not reflected in the captured image.

  In the imaging device according to the aspect of the invention, the light source determination unit may be configured such that the ratio of the evaluation value of the near infrared region component to the evaluation value of the visible region component in the high luminance region is equal to or greater than a first predetermined value. Is determined to be a headlight, and the exposure control unit controls imaging conditions in the imaging unit so that halation due to the headlight is reduced. With this configuration, the shape of the headlight of the subsequent vehicle displayed on the display becomes clear, so that the presence and type of the subsequent vehicle can be easily identified.

  In the imaging apparatus according to the aspect of the invention, the light source determination unit may be configured such that when the ratio of the evaluation value of the near infrared region component to the evaluation value of the visible region component is within a predetermined range in the high luminance region, the light source is The exposure control unit determines that the evaluation value of the visible region component and / or the near infrared region component in a region of the captured image excluding the high luminance region approaches a set value. The imaging condition in the imaging unit is controlled. With this configuration, even when the sun enters the display screen, the brightness of the main subject can be maintained at a certain level, and the presence and type of the following vehicle can be easily identified.

  In the imaging device according to the aspect of the invention, the light source determination unit may include the light source when a ratio of the evaluation value of the near infrared region component to the evaluation value of the visible region component is less than a second predetermined value in the high luminance region. Is determined to be a headlight, and the exposure control unit controls imaging conditions in the imaging unit so that halation due to the headlight is reduced. With this configuration, the shape of the headlight of the subsequent vehicle displayed on the display becomes clear, so that the presence and type of the subsequent vehicle can be easily identified.

  The exposure control method of the present invention includes an imaging step of imaging a subject and generating an image including a visible region component and a near-infrared region component of the subject as a captured image, and the visible region component and the near region from the captured image. An evaluation value calculating step for calculating an evaluation value of an infrared region component and a high value for determining whether or not a high-luminance region is included in the captured image based on the evaluation value of the visible region component and / or the near infrared region component When a luminance region is detected and a high-luminance region is included in the captured image, the evaluation values of the near-infrared region component and the visible region component in the high-luminance region are compared to determine the light source included in the captured image. A light source determination step of determining a type, and an exposure control step of controlling an imaging condition in the imaging unit according to the type of the light source determined in the light source determination unit. With this configuration, the same object as described above can be achieved.

  The exposure control program of the present invention is directed to a computer that captures an image of a subject and generates an image including a visible region component and a near-infrared region component of the subject as a captured image, and from the captured image, the visible region An evaluation value calculating step for calculating an evaluation value of a component and the near-infrared region component, and whether or not a high-luminance region is included in the captured image based on the evaluation value of the visible region component and / or the near-infrared region component The high-intensity region detection step for determining the image, and when the captured image includes a high-intensity region, the evaluation values of the near-infrared region component and the visible region component in the high-intensity region are compared, and the captured image is A light source determination step for determining a type of the included light source, and an exposure control step for controlling an imaging condition in the imaging unit according to the type of the light source determined in the light source determination unit. That. With this configuration, the same object as described above can be achieved.

  According to the present invention, the captured image including the visible region component and the near infrared region component of the subject is generated, and the exposure control is performed according to the type of the light source included in the captured image. And the type can be easily identified.

1 is a functional block diagram illustrating a configuration of an imaging apparatus according to the present invention. Explanatory drawing which shows an example of the arrangement | sequence of the optical filter of an image pick-up element. The graph which shows an example of the spectral characteristic of the optical filter of FIG. The graph which shows an example of the spectral characteristic of a halogen type headlight. The graph which shows an example of the spectral characteristic of a discharge type headlight. The graph which shows an example of the spectral characteristic of sunlight. The flowchart which shows operation | movement of an exposure control apparatus.

  Hereinafter, an in-vehicle imaging device to which an imaging device of the present invention is applied will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an in-vehicle imaging device according to an embodiment of the present invention. The vehicle-mounted imaging device 10 includes a photographing lens 12, an imaging element 14, a CDS circuit 16, an AGC circuit 18, and an A / D converter 20 that are disposed on the optical path.

  The image sensor 14 is a CCD or CMOS solid-state image sensor, for example, and converts an optical image transmitted through the photographing lens 12 into an electric signal (image signal). The imaging device 14 includes a plurality of photodiodes arranged in a matrix, for example, and the optical filter 15 shown in FIG. The optical filter 15 in FIG. 2 includes an optical filter that transmits red (R), green (G), and blue (B) visible light, and an optical filter that mainly transmits light in the near-infrared (IR) region. Are regularly arranged. FIG. 3 shows an example of the spectral characteristics of the optical filter.

  By providing such an optical filter 15, light intensity information of visible region components (R, G, B) and near-infrared region components can be obtained for each pixel constituting the imaging device. A portion composed of these R, G, B, and IR optical filters corresponds to one pixel.

  Further, the exposure time per frame, that is, the electronic shutter speed, in the photodiode constituting the image sensor 14 is controlled by a control signal output from an exposure control unit 30 described later.

  In FIG. 1, the CDS circuit 16 removes reset noise included in the analog signal output from the image sensor 14. The AGC circuit 18 amplifies the analog signal from the CDS circuit 16 at a magnification set based on a control signal input from an exposure control unit 30 described later. The A / D converter 20 converts the analog signal amplified in the AGC circuit 18 into a digital signal.

  The in-vehicle imaging device 10 includes a signal processing unit 22, an evaluation value detection unit 24, a full screen average calculation unit 26, a high brightness area detection unit 28, and an exposure control unit 30. The signal processing unit 22 performs luminance signal (Y) and color difference signal (UV) on the video signal (R, G, and B color signals) digitally converted by the A / D converter 20 by known signal processing. Generates and outputs an aperture signal. The signal processing unit 22 performs known video signal processing such as OB (Optical Black) subtraction for subtracting a signal component corresponding to the black level, white balance adjustment, and noise processing.

  The evaluation value detection unit 24 receives a video signal of visible region components (R, G, B) and near-infrared region components (IR) output from the A / D converter 20, and outputs a plurality of images obtained by imaging. Is divided into smaller areas (evaluation areas). The size of the evaluation area is not limited as long as the presence or absence of a dotted high-luminance area can be identified, and can be, for example, an area of several tens to several hundreds of pixels. The evaluation value detection unit 24 calculates the average value of the visible region component and the near infrared region component in each evaluation region, and outputs it as an evaluation value. Information on the average value (evaluation value) of the visible region component and the near infrared region component in each evaluation region is sent to the full screen average calculation unit 26 and the high luminance region detection unit 28.

  The entire screen average calculation unit 26 calculates the average of the values of the visible light component and the near-infrared region component in each evaluation region, which are input from the evaluation value detection unit 24, so that the visible region component in the entire captured image The average value of near infrared region components is calculated.

  The high brightness area detection unit 28 determines whether or not the evaluation value input from the evaluation value detection unit 24 exceeds a predetermined level, and when there is an evaluation area that exceeds the level, visible light in the evaluation area is displayed. The average value (evaluation value) of the component and the near infrared region component is extracted. When there are a plurality of evaluation areas exceeding a predetermined level, an average value of the evaluation values in the plurality of evaluation areas may be calculated. The evaluation value of the visible region component may be an average value of each color component (R, G, B), or one may be selected from the three evaluation values of R, G, B.

  As shown in FIG. 4, when the light source is a halogen type headlight, the intensity of the near infrared region component is stronger than the visible region component. For this reason, when this type of headlight is reflected in an image obtained by imaging, the value of the intensity ratio between the visible region component and the near infrared region component generally takes a large value.

  As shown in FIG. 5, when the light source is a discharge type (or LED type) headlight, it has peaks at a plurality of wavelengths in the visible range, but contains almost no components in the near infrared range. For this reason, the value of the intensity ratio between the visible range component and the near infrared range component generally takes a small value.

  As shown in FIG. 6, when the light source is the sun, it has a peak intensity in the vicinity of 550 nm, and has a certain level of light intensity from the visible range to the near infrared range. For this reason, when sunlight is reflected in an image obtained by imaging, the value of the intensity ratio between the visible component and the near-infrared component takes an intermediate value between the two types of headlights.

  The exposure control unit 30 calculates an electronic shutter value in the image sensor 14 and a gain value in the AGC circuit 18, and sends control signals for determining the electronic shutter value and the gain value to the image sensor 14 and the AGC circuit 18. Perform exposure control. In addition, the exposure control unit 30 includes a light source determination unit 32 that determines the type of light source reflected in the captured image.

  The light source determination unit 32 outputs the type of the light source reflected in the captured image based on the intensity ratio of the visible region component and the near infrared region component in the high luminance region, which is output from the high luminance region detection unit 28, and the vehicle headlight. Whether it is the sun or the sun. The exposure control unit 30 determines an exposure control method according to the determined type of light source, and performs exposure control.

  These exposure control processes in the present embodiment may be realized by a program stored in a memory or HDD provided in the imaging apparatus.

  The operation of the in-vehicle imaging device having the above configuration will be described with reference to the flowchart of FIG. First, an image behind the vehicle is picked up by the image pickup device 14, signal correction processing is performed on the obtained image pickup signal, and then converted into digital image data by the A / D converter 20 and output (S11). . The generated image data is sent to the signal processing unit 22 where it is converted into a luminance / color difference signal, subjected to image processing such as white balance and gamma correction, and then output and displayed on a monitor (not shown).

  The image data from the A / D converter 20 is also output to the evaluation value detection unit 24. In this evaluation value detection unit 24, the image indicated by the image data is divided into a plurality of evaluation regions, and in each evaluation region, the average value of the intensity of the visible region component and the near infrared region component is calculated and output as an evaluation value. (S12). The information of the calculated evaluation value is sent to the full screen average value calculation unit 26, and the average value of the visible region component and the near infrared region component in the captured whole screen is calculated.

  The high luminance area detection unit 28 detects the presence or absence of an evaluation area (high luminance area) in which the evaluation value of the visible region component or near infrared region component exceeds a predetermined level (S13). When the high brightness area detection unit 28 determines that a high brightness area exists, the high brightness area detection unit 28 sends the average value (evaluation value) information of the visible region component and the near infrared region component in the high brightness region to the exposure control unit 30. .

The light source determination unit 32 of the exposure control unit 30 is based on the average value (intensity) information of the visible region component and the near infrared region component in the high luminance region,
X = (Infrared component intensity) / (Visible component intensity)
The intensity ratio X of the visible region component and the near infrared region component in the high luminance region is calculated (S14).

  Then, the light source determination unit 32 determines whether or not the calculated intensity ratio X is equal to or greater than a predetermined value A (S15). When the intensity ratio X is equal to or greater than the predetermined value A, the light source determination unit 32 determines that the high luminance region is due to the halogen type headlight because the intensity of the near infrared region component in the high luminance region is high. Then, the exposure control unit 30 determines the electronic shutter value of the image sensor 14 and the gain value of the AGC circuit 18 so that the average value of the visible region component in the high luminance region matches a predetermined target value (peak). Exposure control by detection (S16)). Thereby, it controls so that it may become a brightness level which can identify the shape of the headlight of a vehicle contained in a picked-up image, and the influence of the halation by a headlight can be suppressed.

  When it is determined that the intensity ratio X is smaller than the predetermined value A, the light source determination unit 32 determines whether or not the value of the intensity ratio X is a value between the predetermined value C and the predetermined value B (S17). ). If this determination result is positive (Y), the intensity of the near-infrared region component in the high luminance region has a certain level, so that the high luminance region is determined to reflect the sun. Then, the exposure control unit 30 performs exposure control by backlight correction detection so that the main subject is not underexposed due to sunlight (S18). The backlight correction detection means that the electronic shutter value and the gain value are determined so that the average value of the visible region component when the high luminance region in the screen is excluded matches the predetermined level. The average value of the visible region component when the high luminance region in the screen is excluded can be calculated from the average value information input from the full screen average calculating unit 26 and the high luminance region detecting unit 28.

  When the determination result of step S17 is NO (N), the light source determination unit 32 determines whether the intensity ratio X is smaller than the predetermined value D (S19). When the intensity ratio X is smaller than D, since the intensity of the near-infrared region component in the high luminance region is small, it is determined that the high luminance region is a discharge type headlight. Then, as in the case of the halogen type headlight, the exposure control unit 30 sets the electronic shutter value and the gain value so that the intensity value of the visible region component in the high luminance region matches a predetermined target value. Determine (S16).

  On the other hand, when it is determined that the intensity ratio X is greater than D (that is, the intensity ratio X is within the range from C to B, or within the range from D to C), the high-intensity region is a light source such as the sun or a headlight. It is determined that the main subject has a high luminance. Then, the exposure control unit 30 determines the electronic shutter value and the gain value so that the average luminance of the whole screen becomes a predetermined level based on the average luminance information obtained by the whole screen average calculation unit 26 (normal detection ( S20)).

  In addition, when it is determined that the image obtained by imaging does not include the high luminance area (when there is no output from the high luminance area detecting unit 28), the exposure control unit 30 includes the light source in the imaging range. Is determined to be included, and exposure control is performed by normal detection similar to step S20.

  In the above embodiment, the image obtained by imaging is divided into a plurality of evaluation regions, and the average value of the visible region component and the near infrared region component in each evaluation region is calculated as the evaluation value. For example, the values of the visible region component and the near infrared region component in all the pixels (or pixels extracted at a constant interval) of the captured image may be extracted as the evaluation values. However, the embodiment for calculating the evaluation value in each evaluation region is that the size of the memory for storing the evaluation value can be reduced, and the possibility that noise or the like is erroneously determined as the high luminance region can be eliminated. preferable.

  In the above embodiment, the high brightness area detection unit 28 calculates the average value of the signal components of the portion determined as the high brightness area. For example, a halogen type headlight and a discharge type headlight are mixed. The scene that is doing is also assumed. For this reason, based on only the average value of the visible light component and the near-infrared light component in the evaluation region where the average value is determined to be the highest in the portion determined as the high luminance region instead of the average value, The type may be determined.

  In the above embodiment, the exposure control is performed by identifying three types of light sources, ie, halogen type headlights, discharge type headlights, and the sun. Of course, the present invention is limited to these three types of light sources. However, the present invention can be applied to a plurality of types of light sources having different spectral characteristics in the visible region and the near infrared region.

  As described above, according to the imaging apparatus according to the present invention, the exposure control is performed by determining the type of the light source from the light intensity ratio of the visible light component and the near infrared component, so that an image with excellent visibility is output. For example, it is useful as a vehicle-mounted camera.

DESCRIPTION OF SYMBOLS 10 In-vehicle imaging device 14 Imaging element 18 AGC circuit 24 Evaluation value detection part 26 Full screen average calculation part 28 High-intensity area | region detection part 30 Exposure control part 32 Light source determination part

Claims (8)

An imaging unit that images a subject and generates an image including a visible region component and a near-infrared region component of the subject as a captured image;
An evaluation value calculation unit that calculates evaluation values of the visible region component and the near infrared region component from the captured image;
A high-luminance region detection unit that determines whether or not a high-luminance region is included in the captured image based on an evaluation value of the visible region component and / or the near-infrared region component;
Light source determination that determines the type of light source included in the captured image by comparing evaluation values of the near-infrared region component and the visible region component in the high-intensity region when the captured image includes a high-intensity region And
An exposure control unit that controls imaging conditions in the imaging unit according to the type of light source determined in the light source determination unit;
An imaging apparatus comprising: The evaluation value calculation unit divides the captured image into a plurality of evaluation regions, calculates an average value of the visible region component and the near infrared region component in each evaluation region as an evaluation value,
The high-luminance region detection unit determines that the evaluation region is a high-luminance region when the evaluation value of the visible region component and / or the near-infrared region component exceeds a predetermined level,
The light source determination unit determines the type of light source included in the captured image by comparing evaluation values of the near-infrared region component and the visible region component in the evaluation region determined to be a high luminance region. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. A total screen average value calculation unit that calculates an average value of the visible region component and the near infrared region component in the entire captured image,
The exposure control unit, when the light source determination unit determines that a specific light source is not included in the captured image, the visible region component and / or the near infrared region calculated by the full screen average value calculation unit The imaging apparatus according to claim 2, wherein imaging conditions in the imaging unit are controlled so that an average value of components approaches a predetermined level. The light source determination unit determines that the light source is a headlight when a ratio of the evaluation value of the near-infrared region component to the evaluation value of the visible region component in the high luminance region is equal to or greater than a first predetermined value. And
The imaging apparatus according to claim 2, wherein the exposure control unit controls imaging conditions in the imaging unit such that halation due to the headlight is reduced. The light source determination unit determines that the light source is the sun when the ratio of the evaluation value of the near infrared region component to the evaluation value of the visible region component in the high luminance region is within a predetermined range;
The exposure control unit sets an imaging condition in the imaging unit such that an average value of the visible region component and / or the near infrared region component in a region of the captured image excluding the high luminance region approaches a set value. The imaging apparatus according to claim 2, wherein the imaging apparatus is controlled. The light source determination unit determines that the light source is a headlight when a ratio of an evaluation value of the near infrared region component to an evaluation value of the visible region component in the high luminance region is less than a second predetermined value. And
The imaging apparatus according to claim 2, wherein the exposure control unit controls imaging conditions in the imaging unit such that halation due to the headlight is reduced. An imaging step of imaging a subject and generating an image including a visible region component and a near infrared region component of the subject as a captured image;
An evaluation value calculating step for calculating an evaluation value of the visible region component and the near infrared region component from the captured image;
A high-luminance region detection step for determining whether or not a high-luminance region is included in the captured image based on the evaluation values of the visible region component and / or the near-infrared region component;
Light source determination that determines the type of light source included in the captured image by comparing evaluation values of the near-infrared region component and the visible region component in the high-intensity region when the captured image includes a high-intensity region Steps,
An exposure control step for controlling imaging conditions in the imaging unit according to the type of light source determined in the light source determination unit;
An exposure control method comprising: An exposure control program for a computer
An imaging step of imaging a subject and generating an image including a visible region component and a near infrared region component of the subject as a captured image;
An evaluation value calculating step for calculating an evaluation value of the visible region component and the near infrared region component from the captured image;
A high-luminance region detection step for determining whether or not a high-luminance region is included in the captured image based on the evaluation values of the visible region component and / or the near-infrared region component;
Light source determination that determines the type of light source included in the captured image by comparing evaluation values of the near-infrared region component and the visible region component in the high-intensity region when the captured image includes a high-intensity region Steps,
An exposure control step for controlling imaging conditions in the imaging unit according to the type of light source determined in the light source determination unit;
A program characterized by having executed.