JP2009098718A - Image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device capable of acquiring an image that is easy to see even if the image is an image including an object having a high luminance dynamic range. <P>SOLUTION: A plurality of images is generated from image data by the rear camera 10 to extract a light source from one of the generated plurality of images by performing the dynamic range compression. An image is generated by performing the dynamic range compression so that the luminance of a plurality of other images becomes a preset luminance that is lower than that of the image from which the light source is extracted. The light source part of the image from which the light source is extracted is replaced to images with different luminance in the plurality of other images to generate an image by suppressing luminance in a part where a luminance value of the pixel exceeds the saturation luminance to the predetermined luminance. Further, a region of the other vehicle is specified, and in the specified region of the other vehicle, the effect to suppress the luminance in a part where the saturation luminance exceeds the saturation luminance in the luminance saturation suppression image generation processing to the predetermined luminance is reduced. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus capable of clarifying a vehicle image in an imaging apparatus having a low dynamic range.

  In conventional image processing technology, a method to process and provide images to suppress saturation of pixel values (so-called whiteout) and lack of pixel values (so-called blackout) in landscape images with a wide dynamic range is proposed. Has been.

  Basically, in order to suppress overexposure, exposure control may be performed in accordance with the portion having the maximum luminance. However, in that case, the pixel values of other dark portions are insufficient, and the other black portions are completely blackened. Therefore, a method using a logarithmic sensitivity type camera having a very wide dynamic range can be considered. However, also in this case, when the luminance difference between the dark part and the bright part is several thousand times, image generation by min / max processing by simple histogram analysis (see FIG. 5) and tone curve adjustment (see FIG. 9). In such exposure control, an image in which a dark part is blacked out becomes an image, so that an image is generated so that the contrast of the dark part is not lost.

For example, a method of applying dynamic range compression only to a portion of a high illumination light component having a logarithmic characteristic of a Lin / Log camera has been proposed. That is, by taking a means for creating and synthesizing two kinds of images, an image that allows a bright scene and a dark scene to be clearly seen is obtained. As a result, even in an image including a scene with extremely different brightness (for example, tunnel entrance / exit), the contrast of the dark part and the contrast of the bright part can be seen well. That is, an image in which overexposure and underexposure are suppressed is generated (for example, see Patent Document 1).
JP 2006-21650 A

  However, in the above method, when the luminance change of the subject has a large dynamic range, an image including the subject cannot be obtained clearly. For example, when a night road is imaged, the scene has a point light source such as a streetlight having a local high brightness several thousand to several tens of thousands times the darkness of the surroundings and road surface reflection glare. Since the point light source is local in the image, even if dynamic range compression that compresses the low-frequency component is applied as in the above-described technique, the image at that location is subject to whiteout.

  In addition, as a method of suppressing the overexposure, for example, using the above-described composition, an image is separately generated with an extreme setting in which even a local high-intensity portion is difficult to overexposure, and only the overexposed portion is displayed with the picture. It is conceivable to synthesize in the form of replacement.

  In this case, since the brightness value of the light source part is certainly kept low, whiteout can be suppressed, but there is a problem that the image is difficult to see. For example, when considering an in-vehicle side rear monitoring monitor camera, the headlight of an overtaking vehicle that was originally blown out is suppressed, so that it becomes inconspicuous in the image. As a result, it becomes difficult to understand the presence or absence of an overtaking vehicle at the time of a lane change, which is a need in terms of the function of the camera.

  The present invention has been made in view of these problems, and an object thereof is to provide an image processing apparatus capable of obtaining an easy-to-see image even for an image including a subject having a luminance with a large dynamic range.

  The image processing apparatus according to claim 1, which has been made to solve such a problem (5: In this section, in order to facilitate understanding of the invention, “the best mode for carrying out the invention” is necessary as necessary. The reference numerals used in the columns are attached, but this does not mean that the scope of claims is limited by the reference numerals.) The imaging means (10), the luminance saturation suppression image generating means (30), the vehicle area specifying means ( 20 and 30) and luminance correction means (30).

  The imaging means (10) acquires an image around the host vehicle and outputs it as image data including at least the luminance value of the pixel. The luminance saturation suppression image generation means (30) outputs image data output by the imaging means (10). Thus, an image is generated by suppressing the luminance of a portion where the luminance value of the pixel exceeds a predetermined saturation luminance to a predetermined luminance.

  The vehicle area specifying means (20, 30) specifies an area of another vehicle in the image from the image data output by the imaging means (10), and the brightness correcting means (30) is the vehicle area specifying means (20, 30). The brightness of the area of the other vehicle specified by is replaced with a predetermined saturation brightness and corrected.

  In such an image processing device (5), an image in which saturation luminance is suppressed is generated from the image around the host vehicle obtained by the imaging means (10), and the luminance of the vehicle area in the image is set to a predetermined saturation luminance. Replace and correct.

  That is, the vehicle area in the image is displayed with saturated luminance even if the luminance is low, so that the vehicle can be made conspicuous in the image. On the other hand, when brightness saturation occurs outside the vehicle area, for example, when there is a streetlight whose brightness is higher than the headlight of the vehicle and the brightness of that portion causes brightness saturation, the streetlight part The brightness of is suppressed.

  Therefore, if an image subjected to such processing is displayed, the vehicle is displayed prominently in the image, so the driver can appropriately recognize the presence of other vehicles, and safety during driving Can be improved.

  Here, the “saturated luminance” is a luminance that the luminance of the pixel does not become a value higher than that, and when the pixel exceeds the saturated luminance, the pixel does not become the luminance higher than that. So-called whiteout occurs.

  The image processing apparatus (5) according to claim 2 includes an imaging means (10), a luminance saturation suppression image generation means (30), a vehicle area identification means (20, 30), and a luminance saturation suppression effect reduction means (30). ).

  The imaging means (10) acquires an image around the host vehicle and outputs it as image data including at least the luminance value of the pixel. The luminance saturation suppression image generation means (30) outputs image data output by the imaging means (10). Thus, an image is generated by suppressing the luminance of a portion where the luminance value of the pixel exceeds a predetermined saturation luminance to a predetermined luminance.

  The vehicle area specifying means (20, 30) specifies an area of another vehicle in the image obtained from the image data output from the imaging means (10), and the brightness saturation suppression effect reducing means (30) For the area of the other vehicle specified by the specifying unit (20, 30), the effect of suppressing the luminance of the portion exceeding the saturation luminance in the luminance saturation suppression image generating unit (30) to a predetermined luminance is reduced.

  In such an image processing device (5), an image in which the luminance of a portion where the luminance of the pixel exceeds a predetermined saturation luminance in the image around the host vehicle acquired by the imaging means (10) is suppressed to the predetermined luminance. can get. However, the luminance of the region identified as another vehicle in the image is not suppressed even if the luminance of the pixel exceeds a predetermined saturation luminance.

  In other words, the vehicle area in the image is displayed with a high luminance even when the saturation luminance is exceeded, so that the luminance of the other vehicle can be selectively increased in the image. Therefore, for example, a vehicle with headlights turned on at night can be made conspicuous, and if it is displayed, the driver can appropriately recognize the presence of other vehicles, and safety during driving Can be improved.

  By the way, even if the imaging means (10) is relatively narrow and has linear imaging characteristics, it can be used as the image processing device (5) according to claim 1 or 2, As described in the item 3, it is preferable that the imaging means (10) has a logarithmic characteristic at least in a part of the imaging characteristic.

  When the imaging means (10) is relatively narrow and has a linear imaging characteristic, the luminance saturation of the pixel is likely to occur, and the pixel in which the luminance saturation has occurred has no gradation information. This is disadvantageous when generating an image with suppressed image.

  However, having a logarithmic characteristic as part of the imaging characteristic is advantageous in generating an image in which luminance saturation is suppressed because pixel luminance information is difficult to lose because pixel luminance saturation is difficult to occur. . That is, when it is used as the imaging means (10) of the image processing device (5) according to claim 1 or claim 2, the effect of making the vehicle stand out becomes high.

  In addition, as a method for generating an image in which luminance saturation is suppressed, a plurality of images are generated from the image data output by the imaging means (10), and one of the generated plurality of images is provided. Extract the light source from one image, and replace the light source part of the image from which the light source is extracted with other luminance images or mix them with the other luminance images to determine the pixel luminance value The image may be generated by suppressing the luminance of the portion exceeding the saturated luminance to a predetermined luminance.

  In this way, it is possible to generate a plurality of images from the image data output by the imaging means (10) and perform the most appropriate processing on each of the generated images. That is, a process of extracting a light source from one of the plurality of generated images is performed, and in the other plurality of images, the light source portion of the image from which the light source is extracted is replaced with an image having other luminance. Mix with other brightness images.

  In this way, it is possible to generate an image while suppressing the luminance of the portion where the luminance value of the pixel exceeds the predetermined saturation luminance to the predetermined luminance. That is, since different processing can be performed on each of the plurality of images and a processing method suitable for the processing can be adopted, it is possible to efficiently generate an image in which the luminance exceeding the saturation luminance is suppressed.

Further, as described in claim 5, it is preferable that an image from which a light source is extracted is generated by dynamic range compression processing.
The dynamic compression process has an effect of making it easy to see a dark place. Therefore, the dynamic compression process is an image processing technique that is particularly effective at night when the surroundings are dark, but has a characteristic that it is difficult to suppress luminance saturation of pixels. Therefore, if the dynamic compression processing is the target of the luminance saturation suppression image generation means (30), the image processing can take advantage of the dynamic compression processing that makes the dark place easy to see and can suppress the luminance saturation of the pixels. It can be a device (5).

  As for a plurality of images other than the image from which the light source is extracted, as described in claim 6, the luminance saturation suppression image generating means (30) uses the other plurality of images as pixels than the image from which the light source is extracted. Is generated by performing dynamic range compression processing in which parameters are set so as to be low predetermined luminance, and the parameters are spatial directions of filter processing for blurring an image when estimating illumination light components, Alternatively, it is preferable to use at least one of the luminance direction filter radius setting value or the saturation luminance excess area allowable value in the maximum / minimum luminance value setting processing after dynamic range compression.

  Here, the “filtering process for blurring the image” is a process for reducing the overall contrast of the image. For example, a bilateral filter, a low-pass filter, or a method for enlarging the reduced image after reducing the image. This process is to blur the entire image.

  As described above, the dynamic range compression process has an effect of making it easy to see a dark place, and the effect varies depending on parameters such as a compression ratio, for example. A plurality of images other than the light source extraction have lower luminance than the image from which the light source is extracted.

  Therefore, a dynamic range compression process in which parameters are set so that the luminance of the pixels becomes a predetermined luminance lower than that of the image from which the light source is extracted is performed on these images. At that time, processing parameters are saturated in the spatial direction or luminance direction filter radius setting value of the filter processing for blurring the image when estimating the illumination light component, or the maximum and minimum luminance value setting processing after dynamic range compression. If at least one of the over-brightness region allowable values is used, a so-called beautiful image in which luminance saturation hardly occurs can be obtained.

  By the way, various methods can be considered as a method for suppressing pixel saturation in the luminance saturation suppression image generating means (30). As described in claim 7, in each image of a plurality of images, a low luminance region is obtained. It is preferable to generate an image having a larger logarithmic characteristic in the high luminance region.

  In this way, when the luminance value of the image data (luminance data) input to each pixel is low, the luminance value is output as it is, and when the luminance value is high, the value is suppressed. The occurrence of luminance saturation of the pixels can be suppressed.

For example, when a vehicle with a headlight turned on at night is imaged, it is possible to obtain an image in which the luminance of the headlight portion with high luminance is suppressed and the luminance of other portions is not suppressed.
In addition, various vehicle area specifying methods by the vehicle area specifying means (20, 30) are conceivable. Other vehicle position detecting means for detecting the position of another vehicle as described in claim 8. (20), the vehicle area specifying means (20, 30) is a position of the other vehicle detected by the image processing based on the image data output by the imaging means (10) or the other vehicle position detecting means (20). It is good to specify the area | region of another vehicle using at least any one of these.

  In this way, since the position of the other vehicle is detected by the other vehicle position detection means (20), the area of the other vehicle in the image can be easily specified by using the position. Further, based on the image data, for example, the region of the other vehicle in the image can be specified also by image processing such as pattern recognition using the shape data of the other vehicle registered in advance. Further, if both the position of the other vehicle and the image processing are used, the area of the other vehicle in the image can be specified with higher accuracy.

  By the way, although various methods can be considered as a method for reducing the luminance saturation suppression effect in the luminance saturation suppression effect reducing means (30), the vehicle among the images acquired by the imaging means (10) as described in claim 9 Dynamic range compression to prevent the luminance value of the image from locally exceeding a predetermined saturation luminance for an area other than that specified by the area specifying means (20, 30) and an area having a predetermined luminance or higher. It is preferable to generate an image by dynamic range compression capable of setting these parameters.

  In this way, dynamic range compression that allows parameter setting is performed for regions other than the vehicle region in the image, so that an image that allows easy viewing of the region other than the vehicle in a dark place can be obtained as described above.

  In this case, as described in claim 10, the luminance saturation suppression effect reducing means (30) generates an image by dynamic range compression capable of setting a parameter, and the parameter is an image when the illumination light component is estimated. It is preferable to set at least one of the spatial direction of the filter processing for blurring, or the luminance direction filter radius setting value, or the saturation luminance excess region allowable value in the maximum / minimum luminance value setting processing after dynamic range compression.

  In this manner, as in the case of the dynamic range compression processing as in the sixth aspect, the processing parameter is the spatial direction of the filter processing for blurring the image or the luminance direction filter radius setting value when the illumination light component is estimated Alternatively, when at least one of the saturated luminance excess region allowable values in the maximum / minimum luminance value setting process after dynamic range compression is used, a so-called beautiful image in which luminance saturation hardly occurs can be obtained.

Embodiments to which the present invention is applied will be described below with reference to the drawings. The embodiment of the present invention is not limited to the following embodiment, and can take various forms as long as they belong to the technical scope of the present invention.
[First Embodiment]
FIG. 1 is a block diagram showing a schematic configuration of an image display apparatus 1 to which an image processing apparatus 5 to which the present invention is applied is applied. As shown in FIG. 1, the image display device 1 includes an image processing device 5 and a display device 50 and is mounted on a vehicle (not shown).

The image processing device 5 includes a rear camera 10, a millimeter wave radar 20, a headlight switch 30, and an image processing unit 40.
The rear camera 10 acquires an image around the host vehicle and outputs it as image data including at least the luminance value of the pixel. The rear camera 10 is attached to the rear side surface of the host vehicle so that a rear side image of the host vehicle can be acquired. ing.

  Further, the rear camera 10 is a CCD camera having a logarithmic characteristic at least in a part of the imaging characteristics. In the present embodiment, the rear camera 10 has a logarithmic characteristic that suppresses the luminance output for the high luminance region compared to the low luminance region. It is a so-called high dynamic range camera.

  The millimeter wave radar 20 is for detecting the position of another vehicle, and includes an antenna, an oscillator, an amplifier, a modulator / demodulator, etc. (not shown), and transmits a millimeter wave and receives a reflected wave from the other vehicle. The distance and direction of the other vehicle (that is, the position of the other vehicle) are detected. In the case of this embodiment, it is attached to the rear part of the vehicle so that the position of another vehicle located on the rear side can be detected.

The headlight switch 30 is a switch for turning on / off the headlight of the host vehicle.
The image processing unit 40 includes a CPU, ROM, RAM, and I / O (not shown), and executes image processing shown in the following (A) to (G). Here, the portion related to the processing for generating the image with the saturation luminance suppressed in the image processing is the saturation luminance suppression image generation processing, the portion related to the processing for specifying the vehicle region is the vehicle region identification processing, and the saturation luminance suppression effect is reduced. The part related to the processing is called luminance saturation suppression effect reduction processing.

[Image processing]
(Luminance saturation suppression image generation processing)
(A) A plurality of images are generated from the image data output from the rear camera 10.

(A) A light source is extracted from one of the generated images by performing dynamic range compression.
(C) A dynamic range compression process in which parameters are set so that the luminance of the pixels is set to a predetermined luminance lower than that of the image from which the light source is extracted is generated from the other images. The parameter at this time is the spatial direction of filter processing for blurring the image when estimating the illumination light component, or the saturation luminance excess region in the luminance direction filter radius setting value and the maximum and minimum luminance value setting processing after dynamic range compression. Use tolerances.

  (D) The luminance value of the pixel exceeds a predetermined saturation luminance by replacing the light source portion of the image from which the light source has been extracted with another luminance image or mixing it with another luminance image in another plurality of images. An image is generated by suppressing the luminance of the portion to a predetermined luminance.

(Vehicle area identification processing)
(E) The region of the other vehicle is specified using the image processing based on the image data output from the rear camera 10 and the position of the other vehicle detected by the millimeter wave radar 20.

(Brightness saturation suppression effect reduction processing)
(F) For the area of the other vehicle specified by the vehicle area specifying process, the effect of suppressing the luminance of the portion exceeding the saturated luminance in the luminance saturation suppression image generation process to a predetermined luminance is reduced.

  (G) The luminance value of the image does not exceed the predetermined saturated luminance locally in the region acquired by the rear camera 10 other than the region specified by the vehicle region specifying process and the region having the predetermined luminance or higher. An image is generated by dynamic range compression capable of setting dynamic range compression parameters.

  In this case, the parameters are the spatial direction of filter processing for blurring the image when estimating the illumination light component, or the luminance direction filter radius setting value and the saturation luminance excess in the maximum and minimum luminance value setting processing after dynamic range compression. The area tolerance is assumed.

The display device 50 includes an LCD, and is attached to the rearview mirror portion of the vehicle instead of the mirror.
(Contents of image processing)
Next, the contents of the image processing executed in the image processing unit 40 will be described with reference to FIGS. FIG. 2 is a flowchart showing the flow of image processing, FIG. 3 is a diagram showing an example of an image obtained at each stage of image processing, and FIG. 4 shows an example of an image obtained by dynamic range compression processing. FIG. FIG. 5 is a diagram showing a histogram of luminance in the entire night image, and FIG. 6 is a diagram showing a composite image obtained by image processing.

  In the image processing, as shown in FIG. 2, a rear side image of the host vehicle is acquired from the rear camera 10 in S100. An example of the image acquired at this time is shown in FIG. As shown in FIG. 3A, in the captured image (logarithmic image) with the rear camera 10 (high dynamic range camera), the image is blurred as a whole, the contrast of the dark part is not clear, and cannot be clearly identified. It is an image.

  In subsequent S105, a dynamic range compressed image of the rear side image acquired in S100 is generated. Here, the dynamic range compression processing will be described with reference to FIG.

  Dynamic range compression is one of the techniques for making the contrast look good both in dark (low brightness) and bright (high brightness) areas of an image. Specifically, first, an image is obtained by dividing the luminance of the original image by the amount of illumination light falling on that region for each predetermined region of the image. The amount of illumination light is estimated by applying a low-pass filter (blurring) to the image.

  An image obtained in this way is called an illumination light component image, and a filter called a bilateral filter is generally used. FIG. 4B shows an illumination light component image obtained by using a bilateral filter for the original image shown in FIG. FIG. 4C shows an image obtained by dividing the original image by the illumination light component image, that is, an image obtained by performing dynamic range compression processing on the original image.

  The luminance relationship in the image after dynamic range compression is different from the actual luminance relationship, but it looks close to the original image. The luminance histogram of the image after dynamic compression is not necessarily dense. As shown in FIG. 5, a maximum / minimum luminance value setting process (also referred to as min / max process) is performed to obtain a final image. Need to produce. In other words, luminance saturation (whiteout) and luminance reduction (blackout) in an image cannot be eliminated by simple dynamic range compression alone.

Next, min / max processing will be described with reference to FIG. FIG. 5 is a diagram showing a histogram of luminance in the entire night image.
In the night image, as shown in FIG. 5, the distribution of specific luminance is large in the low luminance region, and is distributed in a wide luminance range in the high luminance region. The min / max processing refers to, for example, calculating a 1% portion of the high and low luminance ranges and setting the range to 0 to 255 to create a final image (vertical in FIG. 5). Within the line). In this case, the portion surrounded by the broken line is saturated in brightness (out-of-white). Further, 1%, which is the set value at this time, is called a saturated luminance excess region allowable value.

  An image subjected to dynamic range compression as described above is shown in FIG. As shown in FIG. 3 (b), the dynamic range compression process can clearly recognize the contrast even in a dark part, but the light source part is saturated in brightness and so-called whiteout. The image shown in FIG. 3B is assumed to be a base image.

  In S110, the on / off state of the headlight is acquired from the headlight switch 30, and in the subsequent S115, it is determined whether it is night based on the on / off state of the headlight switch acquired in S110.

  That is, when the headlight switch is on (S115: Yes), it is determined that it is nighttime because the headlight that is nighttime illumination is turned on, and the process proceeds to S120. On the other hand, when the headlight switch is off (S115: No), it is determined that it is not nighttime because the headlight which is night illumination is turned off, and the process proceeds to S135.

  In S120, the position of the other vehicle is acquired from the millimeter wave radar 20, and the other vehicle is detected from the acquired position of the other vehicle and the rear side image acquired in S100. Specifically, another vehicle is extracted from the rear side image obtained in S100 based on the outer shape, the number and position of headlights, or the color. And if the other vehicle extracted by image processing exists in the position of the other vehicle acquired from the millimeter wave radar 20, it will be considered as another vehicle.

  In subsequent S125, a composite ratio image is generated. The details of the synthesis ratio image generation process will be described later. An image obtained as a result of the synthesis ratio image generation process is shown in FIG. As shown in FIG. 3 (c), a portion (particularly a light source) of the area of the other vehicle is replaced with a predetermined luminance from the base image obtained in S105, and a so-called whitened image is generated.

  In subsequent S130, a composition process is performed. That is, an image is generated by combining the base image obtained in S105 and the composition ratio image obtained in S125. FIG. 6A shows the composite image obtained in this way, and FIG. 6B shows a composite image when the processing of the area of the other vehicle is not performed for comparison.

  In subsequent S135, in the case of nighttime, the image synthesized in S130 is displayed on the display device 50, and when it is not nighttime (when it is determined that it is not nighttime in S115), the image acquired by the rear camera 10 is displayed. Is displayed. When the image display is completed, the process returns to S100, and the image processing is repeated.

(Composition ratio image generation processing)
Next, the composition ratio image generation process will be described with reference to FIG. FIG. 7 is a flowchart showing the flow of the composition ratio image generation process.

In the composition ratio image generation process, in S200, a high luminance area is extracted from the image subjected to the dynamic compression process in S105. An image extracted by this processing is shown in FIG.
In subsequent S205, a blurring process is performed on the high-luminance areas such as headlights and street lamps of other vehicles extracted in S200 so that the joints of the combined portions do not become unnatural when the images are combined. An image obtained by this processing is shown in FIG.

In subsequent S210, the luminance of the high luminance portion is replaced with luminance lower than the saturation luminance so as to reduce the luminance of the high luminance region. An image obtained by this processing is shown in FIG.
In subsequent S215, only the luminance of the headlight portion of the other vehicle detected in S120 is replaced with saturated luminance.

(Characteristics of the image display device 1)
In the image display device 1 described above, the luminance of the portion of the rear side image of the host vehicle acquired by the rear camera 10 where the pixel luminance exceeds the predetermined saturation luminance is suppressed to the predetermined luminance. For a region identified as being present, the luminance is not suppressed even if the luminance of the pixel exceeds a predetermined saturation luminance. In other words, the vehicle area in the image is displayed with a high luminance even when the saturation luminance is exceeded, so that the luminance of the other vehicle can be selectively increased in the image.

  Specifically, since the vehicle with the headlights turned on at night is highlighted and displayed on the display device 50 attached instead of the rearview mirror, the driver can appropriately recognize the presence of other vehicles. It is possible to improve safety during driving.

  Further, since the rear camera 10 has a logarithmic characteristic as a part of the imaging characteristic, luminance saturation of the pixel hardly occurs. Since it is difficult for the luminance saturation of the pixel to occur, the gradation information of the pixel is not easily lost, which is advantageous when generating an image with suppressed luminance saturation. That is, the effect of making the vehicle stand out is high.

  Furthermore, a process of generating a plurality of images from the image data output by the direction camera 10 and extracting a light source from one of the generated plurality of images is performed, and a light source is extracted from the other plurality of images. The light source portion of the image to be replaced is replaced with an image having another luminance or mixed with an image having another luminance.

  That is, since different processing is performed on each of the plurality of images and a processing method suitable for the processing is adopted, it is possible to efficiently generate an image in which the luminance exceeding the saturation luminance is suppressed.

  In addition, since dynamic compression processing is used for image generation, it is possible to take advantage of the dynamic compression processing that makes it easy to see a dark place, and to suppress luminance saturation of pixels.

  In addition, dynamic range compression processing is performed in which parameters are set so that the luminance of the pixels is lower than that of the image from which the light source is extracted. At that time, the saturation saturation occurs because the processing parameter uses the saturation saturation excess region allowable value in the maximum and minimum brightness value setting processing after dynamic range compression of the bilateral filter processing when estimating the illumination light component. It is difficult to obtain so-called beautiful images.

  Since the other vehicle position detected by the millimeter wave radar 20 and the image processing such as pattern recognition based on the shape data of the other vehicle registered in advance are specified, the region of the other vehicle in the image is specified. The area of the other vehicle inside can be specified.

  In addition, since dynamic range compression in which parameters can be set is performed for regions other than the vehicle region in the image, it is possible to obtain an image that allows easy viewing of the region other than the vehicle in the dark place as described above.

In addition, when the dynamic range compression processing is performed, the processing parameter uses the saturation luminance excess region allowable value in the maximum and minimum luminance value setting processing after the dynamic range compression of the bilateral filter processing when estimating the illumination light component. Thus, a so-called beautiful image in which luminance saturation hardly occurs can be obtained.
[Second Embodiment]
Next, a second embodiment will be described with reference to FIG. FIG. 9 is a diagram illustrating a luminance value conversion function. In the first embodiment, in S105, the dynamic range compression process using the saturated brightness excess region allowable value in the maximum / minimum brightness value setting process after the dynamic range compression is performed, but in the second embodiment, the tone curve is used instead. Make adjustments.

  Specifically, an image having a larger logarithmic characteristic is generated in the high luminance region than in the low luminance region. That is, as shown in FIG. 9, the luminance value conversion function having a larger logarithmic characteristic is applied to all the pixels in the high luminance region than in the low luminance region.

  When the luminance value conversion function shown in FIG. 9 is applied, a dark portion (low luminance) is stretched so that a contrast appears, and a bright portion (high luminance) is suppressed in luminance. Is less likely to occur.

[Third Embodiment]
Next, a third embodiment will be described. In 1st and 2nd embodiment, when brightness saturation arises in the picture acquired with back camera 10, the brightness of the part of brightness saturation other than vehicles is controlled, and vehicles are displayed in the state where brightness saturation was carried out. The whole image is easy to see and the vehicle part is made conspicuous.

  On the other hand, in the third embodiment, when saturation luminance occurs in the image acquired by the rear camera 10, the luminance of the portion is suppressed, and only the vehicle portion is replaced with a predetermined luminance and displayed. Make the whole area easy to see and make the vehicle stand out.

  Specifically, in the composite ratio image generation process, instead of the processes of S200 to S215, the headlight portion of the other vehicle detected in S120 is replaced with saturated luminance from the image subjected to the dynamic range compression process generated in S105. Correction is performed.

In this way, luminance saturation of the entire image is suppressed, and an image in which only the vehicle portion is conspicuous can be obtained.
[Other Embodiments]
(1) In the above embodiment, the rear camera 10 acquires a rear side image of the host vehicle, performs image processing, and displays the obtained image on the display device 50 installed instead of the rearview mirror. The front camera may acquire a front image of the host vehicle, perform image processing, and display the obtained image on a display device installed in the front part of the room.

  In this way, an easy-to-see image in which other vehicles existing in front of the host vehicle are conspicuous can be obtained, so that the driver can appropriately recognize the presence of the other vehicles ahead and improve safety during driving. can do.

  (2) Alternatively, an illuminometer may be used instead of the headlight switch 30, and it may be determined that it is nighttime when the illuminance falls below a preset value. In this way, it is possible to obtain an easy-to-see image in which other vehicles are conspicuous at night without any operation by the driver.

  (3) In S115, it is determined whether or not it is “nighttime”, but instead of “nighttime”, it may be determined whether or not it is “dark place”. In other words, the headlight switch 30 and the illuminance meter can provide an easy-to-see image in which other vehicles stand out even when entering from a bright place to a dark place such as a tunnel entrance.

  (4) Moreover, in the said embodiment, although the area | region of the other vehicle was specified using the position of the other vehicle detected by the image processing based on the image data which the rear camera 10 outputs, and the millimeter wave radar 20, The area of the other vehicle may be specified by either one of the image processing based on the image data or the position of the other vehicle detected by the millimeter wave radar 20.

  (5) Moreover, in the said embodiment, although the illumination light component image was produced | generated using the bilateral filter, what kind of process may be sufficient as long as an image can be blurred. For example, the image may be blurred using a low-pass filter, or the image may be blurred by reducing the image and then enlarging it.

1 is a block diagram illustrating a schematic configuration of an image display device 1. FIG. It is a flowchart which shows the flow of an image process. It is a figure which shows the example of the image obtained in each step of image processing. It is a figure which shows the example of the image obtained by a dynamic range compression process. It is a figure which shows the histogram of the brightness | luminance in the whole night image. It is a figure which shows the synthesized image obtained by image processing. It is a flowchart which shows the flow of a composite ratio image generation process. It is a figure which shows the image extracted by the synthetic | combination ratio image generation process. It is a figure which shows a luminance value conversion function.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image display apparatus, 5 ... Image processing apparatus, 10 ... Back camera, 20 ... Millimeter wave radar, 30 ... Headlight switch, 40 ... Image processing part, 50 ... Display apparatus.

Claims (10)

Imaging means for acquiring an image around the host vehicle and outputting the image data including at least a luminance value of a pixel;
Luminance saturation-suppressed image generation means for generating an image by suppressing the luminance of a portion where the luminance value of the pixel exceeds a predetermined saturation luminance from the image data output by the imaging means to a predetermined luminance;
Vehicle area specifying means for specifying an area of another vehicle in the image from the image data output by the imaging means;
Luminance correction means for correcting the luminance of the area of the other vehicle specified by the vehicle area specifying means by replacing it with the predetermined saturated luminance;
An image processing apparatus comprising: Imaging means for acquiring an image around the host vehicle and outputting the image data including at least a luminance value of a pixel;
Luminance saturation-suppressed image generation means for generating an image by suppressing the luminance of a portion where the luminance value of a pixel exceeds a predetermined saturation luminance from the image data output by the imaging means to a predetermined luminance;
Vehicle area specifying means for specifying an area of another vehicle in an image obtained from the image data output by the imaging means;
For the area of the other vehicle specified by the vehicle area specifying unit, a luminance saturation suppression effect reducing unit that reduces the effect of suppressing the luminance of the portion exceeding the saturation luminance to a predetermined luminance in the luminance saturation suppression image generating unit. When,
An image processing apparatus comprising: The image processing apparatus according to claim 1 or 2,
The image processing apparatus, wherein the imaging means has logarithmic characteristics at least in part of imaging characteristics. In the image processing device according to any one of claims 1 to 3,
The luminance saturation suppression image generation means includes
A plurality of images are generated from the image data output by the imaging means, a light source is extracted from one of the generated plurality of images, and the light source portion of the image from which the light source is extracted is replaced with another plurality of light sources. In this image, by replacing the image with other luminance or mixing with the image with other luminance, the luminance of the portion where the luminance value of the pixel exceeds the predetermined saturation luminance is suppressed to the predetermined luminance to generate the image. An image processing apparatus. The image processing apparatus according to claim 4.
The luminance saturation suppression image generation means includes
An image processing apparatus for generating an image from which the light source is extracted by dynamic range compression processing. The image processing apparatus according to claim 4 or 5,
The luminance saturation suppression image generation means includes
The other plurality of images are generated by performing dynamic range compression processing in which parameters are set so that the luminance of the pixels is lower than that of the image from which the light source is extracted,
The parameter is a spatial direction of filter processing for blurring an image when estimating an illumination light component, or a luminance direction filter radius setting value, or a saturation luminance excess region allowable value in maximum / minimum luminance value setting processing after dynamic range compression. An image processing apparatus using at least one of values. The image processing apparatus according to claim 4 or 5,
The luminance saturation suppression image generation means includes
An image processing apparatus that generates, in each of the plurality of images, an image having a higher logarithmic characteristic in a high luminance region than in a low luminance region. The image processing apparatus according to any one of claims 1 to 7,
Other vehicle position detection means for detecting the position of the other vehicle,
The vehicle area specifying means includes
The region of the other vehicle is specified using at least one of image processing based on the image data output by the imaging unit or the position of the other vehicle detected by the other vehicle position detection unit. Image processing device. The image processing apparatus according to any one of claims 2 to 8, wherein the luminance saturation suppression effect reducing means is
The luminance value of the image does not locally exceed a predetermined saturation luminance in a region other than the region specified by the vehicle region specifying unit in the image acquired by the imaging unit and a region having a predetermined luminance or higher. An image processing apparatus for generating an image by dynamic range compression capable of setting dynamic range compression parameters for performing The image processing apparatus according to claim 9.
The luminance saturation suppression effect reducing means is
Generate images with dynamic range compression with parameters,
The parameter is a spatial direction of filter processing for blurring an image when estimating an illumination light component, or a luminance direction filter radius setting value, or a saturation luminance excess region allowable value in maximum / minimum luminance value setting processing after dynamic range compression. An image processing apparatus having at least one of values.