JP4935586B2 - Image processing apparatus, in-vehicle image processing apparatus, in-vehicle image display apparatus, and vehicle control apparatus - Google Patents

Description

  The present invention relates to an image processing apparatus that improves the difficulty in viewing an image due to road surface reflection, and an in-vehicle apparatus using the same.

  Conventionally, there are rear side monitors and vehicle detection sensors using an in-vehicle camera. In such a device, in order to improve safety, an object having a relative speed detected by an optical flow or the like is highlighted to create an image that is easier for the driver to understand. (For example, refer to Patent Document 1).

  However, when it is raining at night, the headlamps of the oncoming vehicle, the taillights of the preceding vehicle, streetlights, etc. are reflected on the wet road surface and many glare occurs, and the glare and light source are present in the image acquired by the camera. Therefore, it becomes difficult to distinguish between an actual vehicle and a virtual image of the vehicle due to road surface reflection.

In order to solve this problem, there is a method for detecting a virtual image of a vehicle due to road surface reflection from an image in rainy weather acquired by a camera (see, for example, Patent Document 2).
JP-A-9-48282 JP 2003-252147 A

  However, the method for detecting the virtual image of the vehicle due to the road surface reflection is to detect the virtual image by acquiring distance information by a stereo camera using a plurality of cameras and discriminating between the virtual image and the real image based on the acquired distance information. Yes. Therefore, there is a problem that a plurality of cameras are necessary.

  The present invention has been made in view of these problems, and an object thereof is to provide an image processing apparatus that improves the difficulty in viewing an image due to road surface reflection, and a vehicle control apparatus using the image processing apparatus.

  The image processing apparatus according to claim 1, which has been made to solve such a problem (5: In this section, in order to facilitate understanding of the invention, “the best mode for carrying out the invention” is necessary as necessary. The reference numerals used in the columns are attached, but this does not mean that the claims are limited by the reference numerals.) The imaging means (10, 12), virtual image detection means (30), and road surface reflection suppression means (30 ). The imaging means (10, 12) acquires a peripheral image of the moving body including at least a road image, and the virtual image detection means (30) is a virtual image by road surface reflection from the image acquired by the imaging means (10, 12). Is to detect the occurrence of.

The virtual image detection means (30) extracts a high brightness area in the image acquired by the imaging means (10, 12), and determines the reliability of detection of a virtual image of road surface reflection for each of the extracted high brightness areas. Is calculated based on the degree of vertical deviation of the pair, the degree of difference in size, and the degree of difference in luminance.
Further, the road surface reflection suppression means (30) determines whether or not a virtual image due to road surface reflection has occurred in the virtual image detection means (30), and when a virtual image has occurred, based on the detection result of the virtual image detection means (30). Then, a corrected image in which a virtual image due to road surface reflection in the image acquired by the imaging means (10, 12) is suppressed is generated.
Furthermore, the road surface reflection suppression means (30) suppresses the virtual image due to road surface reflection in the image acquired by the imaging means (10, 12) according to the reliability of the virtual image detection calculated by the virtual image detection means (30). Perform image correction .

  According to such an image processing device, when a virtual image due to road surface reflection is detected from a peripheral image of a moving body including at least a road image acquired by the imaging means (10, 12), the image is displayed according to the detection result. A corrected image in which a virtual image due to road surface reflection is suppressed is generated.

That is, when a virtual image due to road surface reflection is detected, an image in which the virtual image is suppressed can be obtained. Since the virtual image is suppressed in the obtained image, only the image due to road surface reflection is not difficult. That is, it is possible to improve difficulty in viewing an image due to road surface reflection.
Further, when detecting a virtual image by road surface reflection, if the reliability of detection is obtained in addition to the detection of the virtual image, the image can be corrected based on the reliability, but the virtual image detecting means (30) The reliability of road surface reflection virtual image detection is calculated for each extracted high-brightness area, and the road surface reflection suppression means (30) performs correction according to the virtual image detection reliability calculated by the virtual image detection means (30). I am doing so.
Therefore, for example, when the detection reliability of the virtual image is high, the correction is performed by reducing the luminance greatly, and when the reliability of the detection reliability of the virtual image is low, the correction is performed without reducing the luminance so much. Thereby, when the detection detection of the virtual image is delicate, the correction amount is delicately performed. Therefore, it is possible to reduce a sense of incongruity and flicker due to the correction when the determination is not stable.

Here, as a method for detecting the occurrence of a virtual image due to the road surface reflection is as described in claim 2, the virtual image detecting means (30), the shape of the region out extraction, position, luminance value, size, moving speed The occurrence of a road surface reflection virtual image may be detected by combining at least two or more features of the shape stability or the presence or absence of a light source that is a real image corresponding to the road surface reflection virtual image.

  Here, “a light source that is a real image corresponding to a virtual image of road surface reflection” will be briefly described. When a light source is reflected on the road surface and a virtual image is generated on the road surface, the virtual image and the real image of the light source that generated the virtual image are always paired on the screen obtained by the imaging means (10, 12). It will be reflected. That is, a light source that generates a virtual image due to road surface reflection is referred to as a “light source that is a real image corresponding to a virtual image of road surface reflection”.

According to the image processing device of the second aspect, it is possible to detect the generation of the virtual image of the road surface reflection according to the characteristics of the virtual image or the real image, and therefore it is possible to accurately detect the generation of the virtual image.
For example, a virtual image generated by road surface reflection extends in a vertically long shape and is easily visible, and exists only in a region estimated to be a road surface, such as in a car traveling line. On the other hand, the light source is likely to be present in the sky region if it is a street light, and if it is a headlight or taillight of a vehicle, it is present at a low position with strong luminance.

  Furthermore, as described above, the virtual image always has a pair of light sources (real images). Therefore, by combining two or more of these pieces of information, the light spot in the image can be classified as a virtual image or a light source (real image). That is, it is possible to accurately detect the occurrence of a virtual image.

Further, in order to detect the presence or absence of a light source that is a real image corresponding to a virtual image of road surface reflection in the virtual image detection means (30), the virtual image detection means (30) includes a virtual image of road surface reflection as described in claim 3. A light source that is a real image corresponding to the road surface reflection may be detected by using at least one of the shape, position, luminance value, or size of the light source that is a real image corresponding to the road reflection virtual image and the road reflection virtual image. .

  As described above, the real image of the light source and the virtual image by road surface reflection always form a pair. Therefore, for example, the position of the real image and the virtual image is aligned perpendicular to the road surface, the width of both is the same, the brightness of the virtual image is smaller than the real image, etc. It detects.

  Here, "the width of both is the same degree" means that compared to a light source that is a real image, a virtual image due to road surface reflection has a so-called "blur" in the image, so the width of both is not exactly the same . Therefore, in consideration of the “blur”, it means that the widths of both are the same.

  By the way, virtual images are often generated on the road surface when the road surface is in a rainy state. Therefore, if the process in the road surface reflection suppression means (30) is performed only when the road surface is raining, the processing load is reduced.

Therefore, as described in claim 4 , the vehicle is provided with a rain state estimating means (20) for estimating that the road surface is in a rain state, and the virtual image detecting means (30) is provided by the rain state estimating means (20). If it is determined that a virtual image has been generated when it is estimated that the vehicle is in a rainy state and generation of a corrected image is performed, the process in the surface reflection suppression means is performed only when the road surface is in a rainy state. This reduces the processing load.

  The rain state estimating means (20) includes a rain sensor that specifies that the road surface is in a rain state when raindrops are detected, the reflectance of the road surface is measured with a reflectometer, and the reflectance is a predetermined value or more. A method for estimating that the road surface is in a rainy state, a method for measuring the wetness near the road surface with a wet meter, and estimating that the road surface is in a rainy state when the wetness exceeds a predetermined value, There are a method for estimating that the road surface is raining from infrastructure information such as weather information, and a method for extracting a reflected virtual image from the image of the road surface by image processing.

When the image processing apparatus is mounted on a vehicle, it may be estimated that the road surface is in a rainy state by operating the wiper switch.
By the way, as one method of suppressing the influence of glare, there is a method of controlling the exposure of the imaging means (10, 12). That is, normally, when glare with high luminance due to road surface reflection spreads greatly on the screen, the exposure of the imaging means (10, 12) becomes too small and an image suitable for image processing cannot be obtained. Therefore, as described in claim 5 , the imaging means (10, 12) is configured to be able to control exposure of an image to be acquired by an external command, and the road surface reflection suppression means (30) is configured to be the imaging means (10). , 12) is configured so that the exposure can be controlled, and exposure control of the imaging means (10, 12) is performed without using a portion where the luminance in the image acquired by the imaging means (10, 12) is equal to or higher than a predetermined value. Based on the acquired image, it is preferable to perform virtual image detection by road surface reflection.

  In this way, even if glare with high luminance exists, the image is acquired by the imaging means (10, 12) without being affected by the luminance of that portion, so that the exposure of the imaging means (10, 12) is too low. It will not become. Therefore, it is possible to acquire an image with appropriate exposure.

As another method for obtaining an image with appropriate exposure, as described in claim 6 , the road surface reflection suppression means (30) is based on a predetermined value in the image acquired by the imaging means (10, 12). Alternatively, a portion having a higher luminance value may be corrected to a luminance value lower than a predetermined value.

  In this way, even if there is a portion with a luminance greater than a predetermined value in the image, the luminance of that portion becomes smaller than the predetermined value. In other words, since there is no portion of luminance greater than the predetermined value in the image, an image with appropriate exposure can be obtained.

Also, efficiently in order to obtain an image of proper exposure, as described in claim 7, the road surface reflection suppressing means (30) is a virtual image by road reflection from an image acquired by the imaging means (10, 12) The road surface reflection virtual image extraction means (30) for extracting the location where the road is generated is provided, and the image of the location where the virtual image due to the road surface reflection extracted by the road surface reflection virtual image extraction means (30) is generated is the virtual image due to the road surface reflection. You may make it correct | amend with the information of the image of the road surface which has not generate | occur | produced.

  Here, the “image of the road surface where the virtual image due to road surface reflection has not occurred” means an image other than the “image where the virtual image due to road surface reflection extracted by the road surface reflection virtual image extraction means (30)” has occurred. means.

  “Correction with information on the image of the road surface where no virtual image due to road surface reflection has occurred” means, for example, that the inside of the virtual image (glare) is replaced with the brightness value or image pattern of the road surface around the virtual image (glare), or the virtual image This is a process of painting the inside of a virtual image (glare) using a binarization threshold value when extracting (glare).

  In this way, only when a virtual image is generated due to road surface reflection, the image of the portion where the virtual image is generated is replaced with the image of the road surface where no virtual image is generated, so an image with appropriate exposure can be obtained efficiently. be able to.

Furthermore, as described in claim 8 , non-road surface detection means (90) for detecting a non-road surface object that is an object other than a road surface, and not a non-road surface object detected by the non-road surface object detection means (90) Road surface extraction means (30) for extracting a portion that is not a virtual image due to road surface reflection extracted by the road surface reflection virtual image extraction means (30) as a road surface, and the road surface reflection suppression means (30) is road surface extraction means (30). Based on the image information of the road surface extracted in step 1, it is preferable to correct the portion where the virtual image due to the road surface reflection is generated.

  In this case, for example, a non-road surface object such as a radar detects a non-road surface object such as a structure other than a road surface or a vehicle or a pedestrian, and is not a detected non-road surface object and is a virtual image. However, a portion where the virtual image is generated is corrected based on the extracted road surface image information.

  That is, the non-road surface object can be detected by the non-road surface object detection means (90), and the road surface is accurately extracted, so that a virtual image is generated based on the extracted road surface image information. The location is accurately corrected.

Further, as described in claim 9 , the road surface extraction means (30) is configured to be able to extract the color, brightness or pattern of the road surface, and the road surface reflection suppression means (30) is extracted by the road surface extraction means (30). The location where the virtual image is generated may be corrected based on at least one of the color, brightness, or pattern of the road surface. For example, the color of the road surface is averaged, and correction for replacing the entire road surface with the averaged color is performed.

In this way, the road surface can be corrected by combining the color, brightness, or pattern of the road surface in accordance with the state of the road surface, so that an image with a more appropriate exposure can be obtained.
By the way, when an image is obtained by the imaging means (10, 12), for example, there is a case where illumination is irradiated to an imaging target like a headlamp of a vehicle. It may be corrected accordingly.

Therefore, as described in claim 10 , when the road surface reflection suppressing means (30) corrects using a luminance distribution in accordance with illumination characteristics set in advance when correcting, the road surface reflection suppressing means (30) is set in advance. Since the location where the virtual image obtained by the imaging means (10, 12) is generated can be corrected using the luminance distribution corresponding to the illumination characteristics of illumination, more appropriate correction can be performed.

  For example, when irradiating a road surface with a headlight of a vehicle, since the irradiation characteristics such as the luminance distribution according to the irradiation range and distance are known in advance, for example, the distance is dark (the luminance is low). If the correction is made such that the vicinity is bright (the brightness is high), an image with appropriate correction can be obtained.

According to an eleventh aspect of the present invention, there is provided an in-vehicle image processing apparatus in which the image processing apparatus according to any one of the first to tenth aspects is mounted on a vehicle.
When vehicle image processing apparatus according to any one of claims 1 to 10, may be a vehicle image processing device having the features of the image processing apparatus according to any one of claims 1 to 10 .

  By the way, since the virtual image with high brightness is generated in the image due to the road surface reflection of the light source, it is necessary to suppress the generation of the virtual image due to the road surface reflection in the image processing device and make the image easy to see. Therefore, if it is made difficult to generate a virtual image due to road surface reflection, the image becomes easy to see, and the processing load of image processing is reduced.

By the way, when the screen obtained by the image processing apparatus described above is used as the in-vehicle image display apparatus (5), the displayed image is convenient for the driver to see. Therefore, as described in claim 12 , the display means (40) for displaying an image is provided, and the image obtained by the in-vehicle image processing device according to claim 11 is displayed on the display means (40). The vehicle-mounted image display device (5) may be used.

  In this way, even if a virtual image is generated by road surface reflection, an image in which the virtual image is suppressed is displayed on the display means (40), so that an image that is easy to see for the driver is displayed. Thus, the driver can drive the vehicle safely.

Further, the in-vehicle image display device (5) is, as described in claim 13, the display unit (40) is attached to the rear-view mirror for partial imaging means (10, 12) comprises at least road image It is good to be constituted so that the back picture of vehicles may be acquired.

  Here, the “rear viewing mirror” means a mirror for visually recognizing the rear of the vehicle, such as a so-called door mirror, side mirror, and rear view mirror. Also, “attached to the rear viewing mirror part” means that it is attached instead of the rear viewing mirror, is attached to a part of the rear viewing mirror, or functions as a mirror of the rear viewing mirror. This means that the display function of the means (40) can be switched.

  According to such an on-vehicle image display device (5), the vehicle rear image acquired by the imaging means (10, 12) can be displayed on the rear viewing mirror. Therefore, even when a virtual image due to road surface reflection occurs behind the vehicle, the rear view mirror displays an image with the virtual image suppressed, so that the driver can easily confirm the rear, so driving safety Can be improved.

By the way, as described in claim 14 , the road surface is based on the detection result of the virtual image generation by the road surface reflection in the vehicle image processing apparatus according to claim 11 and the virtual image detection means (30) of the vehicle image processing apparatus. Road surface wetness detection means (30) for detecting whether or not the vehicle is wet, light distribution control means (50) for performing light distribution control of the headlight of the vehicle, and road surface detected by the road surface wetness detection means (30) The vehicle control device (1) may control the light distribution of the headlight via the light distribution control means (50) based on the wet state.

  According to such a vehicle control device (1), it is only necessary to detect whether or not a virtual image due to road surface reflection has occurred, for example, by image processing. Therefore, the vehicle control device (1) can have a simple configuration. Further, in a wet state where the virtual image is likely to be generated due to road surface reflection and the generated virtual image has a high luminance, for example, in rainy weather, the light distribution of the headlight can be controlled to be in a rainy state.

  That is, by determining whether or not the road surface is wet by the road surface wetness detection means (30), it is determined that it is rainy if the road surface is wet, and is otherwise clear. If it is raining, for example, the driver of the oncoming vehicle performs light distribution control such as reducing the amount of light in the direction of the closest road surface so that the glare of the road surface reflection glare of the headlight of the own vehicle does not become dazzling.

  In this way, by controlling the light distribution of the headlight, it is possible to suppress the generation of a virtual image or to reduce the brightness of the virtual image even when a virtual image is generated. Accordingly, by viewing the image, for example, a driver other than the own vehicle such as an oncoming vehicle can safely drive.

Further, according to the fifteenth aspect of the invention, there is provided a motion control means (60) for controlling the motion of the vehicle, and the motion control means (60) based on the wet state of the road surface detected by the road surface wetness detection means (30). ) To control the movement of the vehicle.

  If it does in this way, driving control can also be performed by the wet state of a road surface. For example, based on the wet condition of the road surface, if the road surface is wet, it is estimated that the μ value of the road surface is low, and if the road surface is not wet, it is estimated that the μ value of the road surface is high, and based on the estimated μ value, For example, ABS control and stability control can be performed.

By the way, various methods are conceivable for detecting the wet state of the road surface. As described in claim 16 , the road surface wetness detecting means (30) is configured to detect the road surface when a virtual image is generated by road surface reflection. If the road surface is wet and a virtual image due to road surface reflection does not occur, the wet state of the road surface can be easily detected by determining that the road surface is dry.

  By the way, even when a virtual image is generated due to road surface reflection, there is a possibility that the road surface is not necessarily wet. For example, even when the road surface is dry, a virtual image may be generated on the road surface when the amount of light from the headlight of an oncoming vehicle is large and the optical axis is downward.

Therefore, as described in claim 17 , the road surface wetness detection means (30) takes a luminance ratio between the luminance of the real image of the light source obtained by the virtual image detection means (30) and the luminance of the virtual image due to road surface reflection, and On the other hand, when the road surface reflection is brighter than a predetermined luminance ratio, the road surface may be determined to be wet.

  In this way, it is possible to more accurately detect that the road surface is wet. In other words, the luminance ratio between the luminance of the real image of the light source and the luminance of the virtual image due to the road surface reflection differs between when the road surface is wet and when it is dry (the luminance ratio of Is higher).

  Therefore, taking the luminance ratio between the luminance of the real image of the light source and the luminance of the virtual image due to road surface reflection, and determining that the road surface is wet when the road surface reflection with respect to the light source has a brightness of a predetermined luminance ratio or more, It is possible to more accurately determine whether or not the road surface is wet based on the difference in the brightness ratio between the wet state and the dry state.

  By the way, a streetlight, a headlight of a preceding vehicle, a taillight, and the like exist as light sources in an image acquired by the imaging apparatus. Among these, since the optical axis of the headlamp is downward, the luminance ratio of the real image and the virtual image is higher than that of the street lamp and the taillight. In addition, the headlight is lower than the street light.

Accordingly, as described in claim 18 , the light source type determining means (30) for determining whether or not the light source is a headlight of a preceding vehicle according to the position and luminance intensity of the light source is provided, and the road surface wetness detecting means (30) is provided. When the light source type determining means (30) determines that the light source is a headlight, the luminance ratio between the luminance of the real image of the light source and the luminance of the virtual image due to road surface reflection is used to identify light sources other than the headlight and the headlight. If it is not higher than a predetermined value, it may be determined not to be wet.

  If it does in this way, it can be judged whether it is a street light or a vehicle light by the position of a light source, and it can be judged whether it is a headlight or a taillight by the luminance intensity of a light source. That is, it can be determined whether or not the light source in the image is the headlight of the vehicle ahead.

  If it is determined to be a headlight, as described above, the luminance ratio between the real image of the headlight and the virtual image is higher than that of the taillight, so the luminance of the real image of the light source and the luminance of the virtual image due to road surface reflection If the brightness ratio is not higher than a predetermined value for identifying the headlight and the light source other than the headlight, it is determined that the road surface is wet if it is not determined that the road surface is wet. can do.

  By the way, even if the light source generates a virtual image due to road surface reflection, the weight for determining whether or not the light source is wet is not uniform for any light source, and can be varied depending on the position of the light source and the luminance intensity.

Therefore, as described in claim 19 , when the road surface wetness detection means (30) determines whether the road surface is wet or not, whether the light source type determination means (30) is a headlight or not is used as a determination criterion. It is preferable to assign a weight based on the position of the light source and the luminance intensity determined whether or not.

  In this way, for example, if the road surface is wet, a virtual image will be generated by the headlight of the oncoming vehicle or a streetlight that is directly above the traveling road of the host vehicle. If the street lamp does not exist directly above the own road (for example, in the roadside zone), the weight is reduced.

In other words, if the determination criteria for determining whether or not the vehicle is wet is weighted according to the position of the light source and the luminance intensity, the determination of the wet state of the road surface necessary for vehicle motion control is performed more accurately.
Incidentally, even although the road is it often detected light source that does not have a virtual image even in a wet state, the road surface is seldom a virtual image is detected in the dry state, where, as described in claim 20 The road surface wetness detection means (30) determines that the weight of the determination criterion when there is a light source that generates a virtual image due to road surface reflection is greater than the weight of the determination criterion when there is a light source that does not generate a virtual image due to road surface reflection. Good.

  In this way, the weight of the determination criterion when the road surface is wet is large, and the weight of the determination criterion when the road surface is dry is small, so it is easy to determine whether the road surface is wet or dry. It can be judged accurately.

By the way, an image acquired by the imaging means (10, 12) often includes a plurality of light sources such as a lamp of a preceding vehicle and a streetlight. Therefore, as described in claim 21 , when there are a plurality of light sources used for determination, the road surface wetness detection means (30) performs a wet determination for each of the plurality of light sources, and based on a determination result for all of the plurality of light sources. Thus, it is preferable to determine whether or not the road surface is wet.

  In this way, for example, it is determined that the road surface is wet only when it is determined that all of the plurality of light sources are in a wet state, or more than half of the plurality of light sources are determined to be in a wet state. In some cases, it can be determined that the road surface is in a wet state, or a plurality of light sources can be integrated to determine whether the road surface is in a wet state.

Also, if there are multiple light sources, road surface wetting detecting means (30), it is convenient to wet the determination result of each of the plurality of light sources is output in likelihood.
That is, for example, when there are two light sources and one has a virtual image and the other does not have a virtual image, the former has a wet likelihood of 80% (ie, a dry likelihood of 20%) and the latter has a 48% (dry is 5).
2%) and the wet likelihood are output, the determination result of the wet state in which both light sources are integrated by simple multiplication of the likelihood (probability) can be calculated according to the theory with a wet probability of 78.7%.

  As described above, the determination for each light source and its integration make it possible to perform the determination quickly because there is a lot of information that can be used for the determination when there are many light sources. Also, by using the likelihood, the integration process can be simple and conform to the theory.

Embodiments to which the present invention is applied will be described below with reference to the drawings. The embodiment of the present invention is not limited to the following embodiment, and can take various forms as long as they belong to the technical scope of the present invention.
[First Embodiment]
FIG. 1 is a block diagram illustrating a schematic configuration of the vehicle control device 1. As shown in FIG. 1, the vehicle control device 1 includes an in-vehicle image display device 5, a light distribution control unit 50, and an operation control unit 60, and is mounted on a vehicle (not shown).

The in-vehicle image display device 5 includes a front camera 10, a rear camera 12, a wiper switch 20, an image processing unit 30, and a display device 40.
The front camera 10 acquires a peripheral image of the vehicle including at least a road image, and is attached to the vehicle so that a front image of the vehicle can be acquired. The rear camera 12 is
The wiper switch 20 is for estimating that the road surface is raining, and is attached in the vicinity of the handle so as to be operated by the driver.

The image processing unit 30 executes the following processes (a) to (tu).
(A) When it is estimated by the wiper switch 20 that the road surface is in a rainy state, it is determined whether or not a virtual image has been generated and a corrected image is generated.

(A) The generation of a virtual image due to road surface reflection is detected from images acquired by the front camera 10 and the rear camera 12.
(C) It is determined whether or not a virtual image is generated due to road surface reflection. When a virtual image is generated, the road surface in the image acquired by the front camera 10 and the rear camera 12 based on the detection result of the virtual image detection means (30). A corrected image in which a virtual image due to reflection is suppressed is generated.

(D) Extract high-intensity areas in the images acquired by the front camera 10 and the rear camera 12, and support the extracted area shape, position, luminance value, size, moving speed, shape stability, or virtual image of road surface reflection The occurrence of road reflection virtual images is detected by combining at least two features of the presence or absence of a light source that is a real image. (E) The reliability of road image reflection virtual image detection is calculated for each extracted high-intensity region. Then, correction is performed according to the calculated reliability of virtual image detection.

  (F) Using a light source that is a real image corresponding to a virtual image of road surface reflection using at least one of the shape, position, luminance value, or size of the light source that is a real image corresponding to a virtual image of road surface reflection and a virtual image of road surface reflection To detect.

(G) A portion having a luminance value larger than a predetermined value in the images acquired by the front camera 10 and the rear camera 12 is corrected to a luminance value lower than the predetermined value.
(H) Whether or not the road surface is wet is detected based on whether or not generation of a virtual image due to road surface reflection is detected.

(G) Based on the detected wet state of the road surface, the movement of the vehicle is controlled via the movement control unit 60.
(E) When a virtual image due to road surface reflection occurs, the road surface is wet. When a virtual image due to road surface reflection does not occur, it is determined that the road surface is dry.

(Sa) When the luminance ratio between the luminance of the real image of the light source and the luminance of the virtual image due to road surface reflection is lower than a predetermined value, the road surface is determined to be wet.
(H) It is determined whether the light source is a headlight of the vehicle ahead by the position of the light source and the luminance intensity.

  (S) When the light source is determined to be a headlight, the luminance ratio between the luminance of the real image of the light source and the luminance of the virtual image due to road surface reflection is a predetermined value determined to identify a light source other than the headlight and the headlight If it is not higher, it is not determined to be wet.

(C) When determining whether or not the road surface is in a wet state, a weight based on the position and luminance intensity of the light source determined whether or not the headlight is used as a determination criterion.
(E) The weight of the determination criterion when there is a light source that generates a virtual image due to road surface reflection is greater than the weight of the determination criterion when there is a light source that does not generate a virtual image due to road surface reflection.

(T) When there are a plurality of light sources used for the determination, the wet determination is performed for each of the plurality of light sources, and it is determined whether or not the road surface is in a wet state based on the determination results of all the light sources.
(H) Wet determination results for each of a plurality of light sources are output with likelihood.

(Iv) The light distribution of the headlight 70 is controlled via the light distribution control unit 50 based on the detected wet state of the road surface.
The display device 40 includes an LCD and is attached to the rearview mirror portion of the vehicle instead of the mirror. The light distribution control unit 50 performs light distribution control of the headlight 70 of the vehicle, and controls the light amount and irradiation angle of the headlight 70.

The motion control unit 60 controls the motion of the vehicle. In the present embodiment, the motion control unit 60 is an ABS control device that controls the ABS 80 to stop the vehicle at a short distance on a road surface having a small μ value.
(Contents of image processing)
Next, the contents of the image processing executed in the image processing unit 30 will be described with reference to FIG. FIG. 2 is a flowchart showing the flow of image processing.

  In the image processing, as shown in FIG. 2, the state of the wiper switch 20 is acquired in S100, and in the subsequent S105, it is determined whether or not the state of the wiper switch 20 acquired in S100 is on.

  If the wiper switch 20 is on (S105: Yes), the process proceeds to S110. If the wiper switch 20 is off (S105: No), the process returns to S100, and the state of the wiper switch 20 is confirmed. Repeated.

In S110, images are acquired from the front camera 10 and the rear camera 12, and in S115, binarization processing is executed by a known processing method.
In S120, segment processing is executed, and a portion having a predetermined value in position, size, area, or luminance is extracted as a segment in the screen. In addition, numbering is performed on all the extracted segments.

In subsequent S125, a segment feature amount detection process is executed, and the shape, size, position, and luminance of the segment are detected. Details of this feature amount detection processing will be described later.
In subsequent S130, segment classification processing is performed based on the segment feature detected in S125. In this classification process, the segment is classified based on the feature amount shown in FIG. 5 as to whether the segment is glare or paint on the road surface, vehicle taillight or headlight.

  In S135, it is determined whether or not feature amount detection and classification have been performed for all segments. Specifically, it is determined whether or not the feature amount detection process of S125 and the various classification processes of S130 have been executed for all segments numbered in S120.

  If the feature amount detection process and various classification processes have been completed for all segments (S135: Yes), the process proceeds to S140. If the process has not been completed (S135: No), the process proceeds to S125. The process is repeated until the feature amount detection process and various classification processes are completed for all segments.

  In S140, an integrated classification process is performed, and glare in the screen is extracted. That is, light source and glare candidates are selected from the classification of each segment obtained in the various separation processes in S130. Then, the positions, sizes, and luminances of the light source and glare in the candidate are compared, and a pair of glare considered to be generated by the light source and the light source is detected.

  In the pair detection, for example, it is determined whether the light source and the glare are aligned in the vertical direction of the image, the difference in size, or the difference in luminance is within a predetermined range. Is done. A light source and a glare pair are determined to be a light source and a glare generated by the light source, and a light source and a glare are determined to be a single light source.

  Further, when this determination is performed, the reliability of glare is calculated based on the degree of vertical deviation of the light source and glare pair, the magnitude difference, and the brightness difference.

  For example, when there is no vertical deviation between the light source and the glare, the reliability of the deviation is 100%, and the glare is very long compared to a normal light source. When the glare luminance is 80% of the light source luminance, the luminance reliability is 80%, and the total reliability is 100% × 100% × 80%, which is 80%. It is supposed to be.

  In subsequent S145, a non-glare image in which the glare portion is corrected is created for the light source and its glare pair obtained in S140, and the rear image of the created image is displayed on the display device 40. Correction of the glare part is executed by replacing the brightness of the glare part with the brightness calculated by adding the reliability calculated in S140 to the brightness of the entire screen.

  An example of the non-glare image obtained in this way is shown in FIG. 3A is an image acquired by the front camera 10, FIG. 3B is an enlarged image of the oncoming vehicle and the glare portion of the headlight, and FIG. 3C is FIG. 3B. This is an image obtained by correcting the glare part of the image.

  In subsequent S150, a wetness determination process is performed. In subsequent S155, it is determined whether or not the road surface is wet based on the result of the wetness determination process in S150. If the road surface is wet (S155: Yes), the process proceeds to S160. If the road surface is not wet (S155: No), the process returns to S100, and the image processing is repeated. The wetness determination process will be described later.

  In subsequent S160, the light quantity and irradiation angle of the headlight are controlled according to the likelihood obtained in the wetness determination process in S150. That is, when the likelihood is high, the light amount is greatly reduced from the normal light amount, and the irradiation angle is lowered from the normal amount. When the likelihood is low, the light amount is slightly reduced from the normal amount, and the irradiation angle is set to the same angle as usual. It is.

  In subsequent S165, the vehicle motion control is controlled based on the likelihood obtained by the wetness determination in S150. That is, when the likelihood is high, it is estimated that the μ value of the road surface is low, and when the likelihood is low, the μ value is estimated to be high, and the ABS 80 is controlled based on the estimated μ value.

Then, after the vehicle motion control is performed, the process proceeds to S100, and the image processing is repeated.
(Feature amount detection processing)
Next, the feature amount detection process will be described with reference to FIGS. FIG. 4 is a flowchart showing the flow of the feature amount detection process, and FIG. 5 is a diagram showing the feature amount to be detected and the detected contents.

  In the feature amount detection process, as shown in FIG. 3, the segment shape is determined in S200. That is, as shown in FIG. 5, whether the light source or the glare is determined based on the aspect ratio of the segment.

In subsequent S205, the removal of the segment is performed according to the size of the segment. In other words, a segment larger than a predetermined pixel and a segment smaller than a predetermined pixel are removed.
In the following 210, the position of the segment is determined. That is, as shown in FIG. 5, it is determined whether the segment is glare or head depending on whether the segment is on the FOE.

  In the subsequent S215, matching is determined. That is, as shown in FIG. 5, it is determined whether the segment is a light source or a glare based on the horizontal position, width, or luminance intensity of the segment.

  In subsequent S220, moving image determination is performed. That is, as shown in FIG. 5, it is determined whether the segment is road surface paint, a pole, a light source, or glare based on the stop / movement of the segment, the moving speed of the segment, and the shape stability of the segment. Then, after S220 is executed, the process is returned to the main routine.

(Wetting judgment process)
Next, the wetness determination process will be described with reference to FIG. FIG. 6 is a flowchart showing the flow of the wetness determination process.

  In the wetness determination process, in S300, it is determined whether or not there is a virtual image with respect to the light source obtained in S130 (see FIG. 2). If there is a virtual image (S300: Yes), the process proceeds to S305. If there is no virtual image (S300: No), the process proceeds to S325, and the wetness determination process is repeated for other light sources.

  In subsequent S305, it is determined whether or not the light source is a headlight based on the classification result in S130 (see FIG. 2). If the light source is a headlight (S305: Yes), the process proceeds to S310. If the light source is not a headlight (S305: No), the process proceeds to S325, and the wetness determination process is repeated for other light sources.

  In subsequent S310, it is determined whether or not the luminance ratio between the virtual image and the real image (headlight) is equal to or less than a predetermined value determined for identifying the light source other than the headlight and the headlight. If the luminance ratio is equal to or smaller than the predetermined value (S310: Yes), the process proceeds to S315. If the luminance ratio is greater than the predetermined value (S310: No), the process proceeds to S325, and other light sources The wetness determination is repeated for.

In subsequent S315, the likelihood is calculated. That is, the probability of being a light source is calculated for each light source.
In subsequent S320, it is determined whether or not the determination has been made for all the light sources. If the determination has been completed for all the light sources (S320: Yes), the process is returned to the main routine, and the determination has not been completed ( (S320: No), the process proceeds to S325, and the wetness determination process is repeated for the other light sources until the determination is completed for all the light sources.

(Characteristics of the vehicle control device 1)
In the vehicle control apparatus 1 as described above, when a virtual image (glare) due to road surface reflection is detected from a peripheral image of a moving body including at least a road image acquired by the front camera 10 and the rear camera 12, based on the detection result. Thus, a corrected image in which glare in the image is suppressed is generated.

  That is, when a virtual image due to road surface reflection is detected, an image in which the virtual image is suppressed can be obtained. Since the virtual image is suppressed in the image, only the image due to road surface reflection is not difficult. That is, it is possible to improve difficulty in viewing an image due to road surface reflection.

  Also, a high brightness area in the camera image is extracted, and the shape, position, brightness value, size, moving speed, shape stability of the extracted area, or the presence or absence of a light source that is a real image corresponding to a virtual image of road surface reflection, Generation of a virtual image of road surface reflection is detected by combining at least two or more features.

Therefore, since the generation of a virtual image of road surface reflection can be detected according to the characteristics of glare and real images, the generation of virtual images can be accurately detected.
Further, the virtual image always has a paired light source (real image). Therefore, by combining two or more of these pieces of information, the light spot in the image can be classified into a virtual image or a light source (real image). That is, it is possible to accurately detect the occurrence of a virtual image.

  In addition, the reliability of road surface reflection virtual image detection is calculated for each extracted high-brightness area, and correction is performed according to the calculated reliability of virtual image detection. Since the amount is subtly performed, it is possible to reduce a sense of incongruity and flicker by correction when the determination is not stable.

  Further, the real image of the light source and the glare always make a pair. Therefore, since the light source that is a real image corresponding to the glare is detected using at least one of the shape, position, luminance value, and size of the glare and the light source, the light source that is a real image can be accurately detected.

  Further, it is determined whether or not a virtual image has been generated and a corrected image is generated when it is estimated by the wiper switch 20 that the road surface is raining. That is, since the correction process is performed only when the road surface is raining, the processing load is reduced.

  In addition, since a portion having a luminance value higher than a predetermined value in the camera image is corrected to a luminance value lower than the predetermined value, even if there is a portion of luminance higher than the predetermined value in the image, The luminance of the portion becomes smaller than a predetermined value. In other words, since there is no portion of luminance greater than the predetermined value in the image, an image with appropriate exposure can be obtained.

  Further, based on whether or not the occurrence of glare is detected, it is detected whether or not the road surface is wet, and light distribution of the headlight is controlled based on the detected wet state of the road surface. At this time, whether or not the glare has occurred is detected by image processing, so that the vehicle control device 1 can have a simple configuration.

  Further, during rainy weather when glare is likely to occur and the brightness of the generated glare is high, the light distribution of the headlight is controlled to a rainy condition. That is, if the road surface is wet, it is determined to be rainy, otherwise it is determined to be clear. If it is raining, the amount of light in the direction of the closest road surface is reduced so that, for example, the driver of an oncoming vehicle does not dazzle the road surface reflection glare of the headlight of the own vehicle.

  In this way, by controlling the light distribution of the headlight, it is possible to suppress the generation of a virtual image or to reduce the brightness of the virtual image even when a virtual image is generated. Therefore, a driver other than the own vehicle such as the driver of the oncoming vehicle can drive safely.

  Further, since the movement of the vehicle is controlled via the movement control unit 60 based on the wet state of the road surface, if the road surface is wet due to the wet state of the road surface, it is estimated that the μ value of the road surface is low. If the road surface is not wet, it is estimated that the μ value of the road surface is high, and for example, ABS control or stability control can be performed based on the estimated μ value.

  Further, when the glare is generated, the road surface is wet. When the glare is not generated, it is determined that the road surface is dry. Therefore, the wet state of the road surface can be easily detected. it can.

  Further, when the luminance ratio between the luminance of the real image of the light source and the luminance of the glare is lower than a predetermined value, it is determined that the road surface is in a wet state, so that the road surface is more accurately detected. be able to.

  In other words, the luminance ratio between the luminance of the real image of the light source and the luminance of the virtual image due to the road surface reflection differs between when the road surface is wet and when it is dry (the luminance ratio of the light source is low because the reflectance of the road surface in the dry state is low). Is higher). Therefore, if the luminance ratio between the luminance of the real image of the light source and the luminance of the virtual image due to road surface reflection is set in advance as the luminance ratio when the road surface is in a wet state, the difference in luminance ratio between the wet state and the dry state may be caused. It is possible to more accurately determine whether or not the road surface is wet.

  In addition, it is determined whether the light source is a headlight of a vehicle ahead by the position of the light source and the luminance intensity. If the light source is determined to be a headlight, the luminance ratio between the luminance of the real image of the light source and the luminance of the glare is predetermined If it is not higher than the value of, it is determined not to be wet. Therefore, it can be determined whether it is a street light or a vehicle light depending on the position of the light source, and it can be determined whether it is a headlight or a taillight based on the luminance intensity of the light source. That is, it can be determined whether or not the light source in the image is the headlight of the vehicle ahead.

  If it is determined to be a headlight, as described above, the luminance ratio between the real image of the headlight and the virtual image is higher than that of the taillight, so the luminance of the real image of the light source and the luminance of the virtual image due to road surface reflection If the brightness ratio of the headlight and the light source other than the headlight is not higher than a predetermined value determined to identify the light source, it is determined that the road surface is wet. It can be judged accurately.

  In addition, when the road surface is in a wet state, the criteria for determining whether or not the vehicle is wet is weighted according to the position of the light source and the luminance intensity. Becomes more accurate.

  In addition, since the weight of the criterion when there is a light source that generates glare is larger than the weight of the criterion when there is a light source that does not generate glare, the road surface is wet or dry. Can be determined easily and accurately.

  In addition, when there are a plurality of light sources used for determining whether or not the road surface is wet, the wet determination is performed for each of the plurality of light sources. Therefore, based on the determination results of all the light sources, the road surface is in a wet state. Whether or not there is is determined. Therefore, it is possible to determine whether or not the road surface is wet by integrating a plurality of light sources.

  In addition, when there are multiple light sources, the wetness determination results for each of the multiple light sources are output with likelihood, so the determination for each light source and its integration result in a lot of information that can be used for determination when there are many light sources. To be able to do that. Also, by using the likelihood, the integration process can be simple and conform to the theory.

Even if a virtual image is generated due to road surface reflection, an image of the rear of the vehicle in which the virtual image is suppressed is displayed on the display device 40 attached instead of the rearview mirror, so that the driver can easily confirm the rear. Become. Therefore, safety during driving can be improved.
[Second Embodiment]
In the first embodiment, the non-glare image is created by replacing the brightness of the glare portion with the brightness calculated by adding the reliability calculated in S140 to the brightness of the entire screen, but in the second embodiment, The vehicle control device may control the exposure of the front camera 10 and the rear camera 12 to obtain a camera image with appropriate exposure.

  The configuration of the vehicle control device of the second embodiment is the same as that of the vehicle control device 1 of the first embodiment except for the front camera 10 and the rear camera 12, and therefore the description of the same parts is omitted, and only different parts. Will be explained.

The front camera 10 and the rear camera 12 in the second embodiment are configured so that the exposure can be controlled by a command from the outside.
In the image processing (see FIG. 2) executed in the image processing unit 30, in the first embodiment, a camera image is acquired (S100 in FIG. 2), and the acquired image is binarized (S115 in FIG. 2). However, the following processes (a) and (b) are added between S110 and S120.

  (A) As S112, the part more than a predetermined luminance value is deleted from the camera image acquired in S110, and the exposure of the front camera 10 and the rear camera 12 is calculated from the image from which the part more than the predetermined luminance value is deleted.

(B) As S114, the exposure of the front camera 10 and the rear camera 12 is controlled with the exposure calculated in S112, and a camera image is acquired again.
In the vehicle control apparatus of the second embodiment as described above, an image with appropriate exposure can be acquired. In other words, normally, when glare with high luminance due to road surface reflection spreads greatly on the screen, the front camera 10 and the rear camera 12 are underexposed and an image suitable for image processing cannot be obtained.

  However, according to the processing of the second embodiment, even if glare having a large luminance exists, images are acquired by the front camera 10 and the rear camera 12 without being affected by the luminance of that portion. The exposure of the rear camera 12 will not be underexposed. Therefore, it is possible to acquire an image with appropriate exposure.

[Third Embodiment]
Next, the vehicle control device 2 including the millimeter wave radar 90 will be described with reference to FIG. FIG. 7 is a block diagram illustrating a schematic configuration of the vehicle control device 2. As shown in FIG. 7, the vehicle control device 2 includes a millimeter wave radar 90 in the vehicle control device 1 of the first embodiment.

Since the configuration is the same as that of the vehicle control device 1 of the first embodiment except that the millimeter wave radar 90 is provided, description of components other than the millimeter wave radar 90 is omitted.
The millimeter wave radar 90 is for detecting a non-road surface object that is an object other than a road surface, and includes an antenna, an oscillator, an amplifier, a modulator / demodulator, and the like (not shown), and transmits a millimeter wave from the non-road surface object. The reflected wave is received, and the distance and direction of the non-road surface object (that is, the position of the non-road surface object) are detected.

Then, the following processes (c) to (e) are added to the image processing (see FIG. 2) executed in the image processing unit 30.
(C) In S112, the position of the non-road surface object is acquired from the millimeter wave radar 90.

  (D) In S142, the non-road surface object in the image is identified from the position of the non-road surface object acquired in S112, the virtual image is extracted from the segment classified in S140, and it is neither a non-road surface object nor a virtual image. The part which does not exist is extracted as a road surface.

(E) In S145, the screen information is corrected by replacing the portion where the virtual image is generated with the road surface information extracted in S142, and a non-glare image is created.
As described above, in the vehicle control device 2 of the third embodiment, the millimeter wave radar 90 detects a non-road surface object such as a structure other than a road surface or a vehicle or a pedestrian, and is not a detected non-road surface object, and A portion that is not a virtual image is extracted as a road surface, and a portion where a virtual image is generated is corrected based on the extracted image information of the road surface. Therefore, since the road surface is accurately extracted, the portion where the virtual image is generated, which is performed based on the extracted image information of the road surface, is accurately performed.

[Other Embodiments]
(1) In the above embodiment, in the non-glare image creation in S145, the reliability calculated in S140 is integrated with the brightness of the glare portion and the brightness of the entire screen for the light source and the pair of glare obtained in S140. Instead of replacing with the calculated luminance, the glare portion may be corrected by replacing it with information on the road surface where the virtual image obtained in S140 is not generated.

In this case, since the processing load is lighter than the processing of the above embodiment, an image with appropriate exposure can be obtained more efficiently.
(2) In the feature amount detection process in S125, the color, brightness, or pattern of the road surface is extracted from the camera image. In S145, based on at least one of the color, brightness, or pattern of the road surface extracted in S125, The part where the virtual image is generated is corrected. In the correction, the road surface color may be averaged, and the entire road surface may be replaced with the averaged color.

In this way, the road surface can be corrected by combining the color, brightness, or pattern of the road surface in accordance with the state of the road surface, so that an image with a more appropriate exposure can be obtained.
(3) Further, when creating a non-glare image in S145, correction may be performed using a preset luminance distribution in accordance with illumination characteristics. In this way, it is possible to correct the location where the virtual image is generated in the camera image using the preset luminance distribution according to the illumination characteristics of the illumination, and thus it is possible to perform more appropriate correction. .

  For example, when irradiating a road surface with a headlight of a vehicle, since the irradiation characteristics such as the luminance distribution according to the irradiation range and distance are known in advance, for example, the distance is dark (the luminance is low). If the correction is made such that the vicinity is bright (the brightness is high), an image with appropriate correction can be obtained.

  (4) In the above embodiment, the display device 40 is attached instead of the rearview mirror, and the vehicle rear image acquired by the rear camera 12 is corrected and then displayed on the display device 40. However, the display device 40 is displayed on the car navigation device. The image may be shared with the other display device, and after correcting the vehicle front image acquired by the front camera 10, the image may be displayed on the display device 40.

  In this way, even when it is difficult to see the front, such as in rainy weather, the driver can drive while looking at the corrected vehicle front image displayed on the display device 40, so that the driver can drive safely.

1 is a block diagram showing a schematic configuration of a vehicle control device 1. FIG. It is a flowchart which shows the flow of an image process. It is a figure which shows the example of a non-glare image. It is a flowchart which shows the flow of a feature-value detection process. It is a figure which shows the feature-value to detect and the detection content. It is a flowchart which shows the flow of a wet determination process. 2 is a block diagram showing a schematic configuration of a vehicle control device 2. FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1, 2 ... Vehicle control apparatus, 5 ... In-vehicle image display apparatus, 10 ... Front camera, 12 ... Rear camera, 20 ... Wiper switch, 30 ... Image processing part, 40 ... Display apparatus, 50 ... Light distribution control part, 60 ... motion control unit, 70 ... headlight, 90 ... millimeter wave radar.

Claims (21)

Imaging means for acquiring a peripheral image of the moving body including at least a road image;
Virtual image detection means for detecting the occurrence of a virtual image due to road surface reflection from the image acquired by the imaging means;
It is determined whether or not a virtual image due to the road surface reflection is generated by the virtual image detection unit, and when the virtual image is generated, based on a detection result of the virtual image detection unit, the road image reflection by an image acquired by the imaging unit Road surface reflection suppression means for generating a corrected image in which a virtual image is suppressed;
Equipped with a,
The virtual image detecting means includes
Extracting a high brightness area in the image acquired by the imaging means;
The reliability of the virtual image detection of road surface reflection for each of the extracted high luminance regions is calculated based on the degree of vertical deviation of the light source and virtual image pair, the degree of size difference, and the degree of brightness difference,
The road surface reflection suppression means is
An image processing apparatus that performs image correction that suppresses a virtual image due to road surface reflection in an image acquired by the imaging unit according to the reliability of the virtual image detection calculated by the virtual image detection unit. The image processing apparatus according to claim 1.
The virtual image detecting means includes
Shape before Ki抽 out area, the position, luminance value, size, moving speed, the shape stability or the presence or absence of the light source is a real image corresponding to the virtual image of the road surface reflection, a combination of at least two or more characteristics of the An image processing apparatus for detecting occurrence of a virtual image of the road surface reflection. The image processing apparatus according to claim 1 or 2 ,
The virtual image detecting means includes
Using a light source that is a real image corresponding to the road reflection virtual image, using at least one of a shape, a position, a luminance value, or a size of the light source that is a real image corresponding to the road reflection virtual image and the road reflection virtual image. An image processing apparatus characterized by detecting. In the image processing device according to any one of claims 1 to 3 ,
Comprising a rain state estimating means for estimating that the road surface is in a rain state,
The virtual image detecting means includes
An image processing apparatus that executes determination of whether or not the virtual image has occurred and generation of a corrected image when the road surface state is estimated to be raining by the rain state estimating unit. The image processing apparatus according to any one of claims 1 to 4 ,
The imaging means is configured to be able to control exposure of an image to be acquired by an external command,
The road surface reflection suppression means is
The exposure of the imaging means is configured to be controllable,
The virtual image detection by the road surface reflection is performed based on an image acquired by performing exposure control of the imaging unit without using a portion in which the luminance in the image acquired by the imaging unit is not less than a predetermined value. Image processing device. The image processing apparatus according to any one of claims 1 to 4 ,
The road surface reflection suppression means is
An image processing apparatus, wherein a portion having a luminance value larger than a predetermined value in an image acquired by the imaging unit is corrected to a luminance value lower than a predetermined value. The image processing apparatus according to any one of claims 1 to 4 ,
The road surface reflection suppression means is
Road surface reflection virtual image extraction means for extracting a place where a virtual image due to road surface reflection has occurred from the image acquired by the imaging means;
An image processing apparatus, wherein an image of a portion where a virtual image due to road surface reflection extracted by the road surface reflection virtual image extracting means is corrected with information on an image of a road surface where a virtual image due to road surface reflection has not occurred. The image processing apparatus according to claim 7 .
Non-road surface detection means for detecting a non-road surface object that is an object other than the road surface;
Road surface extraction means for extracting, as the road surface, a portion that is not the non-road surface object detected by the non-road surface object detection means but is not a virtual image due to road surface reflection extracted by the road surface reflection virtual image extraction means;
With
The road surface reflection suppression means is
An image processing apparatus that corrects a portion where a virtual image is generated by the road surface reflection based on image information of the road surface extracted by the road surface extraction unit. The image processing apparatus according to claim 8 .
The road surface extraction means is configured to be able to extract the color, brightness or pattern of the road surface,
The road surface reflection suppression means is
An image processing apparatus that corrects a portion where a virtual image is generated based on at least one of a color, luminance, or pattern of a road surface extracted by the road surface extraction unit. The image processing apparatus according to any one of claims 1 to 9 ,
The road surface reflection suppression means is
In the correction, the image processing apparatus is corrected using a luminance distribution set in advance according to the illumination characteristic of illumination. Vehicle image processing apparatus characterized by the onboard image processing apparatus according to any one of claims 1 to 10. A display means for displaying an image;
An in-vehicle image display device, wherein an image obtained by the in-vehicle image processing device according to claim 11 is displayed on the display means. The in-vehicle image display device according to claim 12 ,
The display means is attached to the rear viewing mirror part,
The in-vehicle image display device, wherein the imaging means is configured to acquire a rear image of a vehicle including at least a road image. An in-vehicle image processing device according to claim 11 ;
Road surface wetness detection means for detecting whether or not the road surface is wet based on the detection result of virtual image generation due to road surface reflection in the virtual image detection means of the in-vehicle image processing device;
Light distribution control means for performing light distribution control of the headlight of the vehicle;
A vehicle control device that controls light distribution of the headlight via the light distribution control means based on a wet state of the road surface detected by the road surface wetness detection means. The vehicle control device according to claim 14 , wherein
A motion control means for controlling the motion of the vehicle;
A vehicle control device that controls movement of the vehicle via the movement control means based on a wet state of the road surface detected by the road surface wetness detection means. The vehicle control device according to claim 14 or 15 ,
The road surface wetness detecting means is
Vehicle control characterized by determining that the road surface is wet when a virtual image due to the road surface reflection occurs, and that the road surface is dry when the virtual image due to the road surface reflection is not generated. apparatus. The vehicle control device according to claim 16 , wherein
The road surface wetness detecting means is
Taking the luminance ratio between the luminance of the real image of the light source obtained by the virtual image detecting means and the luminance of the virtual image due to road surface reflection, and when the road surface reflection has a brightness higher than a predetermined luminance ratio with respect to the light source, the road surface It is determined that the vehicle is in a wet state. The vehicle control device according to claim 16 , wherein
A light source type determining means for determining whether the light source is a headlight of a preceding vehicle according to the position and luminance intensity of the light source;
The road surface wetness detecting means is
When the light source type determining means determines that the light source is a headlight,
If the luminance ratio between the luminance of the real image of the light source and the luminance of the virtual image due to road surface reflection is not higher than a predetermined value determined for identifying the light source other than the headlight and the headlight, it is not determined to be wet. A vehicle control device. The vehicle control device according to claim 18 , wherein
The road surface wetness detecting means is
When determining whether or not the road surface is in a wet state, a weight based on the position and luminance intensity of the light source determined by the light source type determination unit as the headlight is attached to a determination criterion. Vehicle control device. The vehicle control device according to claim 19 ,
The road surface wetness detecting means is
The vehicle control device according to claim 1, wherein a weight of a determination criterion when there is a light source that generates a virtual image due to road surface reflection is greater than a weight of a determination criterion when there is a light source that does not generate a virtual image due to road surface reflection. The vehicle control device according to any one of claims 18 to 20 ,
The road surface wetness detecting means is
When there are a plurality of light sources used for the determination, performing wet determination for each of the plurality of light sources, and determining whether or not the road surface is in a wet state based on determination results for all of the plurality of light sources. A vehicle control device.