JP2020180924A - Road surface state estimating device, vehicle control system, and road surface state estimating method - Google Patents

Abstract

To estimate the road surface state and friction coefficient of a road surface with good accuracy.SOLUTION: A road surface state estimating device 1 comprises: a polarization camera 10 for imaging a road surface in the traveling direction of a moving vehicle and thereby generating a polarized image of the road surface; a calculation unit 20 for processing the polarized image of the road surface and thereby calculating a road surface parameter that represents the polarization state of the road surface; a natural light estimation unit 30 for estimating the polarization state of natural light with which the road surface is irradiated; a determination unit 40 for determining, on the basis of the polarization state estimated by the natural light estimation unit 30, the classification parameter used to classify the road surface state of the road surface into one of a plurality of categories corresponding to a moisture state; a classification unit 50 for classifying the road surface state of the road surface into one of the plurality of categories on the basis of the road surface parameter and the classification parameter; and a friction coefficient estimation unit 60 for estimating the friction coefficient of the road surface on the basis of the classified category.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to a road surface condition estimation device, a vehicle control system, and a road surface condition estimation method.

In recent years, the development of various technologies for realizing safe autonomous driving has been promoted. For safe autonomous driving, it is required to accurately determine the road surface condition in the traveling direction and accurately estimate the friction coefficient of the road surface. For example, Non-Patent Document 1 discloses a technique for estimating a road surface state by detecting a polarization ratio of light reflected from a road surface. Further, as a technique for utilizing polarized light, for example, Patent Document 1 discloses a technique for determining forward light or backlight from sky polarized light information.

Japanese Unexamined Patent Publication No. 2012-191267

Muneo Yamada, 4 outsiders, "Development of in-vehicle road surface condition detection sensor by image processing", IEEJ Journal C, 2004, Vol. 124, No. 3, p. 753-760

However, in the above-mentioned conventional technique, the relationship between the estimated road surface condition and the friction coefficient is unknown, and the friction coefficient cannot be estimated appropriately.

Further, the polarization ratio of the light reflected from the road surface changes according to the polarization state of the light before the reflection (light incident on the road surface). Therefore, the estimation result of the road surface condition may differ depending on the polarization state of the incident light. For example, when natural light is used in the daytime, the polarization state of the natural light changes according to the weather, so that the polarization ratio of the reflected light also changes. Therefore, an error may occur in the estimation result of the road surface condition.

Therefore, the present disclosure provides a road surface condition estimation device, a vehicle control system, and a road surface condition estimation method capable of accurately estimating the road surface condition and the friction coefficient of the road surface.

In order to solve the above problems, the road surface condition estimation device according to one aspect of the present disclosure includes a polarized camera that generates a polarized image of the road surface by photographing the road surface in the traveling direction of the moving body, and a polarized image of the road surface. A calculation unit that calculates a road surface parameter representing the polarization state of the road surface, a first estimation unit that estimates the polarization state of natural light irradiating the road surface, and a plurality of categories according to the water content. Based on the determination unit that determines the classification parameter used to classify the road surface state of the road surface to any one based on the polarization state estimated by the first estimation unit, and the road surface parameter and the classification parameter, It includes a classification unit that classifies the road surface condition of the road surface into any of the plurality of categories, and a second estimation unit that estimates the friction coefficient of the road surface based on the classified categories.

Further, the vehicle control system according to one aspect of the present disclosure is based on the road surface condition estimation device, a presentation unit that presents information on the friction coefficient to the driver of the moving body, and the friction coefficient. It includes at least one of the control units that control the operation of the moving body.

Further, in the road surface condition estimation method according to one aspect of the present disclosure, a polarized camera captures the road surface in the traveling direction of the moving body to generate a polarized image of the road surface and to process the polarized image of the road surface. Therefore, the step of calculating the road surface parameter representing the polarization state of the road surface, the step of estimating the polarization state of the natural light applied to the road surface, and the step of estimating the polarization state of the natural light irradiating the road surface, and the road surface of the road surface in any of a plurality of categories according to the moisture state. Based on the step of determining the classification parameter used for classifying the state based on the polarization state of the natural light, and the road surface parameter and the classification parameter, the road surface state of the road surface is classified into one of the plurality of categories. It includes a step of classifying and a step of estimating the friction coefficient of the road surface based on the classified category.

Further, one aspect of the present disclosure can be realized as a program for causing a computer to execute the road surface condition estimation method. Alternatively, it can be realized as a computer-readable non-temporary recording medium in which the program is stored.

According to the present disclosure, the road surface condition and the friction coefficient of the road surface can be estimated with high accuracy.

FIG. 1 is a diagram for explaining an outline of the road surface condition estimation device according to the first embodiment. FIG. 2 is a block diagram showing a configuration of a road surface condition estimation device according to the first embodiment. FIG. 3 is a diagram showing a configuration of a polarized camera included in the road surface condition estimation device according to the first embodiment. FIG. 4 is a diagram showing an example of a captured image generated by the polarized camera according to the first embodiment. FIG. 5 is a diagram showing another configuration of the polarized camera according to the first embodiment. FIG. 6 is a diagram showing the relationship between specular reflection and reflectance generated at the interface between air and water surface. FIG. 7 is a diagram showing the polarization ratio for each observation angle. FIG. 8 is a diagram for explaining the reflected light from the dry road surface in cloudy weather. FIG. 9 is a diagram for explaining the reflected light from the wet road surface in cloudy weather. FIG. 10 is a diagram for explaining the reflected light from the flooded road surface in cloudy weather. FIG. 11 is a diagram for explaining the reflected light from the snow-covered road surface in cloudy weather. FIG. 12 is a diagram for explaining the reflected light from the frozen road surface in cloudy weather. FIG. 13 is a diagram showing a threshold value used for classifying the road surface condition in cloudy weather. FIG. 14 is a diagram for explaining the reflected light from the dry road surface in fine weather. FIG. 15 is a diagram showing a threshold value used for classifying the road surface condition in fine weather. FIG. 16 is a diagram showing the relationship between the polarization ratio and the coefficient of friction. FIG. 17 is a flowchart showing the operation of the road surface condition estimation device according to the first embodiment. FIG. 18 is a flowchart showing a classification parameter determination process in the operation of the road surface condition estimation device according to the first embodiment. FIG. 19 is a block diagram showing the configuration of the road surface condition estimation device according to the second embodiment. FIG. 20 is a block diagram showing a configuration of the vehicle control system according to the third embodiment.

(Summary of this disclosure)
The road surface condition estimation device according to one aspect of the present disclosure is a polarized camera that generates a polarized image of the road surface by photographing the road surface in the traveling direction of the moving body, and processes the polarized image of the road surface. A calculation unit that calculates a road surface parameter representing the polarization state of the road surface, a first estimation unit that estimates the polarization state of natural light irradiating the road surface, and a road surface of the road surface in one of a plurality of categories according to the moisture state. The road surface state of the road surface is determined based on the determination unit that determines the classification parameter used for classifying the state based on the polarization state estimated by the first estimation unit, and the road surface parameter and the classification parameter. It includes a classification unit that classifies into any of a plurality of categories, and a second estimation unit that estimates the friction coefficient of the road surface based on the classified categories.

In this way, the polarization state of natural light is estimated, and the classification parameters are determined based on the estimated polarization state. Therefore, the road surface condition can be classified into an appropriate category according to the polarization state of natural light. This makes it possible to accurately estimate the road surface condition and the friction coefficient of the road surface.

Further, for example, in the road surface condition estimation device according to one aspect of the present disclosure, the polarized camera further generates a polarized image of the sky by photographing the sky in the traveling direction of the moving body, and the first. The estimation unit may estimate the polarized state of the natural light emitted from the sky to the road surface by processing the polarized image of the sky.

Thereby, the polarized image of the sky obtained by the polarized camera can be used to estimate the polarized state of natural light. Therefore, for example, it is not necessary to provide an interface for acquiring information other than the polarized image of the sky, so that the configuration of the road surface state estimation device can be simplified.

Further, for example, in the road surface condition estimation device according to one aspect of the present disclosure, the polarized images in the sky may include polarized images in at least three directions different from each other.

As a result, the degree of polarization can be calculated based on the polarized images in the three directions, so that the estimation accuracy of the polarized state of natural light can be further improved.

By the way, the road surface condition estimation device according to one aspect of the present disclosure is used, for example, to estimate the road surface condition of a distant road surface in the traveling direction while the moving body is moving. Therefore, since the shooting range of the polarized camera changes as the moving body moves, there is a risk that the sky cannot be shot due to obstacles simply by shooting the direction of travel.

On the other hand, in the road surface condition estimation device according to one aspect of the present disclosure, for example, the polarized camera is installed so that the optical axis is substantially horizontal, and the first polarized camera that generates a polarized image of the road surface is used. A second polarized camera, which is installed with its optical axis tilted above the horizontal and generates the polarized image of the sky, may be included.

As a result, a second camera for generating a polarized image of the sky is provided, so that even if the shooting range of the first camera does not include the sky due to an obstacle such as a building, the polarized image of the sky is provided. Can be obtained, and the polarization state of natural light can be estimated. Therefore, the road surface condition and the coefficient of friction can be stably and continuously estimated.

Further, for example, in the road surface condition estimation device according to one aspect of the present disclosure, the first estimation unit is based on the position and orientation of the moving body and the weather information of the moving body or the area including the road surface. The polarization state of the natural light may be estimated.

As a result, even when a polarized image of the sky cannot be acquired, the polarized state of natural light can be estimated based on weather information or the like. Therefore, the road surface condition and the coefficient of friction can be stably and continuously estimated.

Further, for example, in the road surface condition estimation device according to one aspect of the present disclosure, the first estimation unit detects the precipitation and snowfall of the moving body or the area including the road surface, and based on the result, the polarized state of the natural light. May be estimated.

As a result, even when a polarized image of the sky cannot be obtained, the polarized state of natural light can be estimated based on the detection result of rainfall or snowfall. Therefore, the road surface condition and the coefficient of friction can be stably and continuously estimated.

Further, for example, in the road surface condition estimation device according to one aspect of the present disclosure, the determination unit determines the relationship between the unpolarized standard light and the road surface parameter obtained in an environment in which the standard light is irradiated on the road surface. The classification parameter may be determined by correcting the reference parameter to be represented so as to match the polarization state estimated by the first estimation unit.

As a result, the reference parameters under the environment in which the road surface is irradiated with the standard light are appropriately corrected according to the change in the environment, and the appropriate classification parameters according to the environment are determined. This makes it possible to improve the estimation accuracy of the road surface condition and the friction coefficient.

Further, for example, in the road surface condition estimation device according to one aspect of the present disclosure, the plurality of categories may include three categories of dryness, wetness, and freezing or snow cover.

As a result, since there are many categories in which the road surface condition is classified, the estimation accuracy of the friction coefficient can be improved.

Further, for example, in the road surface condition estimation device according to one aspect of the present disclosure, the second estimation unit may estimate the friction coefficient based on the information regarding the tire of the moving body.

Thereby, for example, the estimation accuracy of the friction coefficient can be further improved based on the degree of tire wear and the like.

Further, for example, the road surface condition estimation device according to one aspect of the present disclosure further determines the exposure amount of the polarized camera based on at least one of the polarized image of the road surface and the polarized state estimated by the first estimation unit. The exposure amount control unit may be provided.

As a result, for example, when the road surface and the sky are photographed at the same time, the difference in the amount of light is large. Therefore, by controlling the amount of exposure, signal saturation can be suppressed, or a weak signal can be appropriately detected. can do.

Further, the vehicle control system according to one aspect of the present disclosure is based on the road surface condition estimation device, a presentation unit that presents information on the friction coefficient to the driver of the moving body, and the friction coefficient. It includes at least one of the control units that control the operation of the moving body.

As a result, the coefficient of friction is estimated with high accuracy, so that it is possible to present appropriate information to the driver and appropriately control the vehicle.

Further, in the road surface condition estimation method according to one aspect of the present disclosure, a polarized camera captures the road surface in the traveling direction of the moving body to generate a polarized image of the road surface and to process the polarized image of the road surface. Therefore, the step of calculating the road surface parameter representing the polarization state of the road surface, the step of estimating the polarization state of the natural light applied to the road surface, and the step of estimating the polarization state of the natural light irradiating the road surface, and the road surface of the road surface in any of a plurality of categories according to the moisture state. Based on the step of determining the classification parameter used for classifying the state based on the polarization state of the natural light, and the road surface parameter and the classification parameter, the road surface state of the road surface is classified into one of the plurality of categories. It includes a step of classifying and a step of estimating the friction coefficient of the road surface based on the classified category.

In this way, the polarization state of natural light is estimated, and the classification parameters are determined based on the estimated polarization state. Therefore, the road surface condition can be classified into an appropriate category according to the polarization state of natural light. This makes it possible to accurately estimate the road surface condition and the friction coefficient of the road surface.

Hereinafter, embodiments will be specifically described with reference to the drawings.

It should be noted that all of the embodiments described below show comprehensive or specific examples. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, etc. shown in the following embodiments are examples, and are not intended to limit the present disclosure. Further, among the components in the following embodiments, the components not described in the independent claims will be described as arbitrary components.

Further, each figure is a schematic view and is not necessarily exactly illustrated. Therefore, for example, the scales and the like do not always match in each figure. Further, in each figure, substantially the same configuration is designated by the same reference numerals, and duplicate description will be omitted or simplified.

Further, in the present specification, terms indicating relationships between elements such as horizontal or vertical, and numerical ranges are not expressions expressing only strict meanings, but substantially equivalent ranges, for example, differences of about several percent. It is an expression that means that also includes.

(Embodiment 1)
[1. Overview]
First, an outline of the road surface condition estimation device according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram for explaining an outline of the road surface condition estimation device 1 according to the present embodiment.

FIG. 1 shows a moving body 2 on which the road surface condition estimation device 1 is mounted. The moving body 2 is a vehicle such as a four-wheeled vehicle or a two-wheeled vehicle, but may be an air vehicle called a drone. A polarization camera 10 included in the road surface condition estimation device 1 is attached to the moving body 2. For example, the polarized camera 10 is mounted inside the moving body 2 at a height of about 1.5 m from the ground. The polarized camera 10 may be mounted on the outside of the moving body 2, and the mounting position and height are not particularly limited.

The road surface condition estimation device 1 estimates the road surface condition and the friction coefficient of the road surface 3 in the traveling direction of the moving body 2. Specifically, the road surface 3 is located far in front of the moving body 2, and in the example shown in FIG. 1, it is a road surface located 50 m ahead. The road surface 3 50 m ahead is the distance reached in 3.6 seconds when the moving body 2 moves at a speed of 50 km / h. Since the road surface 3 is far away, it is possible to secure a time during which the moving body 2 can be controlled based on the estimation results of the road surface condition and the friction coefficient. The position of the road surface 3 to be detected may be changed according to the moving speed of the moving body 2. For example, the faster the moving speed, the farther the position may be set as the road surface 3 to be detected. The road surface 3 is, for example, asphalt. The road surface 3 may be a concrete or gravel road.

In this embodiment, the polarized camera 10 generates a polarized image of the road surface 3. As shown in FIG. 1, natural light from the blue sky 4 or the clouds 5 is incident on the road surface 3 as incident light Li. The polarizing camera 10 receives the reflected light Lr generated by the road surface 3 reflecting the incident light Li. The reflection by the road surface 3 includes scattered reflection and specular reflection. The polarization state of the reflected light Lr changes according to the ratio of the diffuse reflection and the specular reflection.

Further, as will be described in detail later, the degree of polarization is different between the natural light from the blue sky 4 and the natural light from the clouds 5. In the present embodiment, the polarized camera 10 captures the sky to generate a polarized image of the sky. The road surface condition estimation device 1 estimates the polarization state of the incident light Li based on the polarized image of the sky. The road surface condition estimation device 1 estimates the road surface condition and the friction coefficient of the road surface 3 based on the estimated polarization state and the modified image of the road surface 3.

In addition, in FIG. 1, the polarization state of light is schematically illustrated by a circle including one or more arrows inside. For example, a circle containing one double-headed arrow indicates that the light is polarized in the direction of the arrow. A circle containing multiple arrows extending radially indicates that the light is unpolarized. Further, the arrows included in the four circles shown in the vicinity of the polarized camera 10 represent the four directions of the polarized image generated by the polarized camera 10. The circle including these arrows means the same in other figures.

[2. Constitution]
Next, a specific configuration of the road surface condition estimation device 1 will be described with reference to FIG. FIG. 2 is a block diagram showing the configuration of the road surface condition estimation device 1 according to the present embodiment.

As shown in FIG. 2, the road surface condition estimation device 1 includes a polarization camera 10, a calculation unit 20, a natural light estimation unit 30, a determination unit 40, a classification unit 50, and a friction coefficient estimation unit 60. Further, the road surface condition estimation device 1 includes an output unit 70, a tire information acquisition unit 80, and an exposure amount control unit 90.

The polarized camera 10 generates a polarized image of the road surface 3 by photographing the road surface 3 in the traveling direction of the moving body 2. In the present embodiment, the polarized camera 10 further captures the sky in the traveling direction of the moving body 2 to generate a polarized image of the sky. The sky in the traveling direction is an empty region (opposing sky region) facing the moving body 2 with the road surface 3 in between.

As shown in FIG. 1, the polarized camera 10 is mounted in the vicinity of the windshield 6 (see FIG. 3) inside the moving body 2. The polarizing camera 10 is installed so that the optical axis J is substantially horizontal, and photographs the front direction of the moving body 2. Approximately horizontal means, for example, a range within ± 10 ° with respect to a horizontal plane.

FIG. 3 is a diagram showing a configuration of a polarized camera 10 included in the road surface condition estimation device 1 according to the present embodiment. As shown in FIG. 3, the polarizing camera 10 includes an image sensor 11 and a mosaic polarizing plate 12. Further, the polarized camera 10 may include an optical system such as a lens and an aperture.

The image sensor 11 has a plurality of pixels 13 arranged in a two-dimensional matrix. The optical axis J is a direction orthogonal to the plane on which the plurality of pixels 13 are arranged. In FIG. 3, only one of the plurality of pixels 13 is schematically shown. The plurality of pixels 13 have the same configuration as each other.

As shown in FIG. 3, pixel 13 has four sub-pixels 14-17. Each of the sub-pixels 14 to 17 photoelectrically converts the incident light to generate an electric signal according to the intensity of the incident light. Light that has passed through the mosaic polarizing plate 12 is incident on each of the sub-pixels 14 to 17. In the mosaic polarizing plate 12, regions having different polarization characteristics are arranged in a two-dimensional matrix so as to correspond to the sub-pixels 14 to 17. As a result, light polarized in different directions is incident on each of the sub-pixels 14 to 17.

Specifically, light polarized in the 0 ° direction (horizontal direction, S-polarized light) is incident on the sub-pixel 14. Light polarized in the 90 ° direction (vertical direction, P-polarized light) is incident on the sub-pixel 15. Light polarized in the 45 ° direction is incident on the sub-pixel 16. Light polarized in the 135 ° direction is incident on the sub-pixel 17.

Since each of the plurality of pixels 13 includes sub-pixels 14 to 17, it is possible to generate a polarized image in a predetermined direction based on the signal intensity of the electric signal generated by the sub-pixels corresponding to the same polarization characteristics. it can. Specifically, a polarized image in the 0 ° direction is generated based on the plurality of sub-pixels 14. A polarized image in the 90 ° direction is generated based on the plurality of sub-pixels 15. A polarized image in the 45 ° direction is generated based on the plurality of sub-pixels 16. A polarized image in the 135 ° direction is generated based on the plurality of sub-pixels 17.

FIG. 4 is an example of a captured image 18 generated by the polarized camera 10 according to the present embodiment. Since the polarized camera 10 is installed so that the optical axis J is substantially horizontal, a horizontal line is included in the center of the captured image 18. The captured image 18 includes a road surface region 19a and an empty region 19b with the horizon as a boundary. The road surface region 19a includes the road surface 3. The image constituting the road surface region 19a is a polarized image of the road surface 3. The image constituting the empty region 19b is a polarized image of the sky. The empty area 19b is an opposite air area including the sky facing the moving body 2 with the road surface 3 in between.

Here, one polarized camera 10 simultaneously captures the road surface 3 and the sky to generate a polarized image of the road surface 3 and a polarized image of the sky as one captured image 18, but the present invention is not limited to this. For example, as shown in FIG. 5, the polarized camera 10 may include two polarized cameras 10a and 10b.

Further, in the example shown in FIG. 3, an example in which the polarized camera 10 generates a polarized image in four directions has been described, but the present invention is not limited to this. The polarized camera 10 may generate a polarized image in two or three directions.

FIG. 5 is a diagram showing another configuration of the polarized camera 10 according to the present embodiment. Here, the differences from the configuration of the polarized camera 10 shown in FIG. 3 will be mainly described, and the description of the common points will be omitted or simplified.

The polarized camera 10a is an example of a first polarized camera that is installed so that the optical axis Ja is substantially horizontal and generates a polarized image of the road surface 3. The polarizing camera 10a includes an image sensor 11a and a mosaic polarizing plate 12a. The polarized camera 10a generates a polarized image in two directions. Specifically, as shown in FIG. 5, each of the plurality of pixels 13a included in the image sensor 11a has sub-pixels 14 and 15, and does not have sub-pixels 16 and 17 shown in FIG. The polarized camera 10a generates a polarized image of the road surface 3 including the 0 ° direction and the 90 ° direction.

The polarized camera 10b is an example of a second polarized camera in which the optical axis Jb is installed at an angle above the horizontal to generate a polarized image of the sky. The angle formed by the optical axis Jb and the horizontal plane (optical axis Ja) is, for example, in the range of 20 ° or more and 70 ° or less, and is 45 ° as an example, but is not limited to this. The angle formed by the optical axis Jb and the horizontal plane may be less than 20 ° and may be larger than 70 °.

The polarizing camera 10b includes an image sensor 11b and a mosaic polarizing plate 12b. The polarized camera 10b produces a polarized image in three directions. Specifically, as shown in FIG. 5, each of the plurality of pixels 13b included in the image sensor 11b has sub-pixels 14 to 16 and does not have the sub-pixel 17 shown in FIG. The polarized camera 10b generates a polarized image of the sky including the 0 ° direction, the 90 ° direction, and the 45 ° direction.

Returning to FIG. 2, the calculation unit 20 calculates the road surface parameter representing the polarization state of the road surface 3 by processing the polarized image of the road surface 3 generated by the polarization camera 10. Specifically, the calculation unit 20 generates a polarization ratio image by calculating the polarization ratio P / S for each pixel constituting the polarization image of the road surface 3.

The polarization ratio P / S is an example of a road surface parameter, and is a value obtained by dividing the signal intensity of P-polarized light by the signal intensity of S-polarized light. The signal intensity of P-polarized light corresponds to the intensity of polarized light in the 90 ° direction, and is the signal intensity of the electric signal generated by the sub-pixel 15. The signal intensity of S-polarized light corresponds to the intensity of polarized light in the 0 ° direction, and is the signal intensity of the electric signal generated by the sub-pixel 14.

The polarization ratio image is a luminance image shown in the numerical range of 0 to 255 so that P / S = 1 becomes 128. The larger P is, the larger the brightness value of the polarization ratio image is, and the smaller P is, the smaller the brightness value of the polarization ratio image is.

The natural light estimation unit 30 is an example of a first estimation unit that estimates the polarization state of the natural light applied to the road surface 3. The natural light estimation unit 30 estimates the polarized state of natural light radiated from the sky to the road surface 3 by processing a polarized image of the sky.

Specifically, the natural light estimation unit 30 estimates the polarization state of the sky by processing the polarized image of the sky generated by the polarized camera 10. Specifically, the natural light estimation unit 30 calculates the degree of polarization DOP (Degree of Polarization) based on the polarized image of the sky. The degree of polarization DOP is represented by the following (Equation 1).

In (Equation 1), S ₀ , S ₁ , S ₂ and S ₃ are parameters constituting the Stokes vector. In the present embodiment, S ₃ is regarded as ₀ , and S ₀ , S ₁ and S ₂ are represented as follows, respectively.

(Equation 2) S ₀ = C ₀ + C ₉₀

(Equation 3) S ₁ = C _0- C ₉₀

(Equation 4) S ₂ = C _45- C ₁₃₅

Note that C ₀ , C ₉₀ , C ₄₅ and C ₁₃₅ correspond to signal strengths in the 0 ° direction, 45 ° direction, 90 ° direction and 135 ° direction, respectively. Specifically, C ₀ , C ₉₀ , C _45, and C ₁₃₅ are values representing the signal strength of the electric signal obtained in each of the sub-pixels 14 to 17, respectively, in the numerical range of 0 to 255.

When the polarized camera 10b shown in FIG. 5 does not have the sub-pixel 17 in the 135 ° direction, the following (Equation 5) can be used instead of (Equation 4).

(Equation 5) S ₂ = 2 × C _45- C _0- C ₉₀

The natural light estimation unit 30 determines the weather in the area including the road surface 3 or the moving body 2 based on the calculated degree of polarization DOP. Specifically, the natural light estimation unit 30 compares the degree of polarization DOP with a predetermined threshold value. When the degree of polarization DOP is equal to or higher than the threshold value, the natural light estimation unit 30 determines that the weather is fine. When the degree of polarization DOP is less than the threshold value, the natural light estimation unit 30 determines that the weather is cloudy.

When the weather is cloudy, the natural light from the blue sky 4 is scattered by the clouds 5 and becomes unpolarized, so that the degree of polarization DOP becomes sufficiently small. On the other hand, when the weather is fine, the natural light of the blue sky 4 is Rayleigh scattered by atmospheric constituent molecules. Since the Rayleigh scattered natural light is polarized, the value of the degree of polarization DOP becomes large. Therefore, it is possible to determine whether the weather is cloudy or sunny by comparing the degree of polarization DOP with the threshold value.

The determination unit 40 determines the classification parameter based on the polarization state estimated by the natural light estimation unit 30. The classification parameter is a parameter used for classifying the road surface condition of the road surface 3 into any of a plurality of categories according to the water content. The classification parameter is the threshold of the polarization ratio between categories. An example of a specific threshold value will be described later with reference to FIG.

The plurality of categories include three categories: dry (Dry), wet (Wet), and frozen (Ice) or snow cover (Snow). Freezing and snow cover may be independent as one category. That is, the plurality of categories may be four categories: dry, wet, frozen and snow-covered. Alternatively, the plurality of categories may further include flooding (Puddle) as one independent category.

The classification unit 50 classifies the road surface condition of the road surface 3 into any of a plurality of categories based on the road surface parameters calculated by the calculation unit 20 and the classification parameters determined by the determination unit 40. Specifically, the classification unit 50 compares the polarization ratio calculated by the calculation unit 20 with the threshold value determined by the determination unit 40 to change the road surface condition of the road surface 3 to dryness, wetness, freezing, and snow cover. Classify into one of.

The friction coefficient estimation unit 60 is an example of a second estimation unit that estimates the friction coefficient of the road surface 3 based on the categories classified by the classification unit 50. Each of the plurality of categories and the coefficient of friction have a predetermined correspondence relationship. Specifically, the coefficient of friction decreases in the order of drying, wetting, and freezing (or snow cover). For example, the coefficient of friction of a dry road surface is about 0.8. The coefficient of friction of the wet road surface is about 0.5. The coefficient of friction of the road surface in the frozen state is about 0.2. The coefficient of friction of the road surface in a snowy state is about 0.4.

In the present embodiment, the friction coefficient estimation unit 60 estimates the friction coefficient of the road surface 3 based on the tire information regarding the tire of the moving body 2. Specifically, the tire information is information indicating the degree of wear of the tire. The tire information may be information indicating the number of days the tire has been used, or may be information indicating the size of the groove of the tire. The tire information may be information indicating the type of tire such as a studless tire or a spiked tire.

For example, the friction coefficient estimation unit 60 corrects the friction coefficient estimated based on the category based on the tire information. Specifically, when the tire wear degree or the number of days of use is large, or when the tire groove is small, the friction coefficient estimated based on the category is corrected to a smaller value. If the tire wears less or has been used for less time, or if the tire groove is large, the coefficient of friction estimated based on the category will not be corrected.

The output unit 70 outputs the estimation result by the friction coefficient estimation unit 60. For example, the output unit 70 outputs numerical information indicating the friction coefficient estimated by the friction coefficient estimation unit 60. Alternatively, the output unit 70 may output a signal to warn that there is a risk of slipping when the estimated friction coefficient is less than the reference value.

For example, the output unit 70 is realized by a communication interface that communicates with other devices by wire or wirelessly. The other device is, for example, a display device or an audio output device, or a vehicle control device such as an ECU (Electronic Control Unit).

The tire information acquisition unit 80 acquires tire information. For example, the tire information acquisition unit 80 is realized by a sensor that detects the degree of tire wear. Alternatively, the tire information acquisition unit 80 may be an input device that receives input of tire information from the user.

The exposure amount control unit 90 controls the exposure amount of the polarizing camera 10 based on at least one of the polarized image of the road surface 3 and the polarized state estimated by the natural light estimation unit 30. For example, the exposure amount control unit 90 adjusts the aperture or the exposure time of the polarizing camera 10 so that the exposure amount for the pixel 13 whose brightness value is saturated is small based on the polarized image of the road surface 3. Further, for example, when the natural light estimation unit 30 determines that the weather is fine, the aperture or exposure time of the polarizing camera 10 is adjusted so that the exposure amount of the plurality of pixels 13 constituting the empty region 19b becomes small. To do.

The calculation unit 20, the natural light estimation unit 30, the determination unit 40, the classification unit 50, the friction coefficient estimation unit 60, and the exposure amount control unit 90 are realized by, for example, a microcomputer. Specifically, the microcomputer includes a non-volatile memory in which the program is stored, a volatile memory which is a temporary storage area for executing the program, an input / output port, a processor for executing the program, and the like. That is, the calculation unit 20, the natural light estimation unit 30, the determination unit 40, the classification unit 50, the friction coefficient estimation unit 60, and the exposure amount control unit 90 are functional processing units realized by software when the processor executes a program. is there. At least one of the calculation unit 20, the natural light estimation unit 30, the determination unit 40, the classification unit 50, the friction coefficient estimation unit 60, and the exposure amount control unit 90 may be realized by using a dedicated electronic circuit.

[3. Relationship between polarized light and reflection]
Subsequently, the relationship between the polarization and reflection of light will be described with reference to FIG. In the present embodiment, the reflection includes a specular reflection and a diffuse (scattering) reflection, and it is assumed that the surface has a smooth surface in each case.

Specular reflection occurs at the interface between air and the water surface, for example, when the road surface 3 is covered with water. In the case of specular reflection, the polarization of the incident light is maintained. Therefore, the reflected light reflected on the mirror surface becomes light that depends on the polarization of the incident light. Further, when specular reflection is performed on the water surface, the reflectance differs between S-polarized light and P-polarized light.

FIG. 6 is a diagram showing the relationship between specular reflection and reflectance generated at the interface between air and water surface. In FIG. 6, the horizontal axis is the angle of incidence with respect to the water surface, and the vertical axis is the reflectance of light. FIG. 6 shows the reflectances of two types of light, P-polarized light and S-polarized light.

As shown in FIG. 6, when specular reflection occurs on the water surface, the reflectance of S-polarized light is higher than that of P-polarized light. When the angle of incidence is about 53 °, which is the Brewster's angle, the reflectance of P-polarized light becomes zero. Therefore, the more specular reflection is likely to occur on the water surface, the more S-polarized light is contained in the reflected light, and the less P-polarized light is.

On the other hand, in the case of diffuse reflection, the polarization of the incident light is eliminated. Therefore, the diffusely reflected reflected light becomes unpolarized. At this time, when the light is diffused by a medium that refracts light such as snow or ice, the amount of P-polarized light increases when the light passes through the medium and then is emitted from the smooth surface of the medium. This is because, as shown in FIG. 6, since the reflectance of P-polarized light is low, the amount of P-polarized light increases in the refracted light.

FIG. 7 is a diagram showing the polarization ratio for each observation angle. In FIG. 7, the horizontal axis represents the observation angle, and the vertical axis represents the polarization ratio P / S. The observation angle is an angle for observing the reflected light, and the direction perpendicular to the road surface 3 (vertical direction) is 0 ° and the direction parallel to the road surface 3 (horizontal direction) is 90 °. As shown in FIG. 1, the observation angle when the road surface 3 is photographed by the polarizing camera 10 of the moving body 2 far away from the road surface 3 is, for example, in the range of about 85 ° or more and less than 90 °.

As shown in FIG. 7, when specular reflection is predominant, the amount of S-polarized light contained in the reflected light increases, so that the polarization ratio becomes smaller than 128 (P / S = 1). Specifically, when the observation angles are 0 ° and 90 °, the polarization ratio is 128. As the observation angle increases from 0 °, the polarization ratio also decreases, and when the observation angle becomes the Brewster angle (about 53 ° at the interface between water and air), the reflectance of P-polarized light becomes 0. Therefore, the polarization ratio also becomes 0. As the observation angle increases from Brewster's angle to 90 °, the polarization ratio also increases.

On the other hand, when diffuse reflection is dominant, the amount of P-polarized light contained in the reflected light increases, so that the polarization ratio becomes larger than 128. Specifically, when the observation angle is 0 °, the polarization ratio is 128. It gradually increases as the observation angle increases from 0 °, and the slope of the increase also increases from around the Brewster angle.

In the range of observation by the polarizing camera 10 mounted on the moving body 2 (about 85 ° or more and less than 90 °), the polarization ratio differs depending on whether specular reflection is dominant or diffuse reflection is dominant. Therefore, the superiority of reflection can be determined based on the polarization ratio, and the state of the road surface 3 can be determined.

Specifically, specular reflection becomes superior to diffuse reflection when the road surface 3 is wet (Wet) or flooded (Puddle). Diffuse reflection tends to be superior to specular reflection when the road surface 3 is dry or snowy or frozen. However, the above description is limited to the ideal case where the surface is smooth. When the surface of the road surface is rough, the polarized light collapses and only the non-polarized light is observed, so that only the non-polarized light is often observed in the snow cover (Snow). Further, even in the dry state, if the asphalt component contains a content that causes specular reflection, the specular reflection may be dominant.

[4. Relationship between road surface condition and polarized condition (in cloudy weather)]
Hereinafter, the relationship between each road surface state of the road surface 3 and the polarized state of the reflected light will be described with reference to FIGS. 8 to 12. In the following, a case where the road surface 3 is asphalt and the weather is cloudy will be described as a standard state. When the weather is cloudy, the natural light emitted to the road surface 3 is the light obtained by the clouds 5 scattering the light from the sun, and is unpolarized light. That is, the degree of polarization DOP of natural light becomes a sufficiently low value.

<Dry, cloudy>
FIG. 8 is a diagram for explaining the reflected light Lr from the dry road surface 3 in cloudy weather. As shown in FIG. 8, the asphalt road surface 3 has random irregularities. Therefore, the incident light Li is diffusely (scattered) reflected by the road surface 3. Since polarized light is eliminated by diffuse reflection, the reflected light Lr by the road surface 3 becomes unpolarized light.

<Wet, cloudy>
FIG. 9 is a diagram for explaining the reflected light Lr from the wet road surface 3 in cloudy weather. As shown in FIG. 9, water 7 is attached to the road surface 3. Since the amount of water 7 is not sufficiently large, the water surface is not horizontal and has an irregular shape along the unevenness of the surface of the road surface 3. Therefore, a part of the incident light Li is specularly reflected by the water surface, and a part is diffusely reflected. As shown in FIG. 6, since the amount of S-polarized light increases in specular reflection, the reflected light Lr becomes light containing a large amount of S-polarized light (that is, light in which S-polarized light is dominant).

<Puddle, cloudy weather>
FIG. 10 is a diagram for explaining the reflected light Lr from the flooded road surface 3 in cloudy weather. As shown in FIG. 10, the unevenness of the road surface 3 is completely covered with water 7. Therefore, the incident light Li is specularly reflected on the water surface of the water 7. Therefore, since the amount of S-polarized light increases in specular reflection, the reflected light Lr becomes light in which S-polarized light is dominant. Since the amount of light reflected by the mirror surface is larger than that in the wet road surface shown in FIG. 9, the reflected light Lr from the flooded road surface 3 is more S-polarized than the reflected light Lr from the wet road surface 3. More.

<Snow, cloudy weather>
FIG. 11 is a diagram for explaining the reflected light Lr from the snow-covered road surface 3 in cloudy weather. As shown in FIG. 11, the unevenness of the road surface 3 is covered with snow 8. The incident light Li is refracted on the surface of the snow 8 and travels inside the snow 8, is diffused inside the snow 8 to be depolarized, and then is refracted and emitted from the surface of the snow 8. .. The snow 8 is an aggregate of ice crystals, and irregularities are randomly formed on the surface of the snow 8. Therefore, the refracted light is diffusely reflected on the surface of the snow 8. Therefore, the reflected light Lr by the road surface 3 becomes unpolarized light.

<Freezing (Ice), cloudy sky>
FIG. 12 is a diagram for explaining the reflected light Lr from the frozen road surface 3 in cloudy weather. As shown in FIG. 12, the unevenness of the road surface 3 is covered with ice 9. A part of the incident light Li is specularly reflected on the surface of the ice 9 (light Lr1), a part is refracted and travels inside the ice 9, and is diffused inside to become unpolarized, and then the ice. It is refracted and emitted from the surface of 9 (light Lr2). At this time, since the surface of the ice 9 is smoother than that of the snow 8, the light Lr1 contains a large amount of S-polarized light, and the light Lr2 contains a large amount of P-polarized light. The polarized state of the integrated reflected light is determined by the addition of the light Lr1 and the light Lr2.

[5. Polarization ratio threshold for classifying road surface conditions (in cloudy weather)]
The threshold value for classifying the road surface state of the road surface 3 based on the polarization state shown in FIGS. 8 to 12 is as shown in FIG. FIG. 13 is a diagram showing a threshold value used for classifying the road surface condition in cloudy weather. FIG. 13 shows an example of classifying the road surface condition of the road surface 3 into four categories. Specifically, the four categories are dry, wet, snowy, and frozen. In addition, submersion is included in the same category as wet.

Since the reflected light from the snow cover is almost unpolarized, P / S = 1 as shown in FIG. 13, and the light is always located near 128. The threshold value T_wd is a threshold value for discriminating between wet and dry, and is, for example, 95. The threshold value T_di is a threshold value for discriminating between drying and freezing, and is, for example, 110. The threshold value T_is is a threshold value for discriminating between freezing and snow cover, and is, for example, 120. Satisfy T_wd <T_di <T_is. In addition, a threshold value for classifying wetness and flooding may be further provided.

The classification unit 50 classifies the road surface state of the road surface 3 into four categories by comparing at least one of the threshold values T_wd, T_di and T_is with the polarization ratio of the road surface 3 calculated by the calculation unit 20. Specifically, the classification unit 50 classifies the road surface state of the road surface 3 as wet (flooded) when the polarization ratio is less than the threshold value T_wd. When the polarization ratio is equal to or greater than the threshold value T_wd and less than the threshold value T_di, the classification unit 50 classifies the road surface state of the road surface 3 as dry. When the polarization ratio is equal to or greater than the threshold value T_di and less than the threshold value T_is, the classification unit 50 classifies the road surface state of the road surface 3 as frozen. The classification unit 50 classifies the road surface condition of the road surface 3 as snow cover when the polarization ratio is equal to or higher than the threshold value T_is or when the polarization ratio is in the vicinity of 128.

Note that FIG. 13 shows an example in which the polarization ratio ranges of each category do not overlap, but the present invention is not limited to this. For example, since the polarization ratio range of snow cover is around 128 regardless of the weather, it may overlap the polarization ratio ranges of dry and frozen. When the polarization ratio of the road surface 3 is included in the overlapping range, the classification unit 50 may classify the categories based on weather information such as air temperature, road surface temperature, snowfall and precipitation.

[6. Difference between fine weather and cloudy weather]
Next, the difference between fine weather and cloudy weather will be described.

In the cloudy weather described above, the incident light Li on the road surface 3 is natural light scattered by the clouds 5, and is unpolarized light. On the other hand, in fine weather, the incident light Li on the road surface 3 is natural light Rayleigh scattered by atmospheric constituent molecules and is polarized in a predetermined direction. Therefore, the characteristics of the reflected light Lr after being diffusely reflected or scattered and reflected by the road surface 3 also change. In particular, when the road surface condition of the road surface 3 is a dry state, a large difference between cloudy weather and fine weather appears.

FIG. 14 is a diagram for explaining the reflected light Lr from the dry road surface 3 in fine weather. As shown in FIG. 14, the incident light Li in fine weather is polarized in a predetermined direction in addition to having a sufficiently higher illuminance than in cloudy weather. On the road surface 3, a part of the incident light Li is diffusely reflected and becomes unpolarized as in the case of cloudy weather. However, a part of the incident light Li is specularly reflected on the road surface 3 and reflected as light that maintains polarized light as it is. Therefore, the reflected light Lr becomes light containing a large amount of components polarized in the same direction as the incident light Li. Specifically, the reflected light Lr often contains a large amount of P-polarized light, so that the polarization ratio is often higher than in cloudy weather.

FIG. 15 is a diagram showing a threshold value used for classifying the road surface condition in fine weather. Since the snow cover is unpolarized, P / S = 1 and is always located near 128 as shown in FIG. The threshold value T_wd for discriminating between wet and dry is larger than the value in cloudy weather (about 95 in the example shown in FIG. 13), and is, for example, 128. That is, in cloudy weather, the polarization ratio of the dry road surface 3 was less than 1 (S polarization predominant), whereas in fine weather, the polarization ratio of the dry road surface 3 was 1 or more (P polarization predominant). )It has become. Moreover, the polarization ratio in the frozen state is even higher than that in the dry state. Therefore, the threshold value T_di for distinguishing between drying and freezing is also 128 or more.

As described above, since the polarization ratio of the road surface 3 differs depending on whether the weather is cloudy or sunny, it is necessary to appropriately determine the threshold value. In the road surface condition estimation device 1 according to the present embodiment, the determination unit 40 determines an appropriate threshold value based on the estimation result of the polarization state of natural light. As a result, the accuracy of categorization of the road surface condition of the road surface 3 can be improved.

[7. Relationship between friction coefficient and polarization ratio]
Next, the relationship between the polarization ratio of the road surface 3 and the friction coefficient will be described.

FIG. 16 is a diagram showing the relationship between the polarization ratio and the coefficient of friction. In FIG. 16, the horizontal axis represents the polarization ratio of the road surface 3, and the vertical axis represents the friction coefficient of the road surface 3.

The coefficient of friction decreases in the order of dryness, wetness, snow cover, and freezing. In addition, the polarization ratio increases in the order of wet ⇒ dry ⇒ freeze ⇒ snow cover in cloudy weather. In fine weather, it increases in the order of wetness ⇒ snowfall ⇒ dryness ⇒ freezing. Therefore, when the relationship between the polarization ratio and the coefficient of friction is graphed, it is represented by a concave-convex graph having drying as the apex, as shown in FIG. Wetness is present on the left side of the dry (smaller polarization ratio), and snow and freezing are present on the right side of the dry (higher polarization ratio).

The positions of dryness, wetness, and freezing other than snow cover are different between cloudy weather and fine weather, as described above. Therefore, as shown in FIG. 16, the shape of the uneven graph is different between cloudy weather and fine weather. Specifically, in the uneven graph in fine weather, the position of the apex moves to the right side (the side having a large polarization ratio) than in the uneven graph in cloudy weather. According to the road surface condition estimation device 1 according to the present embodiment, the category classification is performed with high accuracy based on the estimation result of the polarization state of natural light, so that the estimation accuracy of the friction coefficient can be improved accordingly.

[8. motion]
Subsequently, the operation of the road surface condition estimation device 1 according to the present embodiment will be described with reference to FIGS. 17 and 18.

FIG. 17 is a flowchart showing the operation of the road surface condition estimation device 1 according to the present embodiment. FIG. 18 is a flowchart showing a classification parameter determination process in the operation of the road surface condition estimation device 1 according to the present embodiment.

As shown in FIG. 17, the polarized camera 10 first generates a polarized image (S10). Specifically, the polarized camera 10 generates a polarized image of the road surface 3 by photographing the road surface 3 in the traveling direction of the moving body 2. In the present embodiment, the polarized camera 10 also generates a polarized image of the sky by photographing the sky in the traveling direction.

Next, the calculation unit 20 calculates the road surface parameter representing the polarized state of the road surface 3 by processing the polarized image of the road surface 3 (S11). Specifically, the polarization ratio P / S is calculated for each pixel 13 of the polarized image of the road surface 3.

Next, the natural light estimation unit 30 estimates the polarization state of the natural light applied to the road surface 3 (S12). Specifically, the degree of polarization DOP of natural light is calculated based on the polarized image of the sky. The calculation of the polarization ratio of the road surface 3 (S11) and the calculation of the degree of polarization DOP of natural light (S12) may be performed first, or may be performed simultaneously by parallel processing.

Next, the determination unit 40 determines the classification parameter based on the estimation result of the polarization state of natural light (S13). Specifically, as shown in FIG. 18, the determination unit 40 determines whether the weather is cloudy or sunny based on the calculated degree of polarization DOP (S21). For example, the determination unit 40 compares the degree of polarization DOP with the threshold value. The determination unit 40 determines that the weather is cloudy when the degree of polarization DOP is less than the threshold value. Further, the determination unit 40 determines that the weather is fine when the degree of polarization DOP is equal to or greater than the threshold value.

When it is cloudy (cloudy in S21), the determination unit 40 determines a classification parameter (specifically, a threshold value) for cloudy weather (S22). For example, the determination unit 40 determines the threshold value T_wd, the threshold value T_di, and the threshold value T_is shown in FIG. 13 as classification parameters.

In the case of fine weather (fine weather in S21), the determination unit 40 determines the classification parameter (specifically, the threshold value) for fine weather (S23). For example, the determination unit 40 determines the threshold value T_wd shown in FIG. 15 as a classification parameter.

Next, as shown in FIG. 17, the classification unit 50 classifies the road surface state of the road surface 3 into any of a plurality of categories based on the road surface parameters and the classification parameters (S14). Specifically, the classification unit 50 classifies the road surface state of the road surface 3 by comparing the polarization ratio of the road surface 3 with the threshold value T_wd or the like.

Next, the friction coefficient estimation unit 60 estimates the friction coefficient of the road surface 3 based on the classified categories (S15). For example, if the classified category is dry, the friction coefficient estimation unit 60 estimates that the friction coefficient is 0.8. The friction coefficient estimation unit 60 may estimate the friction coefficient corresponding to the polarization ratio based on the convex graph shown in FIG. Further, the friction coefficient estimation unit 60 may estimate the friction coefficient based on the tire information.

Next, the output unit 70 outputs the estimation result (S16). For example, the output unit 70 outputs numerical information indicating the friction coefficient estimated by the friction coefficient estimation unit 60. Alternatively, the output unit 70 may output a signal to warn that there is a risk of slipping when the estimated friction coefficient is less than the reference value.

The above processing is repeated, for example, while the moving body 2 is traveling on the road. As a result, the road surface condition of the road surface 3 in the traveling direction of the moving body 2 can be constantly estimated, so that it can be used for safe automatic driving control and safe manual driving support.

[9. Modification example]
Here, a modified example of the road surface condition estimation device 1 according to the first embodiment will be described. In this modification, the processing of the determination unit 40 and the classification unit 50 is different from that of the first embodiment. In the following, the differences from the first embodiment will be mainly described, and the common points will be omitted or simplified.

In the first embodiment, the polarization degree DOP of natural light is calculated based on the polarized image of the sky, the calculated polarization degree DOP is compared with the threshold value, and the threshold value which is an example of the classification parameter is determined based on the comparison result. On the other hand, in this modification, reference parameters are predetermined for each category of the road surface 3. The determination unit 40 according to this modification determines the classification parameter by correcting the reference parameter so as to match the polarization state estimated by the natural light estimation unit 30.

The reference parameter is a parameter representing the relationship between the unpolarized standard light and the road surface parameter obtained in an environment in which the standard light is applied to the road surface 3. Specifically, the reference parameter is a Müller matrix M, which is predetermined for each category and is stored in a memory (not shown) of the road surface state estimation device 1. For example, a dry Muller matrix Md, a wet Muller matrix Mw, a flooded Muller matrix Mp, a snow-covered Muller matrix Ms, and a frozen Muller matrix Mi are stored in the memory. In the following description, when the Müller matrix of each category is not distinguished, it is simply described as the Müller matrix M.

The Müller matrix M is a matrix that converts the incident light Li incident on the road surface 3 into the reflected light Lr by the road surface 3. Specifically, when the Stokes vector of the incident light Li is S _Light and the Stokes vector of the reflected light Lr is S _Meas , the following relationship (Equation 6) is satisfied.

(Equation 6) S _Meas = M × S _Light

(Equation 6) is specifically represented by the following (Equation 7).

In principle, the polarizing camera 10 assumed in this modification cannot acquire S3 (circular polarization component), which is the fourth component of the Stokes vector. Therefore, assuming that S3 = 0, the Stokes component is three-dimensional, and the Müller matrix is 3 × 3, the following (Equation 8) can actually be used.

The components of the Stokes vectors S _Meas and S _Light are as shown in (Equation 2) to (Equation 4). The Stokes vector S _Meas is an example of road surface parameters, and is calculated by the calculation unit 20 based on a polarized image of the road surface 3 obtained by photographing the reflected light Lr of the road surface 3 with a polarizing camera 10. The Stokes vector S _Light is calculated by the natural light estimation unit 30 based on a polarized image of the sky obtained by photographing the sky facing the moving body 2 with the road surface 3 sandwiched by the polarized camera 10.

The determination unit 40 uses the Stokes vector S _Light of the incident light Li calculated by the natural light estimation unit 30 and the Muller matrix M of each category, and calculates according to (Equation 8) to obtain the Stokes vector S of the reflected light Lr. _Meas can be calculated as a classification parameter.

The classification unit 50 classifies the categories of the road surface 3 by comparing the calculated classification parameters for each category with the measured values of the Stokes vector S _Meas of the reflected light Lr calculated by the calculation unit 20. Specifically, the category in which the difference between the estimated value and the actually measured value is the smallest is determined as the category of the road surface 3.

As described above, even in the road surface condition estimation device 1 according to the present modification, the classification parameters are appropriately calculated based on the estimation result of the polarization state of natural light, so that the road surface condition and the friction coefficient of the road surface 3 are estimated accurately. can do.

(Embodiment 2)
Subsequently, the second embodiment will be described.

In the road surface condition estimation device 1 according to the first embodiment, the polarized state of natural light was estimated based on the polarized image of the sky generated by the polarized camera 10. On the other hand, in the road surface condition estimation device according to the second embodiment, information other than the polarized image is acquired, and the polarized state of natural light is estimated based on the acquired information. In the following, the differences from the first embodiment will be mainly described, and the common points will be omitted or simplified.

FIG. 19 is a block diagram showing the configuration of the road surface condition estimation device 101 according to the present embodiment. As shown in FIG. 19, the road surface condition estimation device 101 is different from the road surface condition estimation device 1 according to the first embodiment in that it includes a natural light estimation unit 130 instead of the natural light estimation unit 30. Further, the road surface condition estimation device 101 is communicably connected to the server device 102 and the vehicle sensor 103.

The server device 102 is a server device that stores weather information. The weather information is, for example, information that indicates the weather for each region on an hourly basis. Specifically, the weather information indicates whether the weather in each region is cloudy or sunny. The weather is not limited to cloudy and sunny weather, and may include the presence or absence of precipitation and snowfall. In addition, the weather information may indicate the amount of solar radiation, the wind direction, the wind speed, and the like for each region.

The vehicle sensor 103 is a sensor that detects the position and orientation of the moving body 2. For example, the vehicle sensor 103 is a GPS (Global Positioning System) receiver, but is not limited to this. The vehicle sensor 103 acquires the current position of the moving body 2 by latitude and longitude, and acquires the direction (traveling direction) of the moving body 2 based on the time change of the current position.

The natural light estimation unit 130 estimates the polarization state of natural light based on the position and orientation of the moving body 2 and the weather information of the area including the moving body 2 or the road surface 3. Specifically, as shown in FIG. 19, the natural light estimation unit 130 acquires the position and orientation of the moving body 2 from the vehicle sensor 103. The natural light estimation unit 130 transmits the acquired position and direction (traveling direction) of the moving body 2 to the server device 102, and as a response, receives weather information of the area including the moving body 2 or the road surface 3 from the server device 102. get. The natural light estimation unit 130 can determine whether the area including the moving body 2 or the road surface 3 is sunny or cloudy based on the weather information, and outputs the determination result to the determination unit 40 as the estimation result of the natural light. .. As a result, the determination unit 40 can appropriately determine the classification parameters based on the estimation result of natural light in the same manner as in the first embodiment.

The natural light estimation unit 130 may estimate the polarization state of natural light based on the result of detecting rainfall and snowfall in the area including the moving body 2 or the road surface 3. For example, the moving body 2 is provided with a moisture sensor for detecting rainfall and snowfall, and the natural light estimation unit 130 acquires the detection result by the moisture sensor. Alternatively, the natural light estimation unit 130 may determine the presence or absence of rainfall and snowfall by performing image processing on the visible light camera provided on the moving body 2. Further, the natural light estimation unit 130 may acquire weather information indicating the detection results of precipitation and snowfall from the server device 102. The natural light estimation unit 130 estimates that the weather is cloudy when at least one of precipitation and snowfall is detected.

As described above, the road surface condition estimation device 101 according to the present embodiment estimates the polarized state of natural light without using the polarized image of the sky. Therefore, even when the polarized camera 10 cannot capture the sky and cannot acquire a polarized image of the sky, the polarized state of natural light can be estimated based on weather information or the like. Therefore, the road surface condition and the coefficient of friction can be stably and continuously estimated.

When the polarized camera 10 can acquire a polarized image of the sky, the natural light estimation unit 130 may estimate the polarized state of natural light based on the polarized image of the sky as in the first embodiment. The natural light estimation unit 130 may estimate the polarization state of natural light based on weather information or the like only when the polarized image of the sky cannot be acquired by the polarized camera 10.

(Embodiment 3)
Subsequently, the third embodiment will be described.

In the third embodiment, the vehicle control system including the road surface condition estimation device 1 according to the first embodiment will be described. In the following, the differences from the first embodiment will be mainly described, and the common points will be omitted or simplified. The vehicle control system according to the present embodiment may include the road surface condition estimation device 101 according to the second embodiment.

FIG. 20 is a block diagram showing the configuration of the vehicle control system 200 according to the present embodiment. As shown in FIG. 20, the vehicle control system 200 includes a road surface condition estimation device 1, a vehicle control device 201, and a presentation device 202.

The vehicle control device 201 is an example of a control unit that controls the operation of the moving body 2 based on the friction coefficient. The vehicle control device 201 is, for example, an electronic control unit that controls the engine and steering of the moving body 2. The vehicle control device 201 slows down the moving body 2 when, for example, the friction coefficient becomes lower than a predetermined threshold value. As a result, when the moving body 2 passes through the road surface 3, it is possible to prevent slipping.

The vehicle control device 201 may be a switching device for switching between automatic driving and manual driving. For example, the vehicle control device 201 switches from automatic driving to manual driving when the friction coefficient becomes lower than a predetermined threshold value. As a result, the driver can concentrate on driving and can support the careful passage of the road surface 3.

The presentation device 202 is an example of a presentation unit that presents information on the coefficient of friction. The presenting device 202 is, for example, a display or a speaker mounted on the mobile body 2. For example, the presenting device 202 outputs a notification as an image or a sound for carefully passing the road surface 3 when the friction coefficient is lower than a predetermined threshold value.

As described above, according to the vehicle control system 200 according to the present embodiment, the coefficient of friction is estimated with high accuracy, so that it is possible to present appropriate information to the driver and appropriately control the vehicle.

(Other embodiments)
The road surface condition estimation device, the vehicle control system, and the road surface condition estimation method according to one or more aspects have been described above based on the embodiments, but the present disclosure is not limited to these embodiments. Absent. As long as the gist of the present disclosure is not deviated, various modifications that can be conceived by those skilled in the art are applied to the present embodiment, and a form constructed by combining components in different embodiments is also included in the scope of this disclosure. Is done.

For example, the road surface condition estimation device 1 does not have to include at least one of the tire information acquisition unit 80 and the exposure amount control unit 90. The friction coefficient estimation unit 60 may determine the friction coefficient of the road surface 3 without using the tire information. Further, for example, the friction coefficient estimation unit 60 may determine the friction coefficient of the road surface 3 based on the road surface information indicating the type of the road surface (for example, asphalt or concrete).

Further, for example, in the above embodiment, an example in which the road surface state estimation device 1 is mounted on the moving body 2 has been described, but the present invention is not limited to this. The polarized camera 10 and the communication device are mounted on the moving body 2, and components other than the polarized camera 10 (specifically, the calculation unit 20 and the like) may be provided by an external server device. For example, the communication device transmits the polarized image generated by the polarized camera 10 to the server device by wireless communication.

Further, for example, in the above embodiment, the polarization ratio P / S and the degree of polarization DOP are expressed in the numerical range of 0 to 255, but the present invention is not limited to this. By increasing the numerical ranges of the polarization ratio P / S and the degree of polarization DOP, the road surface condition and the friction coefficient can be estimated with higher accuracy. Further, by reducing the numerical ranges of the polarization ratio P / S and the degree of polarization DOP, the amount of processing required for calculating the road surface condition and the friction coefficient can be reduced.

Further, the polarization state of the road surface is estimated by the Stokes vector of the modification of the first embodiment or the same polarization angle and degree of polarization as the polarization state of the sky, as polarization information having a larger amount of information instead of the polarization ratio. It is clear that it may be.

Further, the communication method between the devices described in the above embodiment is not particularly limited. When wireless communication is performed between devices, the wireless communication method (communication standard) is, for example, short-range wireless communication such as ZigBee (registered trademark), Bluetooth (registered trademark), or wireless LAN (Local Area Network). is there. Alternatively, the wireless communication method (communication standard) may be communication via a wide area communication network such as the Internet. Further, wired communication may be performed between the devices instead of wireless communication. Specifically, the wired communication is a power line communication (PLC) or a communication using a wired LAN.

Further, in the above embodiment, another processing unit may execute the processing executed by the specific processing unit. Further, the order of the plurality of processes may be changed, or the plurality of processes may be executed in parallel. Further, the distribution of the components included in the vehicle control system 200 to a plurality of devices is an example. For example, the components of one device may be included in another device. Further, the road surface condition estimation device 1 or 101 may be subjected to distributed processing using a plurality of devices. The vehicle control system 200 may be realized as a single device.

For example, the processing described in the above embodiment may be realized by centralized processing using a single device (system), or may be realized by distributed processing using a plurality of devices. Good. Further, the number of processors that execute the above program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

Further, in the above embodiment, all or a part of the components such as the control unit may be configured by dedicated hardware, or may be realized by executing a software program suitable for each component. May be good. Each component may be realized by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as an HDD (Hard Disk Drive) or a semiconductor memory. Good.

Further, a component such as a control unit may be composed of one or a plurality of electronic circuits. The one or more electronic circuits may be general-purpose circuits or dedicated circuits, respectively.

The one or more electronic circuits may include, for example, a semiconductor device, an IC (Integrated Circuit), an LSI (Large Scale Integration), or the like. The IC or LSI may be integrated on one chip or may be integrated on a plurality of chips. Here, it is called IC or LSI, but the name changes depending on the degree of integration, and it may be called system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). An FPGA (Field Programmable Gate Array) programmed after the LSI is manufactured can also be used for the same purpose.

Also, general or specific aspects of the present disclosure may be implemented in systems, devices, methods, integrated circuits or computer programs. Alternatively, it may be realized by a computer-readable non-temporary recording medium such as an optical disk, HDD or semiconductor memory in which the computer program is stored. Further, it may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program and a recording medium.

Further, in each of the above embodiments, various changes, replacements, additions, omissions, etc. can be made within the scope of claims or the scope thereof.

The present disclosure can be used as a road surface condition estimation device that can accurately estimate the road surface condition and the friction coefficient of the road surface, and can be used, for example, for automatic driving control, road maintenance or management, and the like. ..

1,101 Road surface condition estimation device 2 Moving object 3 Road surface 4 Blue sky 5 Cloud 6 Front glass 7 Water 8 Snow 9 Ice 10, 10a, 10b Polarized camera 11, 11a, 11b Image sensor 12, 12a, 12b Mosaic polarizing plate 13, 13a , 13b Pixel 14, 15, 16, 17 Sub-pixel 18 Captured image 19a Road surface area 19b Empty area 20 Calculation unit 30, 130 Natural light estimation unit (first estimation unit)
40 Decision unit 50 Classification unit 60 Friction coefficient estimation unit (second estimation unit)
70 Output unit 80 Tire information acquisition unit 90 Exposure control unit 102 Server device 103 Vehicle sensor 200 Vehicle control system 201 Vehicle control device 202 Presentation device

Claims (12)

A polarized camera that generates a polarized image of the road surface by photographing the road surface in the traveling direction of the moving body, and
A calculation unit that calculates road surface parameters representing the polarization state of the road surface by processing the polarized image of the road surface.
The first estimation unit that estimates the polarization state of the natural light that irradiates the road surface,
A determination unit that determines the classification parameters used to classify the road surface condition of the road surface into one of a plurality of categories according to the water content based on the polarization state estimated by the first estimation unit.
A classification unit that classifies the road surface condition of the road surface into any of the plurality of categories based on the road surface parameters and the classification parameters.
A road surface condition estimation device including a second estimation unit that estimates the friction coefficient of the road surface based on the classified categories. The polarized camera further generates a polarized image of the sky by photographing the sky in the traveling direction of the moving body.
The road surface condition estimation device according to claim 1, wherein the first estimation unit estimates the polarized state of the natural light emitted from the sky to the road surface by processing a polarized image of the sky. The road surface condition estimation device according to claim 2, wherein the polarized image of the sky includes polarized images in at least three directions different from each other. The polarized camera
A first polarized camera that is installed so that the optical axis is substantially horizontal and generates a polarized image of the road surface,
The road surface condition estimation device according to claim 2 or 3, which includes a second polarized camera that is installed with an optical axis tilted above the horizontal and generates a polarized image of the sky. The road surface condition estimation according to claim 1, wherein the first estimation unit estimates the polarization state of the natural light based on the position and orientation of the moving body and the weather information of the moving body or the area including the road surface. apparatus. The road surface condition estimation device according to claim 1, wherein the first estimation unit estimates the polarization state of the natural light based on the result of detecting precipitation and snowfall in the moving body or the area including the road surface. The determination unit sets a reference parameter representing the relationship between the unpolarized standard light and the road surface parameter obtained in an environment where the standard light is irradiated on the road surface into a polarized state estimated by the first estimation unit. The road surface condition estimation device according to any one of claims 1 to 6, which determines the classification parameter by correcting it so as to match. The road surface condition estimation device according to any one of claims 1 to 7, wherein the plurality of categories include three categories of dryness, wetness, freezing or snow cover. The road surface condition estimation device according to any one of claims 1 to 8, wherein the second estimation unit estimates the friction coefficient based on the information about the tire of the moving body. Further, any one of claims 1 to 9, further comprising an exposure amount control unit that controls the exposure amount of the polarized camera based on at least one of the polarized image of the road surface and the polarized state estimated by the first estimation unit. The road surface condition estimation device according to item 1. The road surface condition estimation device according to any one of claims 1 to 10,
A vehicle control system including at least one of a presenting unit that presents information on the friction coefficient to the driver of the moving body and a control unit that controls the operation of the moving body based on the friction coefficient. A step of generating a polarized image of the road surface by taking a picture of the road surface in the traveling direction of the moving body with a polarized camera, and
A step of calculating a road surface parameter representing a polarized state of the road surface by processing a polarized image of the road surface, and
The step of estimating the polarization state of the natural light applied to the road surface, and
A step of determining a classification parameter used for classifying the road surface condition of the road surface into one of a plurality of categories according to the moisture condition based on the polarization state of the natural light, and
A step of classifying the road surface condition of the road surface into one of the plurality of categories based on the road surface parameter and the classification parameter.
A road surface condition estimation method including a step of estimating the friction coefficient of the road surface based on the classified categories.

Images