JP2020094896A - Road surface state determination device and road surface state determination system - Google Patents

Abstract

To provide a road surface state determination device and a road surface state determination system which increase the accuracy of determining a road surface state more than before by using only reflection light from a road surface region.SOLUTION: A road surface state determination device includes: a storage unit for storing an image data on the front of a vehicle, which is imaged by an imaging device; a road surface region recognition section for specifying a road surface region in which the vehicle travels based on the image data stored in the storage unit; and a road surface state determination section for determining a road surface state based on absorbance of reflection light of infrared light with which the road surface region specified by the road surface region recognition section is irradiated.SELECTED DRAWING: Figure 6A

Description

The present invention relates to a road surface determination device and a road surface determination system that determine a road surface state (dry, wet, freezing).

As a device for detecting a road surface condition while a vehicle is traveling, a "road surface condition detecting device" described in Patent Document 1 is known. In this abstract, "A road surface condition detector that can detect road surface conditions (dry, wet, freezing, etc.) even while the vehicle is running, and can also detect the road surface condition in front of the vehicle" Of the wavelengths λ1 and λ2 are projected toward the road surface in front of the vehicle. The light receiving means is a light receiving element via a spectroscope corresponding to the projected wavelength and the level of the reflected light of each wavelength. The output of each light receiving element is converted into digital data by an A/D converter and supplied to the road surface condition determining means, which is provided with reflected light level data in various road surface conditions. The road surface condition is determined by comparing the light receiving levels of the wavelengths, and the determination result is output."

Further, in Patent Document 1, the wavelength of light used for road surface condition determination is set to a wavelength in the infrared light to far infrared light region (for example, claim 2 of the same document), or about 20 meters in front of the vehicle. It is disclosed that a road surface state in the traveling direction of a vehicle is detected by projecting light on a road surface and receiving reflected light therefrom (for example, paragraph 0023 of the same document).

JP-A-8-247940

However, Patent Document 1 does not mention how to handle reflected light from outside the road surface when detecting the road surface state. For example, infrared light may be an obstacle such as a streetlight or a step outside the road surface. It is considered that there is a problem that the road surface condition cannot be accurately detected due to the influence of the reflected light from the outside of the road surface when the light is also irradiated and the reflected light from them enters the light receiving means.

Therefore, it is an object of the present invention to provide a road surface state determination device and a road surface state determination system in which the accuracy of determination of the road surface state is further improved by using only the reflected light from the road surface area.

In order to solve the above problems, a road surface state determination device of the present invention is a storage device that stores image data of the front of a vehicle captured by an image capturing device, and a road surface region in which a vehicle travels based on the image data stored in the storage device. And a road surface state determination unit that determines the road surface state based on the absorbance of the reflected light of the infrared light applied to the road surface region identified by the road surface region recognition unit. did.

Further, the road surface state determination system of the present invention receives an image pickup device for picking up image data in front of the vehicle, an infrared light irradiator for irradiating infrared light in front of the vehicle, and infrared light reflected in front of the vehicle. Infrared light receiver, based on the image data captured by the imaging device, the absorbance indicating the relationship between the reflected light amount received by the infrared light receiver and the irradiation light amount irradiated by the infrared light irradiator, A road surface state determination system comprising: a road surface state determination device for determining a road surface state, wherein the road surface state determination device is a storage device for storing image data captured by the imaging device; Based on the image data, a road surface area recognition unit that specifies a road surface area on which the vehicle travels, and a road surface condition determination unit that determines the road surface condition based on the absorbance of the reflected light from the road surface area specified by the road surface area recognition unit. When,
To have.

According to the present invention, it is possible to provide a road surface state determination device and a road surface state determination system in which the accuracy of determination of the road surface state is further improved by using only the reflected light from the road surface area.

Problems, configurations, and effects other than those described above will be clarified from the following description of the embodiments.

It is a schematic diagram of a road surface state judging system of one example. FIG. 1 is a top view of a vehicle equipped with a road surface state determination system according to an embodiment, which is running. It is a conceptual diagram of road surface area recognition processing performed on image data. It is a graph which shows the light absorbency of water and ice. It is a control flowchart of a road surface state determination device. It is a control flowchart of a road surface state determination part. It is a control flowchart of a road surface state determination part.

Hereinafter, a road surface state determination system 10 according to an embodiment of the present invention will be described in detail with reference to the drawings.

First, the outline of a road surface state determination system 10 of the present embodiment will be described with reference to FIG. As shown here, the road surface state determination system 10 includes an imaging device 1 such as a stereo camera that images the front of the vehicle with visible light and generates image data 1a, and infrared light irradiation that irradiates the front of the vehicle with infrared light. A device including a device 2, an infrared light receiver 3 that receives infrared light reflected from the front of the vehicle, and generates an absorbance 3a indicating the relationship between the amount of irradiation light and the amount of reflected light, and a road surface condition determination device 4 described later. is there.

The road surface condition determination device 4 classifies the road surface condition in front of the vehicle 20 into one of dry, wet, and frozen, based on the outputs of the image pickup device 1 and the infrared light receiver 3, and an ECU (Electronic Control Unit) not shown. Etc. are output. As a result, the ECU or the like can realize automatic driving control of the vehicle 20 according to the road surface condition.

More specifically, the road surface state determination device 4 includes an image processing unit 4a, a road surface area recognition unit 4b, a road surface state determination unit 4c, and an output unit 4d. Details of each unit will be described later. Note that the road surface condition determination device 4 is actually a computer (computer) including hardware such as an arithmetic unit such as a CPU, a main memory such as a semiconductor memory, an auxiliary memory such as a hard disk, and a communication device. is there. Then, while referring to the database recorded in the auxiliary storage device, the arithmetic device executes the program loaded in the main storage device to realize each of the functions described above. The description will be made while appropriately omitting it.

Next, the road environment in which the vehicle 20 travels and the installation locations of the imaging device 1, the infrared light irradiator 2, and the infrared light receiver 3 will be described with reference to FIG.

In this example, white or orange lane markings 5L and 5R are drawn on the left and right road surfaces of the vehicle 20, and on the left side of the left lane marking 5L, there is a boundary of the road surface determined by curbs, guardrails, or the like. There is a roadside 6.

Further, the image pickup device 1, the infrared light irradiator 2, and the infrared light receiver 3 are installed with the front facing toward the upper inside of the windshield of the vehicle 20. In this example, an infrared light irradiator 2a that irradiates infrared light having a wavelength λ _{1 and} an infrared light irradiator 2b that irradiates infrared light having a wavelength λ ₂ are installed so that their irradiation ranges overlap. Therefore, the infrared light having the wavelength λ ₁ or the wavelength λ ₂ can be simultaneously or exclusively emitted toward the traveling direction of the vehicle 20. The infrared light irradiator 2 is capable of irradiating infrared light to the entire irradiation area according to the specifications, but narrows it down to a specific area (for example, the traveling direction of the vehicle 20 sandwiched between the left and right marking lines 5). Irradiation with infrared light is also possible.

It is desirable that the infrared light emitted by the infrared light irradiator 2 be modulated light. If the modulated light is used, even if the infrared light of the wavelength λ ₁ and the wavelength λ ₂ are simultaneously irradiated, the infrared light receiver 3 which receives the reflected light of both wavelengths changes the frequency of the received modulated light. This is because by classifying with a filter, the absorbances 3a for each wavelength can be generated in parallel, and thereby the influence of ambient light can be removed.

Hereinafter, assuming the vehicle 20 traveling on such a road, the details of each part of the road surface state determination device 4 will be sequentially described.
<Image processing unit 4a>
The image processing unit 4a performs image processing on the image data 1a from the imaging device 1 stored in a storage device (semiconductor memory or the like) (not shown) built in the road surface state determination device 4. The image processing executed here is mainly lane marking detection, road edge detection, unevenness detection, and inclination detection. Since these image processes are well-known techniques, detailed description thereof will be omitted, but the outline is as follows.

The marking line detection is a process of extracting a continuous white portion or orange portion from the image data 1a and detecting this as the marking line 5. The roadside detection is a process of extracting a curbstone, a guardrail, or the like from the image data 1a and detecting the curbstone or the guardrail as an end portion (roadside 6) of the road surface on which the vehicle 20 is allowed to travel. This road edge detection can also be realized by detecting a step or the like by applying a parallax recognition technique by stereoscopic vision of a stereo camera.

Further, the unevenness detection and the inclination detection are processing for detecting the size of the unevenness and the inclination of the road surface by using the parallax of the pair of image data 1a taken when the imaging device 1 is a stereo camera. Strictly speaking, since there are always irregularities and slopes on the road surface, for example, if the irregularities are 5 mm or more, the irregularities are present, and if the irregularities are less than 5 mm, there are no irregularities. Although the presence or absence of unevenness or inclination is classified as, the threshold value may be appropriately changed according to the traveling speed of the vehicle 20 and the like.
<Road surface area recognition unit 4b>
The road surface area recognition unit 4b identifies the road surface area in which the vehicle 20 is about to travel, using the lane markings 5 and road edges 6 detected by the image processing unit 4a. When the image data 1a illustrated in FIG. 3 is imaged, the intersection of the extension lines of the left and right lane markings 5L and 5R is set as the vanishing point, and the lane marking 5 and the lower portion of the image data 1a and the area surrounded by the vanishing points are defined. The road surface area.

When the lane marking 5 is rubbed and becomes thin, or on a road without the lane marking 5, the road surface area may not be discriminated by the above method. In that case, the road surface area may be recognized by using the road edge 6 instead of one or both of the left and right lane markings 5 in the above method.
<Road surface condition determination unit 4c>
Information indicating the road surface area detected by the road surface area recognition unit 4b and information indicating the magnitude of infrared light received by the infrared light receiver 3 (absorbance 3a) are input to the road surface state determination unit 4c. Then, the road surface state (dry, wet, freezing) is determined based on the absorbance 3a of the infrared light reflected by the road surface area, and the determination result is transmitted to an external ECU or the like via the output unit 4d. At this time, the infrared light irradiator 2 may irradiate the infrared light to the entire irradiable area, or may irradiate the infrared light only to the road surface area. Also in this case, the road surface state determination unit 4c determines the road surface state using only the reflected light from the road surface region identified by the road surface region recognition unit 4b. As a result, since the accuracy of determining the road surface state does not deteriorate due to the reflected light from the outside of the road surface, it is possible to improve the accuracy of determining the road surface state as compared with the case of determining the road surface state using the reflected light in the entire area. Moreover, since the amount of calculation can be suppressed, the calculation load can be reduced.

Here, a specific method for determining the road surface condition (dry, wet, freezing) will be outlined. FIG. 4 is a graph showing the absorbance of infrared light for water (broken line) and ice (solid line), respectively, with the horizontal axis representing the wavelength of infrared light and the vertical axis representing the absorbance.

When infrared light passes through water or ice, the energy is absorbed according to the wavelength-dependent absorbance of infrared light, so the amount of reflected light from a wet road surface or a frozen road surface is much larger than the amount of reflected light from a dry road surface. Less. Therefore, the amount of infrared light emitted by the infrared light irradiator 2 and the amount of infrared light received by the infrared light receiver 3 are compared to determine whether or not the latter has a significantly reduced light absorption phenomenon, and the light absorption phenomenon If the light does not occur (that is, if the absorbance 3a, which is the ratio of the amount of received light to the amount of irradiated light, is greater than or equal to a predetermined threshold value), it can be determined that the road surface is dry. On the other hand, when the light absorption phenomenon occurs, it can be determined whether it is in the wet state or the frozen state based on the graph of the absorbance of water (broken line) and ice (solid line) shown in FIG. The relationship between the magnitude of the absorbance 3a and the road surface state changes depending on the wavelength. For example, when infrared light having a wavelength of about 1300 nm is used (see λ _{1 in} FIG. 4), the absorbance 3a in the road surface region is 1 If the degree of absorption is 3 degrees, it can be determined to be a frozen state, and if the absorbance 3a is about 0.7, it can be determined to be a wet state. In this way, the road surface condition can be classified as dry, wet, or frozen based on the absorbance 3a of infrared light having a single wavelength.

However, this determination method may not be able to realize automatic driving. Generally, in order to realize automatic driving, it is necessary to determine the road surface condition at a distance of 20 to 30 m. However, when infrared light is irradiated to such a distance, the influence of unevenness or inclination of the road surface on the absorbance 3a may occur. growing. This effect appears as the entire graph illustrated in FIG. 4 moves upward or downward in parallel, so that, for example, the absorbance 3a at the wavelength λ ₁ is about 0.7, even though the road surface is actually frozen, and the wetness is obtained. There is a fear that it will be misjudged as a road surface.

Therefore, in the present embodiment, by irradiating infrared light having a plurality of wavelengths, even if the entire graph of FIG. 4 moves up and down due to the influence of unevenness and inclination of the road surface, the accuracy of the road surface state determination is high. I was able to improve. Specifically, the road surface region is irradiated with infrared light of wavelength λ ₁ (for example, 1300 nm) and wavelength λ ₂ (for example, 1410 nm), and a difference Δ (=absorbance of λ ₁ −λ ₁ of absorbance 3a at each wavelength. ₂ ) and the magnitude of the difference Δ is the difference between Δ _W (the difference Δ observed in the wet state) and Δ _F (the difference Δ observed in the frozen state) shown in FIG. Whether the road surface condition is wet or frozen is determined depending on which one is closer. Even if the entire graph of FIG. 4 moves up and down in parallel, since the magnitude relationship between Δ _W and Δ _F is uniquely determined by the combination of the wavelengths of infrared light, the difference between Δ _W and Δ _F becomes large. By using infrared light having such a wavelength in combination, the road surface condition can be more accurately determined even when the road surface region has irregularities or slopes.
<Control Flowchart of Road Surface Condition Determination Device 4>
A control flowchart of the road surface state determination by the road surface state determination device 4 described above will be described with reference to FIGS. 5 and 6.

When the road surface state determination device 4 starts the road surface state determination process, the image processing unit 4a stores the image data 1a captured by the image capturing device 1 in a storage medium (FIG. 5, S1). Next, the image processing unit 4a performs a detection process on the image data 1a stored in the storage medium such as the lane marking 5, the road edge 6, and the unevenness and inclination of the road surface (S2). Then, the road surface area recognizing unit 4b identifies the road surface area based on the lane marking 5 and the road edge 6 (S3). The process of S3 corresponds to the above description of FIG. Further, the road surface state determination device 4 acquires the absorbances of the two types of reflected light having different wavelengths from the infrared light receiver 3 in parallel with the processing of S1 to S3.

When the road surface area is specified and the absorbance of the reflected light is acquired, the road surface state determination unit 4c determines whether the reflected light is the reflected light from the road surface area (S5). Then, if it is the reflected light from the outside of the road surface, the processing of FIG. 5 is completed, and if it is the reflected light from the road surface area, the road surface state determination unit 4c causes the road surface state determination processing (S6 shown in FIG. 6A or 6B). ) Is executed.

After the road surface area is determined in S3, the irradiation range of the infrared light irradiator 2 can be limited to the road surface area. In that case, since reflected light from outside the road surface is not generated, S5 can be omitted. Good.
<Control Flowchart of Road Surface Condition Determination Unit 4c>
FIG. 6A is an example of a flowchart illustrating the details of the process in which the road surface state determination unit 4c determines the road surface state based on the absorbance of the reflected light from the road surface region.

First, the road surface state determination unit 4c acquires the absorbance of the reflected light from the road surface area (S61). Note that, in FIG. 1, the configuration in which the absorbance 3a output by the infrared light receiver 3 is input to the road surface state determination unit 4c is illustrated, but the irradiation light amount data output by the infrared light irradiator 2 and the infrared light reception The reflected light amount data output from the device 3 may be input to the road surface state determination unit 4c, and the road surface state determination unit 4c may calculate the absorbance of each reflected light based on the both light amount data.

Next, the road surface state determination unit 4c calculates the difference Δ between the two acquired absorbances (S62). When the road surface is dry, the absorbed light is absorbed by water and ice, and the reflected light returns without causing a decrease in absorption, so the absorbance is almost the same regardless of the wavelength, and the difference Δ is a very small value. Becomes Therefore, the difference Δ and the wetting determination value Th _W are compared (S63), and if the difference Δ is smaller, it is determined to be a dry road surface (S64). If the difference Δ is greater than or equal to the wetting determination value Th _W , the difference Δ and the freezing determination value Th _F are compared (S65), and if the difference Δ is smaller, it is determined to be a wet road surface (S66). On the other hand, when the difference Δ is equal to or greater than the freezing determination value Th _F , it is determined to be a frozen road surface (S67). When it is determined that the road surface condition is one of dry, wet, and frozen, the road surface condition determination unit 4c transmits the determination result to the outside via the output unit 4d (S58).

It should be noted that the above is the flowchart in the case where the infrared lights of the wavelengths λ ₁ and λ ₂ in which the relationship between Δ _F and Δ _W shown in FIG. 4 is Δ _F >Δ _W are combined. Therefore, when the infrared light of wavelengths λ ₁ and λ ₂ where the relationship between Δ _F and Δ _W is Δ _F <Δ _W is combined, S63 and S65 in FIG. 6A are replaced, and S66 and S67 are replaced. Further, the road surface condition is determined according to the flowchart of FIG. 6B. Needless to say, when the infrared light of wavelengths λ ₁ and λ ₂ where the relationship between Δ _F and Δ _W is Δ _F ≈Δ _W is combined, the road surface condition cannot be determined. It is necessary to avoid the combination of λ ₁ and λ ₂ .

The processes of FIGS. 5, 6A, and 6B may be executed regardless of the unevenness or inclination of the road surface. However, when the unevenness or inclination of the road surface is large, the acquired absorbance 3a is not reliable, It is possible that the road surface condition cannot be accurately determined. Therefore, in S2 of FIG. 5, when the image processing unit 4a detects an unevenness or an inclination of a predetermined value or more, the processing of FIG. 5 may be stopped and the processing of FIG. 5 may be continued only when the unevenness or the inclination is small. .. In this way, the determination accuracy can be improved by determining the road surface state only when the absorbance 3a is reliable.

Further, in the above description, the configuration is used in which the difference Δ of the absorbance 3a is calculated by using infrared light of a plurality of wavelengths together and the road surface state is determined based on the difference Δ. When the relationship of the amount of parallel movement in the vertical direction of the graph of FIG. 4 is known, the unevenness or slope of the road surface is detected using the image data 1a from the image pickup device 1 and the influence of the unevenness or slope on the absorbance (that is The amount of parallel movement in the vertical direction of the graph of FIG. 4) is calculated, and the road surface state may be determined using only the absorbance for one wavelength based on the calculation result. As a result, it is possible to realize accurate road surface condition determination with a simple configuration in which the wavelength of infrared light to be emitted is one.

In general, when a frozen road surface melts, the area through which the tire passes is rapidly melted due to frictional heat, so running on a melted road surface (on a wet road surface) enables more stable running. .. The region through which the tire has passed is defined as the vehicle passage region, and a method for detecting the vehicle passage region will be described below. Based on the road surface condition at multiple points measured by the road surface freezing method, the turning point is the point at which the road surface freezing determination result at the measurement points arranged parallel to the vehicle switches from ice to water or dry condition. When it is confirmed that the distance between the two pairs of turning points is a certain distance (for example, the diameter of the tire), the point is set as the vehicle passing point. The above processing is performed for all the measurement points, and the area (quadrangle) in which the two pairs of straight lines connecting the confirmed two pairs of vehicle passage points and the ends of the two pairs of straight lines are connected in parallel is defined as the vehicle passage area. .. In S3 and S5 of FIG. 5, the road surface area is specified and the road surface state is determined using the reflected light from this road surface area. However, instead of the road surface area, the road surface state is determined using the reflected light from the vehicle passage area. It may be done.

According to the road surface state determination device 4 or the road surface state determination system 10 of the present embodiment described above, the reflected light from the outside of the road surface is not used, and only the reflected light from the road surface area is used to It is possible to avoid a situation in which the determination accuracy of the road surface state deteriorates due to the influence of irregular reflection from the streets or steps, and the determination accuracy can be further improved.

Further, by limiting the area referred to when determining the road surface condition to the road surface region, it is possible to suppress the calculation amount required for the road surface condition determination and reduce the calculation load of the vehicle system.

1 Imaging device 1a Image data 2, 2a, 2b Infrared light irradiator 3 Infrared light receiver 4 Road surface condition determination device 4a Image processing unit 4b Road surface region recognition unit 4c Road surface condition determination unit 4d Output unit 5, 5L, 5R Section Line 6 Road edge 10 Road surface condition determination system 20 Vehicle

Claims (6)

A storage device that stores image data of the front of the vehicle captured by the imaging device,
A road surface area recognition unit that specifies a road surface area on which the vehicle travels based on the image data stored in the storage device;
Based on the absorbance of the reflected light of the infrared light applied to the road surface area specified by the road surface area recognition unit, a road surface state determination unit that determines the road surface state,
A road surface condition determination device comprising: The road surface condition determination device according to claim 1,
Furthermore, an image processing unit for detecting left and right lane markings or road edges of the vehicle from the image data is provided,
The road surface area recognition unit,
As the vanishing point, the intersection of the left and right lane markings or road edges of the vehicle,
A road surface state determination device, wherein each of the left and right lane markings or road edges of the vehicle, a lower area of the image data, and an area surrounded by the vanishing points are defined as the road surface area. In the road surface state determination device according to claim 1 or 2,
The road surface state determination unit, the road surface state, a dry state, a wet state, a frozen state according to the magnitude of the difference Δ of the absorbance of the reflected light of two types of infrared light with different wavelengths applied to the road surface region. A road surface state determination device characterized by being classified into any of the following. The road surface condition determination device according to claim 2,
The image processing unit determines the presence or absence of unevenness on the road surface from the image data,
The road surface state determination device, wherein the road surface state determination unit determines the road surface state when it is determined that there is no unevenness on the road surface. An image pickup device for picking up image data in front of the vehicle;
An infrared light irradiator that irradiates infrared light in front of the vehicle,
An infrared light receiver that receives infrared light reflected in front of the vehicle,
A road surface that determines a road surface state based on image data captured by the image capturing device and absorbance indicating a relationship between the reflected light amount received by the infrared light receiver and the irradiation light amount irradiated by the infrared light irradiator. A state determination device,
A road surface condition determination system comprising:
The road surface condition determination device,
A storage device that stores image data captured by the imaging device;
A road surface area recognizing unit that specifies a road surface area on which the vehicle travels based on image data stored in the storage device;
Based on the absorbance of the reflected light from the road surface region specified by the road surface region recognition unit, a road surface state determination unit that determines the road surface state,
A road surface condition determination system having: The road surface condition determination system according to claim 5,
The infrared light irradiator irradiates the road surface area recognized by the road surface area recognition unit with infrared light,
The road surface state determination system characterized in that the road surface state determination device determines the road surface state using only reflected light from the road surface area without using reflected light from outside the road surface area.

Images