JP7134855B2 - ROAD CONDITION DETERMINATION DEVICE AND ROAD CONDITION DETERMINATION SYSTEM - Google Patents

Description

The present invention relates to a road surface determination device and a road surface determination system for determining road surface conditions (dry, wet, frozen).

As a device for detecting a road surface state while a vehicle is running, a "road surface state detection device" described in Patent Document 1 is known. The abstract describes ``a road surface condition detection device that can detect the road surface condition (dry, wet, frozen, etc.) even while the vehicle is running, and can also detect the condition of the road surface in front of the vehicle.'' light of wavelengths λ1 and λ2 are projected onto the road surface in front of the vehicle. The output of each light-receiving element is converted into digital data by an A/D converter, and supplied to road surface condition determination means, which is provided with reflected light level data for various road surface conditions. The road surface condition is determined by comparing the received light levels of the wavelengths, and the determination result is output.

In addition, in Patent Document 1, the wavelength of light used for road surface condition determination is set to a wavelength in the infrared light to far infrared light region (for example, claim 2 of the same document), and about 20 meters ahead of the vehicle Detecting the road surface condition in the traveling direction of the vehicle by projecting light onto the road surface and receiving the light reflected therefrom (for example, paragraph 0023 of the same document) is disclosed.

JP-A-8-247940

However, Patent Document 1 does not mention how to handle when there is reflected light from outside the road surface when the road surface state is detected. When the reflected light from them enters the light-receiving means, it is considered that the problem arises that the road surface condition cannot be accurately detected due to the influence of the reflected light from the outside of the road surface.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a road surface condition determination device and a road surface condition determination system that increase the accuracy of road surface condition determination by using only reflected light from a road surface area.

In order to solve the above problems, the road surface condition determination apparatus of the present invention includes a storage device for storing image data of the front of the vehicle captured by an imaging device, and a road surface area on which the vehicle travels based on the image data stored in the storage device. a road surface area recognition unit that identifies the The road surface condition determination unit classifies the reflected light of the modulated infrared light with wavelengths of 1300 nm and 1410 nm simultaneously irradiated onto the road surface area with a frequency filter, thereby generating absorbances for each wavelength in parallel, and calculating the difference Δ between the two absorbances. According to the magnitude, the road surface condition is classified into one of a dry condition, a wet condition, and a frozen condition .

According to the present invention, it is possible to provide a road surface condition determination device and a road surface condition determination system that improve the accuracy of road surface condition determination by using only reflected light from a road surface area.

Problems, configurations, and effects other than those described above will be made clear from the following description of the embodiments.

1 is a schematic diagram of a road surface condition determination system of one embodiment; FIG. 1 is a top view of a vehicle equipped with a road surface condition determination system according to an embodiment while the vehicle is running; FIG. 4 is a conceptual diagram of road surface area recognition processing applied to image data; FIG. It is a graph which shows the absorbance of water and ice. It is a control flowchart of a road surface state determination device. 4 is a control flowchart of a road surface state determination unit; 4 is a control flowchart of a road surface state determination unit;

A road surface condition determination system 10 according to an embodiment of the present invention will be described in detail below with reference to the drawings.

First, the outline of the road surface condition determination system 10 of this embodiment will be described with reference to FIG. As shown here, the road surface condition determination system 10 includes an imaging device 1 such as a stereo camera that images the front of the vehicle with visible light and generates image data 1a, and an infrared light irradiation that irradiates the front of the vehicle with infrared light. 2, an infrared light receiver 3 that receives infrared light reflected in front of the vehicle and generates absorbance 3a that indicates the relationship between the amount of irradiated light and the amount of reflected light, and a road surface condition determination device 4 that will be described later. be.

The road surface condition determination device 4 classifies the road surface condition in front of the vehicle 20 into any of dry, wet, and frozen based on the outputs of the imaging device 1 and the infrared light receiver 3. etc. As a result, the ECU or the like can realize automatic operation control of the vehicle 20 according to the road surface condition.

More specifically, the road surface condition determination device 4 includes an image processing section 4a, a road surface area recognition section 4b, a road surface condition determination section 4c, and an output section 4d. Details of each part will be described later. Note that the road surface condition determination device 4 is actually a calculator (computer) having hardware such as an arithmetic device such as a CPU, a main storage device such as a semiconductor memory, an auxiliary storage device such as a hard disk, and a communication device. be. Then, while referring to the database recorded in the auxiliary storage device, the arithmetic unit executes the program loaded in the main storage device to realize each function described above. Description will be made while omitting it as appropriate.

Next, the road environment on which the vehicle 20 travels and the installation locations of the imaging device 1, the infrared light illuminator 2, and the infrared light receiver 3 will be described with reference to FIG.

In this example, white or orange marking lines 5L and 5R are drawn on the road surface on the left and right sides of the vehicle 20, and further to the left of the left marking line 5L, there is a road boundary determined by a curbstone, a guardrail, or the like. There is a roadside 6.

In addition, the imaging device 1, the infrared light irradiator 2, and the infrared light receiver 3 are installed on the upper inner side of the windshield of the vehicle 20 facing forward. _In this example, an infrared light irradiator 2a that irradiates infrared light with _a wavelength of λ1 and an infrared light irradiator 2b that irradiates infrared light with a wavelength of λ2 are installed so that their irradiation ranges overlap. , and can simultaneously or exclusively irradiate infrared light with wavelength λ ₁ or wavelength λ ₂ toward the traveling direction of vehicle 20 . The infrared light irradiator 2 can irradiate infrared light to the entire irradiation area according to the specifications, but it can be limited to a specific area (for example, the traveling direction of the vehicle 20 sandwiched between the left and right lane markings 5). Infrared light can also be applied.

It is desirable that the infrared light emitted by the infrared light irradiator 2 be modulated light. If modulated light is used, even when infrared light with wavelengths λ ₁ and λ ₂ are irradiated at the same time, the infrared light receiver 3 that receives the reflected light with both wavelengths can convert the received modulated light into This is because the absorbance 3a for each wavelength can be generated in parallel by classifying by the frequency filter, thereby removing the influence of disturbance light.

Hereinafter, details of each part of the road surface condition determination device 4 will be sequentially described on the assumption that the vehicle 20 is traveling on such a road.
<Image processing unit 4a>
The image processing unit 4a performs image processing on the image data 1a from the imaging device 1, which is stored in a storage device (semiconductor memory or the like) (not shown) incorporated in the road surface condition determination device 4. FIG. The image processing executed here is mainly lane marking detection, road edge detection, unevenness detection, and tilt detection. Since these image processes are well-known techniques, a detailed description thereof will be omitted, but the outline thereof is as follows.

The marking line detection is a process of extracting a continuous white portion or an orange portion from the image data 1a and detecting it as the marking line 5 . The roadside detection is a process of extracting curbs, guardrails, etc. from the image data 1a and detecting them as roadside edges (roadsides 6) on which the vehicle 20 is allowed to travel. This roadside detection can also be realized by detecting a step or the like by applying a parallax recognition technology based on stereoscopic vision of a stereo camera.

Further, when the imaging device 1 is a stereo camera, the unevenness detection and inclination detection are processes for detecting the magnitude of unevenness and inclination of the road surface using the parallax of the pair of captured image data 1a. Strictly speaking, since unevenness and slopes always exist on the road surface, a preset threshold is used as a reference, for example, if the unevenness is 5 mm or more, there is unevenness, and if the unevenness is less than 5 mm, there is no unevenness. , the threshold value may be changed as appropriate according to the running speed of the vehicle 20 or the like.
<Road surface area recognition unit 4b>
The road surface area recognizing unit 4b identifies the road surface area on which the vehicle 20 will travel from now on, using the lane markings 5 and the road edges 6 detected by the image processing unit 4a. When the image data 1a illustrated in FIG. 3 is captured, the vanishing point is the intersection of the extension lines of the left and right partition lines 5L and 5R, and the area surrounded by the vanishing point and the lower part of the partition line 5 and the image data 1a is road area.

If the marking lines 5 are worn and thin, or if the road does not have marking lines 5, the above-described method may not be able to determine the road surface area. In that case, the road surface area may be recognized by using the road edge 6 instead of one or both of the left and right marking lines 5 in the above method.
<Road surface condition determination unit 4c>
Information indicating the road surface area detected by the road surface area recognition unit 4b and information indicating the intensity of the infrared light received by the infrared light receiver 3 (absorbance 3a) are input to the road surface state determination unit 4c. The road surface state (dry, wet, frozen) is determined based on the absorbance 3a of the infrared light reflected on the road surface area, and the determination result is transmitted to an external ECU or the like via the output unit 4d. At this time, the infrared light irradiator 2 may irradiate the infrared light over the entire irradiable area, or may irradiate the infrared light limited to the road surface area. Also in this case, the road surface condition determination unit 4c determines the road surface condition using only the reflected light from the road surface area identified by the road surface area recognition unit 4b. As a result, the road surface condition determination accuracy is not degraded by reflected light from the outside of the road surface. In addition, since the amount of calculation can be suppressed, it is possible to reduce the calculation load.

Here, a specific method for determining road surface conditions (dry, wet, frozen) will be outlined. FIG. 4 is a graph showing the absorbance of infrared light for water (broken line) and ice (solid line), with the horizontal axis representing the wavelength of infrared light and the vertical axis representing absorbance.

When infrared light passes through water or ice, energy is absorbed according to the wavelength-dependent absorbance of infrared light, so the amount of light reflected from wet or frozen road surfaces is significantly greater than that from dry road surfaces. to less. For this reason, the amount of infrared light emitted by the infrared light irradiator 2 is compared with the amount of infrared light received by the infrared light receiver 3, and it is determined whether or not an absorption phenomenon occurs in which the latter is significantly reduced. does not occur (that is, if the absorbance 3a, which is the ratio of the amount of received light to the amount of irradiated light, is equal to or greater than a predetermined threshold value), it can be determined that the road surface is dry. On the other hand, when the light absorption phenomenon occurs, it is possible to determine whether the state is wet or frozen based on the absorbance graph of water (broken line) and ice (solid line) shown in FIG. _The relationship between the magnitude of the absorbance 3a and the road surface condition changes depending on the wavelength. If it is about 0.7, it can be determined that it is in a frozen state, and if it is about 0.7, it can be determined that it is in a wet state. Thus, the road surface condition can be classified into dry, wet, and frozen based on the absorbance 3a of single-wavelength infrared light.

However, this determination method may not be able to realize automatic driving. In general, it is necessary to determine the road surface condition at a distance of 20 to 30 m in order to realize automatic driving. growing. This effect appears in the form of an upward or downward parallel shift of the _entire graph illustrated in FIG. There is also a risk of misjudgment as a road surface.

Therefore, in this embodiment, by irradiating infrared light with a plurality of wavelengths, even if the entire graph in FIG. can be improved. Specifically, the road surface area is irradiated with infrared light having a wavelength λ ₁ (for example, 1300 nm) and _a wavelength λ ₂ (for example, 1410 nm). ₂ ), and the magnitude of the difference Δ is the difference between Δ _W (difference Δ observed when in a wet state) and Δ _F (difference Δ observed when in a frozen state) shown in FIG. It is determined whether the road surface condition is wet or frozen depending on which one is closer. Even if the entire graph in FIG. 4 is vertically translated, the difference between _ΔW and _ΔF increases because the magnitude relationship between _ΔW and _ΔF is uniquely determined by the combination of wavelengths of infrared light. By using a combination of infrared light with such wavelengths, it is possible to more accurately determine the road surface condition even if the road surface area is uneven or sloped.
<Control Flowchart of Road Condition Determining Device 4>
A control flow chart for road surface condition determination by the road surface condition determination device 4 described above will be described with reference to FIGS. 5 and 6. FIG.

When the road surface condition determination device 4 starts the road surface condition determination process, the image processing unit 4a stores the image data 1a captured by the imaging device 1 in a storage medium (FIG. 5, S1). Next, the image processing unit 4a executes a process of detecting lane markings 5, road edges 6, unevenness and inclination of the road surface, etc. on the image data 1a stored in the storage medium (S2). Then, the road surface area recognition unit 4b identifies the road surface area with reference to the marking lines 5 and the road edge 6 (S3). The process of S3 corresponds to the description of FIG. 3 above. In parallel with the processing of S1 to S3, the road surface condition determination device 4 acquires from the infrared light receiver 3 the absorbance of two types of reflected light with different wavelengths.

When the road surface area is identified and the absorbance of the reflected light is obtained, the road surface condition determination unit 4c determines whether the reflected light is from the road surface area (S5). If the light is reflected from the outside of the road surface, the processing of FIG. 5 is completed. ).

After the road surface area is determined in S3, the irradiation range of the infrared light illuminator 2 can be limited to the road surface area. good too.
<Control Flowchart of Road Condition Determination Unit 4c>
FIG. 6A is an example of a flowchart for explaining the details of the process of determining the road surface state based on the absorbance of the reflected light from the road surface area by the road surface state determination unit 4c.

First, the road surface condition determination unit 4c acquires the absorbance of the reflected light from the road surface area (S61). Note that FIG. 1 illustrates a configuration in which the absorbance 3a output by the infrared light receiver 3 is input to the road surface state determination unit 4c. The reflected light amount data output from the device 3 may be input to the road surface state determination section 4c, and the road surface state determination section 4c may calculate the absorbance of each reflected light based on both light amount data.

Next, the road surface condition determination unit 4c calculates the difference Δ between the two acquired absorbances (S62). If the road surface is dry, the reflected light will return without the absorption reduction caused by the absorption of energy by water or ice. becomes. Therefore, the difference Δ is compared with the wetness determination value Th _W (S63), and if the difference Δ is smaller, it is determined that the road surface is dry (S64). If the difference Δ is greater than or equal to the wetness determination value _ThW , the difference Δ is compared with the freeze determination value _ThF (S65), and if the difference Δ is smaller, the road surface is determined to be wet (S66). On the other hand, when the difference Δ is equal to or greater than the frozen determination value _ThF , it is determined that the road surface is frozen (S67). When the road surface condition is determined to be dry, wet, or frozen, the road surface condition determination unit 4c transmits the determination result to the outside via the output unit 4d (S58).

The above is a flow chart for the case of combining infrared light with wavelengths λ ₁ and λ ₂ where the relationship between _ΔF and _ΔW shown in FIG. 4 is _ΔF > _ΔW . Therefore, when combining infrared light with wavelengths λ ₁ and λ ₂ where the relationship between Δ _F and Δ _W is Δ _F <Δ _W , S63 and S65 in FIG. 6A are exchanged, and S66 and S67 are exchanged. Also, the road surface condition is determined according to the flowchart of FIG. 6B. Needless to say, when the infrared _rays of wavelengths λ ₁ and λ ₂ are combined so that the relationship between _ΔF and _ΔW is _ΔF≈ΔW , the road surface condition cannot be determined. A combination of λ ₁ and λ ₂ should be avoided.

5, FIG. 6A, and FIG. 6B may be configured to be executed regardless of the unevenness or inclination of the road surface. In some cases, the road surface condition cannot be accurately determined. Therefore, in S2 of FIG. 5, when the image processing unit 4a detects unevenness or inclination of a predetermined value or more, the processing of FIG. 5 may be stopped, and only when the unevenness or inclination is small, the processing of FIG. 5 may be continued. . In this way, by judging the road surface condition only when the absorbance 3a is reliable, it is possible to improve the judgment accuracy.

In the above, the difference Δ of the absorbance 3a is calculated by using infrared light of a plurality of wavelengths, and the road surface condition is determined based on this difference Δ. , when the relationship between the amount of parallel movement in the vertical direction of the graph in FIG. , parallel displacement in the graph of FIG. 4), and based on the calculation result, the road surface condition may be determined using only the absorbance for one wavelength. As a result, it is possible to realize accurate road surface condition determination even with a simple configuration in which only one wavelength of infrared light is emitted.

In general, when a frozen road surface melts, the area through which the tires pass melts quickly due to frictional heat. . In this way, the area through which the tires have passed is defined as the vehicle passage area, and a method for detecting the vehicle passage area will be described below. Based on the road surface conditions at multiple locations measured by the road surface freezing method, the turning point is the point at which the results of the road surface freezing determination at the measurement points aligned parallel to the vehicle switch from ice to water or dry conditions. When it is confirmed that the distance between the two pairs of turning points is a certain distance (for example, the diameter of the tire), that point is taken as the vehicle passing point. The above processing is performed for all measurement points, and two pairs of straight lines connecting the two confirmed pairs of vehicle passing points and an area (rectangle) connecting the ends of the two pairs of straight lines in parallel are defined as the vehicle passing area. . In S3 and S5 of FIG. 5, the road surface area is specified and the road surface condition is determined using the reflected light from this road surface area. It is good to do.

According to the road surface condition determination device 4 or the road surface condition determination system 10 of the present embodiment described above, by using only the reflected light from the road surface area without using the reflected light from the outside of the road surface, It is possible to avoid a situation in which the accuracy of determination of the road surface state is deteriorated due to the influence of diffused reflection from the streets and steps, etc., and it is possible to further improve the determination accuracy.

In addition, by limiting the area referred to when determining the road surface condition to the road surface area, it is possible to suppress the amount of calculation required for road surface condition determination and reduce the calculation load of the vehicle system.

1 Imaging device 1a Image data 2, 2a, 2b Infrared light illuminator 3 Infrared light receiver 4 Road surface condition determination device 4a Image processing unit 4b Road surface area recognition unit 4c Road surface condition determination unit 4d Output unit 5, 5L, 5R Section Line 6 Road edge 10 Road condition determination system 20 Vehicle

Claims (3)

a storage device for storing image data of the front of the vehicle captured by the imaging device;
a road surface area recognition unit that identifies a road surface area on which the vehicle travels based on the image data stored in the storage device;
a road surface condition determination unit that determines a road surface condition based on the absorbance of reflected infrared light irradiated to the road surface area identified by the road surface area recognition unit;
with
The road surface condition determination unit classifies the reflected light of the modulated infrared light with wavelengths of 1300 nm and 1410 nm simultaneously applied to the road surface area with a frequency filter, thereby generating absorbances for each wavelength in parallel, and calculating the difference between the two absorbances. A road surface state determination device , wherein the road surface state is classified into one of a dry state, a wet state, and a frozen state according to the magnitude of Δ . In the road surface condition determination device according to claim 1,
Further, an image processing unit is provided for detecting left and right lane markings or roadsides of the vehicle from the image data,
The road surface area recognition unit
With the vanishing point at the intersection of the left and right lane markings or the road edge of the vehicle,
A road surface condition determination apparatus, wherein the road surface area is an area surrounded by each of the left and right lane markings or the road edge of the vehicle, a lower area of the image data, and the vanishing point. In the road surface condition determination device according to claim 2,
The image processing unit determines the presence or absence of unevenness on the road surface from the image data,
The road surface condition determination device, wherein the road surface condition determination unit determines the road surface condition when it is determined that the road surface is not uneven.

Images