JP2003240869A - Road surface condition determination method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a road surface condition determination method in a visible image type road surface condition grasping device capable of securing reliability as a system by detecting a case possible causing erroneous detection, and by selecting a determination result on the safe side from a plurality of candidates to output it without forcedly narrow determination results down to one in that case. <P>SOLUTION: This method is characterized in that the first nearest candidate and the second nearest candidate are determined from mathematical distances between an inspection image computed by a calculation and a plurality of reference images, the first object is selected when a ratio of both of them is above a preset value, and when the ratio is below the preset value, the candidate corresponding to the direction on the safe side with respect to a vehicle is selected from both the candidates. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a road surface condition determining method, and more specifically, in road traffic, it improves driving safety by providing road surface information to vehicles, or snow removal by providing information such as snow accumulation and freezing in road management. As a technology that contributes to the efficiency of road management work, it uses a visible image obtained with a general-purpose visible camera to automatically and contactlessly detect the road surface condition information required for that purpose, and It relates to a technology that makes it possible to provide that information.

[0002]

2. Description of the Related Art Conventionally, various devices have been developed as a method for discriminating road surface conditions such as wetness, dryness, snow accumulation, and freezing. For example, methods that can be detected without contact include methods that use changes in the reflection characteristics of laser light and methods that use microwaves and infrared rays, but all of them are limited to a relatively narrow measurement range, or depending on the road surface condition. At present, it is not always possible to meet the needs for monitoring the road surface condition over a wide area such as the road surface due to the need for classification. Further, the method using an infrared camera capable of inspecting a relatively wide range has a drawback that the cost is high.

On the other hand, the method using a visible camera is capable of inspecting a relatively wide area of 100 to 150 m in terms of a monitoring area and is relatively inexpensive in terms of cost. In addition, many visible cameras themselves are already installed for purposes other than road surface detection, such as monitoring traffic volume and accidents. Is also expected to be installed relatively densely on each line. Therefore, if this road surface condition grasping function can be added as one of the installation of visible cameras, it can be a very useful road operation support system in terms of cost.

However, there are some difficulties in accurately and reliably detecting the condition of the road surface under various environmental conditions by the image of the visible camera. The commonly used method is to make a judgment by setting a threshold value corresponding to each of the basic elements such as the color, the brightness of the acquired road surface image, the pattern which is the distribution of the acquired road surface image, and the processed information thereof. However, the optimum threshold value fluctuates in response to disturbances, that is, changes in environmental conditions such as weather, such as brightness and the irradiation angle of the sun, which often causes erroneous detection.

Therefore, in order to construct a method for making a strong judgment against changes in these environmental conditions, an image of the road surface condition to be inspected corresponding to the weather conditions that cause disturbance balls, that is, the environmental conditions such as clear weather, cloudy weather, and nighttime. Is stored in advance as a reference image, and the difference is compared with the newly captured inspection image to calculate which reference image the inspection image is closest to. A road surface condition determination method in a road surface condition grasping apparatus has already been proposed (Japanese Patent Application No. 2000-354863).

[0006]

However, it is virtually impossible to consider in advance any changes in environmental conditions, and it is difficult to guarantee a 100% correct answer.
In that case, to ensure the reliability of the system, such as issuing a safe side instruction to the operation of the vehicle when the determination is difficult, so that the false detection does not harm the operation of the vehicle. Going is an important issue.

[0007]

In order to solve the above problems, the present invention detects a case where there is a possibility of erroneous detection, and in that case, a plurality of judgment results are not forcedly narrowed down to one. The reliability of the road surface information providing device is ensured by selecting and outputting the determination result on the safe side from among the candidates. That is, the invention of claim 1 preliminarily images wet and snow images corresponding to a road surface condition to be detected corresponding to weather conditions such as clear weather, cloudy weather, and nighttime, which are disturbance factors, and stores them as reference images. A road surface condition determination method for determining a road surface state in an inspection image by mathematically calculating which reference image the inspection image is closest to by comparing the difference with a newly captured inspection image, From the academic distance between the inspection image calculated by calculation and the plurality of reference images, the closest first candidate object and the second candidate having the next shortest distance are determined, and the ratio of the two is preset. It is characterized in that the first candidate is selected when the value is equal to or more than the value, and the candidate corresponding to the instruction on the safety side for the vehicle is selected from the two candidates when the value is less than or equal to the value.

The invention of claim 2 is characterized in that, in claim 1, the set value of the distance ratio between the first candidate and the second candidate is set to 1.5 to 2.

[0009]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram showing a device configuration of a road surface condition grasping apparatus according to the present invention. An ordinary TV camera 10 for taking a road surface image is attached to the road side so as to look down on the road surface, an A / D converter 18 for digitizing the video output signal, and an image for processing the image data. A processing device 11, a reference image memory unit 12 for storing the reference image, an inspection image memory unit 17 required when creating the reference image, and an arithmetic processing unit 13 for truly determining the road surface condition based on the image processing result, It comprises a display unit 14 and an interface unit 15 for outputting the determination result to the outside, and a control device 16 for controlling the entire device.

The TV camera 10 has a detection range of several tens of meters.
It is installed on the pole or gantry above the roadside so that the road surface of 100 to several tens of meters can be seen. The road surface image is sent to a building in which an apparatus body such as an image processing apparatus is installed, or a signal line of several tens of kilometers in some cases. The TV camera 10 normally uses a color camera because it is necessary to obtain color information as well. The image acquisition cycle is usually 30 per second
It is a screen. The video signal input to the image processing device 11 is A / D-converted, and then image-processed by the method described later under the control of the control device 16, and the result is input to the arithmetic processing device 13. The arithmetic processing unit 13 refers to the reference image stored in the reference image memory unit 12,
This is also processed and calculated by the procedure described later, the road surface state is determined based on the result, and output to the display unit 14 or another device via the interface unit 15.

Next, the details of the image processing executed by this apparatus will be described in detail. First, the principle of acquiring the reference image will be described. The reference image, which is the basis for calculating the reference image feature amount, is basically acquired according to the road surface condition such as dryness, wetness, water film, snow cover, and freezing on the target road surface, and the reference image feature amount is obtained by image processing. To be converted. However, this road surface condition grasping apparatus is intended for outdoor surface images, and therefore the irradiation environment conditions on the road surface greatly change depending on the position of the sun, the state of clouds, surrounding buildings, etc., and the state of the image also changes accordingly. Even under the same road surface condition, the image condition is greatly different especially in the condition where the sun is shining and the condition when it is cloudy, and the determination becomes difficult by the multivariate analysis that relies only on simple inspection images. Therefore, as shown in part in FIG. 2, dry, wet, water film, snow cover, and freezing in fine weather, as well as dry, wet, water film, snow cover, freezing in cloudy weather, and dry and wet under night illumination. Water film,
The road surface conditions corresponding to each of the two or three environmental conditions of snow cover, freezing, fine weather if necessary, and cloudy weather are acquired and used as reference images for calculating the reference image feature amount. Further, in fine weather, the image also changes depending on the position of the sun, so acquisition of the reference image in a rough time band is also required depending on the target road surface.

It is the time to acquire the above-mentioned reference image feature amount,
It is very difficult to obtain all the above cases in a short period of time, and it is necessary to take some time to maintain them. Basically, it will be done manually, but the judgment result is recorded and displayed in the inspection image, and it is not necessary to perform it for all images, and if you look at the data for typical cases, Well, the operation is relatively simple. By successively increasing the reference image feature amount in this way, the road surface condition grasping device will be improved to have sufficient determination reliability after at least one season.

Next, a method of determining the road surface condition from the inspection image will be described based on the flow of FIG. First, an image of a road surface to be processed is extracted as an inspection image from the input image. This follows the processing set in advance based on the road surface image. Next, the image processing apparatus 11 calculates and acquires a difference image between the inspection image and the reference image. The following feature quantities are extracted based on this difference image. (1) The brightness and the brightness I of the dispersed image thereof can be calculated, for example, by the following formulas based on the RGB signals of the color camera. (2) The color difference and the color H of its dispersed image line can be calculated, for example, by the following formula based on the RGB signals of the color camera. Color H = π / 3 (bg) r = (Imax-R) / (Imax-Imin) [0,2π] H = π / 3 (2 + r-b) g = (Imax-G) / (Imax-Imin) H = π / 3 (4 + g−r) b = (Imax−B) / (Imax−Imin) (3) Texture What is a texture?
When wet or water film occurs, the gradient of the reflection intensity (due to the deflection characteristics) between the front and back of the camera field of view (due to the deflection characteristics), the granular brightness distribution in fresh snow, and other patterns that occur on the road surface are numerically differentiated. It refers to the amount converted into the feature amount.

Although various improvements have been made in the calculation of the image feature amount, they are not directly related to the present invention, and thus will not be described in detail, but basically, the feature amount obtained by a commonly used image analysis tool Is available. Such feature amount extraction processing is performed on the difference image to obtain the feature amount. The feature amount thus obtained is considered as the coordinate of the multidimensional space, and the coordinate point with respect to the reference image is determined from the coordinate value thereof. By performing this operation on a plurality of reference images, the same number of coordinate points is obtained in the multidimensional space, and the reference image corresponding to the one with the shortest distance from the origin to each coordinate point is obtained. It represents the desired road surface condition. A conceptual state of this calculation is shown in FIG. The spread of the reference image feature amount in the figure indicates the statistical dispersion of a plurality of images under the same road surface and environmental conditions, and the feature points are the coordinates indicating the average thereof.

As described above, in the method of judging by the distance in the feature amount space between the reference image and the inspection image, the distance between the first candidate having the shortest distance and the second candidate having the next shortest distance is greatly separated. In this case, there is a high possibility that the road surface condition is obtained by the first candidate, and this can be output as the correct answer. However, when the distance ratio between the first candidate and the second candidate is small, the difference in the image lines between the two is basically small, and the distance difference between the two may fall within the range of variation in the above-mentioned environmental change. The possibility that the first candidate is the correct answer is lower than in the above case. That is, mere output of the state in which the distance is the closest causes an erroneous determination. Therefore, it is considered effective to take the distance ratio between the first candidate and the second candidate as an index of the certainty of the determination result.

FIG. 4 shows an example of the correct answer ratios of the first candidate and the second candidate in the relationship between the actual road surface state and the road surface state measured / determined by the device. At least second from the figure
When the distance of the candidate is more than twice the distance of the first candidate (hereinafter referred to as k = distance of second candidate / distance of first candidate, in this case, k ≧ 2), the first candidate is It turns out that the answer is 100% correct. Also, for k ≧ 1.5, the first
The percentage of correct answers is high.

On the other hand, in the region of k <1.5, the number of cases where the second candidate is correct increases. From this, it is appropriate to use the index k as the index of the certainty of the determination result, and specifically, the determination result of the first candidate can be output for k ≧ 1.5 to 2. .

On the other hand, in the case of k <1.5, although the first candidate is still correct, the number of cases in which the second candidate is correct increases, so that an accurate determination becomes difficult. In that case, it is required to provide information to the person to whom the information is provided, on the safe side, that is, even if the information is incorrect, the information can be dealt with on the safer side. For example, if it is determined that the first candidate is “wet” and the second candidate is “freeze”, the output of the second candidate freeze causes the driver to slow the speed even if the actual road surface is wet. It will not lead to an accident.

As described above, although the accuracy rate of the apparatus is slightly lowered, it is possible to provide the information provider with more reliable information by using the distance ratio k between the first candidate and the second candidate. Will be realized. In this case, the set value of k must be decided according to the contents of the target service, and if the more secure side is required, set k to a large value, and if some errors are allowed, set it to a smaller value. It will be possible. Furthermore, it is possible to change this value depending on the type of candidate. For example, "wet"
In the judgment of “freeze” and “dry” and “wet”, the former requires a safer judgment, so k is set to a large value, and the latter has a small influence on erroneous judgment, so the correct answer rate is secured. In that sense, k can be set small.

The above is the main point of the present invention.
The two cases of the candidate and the second candidate have been described, but naturally, there are cases where the third and subsequent candidates fall into this category depending on the conditions. However, even in that case, the output can be determined by the same process as described above.

Although not described above, it is naturally conceivable to use a road surface temperature that is particularly effective for the determination of freezing etc. as a determination element (feature amount), and if it is added as one of the special amount space, It is considered that its reliability will be further improved.

[0022]

According to the present invention, as described above, by using the determination method of the present invention in the visible image type road surface condition grasping apparatus, the erroneous detection may be conventionally performed depending on the change of environmental conditions such as weather. However, the road surface reference images corresponding to the weather conditions, which are disturbance factors, that is, the environmental conditions such as clear weather, cloudy weather, and nighttime, are stored for the visible image expression, which is difficult to trust. Compare the difference with the inspection image,
While calculating the closest to the reference image to determine the road surface condition, quantitatively grasp the degree of closeness to multiple reference images, and the difference between the determination candidates is the range of fluctuation due to external disturbance as a result of experiments in advance. When entering the set value,
By selecting a safer state from among the candidates as the information to be provided, the influence of the environmental fluctuation can be greatly reduced, and a device having sufficient reliability as a road surface condition grasping device can be provided. Therefore, it can be put to practical use by making the most of the features of the visible image type, which were originally excellent in terms of cost, versatility, and multi-functionality (combined with other functions).

[Brief description of drawings]

FIG. 1 is a block diagram showing a device configuration of a road surface condition ascertaining device according to an embodiment of the present invention.

FIG. 2 is a reference road surface image creation flow.

FIG. 3 is a diagram showing image comparison in an image feature amount space.

FIG. 4 is a diagram showing a ratio of correct answers between a first candidate and a second candidate.

[Explanation of symbols]

10 TV camera 11 Image processing device 12 Reference image memory section 13 Processor 14 Display 15 Interface section 16 Control device 17 Inspection image memory section 18 A / D converter

Continued front page    F term (reference) 2G059 AA05 BB08 CC11 EE13 FF01                       KK04 MM01 MM05 MM09 MM10                 5B057 AA16 CA08 CB08 CH01 CH11                       CH16 DA12 DA16 DB09 DC33                 5H180 AA01 CC05 EE12                 5L096 AA06 BA20 CA02 GA08 HA09                       LA04 LA05 LA11

Claims (2)

[Claims] 1. A visible image type road surface condition grasping apparatus for detecting a condition such as wetness or dryness of a road surface in a certain road surface range based on a road surface image signal from a visible camera mounted so as to look down on the road surface, Wet and snow images that correspond to the road conditions you want to detect are captured in advance according to weather conditions such as clear weather, cloudy weather, and nighttime that are disturbance factors, and these are stored as reference images, and newly captured inspection images A method for determining a road surface condition in an inspection image by mathematically calculating which reference image the inspection image is closest to by comparing the difference with the inspection image calculated by the calculation. From the educational distances from the plurality of reference images, the closest first candidate object and the second closest candidate object are determined, and the ratio of both is equal to or greater than a preset value. In the case of No. 1, the first candidate is selected, and in the case of less than that, the candidate corresponding to the instruction on the safe side for the vehicle is selected from both candidates, and the road surface condition determination method. 2. The road surface condition determining method according to claim 1, wherein the set value of the distance ratio between the first candidate and the second candidate is 1.5 to 2.