JP2018147368A - Road surface state estimating apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate a state of a road surface in front of a vehicle on the basis of a vehicle front image obtained by capturing a front of the vehicle.SOLUTION: A road surface estimating apparatus 1 has: a reference image extracting unit 13 for extracting, as a reference image, a region of a lamp part of a preceding vehicle in a vehicle front image obtained by capturing a front of the vehicle; a search region setting unit 14 for setting, as a search region, a region in the vehicle front image which may have reflection of the lamp part by reflection of the lamp part to the road surface due to a wet road surface in front of the vehicle; and an estimating unit 15 for estimating a degree of wetness of the road surface by comparing color information of pixels in the search region with color information of pixels in the reference image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a road surface state estimating device.

  Patent Document 1 describes an environment estimation device that estimates a surrounding environment of a vehicle or the like based on a camera image captured by a camera fixedly installed on the vehicle or the like, and a vehicle control device that uses this device.

Japanese Patent No. 5501477

  The state of the road surface affects the vehicle traveling on the road surface. The present invention has been made in view of such a situation, and an object of the present invention is to estimate the state of a road surface in front of a vehicle based on a vehicle front image in which the front of the vehicle is captured.

  In order to achieve the above object, a road surface state estimation device according to the present invention includes a reference image extraction unit that extracts a region of a ramp portion of a preceding vehicle as a reference image in a vehicle front image obtained by copying the vehicle front, A search area setting unit that sets, as a search area, an area in the front image of the vehicle in which reflection of the lamp part may occur due to reflection of the lamp part on the road surface due to wetness of the road surface ahead. And an estimation unit that estimates the degree of wetness of the road surface by comparing the color information of the pixels in the search area with the color information of the pixels in the reference image.

  According to the present invention, the state of the road surface in front of the vehicle can be estimated based on the vehicle front image in which the front of the vehicle is captured.

It is explanatory drawing of the system provided with the road surface state estimation apparatus. It is a flowchart of the process performed by a system. It is explanatory drawing which shows a vehicle front image. It is explanatory drawing which shows a reference | standard image. It is explanatory drawing which shows the example of a setting of a 1st search area | region. It is explanatory drawing which shows the example of a setting of a 2nd search area | region. It is a graph which shows the relationship between the degree of wetting and the degree of reflection.

  Embodiments of the present invention will be described below. However, the present invention is not limited to the following embodiments.

  FIG. 1 shows a system S including a road surface state estimation device 1. The system S is mounted on a vehicle, and the vehicle on which the system S is mounted is also called a host vehicle. The system S further includes a vehicle forward imaging device 2, a vehicle information acquisition device 3, an alarm device 4 that informs a driver of the host vehicle of a road surface state, and a communication device 5 that informs a surrounding vehicle or a road manager of the road surface state. These devices are connected to the road surface state estimating device 1.

  When the road surface in front of the host vehicle is wet due to rain or the like, the lamp portion of the preceding vehicle that travels in front of the host vehicle is reflected on the road surface, and the lamp portion is reflected in the vehicle front image that shows the front of the host vehicle. Arise. It is considered that the greater the wetness of the road surface, the greater the degree of reflection. The road surface state estimation device 1 estimates the degree of wetness of the road surface ahead of the host vehicle from the degree of reflection of the ramp portion of the preceding vehicle in the vehicle front image. The road surface state estimation device 1 includes an image acquisition unit 11, a preceding vehicle detection unit 12, a reference image extraction unit 13, a search area setting unit 14, an estimation unit 15, and a signal output unit 16.

  The road surface state estimation device 1 has, as its hardware configuration, a memory that stores a program and data operable to execute the functions of each unit, a processor that performs arithmetic processing, and an interface between other devices in the system S And.

  The vehicle front imaging device 2 captures the front of the host vehicle as a vehicle front image. This vehicle forward image is sent from the vehicle forward imaging device 2 to the road surface state estimating device 1. In recent years, an omnidirectional camera that has been widely used for the purpose of reducing blind spots around the vehicle can be used as the vehicle front imaging device 2. In addition, functions for detecting obstacles and white lines in front of vehicles and preventing rear-end collisions and lane departures are becoming widespread, and stereo cameras and monocular cameras used for these functions are used as the vehicle front imaging device 2. Also good.

  FIG. 2 shows a flow of processing performed by the system S. First, in step S <b> 101, the vehicle information acquisition device 3 acquires vehicle information of the traveling host vehicle from the host vehicle, and inputs the acquired information to the road surface state estimation device 1. Examples of vehicle information include vehicle speed, steering angle, brake switch state, accelerator opening, shift position, wiper operating status, and the like.

In step S102, the image acquisition unit 11 in the road surface state estimation device 1 sends a fixed size (for example, VGA size 640 × vertical 480 pixels) sent from the vehicle front imaging device 2 at a fixed interval (for example, video rate 30 Hz). The vehicle front image of is acquired. An example of the vehicle front image is shown in FIG. The vehicle front image 6 shows a preceding vehicle 61 traveling in front of the host vehicle and a road surface 62 on which the host vehicle and the preceding vehicle travel. Furthermore, before run the vehicle 61 and the front run left brake lamp portion 63 _L and the right brake lamp portion 63 _R of the vehicle, are crowded reflected by the reflection on the wet road surface 62. Before run the vehicle 61 and the brake lamp portion 63 _L and 63 _R are the elaborate reflected regions, respectively, shown as reference numeral 61 ₁ and _{63 L1} and _{63 R1.}

  In step S <b> 103, the preceding vehicle detection unit 12 detects the preceding vehicle 61 in the vehicle front image 6. As a method for detecting the preceding vehicle, a generally known template matching method, neural network method, or the like can be used. In the neural network method, a large number of images actually taken are stored in a database and used as teacher data, and are learned in advance by a neural network.

  When a stereo camera is used as the vehicle forward imaging device 2, an object having a height other than the road surface is extracted from the distance information as a candidate for the preceding vehicle, and it is estimated whether the candidate is a preceding vehicle. By performing this, the detection accuracy can be improved. Alternatively, white line information may be acquired in advance from the vehicle forward imaging device 2 and the detection range may be narrowed down within the white line area in front of the host vehicle.

  In step S104, the preceding vehicle detection unit 12 determines whether a preceding vehicle has been detected. If it is determined that a preceding vehicle has been detected, the process proceeds to step S105, which will be described later. Otherwise, the process returns to step S101.

In step S <b> 105, the reference image extraction unit 13 extracts the area of the left brake lamp 63 </ b> _{L and} the area of the right brake lamp 63 </ b> _R of the preceding vehicle 61 as a left reference image and a right reference image, respectively. Both the left reference image and the right reference image are square or rectangular. Specifically, the reference image extraction unit 13 extracts pixels having color information close to the color information of the brake lamp unit in the vehicle front image 6. As this color information, for example, RGB color information can be used.

  In general, RGB color information is composed of an R value (0 to 255) indicating a red component, a G value (0 to 255) indicating a green component, and a B value (0 to 255) indicating a blue component. In the present embodiment, the range of the color information of the brake lamp portion is, for example, 200 ≦ R value ≦ 255, 0 ≦ G value ≦ 50, and 0 ≦ B value ≦ 50. Pixels in the vehicle front image 6 having R value, G value, and B value within this range are extracted as candidate pixels of the brake lamp portion.

Generally, the brake lamp part is attached to the left and right ends of the rear part of the vehicle symmetrically. Therefore, the reference image extraction unit 13 extracts the candidate pixels of the brake lamp unit that are located symmetrically on the left and right ends of the vehicle and uses them as reference images. FIG. 4 shows an example of the reference image. FIG (a) is a front run left reference image 7 on the left side of the brake lamp portion 63 _L is captured of the vehicle 6 _L. FIG (b) is a front run right brake lamp portion 63 _R is reflected and right reference image 7 _R of the vehicle 6.

  In step S106, the reference image extraction unit 13 determines whether a reference image has been extracted. If it is determined that the reference image has been extracted, the process proceeds to step S107, which will be described later. Otherwise, the process returns to step S101.

The search area setting unit 14 performs two-stage search area setting. In step S107, the search area setting unit 14 sets a first search area in the vehicle front image 6 as a first stage of search area setting. 5 shows an example of setting the first search region _{B L} and _{B R.} Each of the first search area B _L and B _R, is set corresponding to the previous run left side of the vehicle brake lamp section 63 _L and the right side of the vehicle brake lamp portion 63 _R of the vehicle 61. Each of the first search area _{B L} and _{B R,} is set below the area of the left and right brake lamp portion 63 _L and 63 _R of the front run the vehicle 61 in the vehicle front image 6.

Specifically, the first search area _BL on the left side of the brake lamp portion 63 _{L is based on} a straight line A passing through two contact points between the rear wheels 64 _L and 64 _R of the preceding vehicle 61 and the road surface 62. It is set so as to include an area that is in a line-symmetrical positional relationship with the area. That is, the distance L ₁ between the center and the straight line A of the left brake lamp portion 63 _L is equal to the distance L ₂ between the center and the straight line A in the first search area B _L. Similarly, the first search area B _R of the right, relative to the straight line A, is set to be as including a region of the brake lamp portion 63 _R of the region line symmetric positional relationship.

Size of the first search area B _L and B _R may be a size larger than the vertical and horizontal size of the reference image 7 _L and 7 _R. For example, it can be set to twice the vertical and horizontal size of the reference image.

In step S108, the estimation unit 15, the color information of the pixels in the first search area B _L compared with the color information of the pixels of the reference image 7 in _L, matches the reference image 7 _L in the first search area B _L Determine whether the location exists. Further, the estimating unit 15, the color information of the first search area B in _R pixels compared to the color information of the pixels in the reference image 7 _R, are places that match the reference image 7 _R in the first search area B _R Determine if it exists. As a search method, for example, a template matching method can be used. Further, the degree of coincidence of color information may be determined as a threshold value.

At the same step, if there are places that match the reference image 7 _L, and is determined locations match the reference image 7 _R in the first search area B _R are present in the first search area B _L, step Proceed to S109, otherwise proceed to step S110.

  In step S109, it is assumed that water has accumulated on the road surface, and the wetness degree W of the road surface is set to 1.0 by the estimation unit 15. After the wetness degree W is output to the signal output unit 16 by the estimation unit 15, Proceed to S112.

  When the degree of wetness of the road surface is relatively large, that is, when water accumulates on the road surface, the brake lamp portion of the preceding vehicle is reflected in a mirror surface and reflected in an area having a symmetrical relationship with respect to the straight line A Steps S107 to S109 as described above are performed based on the viewpoint that the possibility is high.

  The road surface wetting degree W is an index that numerically represents the degree of road surface wetting. The greater the wetness value W, the greater the wetness. When there is a location that matches the reference image in the first search area, the wetness degree W is set to 1.0 as described above.

In step S110, the search area setting unit 14 sets a square or rectangular second search area in the vehicle front image 6 as the second stage of the search area setting. 6 shows an example of setting the second search area _{C L} and _{C R.} The left second search region C _L is set as a region between the first search region B _L and the straight line A. Width of the second search area C _L of the left side is set to be larger than the width of the brake lamp section 63 _L. For example, it can be the same as the horizontal size of the reference image _7L . The vertical length of the second search area C _L of the left side is equal to the length to the straight line A from the upper side of the first search region B _L.

Similarly, the second search area C _R of the right side is set as the region between the first search area B _R and the line A. Width of the second search area C _R of the right side is set to be larger than the width of the brake lamp portion 63 _R. For example, it can be the same as the horizontal size of the reference image 7 _R. The vertical length of the second search area C _R of the right side is equal to the length to the straight line A from the upper side of the first search area B _R.

In step S111, the estimation unit 15 searches for a pixel having color information close to the color information (generally red) of the pixel in the reference image in the second search region. Specifically, firstly, estimating unit 15 obtains the RGB color information of the reference image _{7 L} and _{7 R.} Red information BASE _R is the average value of R values of all pixels in the reference image _{7 L} and _{7 R,} median, maximum value, or the like is used. The same applies to the green information BASE _G and the blue information BASE _B. For example, BASE _R = 200, BASE _G = 10, and BASE _B = 20.

Subsequently, the estimating unit 15 obtains the RGB color information of the second search area _{C L} and _{C R.} Red information AREA _R is the average value of R values of all pixels in the second search area _{C L} and _{C R,} the median, the maximum value or the like is used. The same applies to the green information AREA _G and the blue information AREA _B. For example, AREA _R = 100, AREA _G = 16, and AREA _B = 12.

In this step, the estimation unit 15 further uses the RGB color information (BASE _R , BASE _G , BASE _B ) of the reference image and the RGB color information of the search area (AREA _R , AREA _G , AREA _B ) to determine the degree of wetness W. Is calculated.

This is based on the viewpoint that there is a correlation between the road surface wetness degree W and the degree that the brake lamp portion is reflected in the second search area. FIG. 7 shows an example of the relationship between the road surface wetness degree W and the degree of reflection. As the degree of wetness W of the road surface increases, the degree of reflection increases. However, when the road surface wetness degree W is 1.0 or more, it is assumed that water is accumulated on the road surface, and the degree of reflection is maximum and constant. When the determination result in step S108 is “No”, the degree of wetness W of the road surface is less than 1.0. From the relationship shown in FIG. 7, the road surface wetness degree W is obtained by the following equation.
W = α × (AREA _R ÷ BASE _R ) (1)

Here, the degree of reflection is represented by AREA _R ÷ BASE _R. Α is a coefficient, and can be set to 0.8, for example. For example, regarding the relationship between the degree of wetness on the road surface and the degree of reflection, the value of the coefficient α may be changed depending on whether the driving location is a general road or a highway, or a high value rather than a straight line (proportional) relationship. It is good also as a relationship of a quadratic curve.

In this step, the estimation unit 15 further calculates the value of AREA _G ÷ BASE _{G and} the value of AREA _B ÷ BASE _B , and determines whether each value is equal to or less than a predetermined threshold value. When these two values are less than or equal to the threshold value, the estimation unit 15 outputs the value of the road surface wetting degree W according to the equation (1) to the signal output unit 16. In other cases, the process returns to step S101 (not shown).

Though not necessarily water is accumulated on the road surface, the area between the when the road surface is wet, the straight line A and the first search region B _L and B _R brake lamp portion before run the vehicle is diffused reflected It is highly probable that it is reflected in Based on this viewpoint, steps S110 and S111 as described above are performed. The areas reflected in this way are indicated by reference numerals 63 _L2 and 63 _R2 in FIG.

  In step S <b> 112, the signal output unit 16 outputs a signal corresponding to the road surface wetness degree W output from the estimation unit 15 to the alarm device 4. Then, the alarm device 4 warns the driver of the own vehicle with the sound, vibration, lamp, display, etc. according to the signal output from the signal output unit 16.

  In step S <b> 113, the signal output unit 16 outputs a signal corresponding to the road surface wetness degree W output from the estimation unit 15 to the communication device 5. And according to the signal output from the signal output part 16, the communication apparatus 5 notifies the state of a road surface to another vehicle around the own vehicle, and a road manager. When step S113 is completed, the process returns to step S101.

  According to the above-described embodiment, the degree of wetness due to rain or the like on the road surface on which the host vehicle is traveling can be estimated based on the vehicle front image that captures the front of the host vehicle. This estimation is performed using a reference image from which a brake lamp portion of a preceding vehicle traveling in front of the host vehicle is extracted as a template. Therefore, the degree of wetness of the road surface can be estimated with a relatively simple configuration while suppressing the complexity of the processing.

  Further, since the estimation is performed using the vehicle front image, the degree of wetness of the road surface can be estimated with an inexpensive hardware configuration without physically contacting the road surface.

  In general, when the road surface is wet, it is considered that the possibility that the host vehicle will collide with the preceding vehicle is higher than when the road surface is dry. By estimating the degree of wetness of the road surface according to the above embodiment and issuing a warning to the driver of the host vehicle, the possibility of the host vehicle colliding with the preceding vehicle can be reduced.

  Since the search area setting unit sets the area below the brake lamp area in the vehicle front image as the search area, estimation can be performed efficiently. In addition, since the search area setting is performed in two stages, the estimation efficiency is further increased.

  Since the road surface is often a blackish color, the estimation error can be suppressed by estimating the road surface state based on the red information.

  Not only the brake lamp part of the preceding vehicle but also an arbitrary lamp part such as a tail lamp part can be extracted as a reference image.

  In expressing color information quantitatively, not only the RGB color space but also another color space or color system may be used.

  The system S can be configured to estimate the road surface state only when the wiper is activated.

In step S <b> 112, the signal output unit 16 can output a signal corresponding to the road surface wetness degree W output from the estimation unit 15 to a vehicle control device (not shown). The vehicle control device can control the brake, accelerator, shift position, engine torque, and the like of the host vehicle so that the host vehicle does not slip or skid.
Furthermore, the control timing and control time in the inter-vehicle distance maintaining device (ACC), the collision damage reducing brake (AEB), the skid prevention device (ESC), and the like may be changed according to the signal of the degree of wetness of the road surface.

The first search area B _L of the left side, relative to the straight line A, can also be set as an area itself in the region line symmetric positional relationship of the brake lamp section 63 _L. It is possible to set the region between the thus set first search area B _L and the straight line A of the left as a second search region C _L.
Similarly, the first search area B _R of the right, relative to the straight line A, can also be set as an area itself in the region line symmetric positional relationship of the brake lamp section 63 _R. It is possible to set the region between the first search area B _R and the straight line A of the setting as has been the right as the second search region C _L.

  The functional configuration and physical configuration of the system S and the road surface state estimation device 1 described above are not limited to the above-described modes. For example, the functions and physical resources are integrated and mounted, or conversely, further distributed. It is also possible to implement it.

1 Road surface condition estimation device

DESCRIPTION OF SYMBOLS 11 Image acquisition part 12 Front running vehicle detection part 13 Reference | standard image extraction part 14 Search area setting part 15 Estimation part 16 Signal output part

2 Vehicle front imaging device 3 Vehicle information acquisition device 4 Alarm device 5 Communication device

6 Vehicle front image 61 Front running vehicle 62 Road surface 63 _L , 63 _R Brake lamp part

7 _{L, 7 R} reference image _B L, _{B R} first search region _C L, _{C R} second search area

Claims (6)

In the vehicle front image that shows the front of the vehicle, a reference image extraction unit that extracts a region of the ramp portion of the preceding vehicle as a reference image;
Search area setting for setting an area in the vehicle front image in which reflection of the lamp part may occur due to reflection of the lamp part on the road surface due to wetness of the road surface in front of the vehicle as a search area And
A road surface state estimation device comprising: an estimation unit that estimates the degree of wetness of the road surface by comparing color information of pixels in the search area with color information of pixels in the reference image.   The road surface state estimation device according to claim 1, wherein the search region is set below the region of the ramp portion in the vehicle front image.   The search area includes a first search area that is in a line-symmetrical positional relationship with the area of the ramp section with reference to a straight line passing through two contact points between a rear wheel of the preceding vehicle and the road surface. 2. The road surface state estimating apparatus according to 2.   The road surface state estimation device according to claim 3, wherein the search area further includes a second search area between the first search area and the straight line.   The road surface state estimation device according to any one of claims 1 to 4, wherein the ramp unit is a brake ramp unit or a tail lamp unit.   The road surface state estimation apparatus according to any one of claims 1 to 5, wherein the color information is red information.

Images