JP2023064793A - Road surface deterioration detection device and road surface deterioration detection method - Google Patents

Abstract

To provide a road surface deterioration detection device for accurately detecting road surface deterioration.SOLUTION: A road surface deterioration detection device comprises: an imaging section 1 for acquiring an image; an own vehicle position information acquisition section 2 for acquiring information of an own vehicle position; a map information acquisition section 3 for acquiring map information which includes road information including at least the number of lanes of a road, and includes attribute information of a position on a map; a lane area extraction section 4 for extracting a lane area image from the image; an own vehicle travel lane determination section 5 for determining an own vehicle travel lane from the own vehicle position and the map information; a detection area extraction section 6 for extracting a detection area image from the lane area image by using the own vehicle position, the road information, and the information of the own vehicle travel lane; and a road surface deterioration detection section 7 for detecting road surface deterioration from the detection area image. The road surface deterioration detection section 7 changes detection reference in detecting the road surface deterioration based on the own vehicle position and the attribute information.SELECTED DRAWING: Figure 1

Description

The present application relates to a road surface deterioration detection device and a road surface deterioration detection method.

With the aging of roads, bridges, facilities, etc., efficient maintenance of buildings is required. In order to carry out building maintenance, it is necessary to find defects in the building, but it is difficult to perform quantitative evaluation and comprehensive analysis by visual confirmation. Therefore, automatic detection by machines is becoming popular. An image acquisition unit that acquires an image of an area including the surface of the building continuously captured by an imaging device mounted on a moving body that moves along the surface of the building as a device for automatic detection, and an image acquisition unit. divides the acquired image and outputs a plurality of partial images which are part of the image; An information processing apparatus including a state determination unit has been proposed (see Patent Document 1, for example).

Japanese Patent No. 6647171

In the information processing apparatus disclosed in Patent Literature 1, the state determination section determines a defect from the image acquired by the image acquisition section. However, when the moving object is in a tunnel, the image data changes depending on the position of the moving object or the circumstances of the surroundings.

SUMMARY OF THE INVENTION The present application has been made to solve the above-described problems, and it is an object of the present invention to provide a road surface deterioration detection device and a road surface deterioration detection method that accurately detect road surface deterioration.

The road surface deterioration detection device disclosed in the present application includes an imaging unit that is mounted on a vehicle and acquires an image including the road surface, a vehicle position information acquisition unit that acquires vehicle position information that is the position of the vehicle, and a map information acquisition unit that acquires map information of an area including a vehicle position; a lane area extraction unit that detects lane marks that separate lanes on a road surface from an image and extracts a lane area image from the image using the lane marks; a vehicle driving lane determination unit that determines a vehicle driving lane in which the vehicle is traveling from the vehicle position and map information or from lane marks; a detection area extraction unit that extracts a detection area image from the lane area image; a road surface deterioration detection unit that detects road surface deterioration from the detection area image, the map information includes road information including at least the number of lanes on the road and attribute information of the position on the map, and the detection area extraction unit detects the vehicle position. , the road information and information on the lane in which the vehicle is traveling are used to extract a detection area image from the lane area image. Characterized by changing.

In the road surface deterioration detection device disclosed in the present application, the map information includes road information including at least the number of lanes on the road and attribute information of the position on the map, and the detection area extraction unit extracts the vehicle position, the road information, and the vehicle The detection area image is extracted from the lane area image using the information of the driving lane, and the road surface deterioration detection unit changes the detection criteria for detecting road surface deterioration based on the vehicle position and attribute information. Road deterioration can be detected.

1 is a block diagram showing the configuration of a road surface deterioration detection device according to Embodiment 1; FIG. 4 is a flowchart for explaining the operation of the road surface deterioration detection device according to Embodiment 1; 4 is a diagram for explaining an imaging unit in Embodiment 1; FIG. FIG. 4 is a diagram for explaining acquisition of map information by a map information acquisition unit according to Embodiment 1; FIG. 4A and 4B are diagrams showing images acquired by an imaging unit according to the first embodiment; FIG. 4 is a diagram for explaining attribute information according to Embodiment 1; FIG. 4 is a diagram showing the relationship between attribute information and thresholds for edge detection processing according to Embodiment 1. FIG. FIG. 7 is a diagram for explaining another example of a method for determining a threshold value for edge detection processing according to Embodiment 1; FIG. 4 is a diagram for explaining the operation of a detection region extraction unit according to Embodiment 1; FIG. 4 is a diagram for explaining the operation of a detection region extraction unit according to Embodiment 1; 4 is a diagram showing the relationship between road information and whether or not to extract as an extraction area image in Embodiment 1. FIG. 4 is a diagram showing the relationship between attribute information and whether or not road surface deterioration is to be detected in Embodiment 1. FIG. FIG. 4 is a diagram for explaining data transmission in a detection result information transmission unit according to Embodiment 1; 4 is a diagram for explaining operations of an alarm determination unit and an image generation unit according to Embodiment 1; FIG. 4 is a diagram for explaining operations of an alarm determination unit and an image generation unit according to Embodiment 1; FIG. 1 is a schematic diagram showing an example of hardware of a road surface deterioration detection device according to Embodiment 1; FIG.

Hereinafter, a road surface deterioration detection device according to an embodiment for carrying out the present application will be described in detail with reference to the drawings. In each figure, the same reference numerals denote the same or corresponding parts.

Embodiment 1.
FIG. 1 is a block diagram showing the configuration of a road surface deterioration detection device 100 according to Embodiment 1. As shown in FIG. The road surface deterioration detection device 100 includes an imaging unit 1, a vehicle position information acquisition unit 2, a map information acquisition unit 3, a lane area extraction unit 4, a vehicle traveling lane determination unit 5, a detection area extraction unit 6, and a road surface deterioration detection unit 7. , a detection result information transmission unit 8 and a road surface deterioration warning unit 10 . The road surface deterioration warning section 10 includes a warning determination section 11 and a warning transmission section 12 . The alarm transmission unit 12 includes an image generation unit 13 , an image output unit 14 , an audio generation unit 15 , an audio output unit 16 and a vibration generation unit 17 .

The imaging unit 1 is an in-vehicle camera that is mounted on the vehicle and acquires an image including the road surface. The vehicle position information acquisition unit 2 is a locator that acquires vehicle position information, which is the position of the vehicle. The map information acquisition unit 3 uses the information of the vehicle position acquired from the vehicle position information acquisition unit 2 to acquire map information of the area around the vehicle including the vehicle position from the map information database. The lane area extraction unit 4 uses the vehicle position information acquired from the vehicle position information acquisition unit 2 and the map information acquired from the map information acquisition unit 3 to segment the road surface from the image acquired from the imaging unit 1. A lane mark is detected, and a lane area image is extracted from the image acquired from the imaging unit 1 using the lane mark. The vehicle traveling lane determination unit 5 determines the vehicle position information from the vehicle position information acquisition unit 2 and the map information acquired from the map information acquisition unit 3, or from the lane mark acquired from the lane area extraction unit 4. , determine the own vehicle driving lane in which the own vehicle is traveling.

The detection area extracting unit 6 extracts the vehicle position information obtained from the vehicle position information obtaining unit 2, the map information obtained from the map information obtaining unit 3, and the vehicle driving lane obtained from the vehicle driving lane determination unit 5. A detection area image is extracted from the lane area image acquired from the lane area extraction unit 4 using the information of . The road surface deterioration detection unit 7 uses the vehicle position information acquired from the vehicle position information acquisition unit 2 and the map information acquired from the map information acquisition unit 3 to generate the detection area image acquired from the detection area extraction unit 6. to detect road surface deterioration. The detection result information transmission unit 8 transmits the road surface deterioration information detected by the road surface deterioration detection unit 7 to the outside of the road surface deterioration detection device 100 .

The road surface deterioration warning unit 10 issues a warning based on the road surface deterioration information detected by the road surface deterioration detection unit 7 and the vehicle driving lane information obtained from the vehicle driving lane determination unit 5 . The warning judgment unit 11 judges the warning notification level based on the road surface deterioration information detected by the road surface deterioration detection unit 7 and the vehicle driving lane information acquired from the vehicle driving lane determination unit 5, and detects road surface deterioration. A warning signal is output as the warning notification level when the vehicle is in the vehicle's driving lane, and a caution signal is output as the warning notification level when the road surface deterioration is not in the vehicle's driving lane. The alarm issuing unit 12 issues an alarm based on the alarm notification level acquired from the alarm determining unit 11 . The image generation unit 13 generates an alarm image based on the alarm notification level acquired from the alarm determination unit 11 . The image output section 14 displays the warning image acquired from the image generation section 13 . The sound generation unit 15 generates warning sound based on the warning notification level acquired from the warning determination unit 11 . The voice output unit 16 outputs the warning voice acquired from the voice generation unit 15 . The vibration generator 17 generates vibration based on the alarm notification level acquired from the alarm determination unit 11 . The alarm transmission unit 12 includes at least an image alarm transmission unit having an image generation unit 13 and an image output unit 14, an audio alarm transmission unit having an audio generation unit 15 and an audio output unit 16, and a vibration generation unit 17. All you have to do is have one.

Next, the operation of the road surface deterioration detection device 100 according to Embodiment 1 will be described. FIG. 2 is a flow chart for explaining the operation of the road surface deterioration detection device 100 according to the first embodiment. Step S01 is an imaging step, Step S02 is a vehicle position information acquisition step, Step S03 is a map information acquisition step, Steps S04 and S05 are lane area extraction steps, and Step S06 is a vehicle travel lane. This is the judgment step. Steps S07, S08 and S09 are detection area extraction steps. Steps S10 and S11 are road surface deterioration detection steps.

In step S01, the imaging unit 1 acquires an image of the surroundings of the vehicle including the road surface, outputs the acquired image to the lane area extraction unit 4, and proceeds to step S02. FIG. 3 is a diagram for explaining the imaging unit 1. As shown in FIG. FIG. 3 shows how the imaging unit 1 mounted on the own vehicle 200 is photographing the front of the own vehicle 200, that is, the traveling direction of the own vehicle 200. As shown in FIG. In the example shown in FIG. 3, the own vehicle 200 is traveling in the center lane of the three lanes indicated by the lane marks 21, 22, 23 and 24, and the road surface deterioration 30 is present in the center lane. The angle of view θc of the imaging unit 1 is set to 120 degrees, for example. However, the angle of view θc of the imaging unit 1 is not limited to 120 degrees.

In step S02, the vehicle position information acquisition unit 2 acquires vehicle position information, which is the position of the vehicle 200, and the map information acquisition unit 3, the lane area extraction unit 4, the vehicle traveling lane determination unit 5, the detection The data is output to the area extraction unit 6 and the road surface deterioration detection unit 7, and the process proceeds to step S03. The own vehicle position information acquisition unit 2 is mounted on the own vehicle 200 . The vehicle position information acquisition unit 2 is a high-definition locator (HDL: High-Definition Locator) that acquires information on the vehicle position. For example, the position information has an error of 30 cm or less. The vehicle position information acquisition unit 2 may be a GPS (Global Positioning System) receiver capable of accurately observing the latitude and longitude position information of the vehicle 200 .

In step S03, the map information acquisition unit 3 acquires map information of an area including the vehicle position based on the information of the vehicle position obtained in step S02. It is output to the determination section 5, the detection area extraction section 6, and the road surface deterioration detection section 7, and the process proceeds to step S04. FIG. 4 is a diagram for explaining acquisition of map information by the map information acquisition unit 3. As shown in FIG. The map information acquisition unit 3 accesses the map information database 19 storing map information via the Internet 18 to acquire map information of an area including the vehicle position. The map information stored in the map information database 19 includes, as attribute information of the position shown on the map, information on what kind of place such as general roads, highways, service areas, private land, etc. . The map information acquired by the map information acquiring unit 3 includes road information of the roads shown on the map. lane information and information on the presence or absence of a median strip. Therefore, the map information acquired by the map information acquisition unit 3 includes attribute information of the vehicle position and road information of the road on which the vehicle 200 travels. Note that, for example, map information is stored in advance in a memory mounted on the vehicle 200, and the map information acquisition unit 3 acquires the map information by accessing the memory. Any method is acceptable.

In step S04, lane area extraction unit 4 acquires the number of lanes of the road on which vehicle 200 is traveling from the vehicle position information obtained in step S02 and the map information obtained in step S03. , the process proceeds to step S05. For example, in the example shown in FIG. 3, the number of lanes acquired by the lane area extraction unit 4 in step S04 is "3".

In step S05, the lane area extraction unit 4 extracts a lane area image from the image acquired by the imaging unit 1 using the information on the number of lanes obtained in step S04, and outputs the lane area image to the detection area extraction unit 6, Proceed to step S06. FIG. 5 is a diagram showing a part of the image acquired by the imaging unit 1 in the example shown in FIG. The lane area extraction unit 4 detects lane marks 21, 22, 23, and 24 that separate the lanes on the road surface from the image shown in FIG. When the lane marks 21, 22, 23, and 24 are detected by image processing, the upper limit of the number of lane marks to be detected is set based on the number of lanes obtained in step S04, and the lane marks are detected. For example, when "3" is obtained as the number of lanes in step S04, the maximum number of detected lane marks is "4". Lane mark detection uses, for example, edge detection processing for detecting edges from luminance or saturation gradient information of each pixel on the image using a Sobel filter or a Canny filter.

The map information includes information such as general roads, highways, service areas, private land, etc. as attribute information of the position shown on the map. Attribute information of the car position can be acquired. Lane area extraction unit 4 acquires the attribute information of the vehicle position from the information of the vehicle position obtained in step S02 and the map information obtained in step S03, and according to the attribute information of the vehicle position, Two thresholds, which are detection criteria for edge detection processing, are changed. FIG. 6 is a diagram for explaining the attribute information of the positions shown on the map. FIG. 6 shows an example in which the own vehicle 200 passes through a tunnel 63 from a general road 61 and moves to a general road 62 . When the vehicle 200 is traveling on the general road 61 or the general road 62, the attribute information of the vehicle position is "general road", and when the vehicle 200 is traveling through the tunnel 63, the attribute information of the vehicle position. becomes a "tunnel". FIG. 7 is a diagram showing the relationship between the attribute information of the vehicle position and two thresholds that are detection criteria for edge detection processing. According to the table shown in FIG. 7, the lane area extraction unit 4 sets two thresholds for edge detection processing to (60, 130) when the vehicle 200 is traveling on the general road 61 or 62 in FIG. When the vehicle 200 is traveling through the tunnel 63, two thresholds for edge detection are set to (30, 70) and edge detection is performed. Note that the values shown in FIG. 7 are just an example, and the relationship between the attribute information and the threshold value for the edge detection processing should be such that the two threshold values for the edge detection processing can easily detect the lane mark at the position of the vehicle.

The detection of lane marks in the lane area extraction unit 4 is not limited to edge detection processing, and any processing that can detect lane marks, such as detection using machine learning, may be used. For example, when machine learning is used to detect lane marks, teacher data, which is a detection criterion for machine learning, is changed according to the attribute information of the vehicle position. In this way, by changing the detection criteria for lane mark detection based on the attribute information of the vehicle's position, it is possible to detect lane marks more accurately and prevent undetected or erroneous detection of lane marks. can be suppressed. As a result, road surface deterioration can be detected more accurately.

FIG. 8 is a diagram for explaining another example of a method of determining a threshold value for edge detection processing in the lane area extraction unit 4. In FIG. When detection of lane marks in the lane area extraction unit 4 is performed by edge detection processing, as shown in FIG. may be obtained, and the two thresholds for edge detection processing may be changed according to the obtained average luminance value. Also, the two thresholds for the edge detection process may be changed according to the brightness value in the brightness detection area 56 or the saturation value in the brightness detection area 56 .

Next, the lane area extraction unit 4 extracts the detected lane marks 21, 22, 23, and 24, the upper limit line 41, and the lower limit line 42 from the image shown in FIG. The images of the respective enclosed areas are extracted as lane area images 51 , 52 and 53 and output to the detection area extraction section 6 . The lane area image is extracted by dividing the lane area image with the detected lane marks 21, 22, 23, and 24 as boundaries. The upper limit line 41 and the lower limit line 42 are determined in advance. For example, the upper limit line 41 is a line half the height of the image acquired by the imaging unit 1, and the lower limit line 42 is acquired by the imaging unit 1. be the line at the bottom edge of the rendered image. The upper limit line 41 and the lower limit line 42 are not limited to the above. 42 can be extracted as a lane area image. For example, by distinguishing and extracting three lane area images 51, 52, and 53 shown in FIG. can be used to determine the positional relationship of each lane, for example, whether it is the lane in which the vehicle is traveling or whether it is the oncoming lane. Only the lane area image can be extracted as the detection area image.

In step S06, vehicle driving lane determination unit 5 determines the vehicle driving lane in which vehicle 200 travels from the vehicle position obtained in step S02 and the map information obtained in step S03, and extracts a detection area. Output to the unit 6 and the alarm determination unit 11, and proceed to step S07. The map information includes at least the number of lanes of the road as road information, and may include the lane width of the road as the road information. From the position of the vehicle on the road on the map and the number of lanes on the road at the position of the vehicle, the lane in which the vehicle 200 runs can be determined. Information on the lane width of the road on which the vehicle is located may be used to determine the lane in which the vehicle is traveling. If only one lane area image is extracted in step S05, the lane indicated by the extracted lane area image is set as the vehicle driving lane.

Alternatively, in step S06, the own vehicle traveling lane determination unit 5 uses the information of the lane marks 21, 22, 23, and 24 detected by the lane area extraction unit 4 to determine the own vehicle traveling lane in which the vehicle 200 travels. You may In the example shown in FIG. 5, in which the road surface in the traveling direction of the vehicle 200 is photographed, let θ1, θ2, θ3, and θ4 be the inclinations of the lane marks 21, 22, 23, and 24 with respect to the vertical direction of the image shown in FIG. , the lane marks 21 and 22 have a positive slope, and the lane marks 23 and 24 have a negative slope. At this time, the lane of the lane area image 52 sandwiched between the lane mark 22 with a positive gradient and the lane mark 23 with a negative gradient is determined as the vehicle's driving lane. In this way, in step S06, the vehicle driving lane determination unit 5 determines the vehicle driving lane based on the difference in inclination of each of the lane marks 21, 22, 23, and 24 detected by the lane region extraction unit 4. may If two lane marks are detected in step S05 and only one lane area image is extracted, the lane of the extracted lane area image is set as the vehicle's driving lane.

In step S07, the detection area extracting unit 6 uses the vehicle position obtained in step S02, the map information obtained in step S03, and the vehicle driving lane information obtained in step S06 to perform step A detection area image is extracted from the lane area image obtained in S05, and the process proceeds to step S08. The map information obtained in step S03 includes road information of the road on which vehicle 200 travels. Therefore, in step S07, the detection area extracting unit 6 acquires the road information of the road on which the vehicle 200 travels from the vehicle position and the road information included in the map information. is used to extract the detection area image.

FIG. 9 is a diagram for explaining the operation of the detection area extraction unit 6 according to the first embodiment. FIG. 9 shows an example in which three lane area images 51, 52, and 53a are extracted in step S05, and the lane of the lane area image 52 is determined to be the vehicle driving lane in step S06. For example, when the detection area extracting unit 6 acquires the information that the lane of the lane area image 53a is under construction as the road information of the road on which the vehicle 200 travels, the detection area extracting unit 6 extracts the lane area image 51 and the lane area image 52 as the detection area. image, and the lane area image 53a is not extracted as a detection area image. 10A and 10B are diagrams showing another example for explaining the operation of the detection area extraction unit 6 according to Embodiment 1. FIG. 10, four images of lane area images 51, 52, 54, and 55 are extracted from the lane marks 21, 22, 23, 25, and 26 in step S05, and the lane of the lane area image 52 is detected in step S06. An example determined as a lane is shown. For example, when the detection area extracting unit 6 acquires the information that the median strip 27 is laid in the center of the road on which the vehicle 200 is traveling as the road information of the road on which the vehicle 200 is traveling, the detection area extraction unit 6 extracts the lane area image 51 and lane area image 52 are extracted as detection area images, and lane area images 54 and 55 located in the oncoming lane are not extracted as detection area images.

FIG. 11 is a diagram showing the relationship between road information of lanes and whether or not to extract as an extraction area image. For example, the table shown in FIG. 11 is stored in advance, and the detection area extraction unit 6 determines whether or not to extract each lane area image as the detection area image based on the road information of lanes and the table shown in FIG. do. In the example shown in FIG. 11, when the road information of the lane is "oncoming lane with no median", it is extracted as a detection area image, and when the road information of the lane is "under construction", it is extracted as a detection area image. do not. Note that the detection area extraction unit 6 may extract the detection area image from the lane area image in accordance with the objects or signs around the vehicle 200 . In this way, by extracting the detection area image from the lane area image using the vehicle position, road information, and information on the lane in which the vehicle is traveling, the lane area image that does not need to detect road surface deterioration can be detected as a road surface deterioration detection target. , the erroneous detection of road surface deterioration can be reduced, and the processing for detecting road surface deterioration can be reduced.

In step S08, the detection area extraction unit 6 confirms whether or not the detection area image has been extracted. If the detection area image has been extracted, the process proceeds to step S09, and if the detection area image has not been extracted, the process proceeds to step S01. return. In step S09, the detection area extracting unit 6 determines whether the vehicle position is a road surface deterioration detection target based on the information on the vehicle position obtained in step S02 and the attribute information included in the map information obtained in step S03. determine whether or not When it is determined that the vehicle position is subject to detection of road surface deterioration, the detection area image is output to the road surface deterioration detecting section 7, and the process proceeds to step S10. When it is determined that the vehicle position is not the road surface deterioration detection target, the process returns to step S01. FIG. 12 is a diagram showing the relationship between the attribute information of the vehicle position and whether or not road surface deterioration is to be detected. For example, the table shown in FIG. 12 is stored in advance, and the detection area extracting unit 6 determines whether or not the vehicle position is subject to detection of road surface deterioration based on the attribute information of the vehicle position and the table shown in FIG. . In the example shown in FIG. 12, when the attribute information of the vehicle position is "general road", it is determined that road surface deterioration is to be detected, and when the attribute information of the vehicle position is "service area", road surface deterioration is detected. determined not to be covered.

In step S10, the road surface deterioration detection unit 7 detects road surface deterioration from the detection area image output from the detection area extraction unit 6, and the process proceeds to step S11. For example, when the lane area image 52 shown in FIG. In FIG. 5, the road surface deterioration 30 is shown as cracks, but the road surface deterioration to be detected is not limited to cracks, and any defect that appears on the road surface, such as holes and ruts, may be used. In addition, the road surface deterioration detection unit 7 detects not only the road surface deterioration, but also objects such as fallen objects on the road surface and fallen branches protruding from the road, which adversely affect driving and need to be repaired. .

The detection of the road surface deterioration 30 uses, for example, edge detection processing that detects edges from luminance or saturation gradient information of each pixel on the image using a Sobel filter or a Canny filter. The road surface deterioration detection unit 7 performs edge detection processing using, for example, a Canny filter. The road surface deterioration detection unit 7 acquires the attribute information of the vehicle position from the information of the vehicle position obtained in step S02 and the map information obtained in step S03, and detects the attribute information of the vehicle position according to the attribute information of the vehicle position. Two thresholds, which are detection criteria for edge detection processing, are changed. For example, like the lane area extraction unit 4, the road surface deterioration detection unit 7 changes two threshold values that are detection criteria for edge detection processing according to the table shown in FIG. Note that the values shown in FIG. 7 are just an example, and the relationship between the attribute information and the threshold value for edge detection processing should be such that the two threshold values for edge detection processing can easily detect road surface deterioration 30 at the position of the vehicle.

Detection of the road surface deterioration 30 by the road surface deterioration detection unit 7 is not limited to edge detection processing, and any processing that can detect the road surface deterioration 30, such as detection using machine learning, may be used. For example, when machine learning is used to detect the road surface deterioration 30, teacher data, which is a detection standard for machine learning, is changed according to the attribute information of the vehicle position. In this way, by changing the detection criteria for detecting the road surface deterioration 30 based on the attribute information of the vehicle position, the road surface deterioration 30 can be detected more accurately, and the road surface deterioration can be undetected or erroneously detected. Detection can be suppressed.

Similarly to the lane area extraction unit 4, the road surface deterioration detection unit 7 sets a luminance detection area 56 in the detection area image as shown in FIG. Two thresholds for edge detection processing may be changed according to the average value. Also, the two thresholds for the edge detection process may be changed according to the brightness value in the brightness detection area 56 or the saturation value in the brightness detection area 56 .

In step S11, the road surface deterioration detector 7 confirms whether the road surface deterioration 30 has been detected. When the road surface deterioration 30 is detected, information on the detected road surface deterioration is output to the detection result information transmitting section 8 and the alarm determination section 11, and the process proceeds to step S12. When road surface deterioration 30 is not detected, the process returns to step S01.

In step S12, the detection result information transmitting unit 8 transmits data obtained by adding information on the vehicle position when road surface deterioration is detected to the detected road surface deterioration information acquired from the road surface deterioration detecting unit 7 to the road surface deterioration detecting device. 100 outside. The destination is, for example, a road surface deterioration information database. FIG. 13 is a diagram for explaining data transmission in the detection result information transmission unit 8. As shown in FIG. The detection result information transmission unit 8 transmits data to the road surface deterioration information database 20 via the Internet 18, for example. The detection result information transmitting unit 8 may transmit data obtained by adding information on the position of the vehicle when road surface deterioration is detected to information on the detected road surface deterioration acquired from the road surface deterioration detecting unit 7 to the cloud. . By transmitting the road surface deterioration information detected by the detection result information transmitting unit 8 to the road surface deterioration information database 20 or the cloud, the road surface deterioration information from a plurality of vehicles is collectively analyzed in the road surface deterioration information database 20 or the cloud. It is possible to obtain more reliable road surface deterioration information.

In the following description, the alarm transmission unit 12 includes an image alarm transmission unit including an image generation unit 13 and an image output unit 14, an audio alarm transmission unit including an audio generation unit 15 and an audio output unit 16, and a vibration generation unit 17. The operation in the case of having and will be described. In step S13, the warning determination unit 11 determines the warning notification level based on the information on the lane in which the vehicle is traveling obtained in step S06 and the information on the road surface deterioration obtained in step S10. 15 and the vibration generator 17, and proceeds to step S14. 14A and 14B are diagrams for explaining the operations of the alarm determination unit 11 and the image generation unit 13. FIG. FIG. 14 shows an example in which the lane of the lane area image 52 is the lane in which the vehicle is traveling, and road surface deterioration 31 is detected outside the lane in which the vehicle is traveling. When the bounding box 33 indicating the position of the road surface deterioration 31 on the image does not overlap the area of the lane area image 52, which is the lane in which the vehicle is traveling, the warning determination unit 11 outputs a caution signal as the warning notification level to the image generation unit 13. , to the sound generator 15 and the vibration generator 17 . FIG. 15 is another diagram for explaining the operations of the warning determination unit 11 and the image generation unit 13. FIG. FIG. 15 shows an example in which the lane of the lane area image 52 is the lane in which the vehicle is traveling, and the road surface deterioration 32 is detected in the lane in which the vehicle is traveling. When the bounding box 34 indicating the position of the road surface deterioration 32 on the image overlaps even partially with the area of the lane area image 52, which is the lane in which the vehicle is traveling, the alarm determination unit 11 sets the alarm signal to the image as the alarm notification level. It outputs to the generation unit 13 , the sound generation unit 15 and the vibration generation unit 17 .

The alarm judgment unit 11 sets the alarm notification level to only the alarm signal, and the alarm judgment unit 11 determines that the bounding box 34 indicating the position on the image of the road surface deterioration 32 is part of the lane area image 52 in which the vehicle is traveling. However, if they overlap, that is, when the road surface deterioration 32 is in the lane area image 52 that is the lane in which the vehicle is traveling, a warning signal is sent to the image generation unit 13, the sound generation unit 15, and the vibration generation unit 17 as the warning notification level. may be output. The warning determination unit 11 does not always output a warning signal when road surface deterioration is detected, but outputs a warning signal when road surface deterioration is present in the own vehicle's driving lane, thereby giving the driver unnecessary strain. There is no

In step S14, the image generation unit 13 creates an alarm image based on the alarm notification level acquired from the alarm determination unit 11, outputs the alarm image to the image output unit 14, and proceeds to step S15. In the example shown in FIG. 14, the image generation unit 13 receives a caution signal from the warning determination unit 11, generates a warning image 71 indicating that the road surface deterioration 31 is detected outside the lane in which the vehicle is traveling, and generates the warning image 71. is output to the image output unit 14 . Alternatively, the image generation unit 13 receives the warning signal from the warning determination unit 11 , generates a warning image 71 indicating that the road surface deterioration 31 is detected outside the lane in which the vehicle is traveling, and generates the warning image 71 from the imaging unit 1 The acquired images may be superimposed to generate an alarm image as shown in FIG. 14 and output to the image output unit 14 . In the example shown in FIG. 15, the image generation unit 13 receives the warning signal from the warning determination unit 11, generates a warning image 72 indicating that the road surface deterioration 31 has been detected in the lane in which the vehicle is traveling, and displays the warning image 72 as an image. Output to the output unit 14 . Alternatively, the image generation unit 13 receives the warning signal from the warning determination unit 11, generates a warning image 72 indicating that the road surface deterioration 31 has been detected in the lane in which the vehicle is traveling, and uses the image acquired from the imaging unit 1 as the warning image. may be superimposed to generate an alarm image as shown in FIG. Note that the warning images 71 and 72 are not limited to those shown in FIGS. 14 and 15, and may be of any type as long as the warning can be recognized. In step S15, the image output unit 14 displays the warning image generated in step S14, and proceeds to step S16. As a result, the driver driving the own vehicle 200 can recognize that road surface deterioration has been detected around the own vehicle 200 by checking the warning image.

In step S16, the sound generating unit 15 generates a sound signal that conveys that road surface deterioration has been detected around the vehicle 200 based on the alarm notification level acquired from the alarm determining unit 11, Output to the voice output unit 16, and proceed to step S17. For example, the sound generator 15 may generate a sound signal and output it to the sound output unit 16 when an alarm signal is received, and may not generate a sound signal when a caution signal is received. The audio generator 15 may generate different audio signals when receiving an alarm signal and when receiving a caution signal. In step S17, the audio output unit 16 outputs the audio signal generated by the audio generation unit 15 as sound, and the process proceeds to step S18. Accordingly, the driver driving the own vehicle 200 can recognize that road surface deterioration has been detected around the own vehicle 200 by confirming the sound emitted from the voice output unit 16 .

In step S18, the vibration generator 17 notifies the driver of the vehicle 200 that road surface deterioration has been detected around the vehicle 200 by generating vibration based on the alarm notification level acquired from the alarm determination unit 11. and terminate the process. The vibration generator 17 is for generating vibrations in the driver's seat or steering wheel, for example. Any vibration that informs the driver that road surface deterioration has been detected may be used. For example, the vibration generator 17 may generate vibration when receiving an alarm signal and may not generate vibration when receiving a caution signal. The vibration generator 17 may generate different vibrations depending on whether the alarm signal is received or the caution signal is received. As a result, the driver driving the own vehicle 200 can recognize that road surface deterioration has been detected around the own vehicle 200 by confirming the vibration generated from the vibration generating section 17 .

During operation of the road surface deterioration detecting device, the processing from step S01 to step S18 shown in FIG. 2 is repeatedly performed.

As described above, the road surface deterioration detection apparatus 100 according to Embodiment 1 includes the imaging unit 1 mounted on the own vehicle 200 to obtain an image including the road surface, and the own vehicle position information, which is the position of the own vehicle 200. A vehicle position information acquisition unit 2, a map information acquisition unit 3 for acquiring map information of an area including the vehicle position, a lane mark that separates lanes on the road surface from the image, and lane markings from the image using the lane mark. A lane area extracting unit 4 for extracting an area image, a vehicle traveling lane determination unit 5 for determining a vehicle traveling lane in which the vehicle 200 travels from the vehicle position and map information or from lane marks, and a lane area. A detection area extraction unit 6 for extracting a detection area image from an image, and a road surface deterioration detection unit 7 for detecting road surface deterioration from the detection area image. The detection area extracting unit 6 extracts the detection area image from the lane area image using the vehicle position, road information, and vehicle driving lane information. Since the detection criteria for detecting road surface deterioration are changed based on the position and attribute information, road surface deterioration can be detected with high accuracy.

Further, since the road surface deterioration detection apparatus 100 according to Embodiment 1 can detect road surface deterioration with high accuracy, when transmitting information on the detected road surface deterioration to the road surface deterioration information database 20, it is possible to An increase in transmission data can be suppressed. In addition, the administrator or repairer of the paved object can use the road surface deterioration information database 20 in which highly reliable data is accumulated.

FIG. 16 is a schematic diagram showing an example of hardware of the road surface deterioration detection device 100 according to the first embodiment. The imaging unit 1 is a camera 303, the vehicle position information acquisition unit 2 is a locator 304, the map information acquisition unit 3 is a transceiver 305, the detection result information transmission unit 8 is a transmitter 306, and the image output unit. 14 is a display 307 , the audio output section 16 is a speaker 308 , and the vibration generating section 17 is a vibrator 309 . Lane area extraction unit 4, vehicle traveling lane determination unit 5, detection area extraction unit 6, road surface deterioration detection unit 7, warning determination unit 11, image generation unit 13, and sound generation unit 15 execute programs stored in memory 302. It is implemented by a processor 301 such as a CPU for execution and a system LSI. Also, a plurality of processing circuits may work together to perform the functions described above. Furthermore, the above functions may be realized by dedicated hardware. Where dedicated hardware implements the above functions, the dedicated hardware may be, for example, a single circuit, multiple circuits, a programmed processor, an ASIC, an FPGA, or a combination thereof. The above functions may be realized by a combination of dedicated hardware and software, or a combination of dedicated hardware and firmware. Processor 301, memory 302, camera 303, locator 304, transceiver 305, transmitter 306, display 307, speaker 308 and vibrator 309 are bussed together.

Although the present application describes exemplary embodiments, the various features, aspects, and functions described in the embodiments are not limited to application of particular embodiments, alone or Various combinations are applicable to the embodiments.
Therefore, countless modifications not illustrated are envisioned within the scope of the technology disclosed in the present application. For example, the modification, addition, or omission of at least one component shall be included.

1 Imaging unit 2 Vehicle position information acquisition unit 3 Map information acquisition unit 4 Lane area extraction unit 5 Vehicle traveling lane determination unit 6 Detection area extraction unit 7 Road surface deterioration detection unit 8 Detection result information transmission unit 10 road surface deterioration warning unit 11 warning determination unit 12 warning transmission unit 13 image generation unit 14 image output unit 15 sound generation unit 16 sound output unit 17 vibration generation unit 18 Internet 19 map information database, 20 road surface deterioration information database, 21, 22, 23, 24, 25, 26 lane mark, 27 median strip, 30, 31, 32 road surface deterioration, 33, 34 bounding box, 41 upper limit line, 42 lower limit line, 51, 52 , 53, 53a, 54, 55 lane area image, 56 luminance detection area, 61, 62 general road, 63 tunnel, 71, 72 warning image, 100 road surface deterioration detection device, 200 own vehicle, 301 processor, 302 memory, 303 camera , 304 locator, 305 transceiver, 306 transmitter, 307 display, 308 speaker, 309 vibrator.

Claims (10)

an imaging unit that is mounted on the vehicle and acquires an image including the road surface;
an own vehicle position information acquisition unit that acquires information on the own vehicle position, which is the position of the own vehicle;
a map information acquisition unit that acquires map information of an area including the vehicle position;
a lane area extraction unit that detects lane marks that separate lanes on the road surface from the image and extracts a lane area image from the image using the lane marks;
a vehicle driving lane determination unit that determines a vehicle driving lane in which the vehicle travels from the vehicle position and the map information or from the lane mark;
a detection area extraction unit that extracts a detection area image from the lane area image;
A road surface deterioration detection unit that detects road surface deterioration from the detection area image,
The map information includes at least road information including the number of lanes on the road and attribute information of a position on the map;
The detection area extraction unit extracts the detection area image from the lane area image using the vehicle position, the road information, and the information on the vehicle's driving lane,
The road surface deterioration detection device, wherein the road surface deterioration detection unit changes a detection criterion for detecting the road surface deterioration based on the vehicle position and the attribute information. 2. The road surface deterioration detection device according to claim 1, further comprising a detection result information transmission section that transmits information of the road surface deterioration detected by the road surface deterioration detection section to the outside. 3. The vehicle according to claim 1, further comprising a road surface deterioration warning unit that issues an alarm based on the road surface deterioration information detected by the road surface deterioration detection unit and the vehicle driving lane information. Road surface deterioration detector. The road surface deterioration warning unit
an alarm determination unit that determines an alarm notification level based on the road surface deterioration information detected by the road surface deterioration detection unit and the vehicle driving lane information;
4. The road surface deterioration detection device according to claim 3, further comprising an alarm issuing unit that issues an alarm based on the alarm notification level. 5. The road surface deterioration detection device according to claim 4, wherein the warning determination unit outputs the alarm notification level when the road surface deterioration detected by the road surface deterioration detection unit is present in the lane in which the vehicle is traveling. . The alarm transmission unit
an image generation unit that generates an alarm image based on the alarm notification level;
6. The road surface deterioration detection device according to claim 4, further comprising an image output unit that displays the warning image. The alarm transmission unit
an image generator that generates an alarm sound based on the alarm notification level;
7. The road surface deterioration detection device according to any one of claims 4 to 6, further comprising an audio output unit that outputs the warning audio. The alarm transmission unit
8. The road surface deterioration detection device according to any one of claims 4 to 7, further comprising a vibration generator that generates vibration based on the alarm notification level. 9. The road surface deterioration detection device according to claim 1, wherein the lane area extraction unit changes detection criteria for detecting the lane mark based on the attribute information. an imaging step of acquiring an image including the road surface mounted on the own vehicle;
an own vehicle position information acquisition step of acquiring information of the own vehicle position, which is the position of the own vehicle;
a map information obtaining step of obtaining map information of an area including the vehicle position;
a lane area extracting step of detecting lane marks that separate lanes on the road surface from the image, and extracting a lane area image from the image using the lane marks;
a vehicle driving lane determination step of determining a vehicle driving lane in which the vehicle is traveling from the vehicle position and the map information or from the lane mark;
a detection area extracting step of extracting a detection area image from the lane area image;
a road surface deterioration detecting step of detecting road surface deterioration from the detection area image;
The map information includes attribute information of locations on the map and road information including at least the number of lanes of the road,
In the detection area extraction step, the detection area image is extracted from the lane area image using the vehicle position, the road information, and the vehicle driving lane information,
In the road surface deterioration detection step, a road surface deterioration detecting method is characterized in that a detection criterion for detecting the road surface deterioration is changed based on the vehicle position and the attribute information.

Images