US20240034327A1 - Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program - Google Patents
Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program Download PDFInfo
- Publication number
- US20240034327A1 US20240034327A1 US18/378,489 US202318378489A US2024034327A1 US 20240034327 A1 US20240034327 A1 US 20240034327A1 US 202318378489 A US202318378489 A US 202318378489A US 2024034327 A1 US2024034327 A1 US 2024034327A1
- Authority
- US
- United States
- Prior art keywords
- road
- indication
- image
- abnormality
- abnormality detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005856 abnormality Effects 0.000 title claims abstract description 198
- 238000001514 detection method Methods 0.000 title claims abstract description 72
- 230000008439 repair process Effects 0.000 claims description 6
- 238000003384 imaging method Methods 0.000 claims description 5
- 230000004044 response Effects 0.000 claims 11
- 238000000034 method Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 11
- 230000004048 modification Effects 0.000 description 7
- 238000012986 modification Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 3
- 238000005299 abrasion Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
- B60W40/06—Road conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
- B60R11/04—Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2552/00—Input parameters relating to infrastructure
- B60W2552/53—Road markings, e.g. lane marker or crosswalk
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2556/00—Input parameters relating to data
- B60W2556/10—Historical data
Definitions
- the present disclosure relates to a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program.
- Japanese Patent Application Publication No. 2017-20303 discloses a method of capturing an image of a road indication applied onto a road surface obliquely from the above, correcting the captured image into an overhead view through homographic transformation, and determining the state of degradation of the road indication from the corrected image.
- the present disclosure is devised in view of the above, and an object thereof is to provide a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program capable of early detecting abnormality in a road facility.
- a road abnormality detection apparatus including: a memory; and a processor having hardware, wherein the processor is configured to: receive, from a traveling vehicle, a road image indicating an image of the surface of a road or the periphery of the road and image capturing position information indicating a position where the road image is captured; recognize a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determine whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulate, in the memory, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
- the road abnormality detection apparatus can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
- the processor may recognize the road facility included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
- the road abnormality detection apparatus can recognize the road facility included in the road image by comparing the pixel values of the adjacent pixels with each other, such, for example, as brightness values of the pixels when performing image processing on the road image.
- the processor may accumulate, in the memory, the road image including the road facility in which the abnormality exists in addition to the facility position information of the road facility in which the abnormality exists, when it is determined that the abnormality exists in the road facility.
- the road abnormality detection apparatus can examine details of the abnormality arising in the road facility by collecting the image of the road facility in which the abnormality exists in addition to the position of the road facility.
- the processor may transmit the facility position information to a server that a road administrator manages, when at least a predetermined quantity of the facility position information regarding the identical road facility has been accumulated in the memory.
- the road abnormality detection apparatus can notify the road administrator side of the position of the road facility in which abnormality exists, and thereby, the road facility can be early repaired.
- the road facility may include a road indication presented on the road and a structure provided on the road or around the road.
- the road abnormality detection apparatus can early detect the abnormality arising in the road indication presented on the road and the structure provided on the road or around the road.
- a road abnormality detection method which is a road abnormality detection method performed by a road abnormality detection apparatus including a processor.
- the road abnormality detection method includes: receiving, by the processor, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognizing, by the processor, a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determining, by the processor, whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulating, by the processor, in a memory of the road abnormality detection apparatus, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
- the road abnormality detection method can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
- a road abnormality detection program which is a road abnormality detection program executed by a road abnormality detection apparatus, the program comprising: receiving, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognizing a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determining whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulating, in a memory of the road abnormality detection apparatus, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
- the road abnormality detection program can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
- abnormality in a road facility can be early detected based on a road image collected from a traveling vehicle.
- FIG. 1 is a block diagram schematically showing a road abnormality detection system to which a road abnormality detection apparatus according to an embodiment of the present disclosure can be applied;
- FIG. 2 is a schematic diagram exemplarily showing road facilities which are targets of the road abnormality detection apparatus according to the embodiment of the present disclosure
- FIG. 3 is a flowchart showing a processing procedure of a road abnormality detection method according to the embodiment of the present disclosure.
- FIG. 4 is a flowchart showing a modification of the processing procedure of the road abnormality detection method according to the embodiment of the present disclosure.
- a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program according to an embodiment are described with reference to the drawings.
- constituents in the embodiment below may include those that can be easily replaced by a person skilled in the art, or those that are substantially identical to these.
- the road abnormality detection system has a processing server 1 , at least one vehicle 2 and a road administrator server 3 .
- the road abnormality detection apparatus according to the present embodiment is configured, specifically, of the processing server 1 .
- the at least one vehicle 2 may be one vehicle 2 or a plurality of vehicles 2 traveling on a road.
- the processing server 1 , the vehicle 2 and the road administrator server 3 can communicate with one another via a network NW.
- the network NW is configured, for example, of any of the Internet network, a mobile phone network and the like.
- the processing server 1 processes an image received from the vehicle 2 (road image) and includes a control unit 11 , a communication unit 12 and a storage unit 13 .
- control unit 11 includes a processor composed of a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA) and the like, and a memory composed of a random access memory (RAM), a read-only memory (ROM) and the like (main storage unit), these omitted from the figure.
- CPU central processing unit
- DSP digital signal processor
- FPGA field-programmable gate array
- main storage unit main storage unit
- the control unit 11 loads a program stored in the storage unit 13 into a working area of the main storage unit to execute it, controls components and the like through the execution of the program, and thereby, realizes functions which meet predetermined purposes.
- the control unit 11 functions as an image recognition unit 111 through the execution of the program. Notably, details of the image recognition unit 111 are mentioned later.
- the communication unit 12 is configured, for example, of a local area network (LAN) interface board, a wireless communication circuit for wireless communication, and the like.
- the communication unit 12 is connected to the network NW such as the Internet which is a public communication network.
- the communication unit 12 communicates with the processing server 1 and the road administrator server 3 through the connection to the network NW.
- the storage unit 13 is configured of an erasable programmable ROM (EPROM), a hard disk drive (HDD), and a recording medium such as a removable medium.
- EPROM erasable programmable ROM
- HDD hard disk drive
- a recording medium such as a removable medium.
- the removable medium include a universal serial bus (USB) memory, and disc recording media such as a compact disc (CD), a digital versatile disc (DVD) and a Blu-ray® disc (BD).
- the storage unit 13 can store an operating system (OS), various programs, various tables, various databases and the like.
- OS operating system
- the storage unit 13 includes a correctness information DB 131 and an abnormality information DB 132 .
- These databases (DB s) are built by a program for a database management system (DBMS) being executed by the processor above and managing data stored in the storage unit 13 .
- DBMS database management system
- the correctness information DB 131 is configured, for example, of a relational database in which a plurality of pieces of correctness information are retrievably stored.
- the correctness information is information as a standard for determining whether or not abnormality exists in a road facility included in a road image transmitted from the vehicle 2 .
- road image denotes an image obtained by imaging a surface of a road or a periphery of the road.
- road facility includes road indications presented on the road and structures provided on the road and around the road.
- Examples of the “road indications” above include a center line Rs 1 composed of a solid line, center lines Rs 2 composed of broken lines, outer lines Rs 3 , a stop line Rs 4 , a pedestrian crossing Rs 5 , pedestrian crossing advance notices Rs 6 , a speed limit indication Rs 7 , a right turn arrow Rs 8 and the like as shown in FIG. 2 , and road division lines and the like such as Botts' dots provided on roads in foreign countries.
- Examples of the “structures” above include guardrails Rs 9 provided around the road, and road signs such as an information sign, a warning sign, a regulatory sign and a direction sign.
- road signs such as an information sign, a warning sign, a regulatory sign and a direction sign.
- road indications and structures include all the things installed on roads or around the roads in conformity with traffic regulations and the like.
- the correctness information stored in the correctness information DB 131 is exemplarily information including types of road facilities which are to be at the site of the vehicle, and shapes and the like of the road facilities, and is stored therein as map information and various kinds of information such as image data. These pieces of correctness information are built as the correctness information DB 131 by the control unit 11 externally collecting those via the network NW and/or an operator manually inputting those. Notably, how to collect the correctness information is not limited to any of these methods for collecting the same.
- the abnormality information DB 132 is configured, for example, of a relational database in which a plurality of pieces of abnormality information are retrievably stored.
- the abnormality information is information regarding a road facility in which abnormality exists, and as mentioned later, for example, includes facility position information indicating a position of a road facility in which abnormality exists, a road image including the road facility, and the cause of the abnormality of the road facility.
- the vehicle 2 is a movable body externally communicable and includes a control unit 21 , a communication unit 22 , a storage unit 23 , a positioning unit 24 and an image capturing unit 25 .
- the control unit 21 and the storage unit 23 are physically similar to the control unit 11 and the storage unit 13 above, respectively.
- the control unit 21 integrally controls operation of various components mounted on the vehicle 2 . Moreover, the control unit 21 stores, in the storage unit 23 , positional information of the vehicle 2 input from the positioning unit 24 in association with a road image input from the image capturing unit 25 . Namely, the control unit 21 stores, in the storage unit 23 , image capturing position information indicating a position where the road image is captured by the image capturing unit 25 in association with the road image thus captured.
- the communication unit 22 is configured, for example, of a data communication module (DCM) and the like, and communicates with the processing server 1 through wireless communication via the network NW.
- the storage unit 23 includes a road image DB 231 .
- the road image DB 231 is exemplarily a relational database (RDB) in which road images captured by the image capturing unit 25 are retrievably stored. Notably, these road images are associated with the image capturing position information.
- RDB relational database
- the positioning unit 24 receives electric waves from global positioning system (GPS) satellites (not shown) to detect a current position of the vehicle 2 . Then, the positioning unit 24 outputs information regarding the detected current position of the vehicle 2 (hereinafter called “vehicle position information”) to the control unit 21 .
- GPS global positioning system
- vehicle position information information regarding the detected current position of the vehicle 2
- the image capturing unit 25 is an image capturing camera or the like, for example, attached to a front portion of a body of the vehicle 2 , captures a road image frontward of the vehicle 2 while the vehicle 2 is traveling, and outputs the captured road image to the control unit 21 .
- the “road image” may be any of still image data and moving image data, or may be moving image data generated using a plurality of chronologically continuous still image data.
- the road administrator server 3 is a server managed by a road administrator.
- the road administrator is an entity liable to repair of the road facilities, and examples thereof include local public entities such as the twenty-three wards of Tokyo, prefectures, cities, towns and villages, the Public Safety Commission, and the like.
- the vehicle 2 captures an image of a road in its traveling with the image capturing unit 25 (step S 1 ). Subsequently, the vehicle 2 transmits the captured road image and the image capturing position information of the road image to the processing server 1 periodically in predetermined timing (step S 2 ).
- predetermined timing for example, various kinds of timing such as periodical timing at predetermined time intervals, and timing when the vehicle 2 passes through predetermined positions.
- the processing server 1 performs image recognition processing on the road image with the image recognition unit 111 (step S 3 ).
- the image recognition unit 111 recognizes a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information using a known image recognition processing technique.
- position of the road facility denotes coordinates (longitude; latitude) where the road facility is installed.
- the image recognition unit 111 compares pixel values of adjacent pixels in the road image with one another, such, for example, as brightness values of the pixels, and thereby, recognizes a type of the road facility included in the road image. Moreover, the image recognition unit 111 recognizes the position of the road facility included in the road image, for example, by a method of estimating a depth distance to that road facility or the similar method.
- the image recognition unit 111 determines whether or not abnormality exists in the road facility (step S 4 ). Specifically, the image recognition unit 111 determines whether or not abnormality exists in the road facility included in the road image by comparing the road facility and the correctness information which is prepared in advance and stored in the correctness information DB 131 with each other.
- step S 4 When it is determined in step S 4 that no abnormality exists in the road facility (No in step S 4 ), the image recognition unit 111 returns the process to step S 3 . On the other hand, when it is determined in step S 4 that abnormality exists in the road facility (Yes in step S 4 ), the image recognition unit 111 identifies the cause of the abnormality (step S 5 ).
- abnormality in road facilities There can be herein considered as abnormality in road facilities, breakage, abrasion, spots due to dirt, deposits of withered leaves, litter or the like, and the like.
- the road facility is the center line Rs 1 drawn in the center of the road (see FIG. 2 ).
- the image recognition unit 111 identifies a reason for the abnormality, for example, as any of breakage, abrasion, spots due to dirt, and withered leaves, litter or the like using a known image recognition processing technique.
- the control unit 11 of the processing server 1 accumulates data of an image recognition processing result (step S 6 ).
- the control unit 11 stores, in the abnormality information DB 132 of the storage unit 13 , facility position information at least indicating a position of the road facility in which the abnormality exists as abnormality information.
- the control unit 11 may store, in the abnormality information DB 132 , the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility as the abnormality information in addition to the facility position information of the road facility in which the abnormality exists.
- the control unit 11 notifies the road administrator that the abnormality arises in the road facility (step S 7 ). Specifically, the control unit 11 notifies the road administrator that the abnormality arises in the road facility by transmitting the abnormality information accumulated in the abnormality information DB 132 to the road administrator server 3 .
- the abnormality information transmitted to the road administrator server 3 at least includes the facility position information of the road facility in which the abnormality exists, and may include the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility.
- control unit 11 may transmit the abnormality information to the road administrator server 3 when a predetermined quantity of abnormality information regarding the identical road facility has been accumulated in the abnormality information DB 132 .
- the control unit 11 may notify the road administrator that the abnormality arises in the road facility only when the image recognition processing (see step S 4 ) is performed on road images including the identical road facility which are captured, for example, by a plurality of vehicles 2 and “existence of abnormality” is determined for all of those. Thereby, the road administrator can be notified that the abnormality arises in the road facility only in a case of a high possibility that the abnormality arises in the road facility.
- the road abnormality detection method and the road abnormality detection program according to the embodiment as described above it can early detect whether or not abnormality exists in a road facility by performing image recognition processing on a road image received from the traveling vehicle 2 .
- the road abnormality detection method and the road abnormality detection program according to the embodiment since road images can be collected from the plurality of vehicles 2 and it can be determined whether or not abnormality exists in road facilities included in the road images, it can early detect whether or not abnormality exists in the road facilities widely installed.
- abnormality information of road facilities can be sold to a road administrator in charge of repair of roads (any of local public entities, the Public Safety Commission, and the like).
- the vehicle 2 side While in the road abnormality detection method described with FIG. 3 , the vehicle 2 side only captures road images and the processing server 1 side performs image recognition processing thereon, the vehicle 2 side can also perform the image recognition processing.
- a modification of the road abnormality detection method according to the embodiment is described with reference to FIG. 4 .
- the contents in steps S 1 , S 6 and S 7 in the figure are similar to the contents in steps S 1 , S 6 and S 7 of the road abnormality detection method described with FIG. 3 and their description is omitted.
- step S 13 the control unit 21 of the vehicle 2 performs image recognition processing on the road image.
- the control unit 21 recognizes a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information using a known image recognition processing technique.
- control unit 21 compares pixel values of adjacent pixels in the road image with one another, such, for example, as brightness values of the pixels, and thereby, recognizes a type of the road facility included in the road image. Moreover, the control unit 21 recognizes the position of the road facility included in the road image, for example, by a method of estimating a depth distance to that road facility or the similar method.
- the control unit 21 determines whether or not abnormality exists in the road facility (step S 14 ). Specifically, the control unit 21 determines whether or not abnormality exists in the road facility included in the road image based on the road facility and the correctness information which is prepared in advance. Notably, the correctness information above may be stored in the storage unit 23 of the vehicle 2 in advance or may be acquired every time from the correctness information DB 131 of the processing server 1 .
- step S 14 When it is determined in step S 14 that no abnormality exists in the road facility (No in step S 14 ), the control unit 21 returns the process to step S 13 . On the other hand, when it is determined in step S 14 that abnormality exists in the road facility (Yes in step S 14 ), the control unit 21 identifies the cause of the abnormality (step S 15 ).
- control unit 21 transmits data of an image recognition processing result to the processing server 1 (step S 16 ).
- the control unit 21 transmits the facility position information at least indicating a position of the road facility in which the abnormality exists as the abnormality information to the processing server 1 .
- the control unit 21 may transmit the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility as the abnormality information to the processing server 1 in addition to the facility position information of the road facility in which the abnormality exists.
- load on the processing server 1 side can be reduced by the vehicle 2 side performing image recognition processing.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Mechanical Engineering (AREA)
- Automation & Control Theory (AREA)
- Mathematical Physics (AREA)
- Transportation (AREA)
- Traffic Control Systems (AREA)
- Road Repair (AREA)
- Image Analysis (AREA)
Abstract
A road abnormality detection apparatus includes: a memory; and a processor having hardware, wherein the processor is configured to: receive, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognize a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determine whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance; and accumulate, in the memory, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
Description
- This application is a continuation application of U.S. patent application Ser. No. 17/961,296, filed Oct. 6, 2022, which is a continuation application of U.S. patent application Ser. No. 16/812,483, filed on Mar. 9, 2020, which is based on and claims priority under 35 U.S.C. 119 from Japanese Patent Application No. 2019-070776 filed on Apr. 2, 2019. The entire contents of the above applications are incorporated herein by reference.
- The present disclosure relates to a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program.
- Japanese Patent Application Publication No. 2017-20303 discloses a method of capturing an image of a road indication applied onto a road surface obliquely from the above, correcting the captured image into an overhead view through homographic transformation, and determining the state of degradation of the road indication from the corrected image.
- For example, in automated driving or traveling during which white lines and the like need to be detected (for example, single lane traveling), accuracy of the traveling lowers when some white lines are faint or fade away. Therefore, when abnormality exists in a road facility such as a white line, the abnormality needs to be early detected to repair the road facility. It has been difficult to early detect abnormality in a road facility with a technology as disclosed in JP 2017-020303 A.
- The present disclosure is devised in view of the above, and an object thereof is to provide a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program capable of early detecting abnormality in a road facility.
- In order to solve the problem above and achieve the object, there is provided a road abnormality detection apparatus according to the present disclosure, including: a memory; and a processor having hardware, wherein the processor is configured to: receive, from a traveling vehicle, a road image indicating an image of the surface of a road or the periphery of the road and image capturing position information indicating a position where the road image is captured; recognize a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determine whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulate, in the memory, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
- Thereby, the road abnormality detection apparatus can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
- Moreover, as to the road abnormality detection apparatus according to the present disclosure, in the disclosure above, the processor may recognize the road facility included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
- Thereby, the road abnormality detection apparatus can recognize the road facility included in the road image by comparing the pixel values of the adjacent pixels with each other, such, for example, as brightness values of the pixels when performing image processing on the road image.
- Moreover, as to the road abnormality detection apparatus according to the present disclosure, in the disclosure above, the processor may accumulate, in the memory, the road image including the road facility in which the abnormality exists in addition to the facility position information of the road facility in which the abnormality exists, when it is determined that the abnormality exists in the road facility.
- Thereby, the road abnormality detection apparatus can examine details of the abnormality arising in the road facility by collecting the image of the road facility in which the abnormality exists in addition to the position of the road facility.
- Moreover, as to the road abnormality detection apparatus according to the present disclosure, in the disclosure above, the processor may transmit the facility position information to a server that a road administrator manages, when at least a predetermined quantity of the facility position information regarding the identical road facility has been accumulated in the memory.
- Thereby, the road abnormality detection apparatus can notify the road administrator side of the position of the road facility in which abnormality exists, and thereby, the road facility can be early repaired.
- Moreover, as to the road abnormality detection apparatus according to the present disclosure, in the disclosure above, the road facility may include a road indication presented on the road and a structure provided on the road or around the road.
- Thereby, the road abnormality detection apparatus can early detect the abnormality arising in the road indication presented on the road and the structure provided on the road or around the road.
- In order to solve the problem above and achieve the object, there is provided a road abnormality detection method according to the present disclosure which is a road abnormality detection method performed by a road abnormality detection apparatus including a processor. The road abnormality detection method includes: receiving, by the processor, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognizing, by the processor, a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determining, by the processor, whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulating, by the processor, in a memory of the road abnormality detection apparatus, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
- Thereby, the road abnormality detection method can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
- In order to solve the problem above and achieve the object, there is provided a road abnormality detection program according to the present disclosure which is a road abnormality detection program executed by a road abnormality detection apparatus, the program comprising: receiving, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognizing a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determining whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulating, in a memory of the road abnormality detection apparatus, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
- Thereby, the road abnormality detection program can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
- According to the present disclosure, abnormality in a road facility can be early detected based on a road image collected from a traveling vehicle.
- Features, advantages, and technical and industrial significance of exemplary embodiments of the disclosure will be described below with reference to the accompanying drawings, in which like numerals denote like elements, and wherein:
-
FIG. 1 is a block diagram schematically showing a road abnormality detection system to which a road abnormality detection apparatus according to an embodiment of the present disclosure can be applied; -
FIG. 2 is a schematic diagram exemplarily showing road facilities which are targets of the road abnormality detection apparatus according to the embodiment of the present disclosure; -
FIG. 3 is a flowchart showing a processing procedure of a road abnormality detection method according to the embodiment of the present disclosure; and -
FIG. 4 is a flowchart showing a modification of the processing procedure of the road abnormality detection method according to the embodiment of the present disclosure. - A road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program according to an embodiment are described with reference to the drawings. Notably, constituents in the embodiment below may include those that can be easily replaced by a person skilled in the art, or those that are substantially identical to these.
- Road Abnormality Detection System
- First, a road abnormality detection system to which a road abnormality detection apparatus according to an embodiment can be applied is described with reference to
FIG. 1 . The road abnormality detection system has aprocessing server 1, at least onevehicle 2 and aroad administrator server 3. The road abnormality detection apparatus according to the present embodiment is configured, specifically, of theprocessing server 1. Moreover, the at least onevehicle 2 may be onevehicle 2 or a plurality ofvehicles 2 traveling on a road. - The
processing server 1, thevehicle 2 and theroad administrator server 3 can communicate with one another via a network NW. The network NW is configured, for example, of any of the Internet network, a mobile phone network and the like. - Processing Server
- Next, a configuration of the
processing server 1 is described with reference toFIG. 1 andFIG. 2 . Theprocessing server 1 processes an image received from the vehicle 2 (road image) and includes a control unit 11, acommunication unit 12 and astorage unit 13. - Specifically, the control unit 11 includes a processor composed of a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA) and the like, and a memory composed of a random access memory (RAM), a read-only memory (ROM) and the like (main storage unit), these omitted from the figure.
- The control unit 11 loads a program stored in the
storage unit 13 into a working area of the main storage unit to execute it, controls components and the like through the execution of the program, and thereby, realizes functions which meet predetermined purposes. The control unit 11 functions as an image recognition unit 111 through the execution of the program. Notably, details of the image recognition unit 111 are mentioned later. - The
communication unit 12 is configured, for example, of a local area network (LAN) interface board, a wireless communication circuit for wireless communication, and the like. Thecommunication unit 12 is connected to the network NW such as the Internet which is a public communication network. Thecommunication unit 12 communicates with theprocessing server 1 and theroad administrator server 3 through the connection to the network NW. - The
storage unit 13 is configured of an erasable programmable ROM (EPROM), a hard disk drive (HDD), and a recording medium such as a removable medium. Examples of the removable medium include a universal serial bus (USB) memory, and disc recording media such as a compact disc (CD), a digital versatile disc (DVD) and a Blu-ray® disc (BD). Moreover, thestorage unit 13 can store an operating system (OS), various programs, various tables, various databases and the like. - The
storage unit 13 includes acorrectness information DB 131 and anabnormality information DB 132. These databases (DB s) are built by a program for a database management system (DBMS) being executed by the processor above and managing data stored in thestorage unit 13. - The
correctness information DB 131 is configured, for example, of a relational database in which a plurality of pieces of correctness information are retrievably stored. The correctness information is information as a standard for determining whether or not abnormality exists in a road facility included in a road image transmitted from thevehicle 2. - Herein, the term “road image” denotes an image obtained by imaging a surface of a road or a periphery of the road. Moreover, “road facility” includes road indications presented on the road and structures provided on the road and around the road.
- Examples of the “road indications” above include a center line Rs1 composed of a solid line, center lines Rs2 composed of broken lines, outer lines Rs3, a stop line Rs4, a pedestrian crossing Rs5, pedestrian crossing advance notices Rs6, a speed limit indication Rs7, a right turn arrow Rs8 and the like as shown in
FIG. 2 , and road division lines and the like such as Botts' dots provided on roads in foreign countries. - Examples of the “structures” above include guardrails Rs9 provided around the road, and road signs such as an information sign, a warning sign, a regulatory sign and a direction sign. Notably, not limited to those shown in the figure, such road indications and structures include all the things installed on roads or around the roads in conformity with traffic regulations and the like.
- The correctness information stored in the
correctness information DB 131 is exemplarily information including types of road facilities which are to be at the site of the vehicle, and shapes and the like of the road facilities, and is stored therein as map information and various kinds of information such as image data. These pieces of correctness information are built as thecorrectness information DB 131 by the control unit 11 externally collecting those via the network NW and/or an operator manually inputting those. Notably, how to collect the correctness information is not limited to any of these methods for collecting the same. - The
abnormality information DB 132 is configured, for example, of a relational database in which a plurality of pieces of abnormality information are retrievably stored. The abnormality information is information regarding a road facility in which abnormality exists, and as mentioned later, for example, includes facility position information indicating a position of a road facility in which abnormality exists, a road image including the road facility, and the cause of the abnormality of the road facility. - Vehicle
- Next, a configuration of the
vehicle 2 is described with reference toFIG. 1 . Thevehicle 2 is a movable body externally communicable and includes acontrol unit 21, a communication unit 22, astorage unit 23, apositioning unit 24 and animage capturing unit 25. Thecontrol unit 21 and thestorage unit 23 are physically similar to the control unit 11 and thestorage unit 13 above, respectively. - The
control unit 21 integrally controls operation of various components mounted on thevehicle 2. Moreover, thecontrol unit 21 stores, in thestorage unit 23, positional information of thevehicle 2 input from thepositioning unit 24 in association with a road image input from theimage capturing unit 25. Namely, thecontrol unit 21 stores, in thestorage unit 23, image capturing position information indicating a position where the road image is captured by theimage capturing unit 25 in association with the road image thus captured. - The communication unit 22 is configured, for example, of a data communication module (DCM) and the like, and communicates with the
processing server 1 through wireless communication via the network NW. Thestorage unit 23 includes aroad image DB 231. Theroad image DB 231 is exemplarily a relational database (RDB) in which road images captured by theimage capturing unit 25 are retrievably stored. Notably, these road images are associated with the image capturing position information. - The
positioning unit 24 receives electric waves from global positioning system (GPS) satellites (not shown) to detect a current position of thevehicle 2. Then, thepositioning unit 24 outputs information regarding the detected current position of the vehicle 2 (hereinafter called “vehicle position information”) to thecontrol unit 21. Notably, there can also be used as a method of detecting the current position of the vehicle 2 a method, for example, having light detection and ranging (LiDAR, laser imaging detection and ranging) and a three-dimensional digital map combined. - The
image capturing unit 25 is an image capturing camera or the like, for example, attached to a front portion of a body of thevehicle 2, captures a road image frontward of thevehicle 2 while thevehicle 2 is traveling, and outputs the captured road image to thecontrol unit 21. Notably, the “road image” may be any of still image data and moving image data, or may be moving image data generated using a plurality of chronologically continuous still image data. - Road Administrator Server
- Next, a configuration of the
road administrator server 3 is described with reference toFIG. 1 . Theroad administrator server 3 is a server managed by a road administrator. The road administrator is an entity liable to repair of the road facilities, and examples thereof include local public entities such as the twenty-three wards of Tokyo, prefectures, cities, towns and villages, the Public Safety Commission, and the like. - Road Abnormality Detection Method
- Next, a road abnormality detection method performed by the road abnormality detection apparatus is described with reference to
FIG. 3 . - First, the
vehicle 2 captures an image of a road in its traveling with the image capturing unit 25 (step S1). Subsequently, thevehicle 2 transmits the captured road image and the image capturing position information of the road image to theprocessing server 1 periodically in predetermined timing (step S2). Notably, there can be set as such predetermined timing, for example, various kinds of timing such as periodical timing at predetermined time intervals, and timing when thevehicle 2 passes through predetermined positions. - Subsequently, after receiving the road image and the image capturing position information from the
vehicle 2, theprocessing server 1 performs image recognition processing on the road image with the image recognition unit 111 (step S3). The image recognition unit 111 recognizes a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information using a known image recognition processing technique. Notably, the term “position of the road facility” denotes coordinates (longitude; latitude) where the road facility is installed. - Specifically, the image recognition unit 111 compares pixel values of adjacent pixels in the road image with one another, such, for example, as brightness values of the pixels, and thereby, recognizes a type of the road facility included in the road image. Moreover, the image recognition unit 111 recognizes the position of the road facility included in the road image, for example, by a method of estimating a depth distance to that road facility or the similar method.
- Subsequently, the image recognition unit 111 determines whether or not abnormality exists in the road facility (step S4). Specifically, the image recognition unit 111 determines whether or not abnormality exists in the road facility included in the road image by comparing the road facility and the correctness information which is prepared in advance and stored in the
correctness information DB 131 with each other. - When it is determined in step S4 that no abnormality exists in the road facility (No in step S4), the image recognition unit 111 returns the process to step S3. On the other hand, when it is determined in step S4 that abnormality exists in the road facility (Yes in step S4), the image recognition unit 111 identifies the cause of the abnormality (step S5).
- There can be herein considered as abnormality in road facilities, breakage, abrasion, spots due to dirt, deposits of withered leaves, litter or the like, and the like. It is herein supposed, for example, that the road facility is the center line Rs1 drawn in the center of the road (see
FIG. 2 ). In this case, when the center line Rs1 is damaged and “existence of abnormality” is determined in step S4, the image recognition unit 111 identifies a reason for the abnormality, for example, as any of breakage, abrasion, spots due to dirt, and withered leaves, litter or the like using a known image recognition processing technique. - Subsequently, the control unit 11 of the
processing server 1 accumulates data of an image recognition processing result (step S6). In this step, the control unit 11 stores, in theabnormality information DB 132 of thestorage unit 13, facility position information at least indicating a position of the road facility in which the abnormality exists as abnormality information. Moreover, in this step, the control unit 11 may store, in theabnormality information DB 132, the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility as the abnormality information in addition to the facility position information of the road facility in which the abnormality exists. As above, by collecting the image of the road facility in which the abnormality exists and the cause of the abnormality in addition to the position of the road facility, there can be examined details of the abnormality arising in the road facility. - Subsequently, the control unit 11 notifies the road administrator that the abnormality arises in the road facility (step S7). Specifically, the control unit 11 notifies the road administrator that the abnormality arises in the road facility by transmitting the abnormality information accumulated in the
abnormality information DB 132 to theroad administrator server 3. Notably, the abnormality information transmitted to theroad administrator server 3 at least includes the facility position information of the road facility in which the abnormality exists, and may include the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility. By notifying the road administrator side of the position of the road facility in which the abnormality exists as above, the road facility can be early repaired. - Notably, the control unit 11 may transmit the abnormality information to the
road administrator server 3 when a predetermined quantity of abnormality information regarding the identical road facility has been accumulated in theabnormality information DB 132. Namely, the control unit 11 may notify the road administrator that the abnormality arises in the road facility only when the image recognition processing (see step S4) is performed on road images including the identical road facility which are captured, for example, by a plurality ofvehicles 2 and “existence of abnormality” is determined for all of those. Thereby, the road administrator can be notified that the abnormality arises in the road facility only in a case of a high possibility that the abnormality arises in the road facility. - According to the road abnormality detection apparatus, the road abnormality detection method and the road abnormality detection program according to the embodiment as described above, it can early detect whether or not abnormality exists in a road facility by performing image recognition processing on a road image received from the traveling
vehicle 2. Moreover, according to the road abnormality detection apparatus, the road abnormality detection method and the road abnormality detection program according to the embodiment, since road images can be collected from the plurality ofvehicles 2 and it can be determined whether or not abnormality exists in road facilities included in the road images, it can early detect whether or not abnormality exists in the road facilities widely installed. Moreover, according to the road abnormality detection apparatus, the road abnormality detection method and the road abnormality detection program according to the embodiment, for example, abnormality information of road facilities can be sold to a road administrator in charge of repair of roads (any of local public entities, the Public Safety Commission, and the like). - Modification
- While in the road abnormality detection method described with
FIG. 3 , thevehicle 2 side only captures road images and theprocessing server 1 side performs image recognition processing thereon, thevehicle 2 side can also perform the image recognition processing. Hereafter, a modification of the road abnormality detection method according to the embodiment is described with reference toFIG. 4 . Notably, the contents in steps S1, S6 and S7 in the figure are similar to the contents in steps S1, S6 and S7 of the road abnormality detection method described withFIG. 3 and their description is omitted. - Subsequently to step S1, the
control unit 21 of thevehicle 2 performs image recognition processing on the road image (step S13). Thecontrol unit 21 recognizes a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information using a known image recognition processing technique. - Specifically, the
control unit 21 compares pixel values of adjacent pixels in the road image with one another, such, for example, as brightness values of the pixels, and thereby, recognizes a type of the road facility included in the road image. Moreover, thecontrol unit 21 recognizes the position of the road facility included in the road image, for example, by a method of estimating a depth distance to that road facility or the similar method. - Subsequently, the
control unit 21 determines whether or not abnormality exists in the road facility (step S14). Specifically, thecontrol unit 21 determines whether or not abnormality exists in the road facility included in the road image based on the road facility and the correctness information which is prepared in advance. Notably, the correctness information above may be stored in thestorage unit 23 of thevehicle 2 in advance or may be acquired every time from thecorrectness information DB 131 of theprocessing server 1. - When it is determined in step S14 that no abnormality exists in the road facility (No in step S14), the
control unit 21 returns the process to step S13. On the other hand, when it is determined in step S14 that abnormality exists in the road facility (Yes in step S14), thecontrol unit 21 identifies the cause of the abnormality (step S15). - Subsequently, the
control unit 21 transmits data of an image recognition processing result to the processing server 1 (step S16). In this step, thecontrol unit 21 transmits the facility position information at least indicating a position of the road facility in which the abnormality exists as the abnormality information to theprocessing server 1. Moreover, in this step, thecontrol unit 21 may transmit the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility as the abnormality information to theprocessing server 1 in addition to the facility position information of the road facility in which the abnormality exists. - According to the modification as to the road abnormality detection apparatus, the road abnormality detection method and the road abnormality detection program according to the embodiment as described above, load on the
processing server 1 side can be reduced by thevehicle 2 side performing image recognition processing. - As above, a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program according to the present disclosure have been described specifically with a mode for carrying out the disclosure. The scope of the present disclosure is not limited by such description but should be broadly construed based on the disclosure in the appended claims. It is needless to say that the scope of the present disclosure includes various alterations, modifications and the like based on such description.
- Further effects and modifications can be easily derived by those skilled in the art. Thus, the broader aspects of the present disclosure are not limited to the specific details and representative embodiment shown and described above. Accordingly, various modifications can be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.
Claims (20)
1. A road abnormality detection apparatus comprising:
a memory; and
a hardware processor programmed to:
receive, from a traveling vehicle, (i) a road image indicating an image of at least a surface of a road, the road image being captured by a camera attached to the vehicle and (ii) image capturing position information indicating a position where the road image is captured, the position being detected by a positioning unit of the vehicle using GPS data or laser imaging data;
recognize a road indication presented on the surface of the road included in the road image and a position of the road indication based on the road image and the image capturing position information;
determine that an abnormality exists in the road indication by using the road image, the abnormality in the road indication including spots due to dirt, deposits of withered leaves, and litter, wherein
the road indication includes at least one of a center line on the surface of the road, an outer line on the surface of the road, a stop line on the surface of the road, a pedestrian crossing indication on the surface of the road, a pedestrian crossing advance notice on the surface of the road, a speed limit indication on the surface of the road, an arrow line on the surface of the road, Bott's dots, and a road division line on the surface of the road; and
in response to determining that the abnormality exists in the same road indication a plurality of times based on a plurality of the road images captured by a plurality of the vehicles, transmit indication position information for repair of the road indication.
2. The road abnormality detection apparatus according to claim 1 , wherein the processor recognizes the road indication included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
3. The road abnormality detection apparatus according to claim 1 , wherein the processor accumulates, in the memory, the indication position information in response to determining that the abnormality exists in the road indication.
4. The road abnormality detection apparatus according to claim 3 , wherein the processor accumulates, in the memory, the road image including the road indication in which the abnormality exists in addition to the indication position information of the road indication in which the abnormality exists, in response to determining that the abnormality exists in the road indication.
5. The road abnormality detection apparatus according to claim 1 , wherein
the processor determines that the abnormality exists in the road indication by comparing the road indication included in the road image with correctness information that is prepared in advance.
6. The road abnormality detection apparatus according to claim 5 , wherein
the correctness information includes map information having road indication information prepared in advance.
7. A road abnormality detection method performed by a road abnormality detection apparatus including a memory and a hardware processor, the road abnormality detection method comprising:
receiving, by the processor, from a traveling vehicle, (i) a road image indicating an image of at least a surface of a road, the road image being captured by a camera attached to the vehicle and (ii) image capturing position information indicating a position where the road image is captured, the position being detected by a positioning unit of the vehicle using GPS data or laser imaging data;
recognizing, by the processor, a road indication presented on the surface of the road included in the road image and a position of the road indication based on the road image and the image capturing position information;
determining, by the processor, that an abnormality exists in the road indication by using the road image, the abnormality in the road indication including spots due to dirt, deposits of withered leaves, and litter, wherein
the road indication includes at least one of a center line on the surface of the road, an outer line on the surface of the road, a stop line on the surface of the road, a pedestrian crossing indication on the surface of the road, a pedestrian crossing advance notice on the surface of the road, a speed limit indication on the surface of the road, an arrow line on the surface of the road, Bott's dots, and a road division line on the surface of the road; and
in response to determining that the abnormality exists in the same road indication a plurality of times based on a plurality of the road images captured by a plurality of the vehicles, transmitting, by the processor, indication position information for repair of the road indication.
8. The road abnormality detection method according to claim 7 , wherein the processor recognizes the road indication included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
9. The road abnormality detection method according to claim 7 , wherein the processor accumulates, in the memory, the indication position information in response to determining that the abnormality exists in the road indication.
10. The road abnormality detection method according to claim 9 , wherein the processor accumulates, in the memory, the road image including the road indication in which the abnormality exists in addition to the indication position information of the road indication in which the abnormality exists, in response to determining that the abnormality exists in the road indication.
11. The road abnormality detection method according to claim 7 , wherein
the processor determines that the abnormality exists in the road indication by comparing the road indication included in the road image with correctness information that is prepared in advance.
12. The road abnormality detection method according to claim 11 , wherein
the correctness information includes map information having road indication information prepared in advance.
13. A non-transitory computer-readable medium storing thereon a road abnormality detection program executed by a road abnormality detection apparatus including a memory and a hardware processor, the program causing the processor to execute:
receiving, from a traveling vehicle, (i) a road image indicating an image of at least a surface of a road, the road image being captured by a camera attached to the vehicle and (ii) image capturing position information indicating a position where the road image is captured, the position being detected by a positioning unit of the vehicle using GPS data or laser imaging data;
recognizing a road indication presented on the surface of the road included in the road image and a position of the road indication based on the road image and the image capturing position information;
determining that an abnormality exists in the road indication by using the road image, the abnormality in the road indication including spots due to dirt, deposits of withered leaves, and litter, wherein
the road indication includes at least one of a center line on the surface of the road, an outer line on the surface of the road, a stop line on the surface of the road, a pedestrian crossing indication on the surface of the road, a pedestrian crossing advance notice on the surface of the road, a speed limit indication on the surface of the road, an arrow line on the surface of the road, Bott's dots, and a road division line on the surface of the road; and
in response to determining that the abnormality exists in the same road indication a plurality of times based on a plurality of the road images captured by a plurality of the vehicles, transmitting indication position information for repair of the road indication.
14. The non-transitory computer-readable medium according to claim 13 , wherein the program causes the processor to recognize the road indication included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
15. The non-transitory computer-readable medium according to claim 13 , wherein the program causes the processor to accumulate, in the memory, the indication position information in response to determining that the abnormality exists in the road indication.
16. The non-transitory computer-readable medium according to claim 15 , wherein the program causes the processor to accumulate, in the memory, the road image including the road indication in which the abnormality exists in addition to the indication position information of the road indication in which the abnormality exists, in response to determining that the abnormality exists in the road indication.
17. The non-transitory computer-readable medium according to claim 13 , wherein the program causes the processor to determine that the abnormality exists in the road indication by comparing the road indication included in the road image with correctness information that is prepared in advance.
18. The non-transitory computer-readable medium according to claim 17 , wherein
the correctness information includes map information having road indication information prepared in advance.
19. The road abnormality detection apparatus according to claim 1 , wherein in response to determining that the abnormality exists in the road indication, the processor is programmed to identify a cause of the abnormality and transmit the cause of the abnormality.
20. The road abnormality detection method according to claim 7 , wherein in response to determining that the abnormality exists in the road indication, identifying, by the processor, a cause of the abnormality and transmitting, by the processor, the cause of the abnormality.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/378,489 US20240034327A1 (en) | 2019-04-02 | 2023-10-10 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2019070776A JP7358762B2 (en) | 2019-04-02 | 2019-04-02 | Road anomaly detection device, road anomaly detection method, and road anomaly detection program |
JP2019-070776 | 2019-04-02 | ||
US16/812,483 US11491992B2 (en) | 2019-04-02 | 2020-03-09 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
US17/961,296 US11827228B2 (en) | 2019-04-02 | 2022-10-06 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
US18/378,489 US20240034327A1 (en) | 2019-04-02 | 2023-10-10 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/961,296 Continuation US11827228B2 (en) | 2019-04-02 | 2022-10-06 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
Publications (1)
Publication Number | Publication Date |
---|---|
US20240034327A1 true US20240034327A1 (en) | 2024-02-01 |
Family
ID=72662877
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/812,483 Active 2041-02-08 US11491992B2 (en) | 2019-04-02 | 2020-03-09 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
US17/961,296 Active US11827228B2 (en) | 2019-04-02 | 2022-10-06 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
US18/378,489 Pending US20240034327A1 (en) | 2019-04-02 | 2023-10-10 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/812,483 Active 2041-02-08 US11491992B2 (en) | 2019-04-02 | 2020-03-09 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
US17/961,296 Active US11827228B2 (en) | 2019-04-02 | 2022-10-06 | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program |
Country Status (3)
Country | Link |
---|---|
US (3) | US11491992B2 (en) |
JP (1) | JP7358762B2 (en) |
CN (1) | CN111797669A (en) |
Families Citing this family (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP7315909B2 (en) * | 2020-12-07 | 2023-07-27 | 宮川興業株式会社 | Computer system for visualizing soundness of crosswalk markings |
CN112702568B (en) * | 2020-12-14 | 2023-11-24 | 广东荣文科技集团有限公司 | Abnormality detection method and related device |
US20240127604A1 (en) * | 2021-04-07 | 2024-04-18 | Mitsubishi Electric Corporation | Road surface information collecting device, road surface deterioration detecting system, and road surface information collecting method |
CN113223295B (en) * | 2021-05-11 | 2024-01-12 | 深圳聚瑞云控科技有限公司 | Automatic alarm monitoring device and method based on Internet |
JP7497781B2 (en) | 2021-06-03 | 2024-06-11 | 株式会社デンソー | Map update system and map update device |
JP7241834B1 (en) | 2021-09-27 | 2023-03-17 | 三菱電機株式会社 | Road information collection device and road information collection system |
CN114170301B (en) * | 2022-02-09 | 2022-05-17 | 城云科技(中国)有限公司 | Abnormal municipal facility positioning method and device and application thereof |
WO2023166555A1 (en) * | 2022-03-01 | 2023-09-07 | 日本電気株式会社 | Road marking deterioration analysis device, method, and computer-readable medium |
CN114758322B (en) * | 2022-05-13 | 2022-11-22 | 安徽省路通公路工程检测有限公司 | Road quality detection system based on machine identification |
CN115984806B (en) * | 2023-03-20 | 2023-06-13 | 四川京炜数字科技有限公司 | Dynamic detection system for road marking damage |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3466169B2 (en) * | 2001-06-04 | 2003-11-10 | 株式会社リオスコーポレーション | Management system for roads and surrounding facilities |
JP4820712B2 (en) * | 2005-08-05 | 2011-11-24 | アイシン・エィ・ダブリュ株式会社 | Road marking recognition system |
CN102741900B (en) * | 2010-03-03 | 2014-12-10 | 松下电器产业株式会社 | Road condition management system and road condition management method |
JP6013739B2 (en) * | 2012-01-30 | 2016-10-25 | 鹿島建設株式会社 | Transport vehicle operation support system and transport vehicle operation support method |
JP2014164492A (en) * | 2013-02-25 | 2014-09-08 | Denso Corp | Block line recognition device |
JP6203208B2 (en) | 2015-02-18 | 2017-09-27 | 株式会社東芝 | Road structure management system and road structure management method |
JP6408434B2 (en) | 2015-07-14 | 2018-10-17 | 株式会社キクテック | Road marking body deterioration detection method |
WO2017145928A1 (en) * | 2016-02-25 | 2017-08-31 | 株式会社村田製作所 | Detection device and road surface information system, and vehicle |
JP2017181168A (en) * | 2016-03-29 | 2017-10-05 | 株式会社デンソー | Road surface condition detection system |
JP6802035B2 (en) * | 2016-10-24 | 2020-12-16 | 株式会社日立製作所 | Image processing device, warning device, image processing system, image processing method |
CN107392139B (en) * | 2017-07-18 | 2020-10-20 | 海信集团有限公司 | Lane line detection method based on Hough transform and terminal equipment |
CN107895151A (en) * | 2017-11-23 | 2018-04-10 | 长安大学 | Method for detecting lane lines based on machine vision under a kind of high light conditions |
-
2019
- 2019-04-02 JP JP2019070776A patent/JP7358762B2/en active Active
-
2020
- 2020-03-09 US US16/812,483 patent/US11491992B2/en active Active
- 2020-03-18 CN CN202010192489.2A patent/CN111797669A/en active Pending
-
2022
- 2022-10-06 US US17/961,296 patent/US11827228B2/en active Active
-
2023
- 2023-10-10 US US18/378,489 patent/US20240034327A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
US20200317201A1 (en) | 2020-10-08 |
US20230026281A1 (en) | 2023-01-26 |
US11827228B2 (en) | 2023-11-28 |
CN111797669A (en) | 2020-10-20 |
US11491992B2 (en) | 2022-11-08 |
JP2020170314A (en) | 2020-10-15 |
JP7358762B2 (en) | 2023-10-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11827228B2 (en) | Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program | |
US9365217B2 (en) | Mobile pothole detection system and method | |
US11361556B2 (en) | Deterioration diagnosis device, deterioration diagnosis system, deterioration diagnosis method, and storage medium for storing program | |
Ryu et al. | Image‐Based Pothole Detection System for ITS Service and Road Management System | |
CN102509457B (en) | Vehicle tracking method and device | |
KR102196255B1 (en) | Apparatus and method of image processing and deep learning image classification for detecting road surface damage | |
US9846823B2 (en) | Traffic lane boundary line extraction apparatus and method of extracting traffic lane boundary line | |
CN113139410B (en) | Pavement detection method, device, equipment and storage medium | |
CN102819951A (en) | Road target detection system, road target detection method and road target detection device | |
US20140350852A1 (en) | Device and method for detecting objects in a stream of sensor data | |
CN113506261A (en) | Road disease detection method, device, equipment and system | |
CN114341962A (en) | Dangerous vehicle display system, dangerous vehicle display device, dangerous vehicle display program, computer-readable recording medium, and apparatus having recorded the program | |
JP2017102672A (en) | Geographic position information specification system and geographic position information specification method | |
JP2022014432A (en) | Deterioration diagnosis system, deterioration diagnosis device, deterioration diagnosis method, and program | |
Kaushik et al. | Pothole detection system: A review of different methods used for detection | |
CN113283539A (en) | Method and system for detecting illegal slag-soil vehicles based on deep learning | |
KR20220138894A (en) | Prediction and recognition method of road marking information and road maintenance method | |
CN113033493A (en) | Target object inspection method and device, electronic equipment and storage medium | |
CN115631420B (en) | Tunnel accumulated water identification method and device, storage medium and electronic device | |
US20220101509A1 (en) | Deterioration diagnostic device, deterioration diagnostic system, deterioration diagnostic method, and recording medium | |
WO2023166555A1 (en) | Road marking deterioration analysis device, method, and computer-readable medium | |
US11538256B2 (en) | Systems and methods for data collection and performance monitoring of transportation infrastructure | |
JP7004543B2 (en) | Road surface condition detection device and road surface condition detection system | |
US11676397B2 (en) | System and method for detecting an object collision | |
CN111709354B (en) | Method and device for identifying target area, electronic equipment and road side equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |