US20240034327A1 - Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program - Google Patents

Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program Download PDF

Info

Publication number
US20240034327A1
US20240034327A1 US18/378,489 US202318378489A US2024034327A1 US 20240034327 A1 US20240034327 A1 US 20240034327A1 US 202318378489 A US202318378489 A US 202318378489A US 2024034327 A1 US2024034327 A1 US 2024034327A1
Authority
US
United States
Prior art keywords
road
indication
image
abnormality
abnormality detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US18/378,489
Inventor
Kazuaki Takemura
Makoto Matsushita
Tae SUGIMURA
Hideo Hasegawa
Tomoaki UMEKI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toyota Motor Corp
Original Assignee
Toyota Motor Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toyota Motor Corp filed Critical Toyota Motor Corp
Priority to US18/378,489 priority Critical patent/US20240034327A1/en
Publication of US20240034327A1 publication Critical patent/US20240034327A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/06Road conditions
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/04Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/56Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2552/00Input parameters relating to infrastructure
    • B60W2552/53Road markings, e.g. lane marker or crosswalk
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00Input parameters relating to data
    • B60W2556/10Historical data

Definitions

  • the present disclosure relates to a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program.
  • Japanese Patent Application Publication No. 2017-20303 discloses a method of capturing an image of a road indication applied onto a road surface obliquely from the above, correcting the captured image into an overhead view through homographic transformation, and determining the state of degradation of the road indication from the corrected image.
  • the present disclosure is devised in view of the above, and an object thereof is to provide a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program capable of early detecting abnormality in a road facility.
  • a road abnormality detection apparatus including: a memory; and a processor having hardware, wherein the processor is configured to: receive, from a traveling vehicle, a road image indicating an image of the surface of a road or the periphery of the road and image capturing position information indicating a position where the road image is captured; recognize a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determine whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulate, in the memory, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
  • the road abnormality detection apparatus can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
  • the processor may recognize the road facility included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
  • the road abnormality detection apparatus can recognize the road facility included in the road image by comparing the pixel values of the adjacent pixels with each other, such, for example, as brightness values of the pixels when performing image processing on the road image.
  • the processor may accumulate, in the memory, the road image including the road facility in which the abnormality exists in addition to the facility position information of the road facility in which the abnormality exists, when it is determined that the abnormality exists in the road facility.
  • the road abnormality detection apparatus can examine details of the abnormality arising in the road facility by collecting the image of the road facility in which the abnormality exists in addition to the position of the road facility.
  • the processor may transmit the facility position information to a server that a road administrator manages, when at least a predetermined quantity of the facility position information regarding the identical road facility has been accumulated in the memory.
  • the road abnormality detection apparatus can notify the road administrator side of the position of the road facility in which abnormality exists, and thereby, the road facility can be early repaired.
  • the road facility may include a road indication presented on the road and a structure provided on the road or around the road.
  • the road abnormality detection apparatus can early detect the abnormality arising in the road indication presented on the road and the structure provided on the road or around the road.
  • a road abnormality detection method which is a road abnormality detection method performed by a road abnormality detection apparatus including a processor.
  • the road abnormality detection method includes: receiving, by the processor, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognizing, by the processor, a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determining, by the processor, whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulating, by the processor, in a memory of the road abnormality detection apparatus, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
  • the road abnormality detection method can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
  • a road abnormality detection program which is a road abnormality detection program executed by a road abnormality detection apparatus, the program comprising: receiving, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognizing a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determining whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulating, in a memory of the road abnormality detection apparatus, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
  • the road abnormality detection program can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
  • abnormality in a road facility can be early detected based on a road image collected from a traveling vehicle.
  • FIG. 1 is a block diagram schematically showing a road abnormality detection system to which a road abnormality detection apparatus according to an embodiment of the present disclosure can be applied;
  • FIG. 2 is a schematic diagram exemplarily showing road facilities which are targets of the road abnormality detection apparatus according to the embodiment of the present disclosure
  • FIG. 3 is a flowchart showing a processing procedure of a road abnormality detection method according to the embodiment of the present disclosure.
  • FIG. 4 is a flowchart showing a modification of the processing procedure of the road abnormality detection method according to the embodiment of the present disclosure.
  • a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program according to an embodiment are described with reference to the drawings.
  • constituents in the embodiment below may include those that can be easily replaced by a person skilled in the art, or those that are substantially identical to these.
  • the road abnormality detection system has a processing server 1 , at least one vehicle 2 and a road administrator server 3 .
  • the road abnormality detection apparatus according to the present embodiment is configured, specifically, of the processing server 1 .
  • the at least one vehicle 2 may be one vehicle 2 or a plurality of vehicles 2 traveling on a road.
  • the processing server 1 , the vehicle 2 and the road administrator server 3 can communicate with one another via a network NW.
  • the network NW is configured, for example, of any of the Internet network, a mobile phone network and the like.
  • the processing server 1 processes an image received from the vehicle 2 (road image) and includes a control unit 11 , a communication unit 12 and a storage unit 13 .
  • control unit 11 includes a processor composed of a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA) and the like, and a memory composed of a random access memory (RAM), a read-only memory (ROM) and the like (main storage unit), these omitted from the figure.
  • CPU central processing unit
  • DSP digital signal processor
  • FPGA field-programmable gate array
  • main storage unit main storage unit
  • the control unit 11 loads a program stored in the storage unit 13 into a working area of the main storage unit to execute it, controls components and the like through the execution of the program, and thereby, realizes functions which meet predetermined purposes.
  • the control unit 11 functions as an image recognition unit 111 through the execution of the program. Notably, details of the image recognition unit 111 are mentioned later.
  • the communication unit 12 is configured, for example, of a local area network (LAN) interface board, a wireless communication circuit for wireless communication, and the like.
  • the communication unit 12 is connected to the network NW such as the Internet which is a public communication network.
  • the communication unit 12 communicates with the processing server 1 and the road administrator server 3 through the connection to the network NW.
  • the storage unit 13 is configured of an erasable programmable ROM (EPROM), a hard disk drive (HDD), and a recording medium such as a removable medium.
  • EPROM erasable programmable ROM
  • HDD hard disk drive
  • a recording medium such as a removable medium.
  • the removable medium include a universal serial bus (USB) memory, and disc recording media such as a compact disc (CD), a digital versatile disc (DVD) and a Blu-ray® disc (BD).
  • the storage unit 13 can store an operating system (OS), various programs, various tables, various databases and the like.
  • OS operating system
  • the storage unit 13 includes a correctness information DB 131 and an abnormality information DB 132 .
  • These databases (DB s) are built by a program for a database management system (DBMS) being executed by the processor above and managing data stored in the storage unit 13 .
  • DBMS database management system
  • the correctness information DB 131 is configured, for example, of a relational database in which a plurality of pieces of correctness information are retrievably stored.
  • the correctness information is information as a standard for determining whether or not abnormality exists in a road facility included in a road image transmitted from the vehicle 2 .
  • road image denotes an image obtained by imaging a surface of a road or a periphery of the road.
  • road facility includes road indications presented on the road and structures provided on the road and around the road.
  • Examples of the “road indications” above include a center line Rs 1 composed of a solid line, center lines Rs 2 composed of broken lines, outer lines Rs 3 , a stop line Rs 4 , a pedestrian crossing Rs 5 , pedestrian crossing advance notices Rs 6 , a speed limit indication Rs 7 , a right turn arrow Rs 8 and the like as shown in FIG. 2 , and road division lines and the like such as Botts' dots provided on roads in foreign countries.
  • Examples of the “structures” above include guardrails Rs 9 provided around the road, and road signs such as an information sign, a warning sign, a regulatory sign and a direction sign.
  • road signs such as an information sign, a warning sign, a regulatory sign and a direction sign.
  • road indications and structures include all the things installed on roads or around the roads in conformity with traffic regulations and the like.
  • the correctness information stored in the correctness information DB 131 is exemplarily information including types of road facilities which are to be at the site of the vehicle, and shapes and the like of the road facilities, and is stored therein as map information and various kinds of information such as image data. These pieces of correctness information are built as the correctness information DB 131 by the control unit 11 externally collecting those via the network NW and/or an operator manually inputting those. Notably, how to collect the correctness information is not limited to any of these methods for collecting the same.
  • the abnormality information DB 132 is configured, for example, of a relational database in which a plurality of pieces of abnormality information are retrievably stored.
  • the abnormality information is information regarding a road facility in which abnormality exists, and as mentioned later, for example, includes facility position information indicating a position of a road facility in which abnormality exists, a road image including the road facility, and the cause of the abnormality of the road facility.
  • the vehicle 2 is a movable body externally communicable and includes a control unit 21 , a communication unit 22 , a storage unit 23 , a positioning unit 24 and an image capturing unit 25 .
  • the control unit 21 and the storage unit 23 are physically similar to the control unit 11 and the storage unit 13 above, respectively.
  • the control unit 21 integrally controls operation of various components mounted on the vehicle 2 . Moreover, the control unit 21 stores, in the storage unit 23 , positional information of the vehicle 2 input from the positioning unit 24 in association with a road image input from the image capturing unit 25 . Namely, the control unit 21 stores, in the storage unit 23 , image capturing position information indicating a position where the road image is captured by the image capturing unit 25 in association with the road image thus captured.
  • the communication unit 22 is configured, for example, of a data communication module (DCM) and the like, and communicates with the processing server 1 through wireless communication via the network NW.
  • the storage unit 23 includes a road image DB 231 .
  • the road image DB 231 is exemplarily a relational database (RDB) in which road images captured by the image capturing unit 25 are retrievably stored. Notably, these road images are associated with the image capturing position information.
  • RDB relational database
  • the positioning unit 24 receives electric waves from global positioning system (GPS) satellites (not shown) to detect a current position of the vehicle 2 . Then, the positioning unit 24 outputs information regarding the detected current position of the vehicle 2 (hereinafter called “vehicle position information”) to the control unit 21 .
  • GPS global positioning system
  • vehicle position information information regarding the detected current position of the vehicle 2
  • the image capturing unit 25 is an image capturing camera or the like, for example, attached to a front portion of a body of the vehicle 2 , captures a road image frontward of the vehicle 2 while the vehicle 2 is traveling, and outputs the captured road image to the control unit 21 .
  • the “road image” may be any of still image data and moving image data, or may be moving image data generated using a plurality of chronologically continuous still image data.
  • the road administrator server 3 is a server managed by a road administrator.
  • the road administrator is an entity liable to repair of the road facilities, and examples thereof include local public entities such as the twenty-three wards of Tokyo, prefectures, cities, towns and villages, the Public Safety Commission, and the like.
  • the vehicle 2 captures an image of a road in its traveling with the image capturing unit 25 (step S 1 ). Subsequently, the vehicle 2 transmits the captured road image and the image capturing position information of the road image to the processing server 1 periodically in predetermined timing (step S 2 ).
  • predetermined timing for example, various kinds of timing such as periodical timing at predetermined time intervals, and timing when the vehicle 2 passes through predetermined positions.
  • the processing server 1 performs image recognition processing on the road image with the image recognition unit 111 (step S 3 ).
  • the image recognition unit 111 recognizes a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information using a known image recognition processing technique.
  • position of the road facility denotes coordinates (longitude; latitude) where the road facility is installed.
  • the image recognition unit 111 compares pixel values of adjacent pixels in the road image with one another, such, for example, as brightness values of the pixels, and thereby, recognizes a type of the road facility included in the road image. Moreover, the image recognition unit 111 recognizes the position of the road facility included in the road image, for example, by a method of estimating a depth distance to that road facility or the similar method.
  • the image recognition unit 111 determines whether or not abnormality exists in the road facility (step S 4 ). Specifically, the image recognition unit 111 determines whether or not abnormality exists in the road facility included in the road image by comparing the road facility and the correctness information which is prepared in advance and stored in the correctness information DB 131 with each other.
  • step S 4 When it is determined in step S 4 that no abnormality exists in the road facility (No in step S 4 ), the image recognition unit 111 returns the process to step S 3 . On the other hand, when it is determined in step S 4 that abnormality exists in the road facility (Yes in step S 4 ), the image recognition unit 111 identifies the cause of the abnormality (step S 5 ).
  • abnormality in road facilities There can be herein considered as abnormality in road facilities, breakage, abrasion, spots due to dirt, deposits of withered leaves, litter or the like, and the like.
  • the road facility is the center line Rs 1 drawn in the center of the road (see FIG. 2 ).
  • the image recognition unit 111 identifies a reason for the abnormality, for example, as any of breakage, abrasion, spots due to dirt, and withered leaves, litter or the like using a known image recognition processing technique.
  • the control unit 11 of the processing server 1 accumulates data of an image recognition processing result (step S 6 ).
  • the control unit 11 stores, in the abnormality information DB 132 of the storage unit 13 , facility position information at least indicating a position of the road facility in which the abnormality exists as abnormality information.
  • the control unit 11 may store, in the abnormality information DB 132 , the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility as the abnormality information in addition to the facility position information of the road facility in which the abnormality exists.
  • the control unit 11 notifies the road administrator that the abnormality arises in the road facility (step S 7 ). Specifically, the control unit 11 notifies the road administrator that the abnormality arises in the road facility by transmitting the abnormality information accumulated in the abnormality information DB 132 to the road administrator server 3 .
  • the abnormality information transmitted to the road administrator server 3 at least includes the facility position information of the road facility in which the abnormality exists, and may include the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility.
  • control unit 11 may transmit the abnormality information to the road administrator server 3 when a predetermined quantity of abnormality information regarding the identical road facility has been accumulated in the abnormality information DB 132 .
  • the control unit 11 may notify the road administrator that the abnormality arises in the road facility only when the image recognition processing (see step S 4 ) is performed on road images including the identical road facility which are captured, for example, by a plurality of vehicles 2 and “existence of abnormality” is determined for all of those. Thereby, the road administrator can be notified that the abnormality arises in the road facility only in a case of a high possibility that the abnormality arises in the road facility.
  • the road abnormality detection method and the road abnormality detection program according to the embodiment as described above it can early detect whether or not abnormality exists in a road facility by performing image recognition processing on a road image received from the traveling vehicle 2 .
  • the road abnormality detection method and the road abnormality detection program according to the embodiment since road images can be collected from the plurality of vehicles 2 and it can be determined whether or not abnormality exists in road facilities included in the road images, it can early detect whether or not abnormality exists in the road facilities widely installed.
  • abnormality information of road facilities can be sold to a road administrator in charge of repair of roads (any of local public entities, the Public Safety Commission, and the like).
  • the vehicle 2 side While in the road abnormality detection method described with FIG. 3 , the vehicle 2 side only captures road images and the processing server 1 side performs image recognition processing thereon, the vehicle 2 side can also perform the image recognition processing.
  • a modification of the road abnormality detection method according to the embodiment is described with reference to FIG. 4 .
  • the contents in steps S 1 , S 6 and S 7 in the figure are similar to the contents in steps S 1 , S 6 and S 7 of the road abnormality detection method described with FIG. 3 and their description is omitted.
  • step S 13 the control unit 21 of the vehicle 2 performs image recognition processing on the road image.
  • the control unit 21 recognizes a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information using a known image recognition processing technique.
  • control unit 21 compares pixel values of adjacent pixels in the road image with one another, such, for example, as brightness values of the pixels, and thereby, recognizes a type of the road facility included in the road image. Moreover, the control unit 21 recognizes the position of the road facility included in the road image, for example, by a method of estimating a depth distance to that road facility or the similar method.
  • the control unit 21 determines whether or not abnormality exists in the road facility (step S 14 ). Specifically, the control unit 21 determines whether or not abnormality exists in the road facility included in the road image based on the road facility and the correctness information which is prepared in advance. Notably, the correctness information above may be stored in the storage unit 23 of the vehicle 2 in advance or may be acquired every time from the correctness information DB 131 of the processing server 1 .
  • step S 14 When it is determined in step S 14 that no abnormality exists in the road facility (No in step S 14 ), the control unit 21 returns the process to step S 13 . On the other hand, when it is determined in step S 14 that abnormality exists in the road facility (Yes in step S 14 ), the control unit 21 identifies the cause of the abnormality (step S 15 ).
  • control unit 21 transmits data of an image recognition processing result to the processing server 1 (step S 16 ).
  • the control unit 21 transmits the facility position information at least indicating a position of the road facility in which the abnormality exists as the abnormality information to the processing server 1 .
  • the control unit 21 may transmit the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility as the abnormality information to the processing server 1 in addition to the facility position information of the road facility in which the abnormality exists.
  • load on the processing server 1 side can be reduced by the vehicle 2 side performing image recognition processing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Transportation (AREA)
  • Traffic Control Systems (AREA)
  • Road Repair (AREA)
  • Image Analysis (AREA)

Abstract

A road abnormality detection apparatus includes: a memory; and a processor having hardware, wherein the processor is configured to: receive, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognize a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determine whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance; and accumulate, in the memory, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.

Description

    CROSS-REFERENCE TO RELATED APPLICATION(S)
  • This application is a continuation application of U.S. patent application Ser. No. 17/961,296, filed Oct. 6, 2022, which is a continuation application of U.S. patent application Ser. No. 16/812,483, filed on Mar. 9, 2020, which is based on and claims priority under 35 U.S.C. 119 from Japanese Patent Application No. 2019-070776 filed on Apr. 2, 2019. The entire contents of the above applications are incorporated herein by reference.
  • BACKGROUND 1. Technical Field
  • The present disclosure relates to a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program.
  • 2. Description of Related Art
  • Japanese Patent Application Publication No. 2017-20303 discloses a method of capturing an image of a road indication applied onto a road surface obliquely from the above, correcting the captured image into an overhead view through homographic transformation, and determining the state of degradation of the road indication from the corrected image.
  • SUMMARY
  • For example, in automated driving or traveling during which white lines and the like need to be detected (for example, single lane traveling), accuracy of the traveling lowers when some white lines are faint or fade away. Therefore, when abnormality exists in a road facility such as a white line, the abnormality needs to be early detected to repair the road facility. It has been difficult to early detect abnormality in a road facility with a technology as disclosed in JP 2017-020303 A.
  • The present disclosure is devised in view of the above, and an object thereof is to provide a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program capable of early detecting abnormality in a road facility.
  • In order to solve the problem above and achieve the object, there is provided a road abnormality detection apparatus according to the present disclosure, including: a memory; and a processor having hardware, wherein the processor is configured to: receive, from a traveling vehicle, a road image indicating an image of the surface of a road or the periphery of the road and image capturing position information indicating a position where the road image is captured; recognize a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determine whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulate, in the memory, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
  • Thereby, the road abnormality detection apparatus can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
  • Moreover, as to the road abnormality detection apparatus according to the present disclosure, in the disclosure above, the processor may recognize the road facility included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
  • Thereby, the road abnormality detection apparatus can recognize the road facility included in the road image by comparing the pixel values of the adjacent pixels with each other, such, for example, as brightness values of the pixels when performing image processing on the road image.
  • Moreover, as to the road abnormality detection apparatus according to the present disclosure, in the disclosure above, the processor may accumulate, in the memory, the road image including the road facility in which the abnormality exists in addition to the facility position information of the road facility in which the abnormality exists, when it is determined that the abnormality exists in the road facility.
  • Thereby, the road abnormality detection apparatus can examine details of the abnormality arising in the road facility by collecting the image of the road facility in which the abnormality exists in addition to the position of the road facility.
  • Moreover, as to the road abnormality detection apparatus according to the present disclosure, in the disclosure above, the processor may transmit the facility position information to a server that a road administrator manages, when at least a predetermined quantity of the facility position information regarding the identical road facility has been accumulated in the memory.
  • Thereby, the road abnormality detection apparatus can notify the road administrator side of the position of the road facility in which abnormality exists, and thereby, the road facility can be early repaired.
  • Moreover, as to the road abnormality detection apparatus according to the present disclosure, in the disclosure above, the road facility may include a road indication presented on the road and a structure provided on the road or around the road.
  • Thereby, the road abnormality detection apparatus can early detect the abnormality arising in the road indication presented on the road and the structure provided on the road or around the road.
  • In order to solve the problem above and achieve the object, there is provided a road abnormality detection method according to the present disclosure which is a road abnormality detection method performed by a road abnormality detection apparatus including a processor. The road abnormality detection method includes: receiving, by the processor, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognizing, by the processor, a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determining, by the processor, whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulating, by the processor, in a memory of the road abnormality detection apparatus, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
  • Thereby, the road abnormality detection method can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
  • In order to solve the problem above and achieve the object, there is provided a road abnormality detection program according to the present disclosure which is a road abnormality detection program executed by a road abnormality detection apparatus, the program comprising: receiving, from a traveling vehicle, a road image indicating an image of a surface of a road or a periphery of the road and image capturing position information indicating a position where the road image is captured; recognizing a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information; determining whether or not abnormality exists in the road facility by comparing the road facility included in the road image and correctness information that is prepared in advance with each other; and accumulating, in a memory of the road abnormality detection apparatus, facility position information indicating the position of the road facility that the abnormality exists in when it is determined that the abnormality exists in the road facility.
  • Thereby, the road abnormality detection program can early detect whether or not the abnormality exists in the road facility by performing image recognition processing on the road image received from the traveling vehicle.
  • According to the present disclosure, abnormality in a road facility can be early detected based on a road image collected from a traveling vehicle.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Features, advantages, and technical and industrial significance of exemplary embodiments of the disclosure will be described below with reference to the accompanying drawings, in which like numerals denote like elements, and wherein:
  • FIG. 1 is a block diagram schematically showing a road abnormality detection system to which a road abnormality detection apparatus according to an embodiment of the present disclosure can be applied;
  • FIG. 2 is a schematic diagram exemplarily showing road facilities which are targets of the road abnormality detection apparatus according to the embodiment of the present disclosure;
  • FIG. 3 is a flowchart showing a processing procedure of a road abnormality detection method according to the embodiment of the present disclosure; and
  • FIG. 4 is a flowchart showing a modification of the processing procedure of the road abnormality detection method according to the embodiment of the present disclosure.
  • DETAILED DESCRIPTION OF EMBODIMENTS
  • A road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program according to an embodiment are described with reference to the drawings. Notably, constituents in the embodiment below may include those that can be easily replaced by a person skilled in the art, or those that are substantially identical to these.
  • Road Abnormality Detection System
  • First, a road abnormality detection system to which a road abnormality detection apparatus according to an embodiment can be applied is described with reference to FIG. 1 . The road abnormality detection system has a processing server 1, at least one vehicle 2 and a road administrator server 3. The road abnormality detection apparatus according to the present embodiment is configured, specifically, of the processing server 1. Moreover, the at least one vehicle 2 may be one vehicle 2 or a plurality of vehicles 2 traveling on a road.
  • The processing server 1, the vehicle 2 and the road administrator server 3 can communicate with one another via a network NW. The network NW is configured, for example, of any of the Internet network, a mobile phone network and the like.
  • Processing Server
  • Next, a configuration of the processing server 1 is described with reference to FIG. 1 and FIG. 2 . The processing server 1 processes an image received from the vehicle 2 (road image) and includes a control unit 11, a communication unit 12 and a storage unit 13.
  • Specifically, the control unit 11 includes a processor composed of a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA) and the like, and a memory composed of a random access memory (RAM), a read-only memory (ROM) and the like (main storage unit), these omitted from the figure.
  • The control unit 11 loads a program stored in the storage unit 13 into a working area of the main storage unit to execute it, controls components and the like through the execution of the program, and thereby, realizes functions which meet predetermined purposes. The control unit 11 functions as an image recognition unit 111 through the execution of the program. Notably, details of the image recognition unit 111 are mentioned later.
  • The communication unit 12 is configured, for example, of a local area network (LAN) interface board, a wireless communication circuit for wireless communication, and the like. The communication unit 12 is connected to the network NW such as the Internet which is a public communication network. The communication unit 12 communicates with the processing server 1 and the road administrator server 3 through the connection to the network NW.
  • The storage unit 13 is configured of an erasable programmable ROM (EPROM), a hard disk drive (HDD), and a recording medium such as a removable medium. Examples of the removable medium include a universal serial bus (USB) memory, and disc recording media such as a compact disc (CD), a digital versatile disc (DVD) and a Blu-ray® disc (BD). Moreover, the storage unit 13 can store an operating system (OS), various programs, various tables, various databases and the like.
  • The storage unit 13 includes a correctness information DB 131 and an abnormality information DB 132. These databases (DB s) are built by a program for a database management system (DBMS) being executed by the processor above and managing data stored in the storage unit 13.
  • The correctness information DB 131 is configured, for example, of a relational database in which a plurality of pieces of correctness information are retrievably stored. The correctness information is information as a standard for determining whether or not abnormality exists in a road facility included in a road image transmitted from the vehicle 2.
  • Herein, the term “road image” denotes an image obtained by imaging a surface of a road or a periphery of the road. Moreover, “road facility” includes road indications presented on the road and structures provided on the road and around the road.
  • Examples of the “road indications” above include a center line Rs1 composed of a solid line, center lines Rs2 composed of broken lines, outer lines Rs3, a stop line Rs4, a pedestrian crossing Rs5, pedestrian crossing advance notices Rs6, a speed limit indication Rs7, a right turn arrow Rs8 and the like as shown in FIG. 2 , and road division lines and the like such as Botts' dots provided on roads in foreign countries.
  • Examples of the “structures” above include guardrails Rs9 provided around the road, and road signs such as an information sign, a warning sign, a regulatory sign and a direction sign. Notably, not limited to those shown in the figure, such road indications and structures include all the things installed on roads or around the roads in conformity with traffic regulations and the like.
  • The correctness information stored in the correctness information DB 131 is exemplarily information including types of road facilities which are to be at the site of the vehicle, and shapes and the like of the road facilities, and is stored therein as map information and various kinds of information such as image data. These pieces of correctness information are built as the correctness information DB 131 by the control unit 11 externally collecting those via the network NW and/or an operator manually inputting those. Notably, how to collect the correctness information is not limited to any of these methods for collecting the same.
  • The abnormality information DB 132 is configured, for example, of a relational database in which a plurality of pieces of abnormality information are retrievably stored. The abnormality information is information regarding a road facility in which abnormality exists, and as mentioned later, for example, includes facility position information indicating a position of a road facility in which abnormality exists, a road image including the road facility, and the cause of the abnormality of the road facility.
  • Vehicle
  • Next, a configuration of the vehicle 2 is described with reference to FIG. 1 . The vehicle 2 is a movable body externally communicable and includes a control unit 21, a communication unit 22, a storage unit 23, a positioning unit 24 and an image capturing unit 25. The control unit 21 and the storage unit 23 are physically similar to the control unit 11 and the storage unit 13 above, respectively.
  • The control unit 21 integrally controls operation of various components mounted on the vehicle 2. Moreover, the control unit 21 stores, in the storage unit 23, positional information of the vehicle 2 input from the positioning unit 24 in association with a road image input from the image capturing unit 25. Namely, the control unit 21 stores, in the storage unit 23, image capturing position information indicating a position where the road image is captured by the image capturing unit 25 in association with the road image thus captured.
  • The communication unit 22 is configured, for example, of a data communication module (DCM) and the like, and communicates with the processing server 1 through wireless communication via the network NW. The storage unit 23 includes a road image DB 231. The road image DB 231 is exemplarily a relational database (RDB) in which road images captured by the image capturing unit 25 are retrievably stored. Notably, these road images are associated with the image capturing position information.
  • The positioning unit 24 receives electric waves from global positioning system (GPS) satellites (not shown) to detect a current position of the vehicle 2. Then, the positioning unit 24 outputs information regarding the detected current position of the vehicle 2 (hereinafter called “vehicle position information”) to the control unit 21. Notably, there can also be used as a method of detecting the current position of the vehicle 2 a method, for example, having light detection and ranging (LiDAR, laser imaging detection and ranging) and a three-dimensional digital map combined.
  • The image capturing unit 25 is an image capturing camera or the like, for example, attached to a front portion of a body of the vehicle 2, captures a road image frontward of the vehicle 2 while the vehicle 2 is traveling, and outputs the captured road image to the control unit 21. Notably, the “road image” may be any of still image data and moving image data, or may be moving image data generated using a plurality of chronologically continuous still image data.
  • Road Administrator Server
  • Next, a configuration of the road administrator server 3 is described with reference to FIG. 1 . The road administrator server 3 is a server managed by a road administrator. The road administrator is an entity liable to repair of the road facilities, and examples thereof include local public entities such as the twenty-three wards of Tokyo, prefectures, cities, towns and villages, the Public Safety Commission, and the like.
  • Road Abnormality Detection Method
  • Next, a road abnormality detection method performed by the road abnormality detection apparatus is described with reference to FIG. 3 .
  • First, the vehicle 2 captures an image of a road in its traveling with the image capturing unit 25 (step S1). Subsequently, the vehicle 2 transmits the captured road image and the image capturing position information of the road image to the processing server 1 periodically in predetermined timing (step S2). Notably, there can be set as such predetermined timing, for example, various kinds of timing such as periodical timing at predetermined time intervals, and timing when the vehicle 2 passes through predetermined positions.
  • Subsequently, after receiving the road image and the image capturing position information from the vehicle 2, the processing server 1 performs image recognition processing on the road image with the image recognition unit 111 (step S3). The image recognition unit 111 recognizes a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information using a known image recognition processing technique. Notably, the term “position of the road facility” denotes coordinates (longitude; latitude) where the road facility is installed.
  • Specifically, the image recognition unit 111 compares pixel values of adjacent pixels in the road image with one another, such, for example, as brightness values of the pixels, and thereby, recognizes a type of the road facility included in the road image. Moreover, the image recognition unit 111 recognizes the position of the road facility included in the road image, for example, by a method of estimating a depth distance to that road facility or the similar method.
  • Subsequently, the image recognition unit 111 determines whether or not abnormality exists in the road facility (step S4). Specifically, the image recognition unit 111 determines whether or not abnormality exists in the road facility included in the road image by comparing the road facility and the correctness information which is prepared in advance and stored in the correctness information DB 131 with each other.
  • When it is determined in step S4 that no abnormality exists in the road facility (No in step S4), the image recognition unit 111 returns the process to step S3. On the other hand, when it is determined in step S4 that abnormality exists in the road facility (Yes in step S4), the image recognition unit 111 identifies the cause of the abnormality (step S5).
  • There can be herein considered as abnormality in road facilities, breakage, abrasion, spots due to dirt, deposits of withered leaves, litter or the like, and the like. It is herein supposed, for example, that the road facility is the center line Rs1 drawn in the center of the road (see FIG. 2 ). In this case, when the center line Rs1 is damaged and “existence of abnormality” is determined in step S4, the image recognition unit 111 identifies a reason for the abnormality, for example, as any of breakage, abrasion, spots due to dirt, and withered leaves, litter or the like using a known image recognition processing technique.
  • Subsequently, the control unit 11 of the processing server 1 accumulates data of an image recognition processing result (step S6). In this step, the control unit 11 stores, in the abnormality information DB 132 of the storage unit 13, facility position information at least indicating a position of the road facility in which the abnormality exists as abnormality information. Moreover, in this step, the control unit 11 may store, in the abnormality information DB 132, the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility as the abnormality information in addition to the facility position information of the road facility in which the abnormality exists. As above, by collecting the image of the road facility in which the abnormality exists and the cause of the abnormality in addition to the position of the road facility, there can be examined details of the abnormality arising in the road facility.
  • Subsequently, the control unit 11 notifies the road administrator that the abnormality arises in the road facility (step S7). Specifically, the control unit 11 notifies the road administrator that the abnormality arises in the road facility by transmitting the abnormality information accumulated in the abnormality information DB 132 to the road administrator server 3. Notably, the abnormality information transmitted to the road administrator server 3 at least includes the facility position information of the road facility in which the abnormality exists, and may include the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility. By notifying the road administrator side of the position of the road facility in which the abnormality exists as above, the road facility can be early repaired.
  • Notably, the control unit 11 may transmit the abnormality information to the road administrator server 3 when a predetermined quantity of abnormality information regarding the identical road facility has been accumulated in the abnormality information DB 132. Namely, the control unit 11 may notify the road administrator that the abnormality arises in the road facility only when the image recognition processing (see step S4) is performed on road images including the identical road facility which are captured, for example, by a plurality of vehicles 2 and “existence of abnormality” is determined for all of those. Thereby, the road administrator can be notified that the abnormality arises in the road facility only in a case of a high possibility that the abnormality arises in the road facility.
  • According to the road abnormality detection apparatus, the road abnormality detection method and the road abnormality detection program according to the embodiment as described above, it can early detect whether or not abnormality exists in a road facility by performing image recognition processing on a road image received from the traveling vehicle 2. Moreover, according to the road abnormality detection apparatus, the road abnormality detection method and the road abnormality detection program according to the embodiment, since road images can be collected from the plurality of vehicles 2 and it can be determined whether or not abnormality exists in road facilities included in the road images, it can early detect whether or not abnormality exists in the road facilities widely installed. Moreover, according to the road abnormality detection apparatus, the road abnormality detection method and the road abnormality detection program according to the embodiment, for example, abnormality information of road facilities can be sold to a road administrator in charge of repair of roads (any of local public entities, the Public Safety Commission, and the like).
  • Modification
  • While in the road abnormality detection method described with FIG. 3 , the vehicle 2 side only captures road images and the processing server 1 side performs image recognition processing thereon, the vehicle 2 side can also perform the image recognition processing. Hereafter, a modification of the road abnormality detection method according to the embodiment is described with reference to FIG. 4 . Notably, the contents in steps S1, S6 and S7 in the figure are similar to the contents in steps S1, S6 and S7 of the road abnormality detection method described with FIG. 3 and their description is omitted.
  • Subsequently to step S1, the control unit 21 of the vehicle 2 performs image recognition processing on the road image (step S13). The control unit 21 recognizes a road facility included in the road image and a position of the road facility based on the road image and the image capturing position information using a known image recognition processing technique.
  • Specifically, the control unit 21 compares pixel values of adjacent pixels in the road image with one another, such, for example, as brightness values of the pixels, and thereby, recognizes a type of the road facility included in the road image. Moreover, the control unit 21 recognizes the position of the road facility included in the road image, for example, by a method of estimating a depth distance to that road facility or the similar method.
  • Subsequently, the control unit 21 determines whether or not abnormality exists in the road facility (step S14). Specifically, the control unit 21 determines whether or not abnormality exists in the road facility included in the road image based on the road facility and the correctness information which is prepared in advance. Notably, the correctness information above may be stored in the storage unit 23 of the vehicle 2 in advance or may be acquired every time from the correctness information DB 131 of the processing server 1.
  • When it is determined in step S14 that no abnormality exists in the road facility (No in step S14), the control unit 21 returns the process to step S13. On the other hand, when it is determined in step S14 that abnormality exists in the road facility (Yes in step S14), the control unit 21 identifies the cause of the abnormality (step S15).
  • Subsequently, the control unit 21 transmits data of an image recognition processing result to the processing server 1 (step S16). In this step, the control unit 21 transmits the facility position information at least indicating a position of the road facility in which the abnormality exists as the abnormality information to the processing server 1. Moreover, in this step, the control unit 21 may transmit the road image including the road facility in which the abnormality exists and the cause of the abnormality of the road facility as the abnormality information to the processing server 1 in addition to the facility position information of the road facility in which the abnormality exists.
  • According to the modification as to the road abnormality detection apparatus, the road abnormality detection method and the road abnormality detection program according to the embodiment as described above, load on the processing server 1 side can be reduced by the vehicle 2 side performing image recognition processing.
  • As above, a road abnormality detection apparatus, a road abnormality detection method and a road abnormality detection program according to the present disclosure have been described specifically with a mode for carrying out the disclosure. The scope of the present disclosure is not limited by such description but should be broadly construed based on the disclosure in the appended claims. It is needless to say that the scope of the present disclosure includes various alterations, modifications and the like based on such description.
  • Further effects and modifications can be easily derived by those skilled in the art. Thus, the broader aspects of the present disclosure are not limited to the specific details and representative embodiment shown and described above. Accordingly, various modifications can be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

Claims (20)

What is claimed is:
1. A road abnormality detection apparatus comprising:
a memory; and
a hardware processor programmed to:
receive, from a traveling vehicle, (i) a road image indicating an image of at least a surface of a road, the road image being captured by a camera attached to the vehicle and (ii) image capturing position information indicating a position where the road image is captured, the position being detected by a positioning unit of the vehicle using GPS data or laser imaging data;
recognize a road indication presented on the surface of the road included in the road image and a position of the road indication based on the road image and the image capturing position information;
determine that an abnormality exists in the road indication by using the road image, the abnormality in the road indication including spots due to dirt, deposits of withered leaves, and litter, wherein
the road indication includes at least one of a center line on the surface of the road, an outer line on the surface of the road, a stop line on the surface of the road, a pedestrian crossing indication on the surface of the road, a pedestrian crossing advance notice on the surface of the road, a speed limit indication on the surface of the road, an arrow line on the surface of the road, Bott's dots, and a road division line on the surface of the road; and
in response to determining that the abnormality exists in the same road indication a plurality of times based on a plurality of the road images captured by a plurality of the vehicles, transmit indication position information for repair of the road indication.
2. The road abnormality detection apparatus according to claim 1, wherein the processor recognizes the road indication included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
3. The road abnormality detection apparatus according to claim 1, wherein the processor accumulates, in the memory, the indication position information in response to determining that the abnormality exists in the road indication.
4. The road abnormality detection apparatus according to claim 3, wherein the processor accumulates, in the memory, the road image including the road indication in which the abnormality exists in addition to the indication position information of the road indication in which the abnormality exists, in response to determining that the abnormality exists in the road indication.
5. The road abnormality detection apparatus according to claim 1, wherein
the processor determines that the abnormality exists in the road indication by comparing the road indication included in the road image with correctness information that is prepared in advance.
6. The road abnormality detection apparatus according to claim 5, wherein
the correctness information includes map information having road indication information prepared in advance.
7. A road abnormality detection method performed by a road abnormality detection apparatus including a memory and a hardware processor, the road abnormality detection method comprising:
receiving, by the processor, from a traveling vehicle, (i) a road image indicating an image of at least a surface of a road, the road image being captured by a camera attached to the vehicle and (ii) image capturing position information indicating a position where the road image is captured, the position being detected by a positioning unit of the vehicle using GPS data or laser imaging data;
recognizing, by the processor, a road indication presented on the surface of the road included in the road image and a position of the road indication based on the road image and the image capturing position information;
determining, by the processor, that an abnormality exists in the road indication by using the road image, the abnormality in the road indication including spots due to dirt, deposits of withered leaves, and litter, wherein
the road indication includes at least one of a center line on the surface of the road, an outer line on the surface of the road, a stop line on the surface of the road, a pedestrian crossing indication on the surface of the road, a pedestrian crossing advance notice on the surface of the road, a speed limit indication on the surface of the road, an arrow line on the surface of the road, Bott's dots, and a road division line on the surface of the road; and
in response to determining that the abnormality exists in the same road indication a plurality of times based on a plurality of the road images captured by a plurality of the vehicles, transmitting, by the processor, indication position information for repair of the road indication.
8. The road abnormality detection method according to claim 7, wherein the processor recognizes the road indication included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
9. The road abnormality detection method according to claim 7, wherein the processor accumulates, in the memory, the indication position information in response to determining that the abnormality exists in the road indication.
10. The road abnormality detection method according to claim 9, wherein the processor accumulates, in the memory, the road image including the road indication in which the abnormality exists in addition to the indication position information of the road indication in which the abnormality exists, in response to determining that the abnormality exists in the road indication.
11. The road abnormality detection method according to claim 7, wherein
the processor determines that the abnormality exists in the road indication by comparing the road indication included in the road image with correctness information that is prepared in advance.
12. The road abnormality detection method according to claim 11, wherein
the correctness information includes map information having road indication information prepared in advance.
13. A non-transitory computer-readable medium storing thereon a road abnormality detection program executed by a road abnormality detection apparatus including a memory and a hardware processor, the program causing the processor to execute:
receiving, from a traveling vehicle, (i) a road image indicating an image of at least a surface of a road, the road image being captured by a camera attached to the vehicle and (ii) image capturing position information indicating a position where the road image is captured, the position being detected by a positioning unit of the vehicle using GPS data or laser imaging data;
recognizing a road indication presented on the surface of the road included in the road image and a position of the road indication based on the road image and the image capturing position information;
determining that an abnormality exists in the road indication by using the road image, the abnormality in the road indication including spots due to dirt, deposits of withered leaves, and litter, wherein
the road indication includes at least one of a center line on the surface of the road, an outer line on the surface of the road, a stop line on the surface of the road, a pedestrian crossing indication on the surface of the road, a pedestrian crossing advance notice on the surface of the road, a speed limit indication on the surface of the road, an arrow line on the surface of the road, Bott's dots, and a road division line on the surface of the road; and
in response to determining that the abnormality exists in the same road indication a plurality of times based on a plurality of the road images captured by a plurality of the vehicles, transmitting indication position information for repair of the road indication.
14. The non-transitory computer-readable medium according to claim 13, wherein the program causes the processor to recognize the road indication included in the road image by comparing pixel values of adjacent pixels in the road image with each other.
15. The non-transitory computer-readable medium according to claim 13, wherein the program causes the processor to accumulate, in the memory, the indication position information in response to determining that the abnormality exists in the road indication.
16. The non-transitory computer-readable medium according to claim 15, wherein the program causes the processor to accumulate, in the memory, the road image including the road indication in which the abnormality exists in addition to the indication position information of the road indication in which the abnormality exists, in response to determining that the abnormality exists in the road indication.
17. The non-transitory computer-readable medium according to claim 13, wherein the program causes the processor to determine that the abnormality exists in the road indication by comparing the road indication included in the road image with correctness information that is prepared in advance.
18. The non-transitory computer-readable medium according to claim 17, wherein
the correctness information includes map information having road indication information prepared in advance.
19. The road abnormality detection apparatus according to claim 1, wherein in response to determining that the abnormality exists in the road indication, the processor is programmed to identify a cause of the abnormality and transmit the cause of the abnormality.
20. The road abnormality detection method according to claim 7, wherein in response to determining that the abnormality exists in the road indication, identifying, by the processor, a cause of the abnormality and transmitting, by the processor, the cause of the abnormality.
US18/378,489 2019-04-02 2023-10-10 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program Pending US20240034327A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/378,489 US20240034327A1 (en) 2019-04-02 2023-10-10 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
JP2019070776A JP7358762B2 (en) 2019-04-02 2019-04-02 Road anomaly detection device, road anomaly detection method, and road anomaly detection program
JP2019-070776 2019-04-02
US16/812,483 US11491992B2 (en) 2019-04-02 2020-03-09 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program
US17/961,296 US11827228B2 (en) 2019-04-02 2022-10-06 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program
US18/378,489 US20240034327A1 (en) 2019-04-02 2023-10-10 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US17/961,296 Continuation US11827228B2 (en) 2019-04-02 2022-10-06 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program

Publications (1)

Publication Number Publication Date
US20240034327A1 true US20240034327A1 (en) 2024-02-01

Family

ID=72662877

Family Applications (3)

Application Number Title Priority Date Filing Date
US16/812,483 Active 2041-02-08 US11491992B2 (en) 2019-04-02 2020-03-09 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program
US17/961,296 Active US11827228B2 (en) 2019-04-02 2022-10-06 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program
US18/378,489 Pending US20240034327A1 (en) 2019-04-02 2023-10-10 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US16/812,483 Active 2041-02-08 US11491992B2 (en) 2019-04-02 2020-03-09 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program
US17/961,296 Active US11827228B2 (en) 2019-04-02 2022-10-06 Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program

Country Status (3)

Country Link
US (3) US11491992B2 (en)
JP (1) JP7358762B2 (en)
CN (1) CN111797669A (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7315909B2 (en) * 2020-12-07 2023-07-27 宮川興業株式会社 Computer system for visualizing soundness of crosswalk markings
CN112702568B (en) * 2020-12-14 2023-11-24 广东荣文科技集团有限公司 Abnormality detection method and related device
US20240127604A1 (en) * 2021-04-07 2024-04-18 Mitsubishi Electric Corporation Road surface information collecting device, road surface deterioration detecting system, and road surface information collecting method
CN113223295B (en) * 2021-05-11 2024-01-12 深圳聚瑞云控科技有限公司 Automatic alarm monitoring device and method based on Internet
JP7497781B2 (en) 2021-06-03 2024-06-11 株式会社デンソー Map update system and map update device
JP7241834B1 (en) 2021-09-27 2023-03-17 三菱電機株式会社 Road information collection device and road information collection system
CN114170301B (en) * 2022-02-09 2022-05-17 城云科技(中国)有限公司 Abnormal municipal facility positioning method and device and application thereof
WO2023166555A1 (en) * 2022-03-01 2023-09-07 日本電気株式会社 Road marking deterioration analysis device, method, and computer-readable medium
CN114758322B (en) * 2022-05-13 2022-11-22 安徽省路通公路工程检测有限公司 Road quality detection system based on machine identification
CN115984806B (en) * 2023-03-20 2023-06-13 四川京炜数字科技有限公司 Dynamic detection system for road marking damage

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3466169B2 (en) * 2001-06-04 2003-11-10 株式会社リオスコーポレーション Management system for roads and surrounding facilities
JP4820712B2 (en) * 2005-08-05 2011-11-24 アイシン・エィ・ダブリュ株式会社 Road marking recognition system
CN102741900B (en) * 2010-03-03 2014-12-10 松下电器产业株式会社 Road condition management system and road condition management method
JP6013739B2 (en) * 2012-01-30 2016-10-25 鹿島建設株式会社 Transport vehicle operation support system and transport vehicle operation support method
JP2014164492A (en) * 2013-02-25 2014-09-08 Denso Corp Block line recognition device
JP6203208B2 (en) 2015-02-18 2017-09-27 株式会社東芝 Road structure management system and road structure management method
JP6408434B2 (en) 2015-07-14 2018-10-17 株式会社キクテック Road marking body deterioration detection method
WO2017145928A1 (en) * 2016-02-25 2017-08-31 株式会社村田製作所 Detection device and road surface information system, and vehicle
JP2017181168A (en) * 2016-03-29 2017-10-05 株式会社デンソー Road surface condition detection system
JP6802035B2 (en) * 2016-10-24 2020-12-16 株式会社日立製作所 Image processing device, warning device, image processing system, image processing method
CN107392139B (en) * 2017-07-18 2020-10-20 海信集团有限公司 Lane line detection method based on Hough transform and terminal equipment
CN107895151A (en) * 2017-11-23 2018-04-10 长安大学 Method for detecting lane lines based on machine vision under a kind of high light conditions

Also Published As

Publication number Publication date
US20200317201A1 (en) 2020-10-08
US20230026281A1 (en) 2023-01-26
US11827228B2 (en) 2023-11-28
CN111797669A (en) 2020-10-20
US11491992B2 (en) 2022-11-08
JP2020170314A (en) 2020-10-15
JP7358762B2 (en) 2023-10-11

Similar Documents

Publication Publication Date Title
US11827228B2 (en) Road abnormality detection apparatus, road abnormality detection method and road abnormality detection program
US9365217B2 (en) Mobile pothole detection system and method
US11361556B2 (en) Deterioration diagnosis device, deterioration diagnosis system, deterioration diagnosis method, and storage medium for storing program
Ryu et al. Image‐Based Pothole Detection System for ITS Service and Road Management System
CN102509457B (en) Vehicle tracking method and device
KR102196255B1 (en) Apparatus and method of image processing and deep learning image classification for detecting road surface damage
US9846823B2 (en) Traffic lane boundary line extraction apparatus and method of extracting traffic lane boundary line
CN113139410B (en) Pavement detection method, device, equipment and storage medium
CN102819951A (en) Road target detection system, road target detection method and road target detection device
US20140350852A1 (en) Device and method for detecting objects in a stream of sensor data
CN113506261A (en) Road disease detection method, device, equipment and system
CN114341962A (en) Dangerous vehicle display system, dangerous vehicle display device, dangerous vehicle display program, computer-readable recording medium, and apparatus having recorded the program
JP2017102672A (en) Geographic position information specification system and geographic position information specification method
JP2022014432A (en) Deterioration diagnosis system, deterioration diagnosis device, deterioration diagnosis method, and program
Kaushik et al. Pothole detection system: A review of different methods used for detection
CN113283539A (en) Method and system for detecting illegal slag-soil vehicles based on deep learning
KR20220138894A (en) Prediction and recognition method of road marking information and road maintenance method
CN113033493A (en) Target object inspection method and device, electronic equipment and storage medium
CN115631420B (en) Tunnel accumulated water identification method and device, storage medium and electronic device
US20220101509A1 (en) Deterioration diagnostic device, deterioration diagnostic system, deterioration diagnostic method, and recording medium
WO2023166555A1 (en) Road marking deterioration analysis device, method, and computer-readable medium
US11538256B2 (en) Systems and methods for data collection and performance monitoring of transportation infrastructure
JP7004543B2 (en) Road surface condition detection device and road surface condition detection system
US11676397B2 (en) System and method for detecting an object collision
CN111709354B (en) Method and device for identifying target area, electronic equipment and road side equipment

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED