CN111611884A

CN111611884A - Road safety facility inspection system and method

Info

Publication number: CN111611884A
Application number: CN202010378683.XA
Authority: CN
Inventors: 赵文秀; 程俭廷; 王身宁; 梁富会; 胡朝辉
Original assignee: Guangzhou Chengan Road And Bridge Inspection Co ltd
Current assignee: Guangzhou Chengan Road And Bridge Inspection Co ltd
Priority date: 2020-05-07
Filing date: 2020-05-07
Publication date: 2020-09-01

Abstract

The embodiment of the application discloses a road safety facility patrol system and a method. According to the technical scheme, the vehicle-mounted acquisition module is used for acquiring the road facility information on the road, the acquired road facility information is sent to the database server to be stored, the patrol request is sent to the application server through the management terminal, the application server is enabled to call the road facility information from the database server and conduct road facility abnormity identification based on the road facility information, an abnormity identification result is generated, the application server returns the abnormity identification result to the management terminal, the abnormity identification result is displayed on the management terminal, patrol personnel can know the abnormity conditions of the safety facilities on the road conveniently, the abnormity conditions are found and processed in time, the safety facility patrol efficiency is improved, and automation, informatization and intellectualization of road patrol are achieved.

Description

Road safety facility inspection system and method

Technical Field

The embodiment of the application relates to a road patrol technology, in particular to a road safety facility patrol system and a method.

Background

In order to ensure the safety of road traffic, various safety auxiliary facilities such as various marking lines, various sign boards, guardrail anti-glare boards and the like are generally arranged on roads, and the safety auxiliary facilities need to be regularly patrolled.

The current traffic safety facility condition inspection is mainly carried out manually, but the traffic safety facility quantity is many, if the distance between the anti-dazzle plates is generally 0.5m, 20000 anti-dazzle plates need to be inspected on a road section of 10km, the traffic safety facility is longitudinally distributed widely and continuously distributed along the length of a road, the traffic safety facility is transversely distributed widely and transversely on the road surface, the manual inspection is limited by the width of a visual field and the distance, and the efficiency of the facility inspection is lower.

Disclosure of Invention

The embodiment of the application provides a system and a method for patrolling road safety facilities, which are used for patrolling the road safety facilities and improving the efficiency of patrolling the facilities.

In a first aspect, an embodiment of the present application provides a road safety facility inspection system, including a vehicle-mounted collection module, a database server, an application server, and a management terminal, wherein:

the vehicle-mounted acquisition module is in communication connection with the database server and is used for acquiring road facility information and transmitting the road facility information to the database server;

the database server is in communication connection with the application server and is used for storing the road facility information sent by the vehicle-mounted acquisition module;

the application server is in communication connection with the management terminal and used for responding to the patrol request sent by the management terminal, calling the road facility information from the database server to identify the abnormality of the road facility, generating an abnormality identification result and sending the abnormality identification result to the management terminal;

and the management terminal is used for sending a patrol request to the application server and receiving and displaying the abnormal recognition result.

Further, on-vehicle collection module is including being used for installing front end collection module, orientation module, power module and the data collection computer on removing the inspection car, power module is arranged in providing the power for the consumer in the on-vehicle collection module, wherein:

the front-end acquisition module is in communication connection with the data collection computer and is used for acquiring the road facility images and sending the road facility images to the data collection computer;

the positioning module is in communication connection with the data collection computer and is used for acquiring positioning information and sending the positioning information to the data collection computer;

the data collection computer is communicatively connected to the database server for collecting the asset images and the positioning information and sending the asset images and the positioning information to the database server as asset information.

Further, the front end collection module include the mounting box and install in the preceding view camera of mounting box, look sideways at camera, strabismus camera and far vision camera, the mounting box is used for fixed mounting in the top of removing the inspection vehicle, the preceding view camera look sideways at the camera the strabismus camera with the equal communication connection of far vision camera is in data collection computer, wherein:

the front-view camera is arranged on the front side of the mounting box and used for collecting road facility images in front of the mobile inspection vehicle;

the side-looking cameras are arranged on two sides of the mounting box and used for collecting road facility images on two sides of the mobile inspection vehicle;

the oblique-looking camera is arranged at an oblique angle of the mounting box and is used for collecting road facility images in the oblique-looking direction of the mobile inspection vehicle;

the far vision camera set up in the mounting box top for gather and remove the road facilities image of the long ahead of patrol car.

Further, the management terminal comprises a fixed terminal and/or a mobile terminal;

the fixed terminal is connected with the routing equipment of the application server end through the routing equipment through internet communication, so that the communication connection with the application server is established;

the mobile terminal is connected with the routing equipment of the application server end through the mobile base station through internet communication, so that communication connection with the application server is established.

In a second aspect, an embodiment of the present application provides a road safety equipment patrol method applied to the road safety equipment patrol system according to the first aspect, including:

calling the road facility information from the database server in response to a patrol request issued by the management terminal;

performing road facility abnormality recognition on road facility information by using a road facility abnormality recognition model to generate an abnormality recognition result, wherein the road facility abnormality recognition model performs road facility abnormality recognition training based on an abnormality sample image;

and sending the abnormal recognition result to a management terminal so that the management terminal displays the abnormal recognition result.

Further, the asset information comprises an asset image and positioning information;

the method for recognizing the road facility abnormality of the road facility information by using the road facility abnormality recognition model to generate an abnormality recognition result comprises the following steps:

performing road facility abnormality recognition on road facility images by using a road facility abnormality recognition model, and recognizing abnormal facility images in the road facility images;

determining abnormal positioning information corresponding to the abnormal facility image according to the time corresponding relation between the positioning information and the abnormal facility image;

and generating an abnormal identification result based on the abnormal facility image and the abnormal positioning information.

Further, the abnormal identification result comprises abnormal basic information and/or abnormal cloud picture information;

the generating an anomaly identification result based on the anomaly facility image and the anomaly locating information comprises:

generating abnormality basic information indicating an abnormality type and an abnormality position based on the abnormal facility image and the abnormality positioning information;

marking an abnormal type at a position corresponding to the facility condition cloud picture and the abnormal positioning information based on the abnormal facility image and the abnormal positioning information to generate abnormal cloud picture information;

and generating an abnormal recognition result based on the abnormal basic information and the abnormal cloud picture information.

In a third aspect, an embodiment of the present application provides a road safety facility inspection device, including a data call module, an anomaly identification module, and a result feedback module, where:

the data calling module is used for responding to a patrol request sent by the management terminal and calling the road facility information from the database server;

the system comprises an anomaly identification module, a road facility anomaly identification module and a data processing module, wherein the anomaly identification module is used for carrying out road facility anomaly identification on road facility information by utilizing a road facility anomaly identification model to generate an anomaly identification result, and the road facility anomaly identification model carries out road facility anomaly identification training on the basis of an anomaly sample image;

and the result feedback module is used for sending the abnormal recognition result to a management terminal so that the management terminal displays the abnormal recognition result.

Further, the asset information includes asset images and positioning information, and the anomaly identification module is specifically configured to:

Further, the anomaly identification result includes anomaly basic information and/or anomaly cloud map information, and the anomaly identification module specifically includes, when generating the anomaly identification result based on the anomaly facility image and the anomaly positioning information:

In a fourth aspect, an embodiment of the present application provides a computer device, including: a memory and one or more processors;

the memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the road safety facility patrol method according to the first aspect.

In a fifth aspect, embodiments of the present application provide a storage medium containing computer-executable instructions for performing the method of road safety equipment patrol according to the first aspect when executed by a computer processor.

According to the embodiment of the application, the vehicle-mounted acquisition module is used for acquiring the road facility information on the road, the acquired road facility information is sent to the database server to be stored, the patrol request is sent to the application server through the management terminal, the application server is enabled to call the road facility information from the database server and perform road facility abnormality identification based on the road facility information, an abnormality identification result is generated, the application server returns the abnormality identification result to the management terminal, the abnormality identification result is displayed on the management terminal, a patrol worker can conveniently know the abnormality of the safety facility on the road, the abnormality is timely found and processed, the patrol efficiency of the safety facility is improved, and automation, informatization and intellectualization of road patrol are realized.

Drawings

Fig. 1 is a system block diagram of a road safety facility patrol system provided in an embodiment of the present application;

FIG. 2 is a system block diagram of another road safety system patrol system provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of an on-board acquisition module provided in an embodiment of the present application;

fig. 4 is a schematic top view of a front-end acquisition module provided in an embodiment of the present application;

fig. 5 is a schematic front view of a front-end acquisition module provided in an embodiment of the present application;

fig. 6 is a flowchart of a method for patrolling a road safety facility according to an embodiment of the present application;

FIG. 7 is a flow chart of another method for a road safety device patrol provided by an embodiment of the present application;

fig. 8 is a schematic structural diagram of a road safety facility inspection device provided in an embodiment of the present application;

fig. 9 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Reference numerals: 1. a vehicle-mounted acquisition module; 11. moving the inspection vehicle; 12. a front-end acquisition module; 121. mounting a box; 122. a forward looking camera; 123. a side-looking camera; 124. a squint camera; 125. a far vision camera; 126. mounting a foot plate; 13. a positioning module; 14. a power supply module; 15. a data collection computer; 2. a database server; 3. an application server; 4. a management terminal; 41. fixing the terminal; 42. a mobile terminal; 5. a routing device; 6. a mobile base station; 7. a switch.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, specific embodiments of the present application will be described in detail with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be further noted that, for the convenience of description, only some but not all of the relevant portions of the present application are shown in the drawings.

In the description of the embodiments of the present application, unless explicitly stated or limited otherwise, the terms "connected," "connected," and "fixed" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral part; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meaning of the above terms in the present application can be understood in a specific case by those of ordinary skill in the art.

Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently or simultaneously. In addition, the order of the operations may be re-arranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.

Fig. 1 shows a system block diagram of a road safety facility patrol system provided in an embodiment of the present application, and as shown in fig. 1, the road safety facility patrol system includes an on-vehicle acquisition module 1, a database server 2, an application server 3, and a management terminal 4.

The vehicle-mounted acquisition module 1 is in communication connection with the database server 2 in a wired and/or wireless mode. The vehicle-mounted acquisition module 1 is used for acquiring the road facility information and storing the road facility information. After establishing communication connection with the database server 2, the vehicle-mounted acquisition module 1 transmits the road facility information to the database server 2.

It is understood that, when acquiring the asset information, the on-board acquisition module 1 disconnects the communication with the database server 2 and loads the on-board acquisition module 1 on the vehicle, and the on-board acquisition module 1 captures an image corresponding to the asset as the asset information as the vehicle moves on the road. The images acquired by the vehicle-mounted acquisition module 1 can be images shot at set time intervals or distance intervals or videos shot along the way.

The database server 2 is in communication connection with the application server 3 in a wired and/or wireless mode, and the database server 2 stores the road facility information after receiving the road facility information sent by the vehicle-mounted acquisition module 1. Optionally, the database server 2 may record the road and the time corresponding to the road facility information when storing the road facility information.

The application server 3 is communicatively connected to the management terminal 4 by wire and/or wireless, and the application server 3 calls the asset information from the database server 2 in response to a patrol request issued by the management terminal 4 to perform asset abnormality identification, generates an abnormality identification result, and transmits the abnormality identification result to the management terminal 4.

The step of identifying the road facility abnormality by using the road facility information may be performed in the application server 3 or the database server 2, and a corresponding process is started in the application server 3 or the database server 2 to identify the road facility abnormality. If the application server 3 executes the road facility abnormality recognition, after the application server 3 receives the patrol request, the corresponding road facility information is called from the database server 2 according to the road or time to which the patrol request points, and the road facility abnormality recognition is performed according to the road facility information to obtain an abnormality recognition result. The road facilities comprise road surfaces, guardrails, signs, marked lines, anti-dazzle plates, sound insulation screens and the like, and the abnormality identification result comprises an abnormality type (such as an anti-dazzle arrangement defect, a guardrail defect, a sound insulation screen defect, a signboard defect, a marked line defect and the like) and an abnormality position (which can be determined through the corresponding relation between positioning information of the positioning device and a time stamp of an image).

For example, if the identification of the road facility abnormality is performed in the database server 2, after the application server 3 receives the patrol request, the patrol request is forwarded to the database server 2, the database server 2 retrieves the corresponding road facility information from the local according to the road or time to which the patrol request is directed, performs the identification of the road facility abnormality according to the road facility information to obtain an abnormality identification result, and returns the abnormality identification result to the application server 3. The embodiment of the present application is described taking as an example the case where the identification of the asset abnormality is performed in the application server 3.

For example, a patrol request button may be provided on the displayed operation interface of the management terminal 4, and a selection interface of the road or time to which the patrol request is directed may be provided, a patrol request directed to the road or time may be generated according to the selection interface of the road or time, and the patrol request may be transmitted to the application server 3.

Furthermore, after receiving the returned abnormal recognition result, the management terminal 4 displays the abnormal recognition result, so that the patrol personnel can conveniently observe the abnormal condition of the road facility.

The wired connection between the devices can be performed through a network cable, a bus, RS485, RS232 and the like, and the wireless connection can be performed through WiFi, Bluetooth, a communication network, the Internet, a router and the like. The selection may be performed according to actual deployment requirements, and the embodiment is not limited.

The vehicle-mounted acquisition module 1 is used for acquiring the road facility information on the road, the acquired road facility information is sent to the database server 2 to be stored, the patrol request is sent to the application server 3 through the management terminal 4, the application server 3 is enabled to call the road facility information from the database server 2 and conduct road facility abnormality identification based on the road facility information, an abnormality identification result is generated, the abnormality identification result is returned to the management terminal 4 through the application server 3, the abnormality identification result is displayed on the management terminal 4, a patrol worker can know the abnormality of the safety facility on the road conveniently, the abnormality is found and processed in time, the safety facility patrol efficiency is improved, and automation, informatization and intellectualization of road patrol are achieved.

Fig. 2 is a system block diagram of another road safety equipment patrol system according to an embodiment of the present application, which is further configured based on the above embodiment. As shown in fig. 2, the road safety facility patrol system includes an on-vehicle collection module 1, a database server 2, an application server 3, and a management terminal 4.

Fig. 3 is a schematic structural diagram of the vehicle-mounted acquisition module 1 provided in the embodiment of the present application. As shown in fig. 3, the vehicle-mounted inspection system provided by the embodiment of the application includes a front-end acquisition module 12, a positioning module 13, a power module 14 and a data collection computer 15, where the power module 14 provides a special power supply for electric devices such as the front-end acquisition module 12, the positioning module 13 and the data collection computer 15, and is matched with a special power management system to ensure stable power supply for each electric device of the vehicle-mounted acquisition module 1.

Fig. 4 is a schematic top view of the front-end acquisition module 12 provided in the embodiment of the present application, and fig. 5 is a schematic front view of the front-end acquisition module 12 provided in the embodiment of the present application. As shown in fig. 4 and 5, the front-end collection module 12 provided in the embodiment of the present application includes a mounting box 121, a front-view camera 122, a side-view camera 123, a squint camera 124, and a far-view camera 125.

The mounting box 121 is fixedly mounted on the top of the mobile inspection vehicle 11, the mounting foot plate 126 fixedly connected with the mobile inspection vehicle 11 is arranged at the bottom of the mounting box 121, and the mounting foot plate 126 can be fixedly connected to the top of the mobile inspection vehicle 11 by fasteners such as bolts. In the present embodiment, the mounting box 121 is provided in a rectangular box shape, and four corners of the mounting box 121 are provided in a chamfered shape.

Further, the front view camera 122 is provided on the front side of the mounting box 121 such that the front view camera 122 faces the near front of the mobile patrol car 11 and is used to collect an image of a road facility near the front of the mobile patrol car 11. Look sideways at camera 123 and be provided with two to set up respectively in the both sides of mounting box 121, look sideways at the camera lens of camera 123 and respectively towards the outside setting on removal inspection car 11 both sides for gather the road facilities image that removes inspection car 11 both sides. The four oblique cameras 124 are respectively installed at four chamfers of the installation units, and lenses of the four oblique cameras 124 respectively face four oblique directions of the mobile inspection vehicle 11, are used for collecting road facility images in the oblique directions of the mobile inspection vehicle 11, and can be used for supplement and assistance of front view and side view fields. The far vision camera 125 is mounted on top of the mounting box 121, and a lens of the far vision camera 125 is disposed toward the far front of the mobile patrol car 11 for collecting an image of the road facility far in front of the mobile patrol car 11. The present embodiment is described by taking a trackball as the far-vision camera 125 as an example.

Further, the front-looking camera 122, the side-looking camera 123, the oblique-looking camera 124, and the far-looking camera 125 are all communicatively connected to the data collection computer 15 by wire and/or wirelessly. The data collection computer 15 is a high performance data analysis system industrial computer, and meets the requirements for device control and front-end data collection of the front-end collection module 12.

The forward-looking camera 122, the side-looking camera 123, the oblique-looking camera 124 and the far-looking camera 125 respectively collect road facility images of the near front, the two sides, the oblique-looking direction and the far front of the mobile inspection vehicle 11 and send the road facility images to the data collection computer 15, and the road facility images are stored by the data collection computer 15.

The positioning module 13 provided in this embodiment is specifically a high-precision satellite positioning GNSS (global navigation satellite system), and the positioning module 13 is in communication connection with the data collection computer 15 in a wired and/or wireless manner, and is configured to collect, in real time, positioning information corresponding to a position where the mobile inspection vehicle 11 is located, and send the collected positioning information to the data collection computer 15 for storage.

The database server 2 and the data collection computer 15 can be connected in a wired and/or wireless communication manner, and after the database server 2 establishes a communication connection with the data collection computer 15, road facility images and positioning information are acquired from the data collection computer 15 and stored locally. Optionally, when the image of the road facility and the positioning information are saved, the road section corresponding to the image of the road facility and the positioning information or the collection time may be marked or stored in a classified manner. In the present embodiment, the data collection computer 15 accesses the database server 2 through the switch 7, and realizes communication connection with the database server 2.

In other embodiments, the data collection computer 15 may also collect the asset images and the location information, access the storage device storing the asset images and the location information to the database server 2 or other computer communicatively coupled to the database server 2, and upload the asset images and the location information in the storage device to the database server 2.

Further, the application server 3 and the database server 2 are connected in a wired and/or wireless communication manner. The application server 3 may call the image of the road facility and the positioning information from the database server 2 in response to the patrol request issued by the management terminal 4 to perform the abnormality recognition of the road facility, generate the abnormality recognition result, and transmit the abnormality recognition result to the management terminal 4.

Further, the database server 2 is connected to the internet via a routing device 5. Optionally, a firewall may be disposed between the database server 2 and the routing device 5, so as to ensure the security of the communication connection between the application server 3 and the external terminal.

In the present embodiment, the management terminal 4 includes a fixed terminal 41 and a mobile terminal 42. The mobile terminal 42 may be a mobile device such as a mobile phone, a laptop, a tablet computer, etc., and is communicatively connected to the routing device 5 at the application server 3 end through the mobile base station 6 via the internet, so as to establish a communication connection with the application server 3.

The fixed terminal 41 includes computer devices or servers installed in different functional departments such as a management unit, an inspection unit, a maintenance unit, and the like, and realizes efficient transfer of detection data among the inspection unit, the maintenance unit, and the management unit. Further, the fixed terminal 41 is communicatively connected to the routing device 5 on the application server 3 side through the routing device 5 via the internet, thereby establishing a communication connection with the application server 3. Wherein the number of the fixed terminals 41 and the mobile terminals 42 can be determined according to actual deployment requirements, and the fixed terminals 41 in the same area or the same functional department are connected to one or more routing devices 5 by means of buses. For example, a plurality of fixed terminals 41 (stations) are provided corresponding to the management unit in fig. 2, and are connected to the routing device 5 through a bus.

Specifically, an interactive interface (such as an application program or a web page) for managing the road facility patrol information may be provided in the management terminal 4, and patrol related personnel may send a patrol request to the application server 3 through the interactive interface and display the abnormal recognition result on the interactive interface after receiving the abnormal recognition result returned by the application server 3.

The vehicle-mounted acquisition module 1 is used for acquiring the road facility information on the road, the acquired road facility information is sent to the database server 2 to be stored, the patrol request is sent to the application server 3 through the management terminal 4, the application server 3 is enabled to call the road facility information from the database server 2 and conduct road facility abnormality identification based on the road facility information, an abnormality identification result is generated, the abnormality identification result is returned to the management terminal 4 through the application server 3, the abnormality identification result is displayed on the management terminal 4, a patrol worker can know the abnormality of the safety facility on the road conveniently, the abnormality is found and processed in time, the safety facility patrol efficiency is improved, and automation, informatization and intellectualization of road patrol are achieved. Meanwhile, different cameras on the vehicle-mounted acquisition module 1 shoot road facilities at different positions on a road, the problem that manual inspection is horizontal and the problem that the longitudinal coverage surface is narrow can be solved, the abnormal conditions of the road facilities at all positions can be conveniently known by matching with the positioning module 13, and the inspection efficiency of the road facilities is further improved.

Fig. 6 is a flowchart of a road safety equipment patrol method according to an embodiment of the present disclosure, which may be implemented by a road safety equipment patrol device, which may be implemented by hardware and/or software, integrated in a computer device (application server), and applied to the road safety equipment patrol system according to the embodiment.

The following description will be given taking as an example a method of the road safety equipment patrol apparatus performing the road safety equipment patrol. Referring to fig. 1, the road safety equipment patrol method includes:

s101: the asset information is called from the database server in response to a patrol request issued by the management terminal.

The road facility information is acquired by a vehicle-mounted acquisition module loaded on the mobile inspection vehicle, the vehicle-mounted acquisition module shoots road safety facilities on the road along with the movement of the mobile inspection vehicle on the road, and the shot images are used as road facility information (pictures or videos, in the embodiment, the videos are taken as an example) and uploaded to a database server by the vehicle-mounted acquisition module for storage.

For example, an interactive interface (e.g., an application or a web page) for managing the asset patrol information may be provided in the management terminal, and patrol related personnel may generate a patrol request through the interactive interface. After generating the patrol request, the management terminal sends the patrol request to the application server.

Upon receiving the patrol request, the application server communicates with the database server and retrieves asset information stored in the database server in response to the patrol request.

S102: and carrying out road facility abnormality recognition on the road facility information by using the road facility abnormality recognition model to generate an abnormality recognition result.

The road facility abnormality recognition model carries out road facility abnormality recognition training based on the abnormal sample image. The abnormal sample images are images generated by photographing or recording abnormal road facilities, and are collected and used for training the road facility abnormality recognition model. The abnormal conditions of the road facilities include but are not limited to the defects of anti-dazzle arrangement, guardrail, sound insulation screen, signboards, marked lines and the like.

For example, after acquiring the asset information, the asset information is input to the asset abnormality recognition model, and the asset abnormality recognition model performs asset abnormality recognition on the asset information, and generates and outputs an abnormality recognition result.

In another embodiment, after the database server updates the stored asset information, the asset abnormality recognition may be performed on the updated asset information using the asset abnormality recognition model to generate an abnormality recognition result. After receiving the patrol request, the corresponding abnormal recognition result can be directly returned to the management terminal according to the road section or time corresponding to the patrol request.

S103: and sending the abnormal recognition result to a management terminal so that the management terminal displays the abnormal recognition result.

Illustratively, after generating the abnormal recognition result, according to the address information carried by the patrol request, the abnormal recognition result is returned to the management terminal which sends the patrol request.

After receiving the abnormal recognition result, the management terminal can display the abnormal recognition result on the interactive interface, so that patrol related personnel can know the abnormal condition of the road facility in time.

The vehicle-mounted acquisition module is used for acquiring the road facility information on the road, the acquired road facility information is sent to the database server for storage, the patrol request is sent to the application server through the management terminal, the application server calls the road facility information from the database server and performs road facility abnormality identification based on the road facility information to generate an abnormality identification result, the application server returns the abnormality identification result to the management terminal, and the abnormality identification result is displayed on the management terminal, so that patrol personnel can conveniently know the abnormality of the safety facility on the road, can timely find and process the abnormality, improve the patrol efficiency of the safety facility, and realize automation, informatization and intellectualization of road patrol.

Fig. 7 is a flowchart of another road safety equipment patrol method provided in an embodiment of the present application, which is an embodiment of the road safety equipment patrol method. Referring to fig. 7, the road safety equipment patrol method includes:

s201: and establishing a road facility abnormality recognition model, and performing road facility abnormality recognition training based on the abnormal sample image.

Specifically, a road facility abnormality recognition model is established based on a neural network, abnormality type labeling is performed on each abnormal sample image, the abnormal sample image is used as input, the abnormality type is used as output, road facility abnormality recognition training is performed on the road facility abnormality recognition model until the model accuracy reaches a preset threshold (for example, the accuracy reaches 90%), and the road facility abnormality recognition model with the accuracy reaching the preset threshold is obtained. After the road facility image is input into the road facility abnormality recognition model, the road facility image with the road facility abnormality and the corresponding abnormality type can be recognized by the road facility abnormality recognition model.

S202: the asset information is called from the database server in response to a patrol request issued by the management terminal.

The asset information provided by the present embodiment includes an asset image and location information. The road facility images are acquired by cameras (a front-view camera, a side-view camera, a strabismus camera and a far-view camera) in different directions on the vehicle-mounted acquisition module, and the time information (such as a time stamp) when the road facility images are shot is attached to the road facility images.

The positioning information is detected by a positioning module mounted on the mobile patrol vehicle, and the positioning information is accompanied by time information (timestamp). The asset image and the positioning information corresponding to the same time can be determined based on the time correspondence (e.g., a timestamp pointing to the same time) between the asset image and the positioning information, thereby determining at which position the asset image was captured.

Specifically, after a patrol request sent by the management terminal is received, the road facility image and the positioning information corresponding to the road facility information to be called are determined according to the road section or the time period pointed by the patrol request, and the road facility image and the positioning information are called from the database server.

S203: and carrying out road facility abnormality identification on the road facility image by using the road facility abnormality identification model, and identifying an abnormal facility image in the road facility image.

Specifically, after the image of the road facility is obtained, the image of the road facility is input to the abnormality recognition model of the road facility, the abnormality recognition model of the road facility recognizes the abnormality of the road facility from the image of the road facility, the image of the abnormal facility in the image of the road facility is recognized, and the abnormality type corresponding to the image of the abnormal facility is output.

Optionally, if the image of the road facility is a video, extracting a video frame from the image of the road facility (for example, extracting 6 frames per second), and inputting the extracted video frame into the road facility abnormality identification model to identify the road facility abnormality, thereby reducing the computational stress of the computer.

In other embodiments, the road facility image can be grayed to further reduce the computer computing pressure.

S204: and determining abnormal positioning information corresponding to the abnormal facility image according to the time corresponding relation between the positioning information and the abnormal facility image.

Illustratively, upon identifying an anomaly image and a corresponding anomaly type in the asset image, an anomaly identification result is generated based on the anomaly image and the anomaly locating information.

In this embodiment, the anomaly identification result includes anomaly basic information and anomaly cloud map information. The abnormal basic information comprises an abnormal facility image and a corresponding abnormal type, and the abnormal cloud picture information records the abnormal type, namely a corresponding position in a cloud picture mode.

Specifically, after the abnormal facility image and the corresponding abnormal type in the road facility image are identified, the corresponding abnormal positioning information when the abnormal facility image is shot is determined according to the time correspondence between the abnormal facility image and the positioning information. For example, a timestamp of the abnormal facility image is determined, and the positioning information corresponding to the same or similar timestamp is retrieved, i.e., the abnormal positioning information corresponding to the abnormal facility image.

S205: generating abnormality basic information indicating an abnormality type and an abnormality position based on the abnormality facility image and the abnormality localization information.

Specifically, the abnormal positioning information corresponding to each abnormal facility image is determined, and the abnormal basic information is generated according to each abnormal positioning information and the corresponding abnormal type.

It can be understood that when multiple road facility abnormalities (for example, a guardrail defect and a sound insulation screen defect) occur at the same position, the same abnormal positioning information corresponds to the situation that multiple abnormal types exist, and at the moment, the abnormal types corresponding to the same abnormal positioning information can be integrated.

S206: and marking an abnormal type at a position corresponding to the facility condition cloud picture and the abnormal positioning information based on the abnormal facility image and the abnormal positioning information to generate abnormal cloud picture information.

Specifically, after identifying the abnormal facility image, the abnormal type and the corresponding abnormal positioning information in the road facility image, a cloud image (a two-dimensional image or a three-dimensional image, which can be selected according to actual needs) corresponding to the road facility image is obtained, the cloud image is used as a facility status cloud image, the abnormal type is marked at the position corresponding to the facility status cloud image and the abnormal positioning information according to the mapping relationship between the abnormal positioning information and the facility status cloud image (the mapping relationship can be recorded according to the position and the actual position of each point of the image when the cloud image is manufactured, or the cloud image and the positioning information share a set of coordinate system), and the abnormal cloud image information is generated.

Optionally, when the anomaly type is indicated in the facility status cloud picture, different anomaly types may be indicated by different colors or shapes.

S207: and generating an abnormal recognition result based on the abnormal basic information and the abnormal cloud picture information.

Specifically, after the abnormal basic information and the abnormal cloud picture information are generated, the abnormal basic information and the abnormal cloud picture information are packaged to serve as an abnormal recognition result.

In other embodiments, the combination or one of the abnormal basic information and the abnormal cloud picture information may be selected to generate the abnormal recognition result according to the type of the management terminal (fixed terminal or mobile terminal) or the recognition result type indicated by the patrol request. It can be understood that, when the management terminal only requires to acquire one of the abnormal basic information or the abnormal cloud picture information as the abnormal recognition result, only the corresponding abnormal basic information or the abnormal cloud picture information may be generated.

S208: and sending the abnormal recognition result to a management terminal so that the management terminal displays the abnormal recognition result.

Specifically, after the abnormal recognition result is generated, the abnormal recognition result is returned to the management terminal that sent the patrol request according to the address information carried by the patrol request. After receiving the abnormal recognition result, the management terminal can display the abnormal recognition result on the interactive interface, so that the patrol related personnel can know the abnormal condition of the road facility in time

The vehicle-mounted acquisition module is used for acquiring the road facility information on the road, the acquired road facility information is sent to the database server for storage, the patrol request is sent to the application server through the management terminal, the application server calls the road facility information from the database server and performs road facility abnormality identification based on the road facility information to generate an abnormality identification result, the application server returns the abnormality identification result to the management terminal, and the abnormality identification result is displayed on the management terminal, so that patrol personnel can conveniently know the abnormality of the safety facility on the road, can timely find and process the abnormality, improve the patrol efficiency of the safety facility, and realize automation, informatization and intellectualization of road patrol. Meanwhile, the specific position of the abnormal type is determined according to the positioning information, so that patrol related personnel can conveniently and quickly position the specific position of the road facility abnormality, and can conveniently know the distribution condition of various abnormal types on the road by providing abnormal cloud picture information, and conveniently appoint a proper road patrol or overhaul scheme.

Fig. 8 is a schematic structural diagram of a road safety facility inspection device according to an embodiment of the present application. Referring to fig. 8, the road safety equipment patrol apparatus provided in the present embodiment includes a data call module 81, an abnormality recognition module 82, and a result feedback module 83.

The data calling module 81 is used for responding to a patrol request sent by the management terminal and calling the road facility information from the database server; an anomaly recognition module 82, configured to perform asset anomaly recognition on asset information by using an asset anomaly recognition model to generate an anomaly recognition result, where the asset anomaly recognition model performs asset anomaly recognition training based on an anomaly sample image; and a result feedback module 83, configured to send the abnormal recognition result to a management terminal, so that the management terminal displays the abnormal recognition result.

In one possible embodiment, the asset information includes an asset image and location information, and the anomaly identification module 82 is specifically configured to:

In a possible embodiment, the anomaly identification result includes anomaly basic information and/or anomaly cloud map information, and the anomaly identification module 82, when generating the anomaly identification result based on the anomaly facility image and the anomaly location information, specifically includes:

The embodiment of the application also provides computer equipment which can be integrated with the road safety facility patrol device provided by the embodiment of the application. Fig. 9 is a schematic structural diagram of a computer device according to an embodiment of the present application. Referring to fig. 9, the computer apparatus includes: an input device 93, an output device 94, a memory 92, and one or more processors 91; the memory 92 for storing one or more programs; when the one or more programs are executed by the one or more processors 91, the one or more processors 91 implement the road safety equipment patrol method provided in the above-described embodiment. The input device 93, the output device 94, the memory 92 and the processor 91 may be connected by a bus or other means, and fig. 9 illustrates the connection by the bus as an example.

The memory 92, which is a computer-readable storage medium, may be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the road safety equipment patrol method according to any embodiment of the present application (e.g., the data call module 81, the abnormality identification module 82, and the result feedback module 83 in the road safety equipment patrol module). The memory 92 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the device, and the like. Further, memory 92 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 92 may further include memory located remotely from the processor 91, which may be connected to the device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input device 93 may be used to receive input numeric or character information and generate key signal inputs relating to user settings and function control of the apparatus. The output device 94 may include a display device such as a display screen.

The processor 91 executes various functional applications of the device and data processing, i.e., implements the above-described road safety facility patrol method, by executing software programs, instructions, and modules stored in the memory 92.

The road safety facility patrol device and the computer provided by the embodiment can be used for executing the road safety facility patrol method provided by the embodiment, and have corresponding functions and beneficial effects.

Embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the method for road safety equipment patrol provided in the above embodiments, the method comprising: calling the road facility information from the database server in response to a patrol request issued by the management terminal; performing road facility abnormality recognition on road facility information by using a road facility abnormality recognition model to generate an abnormality recognition result, wherein the road facility abnormality recognition model performs road facility abnormality recognition training based on an abnormality sample image; and sending the abnormal recognition result to a management terminal so that the management terminal displays the abnormal recognition result.

Storage medium-any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk, or tape devices; computer system memory or random access memory such as DRAM, DDRRAM, SRAM, EDORAM, Lanbas (Rambus) RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system connected to the first computer system through a network (such as the internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations, such as in different computer systems that are connected by a network. The storage medium may store program instructions (e.g., embodied as a computer program) that are executable by one or more processors.

Of course, the storage medium provided in the embodiments of the present application contains computer-executable instructions, and the computer-executable instructions are not limited to the method for patrolling an asset as described above, and may also perform related operations in the method for patrolling an asset as provided in any embodiments of the present application.

The road safety equipment patrol device, the apparatus, and the storage medium provided in the above embodiments may perform the road safety equipment patrol method provided in any of the embodiments of the present application, and the technical details not described in detail in the above embodiments may be referred to the road safety equipment patrol method provided in any of the embodiments of the present application.

The foregoing is considered as illustrative of the preferred embodiments of the invention and the technical principles employed. The present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. The utility model provides a road safety facility inspection system which characterized in that, includes on-vehicle collection module (1), database server (2), application server (3) and management terminal (4), wherein:

the vehicle-mounted acquisition module (1) is in communication connection with the database server (2) and is used for acquiring road facility information and transmitting the road facility information to the database server (2);

the database server (2) is in communication connection with the application server (3) and is used for storing the road facility information sent by the vehicle-mounted acquisition module (1);

the application server (3) is in communication connection with the management terminal (4) and is used for responding to a patrol request sent by the management terminal (4), calling the road facility information from the database server (2) to identify the road facility abnormity, generating an abnormity identification result and sending the abnormity identification result to the management terminal (4);

and the management terminal (4) is used for sending a patrol request to the application server (3) and receiving and displaying the abnormal recognition result.

2. The road safety arrangement patrol system according to claim 1, wherein the on-board collection module (1) comprises a front-end collection module (12) for mounting on a mobile patrol vehicle (11), a positioning module (13), a power supply module (14) and a data collection computer (15), the power supply module (14) being adapted to supply power to the consumers in the on-board collection module (1), wherein:

the front-end acquisition module (12) is in communication connection with the data collection computer (15) and is used for acquiring the road facility images and sending the road facility images to the data collection computer (15);

the positioning module (13) is in communication connection with the data collection computer (15) and is used for acquiring positioning information and sending the positioning information to the data collection computer (15);

a data collection computer (15) is communicatively connected to the database server (2) for collecting the asset images and the location information and transmitting the asset images and the location information as asset information to the database server (2).

3. The road safety equipment patrol system according to claim 2, wherein the front-end acquisition module (12) comprises a mounting box (121) and a front-view camera (122), a side-view camera (123), an oblique-view camera (124) and a far-view camera (125) mounted to the mounting box (121), the mounting box (121) being configured to be fixedly mounted on top of the mobile patrol vehicle (11), the front-view camera (122), the side-view camera (123), the oblique-view camera (124) and the far-view camera (125) each being communicatively connected to a data collection computer (15), wherein:

the front-view camera (122) is arranged on the front side of the mounting box (121) and is used for collecting road facility images in the front of the mobile inspection vehicle (11);

the side-looking cameras (123) are arranged on two sides of the mounting box (121) and are used for collecting road facility images on two sides of the mobile inspection vehicle (11);

the oblique-looking camera (124) is arranged at the oblique angle of the mounting box (121) and is used for collecting road facility images of the mobile inspection vehicle (11) in the oblique-looking direction;

the far-vision camera (125) is arranged at the top of the mounting box (121) and used for collecting road facility images in the far front of the mobile patrol vehicle (11).

4. The road safety facility patrol system according to claim 1, wherein the management terminal (4) comprises a fixed terminal (41) and/or a mobile terminal (42);

the fixed terminal (41) is in communication connection with the routing equipment (5) at the application server (3) end through the routing equipment (5) through the Internet, so that communication connection with the application server (3) is established;

the mobile terminal (42) is connected with the routing equipment (5) at the end of the application server (3) through the mobile base station (6) through internet communication, thereby establishing communication connection with the application server (3).

5. A road safety facility patrol method applied to the road safety facility patrol system according to any one of claims 1 to 4, comprising:

6. The asset patrolling method according to claim 5, wherein the asset information includes an asset image and positioning information;

7. The road safety facility patrol method according to claim 5, wherein the abnormality identification result includes abnormality basic information and/or abnormality cloud map information;

8. The utility model provides a road safety facility inspection device which characterized in that, includes data call module, unusual identification module and result feedback module, wherein:

9. A computer device, comprising: a memory and one or more processors;

the memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the asset patrolling method of any one of claims 5 to 7.

10. A storage medium containing computer-executable instructions for performing the method of road safety line patrol according to any one of claims 5 to 7 when executed by a computer processor.