CN115311637A

CN115311637A - Pavement dirt loss and marking wear detection method and system

Info

Publication number: CN115311637A
Application number: CN202210956107.8A
Authority: CN
Inventors: 张晓明; 李祥勇; 丁新慧; 陈勇勇; 周审章; 钟盛
Original assignee: Shanghai Tongluyun Transportation Technology Co ltd
Current assignee: Shanghai Tongluyun Transportation Technology Co ltd
Priority date: 2022-08-10
Filing date: 2022-08-10
Publication date: 2022-11-08

Abstract

The invention relates to the technical field of road engineering, and particularly discloses a method and a system for detecting road surface dirt loss and marked line abrasion, wherein the method comprises the steps of obtaining a road surface image; inputting the pavement image into a trained convolutional neural network, and extracting a pavement marking area image; extracting a local image from the preprocessed pavement marking area image based on the movable sliding window; inputting the local images into the trained anomaly recognition model one by one to obtain anomaly parameters and uploading the anomaly parameters to a cloud end; the exception parameters comprise an exception type and an exception position; the abnormal type includes road surface dirt and marking wear. Compared with the traditional manual inspection, the method and the system for identifying the fouling and the abrasion of the highway pavement based on the deep convolutional neural network provided by the invention can identify the damage condition of the pavement marking more quickly, efficiently and objectively. The inspection personnel only need drive the vehicle of patrolling and examining and keep the speed of a motor vehicle to march, alright discover the damaged condition of road marking in real time.

Description

Pavement dirt loss and marking wear detection method and system

Technical Field

The invention relates to the technical field of road engineering, in particular to a method and a system for detecting road surface dirt loss and marked line abrasion.

Background

At present, the technology for identifying the fouling and the wear of the marked lines of the highway pavement based on the deep convolutional neural network is researched less, and the fouling and the wear of the marked lines of the highway pavement basically depend on manual inspection. It requires highway maintenance personnel to take the vehicle and manually inspect the sections of each highway one by one.

The method can not only ensure the personal safety of maintenance personnel, but also interfere the normal running traffic flow, and in addition, the method has huge engineering quantity, wastes time and labor, has low efficiency, and can not ensure the timeliness, the periodicity and the full coverage of detection.

Disclosure of Invention

The invention aims to provide a method and a system for detecting road surface pollution damage and marked line abrasion, so as to solve the problems in the background technology.

In order to achieve the purpose, the invention provides the following technical scheme:

a method of detecting road surface dirt and marking wear, the method comprising:

acquiring a road surface image according to a preset acquisition frequency; the road surface image is an RGB image;

inputting the road surface image into a trained convolutional neural network, and extracting a road surface marking area image;

preprocessing the pavement marking area image, and extracting a local image from the preprocessed pavement marking area image on the basis of a movable sliding window;

inputting the local images into the trained abnormal recognition model one by one to obtain abnormal parameters and uploading the abnormal parameters to a cloud end; the exception parameters comprise an exception type and an exception position; the abnormal type includes road surface dirt and marking wear.

As a further scheme of the invention: the step of acquiring the road surface image according to the preset acquisition frequency comprises the following steps:

establishing a connecting channel with a positioning information system, and acquiring vehicle speed data in real time;

and adjusting the acquisition frequency according to the vehicle speed data.

calculating a travel distance of the vehicle in real time based on a pre-installed rotary encoder;

and adjusting the acquisition frequency according to the travel distance.

As a further scheme of the invention: the step of preprocessing the road marking area image and extracting a local image from the preprocessed road marking area image based on a movable sliding window comprises the following steps:

carrying out normalization processing and light equalization processing on the image size according to the image content;

based on the mobile sliding window, extracting local images with fixed length from the road marking area images after normalization processing and light equalization processing in sequence; the fixed length is a preset value;

the normalization processing step comprises the steps of scaling the size of the pavement marking area images to enable the size of the images to be uniform to be X X Y; wherein X and Y are the length and width of the image.

The technical scheme of the invention also provides a pavement dirt loss and marking wear detection system, which comprises:

the image acquisition module is used for acquiring a road surface image according to a preset acquisition frequency; the road surface image is an RGB image;

the first extraction module is used for inputting the road surface image into a trained convolutional neural network and extracting a road surface marking area image;

the second extraction module is used for preprocessing the road marking area image and extracting a local image from the preprocessed road marking area image on the basis of the movable sliding window;

the recognition storage module is used for inputting the local images into the trained abnormal recognition model one by one to obtain abnormal parameters and uploading the abnormal parameters to the cloud end; the exception parameters comprise an exception type and an exception position; the abnormal type includes road surface dirt and marking wear.

As a further scheme of the invention: the image acquisition module includes:

the vehicle speed acquisition unit is used for establishing a connection channel with the positioning information system and acquiring vehicle speed data in real time;

and the first execution unit is used for adjusting the acquisition frequency according to the vehicle speed data.

As a further scheme of the invention: the image acquisition module includes:

a distance acquisition unit for calculating a travel distance of the vehicle in real time based on a pre-installed rotary encoder;

and the second execution unit is used for adjusting the acquisition frequency according to the travel distance.

As a further scheme of the invention: the second extraction module comprises:

the image processing unit is used for carrying out normalization processing and light equalization processing on the image size according to the image content;

the sliding window intercepting unit is used for sequentially extracting a local image with a fixed length from the road marking area image after normalization processing and light equalization processing based on the moving sliding window; the fixed length is a preset value;

Compared with the prior art, the invention has the beneficial effects that: the recognition system provided by the invention only needs the detection personnel to concentrate on driving, does not need to observe the abnormal condition of the road surface, and greatly reduces the safety risk of the detection personnel; maintenance personnel can timely and efficiently locate the stained or worn road surface based on the road surface abnormal information of the cloud database, and data support is provided for timely repairing the road surface marked lines; the inspection frequency and efficiency can be improved, the inspection procedure can be simplified, and the detection cost continuously invested in the later period is saved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention.

FIG. 1 is a schematic diagram of a pavement damage and marking wear detection system.

FIG. 2 is a block flow diagram of a method for detecting road surface soiling and marking wear.

Fig. 3 is a block diagram of the structure of the road surface dirt and marking wear detection system.

FIG. 4 is a schematic view of a pavement damage and marking wear detection system.

Detailed Description

In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Example 1

Fig. 2 is a block flow diagram of a method for detecting road surface dirt and marking wear, in an embodiment of the present invention, the method for detecting road surface dirt and marking wear includes:

referring to fig. 1 and 4, a road image is collected by a detection vehicle with a mobile function, and a monocular high-definition camera, a central industrial personal computer and a cloud server are installed on the detection vehicle, and the relationship is shown in fig. 1. Wherein:

(1) The monocular high-definition camera is used for collecting highway pavement images. During the process of the detection vehicle, the monocular high-definition camera continuously shoots high-definition pictures. The pixels of the high-definition camera are larger than two million pixels, a central industrial personal computer or vehicle-mounted electric power can be used for supplying power, and the high-definition camera can be installed inside or outside a vehicle.

(2) The central industrial personal computer comprises a fourth generation or fifth generation mobile communication technology, an information acquisition system, a high-precision positioning system and a miniature image processor. The function is to send an image acquisition instruction to the monocular high-definition camera and continuously receive, process and upload acquired data at the same time. The mounting position is the vehicle interior.

Inputting the pavement image into a trained convolutional neural network, and extracting a pavement marking area image;

in the process of extracting the pavement marking area image from the pavement image, the number of training samples can be continuously increased by means of the conventional convolutional neural network, so that the identification accuracy of the pavement marking area image can be increased;

the consistency and continuity of the input images can be kept by the moving sliding window method;

inputting the local images into the trained anomaly recognition model one by one to obtain anomaly parameters and uploading the anomaly parameters to a cloud end; the exception parameters comprise an exception type and an exception position; the anomaly types include road surface dirt damage and marking wear.

The anomaly identification model is also a deep convolutional neural network and aims to identify road surface fouling and marking wear, and for the preprocessed local images of the road surface markings, the images are classified into wear or fouling, health and other conditions based on the deep convolutional neural network with complete training.

As shown in fig. 2, in an example of the technical solution of the present invention, a recognition system is started, and during the detection of the vehicle driving, a monocular high-definition camera collects RBG images according to the instruction of a central industrial personal computer or the vehicle driving speed, recognizes a road marking, and extracts an image of a road marking area. And after the pavement marking area image is preprocessed, a local image is extracted by utilizing the movable sliding window. And inputting the local images into the deep convolution neural network pavement fouling and marking wear identification model one by one. And if the model identifies fouling or abrasion, recording the abnormal position and uploading the abnormal position to the cloud.

Specifically, the central industrial personal computer uploads the recorded road surface fouling and marking wear data to the cloud server, and the data content comprises damaged images, damaged types, position information of vehicles, traveling speeds of the vehicles, course angles of the vehicles and the like. After the road surface fouling and marking wear data are uploaded to the cloud server, the cloud server records the data in detail and sends messages to the relevant platform and the service system. And effective information is provided for timely repairing pavement fouling and marking abrasion.

As a preferred embodiment of the technical solution of the present invention, the step of acquiring the road surface image according to a preset acquisition frequency includes:

and adjusting the acquisition frequency according to the vehicle speed data.

and adjusting the acquisition frequency according to the travel distance.

The device is additionally provided with an acquisition frequency adjusting function, and two acquisition frequency adjusting modes are provided, wherein the first mode is adjusted according to the vehicle speed, and the second mode is modulated according to an acquisition signal sent by a central industrial personal computer. When the vehicle speed is adjusted, the acquisition system firstly acquires the vehicle speed data of the detection vehicle according to the positioning information system and dynamically adjusts the image acquisition frequency based on the real-time vehicle speed. When the vehicle-mounted high-definition monocular camera is adjusted according to the central industrial personal computer, the acquisition system calculates the advancing distance of the vehicle by using the central industrial personal computer and a rotary encoder arranged on the wheels of the detection vehicle, and takes a shooting instruction to the vehicle-mounted high-definition monocular camera according to the advancing distance.

As a preferred embodiment of the technical solution of the present invention, the step of preprocessing the road marking region image and extracting a local image from the preprocessed road marking region image based on the moving sliding window includes:

The method for preprocessing the pavement marking image is to perform normalization and light equalization processing on the image size according to the image content. The reason for this is that the difference in the position of the road marking causes the difference in the length and width of the road marking image obtained by recognition and division, and the difference in illumination. Therefore, the size problem is dealt with first. And stretching or filling the high-definition image to make the image size uniform to X X Y. Wherein X and Y are the length and width values of the image. The value can be determined according to actual conditions. And then, the problem of uneven illumination is solved by using a homomorphic filtering and light-equalizing algorithm.

The moving sliding window is that the preprocessed image is sampled in sequence by a sliding window with a fixed width. After an image with a fixed length L is obtained in each wheel, the image is input into a deep convolution neural network road surface fouling and marking wear identification model, and the state of the road surface marking in the sliding window is judged. And after the judgment is finished, the sliding window continuously moves by a fixed step length L along the specified direction to obtain an image in the next window. The fixed length L is a value determined by practical experimental conditions.

Example 2

Fig. 3 is a block diagram of a component structure of a road surface dirt and marking wear detection system, in an embodiment of the present invention, the road surface dirt and marking wear detection system includes:

the identification storage module is used for inputting the local images into the trained abnormal identification model one by one to obtain abnormal parameters and uploading the abnormal parameters to the cloud end; the exception parameters comprise an exception type and an exception position; the abnormal type includes road surface dirt and marking wear.

Further, the image acquisition module comprises:

Specifically, the image acquisition module includes:

Further, the second extraction module includes:

the sliding window intercepting unit is used for sequentially extracting a local image with a fixed length from the road marking area image after normalization processing and light equalization processing based on the movable sliding window; the fixed length is a preset value;

The functions which can be realized by the method for detecting the road surface dirt and the marked line abrasion are all completed by computer equipment, the computer equipment comprises one or more processors and one or more memories, and at least one program code is stored in the one or more memories and is loaded and executed by the one or more processors so as to realize the functions of the method for detecting the road surface dirt and the marked line abrasion.

The processor fetches instructions and analyzes the instructions from the memory one by one, then completes corresponding operations according to the instruction requirements, generates a series of control commands, enables all parts of the computer to automatically, continuously and coordinately act to form an organic whole, realizes the input of programs, the input of data, the operation and the output of results, and the arithmetic operation or the logic operation generated in the process is completed by the arithmetic unit; the Memory comprises a Read-Only Memory (ROM) for storing a computer program, and a protection device is arranged outside the Memory.

Illustratively, a computer program can be partitioned into one or more modules, which are stored in memory and executed by a processor to implement the present invention. One or more of the modules may be a series of computer program instruction segments capable of performing certain functions, the instruction segments being used to describe the execution of the computer program in the terminal device.

It will be appreciated by those skilled in the art that the above description of the serving device is merely an example and does not constitute a limitation of the terminal device, and may include more or less components than those described above, or some of the components may be combined, or different components may include, for example, input output devices, network access devices, buses, etc.

The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like, which is the control center of the terminal equipment and connects the various parts of the entire user terminal using various interfaces and lines.

The memory may be used to store computer programs and/or modules, and the processor may implement various functions of the terminal device by operating or executing the computer programs and/or modules stored in the memory and calling data stored in the memory. The memory mainly comprises a storage program area and a storage data area, wherein the storage program area can store an operating system, application programs (such as an information acquisition template display function, a product information publishing function and the like) required by at least one function and the like; the storage data area may store data created according to the use of the berth-state display system (e.g., product information acquisition templates corresponding to different product types, product information that needs to be issued by different product providers, etc.), and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.

The terminal device integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the modules/units in the system according to the above embodiment may be implemented by a computer program, which may be stored in a computer-readable storage medium and used by a processor to implement the functions of the embodiments of the system. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, recording medium, U.S. disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution media, and the like.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one of 8230, and" comprising 8230does not exclude the presence of additional like elements in a process, method, article, or apparatus comprising the element.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are also included in the scope of the present invention.

Claims

1. A pavement dirt loss and marking wear detection method is characterized by comprising the following steps:

inputting the local images into the trained anomaly recognition model one by one to obtain anomaly parameters and uploading the anomaly parameters to a cloud end; the exception parameters comprise an exception type and an exception position; the abnormal type includes road surface dirt and marking wear.

2. The method for detecting road surface dirt and marking wear according to claim 1, wherein the step of acquiring the road surface image according to a preset acquisition frequency comprises:

and adjusting the acquisition frequency according to the vehicle speed data.

3. The method for detecting road surface dirt and marking wear according to claim 1, wherein the step of acquiring the road surface image according to a preset acquisition frequency comprises:

and adjusting the acquisition frequency according to the travel distance.

4. The method for detecting road surface dirt damage and marking wear according to claim 1, wherein the step of preprocessing the road marking area image and extracting a local image in the preprocessed road marking area image based on a moving sliding window comprises:

5. A pavement marking soiling and wear detection system, the system comprising:

the second extraction module is used for preprocessing the pavement marking area image and extracting a local image from the preprocessed pavement marking area image on the basis of the movable sliding window;

6. The pavement marking wear detection system of claim 5, wherein the image capture module comprises:

7. The pavement marking wear detection system of claim 5, wherein the image capture module comprises:

8. The pavement marking wear detection system of claim 5, wherein the second extraction module comprises:

the normalization processing step comprises the steps of carrying out size scaling on the pavement marking area images, so that the sizes of the images are unified into X X Y; wherein X and Y are the length and width of the image.