CN114283383A

CN114283383A - Smart city highway maintenance method, computer equipment and medium

Info

Publication number: CN114283383A
Application number: CN202111630622.9A
Authority: CN
Inventors: 张丽娟; 杨旭光
Original assignee: Hebei Polytechnic Institute
Current assignee: Hebei Polytechnic Institute
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2022-04-05

Abstract

The application relates to a smart city highway maintenance method, computer equipment and medium, comprising the following steps: acquiring a monitoring video corresponding to a preset road section; determining multimedia information containing road surface and multimedia information containing vehicle information based on the monitoring video corresponding to the preset road section; screening out at least one piece of image information from multimedia information comprising a road surface; carrying out gray level processing on at least one piece of image information to obtain image information subjected to gray level processing; carrying out speed identification and/or driving route identification on multimedia information containing vehicle information to obtain a speed identification result and/or a driving route result; determining whether the road surface of the preset road section has defects or not based on at least one of the speed identification result and the driving route result and the image information after the gray processing; if the defect exists, determining the road surface defect information, wherein the road surface defect information comprises the following components: road surface defect type information and road surface defect position information; and determining a corresponding road maintenance strategy based on the road surface defect information.

Description

Smart city highway maintenance method, computer equipment and medium

Technical Field

The application relates to the technical field of computers, in particular to a smart city highway maintenance method, computer equipment and a medium.

Background

With the rapid development of capital construction projects in China, a plurality of roads are constructed in a plurality of cities, and at present, road networks of the roads are basically formed. The road must be maintained scientifically and reasonably to ensure that the road keeps a good operation state, and the road needs to be maintained in time in order to ensure the safety, functionality and structure of the road. Highway maintenance refers to the maintenance and repair of highways, the maintenance side being directed to the overall maintenance from the building of a traffic vehicle and the maintenance side being directed to the repair of damaged parts. Further, the maintenance of the road must repair the damaged part in time, otherwise, the investment of repair engineering is increased, the service life of the road is shortened, and the road users are lost. Therefore, how to find out the damage of the road in time and repair the road in time becomes a key problem.

Typically, road patrols are performed by highway maintenance personnel to determine whether and where damage exists to a highway, or to schedule maintenance by the masses to reflect which highway has damage.

The inventor finds in the research that: whether road patrol is performed by road maintenance personnel to determine whether a road is damaged or not, or whether the road is damaged or not is found through reflection of the masses, the damage cannot be found in time, so that the road can be found and maintained under the condition that the road is damaged seriously, and the cost for maintaining the road can be increased.

Disclosure of Invention

The application aims to provide a smart city highway maintenance method, computer equipment and a medium, which are used for solving the technical problems.

The above object of the present application is achieved by the following technical solutions:

in a first aspect, a smart city highway maintenance method is provided, which includes:

acquiring a monitoring video corresponding to a preset road section;

determining multimedia information containing road surface and multimedia information containing vehicle information based on the monitoring video corresponding to the preset road section;

screening out at least one piece of image information from the multimedia information containing the road surface;

carrying out gray level processing on the at least one piece of image information to obtain image information subjected to gray level processing;

carrying out speed identification and/or driving route identification on the multimedia information containing the vehicle information to obtain a speed identification result and/or a driving route result;

determining whether a road surface of a preset road section has a defect or not based on at least one of the speed identification result and the driving route result and the image information after the gray processing;

if the defect exists, determining road surface defect information, wherein the road surface defect information comprises: road surface defect type information and road surface defect position information;

and determining a corresponding road maintenance strategy based on the road surface defect information.

In one possible implementation manner, the determining whether the road surface of the preset road section has a defect based on the image information after the gray processing includes:

determining pixel values corresponding to all pixels in the image information after the gray processing;

converting the image information after the gray processing into a depth image;

and determining whether the current road surface has defects or not based on the pixel values respectively corresponding to the pixels and the depth image.

In another possible implementation manner, the speed recognition result includes: speed change information of at least one vehicle during travel, the travel route results comprising: the running route information corresponding to at least one vehicle;

determining whether a road surface of a preset road section has a defect based on the speed recognition result and the driving route result, including:

determining a road section meeting a first preset condition based on speed change information of the at least one vehicle in the driving process;

determining actual driving route information corresponding to at least one vehicle in the road sections meeting the first preset condition;

determining an offset of the actual driving route;

if the offset of the actual driving route meets a second preset condition, determining that the road surface of a preset road section has defects;

wherein, the road section meeting the first preset condition comprises: the speed descending amplitude is larger than the road section corresponding to the preset amplitude; the second preset condition includes: the offset of the driving route is larger than the preset offset.

In another possible implementation, determining an offset of an actual travel route of any vehicle includes:

identifying a lane line in the road sections meeting the first preset condition; and/or the presence of a gas in the gas,

recognizing lane edge lines in the road sections meeting the first preset condition; and/or the presence of a gas in the gas,

identifying the shape of the road section meeting the first preset condition;

determining the driving direction of any vehicle at a preset reference point and the position information of the preset reference point, wherein the preset reference point is a reference point corresponding to any vehicle entering a road section meeting a first preset condition;

predicting a driving route based on at least one of the lane line, the lane edge line, and the shape of the section satisfying a first preset condition, a driving direction of the any vehicle at a preset point, and position information of the preset reference point;

determining an offset amount of an actual travel route of the any vehicle based on the predicted travel route and the actual travel route.

In another possible implementation manner, the determining the offset of the actual driving route of any vehicle based on the predicted driving route and the actual driving route includes at least one of the following:

determining an offset amount between each first reference point and each corresponding second reference point, and determining an offset amount of the actual travel route of any one of the vehicles based on the offset amount between each first reference point and each corresponding second reference point, wherein the first reference point is a reference point on the predicted travel route, and the second reference point is a reference point on the actual travel route;

and determining curvature information of the predicted travel route and curvature information of the actual travel route, and determining an offset of the actual travel route of any vehicle based on the position information of the preset reference point, the curvature information of the predicted travel route and the curvature information of the actual travel route.

In another possible implementation manner, the determining the road surface defect information includes:

if the speed descending amplitude is larger than the preset amplitude and/or the offset of the actual running route is not in the preset offset range, determining a first position, wherein the first position is a position where the speed descending amplitude is larger than the preset amplitude and/or a position where the offset of the actual running route is not in the preset offset range;

screening at least one image meeting a third preset condition from the images based on the first position, and predicting whether a convex area and/or a concave area exist on the current road surface based on the at least one image meeting the third preset condition, wherein the image meeting the third preset condition comprises: an image containing the current road surface;

if yes, determining that the current defect information is a convex type and/or a concave type, and determining attribute information of a convex region and/or attribute information of a concave region, wherein the attribute information of the convex region comprises: the position information corresponding to the raised area, the raised area range of the raised area and the raised height of the raised area; the attribute information of the recess region includes: the position information corresponding to the recessed area, the recessed range of the recessed area and the recessed depth of the recessed area.

In another possible implementation manner, determining attribute information of the recessed area based on at least one image satisfying a third preset condition includes:

determining the position relation between the recessed area and the camera based on the at least one image meeting a third preset condition, and determining the position information of the recessed area based on the position information of the camera and the position relation between the recessed area and the camera;

performing gradient detection on the at least one image meeting a third preset condition to obtain gradient information, and performing non-maximum suppression processing on the gradient information to obtain a processing result;

determining a depression range of the depression region and a depression depth of the depression region based on the processing result.

In another possible implementation manner, determining attribute information of the convex region based on at least one image satisfying a third preset condition includes:

determining the relation between the raised area and the camera based on the at least one image meeting the third preset condition, and determining the position information of the raised area based on the position information of the camera and the relation between the raised area and the camera;

and extracting the features of the raised area, and determining the raised type of the raised area, the raised area range of the raised area and the raised height of the raised area based on the extracted features.

In another possible implementation manner, the determining a corresponding road maintenance strategy based on the road surface defect information includes:

acquiring vehicle information and road surface thickness information of a road section with a road surface defect;

determining a road maintenance strategy based on the road defect information, the road thickness information and the vehicle information of the road section with the road defect, wherein the road maintenance strategy comprises the following steps: the time of pavement maintenance, the location of pavement maintenance, the tools of pavement maintenance, the materials of pavement maintenance, and the time required to complete pavement maintenance.

In a second aspect, a smart city highway maintenance device is provided, including:

the acquisition module is used for acquiring a monitoring video corresponding to a preset road section;

the first determining module is used for determining multimedia information containing road surfaces and multimedia information containing vehicle information based on the monitoring videos corresponding to the preset road sections;

the screening module is used for screening out at least one piece of image information from the multimedia information containing the road surface;

the gray processing module is used for carrying out gray processing on the at least one piece of image information to obtain image information after the gray processing;

the identification module is used for carrying out speed identification and/or driving route identification on the multimedia information containing the vehicle information to obtain a speed identification result and/or a driving route result;

the second determining module is used for determining whether the road surface of the preset road section has defects or not based on at least one of the speed identification result and the driving route result and the image information after the gray processing;

a third determining module for determining road surface defect information when a defect exists, the road surface defect information including: road surface defect type information and road surface defect position information;

and the fourth determining module is used for determining a corresponding road maintenance strategy based on the road surface defect information.

In a possible implementation manner, when determining whether the road surface of the preset road segment has a defect based on the image information after the grayscale processing, the second determining module is specifically configured to:

converting the image information after the gray processing into a depth image;

the second determining module is specifically configured to, when determining whether a road surface of a preset road section is defective based on the speed recognition result and the driving route result:

determining an offset of the actual driving route;

In another possible implementation manner, the second determining module, when determining the offset of the actual driving route of any vehicle, is specifically configured to:

identifying the shape of the road section meeting the first preset condition;

In another possible implementation manner, the second determining module is specifically configured to, when determining the offset of the actual driving route of any vehicle based on the predicted driving route and the actual driving route, at least one of the following:

In another possible implementation manner, when determining the road surface defect information, the third determining module is specifically configured to:

when the speed descending amplitude is larger than the preset amplitude and/or the offset of the actual running route is not in the preset offset range, determining a first position, wherein the first position is a position where the speed descending amplitude is larger than the preset amplitude and/or a position where the offset of the actual running route is not in the preset offset range;

when a convex region and/or a concave region exists, determining that the current defect information is a convex type and/or a concave type, and determining attribute information of the convex region and/or attribute information of the concave region, wherein the attribute information of the convex region comprises: the position information corresponding to the raised area, the raised area range of the raised area and the raised height of the raised area; the attribute information of the recess region includes: the position information corresponding to the recessed area, the recessed range of the recessed area and the recessed depth of the recessed area.

In another possible implementation manner, when determining the attribute information of the recessed area based on at least one image satisfying a third preset condition, the third determining module is specifically configured to:

In another possible implementation manner, when determining the attribute information of the convex region based on at least one image satisfying a third preset condition, the third determining module is specifically configured to:

In another possible implementation manner, when determining the corresponding road maintenance strategy based on the road surface defect information, the fourth determining module is specifically configured to:

In a third aspect, a computer device is provided, comprising:

at least one processor;

a memory;

at least one application, wherein the at least one application is stored in the memory and configured to be executed by the at least one processor, the at least one application configured to: a smart city highway maintenance method according to the first aspect is performed.

In a fourth aspect, there is provided a computer readable storage medium having stored thereon at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by the processor to implement a method of smart urban highway maintenance according to the first aspect.

In summary, the present application includes at least one of the following beneficial technical effects:

compared with the related art, the intelligent city highway maintenance method, the intelligent city highway maintenance computer equipment and the intelligent city highway maintenance medium have the advantages that the monitoring video corresponding to the preset road section is processed to obtain the multimedia information containing the road surface and the multimedia information containing the vehicle information, the multimedia information containing the road surface is subjected to gray scale processing to obtain the image subjected to gray scale processing, the multimedia information containing the vehicle information is subjected to vehicle speed identification and/or vehicle driving route identification to obtain a speed identification result and/or a driving route result, whether the road surface of the preset road section has defects or not and corresponding road surface defect information can be determined based on the information, a corresponding highway maintenance strategy can be determined according to the road surface defect information, and the efficiency of finding the damage to the highway can be improved, and further, the cost for maintaining the road can be reduced.

Drawings

Fig. 1 is a schematic flow chart illustrating a smart city highway maintenance method according to an embodiment of the present disclosure;

fig. 2 is a schematic structural view of a smart city highway maintenance device according to an embodiment of the present disclosure;

fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The present application is described in further detail below with reference to the attached drawings.

The present embodiment is only for explaining the present application, and it is not limited to the present application, and those skilled in the art can make modifications of the present embodiment without inventive contribution as needed after reading the present specification, but all of them are protected by patent law within the scope of the claims of the present application.

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In addition, the term "and/or" herein is only one kind of association relationship describing an associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship, unless otherwise specified.

The embodiments of the present application will be described in further detail with reference to the drawings attached hereto.

As shown in fig. 1, the smart city highway maintenance method provided in the embodiment of the present application may be executed by a computer device, where the computer device may be a server or a terminal device, where the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud computing services. The terminal device may be a smart phone, a tablet computer, a notebook computer, a desktop computer, and the like, but is not limited thereto, and the terminal device and the server may be directly or indirectly connected through wired or wireless communication, and the embodiment of the present application is not limited thereto, and the method includes:

and S101, acquiring a monitoring video corresponding to a preset road section.

Specifically, the preset road section may be carried in a road maintenance request triggered by a user, that is, when detecting road maintenance request information triggered by the user, acquiring a monitoring video corresponding to the preset road section based on the request information; the relationship between the road section and the maintenance time may also be preset, that is, when the maintenance time for the preset road section is reached, the monitoring video corresponding to the preset road section is obtained.

Further, a mapping relation between the road section and the camera is preset, camera information corresponding to the preset road section is obtained, and a monitoring video corresponding to the preset road section is obtained based on the camera information corresponding to the preset road section. In this embodiment of the application, the monitoring video corresponding to the preset road segment is not limited to the monitoring video shot by one camera, and may be the monitoring video shot by at least two cameras, which is not limited in this embodiment of the application.

And S102, determining multimedia information containing road surface and multimedia information containing vehicle information based on the monitoring video corresponding to the preset road section.

Specifically, when a surveillance video corresponding to a preset road section is obtained, target recognition or target detection can be performed on each video frame in the surveillance video respectively to determine multimedia information including a road surface and multimedia information including vehicle information; or when the surveillance video corresponding to the preset road section is obtained, extracting the multi-frame video frames from the surveillance video corresponding to the preset road section, and performing target identification or target detection based on the extracted multi-frame video frames to determine the multimedia information including the road surface and the multimedia information including the vehicle information.

And S103, screening at least one piece of image information from the multimedia information containing the road surface.

For the embodiment of the application, since the multimedia information including the road surface may include many frames of images, in order to reduce the complexity of the gray scale processing, at least one piece of image information may be screened from the included multimedia information, and the gray scale processing may be performed on the at least one piece of image information subsequently. Specifically, the manner of screening out at least one piece of image information from the multimedia information including the road surface may include: and randomly screening at least one piece of image information from the multimedia information containing the road surface, or screening at least one piece of image information containing the complete road surface from the multimedia information containing the road surface.

And step S104, carrying out gray level processing on at least one piece of image information to obtain image information after the gray level processing.

Specifically, in the embodiment of the present application, the grayscale processing is performed on each piece of image information in at least one piece of image information to obtain the grayscale-processed image information, and the manner of the grayscale processing is not limited in the embodiment of the present application.

And step S105, carrying out speed identification and/or driving route identification on the multimedia information containing the vehicle information to obtain a speed identification result and/or a driving route result.

For the embodiment of the application, the speed identification mode of the multimedia information containing the vehicle information can be realized through the trained network model, and the speed identification of the vehicle can also be realized through other algorithms; similarly, the method for identifying the driving route of the multimedia information including the vehicle information can be identified through the trained network model, and the driving route of the vehicle can also be identified through other algorithms.

It should be noted that: step S105 may be performed before step S103, before step S104, after step S104, simultaneously with step S103, or simultaneously with step S104, where fig. 1 is only a schematic diagram of a possible execution sequence, and is not intended to limit the embodiments of the present application.

And S106, determining whether the road surface of the preset road section has defects or not based on at least one of the speed identification result and the driving route result and the image information after the gray processing.

With the embodiment of the present application, after at least one of the speed recognition result and the travel route result and the image information after the gradation processing are obtained by the above-described embodiment, it is determined whether the road surface of the preset road section has a defect based on these information. For example, it may be determined whether defects such as depressions in the road surface, bumps in the road surface, and/or cracks in the road surface are present.

And step S107, if the defects exist, determining the road surface defect information.

Wherein the road surface defect information includes: road surface defect type information and road surface defect position information.

And S108, determining a corresponding road maintenance strategy based on the road surface defect information.

For the embodiment of the application, the highway maintenance strategies corresponding to different types of the road defects are different, and the corresponding highway maintenance strategies may also be different due to different position information of the road defects. In the embodiment of the application, the types of the road defects are different, and the corresponding road maintenance tools, road maintenance materials and road maintenance modes are possibly different; the position information of the road surface defect is different, and the corresponding road maintenance time is also different.

Compared with the related art, the embodiment of the application provides a smart city highway maintenance method, the smart city highway maintenance method obtains multimedia information containing road surfaces and multimedia information containing vehicle information by processing monitoring videos corresponding to preset road sections, obtains images after gray level processing by performing gray level processing on the multimedia information containing road surfaces, obtains speed identification results and/or driving route identification on the multimedia information containing vehicle information, can determine whether the road surfaces of the preset road sections have defects or not and corresponding road defect information based on the information by performing vehicle speed identification and/or vehicle driving route identification on the multimedia information containing vehicle information, and can determine corresponding highway maintenance strategies according to the road defect information, so that the efficiency of finding out highway damages can be improved, and further, the cost for maintaining the road can be reduced.

Specifically, in the embodiment of the present application, steps S101 to S105 may be implemented in the manner described in the embodiment of the present application, and may also be implemented in the manner described in the related art.

Specifically, the step S106 of determining whether the road surface of the preset road section has a defect based on the image information after the gray processing may specifically include: step S1061 (not shown), step S1062 (not shown), and step S1063 (not shown), wherein,

step S1061, determining a pixel value corresponding to each pixel in the image information after the gray scale processing.

And step S1062, converting the image information after the gray level processing into a depth image.

Specifically, in the embodiment of the present application, the corresponding depth image may be determined by using one piece of image information after the gray processing, or the depth image may be determined by using at least two pieces of images after the gray processing. Specifically, the method for determining the depth image from the two gray-scale processed images may include: calculating the offset of pixel points at the same position corresponding to the target in the image after the first gray processing and the image after the second gray processing in the horizontal direction; according to the offset, depth information of the pixel points is obtained by using a calculation formula; and obtaining a corresponding depth image according to the depth information of all pixel points in the image after the first gray processing or the image after the second gray processing.

Specifically, step S1061 may be performed before step S1062, after step S1062, or simultaneously with step S1062, which is not limited in the embodiment of the present application.

And step S1063, determining whether the current road surface has defects or not based on the pixel values respectively corresponding to the pixels and the depth image.

Specifically, determining whether the road surface has a defect based on the pixel value and the depth image respectively corresponding to each pixel may specifically include: and obtaining the distribution of the gray value at each pixel based on the pixel value corresponding to each pixel, and determining whether the current road surface has defects based on the distribution of the gray value at each pixel and the depth image.

Specifically, determining whether the current road surface has a defect based on the distribution of the gray values at each pixel and the depth image may specifically include: determining the relative depth of each corresponding position based on the distribution of the gray value of each pixel, and determining whether the current road surface has defects based on the determined relative depth of each corresponding position; and determining whether the current road surface has defects or not based on the depth image, and if at least one of the defects can be determined, determining that the current road surface has defects.

Specifically, determining whether the current road surface has a defect based on the distribution of the gray values at each pixel and the depth image may further include: and determining the relative depth of each corresponding position based on the distribution of the gray values at each pixel, adjusting the depth image based on the relative depth of each corresponding position, and determining whether the current road surface has defects based on the adjusted depth image.

Further, the speed recognition result includes: speed variation information of at least one vehicle during travel, the travel route results comprising: the running route information corresponding to at least one vehicle; in this embodiment of the application, the determining whether the road surface of the preset road segment has a defect in step S106 based on the speed recognition result and the driving route result may specifically include: step S106a (not shown), step S106b (not shown), step S106c (not shown), and step S106d (not shown), in the present embodiment, steps S106a to S106d may be executed before steps S1061 to S1063, or after steps S1061 to S1063, or simultaneously with steps S1061 to S1063, or in other execution sequences, which are not limited in the present embodiment.

Step S106a, determining road sections meeting first preset conditions based on speed change information of at least one vehicle in the driving process.

Wherein, the highway section that satisfies first preset condition includes: the speed descending amplitude is larger than the road section corresponding to the preset amplitude. In this embodiment of the present application, the preset amplitude may be preset, may also be input by a user, and may also be calculated in a specific manner, which is not limited in this embodiment of the present application. Further, if the preset width is calculated based on a specific manner, it can be determined by the vehicle type and the prescribed vehicle speed information of the preset section.

Step S106, 106b, determining corresponding actual traveling route information of at least one vehicle in the road section meeting the first preset condition.

Step S106c, the offset of the actual travel route is determined. In the embodiment of the present application, determining the offset of the actual driving route may specifically include: an offset of the actual travel path from the predicted travel path is determined.

Specifically, determining the offset of the actual driving route of any vehicle may specifically include: at least one of step S1 (not shown), step S2 (not shown), and step S3 (not shown), step S4 (not shown), step S5 (not shown), and step S6 (not shown), wherein,

step S1, identifying a lane line in a road segment satisfying a first preset condition.

Specifically, the method comprises the steps of determining multimedia information corresponding to a road section meeting a first preset condition from the multimedia information comprising the road surface, and identifying the lane line based on the multimedia information corresponding to the road section meeting the first preset condition.

Further, in order to reduce the complexity of identifying the lane line and reduce the calculation pressure, at least one image may be screened from the multimedia information corresponding to the first preset condition, and the lane line may be identified based on the screened at least one image.

And step S2, recognizing lane edge lines in the road sections meeting the first preset condition.

Specifically, the lane edge lines are identified based on the multimedia information corresponding to the road sections meeting the first preset condition by determining the multimedia information corresponding to the road sections meeting the first preset condition from the multimedia information including the road surface.

Further, in order to reduce the complexity of recognizing the lane edge line and reduce the calculation pressure, at least one image may be screened from the multimedia information corresponding to the condition meeting the first preset condition, and the lane edge line may be recognized based on the screened at least one image.

And step S3, identifying the shape of the road section meeting the first preset condition.

Specifically, the shape of the road section meeting the first preset condition is identified based on the multimedia information corresponding to the road section meeting the first preset condition.

Further, in order to reduce complexity of identifying the shape of the road segment and reduce calculation pressure, at least one image may be screened from the multimedia information corresponding to the first preset condition, and the shape of the road segment meeting the first preset condition may be identified based on the screened at least one image.

For the embodiment of the present application, the multimedia information corresponding to the links meeting the first preset condition utilized in step S1, step S2, and step S3 may be the same multimedia information, or may be different multimedia information, or may be partially the same, and is not limited in the embodiment of the present application.

If steps S1, S2, and S3 are included, the execution sequence of steps S1, S2, and S3 may be any possible execution sequence, for example, steps S1, S2, and S3 may be executed simultaneously.

And step S4, determining the running direction of any vehicle at the preset reference point and the position information of the preset reference point.

The preset reference point is a reference point corresponding to any vehicle entering a road section meeting a first preset condition.

Step S5, predicting a driving route based on at least one of a lane line, a lane edge line, and a shape of a road segment satisfying a first preset condition, a driving direction of any one vehicle at a preset point, and position information of a preset reference point.

Specifically, the manner of predicting the driving route based on at least one of the lane line, the lane edge line, and the shape of the road segment satisfying the first preset condition, the driving direction of any vehicle at the preset point, and the position information of the preset reference point may specifically include predicting the driving route by using a trained driving route prediction model, and predicting the driving route by using another algorithm.

In step S6, an offset amount of the actual travel route of any vehicle is determined based on the predicted travel route and the actual travel route.

Specifically, in the embodiment of the present application, the offset of the actual driving route of any vehicle may specifically include: offset between each reference point in the actual travel route and a reference point of the predicted travel route corresponding to each reference point; may also include: a maximum offset between the actual travel route and the predicted travel route; the method can also comprise the following steps: an average offset between the actual travel route and the predicted travel route.

Specifically, determining the offset amount of the actual travel route of any one of the vehicles based on the predicted travel route and the actual travel route may specifically include at least one of step S61 (not shown in the figure) and step S62 (not shown in the figure), wherein,

and step S61, determining the offset between each first reference point and the corresponding second reference point, and determining the offset of the actual driving route of any vehicle based on the offset between each first reference point and the corresponding second reference point.

Wherein the first reference point is a reference point on the predicted travel route, and the second reference point is a reference point on the actual travel route.

For the embodiment of the present application, the determining the offset between each first reference point and each corresponding second reference point in step S61 can be implemented in at least two ways, wherein,

the first mode is as follows: each first reference point and each second reference point may be plotted on the predicted driving route and the actual driving route, respectively, by the engineer. Further, the amount of offset between each first reference point and the respective corresponding second reference point is determined based on the reference points plotted by the engineer on the predicted travel route and on the actual travel route.

In a second mode, the offset between each first reference point and the corresponding second reference point is determined by a network model.

Further, after the offset amounts (respective offset amounts) between the respective first reference points and the respective corresponding second reference points are obtained through the above embodiment, the respective offset amounts are determined as the offset amounts of any vehicle actual travel route, an average value corresponding to the respective offset amounts may be determined as the offset amount of any vehicle actual travel route, and a maximum offset amount among the respective offset amounts may be determined as the offset amount of any vehicle actual travel route.

And step S62, determining curvature information of the predicted travel route and curvature information of the actual travel route, and determining an offset amount of the actual travel route of any vehicle based on the position information of the preset reference point, the curvature information of the predicted travel route and the curvature information of the actual travel route.

Specifically, after the predicted travel route and the actual travel route are obtained by the above embodiment, the curvature information corresponding to each of the predicted travel route and the actual travel route may be determined, and after the curvature information corresponding to each of the predicted travel route and the actual travel route is obtained, the offset amount of the actual travel route of any vehicle may be determined based on the curvature information corresponding to each of the predicted travel route and the actual travel route and the position information of the preset reference point. In the embodiment of the application, the offset of the actual driving route of any vehicle can be determined through a network model, and the offset of the actual driving route of any vehicle can also be determined through a related curvature algorithm.

And S106, 106d, if the offset of the actual driving route meets a second preset condition, determining that the road surface of the preset road section has defects.

Wherein the second preset condition comprises: the offset of the driving route is larger than the preset offset.

Specifically, after the offset of the actual travel route is obtained, whether the road surface of the preset road section has a defect is predicted according to the offset of the actual travel route. In the embodiment of the application, when the offset of the actual driving route is larger than the preset offset, the defect of the road surface of the preset road section is predicted, and if the offset of the actual driving route is not larger than the preset offset, the defect of the road surface of the preset road section is predicted.

Specifically, if the offset of the actual driving route is each offset (the offset between each first reference point and the corresponding second reference point), if the offset which is greater than the preset threshold in each offset is greater than the preset offset, predicting that the road surface of the preset road section has a defect, otherwise predicting that the road surface of the preset road section has no defect; if the offset of the actual driving route is the average offset, determining that the road surface of the preset road section has defects when the average offset is larger than the preset offset, and otherwise predicting that the road surface of the preset road section has no defects; and if the offset of the actual route is the maximum offset, determining that the road surface of the preset road section has defects when the maximum offset is larger than the preset offset, and otherwise, predicting that the road surface of the preset road section has no defects.

Further, after determining that the road surface of the preset road section has a defect, it is necessary to further determine specific information of the road surface defect, such as the type of the defect and the position of the defect,

in this embodiment of the application, determining the road surface defect information may specifically include: step Sa (not shown), step Sb (not shown), and step Sc (not shown), wherein,

and step Sa, if the speed descending amplitude is larger than the preset amplitude and/or the offset of the actual driving route is not in the preset offset range, determining the first position.

The first position is a position where the speed descending amplitude is larger than a preset amplitude and/or a position where the offset of the actual driving route is not within a preset offset range.

For the embodiment of the application, if the speed descending amplitude is larger than the preset amplitude, determining the position of which the speed descending amplitude is larger than the preset amplitude as a first position; if the offset of the actual driving route is not within the preset offset range, determining that the position of the actual driving route which is not within the preset offset range is a first position; if the speed descending amplitude is larger than the preset amplitude and the offset of the actual running route is not in the preset offset range, determining the first position as the position where the speed descending amplitude is larger than the preset amplitude and the position where the offset of the actual running route is not in the preset offset range based on the first position.

And Sb, screening at least one image meeting a third preset condition from the images based on the first position, and predicting whether a convex region and/or a concave region exists on the current road surface based on the at least one image meeting the third preset condition.

Wherein, the image satisfying the third preset condition includes: including the image of the current road surface.

For the embodiment of the application, the road surface of the road section corresponding to the first position is called the current road surface, after the position information of the first position is determined, at least one image containing the current road surface is screened out from the multimedia information containing the road surface based on the position information of the first position, and whether a convex region and/or a concave region exists on the current road surface is predicted based on the screened at least one image.

Specifically, in the embodiment of the present application, the manner of screening out at least one image satisfying the third preset condition in step Sb and the manner of predicting whether a raised area and/or a depressed area exist on the current road surface may be implemented by a trained network model, or implemented by a related technology of image processing, which is not limited in the embodiment of the present application.

And step Sc, if the defect information exists, determining that the current defect information is a convex type and/or a concave type, and determining attribute information of a convex area and/or attribute information of a concave area.

Wherein the attribute information of the convex region includes: the position information corresponding to the raised area, the raised area range of the raised area and the raised height of the raised area; the attribute information of the recess region includes: the position information corresponding to the recessed area, the recessed range of the recessed area and the recessed depth of the recessed area.

Specifically, determining attribute information of the recessed area based on at least one image satisfying a third preset condition may specifically include: determining the position relation between the recessed area and the camera based on at least one image meeting a third preset condition, and determining the position information of the recessed area based on the position information of the camera and the position relation between the recessed area and the camera; performing gradient detection on at least one image meeting a third preset condition to obtain gradient information, and performing non-maximum suppression processing on the gradient information to obtain a processing result; the depression range of the depression region and the depression depth of the depression region are determined based on the processing result. In this embodiment of the present application, determining the depression range and the depression depth of the depression region may be performed before determining the position information of the depression region, or may be performed after determining the position information of the depression region, or may be performed simultaneously, which is not limited in this embodiment of the present application.

Specifically, the image is considered as a two-dimensional discrete function f (x, y), and the image gradient is actually the derivative of this two-dimensional discrete function (i.e., the derivative of f (x, y) is G (x, y)), where,

image gradient G (x, y) = dx (i, j) + dy (i, j);

dx(i,j) = I(i+1,j) - I(i,j)；

dy(i,j) = I(i,j+1) - I(i,j)；

where I is the value of an image pixel (e.g., RGB value) and (I, j) is the pixel's coordinates.

In another possible implementation, the image gradient may also be generally differentiated by a median value, as shown in detail below:

dx(i,j) = [I(i+1,j) - I(i-1,j)]/2；

dy(i,j) = [I(i,j+1) - I(i,j-1)]/2；

the gradient direction is the direction in which the image function f (x, y) changes most rapidly, when an edge exists in the image, a large gradient value is certain, conversely, when a smooth part exists in the image, the gray value change is small, the corresponding gradient is also small, and the mode of the gradient is called the gradient for short in the image processing. The gradient information in the embodiment of the present application may include: gradient magnitude and gradient direction.

To obtain an accurate edge we need to perform non-maxima suppression, determining only one local maximum and zeroing the others to remove the blur near the edge. In the embodiment of the present application, Non-Maximum Suppression (NMS), which is an element that suppresses a Non-Maximum value as the name implies, may be understood as a local Maximum search. The local representation is a neighborhood, and the neighborhood has two variable parameters, namely the dimension of the neighborhood and the size of the neighborhood.

Further, a final edge detection result may be obtained after passing through the maximum value suppression processing, and the depression range of the depression region may be determined based on the final edge detection result, for example, the depression range of the depression region may be determined by the farthest distance between the point and the upper point of the edge. Further, in the embodiment of the present application, after the final edge detection result is determined, the depth of the recess region is obtained through image depth detection.

Further, in this embodiment of the application, determining attribute information of the recessed area based on any one of the images satisfying the third preset condition may specifically include: determining a depth value corresponding to each pixel in any image meeting a third preset condition, determining a region where the pixel with the depth value smaller than the preset depth value is located, determining a concave range of the concave region based on the region where the pixel with the depth value smaller than the preset depth value is located, and further determining the concave depth of the concave region based on the depth value corresponding to each pixel point of the concave region. For example, the depression depth of the depression region is determined based on the maximum value of the depth values of the pixel points in the depression region.

Specifically, determining attribute information of the convex region based on at least one image satisfying a third preset condition may specifically include: determining the relation between the convex area and the camera based on at least one image meeting a third preset condition, and determining the position information of the convex area based on the position information of the camera and the relation between the convex area and the camera; feature extraction is performed on the raised area, and the type of the raised area, the range of the raised area, and the height of the raised area are determined based on the extracted features.

For the embodiment of the present application, the manner of determining the position of the convex region is similar to the manner of determining the position information of the concave region in the foregoing, and details are not repeated in the embodiment of the present application.

Further, the determining of the position information of the protrusion area may be performed before the determining of the protrusion type, the protrusion range, and the protrusion height of the protrusion area, or after the determining of the protrusion type, the protrusion range, and the protrusion height of the protrusion area, or may be performed simultaneously with the determining of the protrusion type, the protrusion range, and the protrusion height of the protrusion area, which is not limited in the embodiment of the present application.

Further, in the embodiment of the application, the image of the convex region is subjected to feature extraction, so as to extract brightness features, color features, texture features and the like, and further determine the convex type, convex range and convex height of the convex region according to the brightness features, the color features and the like. In the embodiment of the present application, the protrusion types may mainly include: road surface projections and non-road surface projections. In the embodiment of the application, the road surface bulge is used for representing the defect of the road surface, and the non-road surface bulge can be leaves, soil blocks and the like falling on the road surface and is not used for representing the defect of the road surface.

Further, after determining the road surface defect information, determining a corresponding road maintenance strategy based on the road surface defect information may specifically include: acquiring vehicle information and road surface thickness information of a road section with a road surface defect; and determining a highway maintenance strategy based on the road defect information, the road thickness information and the vehicle information of the road section with the road defect.

For the embodiment of the application, the mapping relation between each road and the thickness of the road surface is stored in advance, and even the mapping relation between each road section and the thickness of the road surface can be stored in advance. And after the road section with the road surface defect is determined, determining the thickness of the road surface with the road surface defect. Besides the pre-storage of the road surface thickness corresponding to each road, the thickness of the road surface can be determined through a network model or other image processing methods.

Further, after the monitoring video of the preset road section is obtained, the traffic flow of the preset road section can be determined, and the traffic flow corresponding to the preset road section in each time interval can also be determined. The manner of acquiring the traffic flow is not limited in the embodiment of the present application.

Further, after the vehicle information and the road thickness information of the road section with the road defect are obtained, the road maintenance strategy is determined through the trained network model based on the road defect information, the road thickness information and the vehicle information of the road section with the road defect. Furthermore, the road maintenance strategy determined under the conditions of the current road surface defect information, the road surface thickness information and the vehicle information of the road section with the road surface defect can be recorded in real time, the trained network model is updated based on the road maintenance strategy, so that a more accurate model is obtained, and the accuracy of determining the road maintenance strategy based on the model can be further improved.

Wherein, the highway maintenance strategy includes: the time of pavement maintenance, the location of pavement maintenance, the tools of pavement maintenance, the materials of pavement maintenance, and the time required to complete pavement maintenance.

The above embodiments describe a smart city highway maintenance method from the perspective of method flow, and the following embodiments describe a smart city highway maintenance device from the perspective of modules or units, which are described in detail in the following embodiments.

The embodiment of the application provides a wisdom city highway curing means, as shown in fig. 2, wisdom city highway curing means 20 can include: an acquisition module 21, a first determination module 22, a screening module 23, a gray scale processing module 24, a recognition module 25, a second determination module 26, a third determination module 27, and a fourth determination module 28, wherein,

the obtaining module 21 is configured to obtain a monitoring video corresponding to a preset road segment.

The first determining module 22 is configured to determine multimedia information including road surface information and multimedia information including vehicle information based on the surveillance video corresponding to the preset road segment.

And the screening module 23 is used for screening out at least one piece of image information from the multimedia information comprising the road surface.

And the gray processing module 24 is configured to perform gray processing on at least one piece of image information to obtain gray-processed image information.

And the identification module 25 is used for performing speed identification and/or driving route identification on the multimedia information containing the vehicle information to obtain a speed identification result and/or a driving route result.

And a second determining module 26, configured to determine whether the road surface of the preset road segment has a defect based on at least one of the speed recognition result and the driving route result and the image information after the grayscale processing.

And a third determining module 27 for determining road surface defect information when there is a defect.

And a fourth determining module 28, configured to determine a corresponding road maintenance strategy based on the road surface defect information.

Specifically, the first determining module 22, the second determining module 26, the third determining module 27, and the fourth determining module 28 may be the same determining module, may be different determining modules, or may be partially the same determining module, which is not limited in the embodiment of the present application.

In a possible implementation manner of the embodiment of the application, when determining whether a defect exists on the road surface of the preset road segment based on the image information after the gray processing, the second determining module 26 is specifically configured to:

converting the image information after the gray processing into a depth image;

In another possible implementation manner of the embodiment of the present application, the speed recognition result includes: speed variation information of at least one vehicle during travel, the travel route results comprising: the running route information corresponding to at least one vehicle;

the second determination module 26 is specifically configured to, when determining whether the road surface of the preset road segment has a defect based on the speed recognition result and the driving route result:

determining a road section meeting a first preset condition based on speed change information of at least one vehicle in the driving process;

determining actual driving route information corresponding to at least one vehicle in a road section meeting a first preset condition;

determining the offset of an actual driving route;

if the offset of the actual driving route meets a second preset condition, determining that the road surface of the preset road section has defects;

wherein, the highway section that satisfies first preset condition includes: the speed descending amplitude is larger than the road section corresponding to the preset amplitude; the second preset condition includes: the offset of the driving route is larger than the preset offset.

In another possible implementation manner of the embodiment of the present application, when determining the offset of the actual driving route of any vehicle, the second determining module 26 is specifically configured to:

identifying a lane line in a road section meeting a first preset condition; and/or the presence of a gas in the gas,

recognizing lane edge lines in a road section meeting a first preset condition; and/or the presence of a gas in the gas,

identifying the shape of a road section meeting a first preset condition;

predicting a driving route based on at least one of a lane line, a lane edge line, and a shape of a road section satisfying a first preset condition, a driving direction of any vehicle at a preset point, and position information of a preset reference point;

an offset amount of an actual travel route of any one of the vehicles is determined based on the predicted travel route and the actual travel route.

In another possible implementation manner of the embodiment of the application, when determining the offset of the actual driving route of any vehicle based on the predicted driving route and the actual driving route, the second determining module 26 is specifically configured to at least one of:

determining the offset between each first reference point and the corresponding second reference point, and determining the offset of the actual driving route of any vehicle based on the offset between each first reference point and the corresponding second reference point, wherein the first reference point is a reference point on the predicted driving route, and the second reference point is a reference point on the actual driving route;

and determining the curvature information of the predicted driving route and the curvature information of the actual driving route, and determining the offset of the actual driving route of any vehicle based on the position information of the preset reference point, the curvature information of the predicted driving route and the curvature information of the actual driving route.

In another possible implementation manner of the embodiment of the present application, when determining the road surface defect information, the third determining module 27 is specifically configured to:

screening at least one image meeting a third preset condition from the images based on the first position, and predicting whether a convex area and/or a concave area exist on the current road surface based on the at least one image meeting the third preset condition, wherein the image meeting the third preset condition comprises: an image containing a current road surface;

when the convex region and/or the concave region exist, determining that the current defect information is the convex type and/or the concave type, and determining the attribute information of the convex region and/or the attribute information of the concave region, wherein the attribute information of the convex region comprises: the position information corresponding to the raised area, the raised area range of the raised area and the raised height of the raised area; the attribute information of the recess region includes: the position information corresponding to the recessed area, the recessed range of the recessed area and the recessed depth of the recessed area.

In another possible implementation manner of the embodiment of the application, when determining the attribute information of the recessed area based on at least one image satisfying the third preset condition, the third determining module 27 is specifically configured to:

determining the position relation between the recessed area and the camera based on at least one image meeting a third preset condition, and determining the position information of the recessed area based on the position information of the camera and the position relation between the recessed area and the camera;

performing gradient detection on at least one image meeting a third preset condition to obtain gradient information, and performing non-maximum suppression processing on the gradient information to obtain a processing result;

the depression range of the depression region and the depression depth of the depression region are determined based on the processing result.

In another possible implementation manner of the embodiment of the application, when determining the attribute information of the convex region based on at least one image satisfying the third preset condition, the third determining module 27 is specifically configured to:

determining the relation between the convex area and the camera based on at least one image meeting a third preset condition, and determining the position information of the convex area based on the position information of the camera and the relation between the convex area and the camera;

feature extraction is performed on the raised area, and the type of the raised area, the range of the raised area, and the height of the raised area are determined based on the extracted features.

In another possible implementation manner of the embodiment of the application, when determining the corresponding road maintenance strategy based on the road surface defect information, the fourth determining module 28 is specifically configured to:

determining a highway maintenance strategy based on the road surface defect information, the road surface thickness information and the vehicle information of the road section with the road surface defect, wherein the highway maintenance strategy comprises the following steps: the time of pavement maintenance, the location of pavement maintenance, the tools of pavement maintenance, the materials of pavement maintenance, and the time required to complete pavement maintenance.

Compared with the related art, the embodiment of the application provides a smart city highway maintenance device, the smart city highway maintenance device obtains multimedia information containing road surface and multimedia information containing vehicle information by processing the monitoring video corresponding to the preset road section, obtains the image after gray processing by performing gray processing on the multimedia information containing road surface, obtains the speed recognition result and/or the driving route recognition by performing vehicle speed recognition and/or vehicle driving route recognition on the multimedia information containing vehicle information, can determine whether the road surface of the preset road section has defects or not and corresponding road defect information based on the information, thereby determining a corresponding highway maintenance strategy according to the road defect information and improving the efficiency of discovering the damage of the highway, and further, the cost for maintaining the road can be reduced.

In an embodiment of the present application, an electronic device is provided, as shown in fig. 3, where the electronic device 300 shown in fig. 3 includes: a processor 301 and memory 5003. Wherein the processor 301 is coupled to the memory 303, such as via bus 5002. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that the transceiver 304 is not limited to one in practical applications, and the structure of the electronic device 300 is not limited to the embodiment of the present application.

The Processor 301 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 301 may also be a combination of computing functions, e.g., comprising one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

Bus 302 may include a path that transfers information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 302 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 3, but this does not mean only one bus or one type of bus.

The Memory 303 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these.

The memory 303 is used for storing application program codes for executing the scheme of the application, and the processor 301 controls the execution. The processor 301 is configured to execute application program code stored in the memory 303 to implement the aspects illustrated in the foregoing method embodiments.

Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. But also a server, etc. The electronic device shown in fig. 3 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

The present application provides a computer-readable storage medium, on which a computer program is stored, which, when running on a computer, enables the computer to execute the corresponding content in the foregoing method embodiments. Compared with the prior art, the method and the device have the advantages that the monitoring video corresponding to the preset road section is processed to obtain the multimedia information containing the road surface and the multimedia information containing the vehicle information, the gray scale processing is performed on the multimedia information containing the road surface to obtain the image after the gray scale processing, the vehicle speed identification and/or the vehicle driving route identification are performed on the multimedia information containing the vehicle information to obtain the speed identification result and/or the driving route result, whether the road surface of the preset road section has the defects or not and the corresponding road surface defect information can be determined based on the information, and therefore the corresponding road maintenance strategy can be determined according to the road surface defect information, the efficiency of discovering the road damage can be improved, and the cost of maintaining the road can be reduced.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.

Claims

1. A smart city highway maintenance method is characterized by comprising the following steps:

acquiring a monitoring video corresponding to a preset road section;

2. The method of claim 1, wherein determining whether the road surface of the preset road section has a defect based on the image information after the gray processing comprises:

converting the image information after the gray processing into a depth image;

3. The method of claim 1, wherein the speed identification result comprises: speed change information of at least one vehicle during travel, the travel route results comprising: the running route information corresponding to at least one vehicle;

determining an offset of the actual driving route;

4. The method of claim 3, wherein determining an offset for an actual travel path of any vehicle comprises:

identifying the shape of the road section meeting the first preset condition;

5. The method of claim 4, the determining an offset for the any vehicle actual travel route based on the predicted travel route and the actual travel route comprising at least one of:

6. The method of any of claims 3-5, wherein the determining the road surface defect information comprises:

7. The method according to claim 6, wherein determining attribute information of the recessed area based on at least one image satisfying a third preset condition comprises:

8. The method according to claim 6, wherein determining attribute information of the convex region based on at least one image satisfying a third preset condition comprises:

9. The method of claim 1, wherein determining a corresponding road maintenance strategy based on the road surface defect information comprises:

10. A computer device, comprising:

at least one processor;

a memory;

at least one application, wherein the at least one application is stored in the memory and configured to be executed by the at least one processor, the at least one application configured to: a method of smart city highway maintenance according to any one of claims 1-9.