CN116958684A - Road abnormality detection method, model training method and related devices - Google Patents

Abstract

The application provides a road anomaly detection method, a model training method and a related device, and relates to the field of computers. The method comprises the following steps: obtaining a road image to be processed; detecting the road image to be processed by using a preset detection model to obtain a detection result, wherein the detection objects which can be identified by the detection model comprise a normal object group and an abnormal object group, the normal object group comprises at least one normal object which is similar to the abnormal object in the abnormal object group, and the detection result is used for indicating each target detection object contained in the road image to be processed; and obtaining a road abnormality detection result according to the detection result, wherein the road abnormality detection result is used for indicating whether the road image to be processed contains an abnormal object or not. In this way, by setting both the abnormal object and the normal object similar to the abnormal object as the detection object, it is possible to reduce erroneous recognition of the normal object as the abnormal object, thereby improving the detection accuracy.

Description

Road abnormality detection method, model training method and related devices

Technical Field

The application relates to the technical field of computers, in particular to a road abnormality detection method, a model training method and a related device.

Background

At present, roads (such as national roads, provincial roads, county roads and the like) are generally inspected manually to determine whether abnormal conditions exist on the pavement of the roads, such as road cracks, obstacles, road occupation management, private banner and the like. However, this conventional method is large in labor investment and large in workload. Therefore, how to implement the abnormality detection of the road has become a technical problem to be solved by those skilled in the art.

Disclosure of Invention

The embodiment of the application provides a road abnormality detection method, a model training method and a related device, which can obtain a road abnormality detection result without adopting a manual inspection mode, and in the detection process, by taking an abnormal object and a normal object similar to the abnormal object as detection objects, the situation that the normal object is mistakenly identified as the abnormal object can be reduced, so that the detection precision of the obtained road abnormality detection result is improved.

Embodiments of the application may be implemented as follows:

in a first aspect, an embodiment of the present application provides a method for detecting a road abnormality, where the method includes:

obtaining a road image to be processed;

detecting the road image to be processed by using a preset detection model to obtain a detection result, wherein the detection objects identifiable by the detection model comprise a normal object group and an abnormal object group, the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group, and the detection result is used for indicating each target detection object contained in the road image to be processed;

and obtaining a road abnormality detection result according to the detection result, wherein the road abnormality detection result is used for indicating whether the road image to be processed contains an abnormal object or not.

In a second aspect, an embodiment of the present application provides a model training method, where the method includes:

obtaining a reference video;

decoding the reference video to obtain multi-frame reference image frames;

obtaining a plurality of sample road images from the multi-frame reference image frames, and obtaining labeling information of each sample road image to obtain an image training set, wherein the labeling information of each sample road image is used for indicating each detection object contained in the sample road image, the detection objects corresponding to the image training set comprise a normal object group and an abnormal object group, and the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group;

training a preset neural network according to the image training set to obtain a detection model.

In a third aspect, an embodiment of the present application provides a road abnormality detection apparatus, including:

the image acquisition module is used for acquiring a road image to be processed;

the detection module is used for detecting the road image to be processed by using a preset detection model to obtain a detection result, wherein the detection objects identifiable by the detection model comprise a normal object group and an abnormal object group, the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group, and the detection result is used for indicating each target detection object contained in the road image to be processed;

and the analysis module is used for obtaining a road abnormality detection result according to the detection result, wherein the road abnormality detection result is used for indicating whether the road image to be processed contains an abnormal object or not.

In a fourth aspect, an embodiment of the present application provides a model training apparatus, including:

the training set obtaining module is used for obtaining a reference video;

the training set obtaining module is also used for decoding the reference video to obtain multi-frame reference image frames;

the training set obtaining module is further configured to obtain a plurality of sample road images from the multi-frame reference image frames, and obtain labeling information of each sample road image to obtain an image training set, where the labeling information of each sample road image is used to indicate each detection object included in the sample road image, the detection objects corresponding to the image training set include a normal object group and an abnormal object group, and the normal object group includes at least one normal object similar to the abnormal objects in the abnormal object group;

and the training module is used for training the preset neural network according to the image training set to obtain a detection model.

In a fifth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, where the memory stores machine executable instructions executable by the processor, where the processor may execute the machine executable instructions to implement the method according to the foregoing embodiment.

In a fifth aspect, embodiments of the present application provide a readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method as described in the previous embodiments.

The road abnormality detection method, the model training method and the related device provided by the embodiment of the application utilize the preset detection model to detect the obtained road image to be processed to obtain the detection result, and further obtain the road abnormality result based on the detection result. The detection model is used for identifying a road image to be processed, wherein the detection objects identifiable by the detection model comprise a normal object group and an abnormal object group, the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group, and the detection result is used for indicating each target detection object contained in the road image to be processed; the road abnormality detection result is used for indicating whether an abnormal object is contained in the road image to be processed. In this way, the road abnormality detection result can be obtained without adopting a manual inspection mode, and in the detection process, the abnormal object and the normal object similar to the abnormal object are taken as detection objects, so that the situation that the normal object is mistakenly identified as the abnormal object can be reduced, and the detection precision of the obtained road abnormality detection result is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic block diagram of an electronic device according to an embodiment of the present application;

fig. 2 is a flow chart of a road abnormality detection method according to an embodiment of the present application;

FIG. 3 is a flow chart illustrating the sub-steps included in step S220 in FIG. 2;

FIG. 4 is a schematic flow chart of a model training method according to an embodiment of the present application;

fig. 5 is a schematic block diagram of a road abnormality detection device according to an embodiment of the present application;

fig. 6 is a block schematic diagram of a model training apparatus according to an embodiment of the present application.

Icon: 100-an electronic device; 110-memory; a 120-processor; 130-a communication unit; 200-road abnormality detection means; 210-an image acquisition module; 220-a detection module; 230-an analysis module; 300-model training device; 310-a training set acquisition module; 320-training module.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present application.

It is noted that relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Some embodiments of the present application are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.

Referring to fig. 1, fig. 1 is a block diagram of an electronic device 100 according to an embodiment of the application. The electronic device 100 may be, but is not limited to, a computer, a server, etc. The electronic device 100 may be used as a training device or as a detection device. The electronic device 100 may include a memory 110, a processor 120, and a communication unit 130. The memory 110, the processor 120, and the communication unit 130 are electrically connected directly or indirectly to each other to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines.

Wherein the memory 110 is used for storing programs or data. The Memory 110 may be, but is not limited to, random access Memory (Random Access Memory, RAM), read Only Memory (ROM), programmable Read Only Memory (Programmable Read-Only Memory, PROM), erasable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable Read Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

The processor 120 is used to read/write data or programs stored in the memory 110 and perform corresponding functions. For example, the memory 110 stores a road abnormality detection device and/or a model training device, which may include at least one software function module stored in the memory 110 in the form of software or firmware (firmware), respectively. The processor 120 executes various functional applications and data processing by running software programs and modules stored in the memory 110, such as a road abnormality detection device and/or a model training device in the embodiment of the present application, that is, implements a road abnormality detection method and/or a model training method in the embodiment of the present application.

The communication unit 130 is configured to establish a communication connection between the electronic device 100 and other communication terminals through a network, and is configured to transmit and receive data through the network.

It should be understood that the structure shown in fig. 1 is merely a schematic diagram of the structure of the electronic device 100, and that the electronic device 100 may further include more or fewer components than those shown in fig. 1, or have a different configuration than that shown in fig. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.

Referring to fig. 2, fig. 2 is a flowchart of a road anomaly detection method according to an embodiment of the present application. The method may be applied to a detection apparatus, which is an apparatus for performing road abnormality detection. The detection device may comprise an electronic device 100. In this embodiment, the method may include steps S210 to S230. The specific flow of the path abnormality detection method is described in detail below.

Step S210, obtaining a road image to be processed.

The road image to be processed is an image related to the road aiming at the current road abnormality detection. The road image to be processed can be an image designated by a user, can be a certain image acquired by a camera at certain intervals, and can also be an image determined by other modes. The specific obtaining manner of the road image to be processed is not particularly limited herein, and may be set in combination with actual situations.

And step S220, detecting the road image to be processed by using a preset detection model to obtain a detection result.

The detection model is a model for object detection. The detection objects identifiable by the detection model can be divided into two groups: normal object group and abnormal object group. The normal object group comprises at least one normal object, and the abnormal object group comprises at least one abnormal object. Included in the abnormal object group are objects related to roads and belonging to abnormalities, such as ground cracks.

The normal objects in the normal object group are similar to the abnormal objects in the abnormal object group. For example, the normal object group includes normal objects a1 and a2, the abnormal object group includes abnormal objects b1, b2 and b3, and the normal objects a1 and a2 may be detection objects having a similar appearance to the abnormal object b1. The detection model may be obtained by training the detection device in advance, or may be obtained by training other training devices in advance and sent to the detection device.

And under the condition that the road image to be processed is obtained, performing object detection on the road image to be processed based on the detection model to obtain a detection result. And the detection result is used for indicating each target detection object contained in the road image to be processed. The target detection object is a detection object included in the image to be processed. For example, the detection objects identifiable by the detection model include a1, a2, b1, b2, b3, and the target detection objects a1, b1 can be determined to be included in the road image to be processed through detection.

And step S230, obtaining a road abnormality detection result according to the detection result.

In the case of obtaining the detection result, the detection result may be analyzed to determine whether the road image to be processed includes an abnormal object, and a corresponding road abnormality detection result may be generated. The road abnormality detection result is used for indicating whether the road image to be processed contains an abnormal object or not. The analysis mode adopted can be specifically set in combination with actual requirements.

In this way, by using both the abnormal object and the normal object similar to the abnormal object as the detection objects, that is, by using the false detection object as the detection object, it is possible to reduce the cases of erroneously recognizing the normal object as the abnormal object, thereby improving the road abnormality detection accuracy.

The detection model will be briefly described first.

The detection model may be a neural network model employing a dark net framework. The Darknet framework is a relatively light open source deep learning framework based on C and CUDA completely, and is mainly characterized by easy installation, no dependency (OpenCV can be omitted), good portability and support of two calculation modes of CPU and GPU. The neural network model is represented by a network topology, node characteristics and learning rules, and is described based on a mathematical model of neurons. The neural network model has four files under the dark frame: model files, weight files, data files, category name files.

The detection model may be located in an nvidia edge device. Therefore, the Darknet framework is matched with the nvidia edge equipment, the acceleration strategy based on cuda and cudnn is provided, the framework is compiled and generated for the c language, and only the application layer uses python for calling, so that the calculation speed is greatly improved, and the instantaneity can be improved.

The detection model can be obtained by training a preset neural network model through an image training set. The size of the anchor frame in the preset neural network model may be set based on the size of the detection object corresponding to the image training set, where the size of one anchor frame is set according to the size of the detection object with the smallest size in the detection objects corresponding to the image training set. That is, the size of the anchor frame in the detection model may be set based on the size of the detection object corresponding to the image training set, where the size of one anchor frame is set according to the size of the detection object with the smallest size among the detection objects corresponding to the image training set. The conventional strategy is to add multi-scale feature extraction, but the processing speed of the model is affected, and in the embodiment, the small target size is adapted by adjusting the Anchor strategy, so that the detection precision of the small target is effectively improved, and the processing speed of the model is ensured.

The abnormal object group identifiable by the detection model can comprise at least one road maintenance object and/or at least one road law enforcement object. Among them, highway maintenance objects may be: road surface damage and pits; road surface soil, sand and sundries; road surface water accumulation; road safety facilities (e.g., safety facilities provided for road repair); traffic accidents and road surface pollutants; landslide of the upper and lower side slopes and the roadbed; traffic jams; disaster weather; road production and road right damage; side ditches, drainage ditches, rapid launders, culverts, bridge drainage holes and the like. The road law enforcement objects may be: infringe or damage road construction control area road yield and road right; infringe or damage a road, road land, road right, etc.

The normal object group and the abnormal object group included in the detection model are exemplified below.

The abnormal object group comprises at least any one of ground cracks, a tombstone and a roadblock, the normal object group comprises at least any one of ground shadows, roadside fences and indicators (such as triangular warning signs placed by a vehicle owner due to vehicle faults), the ground cracks are similar to the ground shadows, the tombstone is similar to the roadside fences, and the roadblock is similar to the indicators.

Alternatively, the camera may be mounted on a vehicle travelling along the road to be inspected, for example, a bus in the form of a fixed line. The camera can acquire and analyze view angles of different angles (overlook), so that the camera can use the view angle with highest precision when in installation and implementation, and the resolution of the camera for obtaining video streams can be 1920 x 1080. And then decoding the video stream obtained by the camera, thereby determining the image to be processed. Alternatively, the decoding and detection may be synchronous or asynchronous, and may be specifically determined in conjunction with the actual requirements.

As a possible implementation, the image to be processed may be obtained as shown in fig. 3. Referring to fig. 3, fig. 3 is a flow chart illustrating the sub-steps included in step S220 in fig. 2. In this embodiment, the step S220 may include the sub-step S211 and the sub-step S212.

In sub-step S211, the obtained video stream is decoded by the image acquisition thread to obtain at least one image frame.

Sub-step S212, obtaining the road image to be processed from the at least one image frame by using a detection thread.

In this embodiment, the detection device may include an image capturing thread and a detection thread. And the image acquisition thread can be utilized to carry out image decoding on the obtained video stream to obtain at least one image frame. The video stream is a video stream which needs to be subjected to road abnormality detection, and can be specifically set in combination with actual requirements. And obtaining the image to be processed from the at least one frame of image frame by utilizing the detection thread, and further obtaining a road abnormality detection result based on the image to be processed. Thus, the image decoding and the image processing can be asynchronously performed by using two threads, namely, the multithreading parallel processing is used in the image decoding and the image processing, and the acquisition and the processing are not affected mutually, so that the speed of obtaining the road abnormality detection result is improved.

In order to avoid that the processing speed of the detection thread and the processing speed of the image acquisition thread are greatly different, the detection thread repeatedly detects the road abnormality on the same image, an image frame can be obtained from the at least one image frame as a current image frame, and then whether the current image frame and the last image frame are the same frame is judged. Wherein the last image frame is the last image input into the detection model. And taking the current image frame as the road image to be processed when the current image frame and the last image frame are not the same frame. And if the current image frame and the last image frame are not the same frame, acquiring one frame of image frame from the image acquisition thread again as a new current image frame.

Alternatively, the video stream for decoding may be a video stream that has been captured in advance, or may be a real-time video stream. In order to improve the real-time performance of road anomaly detection, the video stream may be a real-time video stream, in which case, the image acquisition thread may store only the latest image frame obtained by decoding, and the detection thread may obtain the latest image frame from the image acquisition thread as the current image frame at a time.

After the road image to be processed is obtained, if the detection model has a requirement on the image size, the road image to be processed can be processed into a road image to be processed with a preset size, and then the road image to be processed with the preset size is input into the detection model. And the detection model carries out object detection on the road image to be processed with the preset size to obtain a detection result.

As a possible implementation manner, the detection result may include only the object identifier of each target detection object included in the image to be processed. The object identifier included in the detection result is used for indicating a target detection object included in the image to be processed. In the case where an abnormal object exists in the target detection objects, a road abnormality detection result for indicating the existence of an abnormal object in the image to be processed may be directly generated.

As another possible implementation manner, the detection result may also be used to indicate the position information of each target detection object in the road image to be processed. The road abnormality detection result may be obtained from the detection result. And when the road abnormality detection result indicates that the to-be-processed road image contains an abnormal object, the road abnormality detection result is also used for indicating each target detection object serving as the abnormal object and the position information of the target detection object. Therefore, the specific position of the abnormal object in the actual environment can be determined according to the road abnormal detection result, and the subsequent maintenance is convenient.

For example, the detection result may include an object identifier of each target detection object and position information of each target detection object in the road image to be processed. The road abnormality detection result may include an object identifier of each target detection object as an abnormality object and position information of the target detection object.

The detection result may further include confidence of each target detection object. The confidence coefficient of each target detection object can be compared with a preset confidence coefficient, and target detection objects with the confidence coefficient larger than the preset confidence coefficient are screened out. And then the position information of the screened target detection object belonging to the abnormal object and the object identification of the screened target detection object belonging to the abnormal object can be included in the road abnormal detection result. Therefore, the road abnormality detection result which is high in accuracy and convenient for subsequent processing can be obtained.

The conventional detection algorithm has the requirements of a server level or an independent display card on the display memory and the computing power, and when the algorithm is applied to the edge equipment, the algorithm with the high computing power cannot bear the load or needs to spend more time for processing, so that the condition of missed detection can be caused in the road abnormal detection scene. According to the road abnormality detection method provided by the embodiment of the application, the detection model under the Darknet framework is adopted, and the image is acquired and the detection is executed in a multi-process parallel manner, so that the detection equipment is not required to meet the high calculation force requirement or more time is required to obtain the result. Therefore, the road abnormality detection method can be applied to detection equipment serving as edge equipment, the accuracy of road abnormality detection results is guaranteed, and real-time detection and early reporting of abnormal events can be performed.

However, the product mode in the project is edge equipment, an algorithm with high calculation power requirement cannot bear the load or needs to spend more time for processing, and the condition of missed detection is caused in a road abnormal detection scene. In the scheme, no matter in model selection or algorithm frame optimization, good effects are achieved, the precision is guaranteed, the environment of the edge equipment is attached, and real-time algorithm processing and abnormal event reporting are achieved.

Referring to fig. 4, fig. 4 is a flow chart of a model training method according to an embodiment of the application. The model training method can be applied to training equipment, and the training equipment can be a server, a computer or the like. The training device and the detecting device may be the same device or different devices. In this embodiment, the model training method may include steps S310 to S340.

In step S310, a reference video is obtained.

Alternatively, to facilitate rapid acquisition of an image training set that includes a large amount of data, a reference video may be acquired first. The number of the reference videos can be one or a plurality of, and can be specifically set according to actual requirements. The reference video may be a video related to a road collected through different viewing angles.

Step S320, decoding the reference video to obtain multi-frame reference image frames.

Step S330, obtaining a plurality of sample road images from the multi-frame reference image frames, and obtaining labeling information of each sample road image to obtain an image training set.

In the case of obtaining the reference video, the reference video may be decoded to obtain a plurality of frame reference image frames. Alternatively, the multi-frame reference image frame may be directly used as the plurality of sample road images. The similarity between each of the plurality of frame reference image frames may also be calculated, and the image frames having a similarity higher than a preset similarity may be deleted, and the highly blurred image frames may be deleted, thereby obtaining the plurality of sample road images. Next, labeling information for each sample road image may be obtained. The labeling information of each sample road image is used for indicating each detection object contained in the sample road image, and can also be used for indicating the position information of each detection object contained in the sample road image. The detection objects corresponding to the image training set comprise a normal object group and an abnormal object group, and the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group. For the description of the normal object group and the abnormal object group, reference may be made to the above description, and the description is omitted here.

Optionally, under the condition that the preset neural network has a requirement on the format of the picture, format conversion can be performed to obtain an image training set which can be used for training the preset neural network.

And step S340, training a preset neural network according to the image training set to obtain a detection model.

The preset neural network may include preset sizes of a plurality of anchor frames, where the sizes of the anchor frames are set according to the sizes of detection objects corresponding to the image training set, and the sizes of one anchor frame are set according to the size of the detection object with the smallest size in the detection objects corresponding to the image training set. The image training set can be utilized to train the preset neural network, so that a detection model is obtained. The detection model can be used for detecting road abnormality.

In order to perform the corresponding steps in the above embodiments and the various possible ways, an implementation manner of the road abnormality detecting apparatus 200 is given below, and alternatively, the road abnormality detecting apparatus 200 may employ the device structure of the electronic device 100 shown in fig. 1. Further, referring to fig. 5, fig. 5 is a block diagram of a road anomaly detection device 200 according to an embodiment of the present application. It should be noted that, the basic principle and the technical effects of the road abnormality detecting device 200 provided in this embodiment are the same as those of the above embodiment, and for brevity, reference should be made to the corresponding contents of the above embodiment. The road abnormality detection apparatus 200 may include: the device comprises an image obtaining module 210, a detecting module 220 and an analyzing module 230.

The image obtaining module 210 is configured to obtain an image of a road to be processed.

The detection module 220 is configured to detect the road image to be processed by using a preset detection model, so as to obtain a detection result. The detection objects which can be identified by the detection model comprise a normal object group and an abnormal object group, the normal object group comprises at least one normal object which is similar to the abnormal objects in the abnormal object group, and the detection result is used for indicating each target detection object contained in the road image to be processed.

The analysis module 230 is configured to obtain a road abnormality detection result according to the detection result. The road abnormality detection result is used for indicating whether the road image to be processed contains an abnormal object or not.

As shown in fig. 6, an embodiment of the present application further provides a block schematic diagram of the model training apparatus 300. It should be noted that, the basic principle and the technical effects of the model training apparatus 300 provided in this embodiment are the same as those of the foregoing embodiments, and for brevity, reference may be made to the corresponding contents of the foregoing embodiments. The model training apparatus 300 may include: training set acquisition module 310 and training module 320.

The training set obtaining module 310 is configured to obtain a reference video.

The training set obtaining module 310 is further configured to decode the reference video to obtain a multi-frame reference image frame.

The training set obtaining module 310 is further configured to obtain a plurality of sample road images from the multi-frame reference image frame, and obtain labeling information of each of the sample road images, so as to obtain an image training set. The labeling information of each sample road image is used for indicating each detection object contained in the sample road image, the detection objects corresponding to the image training set comprise a normal object group and an abnormal object group, and the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group.

The training module 320 is configured to train a preset neural network according to the image training set to obtain a detection model.

Alternatively, the above modules may be stored in the memory 110 shown in fig. 1 or solidified in an Operating System (OS) of the electronic device 100 in the form of software or Firmware (Firmware), and may be executed by the processor 120 in fig. 1. Meanwhile, data, codes of programs, and the like, which are required to execute the above-described modules, may be stored in the memory 110.

The embodiment of the application also provides a readable storage medium, on which a computer program is stored, which when being executed by a processor, implements the road anomaly detection method or the model training method.

In summary, the embodiment of the application provides a road anomaly detection method, a model training method and a related device, which utilize a preset detection model to detect an obtained road image to be processed to obtain a detection result, and further obtain a road anomaly result based on the detection result. The detection model is used for identifying a road image to be processed, wherein the detection objects identifiable by the detection model comprise a normal object group and an abnormal object group, the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group, and the detection result is used for indicating each target detection object contained in the road image to be processed; the road abnormality detection result is used for indicating whether an abnormal object is contained in the road image to be processed. In this way, the road abnormality detection result can be obtained without adopting a manual inspection mode, and in the detection process, the abnormal object and the normal object similar to the abnormal object are taken as detection objects, so that the situation that the normal object is mistakenly identified as the abnormal object can be reduced, and the detection precision of the obtained road abnormality detection result is improved.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are merely illustrative, for example, of the flowcharts and block diagrams in the figures that illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, functional modules in the embodiments of the present application may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The above description is only of alternative embodiments of the present application and is not intended to limit the present application, and various modifications and variations will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (10)

1. A method for detecting a road abnormality, the method comprising:

obtaining a road image to be processed;

detecting the road image to be processed by using a preset detection model to obtain a detection result, wherein the detection objects identifiable by the detection model comprise a normal object group and an abnormal object group, the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group, and the detection result is used for indicating each target detection object contained in the road image to be processed;

and obtaining a road abnormality detection result according to the detection result, wherein the road abnormality detection result is used for indicating whether the road image to be processed contains an abnormal object or not.

2. The method according to claim 1, wherein the obtaining the road image to be processed comprises:

decoding the obtained video stream by utilizing an image acquisition thread to obtain at least one frame of image frame;

and obtaining the road image to be processed from the at least one image frame by using a detection thread.

3. The method of claim 2, wherein the obtaining the road image to be processed from the at least one image frame using a detection thread comprises:

judging whether a current image frame obtained from the at least one image frame and a last image frame are the same frames, wherein the last image frame is an image which is input into the detection model last time;

and taking the current image frame as the road image to be processed when the current image frame and the last image frame are not the same frame.

4. The method according to claim 1, wherein the detection result is further used for indicating position information of each target detection object in the road image to be processed, and the obtaining the road abnormality detection result according to the detection result includes:

and obtaining the road abnormality detection result according to the target detection objects indicated by the detection result and the position information of the target detection objects, wherein the road abnormality detection result is used for indicating the target detection objects serving as the abnormal objects and the position information of the target detection objects when the road abnormality detection result indicates that the to-be-processed road image contains the abnormal objects.

5. The method according to any one of claims 1-4, wherein the group of abnormal objects comprises at least one road maintenance object and/or at least one road enforcement object.

6. The method of claim 5, wherein the abnormal object group includes at least any one of a ground crack, a tombstone, and a roadblock, and the normal object group includes at least any one of a ground shadow, a roadside fence, and an indicator, wherein the ground crack is similar to the ground shadow, the tombstone is similar to the roadside fence, and the roadblock is similar to the indicator.

7. The method according to any one of claims 1 to 4, wherein the detection model is obtained by training a preset neural network according to an image training set, the preset neural network includes a plurality of preset anchor frames, the sizes of the anchor frames are set according to the sizes of detection objects corresponding to the image training set, and the size of one anchor frame is set according to the size of the detection object with the smallest size among the detection objects corresponding to the image training set.

8. A method of model training, the method comprising:

obtaining a reference video;

decoding the reference video to obtain multi-frame reference image frames;

obtaining a plurality of sample road images from the multi-frame reference image frames, and obtaining labeling information of each sample road image to obtain an image training set, wherein the labeling information of each sample road image is used for indicating each detection object contained in the sample road image, the detection objects corresponding to the image training set comprise a normal object group and an abnormal object group, and the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group;

training a preset neural network according to the image training set to obtain a detection model.

9. The method according to claim 8, wherein the preset neural network includes preset sizes of a plurality of anchor frames, the sizes of the anchor frames are set according to sizes of detection objects corresponding to the image training set, and the size of one anchor frame is set according to a size of a detection object with a smallest size among the detection objects corresponding to the image training set.

10. A road abnormality detection apparatus, characterized by comprising:

the image acquisition module is used for acquiring a road image to be processed;

the detection module is used for detecting the road image to be processed by using a preset detection model to obtain a detection result, wherein the detection objects identifiable by the detection model comprise a normal object group and an abnormal object group, the normal object group comprises at least one normal object similar to the abnormal objects in the abnormal object group, and the detection result is used for indicating each target detection object contained in the road image to be processed;

and the analysis module is used for obtaining a road abnormality detection result according to the detection result, wherein the road abnormality detection result is used for indicating whether the road image to be processed contains an abnormal object or not.