CN113392816A - Pavement disease detection method, device, electronic equipment and computer readable medium - Google Patents
Pavement disease detection method, device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN113392816A CN113392816A CN202110934536.0A CN202110934536A CN113392816A CN 113392816 A CN113392816 A CN 113392816A CN 202110934536 A CN202110934536 A CN 202110934536A CN 113392816 A CN113392816 A CN 113392816A
- Authority
- CN
- China
- Prior art keywords
- image
- road
- disease
- pavement
- road surface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/29—Graphical models, e.g. Bayesian networks
- G06F18/295—Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The embodiment of the disclosure discloses a pavement disease detection method, a pavement disease detection device, electronic equipment and a computer readable medium. One embodiment of the method comprises: a first generation unit configured to perform image extraction on the road video to generate a road image; a second generating unit configured to input the road image to a preset road surface damage detection model to generate a road surface damage information set; a determining unit configured to determine image positioning information corresponding to the road image; a fusion unit configured to fuse the road surface disease information group and the road image based on the image positioning information to obtain a fused road surface disease image; and the transmitting unit is configured to transmit the fused pavement disease image as a pavement disease detection result to a display terminal for displaying. This embodiment can improve the efficiency of generating a road surface disease detection result.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to a pavement disease detection method, a pavement disease detection device, electronic equipment and a computer readable medium.
Background
Along with the continuous improvement of road infrastructure construction, road traffic is more and more developed, and the road is influenced by load capacity, traffic volume, natural factor, can appear the road damage of different degrees, and the condition that the road damage appears will also be more and more common, also can put forward higher requirement to road patrol and examine and road maintenance. At present, when pavement disease detection is carried out, a manual inspection mode is usually adopted to record a pavement disease detection result.
However, when the pavement damage detection is performed in the above manner, the following technical problems often occur:
firstly, the efficiency of recording a pavement disease detection result in a manual inspection mode is low;
secondly, the road surface disease detection result cannot be synchronized to the vehicle display terminal in real time, and more accurate road surface information is difficult to provide for users, so that the safety of vehicle driving is reduced.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose a pavement damage detection method, apparatus, electronic device, and computer-readable medium to solve one or more of the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide an obstacle information generating method, including: extracting images of the road video to generate a road image; inputting the road image into a preset road surface disease detection model to generate a road surface disease information group; determining image positioning information corresponding to the road image; fusing the pavement disease information group and the road image based on the image positioning information to obtain a fused pavement disease image; and sending the fused pavement disease image as a pavement disease detection result to a display terminal for displaying.
In a second aspect, some embodiments of the present disclosure provide an obstacle information generating apparatus, the apparatus comprising: a first generation unit configured to perform image extraction on the road video to generate a road image; a second generating unit configured to input the road image to a preset road surface damage detection model to generate a road surface damage information set; a determining unit configured to determine image positioning information corresponding to the road image; a fusion unit configured to fuse the road surface disease information group and the road image based on the image positioning information to obtain a fused road surface disease image; and the transmitting unit is configured to transmit the fused pavement disease image as a pavement disease detection result to a display terminal for displaying.
In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement the method described in any of the implementations of the first aspect.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium on which a computer program is stored, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect.
The above embodiments of the present disclosure have the following advantages: by the pavement disease detection method of some embodiments of the present disclosure, the efficiency of obtaining a pavement disease detection result can be improved. Specifically, the reason why the efficiency of obtaining a road surface defect detection result is low is that: and recording a pavement disease detection result in a manual inspection mode. Based on this, the road surface disease detection method of some embodiments of the present disclosure first performs image extraction on a road video to generate a road image. Then, the road image is input to a preset road surface disease detection model to generate a road surface disease information group. By introducing the pavement disease detection model, the pavement disease detection can be carried out on the generated road image in real time. And then, determining image positioning information corresponding to the road image. And then, based on the image positioning information, fusing the road surface disease information group and the road image to obtain a fused road surface disease image. Therefore, the generated road surface disease detection information group can be fused into the generated road image in real time. And finally, sending the fused pavement disease image as a pavement disease detection result to a display terminal for displaying. Therefore, the road image marked with the road surface disease detection result can be provided for the user in real time. Compared with a manual inspection mode, the method can be used for detecting the pavement diseases in real time. The efficiency of generating road surface disease testing result has greatly been improved.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.
Fig. 1 is a schematic diagram of an application scenario of a pavement damage detection method of some embodiments of the present disclosure;
FIG. 2 is a flow diagram of some embodiments of a pavement damage detection method according to the present disclosure;
FIG. 3 is a flow chart of further embodiments of a pavement damage detection method according to the present disclosure;
FIG. 4 is a schematic diagram of generating road semantic images according to some embodiments of the pavement disease detection method of the present disclosure;
FIG. 5 is a schematic structural view of some embodiments of a pavement damage detection apparatus of the present disclosure;
FIG. 6 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 is a schematic diagram of an application scenario of a pavement damage detection method according to some embodiments of the present disclosure.
In the application scenario of fig. 1, first, the computing device 101 may perform image extraction on the road video 102 to generate the road image 103. Next, the computing device 101 may input the road image 103 described above to a preset road surface damage detection model 104 to generate a road surface damage information group 105. The computing device 101 may then determine image localization information 106 corresponding to the road image 103 described above. Then, the computing device 101 may fuse the road surface damage information group 105 and the road image 103 based on the image positioning information 106 to obtain a fused road surface damage image 107. Finally, the computing device 101 may send the fused pavement damage image 107 described above as a pavement damage detection result to the display terminal 108 for display.
The computing device 101 may be hardware or software. When the computing device is hardware, it may be implemented as a distributed cluster composed of multiple servers or terminal devices, or may be implemented as a single server or a single terminal device. When the computing device is embodied as software, it may be installed in the hardware devices enumerated above. It may be implemented, for example, as multiple software or software modules to provide distributed services, or as a single software or software module. And is not particularly limited herein.
It should be understood that the number of computing devices in FIG. 1 is merely illustrative. There may be any number of computing devices, as implementation needs dictate.
With continued reference to fig. 2, a flow 200 of some embodiments of a pavement fault detection method according to the present disclosure is shown. The flow 200 of the pavement disease detection method comprises the following steps:
In some embodiments, an executing body of the road surface disease detection method (such as the computing device 101 shown in fig. 1) may perform image extraction on the road video to generate a road image. The road video may be a real-time road video captured by a vehicle-mounted camera of the vehicle. The image extraction may be to cut out the road image from the road video with a preset frequency (e.g., 0.1 second). The above described implementations may be used to characterize the processing of a single road image. Therefore, the method can extract the images of the road video in real time.
In some embodiments, the execution body may input the road image to a preset road surface damage detection model to generate a road surface damage information set. The preset pavement disease detection model may include, but is not limited to, at least one of the following: G-CRF (gaussian-Conditional Random Field) model, DenseCRF (full-Connected Conditional Random Field) model, MRF (MRF-Markov Random Field) model, SPP (Spatial Pyramid Pooling) model, and FCN (full volumetric Conditional Networks) model, etc. The road surface disease information in the road surface disease information group may include: the name of the pavement damage and the image area of the pavement damage. Each piece of pavement disease information in the pavement disease information group can be used for representing one pavement disease. The pavement disease may include, but is not limited to, at least one of: cracks, block cracks, transverse cracks, longitudinal cracks, potholes, sinkages, ruts, potholes, repairs, and the like. The road surface defect image area can represent the area occupied by the road with the road surface defect in the road image.
In some embodiments, the execution subject may determine image positioning information corresponding to the road image. Wherein the road image is extracted from a road video shot by a vehicle-mounted camera of the vehicle. The road image may be used to characterize the position of the vehicle at a certain moment in time. Therefore, the vehicle position coordinate value of the vehicle at the present time may be first determined by a vehicle positioning system (e.g., a global positioning system). Then, a distance value between the road segment where the road surface defect represented by each piece of road surface defect information in the road surface defect information set is located and the vehicle at the current moment can be determined through a distance measurement algorithm (for example, a residual error network or a liquid machine, etc.), so as to obtain a distance value set. Finally, the vehicle position coordinate value and the distance value group may be determined as image location information corresponding to the road image. The current time may be a time when the on-vehicle camera captures the road image. The vehicle position coordinate value may be used to represent a position of the vehicle when the vehicle-mounted camera captures the road image.
And 204, fusing the road surface disease information group and the road image based on the image positioning information to obtain a fused road surface disease image.
In some embodiments, the execution body may fuse the road surface defect information group and the road image based on the image positioning information to obtain a fused road surface defect image. The vehicle position coordinate value and the distance value group included in the image positioning information and the road surface defect name included in the road surface defect information may be marked on the corresponding road surface defect image area in the road image. In addition, the vehicle position coordinate value may be marked on the road image (for example, an upper left corner or an upper right corner). Therefore, the road surface disease information group and the road image can be fused.
And step 205, sending the fused pavement disease image as a pavement disease detection result to a display terminal for displaying.
In some embodiments, the execution body may send the fused pavement damage image to a display terminal for display as a pavement damage detection result. Wherein, the display terminal can be the display terminal of the vehicle. Thus, when the display terminal displays the road image, the detected road surface defect on the road can be displayed in real time. Therefore, more accurate road information can be timely provided for the user. The safety of vehicle driving is improved.
The above embodiments of the present disclosure have the following advantages: by the pavement disease detection method of some embodiments of the present disclosure, the efficiency of obtaining a pavement disease detection result can be improved. Specifically, the reason why the efficiency of obtaining a road surface defect detection result is low is that: and recording a pavement disease detection result in a manual inspection mode. Based on this, the road surface disease detection method of some embodiments of the present disclosure first performs image extraction on a road video to generate a road image. Then, the road image is input to a preset road surface disease detection model to generate a road surface disease information group. By introducing the pavement disease detection model, the pavement disease detection can be carried out on the generated road image in real time. And then, determining image positioning information corresponding to the road image. And then, based on the image positioning information, fusing the road surface disease information group and the road image to obtain a fused road surface disease image. Therefore, the generated road surface disease detection information group can be fused into the generated road image in real time. And finally, sending the fused pavement disease image as a pavement disease detection result to a display terminal for displaying. Therefore, the road image marked with the road surface disease detection result can be provided for the user in real time. Compared with a manual inspection mode, the method can be used for detecting the pavement diseases in real time. The efficiency of generating road surface disease testing result has greatly been improved.
With further reference to fig. 3, a flow 300 of further embodiments of a pavement fault detection method is shown. The flow 300 of the pavement disease detection method comprises the following steps:
In some embodiments, an executing body (such as the computing device 101 shown in fig. 1) of the road surface disease detection method may acquire the speed value of the current vehicle in a wired manner or a wireless manner.
As an example, the speed value may be 30 m/s.
In some embodiments, the executing body may perform image extraction on the road video according to the speed value to generate a road image. The image extraction may be to intercept images in the road video according to a preset time interval, as the road image. The preset interval may be generated by the following formula:
wherein the content of the first and second substances,
representing the time interval.
Indicating the shooting frequency of the onboard camera.
The above speed values are indicated.
As an example, the shooting frequency of the above-described in-vehicle camera may be 30 frames/second. And if the shooting frequency of the vehicle-mounted camera is more than 30 frames/second. The calculation can be performed at 30 frames/second. The speed value may be 30 m/s. Then, the above time interval may be 12 milliseconds.
Due to the fact that the frequency of shooting by the vehicle-mounted camera is high, the image feature similarity between two adjacent road images in the road video is extremely high. Therefore, the similarity of the road surface disease detection results generated after the road surface disease detection is carried out on the two road images is extremely high, and the redundant phenomenon occurs. Moreover, because the frequency of shooting by the vehicle-mounted camera is high, performing road surface disease detection on each frame of road image in the road video consumes a large amount of computing resources, which causes the waste of computing resources. Therefore, a time interval is introduced, so that more different features exist between the extracted two adjacent road images, and the similarity is reduced. Thus, the similarity of the generated road surface disease detection results can be reduced. And the consumption of computing resources can be reduced, and the generation efficiency of the pavement disease detection result is improved. In addition, the formula can also dynamically adjust the time interval for different shooting frequencies of the vehicle-mounted camera. Therefore, the similarity of the generated road surface disease detection result can be further reduced, the consumption of computing resources is reduced, and the generation efficiency of the road surface disease detection result is improved.
And 303, carrying out illumination homogenization treatment on the road image to obtain a treated road image.
In some embodiments, the executing entity may perform illumination homogenization processing on the road image to obtain a processed road image. The road image can be uniformly processed by illumination through a global dynamic threshold method, and the road image is converted into a gray image to be used as a processed road image. Therefore, the influence of illumination on the detection of the road surface diseases can be avoided.
And 304, performing image semantic identification on the processed road image to obtain a road semantic image.
In some embodiments, the executing body may perform image semantic recognition on the processed road image to obtain a road semantic image. The road semantic image can be obtained by performing image semantic recognition on the processed road image through an image semantic recognition algorithm (e.g., deep lab-v4, fourth edition deep convolution semantic segmentation algorithm). The road semantic image may be a road image marked with a road surface area. The road surface area may refer to an area in which the vehicle can normally travel. Therefore, the false detection of the image outside the road surface area can be avoided. This can improve the efficiency of generating a road surface defect detection result. The accuracy of the pavement disease detection result can be improved.
As an example, as shown in fig. 4, image semantic recognition is performed on the processed road image 401, resulting in a road semantic image 402.
In some embodiments, the execution body may input the road semantic image to the road surface damage detection module to generate a set of road surface damage detection information groups. The road surface disease detection model can comprise a road surface disease detection module and a road surface disease processing module. The road surface disease detection module can be used for detecting the road surface disease of the road surface area marked in the road semantic image. Specifically, the road surface disease detecting module may be YoLov5s (You Only Look one), fifth edition target detection algorithm. Each pavement disease detection information group in the pavement disease detection information group set can be used for representing one pavement disease. Each of the road surface disease detection information in the road surface disease detection information group may be a detection result (e.g., a detection frame) for the same road surface disease.
And step 306, inputting the set of the pavement disease detection information group into a pavement disease processing module to generate a pavement disease information group.
In some embodiments, the execution body may input the set of road surface defect detection information sets to the road surface defect processing module to generate a road surface defect information set. The road surface defect processing module may be configured to select an optimal detection result (e.g., one detection frame) from a plurality of detection results (i.e., a road surface defect detection information set) corresponding to one road surface defect. Specifically, the road surface disease processing module may be a Non-Maximum Suppression algorithm (NMS). In addition, the overlap ratio (IoU) in the non-maximum suppression algorithm may be changed to a Distance overlap ratio (Distance interaction of unity). Thus, a plurality of detection results for the same pavement disease can be eliminated. The missing detection phenomenon caused by shielding of the road surface diseases due to vehicles in front or other reasons is reduced, and meanwhile, the convergence speed in model training is increased.
In some embodiments, the specific implementation manner and technical effects of step 307 may refer to step 203 in those embodiments corresponding to fig. 2, which are not described herein again.
And 308, fusing the road surface disease information group and the road image based on the image positioning information to obtain a fused road surface disease image.
In some embodiments, the executing body may fuse the road surface disease information group and the road image based on the image positioning information to obtain a fused road surface disease image, and may include the following steps:
firstly, identifying the type of the road surface disease information in the road surface disease information group to obtain a road surface disease type information group. The image recognition of the pavement disease image area included in the pavement disease information can be performed through a disease recognition model (for example, a residual error network, a deep convolution network, and the like), so as to obtain the pavement disease type (for example, cracks, block cracks, transverse cracks, longitudinal cracks, pits, subsidence, ruts, repair, pits, and the like).
And secondly, in response to the fact that the pavement disease type information matched with the preset type information exists in the pavement disease type information group, removing the matched pavement disease type information from the pavement disease type information group to obtain a removed type information group. The preset type information may be disease type information generated by using a road image extracted from the road video last time, that is, a previous road image. The time interval between the extraction of the previous road image and the extraction of the road image may be the time interval described above. Specifically, the matching with the preset type information may be to determine whether the road surface diseases in the two road images are the same. The matched pavement damage type information is removed from the pavement damage type information group, and can be used for removing the same pavement damage type information. Removing the matched road surface damage type information from the above-mentioned road surface damage type information group can also be used to avoid duplication with the road surface damage detection result of the previous road image. In addition, if the road surface defect type information is removed, the frame of the road surface defect image area corresponding to the removed road surface defect type information may be used as the type information after removal. The method can be used for updating the frame selection position of the road surface diseases when the road surface disease detection result is displayed. Therefore, the road surface diseases can be tracked on the display terminal. If the pavement damage type information is not removed, the pavement damage type information and the frame of the corresponding pavement damage image area can be used as the type information after removal. In this way, the removed type information group can be obtained. The pavement disease type information can correspond to the pavement disease image area through the pavement disease information.
And thirdly, fusing the road image and the pavement disease information matched with each removed type information in the removed type information group in the pavement disease information group based on the image positioning information to obtain a fused pavement disease image. Preferably, the image positioning information may be marked in the road image. Then, the road surface defect names included in the road surface defect information sets that match the respective pieces of the post-removal type information in the post-removal type information sets may be marked in the road image. Finally, the borders of the road surface defect image areas included in each piece of removed type information in the set of removed type information, or the borders of the road surface defect type information and the corresponding road surface defect image areas may be marked in the road image. Therefore, a fused pavement disease image can be obtained.
Optionally, each road surface disease information in the road image may also be tracked by a target Tracking algorithm (e.g., a kalman filter algorithm, a DSRT (Deep Simple Online And real Tracking algorithm).
In some optional implementation manners of some embodiments, the executing body may fuse the road surface disease information group and the road image based on the image positioning information to obtain a fused road surface disease image, and may further include the following steps:
and in response to determining that no pavement disease type information matched with the preset type information exists in the pavement disease type information group, fusing the pavement disease information group and the road image to obtain a fused pavement disease image. And determining that the pavement damage type information matched with the preset type information does not exist in the pavement damage type information group. The method can be used for indicating that the same road surface diseases do not exist between the road image and the previous road image. Accordingly, the names of road surface defects and the areas of the road surface defect images included in the group of road surface defect information, and the types of road surface defects included in the type information of road surface defects can be marked in the road images. Thus, a fused pavement disease image is obtained.
Optionally, the executing main body may further perform the following steps:
firstly, a time stamp corresponding to the road image is determined. Wherein, the timestamp can be used for representing the time for intercepting the road image.
And a second step of storing the time stamp, the road image and the image positioning information. Wherein, storing the time stamp, the road image and the image positioning information can be used for viewing.
And thirdly, storing the road surface damage information corresponding to the removed type information in the road surface damage information group or the removed type information group.
And 309, sending the fused pavement disease image as a pavement disease detection result to a display terminal for displaying.
In some embodiments, the specific implementation manner and technical effects of step 309 may refer to step 205 in those embodiments corresponding to fig. 2, and are not described herein again.
As can be seen from fig. 3, compared with the description of some embodiments corresponding to fig. 2, the flow 300 of the pavement damage detection method in some embodiments corresponding to fig. 3 embodies the step of obtaining a pavement damage image. Due to the fact that the frequency of shooting by the vehicle-mounted camera is high, the image feature similarity between two adjacent road images in the road video is extremely high. Therefore, the similarity of the road surface disease detection results generated after the road surface disease detection is carried out on the two road images is extremely high, and the redundant phenomenon occurs. Moreover, because the frequency of shooting by the vehicle-mounted camera is high, performing road surface disease detection on each frame of road image in the road video consumes a large amount of computing resources, which causes the waste of computing resources. Therefore, a time interval is introduced, so that more different features exist between the extracted two adjacent road images, and the similarity is reduced. Thus, the similarity of the generated road surface disease detection results can be reduced. And the consumption of computing resources can be reduced, and the generation efficiency of the pavement disease detection result is improved. In addition, the formula can also dynamically adjust the time interval for different shooting frequencies of the vehicle-mounted camera. Therefore, the similarity of the generated road surface disease detection result can be further reduced, the consumption of computing resources is reduced, and the generation efficiency of the road surface disease detection result is improved. The matched pavement damage type information is removed from the pavement damage type information group, and can be used for removing the same pavement damage type information. Removing the matched road surface damage type information from the above-mentioned road surface damage type information group can also be used to avoid duplication with the road surface damage detection result of the previous road image. In addition, if the road surface defect type information is removed, the frame of the road surface defect image area corresponding to the removed road surface defect type information may be used as the type information after removal. The method can be used for updating the frame selection position of the road surface diseases when the road surface disease detection result is displayed. Therefore, the road surface diseases can be tracked on the display terminal. Finally, the synchronous vehicle display terminal of road surface disease detection result can be realized in real time, and more accurate road surface information can be provided for users. Thus, the safety of the vehicle driving is improved.
With further reference to fig. 5, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a pavement damage detection apparatus, which correspond to those shown in fig. 2, and which may be specifically applied in various electronic devices.
As shown in fig. 5, a pavement damage detection apparatus 500 of some embodiments includes: a first generation unit 501, a second generation unit 502, a determination unit 503, a fusion unit 504, and a transmission unit 505. The first generating unit 501 is configured to perform image extraction on a road video to generate a road image; a second generating unit 502 configured to input the road image to a preset road surface defect detection model to generate a road surface defect information set; a determining unit 503 configured to determine image positioning information corresponding to the road image; a fusion unit 504 configured to fuse the road surface disease information group and the road image based on the image positioning information to obtain a fused road surface disease image; a transmitting unit 505 configured to transmit the fused road surface defect image as a road surface defect detection result to a display terminal for display.
It will be understood that the elements described in the apparatus 500 correspond to various steps in the method described with reference to fig. 2. Thus, the operations, features and resulting advantages described above with respect to the method are also applicable to the apparatus 500 and the units included therein, and are not described herein again.
Referring now to FIG. 6, a block diagram of an electronic device (e.g., computing device 101 of FIG. 1) 600 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of some embodiments of the present disclosure.
It should be noted that the computer readable medium described above in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the apparatus; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: extracting images of the road video to generate a road image; inputting the road image into a preset road surface disease detection model to generate a road surface disease information group; determining image positioning information corresponding to the road image; fusing the pavement disease information group and the road image based on the image positioning information to obtain a fused pavement disease image; and sending the fused pavement disease image as a pavement disease detection result to a display terminal for displaying.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware. The described units may also be provided in a processor, and may be described as: a processor includes a first generating unit, a second generating unit, a determining unit, a fusing unit, and a transmitting unit. Here, the names of these units do not constitute a limitation to the unit itself in some cases, and for example, the transmission unit may also be described as a "unit that transmits a road surface defect detection result".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (10)
1. A pavement disease detection method comprises the following steps:
extracting images of the road video to generate a road image;
inputting the road image into a preset road surface disease detection model to generate a road surface disease information group;
determining image positioning information corresponding to the road image;
fusing the pavement disease information group and the road image based on the image positioning information to obtain a fused pavement disease image;
and sending the fused pavement disease image as a pavement disease detection result to a display terminal for displaying.
2. The method of claim 1, wherein the image extracting the road video to generate the road image comprises:
acquiring a speed value of a vehicle;
and according to the speed value, carrying out image extraction on the road video to generate a road image.
3. The method according to claim 1, wherein before the inputting the road image to a preset road surface damage detection model to generate a road surface damage information set, the method further comprises:
carrying out illumination homogenization treatment on the road image to obtain a treated road image;
and carrying out image semantic recognition on the processed road image to obtain a road semantic image.
4. The method according to claim 3, wherein the pavement damage detection model comprises a pavement damage detection module and a pavement damage treatment module; and
the inputting the road image into a preset road surface disease detection model to generate a road surface disease information group includes:
inputting the road semantic image into the road surface disease detection module to generate a road surface disease detection information group set;
and inputting the set of the pavement disease detection information group into the pavement disease processing module to generate a pavement disease information group.
5. The method of claim 1, wherein the fusing the road image and the road disease information set based on the image positioning information to obtain a fused road disease image comprises:
identifying the type of the road surface disease information in the road surface disease information group to obtain a road surface disease type information group;
in response to the fact that the pavement disease type information group is determined to have pavement disease type information matched with preset type information, removing the matched pavement disease type information from the pavement disease type information group to obtain a removed type information group;
and fusing the road image and the pavement disease information matched with each removed type information in the removed type information group in the pavement disease information group based on the image positioning information to obtain a fused pavement disease image.
6. The method of claim 5, wherein the fusing the road image with the road disease information set based on the image positioning information to obtain a fused road disease image, further comprises:
and in response to the fact that the pavement disease type information matched with the preset type information does not exist in the pavement disease type information group, fusing the pavement disease information group and the road image to obtain a fused pavement disease image.
7. The method of claim 6, wherein the method further comprises:
determining a timestamp corresponding to the road image;
storing the timestamp, the road image and the image positioning information;
and storing the road surface damage information corresponding to the removed type information in the road surface damage information group or the removed type information group.
8. A pavement damage detection device comprising:
a first generation unit configured to perform image extraction on the road video to generate a road image;
a second generating unit configured to input the road image to a preset road surface disease detection model to generate a road surface disease information group;
a determining unit configured to determine image positioning information corresponding to the road image;
the fusion unit is configured to fuse the pavement disease information group and the road image based on the image positioning information to obtain a fused pavement disease image;
a transmitting unit configured to transmit the fused pavement disease image as a pavement disease detection result to a display terminal for display.
9. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110934536.0A CN113392816B (en) | 2021-08-16 | 2021-08-16 | Pavement disease detection method, device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110934536.0A CN113392816B (en) | 2021-08-16 | 2021-08-16 | Pavement disease detection method, device, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113392816A true CN113392816A (en) | 2021-09-14 |
| CN113392816B CN113392816B (en) | 2021-12-31 |
Family
ID=77622540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110934536.0A Active CN113392816B (en) | 2021-08-16 | 2021-08-16 | Pavement disease detection method, device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113392816B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111126802A (en) * | 2019-12-10 | 2020-05-08 | 福建省高速公路集团有限公司 | Highway inspection and evaluation method and system based on artificial intelligence |
| CN111126460A (en) * | 2019-12-10 | 2020-05-08 | 福建省高速公路集团有限公司 | Pavement disease automatic inspection method, medium, equipment and device based on artificial intelligence |
| CN112330664A (en) * | 2020-11-25 | 2021-02-05 | 腾讯科技（深圳）有限公司 | Pavement disease detection method and device, electronic equipment and storage medium |
| US20210213971A1 (en) * | 2020-08-25 | 2021-07-15 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for determining road information data and computer storage medium |
-
2021
- 2021-08-16 CN CN202110934536.0A patent/CN113392816B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111126802A (en) * | 2019-12-10 | 2020-05-08 | 福建省高速公路集团有限公司 | Highway inspection and evaluation method and system based on artificial intelligence |
| CN111126460A (en) * | 2019-12-10 | 2020-05-08 | 福建省高速公路集团有限公司 | Pavement disease automatic inspection method, medium, equipment and device based on artificial intelligence |
| US20210213971A1 (en) * | 2020-08-25 | 2021-07-15 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for determining road information data and computer storage medium |
| CN112330664A (en) * | 2020-11-25 | 2021-02-05 | 腾讯科技（深圳）有限公司 | Pavement disease detection method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113392816B (en) | 2021-12-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110427917B (en) | Method and device for detecting key points | |
| CN112101305B (en) | Multi-path image processing method and device and electronic equipment | |
| CN113607185B (en) | Lane line information display method, lane line information display device, electronic device, and computer-readable medium | |
| CN112348029B (en) | Local map adjusting method, device, equipment and computer readable medium | |
| CN115326099B (en) | Local path planning method and device, electronic equipment and computer readable medium | |
| KR20200018411A (en) | Method and apparatus for detecting burr of electrode piece | |
| CN114399589B (en) | Three-dimensional lane line generation method and device, electronic device and computer readable medium | |
| CN115540894B (en) | Vehicle trajectory planning method and device, electronic equipment and computer readable medium | |
| CN114926766A (en) | Identification method and device, equipment and computer readable storage medium | |
| CN115817463B (en) | Vehicle obstacle avoidance method, device, electronic equipment and computer readable medium | |
| CN115272182A (en) | Lane line detection method, lane line detection device, electronic device, and computer-readable medium | |
| CN115616937A (en) | Automatic driving simulation test method, device, equipment and computer readable medium | |
| CN112330788A (en) | Image processing method, image processing device, readable medium and electronic equipment | |
| CN112037223A (en) | Image defect detection method and device and electronic equipment | |
| CN114894205A (en) | Three-dimensional lane line information generation method, device, equipment and computer readable medium | |
| CN111965383B (en) | Vehicle speed information generation method and device, electronic equipment and computer readable medium | |
| CN113392816B (en) | Pavement disease detection method, device, electronic equipment and computer readable medium | |
| CN112232326A (en) | Driving information generation method and device, electronic equipment and computer readable medium | |
| CN109816791B (en) | Method and apparatus for generating information | |
| CN113780247B (en) | Traffic light detection method and device, electronic equipment and computer readable medium | |
| CN114723640B (en) | Obstacle information generation method and device, electronic equipment and computer readable medium | |
| CN115409985A (en) | Target object detection method and device, electronic equipment and readable storage medium | |
| CN115049624A (en) | Method, device, equipment, medium and product for sending early warning information of highway cracks | |
| CN114419070A (en) | Image scene segmentation method, device, equipment and storage medium | |
| CN113808134A (en) | Oil tank layout information generation method, oil tank layout information generation device, electronic apparatus, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 201, 202, 301, No. 56-4 Fenghuang South Road, Huadu District, Guangzhou City, Guangdong Province, 510806 Patentee after: Heduo Technology (Guangzhou) Co.,Ltd. Address before: 100099 101-15, 3rd floor, building 9, yard 55, zique Road, Haidian District, Beijing Patentee before: HOLOMATIC TECHNOLOGY (BEIJING) Co.,Ltd. |