WO2024148993A1 - 管道缺陷检测方法、装置、电子设备及存储介质 - Google Patents

管道缺陷检测方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2024148993A1
WO2024148993A1 PCT/CN2023/137184 CN2023137184W WO2024148993A1 WO 2024148993 A1 WO2024148993 A1 WO 2024148993A1 CN 2023137184 W CN2023137184 W CN 2023137184W WO 2024148993 A1 WO2024148993 A1 WO 2024148993A1
Authority
WO
WIPO (PCT)
Prior art keywords
defect
pipeline
frame
image
confidence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/137184
Other languages
English (en)
French (fr)
Inventor
张轩
王亚立
乔宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Publication of WO2024148993A1 publication Critical patent/WO2024148993A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Definitions

  • the underground pipeline system is one of the most important infrastructures in urban construction and development, and is known as the "blood vessels of the city.” In order to ensure its normal operation, regular inspections are required so that defects can be repaired in a timely manner after they are detected.
  • the existing technology usually collects a large number of pipeline videos based on pipeline QV (Quick-View) inspection and pipeline CCTV (Closed Circuit Television Inspection) inspection, and then the inspection personnel analyze whether the pipeline video has defects and damage.
  • the identification of defects or damage is to mark a certain frame in the pipeline video to represent the pipeline defect or damage in that time period. Accordingly, the certain frame becomes the key frame with defects or damage.
  • a pipeline defect detection method includes: acquiring a pipeline video and sampling the pipeline video to obtain a plurality of image frames to be detected; performing frame-level defect detection on the plurality of image frames to be detected to obtain a detection result of each of the image frames; the detection result is used to indicate whether the pipeline has defects, whether the image shooting scene is inside the pipeline, the defect category and its confidence, the defect level of the defect category, and at least one of a defect mask and its confidence; based on the detection results of each of the image frames, performing screening processing and/or deduplication processing on each of the image frames to obtain a key frame; and outputting the detection result of the key frame.
  • the image acquisition module includes: a frame rate acquisition unit, which is used to obtain a frame rate matching the selected input mode in response to a selection operation performed on different input modes; and a sampling unit, which is used to sample the image frames in the pipeline video according to the acquired frame rate to obtain multiple image frames to be detected.
  • the screening and deduplication module includes: a first determination unit, for determining, for each of the image frames, the confidence of each defect category and the confidence of each defect mask based on the detection result of the image frame, and determining the number of defect masks, the maximum confidence of the defect masks, the maximum confidence of the defect category, and the total confidence of all defect categories based on the confidence of each defect category and the confidence of each defect mask; a first screening unit, for taking the image frame as a key frame if the detection result of the image frame indicates that there is a defect in the pipeline and the maximum confidence of the defect category is greater than a first confidence threshold; a second screening unit, for taking the image frame as a key frame if the number of defect masks in the image frame is greater than a set number and the maximum confidence of the defect mask is greater than a second confidence threshold; and a third screening unit, for taking the image frame as a key frame if the total confidence of all defect categories in the image frame is greater than a third confidence threshold.
  • the screening and deduplication module includes: an image traversal unit, which is used to traverse each of the image frames and use the traversed image frame as the current image frame; a second determination unit, which is used to determine the defect category and its confidence of the current image frame based on the detection result of the current image frame; and an image retention unit, which is used to retain the image frame if the interval time between the current image frame and the previous image frame is less than a set time and the confidence of the current image frame and the previous image frame is greater than a fourth confidence threshold and the defect category is the same, then the image frame is retained.
  • the result output module includes: an image output unit, used to mark the corresponding detection result in the key frame and output the marked key frame; a video output unit, used to mark the key frame in the pipeline video based on the detection result of the key frame, and output the marked pipeline video; a text output unit, used to output the detection result of the key frame in text form.
  • an electronic device includes: at least one processor, at least one memory, and at least one communication bus, wherein a computer program is stored in the memory, and the processor reads the computer program in the memory through the communication bus; when the computer program is executed by the processor, the pipeline defect detection method as described above is implemented.
  • a storage medium stores a computer program thereon, and when the computer program is executed by a processor, the pipeline defect detection method as described above is implemented.
  • a computer program product includes a computer program, the computer program is stored in a storage medium, a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the pipeline defect detection method as described above is implemented when the computer device executes the computer program.
  • FIG1 is a schematic diagram of an implementation environment involved in this application.
  • FIG2 is a flow chart of a pipeline detection method according to an exemplary embodiment
  • FIG3 is a flow chart showing step 310 according to an exemplary embodiment
  • FIG4 is a schematic diagram showing a result output according to an exemplary embodiment
  • FIG5 is a flow chart showing step 320 according to an exemplary embodiment
  • FIG6 is a flow chart showing step 330 according to an exemplary embodiment
  • FIG7 is a flowchart showing key frame screening and deduplication according to an exemplary embodiment
  • FIG8 is a schematic diagram of an application scenario according to an exemplary embodiment
  • FIG9 is a structural block diagram of a pipeline defect detection device according to an exemplary embodiment
  • FIG10 is a hardware structure diagram of an electronic device according to an exemplary embodiment
  • pipeline QV detection uses a high-definition zoom camera, combined with lighting assistance, to collect clear images of the inside of the pipeline, and then find defects or damage inside the pipeline
  • pipeline CCTV detection is mainly in the form of closed-circuit television recording, using camera equipment to enter the pipeline and transmit image data to the host computer for pipeline defect detection.
  • Defects in pipeline videos are often displayed continuously for a certain period of time in the pipeline video, but the identification of defects is to mark a certain frame in the pipeline video to represent the defects in this period of time. This certain frame is called the key frame with defects. For this key frame, it is usually necessary to be able to clearly see the pipeline defects and determine the severity of the pipeline defects.
  • pipeline defect detection methods are video-level defect detection, that is, the pipeline video is analyzed and detected as a whole to output the defects contained in the entire pipeline video, without involving temporal positioning.
  • video-level defect detection that is, the pipeline video is analyzed and detected as a whole to output the defects contained in the entire pipeline video, without involving temporal positioning.
  • manual cropping of the pipeline video is required to locate the defects in time.
  • the pipeline defect detection method can accurately locate and classify defects in the pipeline, and evaluate the severity of the defects.
  • the pipeline defects can be accurately located in time and space, avoiding manual pipeline defect detection, and can effectively improve the automation and efficiency of pipeline defect detection.
  • the pipeline defect detection method is applicable to a pipeline defect detection device, which can be deployed in an electronic device, which can be a computer device deployed with a von Neumann architecture, for example, the computer device can be a desktop computer, a laptop computer, a server, etc.
  • FIG1 is a schematic diagram of an implementation environment involved in a pipeline defect detection method, wherein the implementation environment includes an image acquisition device 130 , a gateway 150 , a server 170 and a router 190 .
  • the image acquisition device 130 may be an electronic device used for pipeline defect detection, such as a pipeline QV periscope, a CCTV pipeline inspection robot, etc.
  • the server end 170 can be a desktop computer, a laptop computer, a server or other electronic device with communication connection function, or a computer device cluster composed of multiple servers, or even a cloud computing center composed of multiple servers.
  • the server is used to provide background services, for example, background services include but are not limited to pipeline defect detection services, etc.
  • the image acquisition device 130 interacts with the server 170 through a wide area network
  • the wide area network can be realized by establishing a communication connection in a wired or wireless manner
  • the wired or wireless manner includes but is not limited to 2G, 3G, 4G, 5G, WIFI, etc., so as to realize data transmission with the server 170.
  • the transmitted data can be a pipeline video, etc.
  • the image acquisition device 130 is responsible for acquiring pipeline video and transmitting it to the server 170 to request the server 170 to provide pipeline defect detection service.
  • pipeline defect detection can be performed based on the received pipeline video. Specifically, the pipeline defects are sampled to obtain multiple image frames to be detected, and frame-level defect detection is performed on the multiple image frames to be detected to obtain the detection results of each image frame. Then, according to the detection results of each image frame, each image frame is screened and/or deduplicated to obtain a key frame, and finally the detection result of the key frame is output.
  • the image acquisition device 110 can also be integrated with the server 170 in the same electronic device, so that the pipeline defect detection process is completed by the same electronic device.
  • An embodiment of the present application provides a pipeline defect detection method.
  • the method is applicable to an electronic device.
  • the electronic device may be the server 170 in the implementation environment shown in FIG. 1 .
  • the method may include the following steps:
  • Step 310 Acquire the pipeline video and sample the pipeline video to obtain a plurality of image frames to be detected.
  • the pipeline video may be obtained by photographing the pipeline through a pipeline QV periscope or a CCTV pipeline inspection robot, wherein the image shooting scene may include inside and outside the pipeline.
  • the sample is achieved by sampling the image frames in the pipeline video.
  • sampling is performed at different frame rates, wherein the frame rate can be flexibly set according to actual needs of the application scenario and is not limited here.
  • the above step 310 may include the following steps:
  • Step 311 in response to a selection operation performed on different input modes, obtaining a frame rate matching the selected input mode.
  • Step 312 Sample the image frames in the pipeline video according to the acquired frame rate to obtain a plurality of image frames to be detected.
  • the present application provides four different input modes: performance, balance, precision and custom. Accordingly, the frame rates corresponding to the four different input modes are: 1 frame per second, 10 frames per second, the video frame rate of the pipeline video, and the user-defined frame rate.
  • a selection entry is provided in the electronic device, and the user can select different input modes by triggering the selection operation at the selection entry.
  • the selection entry is a drop-down list
  • the drop-down list contains four input modes to choose from. If the user clicks on one of the input modes, then the one of the input modes is deemed to be the input mode selected by the user, and the click operation is deemed to be the selection operation triggered at the selection entry. It is worth mentioning that the selection operation will be different in specific ways depending on the input components configured by the electronic device.
  • the selection operation may be a gesture operation such as clicking or sliding; if the input component configured by the electronic device is a mouse, the selection operation may be a mechanical operation such as single-clicking, double-clicking, or dragging, which is not limited here.
  • the four input modes respectively acquire matching frame rates according to 1 frame per second, 10 frames per second, the video frequency of the pipeline video, and the user-defined frame rate.
  • the image frames in the pipeline video are extracted according to the acquired frame rates, so that multiple image frames to be detected at different frame rates can be obtained.
  • Step 320 performing frame-level defect detection on a plurality of image frames to be detected, and obtaining detection results for each image frame.
  • frame-level defect detection refers to pipeline defect detection using only one image frame to be detected in the pipeline video as the detection object. Compared with video-level defect detection, frame-level defect detection eliminates the interference and errors that may be caused by adjacent frames, making the accuracy of pipeline defect detection more reliable.
  • the detection result is used to indicate at least one of whether the pipeline has defects, whether the image shooting scene is in the pipeline, the defect category and its confidence, the defect level of the defect category, and the defect mask and its confidence.
  • defect categories are used to represent different defects, and the defect categories may include rupture, deformation, breakage, dislocation, disconnection, leakage, corrosion, rubber ring detachment, branch pipe concealed connection, foreign body insertion, etc.
  • the defect level of the defect category is used to indicate the severity of different defects.
  • the defect level of the defect category may include levels 1-5, and a higher defect level indicates a more severe defect.
  • the confidence of the defect category can be any value between 0 and 1.
  • the confidence of the defect category represents the probability of the defect category predicted by the image frame to be detected. It can be understood that the greater the confidence of the defect category, the more credible the defect category predicted by the image frame to be detected is.
  • the confidence of the defect mask can be any value between 0 and 1.
  • the defect mask refers to a detection frame used to mark the area with defects in the image frame to be detected. It can be understood that the greater the confidence of the defect mask, the more credible the area with defects in the image frame to be detected marked by the detection frame.
  • the detection results can also be used to indicate the maximum confidence of the defect category, the total confidence of all defect categories, the number of defect masks, the maximum confidence of the defect mask, the total confidence of all defect masks, etc., which does not constitute a specific limitation here.
  • Step 330 based on the detection results of each image frame, each image frame is screened and/or deduplicated to obtain a key frame.
  • the multiple image frames determined need to be screened and deduplicated, and only image frames that meet the screening and deduplication conditions will be retained as key frames.
  • Step 340 output the key frame detection result.
  • the output methods of the key frames detected from the pipeline video and the corresponding detection results include but are not limited to image output, video output, text output, etc.
  • the present application can select different output methods according to the actual needs of the application scenario and/or the computing power of the hardware device, which greatly improves the adaptability and versatility of the present application.
  • image output refers to marking the corresponding detection result in the key frame and outputting the marked key frame.
  • video output refers to marking key frames in the pipeline video based on the key frame detection results, and outputting the marked pipeline video.
  • the key frame detection result is output in text form.
  • the text content may include at least one of the following: the clock position of the key frame in the pipeline video, whether the pipeline has defects, whether the image shooting scene is in the pipeline, the defect category and its confidence, the maximum confidence of the defect category, the total confidence of all defect categories, the defect level of the defect category, the number of defect masks, the confidence of each defect mask, the maximum confidence of the defect mask, the total confidence of all defect masks, etc.
  • FIG4(a) a schematic diagram of an image frame to be detected in which defects exist in a pipeline is shown.
  • the defects represented by the four defect categories are respectively detachment represented by TL, deformation represented by BX, rupture represented by PL, and foreign body insertion represented by CR.
  • the confidence of each defect category is 0.77, 0.14, 0.09, and 0.08, respectively.
  • the defect level of each defect category is 4, 1, 1, and 1, respectively.
  • the defect mask corresponding to each defect is represented by the clock position of the defect represented by each defect category in the image frame, which is 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, 260, 270, 280, 290, 300, 310, 320, 330, 340, 360, 370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 560, 570, 580, 590, 600, 610, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710, 720, 730, 2.
  • the frame-level defect detection for the image frame is implemented based on a defect detection model, and the defect detection model is trained with a deep learning model so that it has the ability to perform frame-level defect detection for the image frame.
  • the deep learning model can be any one of FPN (Feature Pyramid Networks, feature pyramid), C3D and TimeSformer, which is not limited here.
  • the training process of the defect detection model may include the following steps: inputting key frames with defects and defect labels as training images into a deep learning model for defect detection, calculating a loss value based on the prediction accuracy between the detection results detected by the training image and the defect labels in the training image, and determining whether the deep learning model has converged to obtain a defect detection model based on the loss value and the model convergence condition.
  • the setting of the model convergence condition can be flexibly set according to the actual needs of the application scenario.
  • the model convergence condition can refer to the prediction accuracy between the detection results detected by the training image and the defect labels in the training image reaching 95%, or it can refer to the number of model iterations reaching a set threshold.
  • step 320 may include the following steps:
  • Step 321 performing multi-dimensional feature extraction on a plurality of image frames to be detected respectively to obtain multi-dimensional feature information of each image frame.
  • Step 322 based on the multi-dimensional feature information of each image frame, the pipeline defect category is detected for each image frame to obtain the detection result of each image frame.
  • the multi-dimensional feature extraction performed on multiple image frames to be inspected is to predict the different types of defects and related information that may exist in the image frames as much as possible, thereby obtaining multi-dimensional feature information of each image frame, wherein the multi-dimensional feature information is used to describe, including but not limited to: whether the pipeline has defects, whether the image shooting scene is inside the pipeline, the defect category and its confidence, the defect level of the defect category, the defect mask and its confidence, etc.
  • the detection results include but are not limited to: whether the pipeline has defects, whether the image shooting scene is inside the pipeline, defect category and its confidence, defect level of defect category, defect mask and its confidence.
  • positioning refers to marking the areas in the image frame that may have defects through the detection frame, so as to obtain each defect mask and its confidence
  • classification refers to predicting the defect category of the areas in the image frame that may have defects marked by each detection frame, so as to obtain each defect category and its confidence.
  • frame-level defect detection eliminates interference and errors that may be caused by adjacent frames, making the accuracy of pipeline defect detection more reliable.
  • the screening process for each image frame according to the detection result of each image frame may include at least one of the following processing methods:
  • each defect category and the confidence of each defect mask For each of the image frames, based on the detection results of the image frames, determine the confidence of each defect category and the confidence of each defect mask, and based on the confidence of each defect category and the confidence of each defect mask, determine the number of defect masks, the maximum confidence of the defect masks, the maximum confidence of the defect category, and the total confidence of all defect categories;
  • the image frame is used as a key frame.
  • the detection result of the image frame indicates that there are defects in the pipeline, it is further determined whether the maximum confidence of the defect category in the image frame exceeds a first confidence threshold. If it exceeds the first confidence threshold, the image frame is retained as a key frame, otherwise the key frame is removed.
  • the image frame is removed; conversely, if the confidence corresponding to the defect category of any of the two defects is greater than the first confidence threshold, the image frame is retained as a key frame.
  • the image frame is taken as a key frame.
  • the detection result of the image frame indicates that the number of defect masks in the image frame exceeds a set number and the maximum confidence of the defect mask is greater than a second confidence threshold, the image frame is retained as a key frame, otherwise the image frame is removed.
  • the confidences of the defect masks corresponding to two of the defects are 0.1 and 0.2 respectively, and the confidence of the defect mask corresponding to the other defect is 0.4. If 0.4 is greater than the second confidence threshold, the image frame is retained as the image frame; otherwise, when 0.4 is less than the second confidence threshold, the image frame is removed.
  • the image frame is used as a key frame.
  • the confidence of each defect category is determined according to the detection result of the image frame, and the total confidence of all defect categories is further obtained from the confidence of each defect category. If the total confidence of all defect categories exceeds the third confidence threshold, the image frame is retained as a key frame, otherwise the image frame is removed.
  • the confidences of the defect categories corresponding to the three defects are 0.1, 0.3, and 0.3, respectively, that is, the total confidence of the defect categories corresponding to the three defects is 0.7. If the total confidence of all defect categories exceeds the third confidence threshold, the image frame is retained as a key frame. Otherwise, when 0.7 is less than the third confidence threshold, the image frame is removed.
  • the number of defects and each confidence threshold listed in this embodiment can be flexibly set according to the actual needs of the application scenario and are not limited here; at the same time, the listed screening and processing methods are not limited to independent implementation, but can also be implemented in free combination. For example, combining the screening and processing methods 1 and 2, if the number of defect masks of the image frame exceeds the set number and the maximum confidence of the defect mask does not exceed the fourth confidence threshold, it is further determined whether there is at least one defect in the pipeline, and whether the maximum confidence of the defect category corresponding to the existing defect exceeds the first confidence threshold, thereby realizing the screening and processing of each image frame to be detected to obtain a key frame.
  • step 330 deduplication processing is performed on each image frame according to the detection result of each image frame, which may include the following steps:
  • Step 331 traverse each image frame, and use the traversed image frame as the current image frame.
  • Step 332 Determine the defect category and confidence level of the current image frame based on the detection result of the current image frame.
  • Step 333 if the interval time between the current image frame and the previous image frame is less than the set time, and the current image frame and the previous image frame have the same defect category with confidence greater than a fourth confidence threshold, then retain the current image frame.
  • the interval time between it and the previous image frame S2 is less than the set time, then further according to each defect category and its confidence of the image frame S1, and each defect category and its confidence of the image frame S2, it is determined whether the image frame S1 can be retained. Specifically, firstly, the defect categories of the image frame S1 and the image frame S2 whose confidence is greater than the fourth confidence threshold are determined respectively, and then based on the determined defect categories whose confidence is greater than the fourth confidence threshold, it is determined whether the defect categories determined by the image frame S1 and the image frame S2 are the same. If they are the same, the image frame S1 is retained, otherwise, if they are different, the image frame S1 is removed.
  • the set time can be flexibly adjusted according to the actual needs of the application scenario, and it is not limited here.
  • the execution order of the screening process and the deduplication process is not limited to the screening process or the deduplication process provided in this embodiment.
  • the screening process and the deduplication process can also be performed at the same time, for example, the screening process is performed first and then the deduplication process, or the deduplication process is performed first and then the screening process. This embodiment does not constitute a specific limitation on this.
  • the key frame screening and deduplication processing may include the following steps:
  • Step 1 If the number of defect masks in the image frame is greater than 2 and the maximum confidence of the defect masks exceeds 0.2, execute step 3 to determine whether the maximum confidence of the defect masks exceeds 0.3; otherwise, execute step 2.
  • Step 2 If there is at least one defect in the pipeline and the maximum confidence of these defects exceeds 0.2, then execute step 5; otherwise, delete the image frame.
  • Step 3 If the maximum confidence of the defect mask exceeds 0.3, execute step 5 to further perform deduplication processing on the image frame; otherwise, execute step 4.
  • Step 4 If the total confidence of the defect category exceeds 0.6, execute step 5 to further deduplicate the image frame; otherwise, delete the image frame.
  • multi-dimensional screening and deduplication processing of the image frames to be detected are realized, which not only realizes the precise positioning of pipeline defects in time and space, but also avoids a large amount of redundancy in the final result output or the inability to accurately find key frames.
  • the defect detection device can be deployed in an image acquisition device, various electronic devices on which the defect detection device can run as an application, various electronic devices that can be equipped with chips with defect detection capabilities, and cloud computing centers, cloud platforms, cloud services, etc. with defect detection capabilities.
  • the defect detection device includes a video input module, a video feature extraction module, an analysis and processing module, and a result output module.
  • the video input module reads the pipeline video, and according to the different input modes selected by the user, the image frames in the pipeline video are sampled at 1 frame per second, 10 frames per second, according to the video frame rate and the user input frame rate, to form an image frame sequence, which includes multiple image frames to be detected.
  • the image frame sequence is input into the video feature extraction module, and the multi-dimensional features at the frame level are extracted for each image frame of the image frame sequence, and then based on the multi-dimensional features of each image frame, the category prediction of whether there are defects in each image frame is performed to obtain the detection results of each image frame.
  • the video feature extraction module is composed of a defect detection model trained by a deep learning model; the extracted multi-dimensional features are used to describe, including but not limited to: whether the image shooting scene of each image frame is in the pipeline, whether the pipeline has defects, defect categories and their confidence, defect levels of defect categories, defect masks and their confidence, etc.; accordingly, the detection results include but are not limited to: whether the pipeline has defects, whether the image shooting scene is in the pipeline, defect categories and their confidence, defect levels of defect categories, defect masks and their confidence.
  • the detection results of each image frame are input into the analysis and processing module, the frame-level detection results are analyzed, and based on this, key frames are screened out for further deduplication.
  • the deduplicated key frames and their detection results are input into the result output module, which annotates the defect category, defect level, confidence of the defect category, defect mask, clock position of the key frame with defects in the pipeline video and other information on the key frames, the key frames in the pipeline video, or stores them as text content as result text, and then outputs the annotated key frames, the annotated pipeline video and the result text.
  • the following is an embodiment of the device of the present application, which can be used to execute the pipeline defect detection method involved in the present application.
  • the method embodiment of the pipeline defect detection method involved in the present application please refer to the method embodiment of the pipeline defect detection method involved in the present application.
  • An embodiment of the present application provides a pipeline defect detection device 600 , including but not limited to: an image acquisition module 610 , a defect detection module 620 , a screening and deduplication module 630 , and a result output module 640 .
  • the image acquisition module 610 is used to acquire the pipeline video and sample the pipeline video to obtain a plurality of image frames to be detected.
  • the defect detection module 620 is used to perform frame-level defect detection on multiple image frames to be detected, and obtain detection results of each image frame.
  • the detection result is used to indicate at least one of whether the pipeline has defects, whether the image shooting scene is inside the pipeline, defect category and its confidence, defect level of defect category, defect mask and its confidence.
  • the screening and deduplication module 630 is used to perform screening and/or deduplication processing on each image frame according to the detection result of each image frame to obtain a key frame.
  • the result output module 640 is used to output the detection result of the key frame.
  • pipeline defect detection device when the pipeline defect detection device provided in the above embodiment performs pipeline defect detection, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the pipeline defect detection device will be divided into different functional modules to complete all or part of the functions described above.
  • pipeline defect detection device and the pipeline defect detection method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiment and will not be repeated here.
  • Fig. 10 is a schematic diagram showing the structure of an electronic device according to an exemplary embodiment.
  • the electronic device is applicable to the server 170 in the implementation environment shown in Fig. 1 .
  • the electronic device is only an example adapted to the present application and cannot be considered to provide any limitation on the scope of use of the present application.
  • the electronic device cannot be interpreted as needing to rely on or necessarily having one or more components in the exemplary electronic device 2000 shown in FIG. 10 .
  • the hardware structure of the electronic device 2000 may vary greatly due to different configurations or performances.
  • the electronic device 2000 includes: a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .
  • CPU central processing unit
  • the power supply 210 is used to provide operating voltage for each hardware device on the electronic device 2000 .
  • the interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices.
  • FIG1 shows an image acquisition device 130 and a server 170 in an implementation environment.
  • the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, as shown in FIG. 10, which is not specifically limited here.
  • the memory 250 is a carrier for storing resources, which may be a read-only memory, a random access memory, a disk or an optical disk, etc.
  • the resources stored thereon include an operating system 251, an application 253 and data 255, etc.
  • the storage method may be temporary storage or permanent storage.
  • the operating system 251 is used to manage and control various hardware devices and application programs 253 on the electronic device 2000 to enable the central processor 270 to calculate and process the massive data 255 in the memory 250. It can be Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
  • the application 253 is a computer program that performs at least one specific task based on the operating system 251, and may include at least one module (not shown in FIG. 10 ), each of which may include a computer program for the electronic device 2000.
  • a pipeline defect detection device may be regarded as an application 253 deployed on the electronic device 2000.
  • the data 255 may be photos, images, etc. stored in a disk, or may be pipeline videos, etc., stored in the memory 250 .
  • the central processor 270 may include one or more processors, and is configured to communicate with the memory 250 through at least one communication bus to read the computer program stored in the memory 250, thereby realizing the operation and processing of the mass data 255 in the memory 250.
  • the pipeline defect detection method is completed in the form of the central processor 270 reading a series of computer programs stored in the memory 250.
  • present application can also be implemented through hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present application is not limited to any specific hardware circuits, software, or a combination of the two.
  • An embodiment of the present application provides an electronic device 4000 .
  • the electronic device 4000 may be a desktop computer, a notebook computer, a server, and the like.
  • the electronic device 4000 includes at least one processor 4001 , at least one communication bus 4002 , and at least one memory 4003 .
  • the processor 4001 and the memory 4003 are connected, such as through a communication bus 4002.
  • the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and/or data reception.
  • the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.
  • Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
  • the communication bus 4002 may include a path to transmit information between the above components.
  • the communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc.
  • the communication bus 4002 may be divided into an address bus, a data bus, a control bus, etc.
  • FIG. 11 only uses one thick line, but it does not mean that there is only one bus or one type of bus.
  • the memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.
  • ROM Read Only Memory
  • RAM Random Access Memory
  • EEPROM Electrically Erasable Programmable Read Only Memory
  • CD-ROM Compact Disc Read Only Memory
  • optical disk storage including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.
  • magnetic disk storage medium or other magnetic storage device or any
  • the memory 4003 stores a computer program
  • the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002 .
  • a storage medium is provided in an embodiment of the present application, on which a computer program is stored.
  • the computer program is executed by a processor, the pipeline defect detection method in the above-mentioned embodiments is implemented.
  • a computer program product includes a computer program, the computer program is stored in a storage medium.
  • a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the computer device executes the pipeline defect detection method in each of the above embodiments.
  • the pipeline defect detection method provided by this application can be applied to various pipelines such as urban drainage pipelines, oil and gas transportation pipelines, water supply and water delivery pipelines, power system pipelines, etc., and can be carried on a variety of platforms such as image acquisition equipment, chips, software and cloud services.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Image Analysis (AREA)

Abstract

本申请实施例提供了一种管道缺陷检测方法、装置、电子设备及存储介质，涉及目标检测领域。其中，该方法包括：获取管道视频，并对所述管道视频进行采样，得到多个待检测的图像帧；对多个待检测的图像帧分别进行帧级别的缺陷检测，得到各所述图像帧的检测结果；所述检测结果用于指示管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度中的至少一种；根据各所述图像帧的检测结果，对各所述图像帧进行筛选处理和/或去重处理，得到关键帧；将所述关键帧的检测结果输出。本申请实施例解决了相关技术中从管道视频中查找存在缺陷的关键帧费时费力的问题。

Description

管道缺陷检测方法、装置、电子设备及存储介质 技术领域
本申请涉及目标检测技术领域，具体而言，本申请涉及一种管道缺陷检测方法、装置、电子设备及存储介质。
 背景技术
地下管网系统作为城市建设发展中最重要的基础设施之一，有着“城市血管”之称，为了保证其正常运行，需要进行定期检测，方可在检测到缺陷后及时进行修复。
现有技术通常是根据管道QV（Quick-View，快速查看）检测和管道CCTV（Closed Circuit Television Inspection，闭路电视检测）检测采集大量的管道视频，然后经由检测人员分析管道视频是否存在缺陷和损坏，其中，对缺陷或损坏的认定是在该管道视频中的某一帧上进行标注，来代表该时间段上的管道缺陷或损坏，相应地，该某一帧成为存在缺陷或损坏的关键帧。然而，从大量管道视频中查找存在缺陷或损坏的关键帧是相当费时费力的。
由上可知，如何从管道视频中自动快速准确地查找到存在缺陷或损坏的关键帧的问题仍有待解决。
 技术问题
本申请各实施例提供了一种管道缺陷检测方法、装置、电子设备及存储介质，可以解决相关技术中存在的从管道视频中查找存在缺陷或损坏的关键帧费时费力的问题。
 技术解决方案
根据本申请实施例的一个方面，一种管道缺陷检测方法，所述方法包括：获取管道视频，并对所述管道视频进行采样，得到多个待检测的图像帧；对多个待检测的图像帧分别进行帧级别的缺陷检测，得到各所述图像帧的检测结果；所述检测结果用于指示管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度中的至少一种；根据各所述图像帧的检测结果，对各所述图像帧进行筛选处理和/或去重处理，得到关键帧；将所述关键帧的检测结果输出。
根据本申请实施例的一个方面，一种管道缺陷检测装置，所述装置包括：图像获取模块，用于获取管道视频，并对所述管道视频进行采样，得到多个待检测的图像帧；缺陷检测模块，用于对多个待检测的图像帧分别进行帧级别的缺陷检测，得到各所述图像帧的检测结果；所述检测结果用于指示管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度中的至少一种；筛选去重模块，用于根据各所述图像帧的检测结果，对各所述图像帧进行筛选处理和/或去重处理，得到关键帧；结果输出模块，用于将所述关键帧的检测结果输出。
在一示例性实施例中，所述图像获取模块包括：帧率获取单元，用于响应于针对不同输入模式进行的选择操作，获取与选中的输入模式匹配的帧率；采样单元，用于按照获取到的帧率对所述管道视频中的图像帧进行采样，得到多个待检测的图像帧。
在一示例性实施例中，所述缺陷检测模块包括：特征提取单元，用于分别对多个待检测的图像帧进行多维度特征提取，得到各所述图像帧的多维度特征信息；类别预测单元，用于根据各所述图像帧的多维度特征信息，分别对各所述图像帧进行管道缺陷类别的预测，得到各所述图像帧的检测结果。
在一示例性实施例中，所述帧级别的缺陷检测是基于缺陷检测模型实现的，所述缺陷检测模型是经过训练、且具有对图像帧进行帧级别的缺陷检测的能力的深度学习模型。
在一示例性实施例中，所述筛选去重模块包括：第一确定单元，用于针对每一个所述图像帧，基于所述图像帧的检测结果，确定各缺陷类别的置信度和各缺陷蒙版的置信度，并基于各缺陷类别的置信度和各缺陷蒙版的置信度，确定缺陷蒙版的数量、缺陷蒙版的最大置信度、缺陷类别的最大置信度、以及所有缺陷类别的总置信度；第一筛选单元，用于若所述图像帧的检测结果指示管道存在缺陷，且缺陷类别的最大置信度大于第一置信度阈值，则将所述图像帧作为关键帧；第二筛选单元，用于若所述图像帧的缺陷蒙版的数量大于设定数量，且缺陷蒙版的最大置信度大于第二置信度阈值，则将所述图像帧作为关键帧；第三筛选单元，用于若所述图像帧中所有缺陷类别的总置信度大于第三置信度阈值，则将所述图像帧作为关键帧。
在一示例性实施例中，所述筛选去重模块包括：图像遍历单元，用于针对各所述图像帧进行遍历，以遍历到的所述图像帧作为当前一个所述图像帧；第二确定单元，用于基于当前一个所述图像帧的检测结果，确定当前一个所述图像帧的缺陷类别及其置信度；图像保留单元，用于若当前一个所述图像帧与前一个所述图像帧的间隔时间小于设定时间，且当前一个所述图像帧与前一个所述图像帧的置信度大于第四置信度阈值的缺陷类别相同，则保留所述图像帧。
在一示例性实施例中，所述结果输出模块包括：图像输出单元，用于在所述关键帧中标注对应的检测结果，并输出标注后的关键帧；视频输出单元，用于基于所述关键帧的检测结果，在所述管道视频中对所述关键帧进行标注，并输出标注后的管道视频；文本输出单元，用于以文本方式输出所述关键帧的检测结果。
根据本申请实施例的一个方面，一种电子设备，包括：至少一个处理器、至少一个存储器、以及至少一条通信总线，其中，存储器上存储有计算机程序，处理器通过通信总线读取存储器中的计算机程序；计算机程序被处理器执行时实现如上所述的管道缺陷检测方法。
根据本申请实施例的一个方面，一种存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现如上所述的管道缺陷检测方法。
根据本申请实施例的一个方面，一种计算机程序产品，计算机程序产品包括计算机程序，计算机程序存储在存储介质中，计算机设备的处理器从存储介质读取计算机程序，处理器执行计算机程序，使得计算机设备执行时实现如上所述的管道缺陷检测方法。
 有益效果
本申请提供的技术方案带来的有益效果是：
在上述技术方案中，在对管道视频中多个待检测的图像帧进行帧级别的缺陷检测时，通过自动地对管道视频进行帧级别的定位和分类，实现了对管道缺陷在时间和空间上的精确定位，并以帧级别的检测结果为依据进行图像帧的筛选和去重，通过筛选和去重去除了大量的冗余，避免人工分析管道视频是否存在缺陷或损坏，实现了自动、高效和准确的管道缺陷检测，能够有效地解决相关技术中存在的从管道视频中查找存在缺陷的关键帧费时费力的问题。
 附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对本申请实施例描述中所需要使用的附图作简单地介绍。
图1是本申请所涉及的实施环境的示意图；
图2是根据一示例性实施例示出的一种管道检测方法的流程图；
图3是根据一示例性实施例示出的步骤310的流程图；
图4是根据一示例性实施例示出的一种结果输出示意图；
图5是根据一示例性实施例示出的步骤320的流程图；
图6是根据一示例性实施例示出的步骤330的流程图；
图7是根据一示例性实施例示出的关键帧筛选和去重的流程图；
图8是根据一示例性实施例示出的一种应用场景示意图；
图9是根据一示例性实施例示出的一种管道缺陷检测装置的结构框图；
图10是根据一示例性实施例示出的一种电子设备的硬件结构图；
图11是根据一示例性实施例示出的一种电子设备的结构框图。
 本发明的实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本申请，而不能解释为对本申请的限制。
本技术领域技术人员可以理解，除非特意声明，这里使用的单数形式“一”、“一个”、“所述”和“该”也可包括复数形式。应该进一步理解的是，本申请的说明书中使用的措辞“包括”是指存在所述特征、整数、步骤、操作、元件和/或组件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。应该理解，当我们称元件被“连接”或“耦接”到另一元件时，它可以直接连接或耦接到其他元件，或者也可以存在中间元件。此外，这里使用的“连接”或“耦接”可以包括无线连接或无线耦接。这里使用的措辞“和/或”包括一个或更多个相关联的列出项的全部或任一单元和全部组合。
如前所述，作为城市建设发展中最重要的基础设施之一的地下管网系统，对其进行定期检测从而及时发现并修复缺陷是至关重要的。
为了实现对地下管网系统的检测，现有技术提出了基于管道QV（Quick-View，快速查看）检测和管道CCTV（Closed Circuit Television Inspection，闭路电视检测）检测两种管道缺陷检测方法。其中，管道QV检测通过高清可变焦摄像头，结合灯光照明辅助，采集清晰的管道内部图像，进而查找管道内部存在的缺陷或损坏；管道CCTV检测主要是通过闭路电视录像的形式，使用摄像设备进入管道将影像数据传输至上位机进行管道缺陷检测。
管道视频中的缺陷往往会在该管道视频中的某个时间段持续显示，但是对缺陷的认定是在该管道视频中的某一帧进行标注，来代表该时间段存在的缺陷，该某一帧称为存在缺陷的关键帧。对于该关键帧，通常需要能够清晰地看到管道缺陷并能够判断该管道缺陷的严重程度。
然而，上述管道缺陷检测方法，大多是视频级别的缺陷检测，即对管道视频整体进行分析和检测，以输出整个管道视频中包含的缺陷，不涉及时序上的定位，对于时长较长且包含多种缺陷的管道视频，需要人工对该管道视频进行裁剪才能对缺陷进行时序定位。
进一步地，虽然少数方法实现了帧级别的缺陷检测，即能够输出管道视频中的每个图像帧上是否包含缺陷及其缺陷类别，但是由于缺少去重步骤或仅仅通过输出的图像帧在时序上的密集程度来去重，导致输出的图像帧存在大量的冗余或无法从输出的图像帧中找到关键帧。
可见，相关技术中仍存在从管道视频中查找关键帧费时费力的问题。
为此，本申请提供的管道缺陷检测方法，能够准确定位并分类管道中的缺陷，并对缺陷严重程度进行了评估，同时通过对管道视频中待检测的图像帧进行筛选和去重查找出关键帧，实现了对管道缺陷在时间和空间上的精准定位，避免人工实现管道缺陷检测，能够有效地提高管道缺陷检测的自动化程度和效率。相应地，该管道缺陷检测方法适用于管道缺陷检测装置，该管道缺陷检测装置可部署于电子设备，该电子设备可以是部署有冯诺依曼体系架构的计算机设备，例如，该计算机设备可以是台式电脑、笔记本电脑、服务器等等。
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
图1为一种管道缺陷检测方法所涉及的实施环境的示意图。该实施环境包括图像采集设备130、网关150、服务器端170和路由器190。
具体地，图像采集设备130可以是管道QV潜望镜、CCTV管道检测机器人等用于管道缺陷检测的电子设备。
服务器端170可以是台式电脑、笔记本电脑、服务器等具有通信连接功能的电子设备，还可以是由多台服务器构成的计算机设备集群，甚至是由多台服务器构成的云计算中心。其中，服务器用于提供后台服务，例如，后台服务包括但不限于管道缺陷检测服务等等。
图像采集设备130与服务器端170之间的交互，在一个应用场景中，图像采集设备130通过局域网络接入网关150，从而与服务器端170进行数据传输。图像采集设备130通过局域网络接入网关150的过程包括：由网关150首先建立一个局域网络，图像采集设备130通过连接该网关150，从而加入该网关150建立的局域网络中。此局域网络包括但不限于：ZIGBEE、蓝牙、WIFI等，进而通过路由器190实现与服务器端170进行数据传输。在另一个应用场景中，图像采集设备130与服务器端170之间通过广域网络交互，该广域网络可以通过建立有线或者无线等方式的通信连接实现，例如，该有线或者无线等方式包括但不限于2G、3G、4G、5G、WIFI等，从而实现与服务器端170进行数据传输。例如，传输的数据可以是管道视频等。
随着图像采集设备130与服务器端170的交互，图像采集设备130负责采集管道视频并传输至服务器端170，以请求服务器端170提供管道缺陷检测服务。
对于服务器端170而言，便可基于接收到的管道视频进行管道缺陷检测，具体是指，对管道缺陷进行采样得到多个待检测的图像帧，并对多个待检测的图像帧分别进行帧级别的缺陷检测，得到各图像帧的检测结果，进而根据各图像帧的检测结果，对各图像帧进行筛选处理和/或去重处理，得到关键帧，最终将关键帧的检测结果输出。
基于上述过程，便实现了自动、高效、准确的管道缺陷检测。
当然，根据实际营运的需求，图像采集设备110也可以与服务器端170整合在同一电子设备，以使管道缺陷检测过程由该同一电子设备完成。
请参阅图2，本申请实施例提供了一种管道缺陷检测方法，该方法适用于电子设备，该电子设备可以是图1所示出实施环境中的服务器端170。
在下述方法实施例中，为了便于描述，以该方法各步骤的执行主体为电子设备为例进行说明，但是并非对此构成具体限定。
如图2所示，该方法可以包括以下步骤：
步骤310，获取管道视频，并对管道视频进行采样，得到多个待检测的图像帧。
具体的，该管道视频可以是通过管道QV潜望镜、CCTV管道检测机器人对管道进行拍摄得到的，其中，图像拍摄场景可以包括管道内以及管道外。
在得到管道视频后，针对管道视频中的大量图像帧，将进一步对管道视频中的大量图像帧进行抽帧，以便于减少后续进行管道缺陷检测的图像帧的数量，从而提高管道缺陷检测的效率。本实施例中，抽帧是通过对管道视频中的图像帧进行采样实现的。
在一种可能的实现方式，采样是按照不同的帧率进行的。其中，帧率可以根据应用场景的实际需要灵活地设置，此处并未加以限定。
具体而言，如图3所示，上述步骤310可以包括以下步骤：
步骤311，响应于针对不同输入模式进行的选择操作，获取与选中的输入模式匹配的帧率。
步骤312，按照获取到的帧率对管道视频中的图像帧进行采样，得到多个待检测的图像帧。
具体的，本申请提供了四种不同的输入模式：性能、平衡、精密以及自定义。相应地，该四种不同的输入模式对应的帧率分别是：每秒1帧、每秒10帧、管道视频的视频帧率、以及用户自定义的帧率。
为了用户能够对不同输入模型进行选择，进而相应地设置采样所需要的帧率，电子设备中设置有选择入口，用户可以通过在该选择入口触发进行的选择操作，选择不同的输入模式。例如，假设该选择入口是一下拉列表，该下拉列表包含可供选择的四种输入模式，若用户点击其中一个输入模式，则该其中一个输入模式视为被用户选中的输入模式，该点击操作视为在该选择入口触发进行的选择操作。值得一提的是，选择操作根据电子设备所配置输入组件的不同，具体方式也将有所区别，例如，若电子设备配置的输入组件为触摸屏幕，则选择操作可以是点击、滑动等手势操作；若电子设备配置的输入组件为鼠标，则选择操作可以是单击、双击、拖拽等机械操作，此处并未加以限定。
响应于用户针对四种输入模式进行的选择操作，四种输入模式分别按照每秒1帧、每秒10帧、管道视频的视频频率、以及用户自定义的帧率对匹配的帧率进行获取，根据获取到的帧率对管道视频中的图像帧进行抽帧，便能够得到不同帧率下的多个待检测的图像帧。
通过设置四种不同的输入模式，满足了不同应用场景不同设备算力下的需求选择，扩展了本申请的泛化性。
步骤320，对多个待检测的图像帧分别进行帧级别的缺陷检测，得到各图像帧的检测结果。
首先说明的是，帧级别的缺陷检测，是指仅以管道视频中的一个待检测的图像帧作为检测对象进行管道缺陷检测，相比于视频级别的缺陷检测，帧级别的缺陷检测排除了相邻帧可能带来的干扰以及误差，使得管道缺陷检测的准确率更加可靠。
其次，检测结果用于指示管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度中的至少一种。
在一种可能的实现方式，缺陷类别用于表示不同缺陷，该缺陷类别可以包括破裂、变形、破裂、错位、脱节、渗漏、腐蚀、胶圈脱落、支管暗接、异物插入等。
在一种可能的实现方式，缺陷类别的缺陷等级用于表示不同缺陷的严重程度。其中，缺陷类别的缺陷等级可以包括1-5级，随着缺陷等级越高，表示缺陷越严重。
在一种可能的实现方式，缺陷类别的置信度可以是0-1内的任意值。其中，缺陷类别的置信度表示待检测的图像帧预测到的缺陷类别的概率，可以理解，缺陷类别的置信度越大，表示待检测的图像帧预测到的缺陷类别越可信。
在一种可能的实现方式，缺陷蒙版的置信度可以是0-1内的任意值。其中，缺陷蒙版是指用于标记待检测的图像帧中存在缺陷的区域的检测框，可以理解，缺陷蒙版的置信度越大，表示检测框所标记的待检测的图像帧中存在缺陷的区域越可信。
当然，在其他实施例中，基于缺陷类别及其置信度和缺陷蒙版及其置信度，检测结果还可以用于指示缺陷类别的最大置信度、所有缺陷类别的总置信度、缺陷蒙版的数量、缺陷蒙版的最大置信度、所有缺陷蒙版的总置信度等等，此处并非构成具体限定。
步骤330，根据各图像帧的检测结果，对各图像帧进行筛选处理和/或去重处理，得到关键帧。
具体的，为了减少相关检测人员的审核工作量，对于确定的多幅图像帧还需经过筛选以及去重处理，只有满足筛选以及去重条件的图像帧才会被保留作为关键帧。
步骤340，将关键帧的检测结果输出。
其中，对于不同应用场景以及不同需求，从管道视频中检测到的关键帧及对应的检测结果的输出方式，包括但不限于图像输出、视频输出、文本输出等。此种方式下，相对于现有技术中受限于硬件设备的算力所导致的输出方式有限，本申请可以根据应用场景的实际需要和/或硬件设备的算力选择不同的输出方式，大大提高了本申请的适应性和通用性。
在一种可能的实现方式，图像输出，是指在关键帧中标注对应的检测结果，并输出标注后的关键帧。
在一种可能的实现方式，视频输出，是指基于关键帧的检测结果，在管道视频中对关键帧进行标注，并输出标注后的管道视频。
在一种可能的实现方式，以文本方式输出关键帧的检测结果。其中，文本内容可以包括以下至少一种：关键帧在管道视频中的时钟位置、管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的最大置信度、所有缺陷类别的总置信度、缺陷类别的缺陷等级、缺陷蒙版的数量、各缺陷蒙版的置信度、缺陷蒙版的最大置信度、所有缺陷蒙版的总置信度等等。
在一示例性实施例中，如图4（a）所示，展示了管道内存在缺陷的待检测的图像帧的示意图，在按照图像输出的输出方式将关键帧的检测结果输出时，如图4（b）所示，展示了关键帧中预测到的四种缺陷类别及相关信息，该四种缺陷类别分别表示的缺陷为TL表示的脱落、BX表示的变形、PL表示的破裂、CR表示的异物插入，各缺陷类别的置信度分别为0.77、0.14、0.09、0.08，各缺陷类别的缺陷等级分别为4、1、1、1；各缺陷对应的缺陷蒙版通过各缺陷类别所表示缺陷在图像帧中的时钟位置表示，分别为1002、1002、0403、1002，其中，1002表示缺陷在图像帧中的区域是十点钟位置到两点钟位置之间的扇形区域，0403则表示缺陷在图像帧中的区域是四点钟位置到三点钟位置之间的扇形区域；图像拍摄场景是否在管道内，其中，（0.71000004，out）表示图像拍摄场景有71%的概率在管道外，（0.29，int）表示图像拍摄场景有29%的概率在管道内；管道有无缺陷，其中，（0.42，normal）表示管道有42%的概率没有缺陷，（0.58，defect）表示管道有58%的概率存在缺陷。
通过上述过程，实现了自动对管道视频进行缺陷识别的同时，实现了时间和空间上的精确定位，并且通过筛选和去重去除了大量的冗余，减少检测人员在核查时的工作量。通过视频、图像以及文本的多种结果呈现方式，更加直观的反应管道内的缺陷情况。除此之外，通过将输入帧率设置为可调节的，满足了不同设备的算力要求以及用户对性能和精确度的权衡。
在一示例性实施例中，针对图像帧进行的帧级别的缺陷检测是基于缺陷检测模型实现的，该缺陷检测模型经过对深度学习模型进行训练，使其具有对图像帧进行帧级别的缺陷检测的能力。其中，该深度学习模型可以是FPN（Feature Pyramid Networks，特征金字塔）、C3D以及TimeSformer中的任意一种，在此并未加以限定。
在一种可能的实现方式，缺陷检测模型的训练过程可以包括以下步骤：将存在缺陷并包含缺陷标签的关键帧作为训练图像输入深度学习模型进行缺陷检测，根据训练图像检测到的检测结果与该训练图像中的缺陷标签之间的预测准确度计算损失值，并基于损失值与模型收敛条件，确定深度学习模型是否收敛得到缺陷检测模型。其中，模型收敛条件的设定可以根据应用场景的实际需要灵活地设置，例如，模型收敛条件可以是指训练图像检测到的检测结果与训练图像中的缺陷标签之间的预测准确度达到95%，还可以是指模型迭代次数达到设定阈值。经过上述训练过程，便得到了具有对图像帧进行帧级别的缺陷检测的能力的缺陷检测模型。
下面基于缺陷检测模型，对帧级别的缺陷检测进行以下详细说明：
请参阅图5，在一示例性实施例中，步骤320可以包括以下步骤：
步骤321，分别对多个待检测的图像帧进行多维度特征提取，得到各图像帧的多维度特征信息。
步骤322，根据各图像帧的多维度特征信息，分别对各图像帧进行管道缺陷类别的检测，得到各图像帧的检测结果。
具体的，针对多个待检测的图像帧进行的多维度特征提取是为了从图像帧中尽可能的预测到可能存在的不同类别的缺陷及其相关信息，从而得到各图像帧的多维度特征信息，其中，该多维度特征信息用于描述包括但不限于：管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度等。
在得到各图像帧的多维度特征信息后，便能够基于各图像帧的多维度特征信息，对各图像帧中可能存在缺陷的区域进行定位和分类，最终得到各图像帧的检测结果。该检测结果包括但不限于：管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度。其中，定位是指通过检测框对图像帧中可能存在缺陷的区域加以标记，从而得到各缺陷蒙版及其置信度；分类则是指对图像帧中各检测框标记的可能存在缺陷的区域进行缺陷类别预测，从而得到各缺陷类别及其置信度。
在上述实施例的作用下，相比于视频级别的缺陷检测，帧级别的缺陷检测排除了相邻帧可能带来的干扰以及误差，使得管道缺陷检测的准确率更加可靠。
在一示例性实施例中，步骤330中根据各图像帧的检测结果，对各图像帧进行筛选处理，可以包括以下至少一种处理方式：
针对每一个所述图像帧，基于所述图像帧的检测结果，确定各缺陷类别的置信度和各缺陷蒙版的置信度，并基于各缺陷类别的置信度和各缺陷蒙版的置信度，确定缺陷蒙版的数量、缺陷蒙版的最大置信度、缺陷类别的最大置信度、以及所有缺陷类别的总置信度；
1、若图像帧的检测结果指示管道存在缺陷，且缺陷类别的最大置信度大于第一置信度阈值，则将图像帧作为关键帧。
具体的，根据图像帧的检测结果指示管道内存在缺陷，则进一步判断该图像帧中缺陷类别的最大置信度是否超过第一置信度阈值，若超过第一置信度阈值则保留该图像帧作为关键帧，否则去除该关键帧。
例如，一个图像帧中存在两处缺陷，若该两处缺陷的缺陷类别所对应的置信度均小于第一置信度阈值，则去除该图像帧；反之，若两处缺陷中有任一缺陷的缺陷类别所对应的置信度大于第一置信度阈值，则保留该图像帧作为关键帧。
2、若图像帧的缺陷蒙版的数量大于设定数量，且缺陷蒙版的最大置信度大于第二置信度阈值，则将图像帧作为关键帧。
具体的，根据图像帧的检测结果指示该图像帧中缺陷蒙版的数量超过设定数量，且缺陷蒙版的最大置信度大于第二置信度阈值，则保留该图像帧作为关键帧，否则去除该图像帧。
例如，一个图像帧中存在三处缺陷，其中两处缺陷对应的缺陷蒙版的置信度分别为0.1、0.2，另一处缺陷对应的缺陷蒙版的置信度为0.4，若0.4大于第二置信度阈值，则保留该图像帧作为图像帧，否则，在0.4小于第二置信度阈值时，去除该图像帧。
3、若图像帧中所有缺陷类别的总置信度大于第三置信度阈值，则将图像帧作为关键帧。
具体的，根据图像帧的检测结果确定各缺陷类别的置信度，并由各缺陷类别的置信度进一步得到所有缺陷类别的总置信度，若所有缺陷类别的总置信度超过第三置信度阈值，则保留该图像帧作为关键帧，否则去除该图像帧。
例如，一个图像帧中存在三处缺陷，该三处缺陷对应的缺陷类别的置信度分别为0.1、0.3、0.3，即该三处缺陷对应的缺陷类别的总置信度为0.7，若所有缺陷类别的总置信度超过了第三置信度阈值，则该图像帧保留作为关键帧，否则，在0.7小于第三置信度阈值时，去除该图像帧。
需要说明的是，本实施例中列举的缺陷数量、各置信度阈值可以根据应用场景的实际需要灵活地设置，此处并未加以限定；同时，所列举的筛选处理方式并非限定在独立实施，还可以自由组合后实施，例如，结合1和2的筛选处理方式，若图像帧的缺陷蒙版的数量超过设定数量且缺陷蒙版的最大置信度未超过第四置信度阈值时，则进一步判断管道内是否存在至少一种缺陷、以及存在的缺陷所对应缺陷类别的最大置信度是否超过第一置信度阈值，从而实现对待检测的各图像帧的筛选处理而得到关键帧。
进一步地，在一示例性实施例中，如图6所示，步骤330中根据各图像帧的检测结果，对各图像帧进行去重处理，可以包括以下步骤：
步骤331，针对各图像帧进行遍历，以遍历到的所述图像帧作为当前一个图像帧。
步骤332，基于当前一个图像帧的检测结果，确定当前一个图像帧的缺陷类别及其置信度。
步骤333，若当前一个图像帧与前一个图像帧的间隔时间小于设定时间，且当前一个图像帧与前一个图像帧的置信度大于第四置信度阈值的缺陷类别相同，则保留当前一个图像帧。
举例来说，对于管道视频中的某一个图像帧S1，若其与前一个图像帧S2的间隔时间小于设定时间，则进一步根据图像帧S1的各缺陷类别及其置信度、以及图像帧S2的各缺陷类别及其置信度，判断图像帧S1是否可以保留，具体地，首先分别确定图像帧S1与图像帧S2置信度大于第四置信度阈值的缺陷类别，然后基于所确定的置信度大于第四置信度阈值的缺陷类别，判断图像帧S1与图像帧S2所确定的缺陷类别是否相同，若相同，则保留该图像帧S1，反之，若不同，则去除该图像帧S1。其中，设定时间可以根据应用场景的实际需要灵活地调整，此处并未加以限定。
值得一提的是，筛选处理和去重处理的执行顺序并非局限于本实施例所提供的筛选处理或者去重处理，还可以同时进行筛选处理和去重处理，例如先筛选处理后去重处理，或者先去重处理后筛选处理，本实施例并非对此构成具体限定。
如图7所示，在一种可能的实现方式，关键帧的筛选处理和去重处理，可以包括以下步骤：
步骤1、若图像帧中缺陷蒙版的数量大于2且缺陷蒙版的最大置信度超过0.2，则执行步骤3，判断缺陷蒙版的最大置信度是否超过0.3；否则，执行步骤2。
步骤2、若管道内存在至少一种缺陷、且该些缺陷的最大置信度超过0.2，则执行步骤5；否则，删除该图像帧。
步骤3、若缺陷蒙版的最大置信度超过0.3，则执行步骤5，进一步对图像帧进行去重处理；否则，执行步骤4。
步骤4、若缺陷类别的总置信度超过0.6，则执行步骤5，进一步对图像帧进行去重处理；否则，删除该图像帧。
步骤5、若该图像帧与前一个图像帧的时间间隔小于5秒，并且二者置信度大于0.2的缺陷类别相同，则保留该图像帧作为关键帧；否则，删除该图像帧。
通过上述实施例的配合，实现了对待检测的图像帧多维度的筛选处理和去重处理，不仅实现了管道缺陷在时间和空间上的精准定位，而且能够避免最终结果输出时存在大量的冗余或者无法准确地查找到关键帧。
请参阅图8，在一应用场景中，缺陷检测装置可以部署在图像采集设备、可供缺陷检测装置作为应用程序运行的各种电子设备、可搭载具有缺陷检测能力的芯片的各种电子设备、以及具有缺陷检测能力的云计算中心、云平台、云服务等等。
在图8中，该缺陷检测装置包括视频输入模块、视频特征提取模块、分析与处理模块、结果输出模块。
具体而言，首先，由视频输入模块对管道视频进行读取，并根据用户所选择输入模式的不同，分别按照每秒1帧、每秒10帧、按照视频帧率和用户输入帧率对管道视频中的图像帧进行采样，形成图像帧序列，该图像帧序列包括多个待检测的图像帧。
然后，将该图像帧序列输入视频特征提取模块，对图像帧序列的各图像帧提取帧级别的多维度特征，进而基于各图像帧的多维度特征，对各图像帧中是否存在缺陷进行类别预测，得到各图像帧的检测结果。其中，该视频特征提取模块由深度学习模型训练的缺陷检测模型构成；提取得到的多维度特征用于描述包括但不限于：每一个图像帧的图像拍摄场景是否在管道内、管道有无缺陷、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度等等；相应地，检测结果包括但不限于：管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度。
接着，将各图像帧的检测结果输入分析与处理模块，对帧级别的检测结果进行分析，并以此为依据筛选出关键帧后作进一步地去重。
最后，将去重后的关键帧及其检测结果输入结果输出模块，由该结果输出模块将缺陷类别、缺陷等级、缺陷类别的置信度、缺陷蒙版、存在缺陷的关键帧在管道视频中时钟位置等信息分别标注在关键帧、管道视频中的关键帧、或者作为文本内容存储为结果文本，进而将标注后的关键帧、标注后的管道视频、结果文本输出。
在上述应用场景，实现自动对管道视频进行帧级别的缺陷检测，实现了时间和空间上的精确定位，同时通过筛选和去重去除了大量冗余，减少了检测人员在核查时的工作量。此外，通过视频、图像以及文本等多种结果呈现方式，更加直观的反映了管道内的缺陷情况，使得本应用场景所提供的管道缺陷检测方法的应用范围广泛，可以适用于城市排水管道、油气运输管道、供水输水管道、电力系统管道等管道系统，具有较好的适应性；而且通过可调节的帧率，满足不同硬件设备的算力要求以及用户对性能和精确度的权衡，使得本应用场景所提供的管道缺陷检测方法可以搭载多种平台，提高了管道检测缺陷方法的通用性。
下述为本申请装置实施例，可以用于执行本申请所涉及的管道缺陷检测方法。对于本申请系统实施例中未披露的细节，请参照本申请所涉及的管道缺陷检测方法的方法实施例。
请参阅图9，本申请实施例提供了一种管道缺陷检测装置600，包括但不限于：图像获取模块610、缺陷检测模块620、筛选去重模块630、结果输出模块640。
其中，图像获取模块610，用于获取管道视频，并对管道视频进行采样，得到多个待检测的图像帧。
缺陷检测模块620，用于对多个待检测的图像帧分别进行帧级别的缺陷检测，得到各图像帧的检测结果。检测结果用于指示管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度中的至少一种。
筛选去重模块630，用于根据各图像帧的检测结果，对各图像帧进行筛选处理和/或去重处理，得到关键帧。
结果输出模块640，用于将关键帧的检测结果进行输出。
需要说明的是，上述实施例所提供的管道缺陷检测装置在进行管道缺陷检测时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即管道缺陷检测装置的内部结构将划分为不同的功能模块，以完成以上描述的全部或者部分功能。
另外，上述实施例所提供的管道缺陷检测装置与管道缺陷检测方法的实施例属于同一构思，其中各个模块执行操作的具体方式已经在方法实施例中进行了详细描述，此处不再赘述。
图10根据一示例性实施例示出的一种电子设备的结构示意图。该电子设备适用于图1所示出实施环境中的服务器端170。
需要说明的是，该电子设备只是一个适配于本申请的示例，不能认为是提供了对本申请的使用范围的任何限制。该电子设备也不能解释为需要依赖于或者必须具有图10示出的示例性的电子设备2000中的一个或者多个组件。
电子设备2000的硬件结构可因配置或者性能的不同而产生较大的差异，如图10所示，电子设备2000包括：电源210、接口230、至少一存储器250、以及至少一中央处理器（CPU, Central Processing Units）270。
具体地，电源210用于为电子设备2000上的各硬件设备提供工作电压。
接口230包括至少一有线或无线网络接口231，用于与外部设备交互。例如，图1所示出实施环境中的图像采集设备130与服务器端170。
当然，在其余本申请适配的示例中，接口230还可以进一步包括至少一串并转换接口233、至少一输入输出接口235以及至少一USB接口237等，如图10所示，在此并非对此构成具体限定。
存储器250作为资源存储的载体，可以是只读存储器、随机存储器、磁盘或者光盘等，其上所存储的资源包括操作系统251、应用程序253及数据255等，存储方式可以是短暂存储或者永久存储。
其中，操作系统251用于管理与控制电子设备2000上的各硬件设备以及应用程序253，以实现中央处理器270对存储器250中海量数据255的运算与处理，其可以是Windows ServerTM、Mac OS XTM、UnixTM、LinuxTM、FreeBSDTM等。
应用程序253是基于操作系统251之上完成至少一项特定工作的计算机程序，其可以包括至少一模块（图10未示出），每个模块都可以分别包含有对电子设备2000的计算机程序。例如，管道缺陷检测装置可视为部署于电子设备2000的应用程序253。
数据255可以是存储于磁盘中的照片、图像等，还可以是管道视频等，存储于存储器250中。
中央处理器270可以包括一个或多个以上的处理器，并设置为通过至少一通信总线与存储器250通信，以读取存储器250中存储的计算机程序，进而实现对存储器250中海量数据255的运算与处理。例如，通过中央处理器270读取存储器250中存储的一系列计算机程序的形式来完成管道缺陷检测方法。
此外，通过硬件电路或者硬件电路结合软件也能同样实现本申请，因此，实现本申请并不限于任何特定硬件电路、软件以及两者的组合。
请参阅图11，本申请实施例提供了一种电子设备4000，该电子设备4000可以台式电脑、笔记本电脑、服务器等等。
在图11中，该电子设备4000包括至少一个处理器4001、至少一条通信总线4002以及至少一个存储器4003。
其中，处理器4001和存储器4003相连，如通过通信总线4002相连。可选地，电子设备4000还可以包括收发器4004，收发器4004可以用于该电子设备与其他电子设备之间的数据交互，如数据的发送和/或数据的接收等。需要说明的是，实际应用中收发器4004不限于一个，该电子设备4000的结构并不构成对本申请实施例的限定。
处理器4001可以是CPU（Central Processing Unit，中央处理器），通用处理器，DSP（Digital Signal Processor，数据信号处理器），ASIC（Application Specific Integrated Circuit，专用集成电路），FPGA（Field Programmable Gate Array，现场可编程门阵列）或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器4001也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等。
通信总线4002可包括一通路，在上述组件之间传送信息。通信总线4002可以是PCI（Peripheral Component Interconnect，外设部件互连标准）总线或EISA（Extended Industry Standard Architecture，扩展工业标准结构）总线等。通信总线4002可以分为地址总线、数据总线、控制总线等。为便于表示，图11中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
存储器4003可以是ROM（Read Only Memory，只读存储器）或可存储静态信息和指令的其他类型的静态存储设备，RAM（Random Access Memory，随机存取存储器）或者可存储信息和指令的其他类型的动态存储设备，也可以是EEPROM（Electrically Erasable Programmable Read Only Memory，电可擦可编程只读存储器）、CD-ROM（Compact Disc Read Only Memory，只读光盘）或其他光盘存储、光碟存储（包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等）、磁盘存储介质或者其他磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。
存储器4003上存储有计算机程序，处理器4001通过通信总线4002读取存储器4003中存储的计算机程序。
该计算机程序被处理器4001执行时实现上述各实施例中的管道缺陷检测方法。
此外，本申请实施例中提供了一种存储介质，该存储介质上存储有计算机程序，该计算机程序被处理器执行时实现上述各实施例中的管道缺陷检测方法。
本申请实施例中提供了一种计算机程序产品，该计算机程序产品包括计算机程序，该计算机程序存储在存储介质中。计算机设备的处理器从存储介质读取该计算机程序，处理器执行该计算机程序，使得该计算机设备执行上述各实施例中的管道缺陷检测方法。
与相关技术相比，一方面，在实现了自动对管道视频进行缺陷检测识别的同时，实现了时间和空间上的精确定位，并且通过筛选和去重去除了大量的冗余，减少了相关检测人员在核查时的工作量；另一方面，通过视频、图像以及文本的多种结果呈现方式，更加直观的反应了管道的缺陷情况。除此之外，根据对管道视频设置不同频率的抽帧方式，满足了不同设备的算力要求以及用户对性能和精确度的权衡。同时，本申请所提供的管道缺陷检测方式可适用于城市排水管道、油气运输管道、供水输水管道、电力系统管道等各式各样的管道，并且可搭载于图像采集设备、芯片、软件以及云服务等多种平台。
应该理解的是，虽然附图的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，其可以以其他的顺序执行。而且，附图的流程图中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，其执行顺序也不必然是依次进行，而是可以与其他步骤或者其他步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
以上所述仅是本申请的部分实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本申请的保护范围。

Claims (10)

  1. 一种管道缺陷检测方法，其特征在于，所述方法包括：
    获取管道视频，并对所述管道视频进行采样，得到多个待检测的图像帧；
    对多个待检测的图像帧分别进行帧级别的缺陷检测，得到各所述图像帧的检测结果；所述检测结果用于指示管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度中的至少一种；
    根据各所述图像帧的检测结果，对各所述图像帧进行筛选处理和/或去重处理，得到关键帧；
    将所述关键帧的检测结果输出。
  2. 如权利要求1所述的方法，其特征在于，所述对所述管道视频进行采样，得到多个待检测的图像帧，包括：
    响应于针对不同输入模式进行的选择操作，获取与选中的输入模式匹配的帧率；
    按照获取到的帧率对所述管道视频中的图像帧进行采样，得到多个待检测的图像帧。
  3. 如权利要求1所述的方法，其特征在于，所述对多个待检测的图像帧分别进行帧级别的缺陷检测，得到各所述图像帧的检测结果，包括：
    分别对多个待检测的图像帧进行多维度特征提取，得到各所述图像帧的多维度特征信息；
    根据各所述图像帧的多维度特征信息，分别对各所述图像帧进行管道缺陷类别的预测，得到各所述图像帧的检测结果。
  4. 如权利要求1所述的方法，其特征在于，所述帧级别的缺陷检测是基于缺陷检测模型实现的，所述缺陷检测模型是经过训练、且具有对图像帧进行帧级别的缺陷检测的能力的深度学习模型。
  5. 如权利要求1所述的方法，其特征在于，所述根据各所述图像帧的检测结果，对各所述图像帧进行筛选处理包括以下至少一种：
    针对每一个所述图像帧，基于所述图像帧的检测结果，确定各缺陷类别的置信度和各缺陷蒙版的置信度，并基于各缺陷类别的置信度和各缺陷蒙版的置信度，确定缺陷蒙版的数量、缺陷蒙版的最大置信度、缺陷类别的最大置信度、以及所有缺陷类别的总置信度；
    若所述图像帧的检测结果指示管道存在缺陷，且缺陷类别的最大置信度大于第一置信度阈值，则将所述图像帧作为关键帧；
    若所述图像帧的缺陷蒙版的数量大于设定数量，且缺陷蒙版的最大置信度大于第二置信度阈值，则将所述图像帧作为关键帧；
    若所述图像帧中所有缺陷类别的总置信度大于第三置信度阈值，则将所述图像帧作为关键帧。
  6. 如权利要求1所述的方法，其特征在于，所述根据各所述图像帧的检测结果，对各所述图像帧进行去重处理，包括：
    针对各所述图像帧进行遍历，以遍历到的所述图像帧作为当前一个所述图像帧；
    基于当前一个所述图像帧的检测结果，确定当前一个所述图像帧的缺陷类别及其置信度；
    若当前一个所述图像帧与前一个所述图像帧的间隔时间小于设定时间，且当前一个所述图像帧与前一个所述图像帧的置信度大于第四置信度阈值的缺陷类别相同，则保留所述图像帧。
  7. 如权利要求1至6任一项所述的方法，其特征在于，所述将所述关键帧的检测结果输出包括以下至少一种：
    在所述关键帧中标注对应的检测结果，并输出标注后的关键帧；
    基于所述关键帧的检测结果，在所述管道视频中对所述关键帧进行标注，并输出标注后的管道视频；
    以文本方式输出所述关键帧的检测结果。
  8. 一种管道缺陷检测装置，其特征在于，所述装置包括：
    图像获取模块，用于获取管道视频，并对所述管道视频进行采样，得到多个待检测的图像帧；
    缺陷检测模块，用于对多个待检测的图像帧分别进行帧级别的缺陷检测，得到各所述图像帧的检测结果；所述检测结果用于指示管道有无缺陷、图像拍摄场景是否在管道内、缺陷类别及其置信度、缺陷类别的缺陷等级、缺陷蒙版及其置信度中的至少一种；
    筛选去重模块，用于根据各所述图像帧的检测结果，对各所述图像帧进行筛选处理和/或去重处理，得到关键帧；
    结果输出模块，用于将所述关键帧的检测结果输出。
  9. 一种电子设备，其特征在于，包括：至少一个处理器、至少一个存储器、以及至少一条通信总线，其中，
    所述存储器上存储有计算机程序，所述处理器通过所述通信总线读取所述存储器中的所述计算机程序；
    所述计算机程序被所述处理器执行时实现权利要求1至7中任一项所述的管道缺陷检测方法。
  10. 一种存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7中任一项所述的管道缺陷检测方法。
PCT/CN2023/137184 2023-01-09 2023-12-07 管道缺陷检测方法、装置、电子设备及存储介质 Ceased WO2024148993A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310025186.5A CN116416208A (zh) 2023-01-09 2023-01-09 管道缺陷检测方法、装置、电子设备及存储介质
CN202310025186.5 2023-01-09

Publications (1)

Publication Number Publication Date
WO2024148993A1 true WO2024148993A1 (zh) 2024-07-18

Family

ID=87058777

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/137184 Ceased WO2024148993A1 (zh) 2023-01-09 2023-12-07 管道缺陷检测方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN116416208A (zh)
WO (1) WO2024148993A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118587514A (zh) * 2024-08-06 2024-09-03 浙江管卫建设有限公司 一种管道cctv检测信息管理系统
CN120070430A (zh) * 2025-04-27 2025-05-30 中国特种设备检测研究院 一种融合标注与识别的管道缺陷检测分析方法及系统
CN120142324A (zh) * 2025-05-16 2025-06-13 西安中财型材有限责任公司 管材在线缺陷检测方法及系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116416208A (zh) * 2023-01-09 2023-07-11 中国科学院深圳先进技术研究院 管道缺陷检测方法、装置、电子设备及存储介质
CN120598889B (zh) * 2025-05-27 2026-01-23 北京首创生态环保集团股份有限公司 一种用于排水管道检测视频的缺陷智能识别去重方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113221710A (zh) * 2021-04-30 2021-08-06 深圳市水务工程检测有限公司 基于神经网络的排水管道缺陷识别方法、装置、设备及介质
US20220036541A1 (en) * 2020-07-29 2022-02-03 Tata Consultancy Services Limited Identification of defect types in liquid pipelines for classification and computing severity thereof
CN114120209A (zh) * 2022-01-27 2022-03-01 深圳市博铭维技术股份有限公司 管道缺陷检测方法、系统、设备及存储介质
CN114140625A (zh) * 2022-01-30 2022-03-04 深圳市博铭维技术股份有限公司 管道缺陷检测方法、系统、设备与计算机可读存储介质
CN115015264A (zh) * 2022-05-05 2022-09-06 阿里巴巴达摩院(杭州)科技有限公司 视频处理方法、管道缺陷信息的显示方法及对应装置
WO2022204788A1 (en) * 2021-03-29 2022-10-06 Musashi Auto Parts Canada Inc. System, method, and computer device for automated visual inspection using adaptive region of interest segmentation
CN116416208A (zh) * 2023-01-09 2023-07-11 中国科学院深圳先进技术研究院 管道缺陷检测方法、装置、电子设备及存储介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111259790B (zh) * 2020-01-15 2023-06-20 上海交通大学 用于中短时视频的从粗到细的行为快速检测与分类方法及系统
CN112669269A (zh) * 2020-12-21 2021-04-16 广东爱科环境科技有限公司 一种基于图像识别的管道缺陷分类分级方法及系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220036541A1 (en) * 2020-07-29 2022-02-03 Tata Consultancy Services Limited Identification of defect types in liquid pipelines for classification and computing severity thereof
WO2022204788A1 (en) * 2021-03-29 2022-10-06 Musashi Auto Parts Canada Inc. System, method, and computer device for automated visual inspection using adaptive region of interest segmentation
CN113221710A (zh) * 2021-04-30 2021-08-06 深圳市水务工程检测有限公司 基于神经网络的排水管道缺陷识别方法、装置、设备及介质
CN114120209A (zh) * 2022-01-27 2022-03-01 深圳市博铭维技术股份有限公司 管道缺陷检测方法、系统、设备及存储介质
CN114140625A (zh) * 2022-01-30 2022-03-04 深圳市博铭维技术股份有限公司 管道缺陷检测方法、系统、设备与计算机可读存储介质
CN115015264A (zh) * 2022-05-05 2022-09-06 阿里巴巴达摩院(杭州)科技有限公司 视频处理方法、管道缺陷信息的显示方法及对应装置
CN116416208A (zh) * 2023-01-09 2023-07-11 中国科学院深圳先进技术研究院 管道缺陷检测方法、装置、电子设备及存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118587514A (zh) * 2024-08-06 2024-09-03 浙江管卫建设有限公司 一种管道cctv检测信息管理系统
CN120070430A (zh) * 2025-04-27 2025-05-30 中国特种设备检测研究院 一种融合标注与识别的管道缺陷检测分析方法及系统
CN120142324A (zh) * 2025-05-16 2025-06-13 西安中财型材有限责任公司 管材在线缺陷检测方法及系统

Also Published As

Publication number Publication date
CN116416208A (zh) 2023-07-11

Similar Documents

Publication Publication Date Title
WO2024148993A1 (zh) 管道缺陷检测方法、装置、电子设备及存储介质
CN111311475B (zh) 检测模型训练方法、装置、存储介质和计算机设备
CN108830837A (zh) 一种用于检测钢包溶蚀缺陷的方法和装置
WO2024114452A1 (zh) 一种气体泄漏检测方法、装置、电子设备及存储介质
CN109947079A (zh) 基于边缘计算的区域异常检测方法和边缘计算设备
WO2022142685A1 (zh) 传染病的传染概率预测方法及装置、存储介质、电子设备
CN114078127B (zh) 物件的缺陷检测及计数方法、装置、设备和存储介质
CN115049493B (zh) 一种区块链数据追踪方法、装置及电子设备
US8705800B2 (en) Profiling activity through video surveillance
CN110737726A (zh) 一种确定待测试接口的测试数据的方法和装置
CN110347605B (zh) 防止软件退化的方法、装置、设备及存储介质
CN110647447A (zh) 用于分布式系统的异常实例检测方法、装置、设备和介质
CN111083469A (zh) 一种视频质量确定方法、装置、电子设备及可读存储介质
CN110515758B (zh) 一种故障定位方法、装置、计算机设备及存储介质
CN113111825B (zh) 施工监控方法和装置、电子设备和存储介质
Bhargava et al. A study on potential of big visual data analytics in construction Arena
CN113158842A (zh) 一种识别方法、系统、设备及介质
CN109256341B (zh) 用于检测单晶硅太阳能电池的方法和装置
CN113140109B (zh) 一种路测数据处理方法、装置、计算机设备及存储介质
CN111343416A (zh) 一种分布式图像分析方法、系统及存储介质
CN114742992A (zh) 视频异常检测方法、装置和电子设备
Calcagni et al. Concrete defect identification and measurement in buildings
Zhang et al. Challenges of automating interior construction progress monitoring
CN112153341B (zh) 一种任务监督方法、装置、系统、电子设备及存储介质
CN114425524B (zh) 用于检测设备的收料控制方法、装置及检测设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23915751

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23915751

Country of ref document: EP

Kind code of ref document: A1