CN117593300B - PE pipe crack defect detection method and system - Google Patents

PE pipe crack defect detection method and system Download PDF

Info

Publication number
CN117593300B
CN117593300B CN202410072606.XA CN202410072606A CN117593300B CN 117593300 B CN117593300 B CN 117593300B CN 202410072606 A CN202410072606 A CN 202410072606A CN 117593300 B CN117593300 B CN 117593300B
Authority
CN
China
Prior art keywords
edge
abnormal
linear
annular
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410072606.XA
Other languages
Chinese (zh)
Other versions
CN117593300A (en
Inventor
艾心乐
万星星
谭铁军
熊海龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangxi Hanyong New Material Co ltd
Original Assignee
Jiangxi Hanyong New Material Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangxi Hanyong New Material Co ltd filed Critical Jiangxi Hanyong New Material Co ltd
Priority to CN202410072606.XA priority Critical patent/CN117593300B/en
Publication of CN117593300A publication Critical patent/CN117593300A/en
Application granted granted Critical
Publication of CN117593300B publication Critical patent/CN117593300B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a PE pipe crack defect detection method and system, and relates to the technical field of defect detection; the PE pipe crack defect detection method comprises the following steps: extracting edge features of a target image to obtain a plurality of edge feature sets, carrying out Hough transform analysis on each edge feature set, and generating an annular edge feature set and a linear edge feature set; constructing annular edge combinations and linear edge combinations, and determining a first abnormal region and a first associated feature vector of each annular edge combination; determining a second associated feature vector of a second anomaly region for each linear edge combination; and determining a plurality of first target areas and a plurality of second target areas in the target image, and inputting the plurality of first target areas and the plurality of second target areas into the trained crack detection model for defect detection. The invention reduces the training difficulty of the crack detection model and improves the crack defect detection efficiency.

Description

PE pipe crack defect detection method and system
Technical Field
The invention relates to the technical field of defect detection, in particular to a PE pipe crack defect detection method and system.
Background
Polyethylene (PE) pipe is a tubular product made of polyethylene, has the characteristics of light weight, corrosion resistance, easy installation and the like, and is widely applied to various fields including urban water supply and drainage, gas transportation, agricultural irrigation and the like. The complicated pipeline route causes the PE tubular product to have the potential safety hazard in the use, and factors such as tubular product ageing, mechanical stress, installation mistake lead to the PE tubular product to appear crack defect in the use easily, bring relevant incident, have important meaning to the periodic inspection maintenance of pipeline.
The method for acquiring the video image in the pipeline through the overhaul robot is one of the common pipeline overhaul means at present, and the analysis processing mode of the video image comprises modes based on manual analysis, neural network analysis and the like. Because of the complexity of the pipeline environment, the acquired image comprises various interference information and noise data, and an accurate crack defect detection method and system are needed to realize efficient detection of the PE pipe crack defects.
Disclosure of Invention
In order to solve the technical problems, the embodiment of the invention provides a PE pipe crack defect detection method and system.
As an aspect of the embodiment of the present invention, there is provided a method for detecting a crack defect of a PE pipe, including: acquiring a detection video in the PE pipe, extracting an image frame of the detection video, and generating an image set to be detected;
Traversing an image set to be detected, marking an ith image in the image set to be detected as a target image, extracting edge characteristics of the target image to obtain a plurality of edge characteristic sets, wherein each edge characteristic set comprises gray values and coordinates of all pixel points of a single edge characteristic; carrying out Hough transformation analysis on each edge feature set to generate an annular edge feature set and a linear edge feature set;
Mapping annular edges in the annular edge feature set into an image space, carrying out relevance analysis on any two annular edges based on a preset rectangular frame to obtain a plurality of annular edge combinations, extracting a first relevance feature vector of each annular edge combination, and determining a first abnormal region of each annular edge combination;
Mapping linear edges in the linear edge feature set into an image space, carrying out relevance analysis on any two linear edges based on a preset rectangular frame to obtain a plurality of linear edge combinations, extracting a second relevance feature vector of each linear edge combination, and determining a second abnormal region of each linear edge combination;
the gray scale difference factor of each first abnormal region is calculated, a first abnormal value of each first abnormal region is determined according to the gray scale difference factor and the first association feature vector, and a plurality of first target regions in the target image are determined based on a first abnormal threshold;
Calculating a gray scale difference factor of each second abnormal region, determining a second abnormal value of each second abnormal region according to the gray scale difference factor and the second associated feature vector, and determining a plurality of second target regions in the target image based on a second abnormal threshold;
and inputting the plurality of first target areas and the plurality of second target areas into a trained crack detection model for defect detection.
Further, performing hough transform analysis on each edge feature set to generate a ring-shaped edge feature set and a linear edge feature set, including:
Respectively carrying out straight line detection and circle detection on each edge feature set based on a Hough transform algorithm, and determining a target straight line and a target circle of each edge feature set;
for any edge feature set, if the number of pixels forming the target straight line is greater than the number of pixels forming the target circle, adding the edge feature set to the linear edge feature set, otherwise adding the edge feature set to the annular edge feature set.
Further, for the annular edge combination and the linear edge combination, further comprising:
For any two annular edges, in the process of traversing all pixel points of one annular edge through preset rectangular frames, if pixel points of the other annular edge exist in each preset rectangular frame at the same time, associating the two annular edges to obtain an annular edge combination;
and for any two linear edges, in the process of traversing all pixel points of one linear edge through the preset rectangular frames, if the pixel points of the other linear edge exist in each preset rectangular frame at the same time, correlating the two linear edges to obtain a linear edge combination.
Further, for the first associated feature vector and the second associated feature vector, further comprising:
Recording the annular edge of any annular edge combination as a first annular edge and a second annular edge, determining the distance between each straight line and the intersection point of the first annular edge and the second annular edge in a plurality of straight lines formed by the circle center of the first annular edge and each pixel point of the first annular edge, and generating a first distance vector of the first annular edge; determining the distance between the first annular edge and the pixel point of the second annular edge in a plurality of straight lines formed by the circle center of the second annular edge and each pixel point of the second annular edge, generating a first distance vector of the second annular edge, respectively calculating the variance value of each first feature vector, and taking the first feature vector with larger variance value as a first associated feature vector of the annular edge combination;
Marking the linear edge of any one linear edge combination as a first linear edge and a second linear edge, and determining a distance dividing line of the first linear edge and the second linear edge;
Determining the vertical line of each pixel point on the first linear edge and the distance dividing line, calculating the distance between each vertical line and the intersection point of the first linear edge and the second linear edge, generating a second distance vector of the first linear edge, determining the vertical line of each pixel point on the second linear edge and the distance dividing line, calculating the distance between each vertical line and the intersection point of the first linear edge and the second linear edge, generating a second distance vector of the second linear edge, respectively extracting a gradient sequence of each second distance vector, including the difference value of every two adjacent elements, performing discrete analysis on the gradient sequence of each second distance vector, determining the discrete value of each second distance vector, and taking the second distance vector with a large discrete value as a second association feature vector of the linear edge combination.
Further, for the first outlier and the second outlier, further comprising:
For any one abnormal region, determining a first reference gray value of the abnormal region, traversing all pixel points in the abnormal region, determining a second reference gray value in 8 adjacent regions of each pixel point, wherein if all pixel points in the 8 adjacent regions of the pixel points are the pixel points of the abnormal region, the second reference gray value is the first reference gray value, otherwise, the second reference gray value is the minimum gray value of all pixel points in the 8 adjacent regions, calculating the difference value between the second reference gray value and the first reference gray value of each pixel point, and determining a gray difference factor of the abnormal region according to a plurality of difference values;
the calculating of the first outlier region includes:
In the method, in the process of the invention, Is the first outlier of the first outlier region,/>For the variance value of the first associated feature vector,/>Is the gray scale difference factor of the first abnormal region,/>、/>Variance values/>, respectively, of the first associated feature vectorAnd a gray scale difference factor/>, of the first abnormal regionIs used for normalizing parameters of the (a);
The calculating of the second outlier region includes:
In the method, in the process of the invention, Is the second outlier of the second outlier region,/>Is the discrete value of the gradual sequence of the second associated feature vector,/>Is the gray level difference factor of the second associated feature vector,/>、/>Discrete values/>, respectively, of the fade sequence of the second associated feature vectorAnd a gray scale difference factor/>, of the second associated feature vectorIs used for the normalization parameters of (a).
Further, determining a plurality of first target areas in the target image based on the first anomaly threshold value, and determining a plurality of second target areas in the target image based on the second anomaly threshold value, includes:
For a first abnormal region with a first abnormal value larger than a first abnormal threshold value and a second abnormal region with a second abnormal value larger than a second abnormal threshold value, respectively determining the positions of the first abnormal region and the second abnormal region in the target image according to the position information of each abnormal region, and respectively selecting a first target region and a second target region corresponding to the first abnormal region and the second abnormal region from the target image based on the circumscribed rectangle of the abnormal region.
Further, training of the crack detection model includes:
And acquiring a plurality of PE pipe defect images with crack defects, determining a plurality of defect images with crack defect textures only based on a manual screening mode, constructing a training data set, and training a crack detection model, wherein the crack detection model is a convolutional neural network model.
As another aspect of the embodiments of the present invention, there is further provided a system for detecting a crack defect of a PE pipe, where the defect detecting system executes the method for detecting a crack defect of a PE pipe, including:
the image acquisition module is used for acquiring a detection video in the PE pipe, extracting an image frame of the detection video and generating an image set to be detected;
The edge extraction module is used for extracting edge features of the target image in the image set to be detected to obtain a plurality of edge feature sets;
The edge analysis module is used for carrying out Hough transformation analysis on each edge feature set to generate an annular edge feature set and a linear edge feature set;
the first area analysis module is used for mapping annular edges in the annular edge feature set into an image space, carrying out relevance analysis on any two annular edges based on a preset rectangular frame to obtain a plurality of annular edge combinations, extracting a first relevance feature vector of each annular edge combination, and determining a first abnormal area of each annular edge combination;
The second region analysis module is used for mapping the linear edges in the linear edge feature set into an image space, carrying out relevance analysis on any two linear edges based on a preset rectangular frame to obtain a plurality of linear edge combinations, extracting a second relevance feature vector of each linear edge combination, and determining a second abnormal region of each linear edge combination;
The first region extraction module is used for calculating a gray scale difference factor of each first abnormal region, determining a first abnormal value of each first abnormal region according to the gray scale difference factor and the first association feature vector, and determining a plurality of first target regions in the target image based on a first abnormal threshold;
the second region extraction module is used for calculating a gray scale difference factor of each second abnormal region, determining a second abnormal value of each second abnormal region according to the gray scale difference factor and the second associated feature vector, and determining a plurality of second target regions in the target image based on a second abnormal threshold;
And the crack defect detection module is used for inputting the plurality of first target areas and the plurality of second target areas into the trained crack detection model to detect defects.
Further, for the edge analysis module, performing hough transform analysis on each edge feature set to generate a ring edge feature set and a linear edge feature set, including:
Respectively carrying out straight line detection and circle detection on each edge feature set based on a Hough transform algorithm, and determining a target straight line and a target circle of each edge feature set;
for any edge feature set, if the number of pixels forming the target straight line is greater than the number of pixels forming the target circle, adding the edge feature set to the linear edge feature set, otherwise adding the edge feature set to the annular edge feature set.
Further, the method further comprises the following steps:
the model training module is used for carrying out model training on the crack detection model through the training data set;
and acquiring a plurality of PE pipe defect images with crack defects for the training data set, determining a plurality of defect images with crack defect textures only based on a manual screening mode, and constructing the training data set for model training of the crack detection model.
The invention has the following advantages:
According to the invention, the similarity of a plurality of texture areas is determined by carrying out morphological analysis and clustering treatment on the edge characteristics of the PE pipe internal image, the interference information in the PE pipe internal image is abandoned, the image area with high crack probability is extracted for crack defect detection, the detection efficiency is improved, and the training difficulty of a crack detection model is reduced.
Drawings
Fig. 1 is a flowchart of a method for detecting a crack defect of a PE pipe according to an embodiment of the present invention.
Fig. 2 is a block diagram of a system for detecting a crack defect of a PE pipe according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, some embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.
As shown in fig. 1, an embodiment of the present invention provides a method for detecting a crack defect of a PE pipe, which specifically includes:
S1, acquiring a detection video in the PE pipe, extracting an image frame of the detection video, and generating an image set to be detected.
Traditional pipeline detection such as manual excavation's detection mode detection cycle is long, and inefficiency, along with intelligent control technology's development, pipeline detection robot gradually becomes the instrument of detecting commonly used in each field, through the built-in video image acquisition system of robot and the camera equipment who carries on, the inside video image data of pipeline of collection that the robot can be efficient supplies relevant technical personnel to carry out subsequent analysis and uses.
In this embodiment, for the collected detection video inside the PE pipe, a corresponding image set to be detected is constructed by extracting image frames in the detection video, and considering the local content repeatability of each frame of image in the detection video, partial images may be discarded at intervals in the process of extracting multi-frame images, and a data set to be detected is constructed according to a series of images with frame intervals, and a specific image selection method may be determined in combination with movement data of a robot, a pipeline size, and the like, which is not specifically limited in this embodiment.
S2, traversing an image set to be detected, marking an ith image in the image set to be detected as a target image, and extracting edge features of the target image to obtain a plurality of edge feature sets; and carrying out Hough transformation analysis on each edge feature set to generate an annular edge feature set and a linear edge feature set.
Taking an ith image in an image set to be detected as an example to carry out the explanation of an image processing step, recording the ith image as a target image, extracting a plurality of edge features of the target image through an edge extraction algorithm, constructing an edge feature set of each edge feature according to gray values and coordinate information of all pixel points of each edge feature, specifically, converting the target image into a gray image, and extracting the edge features in the gray image through a Canny operator, a Sobel operator and the like, wherein the edge detection operator is a technical means well known to a person skilled in the art, and is not repeated here.
In this embodiment, in consideration of environmental noise and data transmission noise in the video image acquisition process, a person skilled in the art may perform denoising processing on the target image before extracting edge features of the target image, for example, performing denoising processing on the image by using a median filter, a gaussian filter, or the like. After extracting a plurality of edge features of the target image, each edge feature is analyzed by hough transform.
Specifically, based on a Hough transform algorithm, line detection and circle detection are respectively carried out on each edge feature set, and a target line and a target circle of each edge feature set are determined.
Among a plurality of straight lines and a plurality of circles formed by a plurality of pixel points in the edge feature set, the straight line and the circle with the largest number of the pixel points are respectively marked as a target straight line and a target circle.
In this embodiment, in order to improve the detection accuracy of the line detection and the circle detection, the number of the pixel points involved in the target line and the target circle may be specifically limited, and, for example, the ratio of the number of the pixel points involved in the target line and the target circle to the total number of all the pixel points in the corresponding feature set is greater than a preset ratio, for example, 40%.
For any edge feature set, after determining the target straight line and the target circle of the edge feature set, if the number of pixels forming the target straight line is greater than the number of pixels forming the target circle, adding the edge feature set to the linear edge feature set, otherwise adding the edge feature set to the annular edge feature set. And analyzing each edge feature set in the mode to generate an annular edge feature set and a linear edge feature set.
S3, mapping annular edges in the annular edge feature set into an image space, carrying out relevance analysis on any two annular edges based on a preset rectangular frame to obtain a plurality of annular edge combinations, extracting a first relevance feature vector of each annular edge combination, and determining a first abnormal region of each annular edge combination.
In this embodiment, the meaning of performing relevance analysis on the annular edges is that the region with the annular lines is located, so as to obtain a first abnormal region corresponding to the position of each annular line, and the change of the distance between the two annular edges with relevance is analyzed, so as to extract a first relevance feature vector representing the distance relationship between the two annular edges in the annular edge combination.
In this embodiment, performing the relevance analysis on the annular edges specifically includes taking any two annular edges as an example, in the process of traversing all the pixels of one annular edge through the preset rectangular frames, if the pixels of the other annular edge exist in each preset rectangular frame involved in the traversing process at the same time, correlating the two annular edges to obtain an annular edge combination, where the size of the preset rectangular frame can be comprehensively set in combination with the resolution of the image, for example, the size isAnd the like, the size of the preset rectangular frame is not particularly limited in this embodiment.
After determining that the two associated annular edges exist, constructing a corresponding annular edge combination, and marking an area formed by the two annular edges as a first abnormal area corresponding to the annular edge combination.
And S4, mapping the linear edges in the linear edge feature set into an image space, carrying out correlation analysis on any two linear edges based on a preset rectangular frame to obtain a plurality of linear edge combinations, extracting a second correlation feature vector of each linear edge combination, and determining a second abnormal region of each linear edge combination.
In this embodiment, performing association analysis on any two linear edges specifically includes taking any two linear edges as an example, in a process of traversing all pixel points of one linear edge through preset rectangular frames, if pixel points of another linear edge exist in each preset rectangular frame at the same time, associating the two linear edges to obtain a linear edge combination, marking an area formed by the two linear edges of the linear edge combination as a second abnormal area of the linear edge combination, and extracting a second association feature vector representing a distance relationship between the two linear edges by analyzing coordinate information of each pixel point of the two linear edges of the linear edge combination.
S5, calculating gray scale difference factors of each first abnormal region, determining first abnormal values of each first abnormal region according to the gray scale difference factors and the first associated feature vectors, and determining a plurality of first target regions in the target image based on the first abnormal threshold.
In this embodiment, the gray scale difference factor is used to quantify the gray scale difference characterizing the abnormal region and the surrounding background region, and is determined by analyzing the gray scale value difference between the plurality of pixel points.
In this embodiment, for any one of the abnormal regions, a first reference gray value of the abnormal region is determined, where the first reference gray value is used to represent a gray value average value of all pixel points in the abnormal region.
It is worth to say that the abnormal region represents linear lines and annular lines, the cracks of the PE pipe are mostly annular cracks and longitudinal cracks, wherein the longitudinal cracks are along the extending direction of the pipe, in the process of detecting the internal cracks of the pipe, the splicing region, the annular crack region, the pipe fluid dipping region, the longitudinal crack region, the scratch region and the like among the pipes, and the light lines (the smoother the inside of the pipe is, the more obvious) which are divergent from the center of the pipe in the extending direction of the pipe form relevant annular lines and longitudinal lines, the positioning of the regions is realized after the correlation of the edge characteristics is analyzed, so that a plurality of abnormal regions are obtained, and the abnormal regions are distinguished through further analysis of gray values. For the above-mentioned regions, the gray values of each region are different, for example, the light line region is light color, the crack region and the welding and splicing region are dark color, the gray values of each pixel point in each region are relatively uniform. Therefore, for the first reference gray level, the gray level (for example, the number of the pixel points is the largest) most representative in the gray level histogram of the abnormal region can be selected as the first reference gray level, and the average value of the gray levels of all the pixel points in the abnormal region can be selected for characterization, so that the difference of different values is not large, and the embodiment is not limited specifically.
After the first reference gray value of the abnormal area is determined, traversing all pixel points in the abnormal area, and determining the second reference gray value in 8 adjacent areas of each pixel point.
Specifically, taking any pixel as an example, if all the pixels in the 8-neighbor region of the pixel are pixels in the abnormal region, the pixel is indicated to be located in the abnormal region, in this case, the second reference gray value of the pixel is the first reference gray value, if all the pixels in the 8-neighbor region of the pixel have pixels in the non-abnormal region, the pixel is indicated to be located at the edge of the abnormal region, and in this case, the gray value minimum value of all the pixels in the 8-neighbor region of the pixel is taken as the second reference gray value of the pixel.
After determining a first reference gray level of the abnormal region and a second reference gray level of each pixel point in the abnormal region, calculating a difference value between the second reference gray level of each pixel point and the first reference gray level, and determining a gray level difference factor of the abnormal region according to the plurality of difference values, wherein an average value is calculated after taking absolute values of the plurality of difference values as the gray level difference factor of the abnormal region.
The calculating of the first outlier region specifically includes:
In the method, in the process of the invention, Is the first outlier of the first outlier region,/>For the variance value of the first associated feature vector,/>Is the gray scale difference factor of the first abnormal region,/>、/>Variance values/>, respectively, of the first associated feature vectorAnd a gray scale difference factor/>, of the first abnormal regionIs used for the normalization parameters of (a).
And further analyzing each first abnormal region through a first abnormal threshold value and a first abnormal value, wherein the larger the first abnormal value is, the greater the possibility that the first abnormal region is a crack region is indicated, so that a plurality of first target regions suspected to have annular cracks are determined.
S6, calculating a gray scale difference factor of each second abnormal region, determining a second abnormal value of each second abnormal region according to the gray scale difference factor and the second associated feature vector, and determining a plurality of second target regions in the target image based on a second abnormal threshold.
In the present embodiment, the calculation of the second abnormal value of the second abnormal region includes:
In the method, in the process of the invention, Is the second outlier of the second outlier region,/>Is the discrete value of the gradual sequence of the second associated feature vector,/>Is the gray level difference factor of the second associated feature vector,/>、/>Discrete values/>, respectively, of the fade sequence of the second associated feature vectorAnd a gray scale difference factor/>, of the second associated feature vectorWherein the gradient sequence of the second associated feature vector is specifically an array formed by the differences of every two adjacent elements, and the discrete value of the gradient sequence is specifically the variance value of the gradient sequence.
For any one of the second abnormal regions, the greater the second abnormal value is, the greater the possibility that the longitudinal crack exists in the second abnormal region is, and the second target regions suspected of having the linear crack are determined through the second abnormal threshold value.
S7, inputting the plurality of first target areas and the plurality of second target areas into the trained crack detection model to detect defects.
In this embodiment, a convolutional neural network is taken as an example, a crack detection model is constructed based on a convolutional neural network architecture, and model training is performed through a training data set, wherein the training data set comprises a plurality of PE pipe defect images with crack defect marks, a plurality of defect images in the training data set are taken as inputs, and crack defect position information of each image is taken as a training target, so that the crack detection model for performing crack defect detection is obtained through training.
The method and the device have the advantages that the edge analysis processing is carried out on the target image in a preprocessing mode, the image area with the high probability of crack defects in the target image is extracted and then is sent into the model for detection, and the processing efficiency of the model on the image is improved.
On the basis, because the texture of the target image is subjected to related analysis in advance, in the process of training a model, after a plurality of PE pipe defect images with crack defects are obtained, a plurality of defect images with crack defect textures can be determined based on a manual screening mode to reconstruct the training data set, specifically, the images with interference information such as pipeline splicing textures, scratch textures, light textures, dipping textures and the like are abandoned or modified in a manual analysis mode, the areas with the interference information in the images are cut and removed, the images with the crack defect textures are subjected to defect labeling, and then the training data set is constructed, so that the training speed of the model is higher under the condition that the interference information is less, the difference between various interference textures and crack defects is not needed to be specifically analyzed and identified, and the efficiency of crack defect detection is improved on the basis of reducing the training difficulty of the model.
Further, for the first associated feature vector and the second associated feature vector in step S3 and step S4, the following manner may be adopted for extraction:
For the first associated feature vector, an arbitrary one of the annular edge combinations is taken as an example, and for convenience of description, two annular edges in the annular edge combination are respectively denoted as a first annular edge and a second annular edge.
Determining the distance between the circle center of the first annular edge and the intersection point of each straight line and the first annular edge and the second annular edge in a plurality of straight lines formed by each pixel point of the first annular edge, and generating a first distance vector of the first annular edge;
And determining the distance between the circle center of the second annular edge and each pixel point of the second annular edge in a plurality of straight lines formed by the circle center of the second annular edge and each pixel point of the second annular edge, and generating a first distance vector of the second annular edge.
In this embodiment, the center of the circle of the first annular edge and the center of the circle of the second annular edge may be determined by hough transform, and considering that there is a certain error in the positions between the plurality of pixels extracted to obtain the annular edge, so that the distance vector of each annular edge is extracted by taking the first annular edge and the second annular edge as references, the variance value of each first feature vector is calculated, the first feature vector with larger variance value is calculated as the first associated feature vector of the annular edge combination, the first associated feature vector is used to characterize the distance change under different positions of the two annular edges, and for the difference between the annular crack defect and the annular interference line such as the pipeline welding line in the image, the distance change between the different positions of the pipeline welding line is relatively regular, that is, the degree of dispersion of the feature vector is small, the crack defect is generally the distance between the two ends with large middle distance, the degree of dispersion of the feature vector is large, and the type of the annular edge can be more specifically analyzed and identified by analyzing the first associated feature vector of each annular edge combination.
For the second associated feature vector, an arbitrary one of the linear edge combinations is taken as an example, and two linear edges of the linear edge combination are respectively denoted as a first linear edge and a second linear edge, and a distance dividing line of the first linear edge and the second linear edge is determined.
The distance dividing line is specifically a position reference line of two linear edges, specifically, if the lines corresponding to the two linear edges are parallel, the distance dividing line is parallel to the two linear edges and the distances are equal, if the lines corresponding to the two linear edges are intersected, an angular bisector of an included angle between the two linear edges is taken as the distance dividing line, and the function of the distance dividing line is to assist in analyzing the relative distance change of the two linear edges.
Determining the vertical line of each pixel point on the first linear edge and the distance dividing line, calculating the distance between each vertical line and the intersection point of the first linear edge and the second linear edge, and generating a second distance vector of the first linear edge;
And determining the vertical line between each pixel point on the second linear edge and the distance dividing line, calculating the distance between each vertical line and the intersection point of the first linear edge and the second linear edge, and generating a second distance vector of the second linear edge.
In this embodiment, regarding the near-far change of the shooting view angle, for the light texture, the dipping texture, and the like that may exist in the target image, the relative distance between two linear edges that form the texture as the view angle changes from near to far gradually becomes smaller, that is, the distal ends of two straight lines parallel to each other gradually approach each other in the view field, so that the second associated feature vector of the linear edge combination is extracted by means of the distance dividing line, specifically, for the two second distance vectors, the gradient sequence of each second distance vector is extracted separately, discrete analysis is performed on the gradient sequence of each second distance vector, the discrete value of each second distance vector is determined, specifically, the variance value of the gradient sequence is calculated as the discrete value, the second distance vector with a large discrete value is taken as the second associated feature vector of the linear edge combination, and specific analysis and identification can be performed by analyzing the type of the second associated feature vector, where the smaller the discrete degree of the gradient sequence of the second associated feature vector indicates that the second abnormal region formed by the linear edge combination is a larger possibility that the region corresponding to the light texture, the dipping texture, and the like is a larger possibility that the region is a crack is a larger vice versa.
Further, for the extraction of the first target area and the second target area, specifically, for the first abnormal area with the first abnormal value greater than the first abnormal threshold value and the second abnormal area with the second abnormal value greater than the second abnormal threshold value, the positions of the first abnormal area and the second abnormal area in the target image are respectively determined according to the position information of each abnormal area, and the first target area and the second target area corresponding to the first abnormal area and the second abnormal area are respectively selected from the target image based on the circumscribed rectangle of the abnormal area.
In this embodiment, the first abnormal region and the second abnormal region are arc-shaped or linear, so that in order to improve the comparison between the abnormal region and the background region, a plurality of first target regions and second target regions can be obtained by dividing the first abnormal region and the second abnormal region in a manner of determining an circumscribed rectangle, and therefore crack detection is performed on the first target regions and the second target regions through a crack detection model, and detection accuracy is improved.
According to the PE pipe crack defect detection method provided by the embodiment of the invention, the similarity of a plurality of texture areas is determined by carrying out morphological analysis and clustering treatment on the edge characteristics of the PE pipe internal image, interference information in the PE pipe internal image is abandoned, and the image area with high crack existence probability is extracted for crack defect detection, so that the detection efficiency is improved, and the training difficulty of a crack detection model is reduced.
As shown in fig. 2, on the basis of the method for detecting a crack defect of a PE pipe provided above, an embodiment of the present invention further provides a system for detecting a crack defect of a PE pipe, including:
the image acquisition module is used for acquiring a detection video in the PE pipe, extracting an image frame of the detection video and generating an image set to be detected;
the edge extraction module is used for extracting edge features of a target image in the image set to be detected to obtain a plurality of edge feature sets, wherein each edge feature set comprises gray values and coordinates of all pixel points of a single edge feature;
The edge analysis module is used for carrying out Hough transformation analysis on each edge feature set to generate an annular edge feature set and a linear edge feature set;
the first area analysis module is used for mapping annular edges in the annular edge feature set into an image space, carrying out relevance analysis on any two annular edges based on a preset rectangular frame to obtain a plurality of annular edge combinations, extracting a first relevance feature vector of each annular edge combination, and determining a first abnormal area of each annular edge combination;
The second region analysis module is used for mapping the linear edges in the linear edge feature set into an image space, carrying out relevance analysis on any two linear edges based on a preset rectangular frame to obtain a plurality of linear edge combinations, extracting a second relevance feature vector of each linear edge combination, and determining a second abnormal region of each linear edge combination;
The first region extraction module is used for calculating a gray scale difference factor of each first abnormal region, determining a first abnormal value of each first abnormal region according to the gray scale difference factor and the first association feature vector, and determining a plurality of first target regions in the target image based on a first abnormal threshold;
the second region extraction module is used for calculating a gray scale difference factor of each second abnormal region, determining a second abnormal value of each second abnormal region according to the gray scale difference factor and the second associated feature vector, and determining a plurality of second target regions in the target image based on a second abnormal threshold;
specifically, for a first abnormal region with a first abnormal value greater than a first abnormal threshold value and a second abnormal region with a second abnormal value greater than a second abnormal threshold value, positions of the first abnormal region and the second abnormal region in a target image are respectively determined according to position information of each abnormal region, and a first target region and a second target region corresponding to the first abnormal region and the second abnormal region are respectively selected from the target image based on circumscribed rectangles of the abnormal regions.
And the crack defect detection module is used for inputting the plurality of first target areas and the plurality of second target areas into the trained crack detection model to detect defects.
Further, for the edge analysis module, performing hough transform analysis on each edge feature set to generate an annular edge feature set and a linear edge feature set, which specifically includes:
Respectively carrying out straight line detection and circle detection on each edge feature set based on a Hough transform algorithm, and determining a target straight line and a target circle of each edge feature set;
for any edge feature set, if the number of pixels forming the target straight line is greater than the number of pixels forming the target circle, adding the edge feature set to the linear edge feature set, otherwise adding the edge feature set to the annular edge feature set.
Further, the system for detecting the crack defect of the PE pipe provided by the embodiment of the invention further comprises:
the model training module is used for carrying out model training on the crack detection model through the training data set;
After a plurality of PE pipe defect images with crack defects are obtained for the training data set, determining a plurality of defect images with crack defect textures only based on a manual screening mode, and constructing the training data set for model training of a crack detection model.
It will be understood that modifications and variations will be apparent to those skilled in the art from the foregoing description, and it is intended that all such modifications and variations be included within the scope of the following claims. Parts of the specification not described in detail belong to the prior art known to those skilled in the art.

Claims (9)

1. The PE pipe crack defect detection method is characterized by comprising the following steps of:
acquiring a detection video in the PE pipe, extracting an image frame of the detection video, and generating an image set to be detected;
Traversing an image set to be detected, marking an ith image in the image set to be detected as a target image, extracting edge characteristics of the target image to obtain a plurality of edge characteristic sets, wherein each edge characteristic set comprises gray values and coordinates of all pixel points of a single edge characteristic; carrying out Hough transformation analysis on each edge feature set to generate an annular edge feature set and a linear edge feature set;
Mapping annular edges in the annular edge feature set into an image space, carrying out relevance analysis on any two annular edges based on a preset rectangular frame to obtain a plurality of annular edge combinations, extracting a first relevance feature vector of each annular edge combination, and determining a first abnormal region of each annular edge combination;
Mapping linear edges in the linear edge feature set into an image space, carrying out relevance analysis on any two linear edges based on a preset rectangular frame to obtain a plurality of linear edge combinations, extracting a second relevance feature vector of each linear edge combination, and determining a second abnormal region of each linear edge combination;
The gray scale difference factor of each first abnormal region is calculated, a first abnormal value of each first abnormal region is determined according to the gray scale difference factor and the first association feature vector, and a plurality of first target regions in the target image are determined based on a first abnormal threshold; in this step, the calculation of the first outlier region specifically includes:
In the method, in the process of the invention, Is the first outlier of the first outlier region,/>For the variance value of the first associated feature vector,/>Is the gray scale difference factor of the first abnormal region,/>、/>Variance values/>, respectively, of the first associated feature vectorAnd a gray scale difference factor/>, of the first abnormal regionIs used for normalizing parameters of the (a);
Further analyzing each first abnormal region through a first abnormal threshold value and a first abnormal value, wherein the larger the first abnormal value is, the greater the possibility that the first abnormal region is a crack region is indicated, so that a plurality of first target regions suspected to have annular cracks are determined; in this step, the calculation of the second outlier region includes:
In the method, in the process of the invention, Is the second outlier of the second outlier region,/>Is the discrete value of the gradual sequence of the second associated feature vector,/>Is the gray level difference factor of the second associated feature vector,/>、/>Discrete values/>, respectively, of the fade sequence of the second associated feature vectorAnd a gray scale difference factor/>, of the second associated feature vectorWherein the gradual change sequence of the second association feature vector is specifically an array formed by the difference values of every two adjacent elements, and the discrete value of the gradual change sequence is specifically the variance value of the gradual change sequence;
For any one second abnormal region, the larger the second abnormal value is, the greater the possibility that the longitudinal crack exists in the second abnormal region is, and a plurality of second target regions suspected to exist linear cracks are determined through a second abnormal threshold value;
and inputting the plurality of first target areas and the plurality of second target areas into a trained crack detection model for defect detection.
2. The method for detecting a crack defect in a PE pipe according to claim 1, wherein performing hough transform analysis on each edge feature set to generate a ring-shaped edge feature set and a line-shaped edge feature set comprises:
Respectively carrying out straight line detection and circle detection on each edge feature set based on a Hough transform algorithm, and determining a target straight line and a target circle of each edge feature set;
for any edge feature set, if the number of pixels forming the target straight line is greater than the number of pixels forming the target circle, adding the edge feature set to the linear edge feature set, otherwise adding the edge feature set to the annular edge feature set.
3. The method for detecting a crack defect in a PE pipe according to claim 1, further comprising, for the annular edge combination and the linear edge combination:
For any two annular edges, in the process of traversing all pixel points of one annular edge through preset rectangular frames, if pixel points of the other annular edge exist in each preset rectangular frame at the same time, associating the two annular edges to obtain an annular edge combination;
and for any two linear edges, in the process of traversing all pixel points of one linear edge through the preset rectangular frames, if the pixel points of the other linear edge exist in each preset rectangular frame at the same time, correlating the two linear edges to obtain a linear edge combination.
4. The method for detecting a crack defect in a PE pipe according to claim 1, further comprising, for the first associated feature vector and the second associated feature vector:
Recording the annular edge of any annular edge combination as a first annular edge and a second annular edge, determining the distance between each straight line and the intersection point of the first annular edge and the second annular edge in a plurality of straight lines formed by the circle center of the first annular edge and each pixel point of the first annular edge, and generating a first distance vector of the first annular edge; determining the distance between the first annular edge and the pixel point of the second annular edge in a plurality of straight lines formed by the circle center of the second annular edge and each pixel point of the second annular edge, generating a first distance vector of the second annular edge, respectively calculating the variance value of each first feature vector, and taking the first feature vector with larger variance value as a first associated feature vector of the annular edge combination;
Marking the linear edge of any one linear edge combination as a first linear edge and a second linear edge, and determining a distance dividing line of the first linear edge and the second linear edge;
Determining the vertical line of each pixel point on the first linear edge and the distance dividing line, calculating the distance between each vertical line and the intersection point of the first linear edge and the second linear edge, generating a second distance vector of the first linear edge, determining the vertical line of each pixel point on the second linear edge and the distance dividing line, calculating the distance between each vertical line and the intersection point of the first linear edge and the second linear edge, generating a second distance vector of the second linear edge, respectively extracting a gradient sequence of each second distance vector, including the difference value of every two adjacent elements, performing discrete analysis on the gradient sequence of each second distance vector, determining the discrete value of each second distance vector, and taking the second distance vector with a large discrete value as a second association feature vector of the linear edge combination.
5. The PE pipe crack defect detection method of claim 1, wherein determining a plurality of first target areas in the target image based on the first anomaly threshold value, and determining a plurality of second target areas in the target image based on the second anomaly threshold value, comprises:
For a first abnormal region with a first abnormal value larger than a first abnormal threshold value and a second abnormal region with a second abnormal value larger than a second abnormal threshold value, respectively determining the positions of the first abnormal region and the second abnormal region in the target image according to the position information of each abnormal region, and respectively selecting a first target region and a second target region corresponding to the first abnormal region and the second abnormal region from the target image based on the circumscribed rectangle of the abnormal region.
6. The method for detecting a crack defect in a PE pipe according to claim 1, wherein the training of the crack detection model comprises:
And acquiring a plurality of PE pipe defect images with crack defects, determining a plurality of defect images with crack defect textures only based on a manual screening mode, constructing a training data set, and training a crack detection model, wherein the crack detection model is a convolutional neural network model.
7. A PE pipe crack defect detection system that performs the PE pipe crack defect detection method of any of the preceding claims 1-6, comprising:
the image acquisition module is used for acquiring a detection video in the PE pipe, extracting an image frame of the detection video and generating an image set to be detected;
The edge extraction module is used for extracting edge features of the target image in the image set to be detected to obtain a plurality of edge feature sets;
The edge analysis module is used for carrying out Hough transformation analysis on each edge feature set to generate an annular edge feature set and a linear edge feature set;
the first area analysis module is used for mapping annular edges in the annular edge feature set into an image space, carrying out relevance analysis on any two annular edges based on a preset rectangular frame to obtain a plurality of annular edge combinations, extracting a first relevance feature vector of each annular edge combination, and determining a first abnormal area of each annular edge combination;
The second region analysis module is used for mapping the linear edges in the linear edge feature set into an image space, carrying out relevance analysis on any two linear edges based on a preset rectangular frame to obtain a plurality of linear edge combinations, extracting a second relevance feature vector of each linear edge combination, and determining a second abnormal region of each linear edge combination;
The first region extraction module is used for calculating a gray scale difference factor of each first abnormal region, determining a first abnormal value of each first abnormal region according to the gray scale difference factor and the first association feature vector, and determining a plurality of first target regions in the target image based on a first abnormal threshold;
the second region extraction module is used for calculating a gray scale difference factor of each second abnormal region, determining a second abnormal value of each second abnormal region according to the gray scale difference factor and the second associated feature vector, and determining a plurality of second target regions in the target image based on a second abnormal threshold;
And the crack defect detection module is used for inputting the plurality of first target areas and the plurality of second target areas into the trained crack detection model to detect defects.
8. The PE pipe crack defect detection system of claim 7, wherein for the edge analysis module, performing hough transform analysis on each edge feature set to generate a ring edge feature set and a line edge feature set, comprising:
Respectively carrying out straight line detection and circle detection on each edge feature set based on a Hough transform algorithm, and determining a target straight line and a target circle of each edge feature set;
for any edge feature set, if the number of pixels forming the target straight line is greater than the number of pixels forming the target circle, adding the edge feature set to the linear edge feature set, otherwise adding the edge feature set to the annular edge feature set.
9. The PE pipe crack defect detection system of claim 8, further comprising:
the model training module is used for carrying out model training on the crack detection model through the training data set;
and acquiring a plurality of PE pipe defect images with crack defects for the training data set, determining a plurality of defect images with crack defect textures only based on a manual screening mode, and constructing the training data set for model training of the crack detection model.
CN202410072606.XA 2024-01-18 2024-01-18 PE pipe crack defect detection method and system Active CN117593300B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410072606.XA CN117593300B (en) 2024-01-18 2024-01-18 PE pipe crack defect detection method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410072606.XA CN117593300B (en) 2024-01-18 2024-01-18 PE pipe crack defect detection method and system

Publications (2)

Publication Number Publication Date
CN117593300A CN117593300A (en) 2024-02-23
CN117593300B true CN117593300B (en) 2024-04-26

Family

ID=89918680

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410072606.XA Active CN117593300B (en) 2024-01-18 2024-01-18 PE pipe crack defect detection method and system

Country Status (1)

Country Link
CN (1) CN117593300B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9092691B1 (en) * 2014-07-18 2015-07-28 Median Technologies System for computing quantitative biomarkers of texture features in tomographic images
CN108765456A (en) * 2018-04-02 2018-11-06 上海鹰觉科技有限公司 Method for tracking target, system based on linear edge feature
CN112581434A (en) * 2020-12-07 2021-03-30 无锡智创云图信息科技有限公司 Image identification method for product defect detection
CN114723681A (en) * 2022-03-22 2022-07-08 江苏禹润智能科技有限公司 Concrete crack defect detection method based on machine vision
CN116703909A (en) * 2023-08-07 2023-09-05 威海海泰电子有限公司 Intelligent detection method for production quality of power adapter
CN116934740A (en) * 2023-09-11 2023-10-24 深圳市伟利达精密塑胶模具有限公司 Plastic mold surface defect analysis and detection method based on image processing

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9092691B1 (en) * 2014-07-18 2015-07-28 Median Technologies System for computing quantitative biomarkers of texture features in tomographic images
CN108765456A (en) * 2018-04-02 2018-11-06 上海鹰觉科技有限公司 Method for tracking target, system based on linear edge feature
CN112581434A (en) * 2020-12-07 2021-03-30 无锡智创云图信息科技有限公司 Image identification method for product defect detection
CN114723681A (en) * 2022-03-22 2022-07-08 江苏禹润智能科技有限公司 Concrete crack defect detection method based on machine vision
CN116703909A (en) * 2023-08-07 2023-09-05 威海海泰电子有限公司 Intelligent detection method for production quality of power adapter
CN116934740A (en) * 2023-09-11 2023-10-24 深圳市伟利达精密塑胶模具有限公司 Plastic mold surface defect analysis and detection method based on image processing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于机器视觉的液晶屏字符缺陷检测装置研究;许炜东;《中国优秀硕士学位论文全文数据库》;20160615;全文 *

Also Published As

Publication number Publication date
CN117593300A (en) 2024-02-23

Similar Documents

Publication Publication Date Title
CN113469177B (en) Deep learning-based drainage pipeline defect detection method and system
US11221107B2 (en) Method for leakage detection of underground pipeline corridor based on dynamic infrared thermal image processing
CN109255317B (en) Aerial image difference detection method based on double networks
CN103400151B (en) The optical remote sensing image of integration and GIS autoregistration and Clean water withdraw method
CN113592828B (en) Nondestructive testing method and system based on industrial endoscope
CN111539330B (en) Transformer substation digital display instrument identification method based on double-SVM multi-classifier
CN114549981A (en) Intelligent inspection pointer type instrument recognition and reading method based on deep learning
CN115841434B (en) Infrared image enhancement method for gas concentration analysis
CN113393426B (en) Steel rolling plate surface defect detection method
CN112198170B (en) Detection method for identifying water drops in three-dimensional detection of outer surface of seamless steel tube
CN111667470B (en) Industrial pipeline flaw detection inner wall detection method based on digital image
AU2020272936B2 (en) Methods and systems for crack detection using a fully convolutional network
CN111598856A (en) Chip surface defect automatic detection method and system based on defect-oriented multi-point positioning neural network
CN110334727B (en) Intelligent matching detection method for tunnel cracks
CN112669269A (en) Pipeline defect classification and classification method and system based on image recognition
CN113705564B (en) Pointer type instrument identification reading method
CN116188943A (en) Solar radio spectrum burst information detection method and device
CN110084587B (en) Automatic dinner plate settlement method based on edge context
CN114639064A (en) Water level identification method and device
CN111178405A (en) Similar object identification method fusing multiple neural networks
TW201601119A (en) Method for recognizing and locating object
CN111626104B (en) Cable hidden trouble point detection method and device based on unmanned aerial vehicle infrared thermal image
CN117593300B (en) PE pipe crack defect detection method and system
CN116385477A (en) Tower image registration method based on image segmentation
CN115019306A (en) Embedding box label batch identification method and system based on deep learning and machine vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant