CN111860519B - Method and system for segmenting pipeline image of aircraft engine - Google Patents
Method and system for segmenting pipeline image of aircraft engine Download PDFInfo
- Publication number
- CN111860519B CN111860519B CN202010688942.9A CN202010688942A CN111860519B CN 111860519 B CN111860519 B CN 111860519B CN 202010688942 A CN202010688942 A CN 202010688942A CN 111860519 B CN111860519 B CN 111860519B
- Authority
- CN
- China
- Prior art keywords
- pipeline
- characteristic
- image
- value
- binary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
Abstract
The invention discloses a method and a system for segmenting an aircraft engine pipeline image, wherein the method comprises the following steps: step 1, obtaining a multi-scale anisotropic feature extraction result according to a pipeline image; step 2, fusing a plurality of features into a conditional random field for feature fusion, and obtaining a pipeline binary segmentation result after training the feature weight; and 3, tracking the pipeline according to the skeleton line of the binary segmentation result, and connecting the pipeline candidate area based on the section similarity to obtain an example segmentation result. The pipeline image segmentation method and the pipeline image segmentation system provided by the invention fully utilize the geometric information and the local information of the pipeline to achieve the purpose of segmenting the pipeline, and solve the problem of aircraft engine pipeline image segmentation in a complex industrial scene.
Description
Technical Field
The invention relates to the field of image processing and machine vision, in particular to a method and a system for segmenting an aircraft engine pipeline image.
Background
An aircraft engine is a highly complex and precise mechanical structure, and ensures the stability and reliability of an aircraft in the operation process. The pipeline is responsible for transmitting liquid and gaseous media such as fuel oil, lubricating oil and air, and is an important component of the aircraft engine. A large amount of vibration can exist in the operation of the aircraft engine, and if the clearance between pipelines is too small, a large amount of friction is generated due to the vibration, so that the reliable operation of the engine is influenced. In the traditional method, workers rely on plug gauges to perform clearance measurement to eliminate hidden troubles, so that the workload is complicated and the efficiency is low. In recent years, machine vision methods are widely applied in the industrial field, and when a stereoscopic vision method is adopted to perform three-dimensional reconstruction on an aircraft engine pipeline, image segmentation on the pipeline in a complex industrial scene is an important step.
The existing technology for pipeline image segmentation mainly utilizes the gray information of pipelines and backgrounds. When the bent pipe parameters are measured by adopting a multi-view vision method in the patent 102410811B, because the pipeline is directly placed in a relatively good backlight light source environment, the pipeline area can be distinguished from the background by using a simple threshold segmentation method, and the method cannot be applied to image segmentation of aircraft engine pipelines under complex illumination and industrial backgrounds. In the method for identifying different structures on a pipeline, in patent CN109636790A, a boundary contour of a pipeline structure is determined by calculating a maximum inter-class variance of gray levels of foreground and background pixels, and this method is also only applicable to a pipeline image with a high contrast between the pipeline and the background. When the pipeline of the aircraft engine is in an industrial environment similar to the pipeline of the aircraft engine in material, the pipeline and the background cannot be distinguished only by utilizing the gray scale information.
Image segmentation has been one of the subjects of intensive research in the field of computer vision. In recent years, algorithms based on deep learning have led to further development of image segmentation techniques, which improve the accuracy of segmentation by building complex mathematical models and extensive data training. The patent CN110287932A improves the accuracy of the model for the road segmentation blocked by trees and shadows by optimizing the loss function in the convolutional neural network. In patent CN110766643A, a blood vessel is segmented by using U-Net, however, this method needs a lot of images and their labeling information, and this method is applied to the pipeline segmentation, which can not extract the pipeline correctly at the position where the pipeline is blocked or staggered, and reduces the segmentation accuracy. Because the pipeline is different, the shape is changeable in the picture mesoscale, can appear crisscross or pipeline part region receives sheltering from of industry connecting piece between the different pipelines, the geometric distribution of pipeline in the space can't be considered in current image segmentation method to pipeline fracture scheduling problem can appear in the segmentation result. Meanwhile, due to the complexity of the environment, the problem of mistaken segmentation caused by mistaken identification of the background as a pipeline is easy to occur. Therefore, the invention provides an image segmentation method for an aircraft engine pipeline, which aims at solving the problem that the image segmentation method for the aircraft engine pipeline is not solved, can segment the pipeline from a complex industrial environment, and simultaneously solves the influence of the interleaving and shielding of the pipeline on the segmentation precision.
In summary, no solution for automatic segmentation exists at present for a metal tubular structure of an aircraft engine pipeline in a complex scene. The method has the main difficulty that the geometric information and the local information of the pipeline in the image need to be comprehensively considered, so that the image segmentation task of the aircraft engine pipeline is realized.
Disclosure of Invention
The application mainly aims to provide an aircraft engine pipeline image segmentation method and system to solve the problem of automatic segmentation of pipelines in complex industrial scenes.
In order to achieve the above object, according to one aspect of the present application, an aircraft engine pipeline image segmentation method is provided.
The aircraft engine pipeline image segmentation method comprises the following steps: step 1, obtaining a multi-scale anisotropic feature extraction result according to a pipeline image; step 2, fusing a plurality of features into a conditional random field for feature fusion, and obtaining a pipeline binary segmentation result after training the feature weight; and 3, tracking the pipeline according to the skeleton line of the binary segmentation result, and connecting the pipeline candidate area based on the section similarity to obtain an example segmentation result.
Further, the acquiring a multi-scale anisotropic feature extraction result according to the pipeline image includes:
(11) Preprocessing an original image I by utilizing path morphology, reserving a linear target in the image and inhibiting a nonlinear object;
(12) The linear structure features are extracted by calculating the gray level accumulation amount in each direction by using a certain linear operator, the gray level accumulation amounts in different directions are obviously different when the point to be measured is positioned in the pipeline, and the gray level accumulation amount difference is not large when the point to be measured is positioned in the background. Sum of maximum gray scale accumulation amountsThe difference of the average gray level values in the window is used as a characteristic response value, and operators with different window sizes correspond to pipelines with different widths to form a multi-scale linear structure characteristic F 1 ；
(13) The optimal directional gradient flux characteristic is extracted by calculating the gradient integral over a circle of a certain radius, where the flux of the gradient component in a certain direction on the circle has an extreme value when the circle coincides with the edge of the pipeline. Solving gradient components by Gaussian convolution and constructing a characteristic matrix, wherein the absolute value of the characteristic value with a negative value can represent the optimal directional gradient flux characteristic response value of the current point, and the multi-scale characteristic F is formed by changing the radius value to correspond to pipelines with different widths 2 ；
(14) And acquiring a pipeline enhancement characteristic based on a Hessian matrix, and solving the characteristic by utilizing the maximum output response when the matching with the actual width of the tubular target is maximum under different scales. Whether the characteristic value of a Hessian matrix formed by the points meets a certain condition is analyzed, and whether the characteristic value belongs to a pipeline is judged. Changing the scale of the Gaussian kernel to adapt to pipelines with different widths to obtain a pipeline enhancement characteristic F 3 。
Further, the step of fusing the plurality of features into the conditional random field for feature fusion, and obtaining a pipeline binary segmentation result after training the feature weight includes:
(21) And fusing the multi-scale anisotropic characteristics of the pipeline by using the full-connection conditional random field. When the conditional random field is used for image segmentation, the problem of minimization of the energy function E can be regarded as expressed by formula (1),
wherein the unary term θ of the energy function u From multi-scale anisotropic features F i To find a binary term theta p Calculating gray scale and distance information among different pixels in the image to obtain the correlation;
(22) And the weight vector between the unary term and the binary term is obtained by the training of the structured SVM. And finally, obtaining a pipeline two-value segmentation result y.
Further, the performing pipeline tracking according to the skeleton line of the binary segmentation result, and connecting the candidate pipeline regions based on the section similarity to obtain an example segmentation result includes:
(31) And (3) solving a main skeleton line by utilizing a connected domain skeleton thinning algorithm of the binary image and pruning. Meanwhile, according to the characteristic that the local edge of the pipeline can be approximately regarded as a line segment, the edge linear response of the pipeline is obtained. The method is mainly characterized in that the method is obtained by performing open operation on a group of linear operators in different directions and a binary image subjected to Laplace transform, and the maximum value of the open operation in different directions at each point is obtained and is the linear response value;
(32) And tracking the pipeline according to the skeleton line result, providing a pipeline cross-point pair extraction algorithm, correctly acquiring a pipeline candidate region by using local information of the pipeline, and further screening the pipeline and the background in the binary image. The cross-point pair extraction algorithm mainly comprises three steps, namely, firstly, selecting an initial tracking point x 1 And using the main skeleton line as a fitting central line, and forming a first group of section point pairs (t) by two points of which the normal lines are intersected with the edge of the pipeline 1 ,x 1 ,p 1 ) (ii) a Then tracking along the central line to obtain a series of cross-sectional point pairs { (t) 1 ,x 1 ,p 1 ),...,(t i ,x i ,p i ) When no central line can be tracked at the tail end of the central line, moving a proper amount of positions along the tail end direction, and calculating two points t which are shortest to the sectional line of the pipeline edge at the points i+1 And p i+1 And two points and the middle point thereof are taken as a new cross-sectional point pair (t) i+1 ,x i+1 ,p i+1 ) (ii) a And finally, finishing pipeline tracking when the width of the obtained cross-point pair is different from the previous average width by a certain threshold value to obtain a pipeline candidate area S i ＝{(t 1 ,x 1 ,p 1 ),...,(t m ,x m ,p m )}；
(33) And after the pipeline candidate area is obtained, judging whether the pipeline candidate area belongs to the same pipeline or not by calculating the similarity of the pipeline section. The calculation of the similarity of the pipeline cross section mainly comprises the judgment of the width, the direction and the distance of the cross section.
To achieve the above object, according to another aspect of the present application, an aircraft engine pipeline image segmentation system is provided.
An aircraft engine pipeline image segmentation system according to the application comprises: the characteristic extraction module is used for extracting the multi-scale anisotropic characteristics of the pipeline image according to the geometric characteristics of the pipeline; the characteristic fusion module is used for fusing a plurality of characteristics into a conditional random field for characteristic fusion, and obtaining a pipeline binary segmentation result after training the characteristic weight; and the example segmentation module is used for tracking the pipeline according to the skeleton line of the binary segmentation result and connecting the pipeline candidate region based on the section similarity to obtain an example segmentation result.
Further, the feature extraction module comprises:
(11) Preprocessing an original image I by utilizing path morphology, reserving a linear target in the image, and inhibiting a nonlinear object;
(12) The linear structure features are extracted by calculating the gray level accumulation amount in each direction by using a certain linear operator, the gray level accumulation amounts in different directions are obviously different when the point to be measured is positioned in the pipeline, and the gray level accumulation amount difference is not large when the point to be measured is positioned in the background. Taking the difference between the maximum gray scale accumulation amount and the average gray scale value in the window as a characteristic response value, and forming a multi-scale linear structure characteristic F by using operators with different window sizes corresponding to pipelines with different widths 1 ；
(13) The optimal directional gradient flux characteristic is extracted by calculating the gradient integral over a circle of a certain radius, when the circle coincides with the edge of the pipeline, the flux of the gradient component in a certain direction on the circle has an extreme value. Solving gradient components by Gaussian convolution and constructing a characteristic matrix, wherein the absolute value of the characteristic value with a negative value can represent the optimal directional gradient flux characteristic response value of the current point, and the multi-scale characteristic F is formed by changing the radius value to correspond to pipelines with different widths 2 ；
(14) Acquiring pipeline enhancement characteristics based on Hessian matrix and utilizing the enhancement characteristics under different scalesMatching the actual width of the tubular object with the maximum output response maximizes to find the feature. Whether the characteristic value of a Hessian matrix formed by the points meets a certain condition is analyzed, and whether the characteristic value belongs to a pipeline is judged. Changing the dimension of the Gaussian kernel to adapt to pipelines with different widths to obtain a pipeline enhancement feature F 3 。
Further, the feature fusion module comprises:
(21) And fusing the multi-scale anisotropic characteristics of the pipeline by using the full-connection conditional random field. When the conditional random field is used for image segmentation, it can be regarded as a minimization problem of the energy function E, and can be expressed by formula (1).
Wherein the unary term θ of the energy function u From multi-scale anisotropic features F i To obtain a binary term theta p Obtaining the correlation relationship by calculating the gray scale and distance information among different pixels in the preprocessed image;
(22) And the weight vector between the unary item and the binary item is obtained by the training of the structured SVM. And finally, obtaining a pipeline two-value segmentation result y.
Further, the instance partitioning module includes:
(31) And solving a main skeleton line by using a connected domain skeleton thinning algorithm of the binary image and pruning. Meanwhile, according to the characteristic that the local edge of the pipeline can be approximately regarded as a line segment, the edge linear response of the pipeline is obtained. The method is mainly characterized in that a group of linear operators in different directions and a binary image subjected to Laplace transform are subjected to open operation to obtain the maximum value of the open operation at each point in different directions, namely the linear response value;
(32) And tracking the pipeline according to the skeleton line result, providing a pipeline cross-point pair extraction algorithm, correctly acquiring a pipeline candidate region by using local information of the pipeline, and further screening the pipeline and the background in the binary image. The cross-point pair extraction algorithm mainly comprises three stepsFirstly, selecting an initial tracking point x 1 And using the main skeleton line as the fitting central line, and forming a first group of section point pairs (t) by two points of the intersection of the normal line and the pipeline edge 1 ,x 1 ,p 1 ) (ii) a Then tracking along the central line to obtain a series of cross-sectional point pairs { (t) 1 ,x 1 ,p 1 ),...,(t i ,x i ,p i ) When no central line can be tracked at the tail end of the central line, moving a proper amount of positions along the tail end direction, and calculating two points t which are shortest to the sectional line of the pipeline edge at the points i+1 And p i+1 And two points and the middle point are taken as a new cross-sectional point pair (t) i+1 ,x i+1 ,p i+1 ) (ii) a And finally, finishing pipeline tracking when the width of the obtained cross-point pair is different from the previous average width by a certain threshold value to obtain a pipeline candidate area S i ＝{(t 1 ,x 1 ,p 1 ),...,(t m ,x m ,p m )}；
(33) And after the pipeline candidate area is obtained, judging whether the pipeline candidate area belongs to the same pipeline or not by calculating the similarity of the pipeline section. The calculation of the similarity of the pipeline cross section mainly comprises the judgment of the width, the direction and the distance of the cross section.
The invention provides a method and a system for segmenting an aircraft engine pipeline image. In the characteristic extraction stage, three multi-scale anisotropic characteristic extraction pipelines are utilized; in the feature fusion stage, a full-connection conditional random field is adopted to fuse the three features to obtain a binary segmentation result; in the example segmentation stage, a pipeline section point pair is proposed to perform pipeline tracking and acquisition on a candidate region by an algorithm, the section similarity is calculated and the regions belonging to the same pipeline are connected, and a final example segmentation result is obtained.
Compared with the prior art, the invention has the advantages that: the method and the system for segmenting the pipeline image of the aircraft engine can meet the requirements of no fracture and high precision for pipeline segmentation, fully consider the geometric information and the local information of the pipeline of the aircraft engine compared with the existing pipeline, remote sensing road, blood vessel and other image segmentation technologies, and enhance the connectivity of the pipeline. The characteristic extraction technology adopted in the invention can utilize the geometric characteristics of the pipeline slenderness to enhance the contrast between the pipeline and the background; the full-connection conditional random field is used for feature fusion, so that a more refined segmentation result can be obtained; in the example segmentation stage, local information of the pipeline is utilized to connect the two-value segmentation results of the fracture, so that the segmentation precision is improved.
Drawings
The accompanying drawings are included to provide a further understanding of the present method and system, and to further clarify the features and advantages of the present application. The drawings are only for purposes of illustrating the present application and are not to be construed as limiting the present application. In the drawings:
FIG. 1 is a flowchart of an overall implementation of the method and system for segmenting an aircraft engine pipeline image according to the present invention;
FIG. 2 is a flow diagram of a feature extraction module;
FIG. 3 is a feature fusion module flow diagram;
FIG. 4 is an example segmentation module flow diagram;
FIG. 5 is a schematic diagram of a pipeline tracking algorithm; wherein, fig. 5 (a) is an initial pipeline tracking diagram, fig. 5 (b) is an intermediate tracking effect diagram from tracking to the end of the skeleton line, and fig. 5 (c) is a complete pipeline tracking result diagram;
FIG. 6 is a schematic diagram of a connection of candidate areas of a pipeline; wherein, fig. 6 (a) is a candidate area of a pipeline to be connected, fig. 6 (b) is a candidate area of a pipeline which is partially connected, and fig. 6 (c) is a final complete pipeline connection result;
FIG. 7 is a diagram illustrating the effect of the pipeline image segmentation experiment.
Detailed Description
The present invention will be described in further detail with reference to the following drawings and specific embodiments.
Fig. 1 is a general flow chart of an implementation of the method and system for segmenting an aircraft engine pipeline image according to the present invention, which specifically includes the following steps:
step 1: and acquiring an aircraft engine pipeline image and establishing a pipeline image data set.
In this example, a total of 186 images were acquired at approximately 40 locations on a large aircraft engine. 112 images of the training set were used as the training set, and the remaining 74 images were used as the test set. The size of the image is 512 × 612 pixels. Wherein the width of the pipeline in the image ranges from 7 pixels to 50 pixels, and the pipeline comprises pipelines with various dimensions.
Step 2: and acquiring a multi-scale anisotropic feature extraction result according to the pipeline image.
And (3) preprocessing the original image I by utilizing path morphology, reserving a linear target in the image and inhibiting a nonlinear object. The path shape operator chosen in this example is 100 pixels in length. As shown in fig. 2, after image preprocessing, three features are extracted according to the structural features of the slender pipeline: multi-scale linear structural feature F 1 Optimal directional gradient flux characteristic F 2 And a pipeline enhancement feature F 3 。
The linear structure features are extracted by calculating the gray level accumulation amount in each direction by using a certain linear operator, the gray level accumulation amounts in different directions have obvious difference when the point to be measured is positioned in the pipeline, and the gray level accumulation amount difference is not large when the point to be measured is positioned in the background. Taking the difference between the maximum gray scale accumulation amount and the average gray scale value in the window as a characteristic response value, wherein operators with different window sizes correspond to pipelines with different widths to form a multi-scale linear structure characteristic F 1 . The operator sizes selected in this example are 5, 13, 21, 29, 3, 53 pixels, for a total of 7 lengths.
The optimal directional gradient flux characteristic is extracted by calculating the gradient integral over a circle of a certain radius, where the flux of the gradient component in a certain direction on the circle has an extreme value when the circle coincides with the edge of the pipeline. Solving gradient components by Gaussian convolution and constructing a characteristic matrix, wherein the absolute value of the characteristic value with a negative value can represent the optimal directional gradient flux characteristic response value of the current point, and the multi-scale characteristic F is formed by changing the radius value to correspond to pipelines with different widths 2 . The radius values selected in this example are 3, 6, 9, 12, 15 pixels, for a total of 5 lengths.
Acquiring pipeline enhancement characteristics based on Hessian matrix, and utilizing the enhancement characteristics of pipelines with tubular meshes under different scalesThe actual width of the target matches the maximum response output at maximum to solve for the feature. Whether the characteristic value of a Hessian matrix formed by the points meets a certain condition is analyzed, and whether the characteristic value belongs to a pipeline is judged. Changing the dimension of the Gaussian kernel to adapt to pipelines with different widths to obtain a pipeline enhancement characteristic F 3 . The scale sizes of the gaussian kernels selected in this example are 1, 3, 5, 7, 9, 12, 15 pixels, for a total of 7.
And step 3: and (4) fusing a plurality of features into a conditional random field for feature fusion, and obtaining a pipeline binary segmentation result after training the feature weight.
As shown in fig. 3, the full-connection conditional random field is used to fuse the multi-scale anisotropic features of the three pipelines. When the conditional random field is used for image segmentation, it can be regarded as a minimization problem of the energy function E, and can be expressed by formula (1).
Wherein the unary term θ of the energy function u From multi-scale anisotropic features F i To obtain a binary term theta p Calculating gray scale and distance information among different pixels in the image to obtain the correlation;
and the weight vector between the unary term and the binary term is obtained by the training of the structured SVM. And finally, obtaining a pipeline binary segmentation result y.
And 4, step 4: and tracking the pipeline according to the skeleton line of the binary segmentation result, and connecting the pipeline candidate area based on the section similarity to obtain an example segmentation result.
As shown in fig. 4, a main skeleton line is obtained by a pruning operation using a connected domain skeleton refinement algorithm of a binary image. Meanwhile, according to the characteristic that the local edge of the pipeline can be approximately regarded as a line segment, the edge linear response of the pipeline is obtained. The method is mainly characterized in that a group of linear operators in different directions and a binary image subjected to Laplace transform are subjected to open operation to obtain the linear response value, and the maximum value of the open operation in different directions at each point is obtained;
as shown in fig. 5, pipeline tracking is performed according to the skeleton line result, and the present application provides a pipeline cross-point pair extraction algorithm, which uses local information of a pipeline to correctly obtain a candidate area of the pipeline, and further screens the pipeline and the background in a binary image. The cross-point pair extraction algorithm mainly comprises three steps, namely, firstly, selecting an initial tracking point x 1 And using the main skeleton line as a fitting center line, and forming a first group of section point pairs (t) by two points of which the normal line intersects with the edge of the pipeline 1 ,x 1 ,p 1 ) (ii) a Then tracking along the central line to obtain a series of cross-sectional point pairs { (t) 1 ,x 1 ,p 1 ),...,(t i ,x i ,p i ) When no central line can be tracked at the tail end of the central line, moving a proper amount of positions along the tail end direction, and calculating two points t which are shortest to the sectional line of the pipeline edge at the points i+1 And p i+1 And two points and the midpoint thereof are taken as a new cross-sectional point pair (t) i+1 ,x i+1 ,p i+1 ) (ii) a Finally, when the width of the obtained cross-section point pair is different from the previous average width by a certain threshold value, the pipeline tracking is finished to obtain a pipeline candidate area S i ＝{(t 1 ,x 1 ,p 1 ),...,(t m ,x m ,p m )}；
And after the pipeline candidate area is obtained, judging whether the pipeline candidate area belongs to the same pipeline or not by calculating the similarity of the pipeline sections. The calculation of the similarity of the pipeline section mainly comprises the judgment of the width, the direction and the distance of the section. As shown in fig. 6, for one pipeline candidate region S 1 Two candidate regions S exist within a certain range 2 And S 3 . To determine the similarity relationship, S is calculated separately 1 And the section similarity of the end surfaces at the two sides and the rest two candidate areas. Is calculated to obtain S 1 Right side cross section and S 2 The left side section has the highest similarity, and is connected to obtain a new pipeline candidate region S' 1 And S' 2 . Similarly, mixing S' 1 To the right side and S' 2 Is connected to obtain a final pipeline S ″) 1 . FIG. 7 shows the final circuit, as shown in FIG. 7As a result of the segmentation, different pipelines are labeled with different gray scales.
Finally, tests are carried out on 74 images, and the actual segmentation precision of the aviation engine pipeline image segmentation method and system provided by the invention is as follows: f1 is 0.732, the pipeline segmentation accuracy is 84.2%, and the pipeline segmentation recall rate is 89.5%.
While the invention has been described with respect to specific preferred embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.
Claims (2)
1. An aircraft engine pipeline image segmentation method is characterized by comprising the following steps:
step 1, obtaining a multi-scale anisotropic feature extraction result according to a pipeline image;
step 2, fusing a plurality of features into a conditional random field for feature fusion, and obtaining a pipeline binary segmentation result after training the feature weight;
step 3, performing pipeline tracking according to the skeleton lines of the binary segmentation result, and connecting pipeline candidate areas based on section similarity to obtain an example segmentation result;
the method for acquiring the multi-scale anisotropic feature extraction result according to the pipeline image comprises the following steps:
(11) Preprocessing an original image I by utilizing path morphology, reserving a linear target in the image and inhibiting a nonlinear object;
(12) Linear structural features are extracted by calculating gray scale accumulation amounts in all directions by linear operators with certain size, when a point to be measured is positioned in a pipeline, the gray scale accumulation amounts in different directions are obviously different, when the point to be measured is positioned in a background, the gray scale accumulation amount difference is not large, the difference between the maximum gray scale accumulation amount and the average value of the gray scales in a window is used as a feature response value, and the operators with different window sizes correspond to pipelines with different widths to form a multi-scale linear structural feature F 1 ；
(13) Extracting optimal directional gradient channel by calculating gradient integral over a circle of a certain radiusQuantity characteristic, when the circle is coincident with the edge of the pipeline, the flux of the gradient component in a certain direction on the circumference has an extreme value, the Gaussian convolution is utilized to solve the gradient component and construct a characteristic matrix, the absolute value of the characteristic value with a negative value can represent the optimal directional gradient flux characteristic response value of the current point, and the multi-scale characteristic F is formed by changing the radius value to correspond to the pipelines with different widths 2 ；
(14) Acquiring a pipeline enhancement characteristic based on a Hessian matrix, solving the characteristic by utilizing the maximum response output when the Hessian matrix is matched with the actual width of a tubular target at different scales, judging whether the characteristic value of the Hessian matrix formed by points at the maximum matching meets a certain condition or not, further judging whether the characteristic value belongs to the pipeline or not, changing the scale of a Gaussian kernel to adapt to the pipelines with different widths, and obtaining a pipeline enhancement characteristic F 3 ；
The method for fusing the plurality of features into the conditional random field for feature fusion and obtaining the pipeline binary segmentation result after training the feature weight comprises the following steps:
(21) The full-connection conditional random field is used for fusing the multi-scale anisotropic characteristics of the pipeline, when the conditional random field is used for image segmentation, the problem can be regarded as the minimization of an energy function E, and can be expressed by a formula (1),
wherein the unary term θ of the energy function u From multi-scale anisotropic features F i To obtain a binary term theta p Calculating gray scale and distance information among different pixels in the image to obtain the correlation;
(22) The weight vector between the unary item and the binary item is obtained by the training of a structured SVM, and finally a pipeline binary segmentation result y can be obtained;
the pipeline tracking is carried out according to the skeleton line of the binary segmentation result, and the pipeline candidate region is connected based on the section similarity, so that the example segmentation result is obtained and comprises the following steps:
(31) Obtaining a main skeleton line by utilizing a connected domain skeleton thinning algorithm of a binary image and pruning, and meanwhile obtaining edge linear response of the pipeline according to the characteristic that local edges of the pipeline are approximately regarded as a line segment;
(32) Performing pipeline tracking according to the skeleton line result, providing a pipeline cross-point pair extraction algorithm, correctly obtaining a pipeline candidate region by using local information of a pipeline, and further screening the pipeline and a background in a binary image, wherein the cross-point pair extraction algorithm mainly comprises three steps of firstly selecting an initial tracking point x 1 And using the main skeleton line as the fitting central line, and forming a first group of section point pairs (t) by two points of the intersection of the normal line and the pipeline edge 1 ,x 1 ,p 1 ) (ii) a Then tracking along the central line to obtain a series of cross-sectional point pairs { (t) 1 ,x 1 ,p 1 ),...,(t i ,x i ,p i ) When there is no central line tracked at the end of the central line, moving a proper amount of position along the direction of the end, and finding two points t shortest to the section line of the pipeline edge at the tracking point i+1 And p i+1 And two points and the midpoint thereof are taken as a new cross-sectional point pair (t) i+1 ,x i+1 ,p i+1 ) (ii) a Finally, when the width of the obtained cross-section point pair is different from the previous average width by a certain threshold value, the pipeline tracking is finished to obtain a pipeline candidate area S i ＝{(t 1 ,x 1 ,p 1 ),...,(t m ,x m ,p m )}；
(33) After the pipeline candidate area is obtained, whether the pipeline candidate area belongs to the same pipeline is judged by calculating the similarity of the pipeline section, wherein the calculation of the similarity of the pipeline section mainly comprises the judgment of the section width, the direction and the distance.
2. An aircraft engine pipeline image segmentation system, comprising:
the characteristic extraction module is used for extracting the multi-scale anisotropic characteristics of the pipeline image according to the geometric characteristics of the pipeline;
the characteristic fusion module is used for fusing a plurality of characteristics into a conditional random field for characteristic fusion, and obtaining a pipeline binary segmentation result after training the characteristic weight;
the example segmentation module is used for tracking the pipeline according to the skeleton line of the binary segmentation result and connecting the pipeline candidate area based on the section similarity to obtain an example segmentation result;
the feature extraction module includes:
(11) Preprocessing an original image I by utilizing path morphology, reserving a linear target in the image and inhibiting a nonlinear object;
(12) Linear structural features are extracted by calculating gray scale accumulation amounts in all directions by linear operators with certain size, when a point to be measured is positioned in a pipeline, the gray scale accumulation amounts in different directions are obviously different, when the point to be measured is positioned in a background, the gray scale accumulation amount difference is not large, the difference between the maximum gray scale accumulation amount and the average value of the gray scales in a window is used as a feature response value, and the operators with different window sizes correspond to pipelines with different widths to form a multi-scale linear structural feature F 1 ；
(13) Extracting optimal directional gradient flux characteristics by calculating gradient integrals on a circle with a certain radius, when the circle is overlapped with the edge of a pipeline, the flux of a gradient component in a certain direction on the circle has an extreme value, solving the gradient component by utilizing Gaussian convolution and constructing an characteristic matrix, wherein the absolute value of a characteristic value with a negative value can represent the optimal directional gradient flux characteristic response value of the current point, and forming a multi-scale characteristic F by changing the radius value to correspond to pipelines with different widths 2 ；
(14) Acquiring a pipeline enhancement characteristic based on a Hessian matrix, solving the characteristic by utilizing the maximum response output when the Hessian matrix is matched with the actual width of a tubular target at different scales, judging whether the characteristic value of the Hessian matrix formed by points at the maximum matching meets a certain condition or not, changing the scale of a Gaussian kernel to adapt to pipelines with different widths, and obtaining a pipeline enhancement characteristic F 3 ；
The feature fusion module includes:
(21) The full-connection conditional random field is used for fusing the multi-scale anisotropic characteristics of the pipeline, when the conditional random field is used for image segmentation, the problem can be regarded as the minimization of an energy function E, and can be expressed by a formula (1),
wherein the unary term θ of the energy function u From multi-scale anisotropic features F i To obtain a binary term theta p Obtaining the correlation relationship by calculating the gray scale and distance information among different pixels in the image;
(22) The weight vector between the unary term and the binary term is obtained by the training of a structured SVM, and finally a pipeline binary segmentation result y can be obtained;
the instance partitioning module includes:
(31) Obtaining a main skeleton line by utilizing a connected domain skeleton thinning algorithm of a binary image and pruning, and meanwhile obtaining edge linear response of the pipeline according to the characteristic that local edges of the pipeline are approximately regarded as a line segment;
(32) Performing pipeline tracking according to the skeleton line result, providing a pipeline cross-point pair extraction algorithm, correctly obtaining a pipeline candidate region by using local information of a pipeline, and further screening the pipeline and a background in a binary image, wherein the cross-point pair extraction algorithm mainly comprises three steps of firstly selecting an initial tracking point x 1 And using the main skeleton line as the fitting central line, and forming a first group of section point pairs (t) by two points of the intersection of the normal line and the pipeline edge 1 ,x 1 ,p 1 ) (ii) a Then tracking along the central line to obtain a series of cross-sectional point pairs { (t) 1 ,x 1 ,p 1 ),...,(t i ,x i ,p i ) At the end of the center lineWhen the end has no tracking central line, the position is moved along the end direction, and two points t with the shortest line to the pipeline edge are found at the tracking point i+1 And p i+1 And taking the two points and the middle point thereof as a new cross-sectional point pair (t) i+1 ,x i+1 ,p i+1 ) (ii) a And finally, finishing pipeline tracking when the width of the obtained cross-point pair is different from the previous average width by a certain threshold value to obtain a pipeline candidate area S i ＝{(t 1 ,x 1 ,p 1 ),...,(t m ,x m ,p m )}；
(33) After the pipeline candidate area is obtained, whether the pipeline candidate area belongs to the same pipeline is judged by calculating the similarity of the pipeline section, wherein the calculation of the similarity of the pipeline section mainly comprises the judgment of the section width, the direction and the distance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010688942.9A CN111860519B (en) | 2020-07-17 | 2020-07-17 | Method and system for segmenting pipeline image of aircraft engine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010688942.9A CN111860519B (en) | 2020-07-17 | 2020-07-17 | Method and system for segmenting pipeline image of aircraft engine |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111860519A CN111860519A (en) | 2020-10-30 |
| CN111860519B true CN111860519B (en) | 2023-02-28 |
Family
ID=72984788
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010688942.9A Active CN111860519B (en) | 2020-07-17 | 2020-07-17 | Method and system for segmenting pipeline image of aircraft engine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111860519B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107256550A (en) * | 2017-06-06 | 2017-10-17 | 电子科技大学 | A kind of retinal image segmentation method based on efficient CNN CRF networks |
| CN109102511A (en) * | 2018-07-06 | 2018-12-28 | 深圳先进技术研究院 | A kind of cerebrovascular dividing method, system and electronic equipment |
-
2020
- 2020-07-17 CN CN202010688942.9A patent/CN111860519B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107256550A (en) * | 2017-06-06 | 2017-10-17 | 电子科技大学 | A kind of retinal image segmentation method based on efficient CNN CRF networks |
| CN109102511A (en) * | 2018-07-06 | 2018-12-28 | 深圳先进技术研究院 | A kind of cerebrovascular dividing method, system and electronic equipment |
Non-Patent Citations (8)
| Title |
|---|
| A Fast Bolt-Loosening Detection Method of Running Train’s Key Components ed on Binocular Vision;Junhua Sun 等;《IEEE ACCESS》;20190326;全文 * |
| 一种从X射线血管造影图像中自动分割冠状动脉的新方法;周寿军等;《中国科学(E辑:信息科学)》;20071215(第12期);全文 * |
| 一种基于形状特征进行高分辨率遥感影像道路提取方法;雷小奇等;《测绘学报》;20091015(第05期);全文 * |
| 从SPOT全色影像中提取道路网络的算法研究;贾鹏等;《测绘与空间地理信息》;20090225(第01期);全文 * |
| 基于HESSIAN增强和形态学尺度空间的视网膜血管分割;于挥等;《计算机应用与软件》;20160815(第08期);全文 * |
| 基于全连接条件随机场模型的眼底图像血管分割;郭莹等;《微处理机》;20180831(第04期);第2部分 * |
| 基于多尺度线性检测的视网膜血管分割;高向军;《科学技术与工程》;20130818(第23期);全文 * |
| 基于多特征融合和条件随机场的道路分割;闫昭帆等;《计算机系统应用》;20200228(第03期);摘要，第1部分 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111860519A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106875381B (en) | Mobile phone shell defect detection method based on deep learning | |
| Alidoost et al. | A CNN-based approach for automatic building detection and recognition of roof types using a single aerial image | |
| Cheng et al. | Outdoor scene image segmentation based on background recognition and perceptual organization | |
| CN108596165B (en) | Road traffic marking detection method and system based on unmanned plane low latitude Aerial Images | |
| CN108830171B (en) | Intelligent logistics warehouse guide line visual detection method based on deep learning | |
| CN108734694A (en) | Thyroid tumors ultrasonoscopy automatic identifying method based on faster r-cnn | |
| CN107346420A (en) | Text detection localization method under a kind of natural scene based on deep learning | |
| CN104751187A (en) | Automatic meter-reading image recognition method | |
| Wang et al. | Adaptive road detection via context-aware label transfer | |
| CN103295013A (en) | Pared area based single-image shadow detection method | |
| CN111611861B (en) | Image change detection method based on multi-scale feature association | |
| CN112465759A (en) | Convolutional neural network-based aeroengine blade defect detection method | |
| Xing et al. | Traffic sign recognition using guided image filtering | |
| CN117237646B (en) | PET high-temperature flame-retardant adhesive tape flaw extraction method and system based on image segmentation | |
| CN113435452A (en) | Electrical equipment nameplate text detection method based on improved CTPN algorithm | |
| CN116630971A (en) | Wheat scab spore segmentation method based on CRF_Resunate++ network | |
| Mlyahilu et al. | Morphological geodesic active contour algorithm for the segmentation of the histogram‐equalized welding bead image edges | |
| Naiemi et al. | Scene text detection using enhanced extremal region and convolutional neural network | |
| Selamet et al. | Automatic detection and classification of defective areas on metal parts by using adaptive fusion of faster R-CNN and shape from shading | |
| CN111860519B (en) | Method and system for segmenting pipeline image of aircraft engine | |
| Kofler et al. | Classifying defects in topography images of silicon wafers | |
| Cheng et al. | Power pole detection based on graph cut | |
| Nasr-Isfahani et al. | A new approach for touching cells segmentation | |
| Mao et al. | Disparity filtering with 3D convolutional neural networks | |
| Li et al. | Superpixel-based adaptive salient region analysis for infrared and visible image fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |