WO2023036346A1

WO2023036346A1 - Yolov5-based method and apparatus for performing real-time detection of blade crack during operation and maintenance of aero engine

Info

Publication number: WO2023036346A1
Application number: PCT/CN2022/119657
Authority: WO
Inventors: 李双宝; 于婧仪
Original assignee: 中国民航大学
Priority date: 2021-09-13
Filing date: 2022-09-19
Publication date: 2023-03-16
Also published as: CN113838013A

Abstract

The present invention provides a YOLOv5-based method and apparatus for performing real-time detection of a blade crack during the operation and maintenance of an aero engine, and relates to the technical field of aero generator detection. The method comprises: sending a first instruction to acquire an internal blade image of an engine; preprocessing the internal blade image of the engine to acquire a test data set, a training data set and a verification data set; inputting the training data set into a preset YOLOv5 network model to perform training, performing, by using the verification data set, a preliminary evaluation on a model effect that is exported from the training, so as to adjust the model, performing a test by using a trained weight file and the test data set, and acquiring an mAP value and a P-R curve to perform a final evaluation on the model; and acquiring the internal blade image of the engine in real time, performing real-time detection on an engine blade by using the weight file, and then outputting a detection result. By means of the present invention, the detection speed of a blade crack in the prior art can be increased, and the network flexibility of a blade crack detection algorithm is improved.

Description

Method and device for real-time detection of blade cracks in aero-engine operation and maintenance based on YOLOv5

technical field

The invention relates to the technical field of aero-generator detection, in particular to a method and device for real-time detection of blade cracks in aero-engine operation and maintenance based on YOLOv5.

Background technique

The normal operation of aero-engine blades can provide continuous flight power for the engine, and its service time is often longer. Such an environment is prone to fatigue cracks, and these cracks on the engine blades pose a potential threat to the normal operation of the aero-engine. . Cracks that are not treated in time will further deteriorate, leading to the paralysis of the entire engine, which poses a serious threat to normal aviation flight. In fact, as long as there are cracks on the blades of the engine, no matter how big or small it is, it will endanger people and pose a serious threat to the machine. For a long time, flight accidents caused by fracture of turbine blades have been common in flight. Therefore, in order to ensure the safe operation of aero-engines, regular detection of blade cracks is very important.

Existing methods for detecting blade cracks include traditional methods such as borescope and penetrant testing, X-ray and magnetic particle testing, eddy current and ultrasonic testing, and image processing methods such as Faster R-CNN two-stage algorithm. The main problems of the traditional method are: limited number of manual marking and poor robustness, many process steps, time-consuming and labor-intensive, etc.

The target detection algorithm is divided into single-stage and two-stage algorithms. The single-stage algorithm performs positioning prediction after the image information is input, and directly outputs the result. The detection speed is fast but there are many anchor frames, and the selection of anchor frames needs to be optimized. YOLO is a single-stage algorithm. The representative algorithm of stage target detection, which outputs the position and category confidence of the target frame at one time; the two-stage algorithm classifies and regresses the anchor frame, performs multiple detections and updates, the speed is slower than the single-stage algorithm, and the structure is not flexible enough, but the network fusion high degree.

To sum up, the blade crack detection method in the prior art relies on manual marking, which is inefficient, and the target detection algorithm cannot satisfy both the speed of blade detection and the flexibility of blade detection network.

Contents of the invention

In view of this, the purpose of the present invention is to provide a method and device for real-time detection of blade cracks in aero-engine operation and maintenance based on YOLOv5, so as to improve the detection speed of blade cracks in the prior art and improve the network flexibility of blade crack detection algorithms.

In the first aspect, the embodiment of the present invention provides a method for real-time detection of blade cracks in aero-engine operation and maintenance based on YOLOv5, which is applied to a host computer and specifically includes the following steps:

Sending a first command to obtain an image of the blades inside the engine;

Preprocessing the internal blade image of the engine to obtain a test data set, a training data set and a verification data set;

Input the training data set to the pre-set YOLOv5 network model for training, use the verification data set to conduct a preliminary evaluation of the model effect derived from the training to adjust the model, use the trained weight file and test data set to test, and obtain mAP value and P-R curve for final evaluation of the model;

Obtain the blade image inside the engine in real time and use the weight file to detect the engine blade in real time and output the detection result.

Preferably, the step of preprocessing the internal blade image of the engine to obtain a test data set, a training data set and a verification data set includes:

Crack marking, zooming and flipping are performed on the internal blade image of the engine to expand the data set to obtain the test data set, training data set and verification data set.

Preferably, the preset YOLOv5 network model includes an input terminal, a Backbone backbone network, a Neck neck network and an output terminal;

The input terminal includes a data enhancement module and an anchor frame selection module, which are used to splicing the input data in a manner of scaling, clipping, and random arrangement, and the anchor frame selection module is used for the internal blade image of the engine The marked crack anchor box, calculate and update the size of the crack anchor box;

Described Backbone backbone network comprises Focus structure, CBL structure, CSP structure and SPP module;

The Focus structure obtains the crack picture at the input end to perform a slice parallel convolution operation, and the CBL structure extracts the feature information of the crack picture through the parallel convolution operation of the Focus structure slice;

The SPP module is used to equally divide the feature map of the crack image feature and perform a pooling operation;

The Neck network includes an FPN structure and a PAN structure, the FPN structure and the CBL structure increase the size of the crack picture feature map, and the PAN structure and the FPN structure perform feature fusion to reduce the crack picture features Figure size;

The output terminal marks the crack information and outputs the confidence level to obtain the precision rate and recall rate.

Preferably, the pooling operation is performed using the following formula:

S _H — the height of the matrix;

S _W — the width of the matrix;

h—the height of the image;

w—the width of the picture;

p—fill quantity;

f—filter size;

s—step size.

Preferably, the accuracy rate is obtained using the following formula

n—the total number of identified pictures;

TP—the number of correctly identified pictures;

FP—Number of incorrectly identified images.

Use the following formula to obtain the recall rate:

m—the total number of pictures to be identified;

FN—Number of pictures that were targeted but not recognized by the system.

Preferably, the following steps are used to obtain TP and FP:

Obtain the confidence degree and the degree of overlap IOU, and the degree of overlap IOU is obtained by the following formula:

B _p —prediction box;

B _gt — ground truth box;

Obtain the TP and the FP based on the overlap IOU:

If the recognized area is larger than the IOU threshold, it is judged as TP, and if it is smaller than the IOU threshold, it is judged as FP.

Preferably, the following formula is used to obtain the mAP value:

P—precision;

r—recall rate;

The mAP value is the average of the AP values of the categorical features.

On the other hand, the present invention provides a real-time detection device for blade cracks in aero-engine operation and maintenance based on YOLOv5, including:

Image acquisition instruction sending module: send the first instruction to acquire the internal blade image of the engine;

Sample collection module: used to preprocess the internal blade image of the engine to obtain a test data set, a training data set and a verification data set;

Training and testing module: used to input the training data set to the pre-set YOLOv5 network model for training, use the verification data set to conduct a preliminary evaluation of the model effect derived from training to adjust the model, use the trained weight file and test data The set is tested, and the mAP value and P-R curve are obtained for the final evaluation of the model;

Output module: acquire the internal blade image of the engine in real time and use the weight file to detect the engine blade in real time and output the detection result.

The embodiment of the present invention brings the following beneficial effects: the present invention provides a method and device for real-time detection of blade cracks in aero-engine operation and maintenance based on YOLOv5. The specific steps of the method include: sending a first command to obtain an image of the blade inside the engine; Preprocess the internal blade image of the engine to obtain the test data set, training data set and verification data set; input the training data set to the preset YOLOv5 network model for training, and use the verification data set to conduct a preliminary evaluation of the model effect derived from the training. Adjust the model, use the trained weight file and test data set to test, and obtain the mAP value and P-R curve for final evaluation of the model; obtain the internal blade image of the engine in real time and use the weight file to detect the engine blade in real time and output the detection result. The invention can improve the detection speed of blade cracks in the prior art, and improve the network flexibility of the blade crack detection algorithm.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

In order to make the above-mentioned objects, features and advantages of the present invention more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

Description of drawings

In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings that need to be used in the specific implementation or description of the prior art. Obviously, the accompanying drawings in the following description The drawings show some implementations of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative work.

Fig. 1 is a flow chart of a method for real-time detection of blade cracks in aero-engine operation and maintenance based on YOLOv5 provided by an embodiment of the present invention;

Figure 2 is a schematic diagram of the overlapping degree of a real-time detection method for blade cracks in aero-engine operation and maintenance based on YOLOv5;

Figure 3 is a P-R curve of a YOLOv5-based real-time detection method for blade cracks in aero-engine operation and maintenance at a mAP value of 0.625.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. the embodiment. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

At present, the target detection algorithm is divided into single-stage and two-stage algorithms. The single-stage algorithm performs positioning prediction after the image information is input, and directly outputs the result. The detection speed is fast but there are many anchor boxes. The selection of anchor boxes needs to be optimized. YOLO It is a representative algorithm of single-stage target detection, which outputs the position and category confidence of the target frame at one time; the two-stage algorithm classifies and regresses the anchor frame, performs multiple detections and updates, and is slower than the single-stage algorithm, and the structure is not flexible enough, but The degree of network integration is high. Based on this, a method and device for real-time detection of blade cracks in aero-engine operation and maintenance based on YOLOv5 provided by the embodiment of the present invention can improve the detection speed of blade cracks in the prior art and improve the performance of blade crack detection algorithms. Network flexibility.

In order to facilitate the understanding of this embodiment, a YOLOv5-based method for real-time detection of blade cracks in aero-engine operation and maintenance disclosed in the embodiment of the present invention is firstly introduced in detail.

Embodiment one:

As shown in Figure 1, an embodiment of the present invention provides a YOLOv5-based method for real-time detection of blade cracks in the operation and maintenance of aero-engines, which is applied to a host computer and specifically includes the following steps:

Sending a first command to obtain an image of the blades inside the engine;

Input the training data set to the pre-set YOLOv5 network model for training, use the verification data set to conduct a preliminary evaluation of the model effect derived from the training to adjust the model, use the trained weight file and test data set for testing, and obtain the mAP value And the P-R curve for the final evaluation of the model;

Further, the Labelimg image annotation tool can be used to expand the data set by turning the image horizontally, vertically, and horizontally and vertically;

The present invention obtains 300 pictures and preprocesses them to obtain 1500 pictures. Specifically, the division of data sets is shown in Table 1:

Table 1: Dataset distribution table

The output terminal marks the crack information and outputs the confidence level to obtain the precision rate and recall rate. The output includes GIOU loss function and NMS non-maximum suppression, further

Preferably, the pooling operation is performed using the following formula:

S _H — the height of the matrix;

S _W — the width of the matrix;

h—the height of the picture;

w—the width of the image;

p—fill quantity;

f—filter size;

s—step size.

Preferably, the accuracy rate is obtained using the following formula

n—the total number of identified pictures;

TP—the number of correctly identified pictures;

FP—Number of incorrectly identified images.

Use the following formula to obtain the recall rate:

m—the total number of pictures to be identified;

FN—Number of pictures that were targeted but not recognized by the system.

Preferably, the following steps are used to obtain TP and FP:

In conjunction with Fig. 2, the confidence degree and the degree of overlap IOU are obtained, and the degree of overlap IOU is obtained by the following formula:

B _p —prediction box;

B _gt — ground truth box;

Obtain the TP and the FP based on the overlap IOU:

Preferably, the following formula is used to obtain the mAP value:

P—precision;

r—recall rate;

The mAP value is the average of the AP values of the categorical features.

Further, the classification contents of TP, FP, NP, and FN are shown in the following table:

Table 2: TP, FP, NP, FN classification content

It should be noted that in the embodiment provided by the present invention, the mAP value under the weight file is 0.625, the specific test results are as follows, and the P-R curve is shown in Figure 3:

Table 3: Test results of mAP at 0.625:

Embodiment two:

Embodiment 2 of the present invention provides a real-time detection device for blade cracks in aero-engine operation and maintenance based on YOLOv5, including:

Image acquisition instruction sending module: used to send the first instruction to acquire the image of the internal blade of the engine;

Training and testing module: It is used to input the training data set to the pre-set YOLOv5 network model for training, use the verification data set to conduct a preliminary evaluation of the model effect derived from training to adjust the model, and use the trained weight file and test data set for training. Test, and obtain mAP value and P-R curve for final evaluation of the model;

Output module: used for real-time acquisition of blade images inside the engine, real-time detection of engine blades using the weight file, and output of detection results.

Relative steps, numerical expressions and numerical values of components and steps set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

The implementation principles and technical effects of the device provided by the embodiment of the present invention are the same as those of the foregoing method embodiment. For brief description, for the parts not mentioned in the device embodiment, reference may be made to the corresponding content in the foregoing method embodiment.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of code that includes one or more Executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In addition, in the description of the embodiments of the present invention, unless otherwise specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection , or integrally connected; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention in specific situations.

In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, or in a specific orientation. construction and operation, therefore, should not be construed as limiting the invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and should not be construed as indicating or implying relative importance.

Finally, it should be noted that: the above-described embodiments are only specific implementations of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting them, and the scope of protection of the present invention is not limited thereto, although referring to the foregoing The embodiment has described the present invention in detail, and those skilled in the art should understand that any person familiar with the technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention Changes can be easily thought of, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of the present invention within the scope of protection. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

A method for real-time detection of blade cracks in aero-engine operation and maintenance based on YOLOv5 is characterized in that it is applied to a host computer, and specifically includes the following steps:

Sending a first command to obtain an image of the blades inside the engine;

Preprocessing the internal blade image of the engine to obtain a test data set, a training data set and a verification data set;

Input the training data set to the pre-set YOLOv5 network model for training, use the verification data set to conduct a preliminary evaluation of the model effect derived from the training to adjust the model, use the trained weight file and test data set to test, and obtain mAP value and P-R curve for final evaluation of the model;

Obtain the blade image inside the engine in real time and use the weight file to detect the engine blade in real time and output the detection result.
The method according to claim 1, wherein the step of preprocessing the internal blade image of the engine to obtain a test data set, a training data set and a verification data set comprises:

Crack marking, zooming and flipping are performed on the internal blade image of the engine to expand the data set to obtain the test data set, training data set and verification data set.
The method according to claim 1, wherein the preset YOLOv5 network model includes an input terminal, a Backbone backbone network, a Neck neck network and an output terminal;

The input terminal includes a data enhancement module and an anchor frame selection module, which are used to splicing the input data in a manner of scaling, clipping, and random arrangement, and the anchor frame selection module is used for the internal blade image of the engine The marked crack anchor box, calculate and update the size of the crack anchor box;

Described Backbone backbone network comprises Focus structure, CBL structure, CSP structure and SPP module;

The Focus structure obtains the crack picture at the input end to perform a slice parallel convolution operation, and the CBL structure extracts the feature information of the crack picture through the parallel convolution operation of the Focus structure slice;

The SPP module is used to equally divide the feature map of the crack image feature and perform a pooling operation;

The Neck network includes an FPN structure and a PAN structure, the FPN structure and the CBL structure increase the size of the crack picture feature map, and the PAN structure and the FPN structure perform feature fusion to reduce the crack picture features Figure size;

The output terminal marks the crack information and outputs the confidence level to obtain the precision rate and recall rate.
The method according to claim 3, wherein the pooling operation is performed using the following formula:

S H — the height of the matrix;

S W — the width of the matrix;

h—the height of the image;

w—the width of the image;

p—fill quantity;

f—filter size;

s—step size.
The method according to claim 3, characterized in that, the following formula is used to obtain the accuracy

n—the total number of identified pictures;

TP—the number of correctly identified pictures;

FP—Number of incorrectly identified images.

Use the following formula to obtain the recall rate:

m—the total number of pictures to be identified;

FN—Number of pictures that were targeted but not recognized by the system.
The method according to claim 5, wherein the following steps are used to obtain TP and FP:

Obtain the confidence degree and the degree of overlap IOU, and the degree of overlap IOU is obtained by the following formula:

B p —prediction box;

B gt — ground truth box;

Obtain the TP and the FP based on the overlap IOU:

If the recognized area is larger than the IOU threshold, it is judged as TP, and if it is smaller than the IOU threshold, it is judged as FP.
method according to claim 6, is characterized in that, adopts following formula to obtain mAP value:

P—precision;

r—recall rate;

The mAP value is the average of the AP values of the categorical features.
A YOLOv5-based real-time detection device for blade cracks in aero-engine operation and maintenance, characterized in that it includes:

Image acquisition instruction sending module: send the first instruction to acquire the internal blade image of the engine;

Sample collection module: used to preprocess the internal blade image of the engine to obtain a test data set, a training data set and a verification data set;

Training and testing module: used to input the training data set to the pre-set YOLOv5 network model for training, use the verification data set to conduct a preliminary evaluation of the model effect derived from training to adjust the model, and use the trained weight file and test data The set is tested, and the mAP value and P-R curve are obtained for the final evaluation of the model;

Output module: for real-time acquisition of blade images inside the engine, real-time detection of engine blades using the weight file, and output of detection results.