CN118365973B - Hybrid line status assessment method and system based on multi-feature information fusion - Google Patents

Abstract

The present application relates to the field of power systems, and in particular to a method and system for evaluating the state of a mixed-frame line based on multi-feature information fusion, the method comprising: acquiring a first image of the mixed-frame line, extracting features from the first image, and obtaining geometric feature data, texture feature data, and connection feature data; acquiring a second image of the mixed-frame line, extracting features from the second image, and obtaining line tilt feature data; acquiring a third image of the mixed-frame line, extracting features from the third image, and obtaining thermal distribution feature data; fusing the obtained feature data to obtain a multi-feature fusion vector; inputting the multi-feature information fusion vector into the mixed-frame line state evaluation model of the multi-feature information fusion to perform mixed-frame line state evaluation and obtain a mixed-frame line state evaluation result. Through the above method, the reliability and accuracy of the mixed-frame line state evaluation result are improved.

Description

Hybrid line status assessment method and system based on multi-feature information fusion

Technical Field

The present application relates to the field of power systems, and in particular to a method and system for evaluating the state of a hybrid line based on multi-feature information fusion.

Background Art

With the rapid development of smart grid technology, information technology and intelligent operation and maintenance technology, the status data of power transmission and transformation equipment has gradually shown big data characteristics such as large volume, multiple types and rapid growth. How to achieve efficient and comprehensive utilization of multi-source heterogeneous data, in-depth mining of the relationship between various types of data and equipment status, and refined status evaluation of mixed-frame lines under different operating conditions have become urgent problems to be solved. Therefore, how to improve the reliability and accuracy of mixed-frame line status evaluation needs to be solved urgently.

Summary of the invention

The main purpose of this application is to provide a mixed-frame line status assessment method and system based on multi-feature information fusion, aiming to improve the accuracy of mixed-frame line status assessment.

In order to achieve the above-mentioned invention object, the present application proposes a hybrid line status assessment method based on multi-feature information fusion, comprising:

A hybrid line status assessment method based on multi-feature information fusion, the method comprising:

Collecting a first image of the mixed rack line, performing feature extraction on the first image to obtain geometric feature data, texture feature data and connection feature data;

Collecting a second image of the mixed-frame line, performing feature extraction on the second image, and obtaining line tilt feature data;

Collecting a third image of the mixed rack circuit, performing feature extraction on the third image, and obtaining thermal distribution feature data;

Fusing the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data and the heat distribution feature data to obtain a multi-feature fusion vector;

A mixed-frame line state assessment model based on multi-feature information fusion is constructed, and the multi-feature information fusion vector is input into the mixed-frame line state assessment model based on multi-feature information fusion to perform mixed-frame line state assessment and obtain a mixed-frame line state assessment result.

Further, the constructing of a mixed-frame line state assessment model based on multi-feature information fusion, inputting the multi-feature information fusion vector into the mixed-frame line state assessment model based on multi-feature information fusion to perform mixed-frame line state assessment, and obtaining a mixed-frame line state assessment result specifically includes:

Obtain the historical multi-feature information fusion vector, and mark each fusion feature vector as three states: A/T/E, which represent the normal state, warning state, and fault state respectively;

Dividing the historical multi-feature information fusion vector set into a training set and a test set, and constructing a decision tree based on the Gini index and information gain rate of the probability distribution of the training set;

Inputting the test set into the decision tree and calculating the prediction accuracy;

When the prediction accuracy is lower than a preset threshold, the decision tree is pruned to generate a hybrid line status assessment model integrating multiple feature information;

Obtain the current multi-feature information fusion vector, input the current multi-feature information fusion vector into the mixed line status assessment model for evaluation, and classify the status category according to the trained rules. The mixed line status assessment model outputs the current status of the mixed line: normal/warning/fault based on the feature data concentratedly reflected by the current multi-feature information fusion vector.

Furthermore, the extracting features from the first image to obtain geometric feature data specifically includes:

Performing denoising processing on the first image using a Gaussian filter to obtain a fourth image;

Binarizing the fourth image to obtain a binary image corresponding to the fourth image;

Performing edge extraction on the binary image to obtain edge information in the binary image, wherein the edge information is a set of coordinate points of the edge of the binary image;

Geometric feature data is acquired according to the edge information.

Furthermore, the extracting features of the first image to obtain texture feature data specifically includes:

Dividing the first image into a plurality of sub-images, obtaining the gradient direction and gradient magnitude corresponding to all pixels in each sub-image, constructing a gradient histogram corresponding to each sub-image based on the gradient direction and the gradient magnitude, and normalizing the gradient histogram to obtain a normalized gradient histogram;

The normalized gradient histograms corresponding to each sub-image are connected to obtain the HOG texture feature vector of the first image, and the HOG texture feature vector is used as texture feature data.

Furthermore, the collecting of the second image of the mixed-frame line and extracting features from the second image to obtain line tilt feature data specifically includes:

performing image correction processing on the second image to obtain a fifth image;

Extracting line data points from the fifth image to obtain a line data point set;

Performing least squares straight line fitting processing on the pole data point set to obtain a line fitting straight line;

The line inclination characteristic data is obtained according to the line fitting straight line.

Furthermore, the collecting of the third image of the mixed rack circuit and extracting features from the third image to obtain thermal distribution feature data specifically includes:

The mixed rack line to be evaluated is photographed using an infrared thermal imaging instrument to obtain a third image;

Acquire first temperature values corresponding to all pixels in the third image; and generate a temperature matrix of the third image based on the first temperature values;

Abnormal temperature points are extracted from the temperature matrix to obtain abnormal temperature points, and thermal distribution feature data of the third image is determined based on the abnormal temperature points.

Furthermore, the fusing of the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data and the thermal distribution feature data to obtain a multi-feature fusion vector specifically includes:

Based on a convolutional neural network, the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data, and the thermal distribution feature data are respectively converted into a geometric feature vector, a texture feature vector, a connection feature vector, a line tilt feature vector, and a thermal distribution feature vector;

assigning weights to the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector, and the heat distribution feature vector respectively to obtain corresponding weight coefficients;

Multiplying the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector and the heat distribution feature vector by their corresponding weight coefficients respectively to obtain corresponding weighted feature vectors;

The weighted feature vectors corresponding to the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector, and the heat distribution feature vector are linearly added to obtain a multi-feature fusion vector.

A hybrid line status assessment system based on multi-feature information fusion, the system comprising:

A first image feature extraction module, used for collecting a first image of the mixed rack line, performing feature extraction on the first image, and obtaining geometric feature data, texture feature data, and connection feature data;

A second image feature extraction module, used for collecting a second image of the mixed-frame line, performing feature extraction on the second image, and obtaining line tilt feature data;

A third image feature extraction module collects a third image of the mixed rack circuit, performs feature extraction on the third image, and obtains thermal distribution feature data;

A multi-feature data fusion vector module, used for fusing the geometric feature data, the texture feature data, the connection feature data, the line inclination feature data and the thermal distribution feature data to obtain a multi-feature fusion vector;

The mixed line state assessment module is used to construct a mixed line state assessment model of multi-feature information fusion, input the multi-feature information fusion vector into the mixed line state assessment model of multi-feature information fusion to perform mixed line state assessment, and obtain a mixed line state assessment result.

The present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above-mentioned mixed-frame line status assessment methods based on multi-feature information fusion are implemented.

The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the above-mentioned mixed-frame line status assessment methods based on multi-feature information fusion are implemented.

The present application proposes a method and system for evaluating the state of a mixed rack line based on multi-feature information fusion, wherein the method acquires a first image of the mixed rack line, extracts features from the first image, and obtains geometric feature data, texture feature data, and connection feature data; acquires a second image of the mixed rack line, extracts features from the second image, and obtains line tilt feature data; acquires a third image of the mixed rack line, extracts features from the third image, and obtains thermal distribution feature data; fuses the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data, and the thermal distribution feature data to obtain a multi-feature fusion vector; constructs a mixed rack line state evaluation model with multi-feature information fusion, and inputs the multi-feature information fusion vector into the mixed rack line state evaluation model with multi-feature information fusion to perform mixed rack line state evaluation and obtain a mixed rack line state evaluation result. Through the above method, the reliability and accuracy of the mixed rack line state evaluation result are improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of an embodiment of a hybrid line status assessment method based on multi-feature information fusion according to the present application;

FIG2 is a flow chart of an embodiment of a hybrid line status assessment method based on multi-feature information fusion according to the present application;

FIG3 is a flow chart of an embodiment of a hybrid line status assessment method based on multi-feature information fusion according to the present application;

FIG4 is a flow chart of an embodiment of a hybrid line status assessment method based on multi-feature information fusion according to the present application;

FIG5 is a flow chart of an embodiment of a hybrid line status assessment method based on multi-feature information fusion according to the present application;

FIG6 is a flow chart of an embodiment of a hybrid line status assessment method based on multi-feature information fusion according to the present application;

FIG7 is a flow chart of an embodiment of a hybrid line status assessment method based on multi-feature information fusion according to the present application;

FIG8 is a schematic structural diagram of an embodiment of a hybrid line status assessment system for multi-feature information fusion of the present application;

FIG. 9 is a schematic block diagram of the structure of a computer device according to an embodiment of the present application.

The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

1 , an embodiment of the present application provides a hybrid line status assessment method based on multi-feature information fusion, including steps S10-S50. The steps of the hybrid line status assessment method based on multi-feature information fusion are described in detail as follows.

S10, collecting a first image of the mixed-frame line, extracting features from the first image to obtain geometric feature data, texture feature data, and connection feature data;

In this embodiment, feature extraction is performed on the first image to obtain geometric feature data, which specifically includes: denoising the first image using a Gaussian filter to obtain a fourth image; binarizing the fourth image to obtain a binary image corresponding to the fourth image; edge extraction is performed on the binary image to obtain edge information in the binary image, wherein the edge information is a set of coordinate points of the edge of the binary image; and geometric feature data is obtained according to the edge information. Feature extraction is performed on the first image to obtain texture feature data, which specifically includes: dividing the first image into multiple sub-images, obtaining the gradient direction and gradient amplitude corresponding to all pixels in each sub-image, constructing a gradient histogram corresponding to each sub-image based on the gradient direction and the gradient amplitude, normalizing the gradient histogram to obtain a normalized gradient histogram; connecting the normalized gradient histograms corresponding to each sub-image to obtain the HOG texture feature vector of the first image, and using the HOG texture feature vector as texture feature data.

S20, collecting a second image of the mixed-frame line, performing feature extraction on the second image, and obtaining line tilt feature data;

In this embodiment, image correction processing is performed on the second image to obtain a fifth image; line data points are extracted from the fifth image to obtain a line data point set; least squares straight line fitting processing is performed on the pole data point set to obtain a line fitting straight line; and line inclination feature data is obtained based on the line fitting straight line.

S30, collecting a third image of the mixed rack circuit, performing feature extraction on the third image, and obtaining thermal distribution feature data;

In this embodiment, the mixed-rack line to be evaluated is photographed by an infrared thermal imaging instrument to obtain a third image; the first temperature values corresponding to all pixels in the third image are obtained; based on the first temperature values, a temperature matrix of the third image is generated; abnormal temperature points are extracted from the temperature matrix to obtain abnormal temperature points, and based on the abnormal temperature points, thermal distribution characteristic data of the third image is determined.

S40, fusing the geometric feature data, the texture feature data, the connection feature data, the line inclination feature data and the thermal distribution feature data to obtain a multi-feature fusion vector;

In this embodiment, based on a convolutional neural network, the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data and the thermal distribution feature data are respectively converted into geometric feature vectors, texture feature vectors, connection feature vectors, line tilt feature vectors and thermal distribution feature vectors; weights are respectively assigned to the geometric feature vectors, the texture feature vectors, the connection feature vectors, the line tilt feature vectors and the thermal distribution feature vectors to obtain corresponding weight coefficients; the geometric feature vectors, the texture feature vectors, the connection feature vectors, the line tilt feature vectors and the thermal distribution feature vectors are respectively multiplied by their corresponding weight coefficients to obtain corresponding weighted feature vectors; the weighted feature vectors corresponding to the geometric feature vectors, the texture feature vectors, the connection feature vectors, the line tilt feature vectors and the thermal distribution feature vectors are linearly added to obtain a multi-feature fusion vector.

S50: construct a mixed-frame line state assessment model based on multi-feature information fusion, input the multi-feature information fusion vector into the mixed-frame line state assessment model based on multi-feature information fusion to perform mixed-frame line state assessment, and obtain a mixed-frame line state assessment result.

In this embodiment, a historical multi-feature information fusion vector is obtained, and each fused feature vector is labeled as three states A/T/E, which represent normal state, warning state and fault state respectively; the historical multi-feature information fusion vector set is divided into a training set and a test set, and a decision tree is constructed based on the Gini index of the probability distribution of the training set and the information gain rate; the test set is input into the decision tree and the prediction accuracy is calculated; when the prediction accuracy is lower than a preset threshold, the decision tree is pruned to generate a mixed line state assessment model of multi-feature information fusion; the current multi-feature information fusion vector is obtained, and the current multi-feature information fusion vector is input into the mixed line state assessment model for assessment, and the state category is classified according to the training rules. The mixed line state assessment model outputs the current state of the mixed line: normal/warning/fault based on each feature data reflected in the current multi-feature information fusion vector set.

In one embodiment, the construction of a mixed-frame line state assessment model based on multi-feature information fusion, inputting the multi-feature information fusion vector into the mixed-frame line state assessment model based on multi-feature information fusion to perform mixed-frame line state assessment, and obtaining a mixed-frame line state assessment result, specifically includes: obtaining historical multi-feature information fusion vectors, and marking each fusion feature vector as three states A/T/E, representing a normal state, a warning state, and a fault state, respectively; dividing the historical multi-feature information fusion vector set into a training set and a test set, and constructing a decision tree based on the Gini index of the probability distribution of the training set and the information gain rate. ; Input the test set into the decision tree and calculate the prediction accuracy; Input the test set into the decision tree and calculate the prediction accuracy; When the prediction accuracy is lower than a preset threshold, prune the decision tree to generate a mixed line status assessment model with multi-feature information fusion; Obtain the current multi-feature information fusion vector, input the current multi-feature information fusion vector into the mixed line status assessment model for assessment, and classify the status category according to the training rules. The mixed line status assessment model outputs the current status of the mixed line: normal/warning/fault based on the feature data reflected in the current multi-feature information fusion vector.

In this embodiment, specifically, the decision tree model is generated with the full set of historical multi-feature information fusion vectors as the root node, the feature attributes selected in the historical multi-feature information fusion vectors as the internal nodes, and the classification results as the leaf nodes. The feature attribute with the largest information gain is selected as the selected feature attribute, and the information gain is determined according to the following formula: (S) =－ , where E(S) is the information entropy of the known historical multi-feature information fusion vector S; γ _m is the proportion of the mth class of samples in the sample data set S; L is the number of categories in the sample data set S, and each fused feature vector is labeled as three categories A/T/E, representing normal state, warning state and fault state respectively. These labels are based on known operating data and fault cases to ensure the accuracy of the labels; assuming that there is a test set containing 100 samples, these samples are predicted by a decision tree model, and the prediction results of 85 samples are correct, and the prediction results of 15 samples are wrong, then the prediction accuracy of the decision tree model on this test set is 85%, which is higher than the preset threshold of 80%. Starting from the root node, split according to the data features until all leaf nodes meet the conditions for stopping splitting. If it is lower than the preset threshold, stop splitting, generate a hybrid line state evaluation model with multi-feature information fusion, obtain the current multi-feature information fusion vector, input the current multi-feature information fusion vector into the hybrid line state evaluation model for evaluation, and classify the state categories according to the training rules. The hybrid line state evaluation model outputs the current state of the hybrid line: normal/warning/fault based on the feature data reflected in the current multi-feature information fusion vector.

In one embodiment, the feature extraction of the first image to obtain geometric feature data specifically includes: denoising the first image using a Gaussian filter to obtain a fourth image; binarizing the fourth image to obtain a binarized image corresponding to the fourth image; performing edge extraction on the binarized image to obtain edge information in the binarized image, wherein the edge information is a set of coordinate points of the edge of the binarized image; and obtaining geometric feature data based on the edge information.

In this embodiment, specifically, the fourth image is converted into a grayscale image, so that each pixel has only one grayscale value, and the grayscale image is binarized using a global threshold method, and the pixel threshold is set to 128, and the pixels greater than 128 are set to 255, which are represented as white, and the pixels less than or equal to 128 are set to 0, which are represented as black. A black and white binary image is obtained, in which the white part represents the object and the black part represents the background. The black and white binary image is processed using the Canny edge detection algorithm to extract the boundary between the object and the background in the image. The Canny algorithm detects the strong edge in the image, displays it in a new image, and obtains a white edge line on a black background. Find the outline of the edge and calculate the length of the edge. By traversing all the outlines and using the cv2.arcLength() function to calculate the length of each outline, the total length of the edge in the fourth image is finally obtained as the feature data output.

In one embodiment, the feature extraction of the first image to obtain texture feature data specifically includes: dividing the first image into multiple sub-images, obtaining the gradient direction and gradient amplitude corresponding to all pixels in each sub-image, constructing a gradient histogram corresponding to each sub-image based on the gradient direction and the gradient amplitude, normalizing the gradient histogram to obtain a normalized gradient histogram; connecting the normalized gradient histograms corresponding to each sub-image to obtain the HOG texture feature vector of the first image, and using the HOG texture feature vector as the texture feature data.

In this embodiment, specifically, in one embodiment, the first image is converted into a grayscale image because the HOG method works better for grayscale images. Then, the first image can be preprocessed, such as resizing, removing noise, etc. Calculate the gradient: Calculate the gradient of the preprocessed first image, usually using methods such as the Sobel operator to calculate the gradient of the image in the horizontal and vertical directions. Calculate the gradient histogram: Divide the first image into 8*8 cells, count the direction and size information of the gradient in each cell, and form a histogram. In this embodiment, the gradient direction is divided into 9 direction intervals, and then the cumulative value of the gradient in each interval is counted. Block normalization: Divide the image into larger areas, each block contains multiple cells. In each block, the histograms of all cells are normalized, and the feature vector is spliced: The normalized histograms in all blocks are connected into a feature vector, which is the HOG feature.

Specifically, in another embodiment, feature extraction is performed on the first image to obtain texture feature data, which also includes: feature extraction is performed on the first image to obtain texture feature data; specifically, the first image is divided into multiple sub-images, and a first LBP value corresponding to all pixels in each sub-image is obtained, and a histogram corresponding to each sub-image is calculated based on the first LBP value, and the histogram is normalized to obtain a normalized histogram; the normalized histogram corresponding to each sub-image is connected to obtain an LBP texture feature vector of the second image, and the LBP texture feature vector is used as texture feature data. Specifically, by selecting a first pixel point in each sub-image, based on the first pixel point, obtaining the first grayscale value of 8 pixels adjacent to the first pixel point, and respectively comparing the first grayscale value with the second grayscale value corresponding to the first pixel point, if the first grayscale value is greater than the second grayscale value, the position of the pixel point corresponding to the first grayscale value is marked as 1, otherwise it is 0; based on the above operation, in this way, the binary numbers corresponding to the 8 pixels in the 3*3 neighborhood except the central pixel point, and based on the binary numbers corresponding to the 8 pixels, the LBP value of the central pixel point of the window, that is, the first LBP value corresponding to the first pixel point, is obtained; repeat the above operation until the first LBP value corresponding to all pixels in each sub-image is obtained. In one embodiment, the histogram corresponding to each sub-image is calculated based on the first LBP value. Specifically, based on the first LBP values corresponding to all pixels in each sub-image, the probability of each first LBP value appearing is counted, and based on the probability, the histogram corresponding to each sub-image is generated.

In one embodiment, the method of acquiring a second image of the mixed-frame line and extracting features from the second image to obtain line inclination feature data specifically includes: performing image correction processing on the second image to obtain a fifth image; extracting line data points from the fifth image to obtain a line data point set; performing least squares straight line fitting processing on the pole data point set to obtain a line fitting straight line; and obtaining line inclination feature data based on the line fitting straight line.

In this embodiment, the second image is converted into a grayscale image, and then the Canny edge detection algorithm is applied to detect the contour of the line to obtain the edge information of the line in the second image. After edge detection, the contour of the line in the second image is obtained, and the contour is traversed and the pixel points on the contour are extracted, which are expressed as [(x1, y1), (x2, y2), ..., (xn, yn)]. The least squares fitting method is used to perform straight line fitting on the extracted line data points to obtain the fitted straight line equation of the line, which is expressed as: y = mx + b. According to the fitted straight line equation, the inclination angle m of the line, as well as the intercept b and other information are calculated to describe the inclination of the line.

In one embodiment, the method of acquiring a third image of the mixed-rack circuit, performing feature extraction on the third image, and obtaining thermal distribution feature data specifically includes: photographing the mixed-rack circuit to be evaluated based on an infrared thermal imaging instrument to obtain a third image; obtaining a first temperature value corresponding to all pixels in the third image; generating a temperature matrix of the third image based on the first temperature value; extracting abnormal temperature points from the temperature matrix to obtain abnormal temperature points, and determining the thermal distribution feature data of the third image based on the abnormal temperature points.

In this embodiment, specifically, the mixed-frame circuit to be evaluated is photographed based on an infrared thermal imaging instrument to obtain a third image of 320x240 pixels. According to the parameters and calibration information of the thermal imaging instrument, the pixel values of the third image are converted into temperature values, the temperature values corresponding to each pixel are extracted, and a 320*240 matrix is created, and all elements are initialized to 0. The third image is filled with the temperature values corresponding to the extracted pixels into the corresponding positions of the matrix according to the pixel order of the image. More specifically, assuming that we have obtained the following partial temperature values, the temperature value corresponding to the first pixel point (0,0) in the image is 25°C, the temperature value corresponding to the second pixel point (0,1) in the image is 26°C, the temperature value corresponding to the third pixel point (0,2) in the image is 27°C, and so on... Fill these temperature values into the first row, second row, third row, and so on, until the entire matrix is filled. Finally, a 320x240 temperature matrix is obtained, in which each element represents the temperature value of the corresponding pixel point in the image. Traverse each pixel point in the temperature matrix. For each pixel point, check whether its temperature exceeds the set abnormal range. In this embodiment, a temperature range of less than 30°C or greater than 40°C is set. If the temperature exceeds the abnormal range, the pixel point is marked as an abnormal temperature point, and the abnormal temperature points are obtained by traversing until all pixel points are checked. Cluster analysis is performed on the abnormal temperature points, such as K-means clustering, to divide the abnormal temperature points into different clusters, each cluster representing the thermal distribution of a line or part of the line. The shape, size and position of each cluster are further analyzed to determine the thermal distribution feature data of the third image.

In one embodiment, the fusing of the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data and the thermal distribution feature data to obtain a multi-feature fusion vector specifically includes: based on a convolutional neural network, converting the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data and the thermal distribution feature data into a geometric feature vector, a texture feature vector, a connection feature vector, a line tilt feature vector and a thermal distribution feature vector, respectively; assigning weights to the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector and the thermal distribution feature vector, respectively, to obtain corresponding weight coefficients; multiplying the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector and the thermal distribution feature vector with their corresponding weight coefficients, respectively, to obtain corresponding weighted feature vectors; linearly adding the weighted feature vectors corresponding to the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector and the thermal distribution feature vector, to obtain a multi-feature fusion vector.

In this embodiment, specifically, there are two feature vectors A and B, which represent two different features of geometric feature data and texture feature vectors respectively. Feature vector A is represented as [0.2, 0.3, 0.5], feature vector B is represented as [0.1, 0.4, 0.6], and a weight is assigned to each feature vector. Assuming that the weights are 0.7 and 0.3 respectively, using the weighted average method, we perform feature fusion in the following steps: multiply feature vector A by weight 0.7, [0.2*0.7, 0.3*0.7, 0.5*0.7] = [0.14, 0.21, 0.35], multiply feature vector B by Weight 0.3: [0.1*0.3, 0.4*0.3, 0.6*0.3]=[0.03, 0.12, 0.18], add the two weighted results to get the fused feature vector: [0.14+0.03, 0.21+0.12, 0.35+0.18]=[0.17, 0.33, 0.53] Finally, we get a fused feature vector [0.17, 0.33, 0.53], thus getting a multi-feature fusion vector.

8 , the present application provides a hybrid line status assessment system based on multi-feature information fusion, the system comprising:

A first image feature extraction module 10 is used to collect a first image of the hybrid line, perform feature extraction on the first image, and obtain geometric feature data, texture feature data, and connection feature data;

A second image feature extraction module 20 is used to collect a second image of the mixed-frame line, perform feature extraction on the second image, and obtain line tilt feature data;

A third image feature extraction module 30 collects a third image of the mixed rack circuit, performs feature extraction on the third image, and obtains thermal distribution feature data;

A multi-feature data fusion vector module 40 is used to fuse the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data and the heat distribution feature data to obtain a multi-feature fusion vector;

The mixed line state assessment module 50 is used to construct a mixed line state assessment model of multi-feature information fusion, input the multi-feature information fusion vector into the mixed line state assessment model of multi-feature information fusion to perform mixed line state assessment, and obtain a mixed line state assessment result.

As described above, it can be understood that the various components of the mixed-frame line state assessment system based on multi-feature information fusion proposed in the present application can realize the functions of any one of the mixed-frame line state assessment methods based on multi-feature information fusion described above.

In one embodiment, the first image feature extraction module 10 further comprises executing:

Performing denoising processing on the first image using a Gaussian filter to obtain a fourth image;

Binarizing the fourth image to obtain a binary image corresponding to the fourth image;

Performing edge extraction on the binary image to obtain edge information in the binary image, wherein the edge information is a set of coordinate points of the edge of the binary image;

Geometric feature data is acquired according to the edge information.

In one embodiment, the first image feature extraction module 10 further comprises executing:

Dividing the first image into a plurality of sub-images, obtaining the gradient direction and gradient magnitude corresponding to all pixels in each sub-image, constructing a gradient histogram corresponding to each sub-image based on the gradient direction and the gradient magnitude, and normalizing the gradient histogram to obtain a normalized gradient histogram;

The normalized gradient histograms corresponding to each sub-image are connected to obtain the HOG texture feature vector of the first image, and the HOG texture feature vector is used as texture feature data.

In one embodiment, the second image feature extraction module 20 further comprises executing:

performing image correction processing on the second image to obtain a fifth image;

Extracting line data points from the fifth image to obtain a line data point set;

Performing least squares straight line fitting processing on the pole data point set to obtain a line fitting straight line;

The line inclination characteristic data is obtained according to the line fitting straight line.

In one embodiment, the third image feature extraction module 30 further includes executing:

The mixed rack line to be evaluated is photographed using an infrared thermal imaging instrument to obtain a third image;

Acquire first temperature values corresponding to all pixels in the third image; and generate a temperature matrix of the third image based on the first temperature values;

Abnormal temperature points are extracted from the temperature matrix to obtain abnormal temperature points, and thermal distribution feature data of the third image is determined based on the abnormal temperature points.

In one embodiment, the multi-feature data fusion vector module 40 further includes executing:

Based on a convolutional neural network, the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data, and the thermal distribution feature data are respectively converted into a geometric feature vector, a texture feature vector, a connection feature vector, a line tilt feature vector, and a thermal distribution feature vector;

assigning weights to the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector, and the heat distribution feature vector respectively to obtain corresponding weight coefficients;

Multiplying the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector and the heat distribution feature vector by their corresponding weight coefficients respectively to obtain corresponding weighted feature vectors;

The weighted feature vectors corresponding to the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector, and the heat distribution feature vector are linearly added to obtain a multi-feature fusion vector.

In one embodiment, the hybrid line status evaluation module 50 further includes executing:

Obtain the historical multi-feature information fusion vector, and mark each fusion feature vector as three states: A/T/E, which represent the normal state, warning state, and fault state respectively;

Dividing the historical multi-feature information fusion vector set into a training set and a test set, and constructing a decision tree based on the Gini index and information gain rate of the probability distribution of the training set;

Inputting the test set into the decision tree and calculating the prediction accuracy;

When the prediction accuracy is lower than a preset threshold, the decision tree is pruned to generate a hybrid line status assessment model integrating multiple feature information;

Obtain the current multi-feature information fusion vector, input the current multi-feature information fusion vector into the mixed line status assessment model for evaluation, and classify the status category according to the trained rules. The mixed line status assessment model outputs the current status of the mixed line: normal/warning/fault based on the feature data concentratedly reflected by the current multi-feature information fusion vector.

Referring to Figure 9, a computer device is also provided in an embodiment of the present application, and the internal structure of the computer device can be shown in Figure 9. The computer device includes a processor, a memory, a network interface, a display device and an input device connected through a system bus. Among them, the network interface of the computer device is used to communicate with an external terminal through a network connection. The display device of the computer device is used to display an interactive page. The input device of the computer device is used to receive user input. The processor designed for the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium. The non-volatile storage medium stores an operating system, a computer program and a database. The database of the computer device is used to store raw data. When the computer program is executed by the processor, a mixed rack line status assessment method based on multi-feature information fusion is implemented.

The processor executes the mixed line state assessment method based on multi-feature information fusion, the method comprising: collecting a first image of the mixed line, extracting features from the first image, obtaining geometric feature data, texture feature data and connection feature data; collecting a second image of the mixed line, extracting features from the second image, obtaining line tilt feature data; collecting a third image of the mixed line, extracting features from the third image, obtaining thermal distribution feature data; fusing the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data and the thermal distribution feature data to obtain a multi-feature fusion vector; constructing a mixed line state assessment model with multi-feature information fusion, inputting the multi-feature information fusion vector into the mixed line state assessment model with multi-feature information fusion to perform mixed line state assessment, and obtaining a mixed line state assessment result. Through the method, the reliability and accuracy of the mixed line state assessment result are improved.

The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by the processor, a mixed line state assessment method based on multi-feature information fusion is implemented, which includes collecting a first image of the mixed line, performing feature extraction on the first image, and obtaining geometric feature data, texture feature data, and connection feature data; collecting a second image of the mixed line, performing feature extraction on the second image, and obtaining line tilt feature data; collecting a third image of the mixed line, performing feature extraction on the third image, and obtaining thermal distribution feature data; fusing the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data, and the thermal distribution feature data to obtain a multi-feature fusion vector; constructing a mixed line state assessment model with multi-feature information fusion, and inputting the multi-feature information fusion vector into the mixed line state assessment model with multi-feature information fusion to perform mixed line state assessment, and obtaining a mixed line state assessment result.

The computer-readable storage medium provides a method for evaluating the state of a mixed rack line based on multi-feature information fusion, the method comprising: collecting a first image of the mixed rack line, extracting features from the first image, and obtaining geometric feature data, texture feature data, and connection feature data; collecting a second image of the mixed rack line, extracting features from the second image, and obtaining line tilt feature data; collecting a third image of the mixed rack line, extracting features from the third image, and obtaining thermal distribution feature data; fusing the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data, and the thermal distribution feature data to obtain a multi-feature fusion vector; constructing a mixed rack line state evaluation model with multi-feature information fusion, inputting the multi-feature information fusion vector into the mixed rack line state evaluation model with multi-feature information fusion to perform mixed rack line state evaluation, and obtaining a mixed rack line state evaluation result. Through the above method, the reliability and accuracy of the mixed rack line state evaluation result are improved.

Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and/or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims (9)

1. A hybrid line status assessment method based on multi-feature information fusion, characterized by comprising: Collecting a first image of the mixed rack line, performing feature extraction on the first image to obtain geometric feature data, texture feature data and connection feature data; Collecting a second image of the mixed-frame line, performing feature extraction on the second image, and obtaining line tilt feature data; Collecting a third image of the mixed rack circuit, performing feature extraction on the third image, and obtaining thermal distribution feature data; Fusing the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data and the heat distribution feature data to obtain a multi-feature fusion vector; Constructing a mixed-frame line state assessment model based on multi-feature information fusion, inputting the multi-feature information fusion vector into the mixed-frame line state assessment model based on multi-feature information fusion to perform mixed-frame line state assessment, and obtaining a mixed-frame line state assessment result; The step of constructing a mixed-frame line state assessment model by fusion of multiple feature information, inputting the multiple feature information fusion vector into the mixed-frame line state assessment model by fusion of multiple feature information to perform mixed-frame line state assessment, and obtaining a mixed-frame line state assessment result specifically includes: Obtain the historical multi-feature information fusion vector, and mark each fusion feature vector as three states: A/T/E, which represent the normal state, warning state, and fault state respectively; Dividing the historical multi-feature information fusion vector set into a training set and a test set, and constructing a decision tree based on the Gini index and information gain rate of the probability distribution of the training set; Inputting the test set into the decision tree and calculating the prediction accuracy; When the prediction accuracy is lower than a preset threshold, the decision tree is pruned to generate a hybrid line status assessment model integrating multiple feature information; Obtain the current multi-feature information fusion vector, input the current multi-feature information fusion vector into the mixed line status assessment model for evaluation, and classify the status category according to the trained rules. The mixed line status assessment model outputs the current status of the mixed line: normal/warning/fault based on the feature data concentratedly reflected by the current multi-feature information fusion vector. 2. A hybrid line status assessment method based on multi-feature information fusion according to claim 1, characterized in that the feature extraction of the first image to obtain geometric feature data specifically includes: Performing denoising processing on the first image using a Gaussian filter to obtain a fourth image; Binarizing the fourth image to obtain a binary image corresponding to the fourth image; Performing edge extraction on the binary image to obtain edge information in the binary image, wherein the edge information is a set of coordinate points of the edge of the binary image; Geometric feature data is acquired according to the edge information. 3. A hybrid line status assessment method based on multi-feature information fusion as claimed in claim 1, characterized in that the feature extraction of the first image to obtain texture feature data specifically includes: Divide the first image into multiple sub-images, and obtain the gradient direction and gradient amplitude corresponding to all pixels in each sub-image; constructing a gradient histogram corresponding to each sub-image based on the gradient direction and the gradient amplitude, and normalizing the gradient histogram to obtain a normalized gradient histogram; The normalized gradient histograms corresponding to each sub-image are connected to obtain the HOG texture feature vector of the first image, and the HOG texture feature vector is used as texture feature data. 4. A hybrid line status assessment method based on multi-feature information fusion according to claim 1, characterized in that said collecting a second image of the hybrid line, extracting features from the second image, and obtaining line tilt feature data specifically comprises: performing image correction processing on the second image to obtain a fifth image; Extracting line data points from the fifth image to obtain a line data point set; Performing least squares straight line fitting processing on the line data point set to obtain a line fitting straight line; The line inclination characteristic data is obtained according to the line fitting straight line. 5. A method for evaluating the state of a hybrid line based on multi-feature information fusion according to claim 1, characterized in that said collecting a third image of the hybrid line, extracting features from the third image, and obtaining thermal distribution feature data specifically comprises: The mixed rack line to be evaluated is photographed using an infrared thermal imaging instrument to obtain a third image; Acquire first temperature values corresponding to all pixels in the third image; generating a temperature matrix of the third image based on the first temperature value; Abnormal temperature points are extracted from the temperature matrix to obtain abnormal temperature points, and thermal distribution feature data of the third image is determined based on the abnormal temperature points. 6. A hybrid line status assessment method based on multi-feature information fusion as claimed in claim 1, characterized in that the step of fusing the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data and the thermal distribution feature data to obtain a multi-feature fusion vector specifically comprises: Based on a convolutional neural network, the geometric feature data, the texture feature data, the connection feature data, the line tilt feature data, and the thermal distribution feature data are respectively converted into a geometric feature vector, a texture feature vector, a connection feature vector, a line tilt feature vector, and a thermal distribution feature vector; assigning weights to the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector, and the heat distribution feature vector respectively to obtain corresponding weight coefficients; Multiplying the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector and the heat distribution feature vector by their corresponding weight coefficients respectively to obtain corresponding weighted feature vectors; The weighted feature vectors corresponding to the geometric feature vector, the texture feature vector, the connection feature vector, the line tilt feature vector, and the heat distribution feature vector are linearly added to obtain a multi-feature fusion vector. 7. A hybrid line status assessment system based on multi-feature information fusion, used to implement the assessment method according to any one of claims 1 to 6, characterized in that the system comprises: A first image feature extraction module, used for collecting a first image of the mixed rack line, performing feature extraction on the first image, and obtaining geometric feature data, texture feature data, and connection feature data; A second image feature extraction module, used for collecting a second image of the mixed-frame line, performing feature extraction on the second image, and obtaining line tilt feature data; A third image feature extraction module collects a third image of the mixed rack circuit, performs feature extraction on the third image, and obtains thermal distribution feature data; A multi-feature data fusion vector module, used for fusing the geometric feature data, the texture feature data, the connection feature data, the line inclination feature data and the thermal distribution feature data to obtain a multi-feature fusion vector; The mixed line state assessment module is used to construct a mixed line state assessment model of multi-feature information fusion, input the multi-feature information fusion vector into the mixed line state assessment model of multi-feature information fusion to perform mixed line state assessment, and obtain a mixed line state assessment result. 8. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the mixed line status assessment method based on multi-feature information fusion according to any one of claims 1 to 6 when executing the computer program. 9. A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the method for evaluating the state of a hybrid line based on multi-feature information fusion according to any one of claims 1 to 6 are implemented.