CN112733824B

CN112733824B - Transformer equipment defect diagnosis method and system based on video image intelligent front end

Info

Publication number: CN112733824B
Application number: CN202110364984.1A
Authority: CN
Inventors: 杨宁; 高飞; 杨洋; 李丽华; 张博文; 韩帅; 贾鹏飞
Original assignee: China Electric Power Research Institute Co Ltd CEPRI
Current assignee: China Electric Power Research Institute Co Ltd CEPRI
Priority date: 2021-04-06
Filing date: 2021-04-06
Publication date: 2022-04-15
Anticipated expiration: 2041-04-06
Also published as: CN112733824A

Abstract

The invention discloses a method and a system for diagnosing defects of power transformation equipment based on an intelligent front end of a video image. The method comprises the following steps: acquiring a video image of the power transformation equipment in real time; and adopting an optimal scheduling scheme, executing a lightweight defect identification model to process the video image, and determining the defects of the power transformation equipment according to a processing result. The method utilizes edge computing power located on the side of the smart front end to process video images in real time. Executing a power transformation equipment defect image diagnosis algorithm by using a video image processing chip integrated at the intelligent front end to generate a power transformation equipment state early warning conclusion or a defect identification conclusion; and the rapid and efficient image processing is realized at the side end, and the rapidness, timeliness and accuracy of the defect diagnosis of the transformer substation equipment are ensured.

Description

Transformer equipment defect diagnosis method and system based on video image intelligent front end

Technical Field

The invention belongs to the technical field of intelligent inspection of power transformation equipment, and particularly relates to a power transformation equipment defect diagnosis method and system based on an intelligent front end of a video image.

Background

At present, the following problems exist in the routing inspection and analysis of power transformation equipment:

firstly, a means for intelligently acquiring a video image of the power transformation equipment is lacked. At the development initial stage that substation equipment patrolled, adopt handheld infrared thermal imager, handheld ultraviolet imager and artifical mode of patrolling usually, promoted equipment defect discovery ability, but increased the daily operating pressure that patrols of fortune inspector, also proposed higher technical requirement to the fortune inspector. Along with the construction of the intelligent substation, the power transformation equipment can adopt video image inspection terminals such as a mobile inspection robot and a video image camera for fixing a machine position, so that the manual inspection pressure of a part of operators is reduced, and the pressure of the operators in the aspects of equipment maintenance, system upgrading, defect review and the like is increased. At present, the video image camera and the mobile inspection robot which are fixed at a machine position are limited by software and hardware technical levels, data acquisition can only be carried out at fixed preset points, the intelligent image acquisition of a self-adaptive angle cannot be realized, and the problems of low cloud diagnosis efficiency and high false detection rate caused by low acquired image quality exist.

Secondly, the current video image inspection terminal lacks real-time intelligent analysis capability. At present, video images of the power transformation equipment are uniformly transmitted to a background server through channels such as optical fibers and the like for analysis and processing, the cost and pressure of communication and data centralized management are increased, and the timeliness and instantaneity of data analysis are insufficient. Especially, when the number of video image inspection terminals arranged in a transformer substation is continuously increased and the imaging quality and resolution of the intelligent front end for video image acquisition are continuously improved, the hardware cost of a communication channel and a background server is increased, and the image identification and processing efficiency is greatly reduced.

Therefore, although a substation/converter station regularly inspects substation equipment by using an inspection robot, a video camera, an infrared camera and other video image inspection terminals, the subsequent process needs to process video images by using computing capacity arranged in the cloud, and the defect diagnosis timeliness is insufficient.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a transformer equipment defect diagnosis method and system based on an intelligent front end of a video image, so as to solve the problem of insufficient timeliness of the existing transformer equipment defect diagnosis.

In a first aspect, the present invention provides a method for diagnosing defects of a power transformation device based on an intelligent front end of a video image, including:

acquiring a video image of the power transformation equipment in real time;

and adopting an optimal scheduling scheme, executing a lightweight defect identification model to process the video image, and determining the defects of the power transformation equipment according to a processing result.

Further, before acquiring the video image of the power transformation device in real time, the method further comprises the following steps:

acquiring a transformer equipment image diagnosis model from a cloud server through an edge side cloud edge cooperative component and a cloud side cloud edge cooperative component, wherein the transformer equipment image diagnosis model is generated after deep learning neural network training and evaluation are completed according to typical video images and typical defects of a transformer substation; wherein the content of the first and second substances,

the edge side cloud edge cooperative component is arranged at the intelligent front end of the video image;

the cloud side and cloud edge cooperative component is arranged on the cloud server;

and performing pseudo-quantization processing and/or pruning processing on the acquired image diagnosis model of the power transformation equipment to obtain a lightweight defect identification model.

Furthermore, a video image processing chip and a central processing unit are arranged at the intelligent front end of the video image;

before the real-time acquisition of the video image of the power transformation equipment, the method further comprises the following steps:

and executing an operator tuning algorithm on the lightweight defect identification model in cooperation with hardware resources of a video image processing chip and hardware resources of a central processing unit to obtain an optimal scheduling scheme corresponding to the lightweight defect identification model, and mapping the optimal scheduling scheme into the central processing unit.

Furthermore, a camera sensing unit is arranged at the intelligent front end of the video image, and is fixedly arranged on the controllable holder and swings along with the controllable holder;

acquiring a video image of the power transformation equipment acquired by the camera sensing unit,

executing a lightweight instance segmentation model to process the video image, and determining a target central value of a target subject;

calculating the offset according to the target central value of the target main body and the picture central value of the image collected by the camera sensing unit;

and according to the offset, controlling the controllable holder through proportional-integral-derivative so that the target main body is positioned in the center of the picture acquired by the camera sensing unit.

Further, when deep learning neural network training and evaluation are completed according to typical video images and typical defects of the transformer substation to generate the image diagnosis model of the transformer equipment,

the power transformation equipment defect image diagnosis model is obtained based on one or more of the following deep learning neural network frameworks: YOLO, fast RCNN, SSD, and Mask RCNN;

and when the model prediction accuracy is greater than or equal to the preset accuracy, outputting the weight matrix and the model parameters of the power transformation equipment defect image diagnosis model.

In a second aspect, the present invention provides a power transformation device defect diagnosis system based on a video image intelligent front end, including:

the intelligent front ends are arranged at different positions in the transformer substation;

each video image intelligent front end comprises an intelligent camera and an intelligent front end board card;

the intelligent camera is used for acquiring a video image of the power transformation equipment;

the intelligent front end plate card is provided with a video image processing chip and a central processing unit;

the intelligent front-end board card is further provided with an intelligent analysis unit, and the intelligent analysis unit is used for coordinating hardware resources of the video image processing chip and hardware resources of the central processing unit by adopting an optimal scheduling scheme, executing the lightweight defect identification model to process the video image, and determining the defects of the power transformation equipment according to the processing result.

Furthermore, the intelligent camera comprises a camera sensing unit, wherein the camera sensing unit is an infrared camera, a visible light camera or an ultraviolet camera;

the intelligent camera is fixedly arranged on the controllable holder and swings along with the controllable holder;

the video image processing chip and the central processing unit are also used for example segmentation of the video image of the power transformation equipment and determination of a target central value of a target main body;

and the central processing unit is also used for executing a tripod head visual angle self-adaptive adjustment algorithm according to the target central value of the target main body and controlling the controllable tripod head to swing so that the target main body is positioned in the center of the picture collected by the camera sensing unit.

Further, the central processing unit is further configured to cooperate with hardware resources of a video image processing chip and hardware resources of the central processing unit, execute an operator tuning algorithm on the lightweight defect identification model, obtain an optimal scheduling scheme corresponding to the lightweight defect identification model, and map the optimal scheduling scheme into the central processing unit.

Furthermore, the intelligent front-end board card also comprises a compression algorithm processing unit;

the compression algorithm processing unit is used for performing pseudo-quantization processing and/or pruning processing on the transformer equipment image diagnosis model acquired from the cloud end to obtain a lightweight defect identification model;

the power transformation equipment defect image diagnosis model is obtained based on one or more of the following deep learning neural network frameworks: YOLO, fast RCNN, SSD, and Master RCNN.

Further, still include:

a cloud server in communication connection with the plurality of intelligent front ends;

the cloud server is provided with a cloud side cloud edge cooperative component, a defect image sample library, a defect identification model library and a defect identification model training platform;

correspondingly, each intelligent front end further comprises a side edge cloud edge cooperative component;

the defect image indicating the defects of the power transformation equipment, which is determined by the intelligent analysis unit, is transmitted through the edge side cloud edge cooperative component and the cloud side cloud edge cooperative component;

in a cloud server, after the defect image indicating the defects of the power transformation equipment is labeled and audited, the defect image serving as a labeled sample with defect type identification is stored in a defect image sample library;

the defect recognition model training platform performs model training and optimization according to the labeled sample and pushes the model evaluated by the model to a defect recognition model library;

the defect identification model library is further used for carrying out version management on the models and pushing the updated models to the intelligent front ends through the cloud side and edge side cloud side cooperative components.

According to the method and the system for diagnosing the defects of the power transformation equipment based on the intelligent front end of the video image, provided by the invention, the video image processing chip integrated at the intelligent front end is utilized to execute a power transformation equipment defect image diagnosis algorithm and generate a power transformation equipment state early warning conclusion or a defect identification conclusion; the video images are processed in real time by utilizing the edge computing power arranged on the side of the edge, the image processing with high speed and high efficiency is realized on the side of the edge, and the rapidity, the timeliness and the accuracy of the defect diagnosis of the transformer substation equipment are ensured.

Drawings

A more complete understanding of exemplary embodiments of the present invention may be had by reference to the following drawings in which:

FIG. 1 is a schematic diagram of a power transformation equipment defect image diagnosis system according to a preferred embodiment of the present invention;

FIG. 2 is a schematic flow chart of a defect image diagnosis model operator tuning algorithm according to a preferred embodiment of the present invention;

fig. 3 is a schematic flow chart of a pan-tilt-zoom adaptive adjustment algorithm according to a preferred embodiment of the present invention;

FIG. 4 is a flow chart illustrating the compression algorithm process according to the preferred embodiment of the present invention.

Detailed Description

The exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, however, the present invention may be embodied in many different forms and is not limited to the embodiments described herein, which are provided for complete and complete disclosure of the present invention and to fully convey the scope of the present invention to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the accompanying drawings is not intended to be limiting of the invention. In the drawings, the same units/elements are denoted by the same reference numerals.

Unless otherwise defined, terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Further, it will be understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense.

Video images, i.e., video data, are composed of temporally successive frames of images.

The central processing unit is also called a Central Processing Unit (CPU).

YOLO, also known as youonly Look Once, is an object detection algorithm used to detect objects/targets in images in real time.

The SSD, i.e., Single Shot multitox Detector, is a multi-target detection algorithm that directly predicts a target class and a bounding box. It uses the low-level feature map to detect small targets and the high-level feature map to detect large targets.

Mask RCNN is a branch to which a predictive segmentation Mask is added on the basis of fast RCNN.

The method for diagnosing the defects of the power transformation equipment based on the intelligent front end of the video image comprises the following steps:

acquiring a video image of the power transformation equipment in real time;

In the method for diagnosing the defects of the power transformation equipment, the intelligent analysis unit integrated in the intelligent front-end board card processes the video image based on the intelligent front end of the video image, namely the intelligent AI chip, executes a power transformation equipment defect image diagnosis algorithm, and generates a power transformation equipment state early warning conclusion or a defect identification conclusion.

Specifically, in the method for diagnosing the defects of the power transformation equipment, an intelligent front end plate card carries an embedded chip for edge calculation, namely an embedded chip special for artificial intelligent image processing, and video image intelligent identification is carried out on the side end; and the cloud end issues the trained and evaluated transformer equipment defect image diagnosis model to the cloud edge cooperative component of the intelligent front-end board card through the cloud edge cooperative component at the cloud end side.

And the compression algorithm processing unit of the intelligent front-end board card compresses the received defect image diagnosis model to obtain a lightweight defect identification model, so that model compression is realized. The embedded operating system is combined with hardware resources of an embedded chip special for artificial intelligent image processing and a central processing unit, an operator tuning algorithm is executed, an optimal scheduling scheme corresponding to the lightweight defect identification model is obtained, and the execution efficiency of the diagnosis model is improved. And the intelligent analysis unit of the intelligent front-end board card executes the optimal scheduling scheme/optimized model, and performs the example segmentation and defect diagnosis of the power transformation equipment in real time.

Specifically, the embedded chip for edge calculation used by the intelligent analysis unit is a rising 310AI processing chip. As an efficient, flexible, programmable AI processor, the Boolean 310 in its typical configuration achieves 22TOPS for eight bit integer precision (INT 8) and 11TFLOPS for 16 bit floating point (FP 16), supporting the simultaneous recognition of 200 different objects, which can process thousands of pictures in one second.

It should be understood that the rising 310AI processing chip is not a general purpose embedded chip and general purpose processor that is used to replace a central processing unit such as an ARM processor running an embedded operating system; but rather, in cooperation with the generic embedded chip and the embedded operating system, adds an AI accelerator function to the generic processor. In addition, AI calculation is computationally intensive, and compared with a CPU, the AI calculation has low requirement on control and is relatively simple to control.

The itantegra 310 adopts a da vinci architecture, uses a high-efficiency flexible operation-intensive CISC instruction set (special instructions for neural networks), can complete 4096 MAC calculations in 1 cycle for each AI core, integrates various operation units such as tensors, vectors, scalars and the like, supports various mixed precision calculations, supports high-precision operations of training and reasoning two scene data, and conveniently extends artificial intelligence from a data center to edge computing equipment.

TOPS is the abbreviation for Tera Ops/Second, that is Tera Operations Per Second, and 1 TOPS stands for that a processor can perform one trillion (10 ^ 12) Operations Per Second, which is the basic operator operand Per Second in the deep learning field. FLOPS, also known as the abbreviation of Floating-point operations per second, is the number of Floating-point operations performed per second. One TFLOPS (i.e., tera FLOPS) is equal to 10^12 floating point operations per trillion per second. Floating-point operations are more time consuming than integer operations because they involve all fractional operations.

Operands are somewhat different in view of different word lengths and precisions, and therefore word lengths such as 16 bit floating point, or 8bit integer are typically labeled. Generally, the shorter the word length, the worse the accuracy. Therefore, if the model optimization effect is not good, the 4-bit precision is not good enough compared with the traditional deep learning model. It is generally considered that a minimum of 8 bits is required to ensure good accuracy.

Specifically, before acquiring a video image of the power transformation device in real time, the method further comprises the following steps:

Specifically, the embedded operating system running on the general embedded chip (including the central processing unit) is also used for executing the compression algorithm, and corresponds to a virtual device, namely a compression algorithm processing unit.

The compression algorithm processing unit is used for compressing the transformer equipment defect image diagnosis model so as to deploy a neural network deep learning framework corresponding to the transformer equipment defect image diagnosis model in the central processing unit and the intelligent analysis unit of the intelligent front-end board card.

Specifically, the compression algorithm includes a pseudo quantization process, such as changing the depth learning model parameters of the 32-bit floating point representation to a double-precision or single-precision representation, and storing. The compression algorithm further comprises pruning, wherein unimportant parameters in the deep learning weight matrix are set to be 0, and the deep learning weight matrix is stored by utilizing the sparse matrix.

It should be understood that the deep learning model is a pre-specified neural network deep learning framework of some kind.

In specific implementation, the sequence between the pseudo quantization processing and the pruning processing is not limited. The sequence between the two does not influence the compression effect.

Specifically, the compression algorithm executed by the compression algorithm processing unit includes:

1) and (5) pseudo quantization processing. And converting the 32-bit floating point (float) deep learning model parameters acquired by the cloud edge cooperative component into 8-bit integer (int) for storage.

That is, during storage, the model parameters are quantized using low precision integer. On the other hand, in the deep learning framework, the majority operator only supports 32-bit floating point operation, so that in the process of executing the defect diagnosis model after the weight reduction by the intelligent analysis unit, the model parameters expressed by the low-precision integer need to be converted into the model parameters expressed by the high-precision floating point, and then the subsequent calculation needs to be performed. Therefore, the pseudo quantization process can compress the storage space occupied by the model in the model storage step, but has little effect on acceleration in the model operation process.

Specifically, the pseudo quantization process includes both cases of symmetric quantization and asymmetric quantization.

Note that the scale of quantization is scale, which is typically a 32-bit floating point number; recording a fixed point value after the floating point value 0 is quantized as offset; and recording the value range of the 32-bit floating point number as [ Fmin, Fmax ], and the value range of the quantized integer number as [ Qmin, Qmax ].

1.1) symmetric quantization

When symmetrically quantizing, after the floating point value 0 is mapped to the zero point, the offset value is 0, that is: offset = 0.

The value representing the quantized scale is calculated according to:

。

at this time, the floating point value

And the integer value after quantization

The interconversion formula of (1) is as follows:

；

；

wherein the content of the first and second substances,

deep learning model parameters for floating point value precision;

deep learning model parameters for integer precision;

as a function of rounding operations.

1.2) asymmetric quantization

In the case of asymmetric quantization, the fixed-point value of floating-point number 0 is not fixed to zero.

The value representing the quantized scale is calculated according to:

；

the value of the floating point value 0 quantized fixed point value offset is calculated according to the following formula:

。

at this time, the formula of interconversion of the floating point value and the quantized value is as follows:

；

；

wherein the content of the first and second substances,

deep learning model parameters for floating point value precision;

model parameters are deeply learned for integer precision.

Generally, when | F_minI and F_maxWhen they are close (usually the difference between the absolute values of the two is F_maxWithin 10%) of the total amount of_minI and F_maxSymmetry, in which case, a symmetric quantization process can be adopted; if not, then judge | F_minI and F_maxAsymmetric, and asymmetric quantization processing is employed.

By adopting the processing strategy, the capability of expressing data by 8-bit integer can be fully utilized.

2) And (5) pruning. And setting the unimportant weight value in the deep learning weight matrix to be 0, and storing the weight matrix subjected to pruning treatment through a sparse matrix.

Specifically, the basic principle for evaluating the importance of each weight value in the weight matrix is the distance between each weight value and the 0 value. The closer a certain weight value is to the value 0, the less important the weight value is; the farther a certain weight value is from a value of 0, the more important the weight value is.

Specifically, the pruning process includes:

2.1) disconnecting the connection between two weakly connected neurons. Each element of the weight matrix (i.e., the weight value) reflects the connection between neurons; the smaller the weight value, the weaker the connection between neurons.

Specifically, when a disconnection operation between two weakly connected neurons is performed, the weight values in the weight matrix are sorted from large to small according to the magnitude of an absolute value, and the ranked k% weight value is set to zero.

2.2) removing insignificant neurons. The mapping to the weight matrix is equivalent to zeroing a certain row and/or a certain column in the weight matrix.

Specifically, when the unimportant neuron removing operation is executed, calculating the root of the square sum of the weighted values of the row and the column corresponding to each neuron, and sorting the importance of each neuron from large to small according to the size of the root; and setting the weight values of the row and the column corresponding to k% of the neurons with ranked importance to zero.

2.3) storing the deep learning weight matrix through the sparse matrix. Specifically, in the sparse matrix, only the sequence number of the non-zero element in the weight matrix and the numerical value of the non-zero element are stored. Therefore, compared with the weight matrix, the sparse matrix has lower dimensionality and occupies less storage space.

Specifically, the intelligent front end of the video image is provided with a video image processing chip and a central processing unit;

Specifically, an embedded operating system running on a general embedded chip executes an operator tuning algorithm by combining hardware resources of an artificial intelligent image processing special embedded chip and a central processing unit, so that an optimal scheduling scheme corresponding to a lightweight defect identification model is obtained, and the execution efficiency of a diagnosis model is improved.

The traditional deep learning framework usually abstracts a deep learning neural network model into a data flow graph constructed by operators and dependency relations, wherein the operators refer to sub-operations in the deep learning framework such as convolution, pooling, full connection, normalization and the like.

Currently, for data flow diagrams, a first layer scheduler and a second layer scheduler are involved in execution. The first-layer scheduler is responsible for scheduling parallelism among operators, and the operators are scheduled to the central processing unit one by one or a plurality of operators simultaneously according to a topological sequence of a data flow graph. Under this, there is again a second layer scheduler that is responsible for exploiting the parallelism of scheduling within operators to map computational tasks to smaller granularity processing units. However, in actual deployment, the two layers of schedulers are not aware of each other, and the scheduling overhead during operation is large, for example, the direct overhead includes command time that must be executed when the central processing unit switches. For example, the indirect overhead refers to the time of cache warm start after switching a new process, and the like. In the scheduling mode, the parallelism among operators is not effectively utilized, and the mutual influence of the parallelism in the operators and the parallelism among the operators is ignored.

As shown in fig. 2, in the operator tuning algorithm of this embodiment, the embedded operating system generates a scheduling scheme according to the deep learning neural network model after pseudo quantization and pruning and the bottom layer computing resource on the front-end smart board. Specifically, operators in the original dataflow graph are decomposed into smaller scheduling units, and the underlying hardware is abstracted to be composed of a plurality of virtual execution units. Under the new set of abstraction, the dataflow graph is dispatched to a plurality of virtual execution units through finer unit granularity, and the coordination of two kinds of parallelism in the calculation task and the bottom layer calculation resource is considered. Then, in the compiling period, the embedded operating system generates the whole scheduling scheme, and the whole scheduling scheme is generated in the compiling period and is mapped to a hardware computing unit (comprising an artificial intelligent image processing special embedded chip and a central processing unit) in a static mode, so that many original scheduling overheads of direct overheads and indirect overheads in scheduling switching can be eliminated.

Specifically, as shown in fig. 2, the operator tuning algorithm includes the following steps:

3.1) analyzing a data flow graph of the transformer equipment defect diagnosis model into an operator and dependency relationship;

3.2) decomposing each operator into smaller uncoupled scheduling units respectively;

3.3) abstracting the bottom hardware of the central processing unit into a plurality of virtual execution units;

3.4) scheduling the data flow graph on a plurality of virtual execution units through finer scheduling unit granularity to obtain a scheduling scheme, wherein the scheduling scheme takes two parallelities in the computing task into account and coordinates with bottom computing resources;

3.5) generating a scheduling scheme at compile time and "statically" mapping onto the hardware computational units, thus eliminating much of the scheduling overhead that would otherwise be present.

Specifically, a camera sensing unit is arranged at the intelligent front end of the video image, and is fixedly arranged on the controllable holder and swings along with the controllable holder;

Specifically, the embedded operating system running on the general embedded chip (including the central processing unit) is also used for controlling the controllable pan/tilt head. The rotation angle and/or the pitch angle of the holder corresponding to different preset point positions are recorded in advance, and flexible switching among the preset point positions is realized.

It should be noted that the camera sensing unit is fixedly connected with the controllable pan/tilt; the rotation/pitching of the holder is controlled to change the visual angle of the holder, so that the image acquisition angle of the camera sensing unit is changed at the same time.

Specifically, at any preset point position, adaptive adjustment of the visual angle of the holder is realized through a center offset algorithm. Firstly, the intelligent analysis unit cooperates with the CPU to divide the transformer equipment from the acquired video image (it should be understood that, at this time, the transformer equipment is used as a target main body, which is a foreground, and the environment around the transformer equipment or other accessories are used as a background; the target main body used as the foreground needs to be divided from the background, and the information such as the type, model and number of the specific transformer equipment corresponding to the target main body is determined), and then the target central value of the target main body is determined. Subsequently, the CPU calculates the offset amount based on the target center value of the target subject acquired from the intelligent analysis unit and the screen center value of the image collected by the camera sensing unit. And then, normalizing the offset, determining the rotation direction of the holder according to the positive and negative of the offset, and controlling the rotation of the holder through proportional-Integral-Derivative (PID), so that the target main body is placed in the center of a picture acquired by a camera sensing unit, and the detection efficiency of the target main body based on a deep learning model and the accuracy of image defect identification of the power transformation equipment are improved.

Specifically, the intelligent camera is provided with a controllable holder, a central processing unit at the front end of the intelligent camera is provided with a holder visual angle self-adaptive adjustment algorithm, the angle self-adaptive adjustment of the camera is realized by controlling the rotation or pitching of the controllable holder, and the stability and reliability of the image acquired by the camera are ensured.

As shown in fig. 3, the adaptive pan-tilt-angle adjustment algorithm includes the following steps:

4.1) identifying the target of the power transformation equipment.

The intelligent analysis unit performs example segmentation on the video image returned by the intelligent camera by using a transformer equipment example segmentation model to obtain an example segmentation area (a minimum rectangle surrounding the transformer equipment) of each transformer equipment, namely a rectangular frame for target identification, and determines boundary coordinates of the rectangular frame for target identification in a picture, namely,

the value of (c).

4.2) offset calculation

Calculating the coordinates of the center of the rectangular frame for object recognition according to the following equation (

)：

；

；

Recording the coordinate of the center of the picture as (

) The picture center coordinates are calculated according to the following formula:

；

；

wherein the content of the first and second substances, Widthrepresenting a picture width;Heightrepresenting a picture height;

calculating the offset between the center of the screen and the center of the rectangular frame for object recognition according to the following equation

)：

；

。

In the method, the influence of different image resolutions of the picture and the rectangular frame is considered, the offset is scaled to the range of [ -1,1] through normalization processing, and the rotating direction of the holder is represented by a positive sign or a negative sign to be anticlockwise or clockwise. Specifically, a positive sign indicates that the pan/tilt head is rotating in a counterclockwise direction; the minus sign indicates that the head is rotating clockwise.

4.3) PID control of the rotation of the holder.

Due to the limitation of the minimum stepping angle when the pan/tilt is rotated, the pan/tilt may shake near the target value, and for this reason, a dead zone range (self-adaptation adjustment of the pan/tilt with the change of the angle increment after the offset amount falls within the dead zone range is stopped) is set as the minimum stepping angle of the pan/tilt.

Because the pan-tilt control amount is an angle increment (an angle for controlling the pan-tilt to move each time), and the offset amount is a two-dimensional coordinate increment, the two-dimensional coordinate increment is firstly converted into the angle increment according to a coordinate-angle conversion constant calibrated in advance, and then the pan-tilt is controlled to rotate.

Within a plurality of continuous sampling period spans, in each sampling period, estimating an angle increment in real time according to the offset between the center of the current picture and the center of the rectangular frame for target identification; and when the current angle increment is determined to be smaller than the dead zone range, controlling the holder to stop rotating.

In particular, when deep learning neural network training and evaluation are completed according to typical video images and typical defects of a transformer substation to generate the transformer equipment image diagnosis model,

Specifically, deep learning model training is carried out on a positive sample of the power transformation equipment at the cloud end, and a power transformation equipment instance segmentation model is constructed and used for target identification and cloud deck control; and carrying out deep learning model training by using the negative sample of the defects of the power transformation equipment at the cloud end, and constructing a power transformation equipment defect diagnosis model for detecting the defects of the power transformation equipment.

Specifically, the transformer equipment defect image diagnosis model can be obtained by deep learning neural network framework training based on YOLO, fasternn, SSD, Mask RCNN and the like.

Correspondingly, the transformer equipment defect image diagnosis model is obtained by training aiming at the 4 types of image data sources respectively.

As shown in fig. 4, the deep learning neural network model training step includes:

5.1) initializing the weight;

5.2) forward propagation calculation;

5.3) loss calculation;

5.4) backward propagation calculation;

5.5) updating the weight matrix;

5.6) outputting the model parameters meeting the preset accuracy in the model prediction and the weight matrix.

If the model prediction accuracy is less than the preset accuracy, repeating the steps from 5.2) to 5.5); and if the model prediction accuracy is greater than or equal to the preset accuracy, outputting a model file, a weight file, model parameters and a configuration file.

As shown in fig. 1, the power transformation equipment defect diagnosis system based on the video image intelligent front end of the embodiment of the present invention includes:

Specifically, the intelligent camera comprises a camera sensing unit, wherein the camera sensing unit is an infrared camera, a visible light camera or an ultraviolet camera;

Specifically, the central processing unit is further configured to cooperate with hardware resources of a video image processing chip and hardware resources of the central processing unit, execute an operator tuning algorithm on the lightweight defect identification model, obtain an optimal scheduling scheme corresponding to the lightweight defect identification model, and map the optimal scheduling scheme into the central processing unit.

Specifically, the intelligent front-end board card further comprises a compression algorithm processing unit;

Specifically, the method further comprises the following steps:

According to the method for diagnosing the defects of the transformer equipment, an advanced intelligent chip technology is utilized on the side of the side, an embedded chip special for artificial intelligent image processing and an embedded operating system are implanted at the intelligent front end of video image acquisition, so that the quick and efficient image processing is realized on the side of the side, and the rapidness, timeliness and accuracy of the defect diagnosis of the transformer equipment are guaranteed; and the visual angle of the holder is self-adaptively adjusted, so that the intelligent level of the video image acquisition intelligent front end is improved, and the stability and the reliability of the acquired image are ensured.

According to the defect diagnosis method for the power transformation equipment, the controllable cloud deck is carried by the intelligent camera on the side of the side end, the cloud deck visual angle adaptive adjustment algorithm is deployed on the central processing unit of the intelligent front end, adaptive adjustment of the angle of the camera is achieved by controlling the cloud deck, and stability and reliability of collected images are guaranteed.

In the method for diagnosing the defects of the power transformation equipment, an intelligent analysis unit generates a defect diagnosis conclusion (including a defect picture and alarm information) of the power transformation equipment, the defect picture and the alarm information are transmitted to a cloud side cooperative component through a side cloud side cooperative component, a labeled sample is obtained after automatic and/or manual labeling and checking, and the labeled sample is put into a sample library; on the other hand, the labeled sample is sent to a defect recognition model training platform for model training and optimization to train a transformer equipment defect image diagnosis model, and after the defect recognition model training platform carries out model evaluation, the image diagnosis model meeting the requirements is pushed to an image diagnosis model library; the image diagnosis model library is used for managing model versions (for example, version numbering is carried out according to receiving dates), the latest version of the defect image diagnosis model is pushed to the intelligent front-end side cloud side cooperation assembly through the cloud side cooperation assembly, and the intelligent front-end compression algorithm processing unit is used for updating the transformer equipment defect image diagnosis model executed by the intelligent analysis unit in cooperation with the central processing unit, so that closed-loop optimization of the annotation sample and the image diagnosis model between the cloud side and the side is achieved.

The intelligent front-end board card is also provided with a cloud edge cooperative component (edge end) for returning a power transformation equipment state early warning conclusion (namely warning information) and a defect identification conclusion (namely defect data) generated after the intelligent analysis unit executes a power transformation equipment defect image diagnosis algorithm; the cloud edge cooperative component is also used for receiving a mirror image file or an update package of a power transformation equipment defect image diagnosis algorithm issued by the cloud end.

The cloud edge cooperative component receives an image file or an update packet of a power transformation equipment defect image diagnosis algorithm at an edge end, and the image file or the update packet is sequentially subjected to compression algorithm processing and operator tuning, and finally the intelligent analysis unit cooperates with the central processing to execute the updated power transformation equipment defect image diagnosis algorithm/model.

The cloud side cooperative component (cloud end) is deployed on the cloud server and used for receiving defect data and alarm information generated after the intelligent analysis unit executes a transformer equipment defect image diagnosis algorithm; and the cloud server is also used for issuing a mirror image file or an update package of the power transformation equipment defect image diagnosis algorithm generated after the cloud server is trained.

The defect recognition model training platform is deployed on the cloud server and used for training a transformer equipment defect image recognition algorithm model; the container technology is adopted, and the mirror images of learning frames with different depths can be deployed; and carrying out iterative optimization on the transformer equipment defect image recognition algorithm model by utilizing sufficient computing resources of the cloud server.

And the model library is deployed in the cloud server and is used for model version management, model feedback and downloading, model evaluation and release and model mirror image packaging.

And the sample library is deployed on the cloud server and is used for image data storage and management, data cleaning and preprocessing, data labeling and label management. Specifically, modules for sample returning/downloading, sample management, sample expansion, sample marking, sample evaluation and the like are provided, and collection and management of sample resources are effectively supported.

In the method for diagnosing the defects of the power transformation equipment, the intelligent analysis unit transmits the obtained diagnosis conclusion of the defects of the power transformation equipment to the cloud side cooperative component at the cloud side through the cloud side cooperative component at the edge side, and transmits the defect picture and the alarm information to the cloud side cooperative component at the cloud side. And at the cloud end, after self-adaptive/manual marking and checking, the defect picture and the alarm information are put into a sample library.

And at the cloud end, the labeled sample is sent to a defect recognition model training platform to train a transformer equipment defect image diagnosis model, model evaluation is carried out on the defect recognition model training platform, and the diagnosis model meeting the accuracy requirement after evaluation is pushed to a model base.

And at the cloud end, the model library carries out version management on the diagnosis model, and pushes the transformer equipment defect image diagnosis model to a cloud edge cooperative component at the intelligent front end (namely the edge end) through the cloud edge cooperative component at the cloud end side.

The intelligent front end updates the lightweight transformer equipment defect image diagnosis model stored in the central processing unit/embedded chip through the compression algorithm and the operator tuning algorithm, and realizes the closed-loop optimization of the model at the edge end.

The transformer substation defect diagnosis method of one embodiment of the invention comprises the following steps:

and step S10, finishing the real-time acquisition of the video image through the camera sensing unit at the intelligent front end.

Step S20, finishing H.265 coding of the video image through an image digital coder at the intelligent front end to obtain an H.265 code stream; the H.265 code Stream is encapsulated into a Real-Time streaming Protocol (RTSP) data packet, and the RTSP data packet is transmitted to an encoding and decoding module of the intelligent front-end board card, so that the RTSP data packet is decoded into a video image which can be processed by the intelligent analysis unit.

In specific implementation, the video encoding module and the video encoding/decoding module disclosed in the prior art are optionally configured according to a video image transmission format supported by the camera sensing unit.

And step S30, carrying out transformer equipment infrared instance segmentation and defect diagnosis by using a preset diagnosis model.

Specifically, the smart front panel card carries an artificial intelligence video image processing chip that implements edge computing (as opposed to cloud computing), such as an ascend heave 310AI processor chip.

As shown in fig. 1, the system for diagnosing defects of a power transformation device includes: a plurality of intelligent front ends and a cloud end; the intelligent front end comprises an intelligent camera and an intelligent front end board card; the intelligent camera is integrated with a camera sensing unit; the intelligent front end/intelligent camera can be installed at the camera point of the transformer substation as an independent unit.

When the camera sensing unit adopts the visible light spectrum section module, the camera sensing unit is used for collecting visible light video images of the power transformation equipment; when the camera sensing unit adopts the infrared spectrum section module, the infrared spectrum section module is used for acquiring an infrared video image of the power transformation equipment; when the camera sensing unit adopts the ultraviolet spectrum section module, the camera sensing unit is used for acquiring an ultraviolet video image of the power transformation equipment; when the acoustic imaging probe is adopted, the acoustic imaging probe can be used for acquiring acoustic imaging of the power transformation equipment.

The intelligent camera is further integrated with an image digital encoder, and the image digital encoder is used for receiving the video image signals acquired by the camera sensing unit and encoding the video image signals into an H.265 format, so that the compression efficiency is improved, the robustness and the error recovery capability are improved, the real-time delay is reduced, the channel acquisition time and the random access time delay are reduced, the complexity is reduced, and the like.

The intelligent camera is further integrated with a controllable holder, and the controllable holder is used for completing mechanical movement of the camera sensing unit in switching at different preset points and completing mechanical movement of adaptive adjustment of holder visual angles.

The intelligent front panel card is integrated with a coding and decoding module (which can be a hardware module or a software module) for coding and decoding the h.265 format video image sent by the image digital encoder.

The intelligent front end plate card is integrated with a central processing unit, and the central processing unit and the embedded chip special for artificial intelligent image processing integrated on the intelligent front end plate card cooperate to execute a power transformation equipment defect image diagnosis algorithm.

The intelligent front-end board card is also provided with a compression algorithm processing unit. And the compression algorithm processing unit executes a power transformation equipment defect image diagnosis model compression algorithm, realizes model lightweight conversion and obtains a lightweight diagnosis model.

The intelligent front-end board card is also provided with a central processing unit. The central processing unit executes a pan-tilt visual angle self-adaptive adjustment algorithm and controls the camera sensing unit to move along with the pan-tilt so as to realize target body tracking. The central processing unit also executes operator tuning, and execution efficiency of the intelligent analysis/unit is improved.

The intelligent front-end board card is also provided with an intelligent analysis unit, the intelligent analysis unit executes a light-weighted defect diagnosis model according to a scheduling scheme after operator tuning, processes video images obtained by the camera sensing unit, comprises example segmentation and defect diagnosis, and generates an equipment state early warning conclusion (namely warning information) and a defect identification conclusion (namely defect data) conclusion.

And issuing the transformer equipment defect image diagnosis model pre-trained and evaluated at the cloud end to a cloud edge cooperative component arranged on the intelligent front-end board card through the cloud edge cooperative component arranged at the cloud end.

Namely, each intelligent front end installed at each preset position of each camera of the transformer substation is used as a cloud edge, and the cloud edge and a cloud end arranged on a cloud server form a transformer equipment defect diagnosis system based on the Internet of things through a cloud edge cooperative component; cloud computing is adopted at the cloud end, edge computing is adopted at each intelligent front end, and transformer equipment defect diagnosis is completed in a coordinated mode.

In conclusion, the method and the system for diagnosing the defect images of the power transformation equipment realize the self-adaptive adjustment of the image acquisition angle of the camera sensing unit through the self-adaptive adjustment of the visual angle of the holder, intelligently acquire the video images of the power transformation equipment and improve the intelligent level of video acquisition at the intelligent front end; at the side end, based on the video image processing chip, the video image of the substation equipment is processed by utilizing the deep learning model, the timeliness and instantaneity of video data analysis are improved, the state early warning and defect identification of the substation equipment are realized, and the real-time monitoring and control of the state of the substation equipment are realized.

The transformer equipment defect image diagnosis method and system can greatly improve the inspection efficiency of the transformer substation, timely find the defect of equipment overheating, eliminate the potential fault risk of the equipment, ensure the safe and reliable operation of the equipment, reduce the probability of power failure, play the active auxiliary support role of equipment state evaluation and maintenance decision and save the operation and maintenance cost.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The invention has been described above by reference to a few embodiments. However, other embodiments of the invention than the one disclosed above are equally possible within the scope of the invention, as would be apparent to a person skilled in the art from the appended patent claims.

Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a// the [ device, apparatus/line, etc. ]" are to be interpreted openly as at least one instance of a device, apparatus/line, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

Claims

1. A transformer equipment defect diagnosis method based on video image intelligent front ends is characterized in that the method is used for a plurality of video image intelligent front ends arranged at different positions in a transformer substation and comprises the following steps:

the intelligent front end of the video image is provided with a video image processing chip and a central processing unit, wherein the central processing unit is provided with an embedded operating system; the video image processing chip is matched with the universal embedded chip and the embedded operating system to add an AI accelerator function to the central processing unit;

the method comprises the following steps that a camera sensing unit collects a video image of the power transformation equipment, wherein the camera sensing unit is arranged at the intelligent front end of the video image and is fixedly arranged on a controllable cloud deck to swing along with the controllable cloud deck;

executing a lightweight instance segmentation model to process the video image, and determining a target central value of a target main body, wherein the target main body is power transformation equipment;

the central processing unit calculates the offset according to the target central value of the target main body and the picture central value of the image collected by the camera sensing unit; according to the offset, the controllable holder is controlled through proportional-integral-differential, so that the target main body is positioned in the center of the picture collected by the camera sensing unit;

the camera sensing unit collects video images of the power transformation equipment in real time, wherein the embedded operating system controls the controllable holder through proportional-integral-differential control so that the power transformation equipment serving as a target main body is positioned in the center of a picture collected by the camera sensing unit;

the intelligent analysis unit executes a lightweight defect identification model to process the video image by adopting an optimal scheduling scheme, and determines the defects of the power transformation equipment according to a processing result;

the lightweight defect identification model is obtained by performing pseudo-quantization processing and/or pruning processing on the acquired transformer equipment image diagnosis model by using a compression algorithm processing unit; wherein, the compression algorithm processing unit runs in an embedded operating system;

and the optimal scheduling scheme is obtained by executing an operator tuning algorithm on the lightweight defect identification model by the embedded operating system in cooperation with hardware resources of a video image processing chip and hardware resources of a central processing unit, and mapping the lightweight defect identification model into the central processing unit.

2. The method of claim 1,

the cloud side and cloud edge cooperative component is arranged on the cloud server.

3. The method of claim 1,

when deep learning neural network training and evaluation are completed according to typical video images and typical defects of a transformer substation to generate the transformer equipment image diagnosis model,

4. A transformer equipment defect diagnosis system based on video image intelligence front end, characterized by comprising:

the intelligent front end plate card is provided with a video image processing chip and a central processing unit, wherein the central processing unit is provided with an embedded operating system; the video image processing chip is matched with the universal embedded chip and the embedded operating system to add an AI accelerator function to the central processing unit;

the intelligent front-end board card is also provided with an intelligent analysis unit, and the intelligent analysis unit is used for coordinating hardware resources of a video image processing chip and hardware resources of a central processing unit by adopting an optimal scheduling scheme, executing a lightweight defect identification model to process the video image, and determining the defects of the power transformation equipment according to the processing result;

5. The system of claim 4,

the intelligent camera comprises a camera sensing unit, and the camera sensing unit is an infrared camera, a visible light camera or an ultraviolet camera.

6. The system of claim 4,

the central processing unit is further used for cooperating with hardware resources of a video image processing chip and hardware resources of the central processing unit, executing an operator tuning algorithm on the lightweight defect identification model to obtain an optimal scheduling scheme corresponding to the lightweight defect identification model, and mapping the optimal scheduling scheme into the central processing unit.

7. The system of claim 4,

the intelligent front-end board card also comprises a compression algorithm processing unit;

the compression algorithm processing unit is used for performing pseudo-quantization processing and/or pruning processing on the power transformation equipment image diagnosis type acquired from the cloud end to obtain a lightweight defect identification model;

the power transformation equipment defect image diagnosis model is obtained based on one or more of the following deep learning neural network frameworks: YOLO, fast RCNN, SSD, and Mask-RCNN.

8. The system of claim 4, further comprising:

the cloud server is in communication connection with the intelligent front ends;