CN117853445A - Target tracking analysis method and system based on machine vision - Google Patents

Abstract

The application provides a target tracking analysis method and system based on machine vision, wherein the method comprises the following steps: installing a plurality of image information acquisition devices at target devices of the wind turbine generator, and acquiring a plurality of groups of image data; normalizing the plurality of groups of image data, and performing alignment processing through an image alignment network; selecting sample data with preset time length from the processed image data, performing fast Fourier transform on the sample data to obtain DC-removed frequency spectrum data corresponding to the sample data, and analyzing the frequency domain characteristics of the sample data; and analyzing the vibration condition of the target equipment based on the DC-free frequency spectrum data, and analyzing the transverse vibration characteristics of the target equipment based on the frequency domain characteristics to obtain the structural vibration parameters and the strain parameters of the target equipment. According to the method, vibration monitoring of the wind turbine generator related equipment is performed based on machine vision, vibration signals of the related equipment can be accurately obtained, and accuracy, convenience and implementation efficiency of vibration and strain monitoring are improved.

Description

Target tracking analysis method and system based on machine vision

Technical Field

The application relates to the technical field of operation and maintenance of wind turbines, in particular to a target tracking analysis method and system based on machine vision.

Background

With the development of new energy technology, the popularity of wind generating sets is gradually increased, and the wind generating sets are maintained so as to ensure the safe operation of the wind generating sets, so that the wind generating sets are an important link in the actual operation process of the wind generating sets. Because maintenance personnel of the wind turbine generator cannot carry out manual inspection frequently and the experience of the maintenance personnel is insufficient, tracking analysis of multiple targets such as structural vibration and strain is required to be carried out on the wind turbine generator so as to determine the vibration state of the wind turbine generator in time, and then reasonable operation and maintenance processing is carried out.

In the related art, when monitoring and analyzing the vibration condition of the wind turbine generator, a linear vibration analysis method is generally used, which includes an analysis method of free vibration and forced vibration of a single degree of freedom system, a modal analysis method of inherent vibration and response analysis of a two degree of freedom and multiple degree of freedom system, and an approximation method of vibration analysis, namely a rayleigh energy method and Lv Ci analysis method.

However, in the vibration analysis method in the related art, the tracking analysis efficiency in practical application is low, the obtained monitoring result may have a large deviation, and the implementation process is complex and inconvenient.

Disclosure of Invention

The present application aims to solve, at least to some extent, one of the technical problems in the related art.

Therefore, a first object of the present application is to provide a target tracking analysis method based on machine vision, which is based on machine vision to perform vibration monitoring of wind turbine generator related equipment, and can accurately obtain vibration signals of the related wind turbine generator related equipment, thereby improving accuracy, convenience and implementation efficiency of vibration and strain monitoring.

A second object of the present application is to propose a machine vision based target tracking analysis system.

A third object of the present application is to propose a non-transitory computer readable storage medium.

To achieve the above object, a first aspect of the present application provides a machine vision-based target tracking analysis method, which includes the following steps:

installing a plurality of image information acquisition devices at target devices to be tracked and analyzed in a wind turbine generator, and acquiring a plurality of groups of image data of the target devices by adjusting each image information acquisition device after setting parameters;

normalizing the plurality of groups of image data, and aligning the normalized image data through a trained image alignment network;

Selecting sample data with preset time length from each group of processed image data, performing fast Fourier transform on each group of sample data to obtain DC-removed frequency spectrum data corresponding to the sample data, and analyzing the frequency domain characteristics of each group of sample data;

analyzing the vibration condition of the target equipment based on the DC-free frequency spectrum data, analyzing the transverse vibration characteristic of the target equipment based on the frequency domain characteristic, acquiring the structural vibration parameter and the strain parameter of the target equipment, and repeatedly carrying out tracking analysis on the target parameter according to a preset period, wherein the target parameter comprises the structural vibration parameter and the strain parameter.

Optionally, in an embodiment of the present application, the normalizing the multiple sets of image data includes: for each group of image data, respectively calculating the average value of the image gray levels, and subtracting the average value from the gray level of each image; and calculating the standard deviation of each group of image data, calculating the sum value of the standard deviation and a preset constant, and dividing the gray scale of each image by the sum value.

Optionally, in an embodiment of the present application, the performing alignment processing on the normalized image data includes: performing feature point detection and feature point rough matching on the normalized image data; combining the feature point rough matching result and an image alignment network trained based on an unsupervised deep learning algorithm to obtain a homography matrix for pixel mapping; all pixel points in each orthogonal polarized image are mapped into a single polarized image based on the homography matrix.

Optionally, in one embodiment of the present application, training the image alignment network includes: acquiring a training data set, and inputting training images in the training data set into a pre-built image alignment network to acquire a predicted alignment image output by the network; acquiring a target image corresponding to the training image, and acquiring first edge characteristic information of the prediction alignment image and second edge characteristic information of the target image; and calculating edge characteristic loss between the first edge characteristic information and the second edge characteristic information, and updating parameter values of an image alignment network based on the edge characteristic loss.

Optionally, in an embodiment of the present application, performing a fast fourier transform on each set of the sample data to obtain dc-removed spectrum data corresponding to the sample data includes: removing a direct current component in the sample data and a periodic sinusoidal signal caused by rotation of the wind turbine generator; substituting the sample data excluding the interference signals into a preset discrete Fourier transform formula to obtain the frequency spectrum data corresponding to each group of sample data.

Optionally, in one embodiment of the present application, the analyzing the frequency domain characteristics of each set of the sample data includes: acquiring a main component in the DC-removed frequency spectrum data; and taking the amplitude ratio and the phase difference of the main components as frequency domain characteristic parameters of corresponding sample data.

Optionally, in an embodiment of the present application, the target device includes a lifting device of the wind turbine, and the transverse vibration characteristic analysis is performed on the lifting device by the following formula:

wherein f _n For the natural vibration frequency of the lifting equipment, T is the tension at two ends of the lifting equipment, L is the length of the steel wire suspension rope, n is the order of the natural frequency, n is 1, E is the elastic modulus of the steel wire suspension rope, I is the moment of inertia of the steel wire suspension rope, and ρ is the linear density of the steel wire suspension rope.

To achieve the above object, a second aspect of the present application further provides a machine vision-based target tracking analysis system, including:

the system comprises an acquisition module, a tracking module and a control module, wherein the acquisition module is used for installing a plurality of image information acquisition devices at target devices to be tracked and analyzed in a wind turbine, and acquiring a plurality of groups of image data of the target devices by adjusting each image information acquisition device after setting parameters;

the preprocessing module is used for carrying out normalization processing on the plurality of groups of image data and carrying out alignment processing on the normalized image data through a trained image alignment network;

the transformation module is used for selecting sample data with preset duration from each group of processed image data, performing fast Fourier transformation on each group of sample data to obtain DC-removed frequency spectrum data corresponding to the sample data, and analyzing the frequency domain characteristics of each group of sample data;

The tracking analysis module is used for analyzing the vibration condition of the target equipment based on the DC-free frequency spectrum data, analyzing the transverse vibration characteristic of the target equipment based on the frequency domain characteristic, acquiring the structural vibration parameter and the strain parameter of the target equipment, and repeatedly carrying out tracking analysis on the target parameter according to a preset period, wherein the target parameter comprises the structural vibration parameter and the strain parameter.

Optionally, in an embodiment of the present application, the preprocessing module is specifically configured to: for each group of image data, respectively calculating the average value of the image gray levels, and subtracting the average value from the gray level of each image; and calculating the standard deviation of each group of image data, calculating the sum value of the standard deviation and a preset constant, and dividing the gray scale of each image by the sum value.

In order to implement the above embodiment, a third aspect of the present application further proposes a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the machine vision-based target tracking analysis method in the above first aspect.

The technical scheme provided by the embodiment of the application at least brings the following beneficial effects: according to the method, machine vision is utilized to perform vibration detection on relevant equipment of the wind turbine generator, and a hardware module and a software processing module are designed and selected. The machine vision-based unit equipment vibration monitoring method can accurately acquire vibration signals of related unit equipment, and acquire various target parameters such as structural vibration parameters, strain parameters and the like. The hardware module used in the practical application has the advantages of small volume, small occupied area, stable system operation, small data acquisition error, accurate monitoring result and low misjudgment rate, and can meet the vibration monitoring requirement of the related unit equipment to the maximum extent, thereby being beneficial to ensuring the safety of the later unit equipment in the operation process. In addition, the image preprocessing module is arranged, the collected unit equipment images are preprocessed, so that the definition of the obtained images is increased in the subsequent use process, and meanwhile, the processing efficiency and accuracy in the subsequent analysis processing process of the images are accelerated, and the running speed of the whole tracking analysis process is improved.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

fig. 1 is a flowchart of a target tracking analysis method based on machine vision according to an embodiment of the present application;

fig. 2 is a flowchart of an image alignment processing method according to an embodiment of the present application;

fig. 3 is a flowchart of a method for generating spectrum data according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a machine vision-based target tracking analysis system according to an embodiment of the present application.

Detailed Description

Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present invention and should not be construed as limiting the invention.

The following describes in detail a target tracking analysis method and system based on machine vision according to an embodiment of the present application with reference to the accompanying drawings.

Fig. 1 is a flowchart of a target tracking analysis method based on machine vision according to an embodiment of the present application, as shown in fig. 1, the method includes the following steps:

step S101, installing a plurality of image information acquisition devices at target devices to be tracked and analyzed in the wind turbine generator, and acquiring a plurality of groups of image data of the target devices by adjusting each image information acquisition device after setting parameters.

The target tracking analysis method of the present application is to perform continuous tracking detection analysis on a plurality of parameters of related devices needing to perform vibration detection in a wind turbine generator by using a machine vision technology, that is, the target performing tracking analysis refers to a plurality of parameters related to vibration conditions, such as structural vibration parameters and strain parameters, of target devices in the wind turbine generator.

Wherein, the machine vision is to use a machine to replace human eyes to make measurement and judgment. The machine vision system converts the shot target equipment into image signals through a machine vision product (namely image information acquisition equipment), and then transmits the image signals to a special image processing system to obtain the form information of the shot target equipment.

Specifically, a target device that needs to perform vibration state tracking analysis detection at present is selected, where the target device may be a related device in each subsystem in the wind turbine, for example, a lifter in a lifting system, which is not limited in this application. After the target equipment is determined, the image information acquisition equipment is firstly installed, and the quality and effect of the follow-up detection analysis processing by the machine vision system are directly determined by the image data related to the vibration signals acquired on site, so that a plurality of image information acquisition equipment can be installed around the target equipment, and the setting parameters of the image information acquisition equipment can be adjusted, so that the image data of the target equipment can be accurately and comprehensively acquired.

For example, the application may set 8 sets of image information acquisition devices around the target device, where each set of image information acquisition devices includes a pre-selected lens and a camera for image acquisition. When the equipment is set, high-definition unit equipment detection images can be obtained by mutually coordinating the selected lenses and the cameras, for example, reasonably adjusting the setting parameters such as the distance between the lenses and the cameras as well as the distance between the lenses and the target equipment.

Furthermore, a plurality of image information acquisition devices which are installed and parameter-adjusted are used for respectively carrying out image acquisition, so that a plurality of groups of image data of the target device can be acquired.

As an example, the collection device based on machine vision for structural vibration and strain tracking may be a MER-503-40GC-P gigabit network CCD camera with high reliability and stable performance, and for obtaining a higher-definition image and vibration signal, a corresponding lens is selected, where the function of the lens is to collect reflected light of an actual object on a corresponding light sensing element. And by reasonably adjusting the lens distance, a high-definition detection image is obtained. For example, when camera and lens parameters are set, MER-503-40GC-P gigabit network CCD camera is selected, the resolution is 2448 (H) multiplied by 2048 (V), the maximum frame rate is 40fps, and the triggering mode is hardware triggering/software triggering; the M2518-MPW2 industrial lens selected by the image information acquisition equipment can automatically adjust the focal length, the focal length is 25mm, the target surface size is 16.9mm, the aperture range is F1.8, and the working temperature is-20-50 ℃.

Step S102, carrying out normalization processing on a plurality of groups of image data, and carrying out alignment processing on the normalized image data through a trained image alignment network.

Specifically, multiple groups of image data acquired in the previous step are input into an image preprocessing module for machine vision-based structural vibration and strain tracking through an information communication module, preprocessing of the image data is carried out, and the preprocessing process comprises normalization processing and then image alignment processing.

In one embodiment of the application, more cables can be effectively prevented from being arranged in the actual measurement environment through the wireless communication equipment inside the information communication module, and data transmission among the control modules is achieved. In this embodiment, the wireless communication device may be an LF-P691 type wireless bridge. The performance parameters of the selected wireless network bridge are as follows: the data transmission rate is 450M/s, the wireless transmission distance is 3km, the network standard is IEEE802.11an/a/ac, the memory/flash memory is 64MB+8MB, and the number of the band cameras is 1-11.

In this embodiment, the wireless bridge, that is, the wireless network bridge, uses the wireless bridge system to connect the image acquisition module and the software processing module, so that the data transmission in the middle and long distances can be realized. In practical application, the data transmission mode of the wireless network bridge is not affected by the geographic environment, and the wireless network bridge has the advantages of long transmission distance and high transmission signal quality. The working frequency band of the wireless bridge is in the 2.4G frequency band and the 5.8G frequency band, and the wireless transmission distance is 3km.

Further, after receiving the image data, normalization is performed in the image processing application of machine vision. In one embodiment of the present application, the normalization processing is performed on the multiple sets of image data, including the following steps: for each group of image data, respectively calculating the average value of the image gray levels, and subtracting the average value from the gray level of each image; then, the standard deviation of each group of image data is calculated, the sum value of the standard deviation and a preset constant is calculated, and the gray scale of each image is divided by the sum value.

Specifically, in the present embodiment, when performing the image normalization processing, first, the average value of the sample gray scale is subtracted from each image data. Then divided by the standard deviation, wherein a small constant is added to the standard deviation to avoid denominator of 0 and suppress noise. For gray data plots in the [0, 255] range, the variance can be increased by 10. In this embodiment, the sample-by-sample mean subtraction is mainly applied to those data sets with stability, i.e. the statistical properties are the same among each dimension of those data.

Further, the normalized image is input into an image alignment network for structure vibration and strain tracking, and an aligned vibration and strain image output by the structure vibration and strain tracking image alignment network based on machine vision is obtained.

Here, since the present application performs image acquisition by a plurality of image information acquisition apparatuses at different positions, image alignment processing is required. The image alignment processing in the application refers to the processes of splicing, noise reduction, image super-resolution application, video anti-shake and the like on pictures shot under different visual angles, and one picture is perfectly aligned with the other picture through twisting and rotation. The key idea is that a group of characteristic points are detected in one image and matched with the characteristic points of the other image, and then a conversion rule is generated according to the matched characteristic points to map the images.

In order to more clearly illustrate the specific implementation of the image alignment process of the present application, an exemplary alignment method set forth in one embodiment of the present application is described below. Fig. 2 is a flowchart of an image alignment processing method according to an embodiment of the present application, as shown in fig. 2, where the method includes:

step S201, performing feature point detection and feature point rough matching on the normalized image data.

Specifically, feature points in the graph are detected first, the detection quantity of the feature points can be determined according to actual needs, and the detection quantity of the feature points is controlled through a max-features function. And then carrying out characteristic point matching, and determining the matched characteristic points in the two graphs.

Step S202, combining the feature point rough matching result and an image alignment network trained based on an unsupervised deep learning algorithm to obtain a homography matrix for pixel mapping.

Specifically, the core of image alignment is to determine a homography matrix, and in related embodiments, the homography matrix for pixel mapping is generally obtained through a random sample consensus algorithm (Random Sample Consensus, RANSAC for short) algorithm, however, this approach has a plurality of defects such as more interference. Therefore, in this embodiment, the homography matrix is calculated by combining the feature point matching and RANSAC algorithm in the related art and the image alignment network trained in advance in the application, that is, the final homography matrix is determined by integrating the two calculation results, so as to improve the accuracy of image alignment.

The image alignment network is trained based on an unsupervised Deep learning algorithm, and a homography matrix suitable for the current scene is calculated through the image alignment network, namely, the output homography matrix is unsupervised-Deep-homography. The homography matrix generated by the method of the embodiment of the application can improve the robustness of image characteristic expression and reduce the dependence on characteristic points.

As one possible implementation, training the image alignment network includes: acquiring a training data set, inputting training images in the training data set into a pre-built image alignment network to acquire a predicted alignment image output by the network; acquiring a target image corresponding to the training image, and acquiring first edge characteristic information of a predicted alignment image and second edge characteristic information of the target image; edge feature loss between the first edge feature information and the second edge feature information is calculated and parameter values of the image alignment network are updated based on the edge feature loss.

Specifically, the training operation is repeatedly performed on the pre-built image alignment network until the training of the machine vision based structural vibration and strain tracking image alignment network is completed, the training operation including: inputting the training image into a structural vibration and strain tracking image alignment network based on machine vision to obtain a predicted image corresponding to the training image, wherein the predicted image is output by the structural vibration and strain tracking image alignment network based on machine vision; acquiring edge characteristic information of the predicted image and edge characteristic information of a target image related to the training image; calculating edge feature loss between the predicted image and the target image based on the edge feature information of the predicted image and the edge feature information of the target image, wherein the edge feature loss is used for indicating the difference between the predicted alignment image and the target image on the edge feature; and updating parameter values of the alignment network of the structural vibration and strain tracking image based on the machine vision based on the edge characteristic loss until the output of the alignment network meets the requirement.

Step S203, mapping all pixel points in each orthogonal polarized image into a single polarized image based on the homography matrix.

Specifically, all pixels of the orthogonal polarized light image are mapped into a unit equipment image of single polarization based on the homography matrix, so that the alignment of structural vibration and strain tracking unit equipment image based on machine vision is realized. The orthogonal polarized light image and the single polarized light image at two ends of the mapping relation can correspond to the two images of the feature point detection and the rough matching in the steps.

In an embodiment of the present application, in an actual application, the module for performing the image preprocessing operation may be an IPC-610L host computer, where the host computer is mounted with an IBM motherboard and an IntelI7 processor, the memory is 2GDDRII, and the optical drive is 16XDVD, so as to meet the operation requirement of the image processing algorithm.

Thus, after the preprocessing operation is finished, the structural vibration and strain tracking analysis module can be used for carrying out structural vibration and strain tracking analysis on the image layout of the target equipment.

Step S103, selecting sample data with preset time length from each group of processed image data, performing fast Fourier transform on each group of sample data to obtain DC-removed frequency spectrum data corresponding to the sample data, and analyzing the frequency domain characteristics of each group of sample data.

Specifically, the preprocessed image data is subjected to fast fourier transform to analyze the frequency domain characteristics of each measurement point in the image data.

In order to more clearly illustrate the specific implementation process of the fast fourier transform in the present application, an exemplary method for performing data transformation to obtain dc-removed spectrum data corresponding to sample data is described below in an embodiment of the present application. Fig. 3 is a flowchart of a method for generating spectrum data according to an embodiment of the present application, as shown in fig. 2, where the method includes:

in step S301, the dc component in the sample data and the periodic sinusoidal signal caused by the rotation of the wind turbine generator are removed.

For example, with the image data collected by the 8 sets of image information collecting devices in the above embodiments, the frequency domain response of the target device under the natural wind load is analyzed by using the image measurement data collected by the 8 sensing devices as the sample signal. In the specific implementation, firstly, data (10 h in total) collected by the target equipment in a rotating speed stable working state are extracted, and one hour (namely a preset time length) is taken as a sample time length to obtain 10 groups of sample data. And then, removing the direct current component in the sample data and the periodic sinusoidal signals and other interference caused by the rotation of the fan, and performing subsequent fast Fourier transform to obtain the frequency spectrum of 10 sections of sample data.

Step S302, substituting the sample data excluding the interference signals into a preset discrete Fourier transform formula to obtain the frequency spectrum data corresponding to each group of sample data.

Specifically, the sample data excluding the dc component and the periodic sinusoidal signal in each group in step S301 is substituted into the discrete fourier transform X (k) calculation formula shown in the following formula:

in the formula, each group of sample data is an input signal X (n), and the frequency spectrum X (k) corresponding to each group of input data is obtained through calculation by the method.

For some devices in the wind turbine generator, the vibration state of the devices may be affected by control parameters of other devices, and for these controlled devices, the last period (i.e. each selected measurement point) of the input time sequence and the output time sequence of each structure vibration and strain tracking data frequency value may be selected to perform the fast fourier transform. Therefore, the data calculation amount is reduced, and the image data of the input and output aspects of the target equipment are subjected to fast Fourier transform and spectrum analysis through the equipment control angle, so that the vibration condition of the target equipment is more comprehensively determined.

Further, frequency domain data obtained by the fast fourier transform is analyzed. In one embodiment of the present application, analyzing the frequency domain characteristics of each set of sample data includes: obtaining a main component in the direct current spectrum data; the amplitude ratio and the phase difference of the principal components are used as the frequency domain characteristic parameters of the corresponding sample data.

Specifically, the main components of the input time sequence and the output time sequence of the structural vibration and the strain tracking data after fast Fourier transformation are respectively taken out, and finally the amplitude ratio and the phase difference of the main components of the structural vibration and the strain tracking data are used as the frequency domain characteristics of the algorithm under the frequency values of the structural vibration and the strain tracking data.

Step S104, analyzing the vibration condition of the target equipment based on the DC-free frequency spectrum data, analyzing the transverse vibration characteristic of the target equipment based on the frequency domain characteristic, acquiring the structural vibration parameter and the strain parameter of the target equipment, and repeatedly carrying out tracking analysis on the target parameter according to a preset period, wherein the target parameter comprises the structural vibration parameter and the strain parameter.

Specifically, the vibration condition of the target equipment in the stable working state is obtained through statistics by combining the DC-removed frequency spectrums corresponding to the image data acquired by each image information acquisition equipment, wherein the vibration condition comprises parameters such as strain amplitude and the like. For example, the dc-removed spectrum data corresponding to each group of data is analyzed, and the sample data with the same frequency domain characteristics is calculated and analyzed to obtain the strain amplitude corresponding to the sample data.

Further, by means of the transverse vibration characteristic analysis method of the machine vision-based structural vibration and strain related equipment, structural vibration and strain parameters of the target equipment can be calculated.

For example, when the target device is a lifting device of a wind turbine generator, the lifting device may be subjected to a lateral vibration characteristic analysis by the following formula:

wherein f _n For the natural vibration frequency of the lifting equipment, T is the tension at two ends of the lifting equipment, L is the length of the steel wire suspension rope, n is the order of the natural frequency, n is 1, E is the elastic modulus of the steel wire suspension rope, I is the moment of inertia of the steel wire suspension rope, and ρ is the linear density of the steel wire suspension rope.

It should be noted that, for the whole lifting system of the wind turbine generator, the vibration of the equipment such as the hoisting machine is mainly dependent on the natural vibration frequency of the equipment and the operation speed of the lifting system. The sensor can detect each parameter related to the hoisting machine in the formula to calculate the natural frequency, compare and verify the frequency data obtained by frequency domain characteristic analysis in the embodiment, determine the transformation condition of the natural frequency after the parameters such as the suspension rope in the hoisting machine are changed in practical application, and analyze the change of strain parameters such as the change of the length of the steel wire suspension rope.

In summary, according to the machine vision-based target tracking analysis method in the embodiment of the present application, vibration detection is performed on relevant devices of a wind turbine generator set by using machine vision, and design and model selection are performed on the hardware modules and the software processing modules. The method is based on machine vision unit equipment vibration monitoring, can accurately acquire vibration signals of related unit equipment, and acquires various target parameters such as structural vibration parameters, strain parameters and the like. The method has the advantages that the hardware module used in the practical application is small in size, small in occupied area, stable in system operation, small in data acquisition error, accurate in monitoring result and low in misjudgment rate, the vibration monitoring requirement of related unit equipment is met to the maximum extent, and the method is beneficial to guaranteeing the safety in the operation process of the later unit equipment. In addition, the image preprocessing module is arranged in the method, and the acquired unit equipment images are preprocessed, so that the definition of the acquired images is increased in the subsequent use process, and meanwhile, the processing efficiency and the accuracy in the subsequent analysis processing process of the images are accelerated, so that the running speed of the whole tracking analysis process is improved.

In order to implement the above embodiment, the present application further provides a machine vision-based target tracking analysis system, and fig. 4 is a schematic structural diagram of the machine vision-based target tracking analysis system according to the embodiment of the present application, as shown in fig. 4, where the system includes an acquisition module 100, a preprocessing module 200, a transformation module 300, and a tracking analysis module 400.

The collection module 100 is configured to install a plurality of image information collection devices at a target device to be tracked and analyzed in the wind turbine, and collect multiple groups of image data of the target device by adjusting each image information collection device after setting parameters.

The preprocessing module 200 is configured to normalize the multiple sets of image data, and align the normalized image data through the trained image alignment network.

The transformation module 300 is configured to select sample data with a preset duration from each set of processed image data, perform fast fourier transform on each set of sample data to obtain dc-removed spectrum data corresponding to the sample data, and analyze frequency domain characteristics of each set of sample data.

The tracking analysis module 400 is configured to analyze a vibration condition of the target device based on the dc-removed spectrum data, perform a transverse vibration characteristic analysis on the target device based on the frequency domain characteristic, obtain a structural vibration parameter and a strain parameter of the target device, and repeat the tracking analysis of the target parameter according to a preset period, where the target parameter includes the structural vibration parameter and the strain parameter.

Optionally, in one embodiment of the present application, the preprocessing module 200 is specifically configured to: : for each group of image data, respectively calculating the average value of the image gray levels, and subtracting the average value from the gray level of each image; and calculating the standard deviation of each group of image data, calculating the sum value of the standard deviation and a preset constant, and dividing the gray scale of each image by the sum value.

Optionally, in an embodiment of the present application, the preprocessing module 200 is further configured to: performing feature point detection and feature point rough matching on the normalized image data; combining the feature point rough matching result and an image alignment network trained based on an unsupervised deep learning algorithm to obtain a homography matrix for pixel mapping; all pixel points in each orthogonal polarized image are mapped into a single polarized image based on the homography matrix.

Optionally, in an embodiment of the present application, the preprocessing module 200 is further configured to: acquiring a training data set, inputting training images in the training data set into a pre-built image alignment network to acquire a predicted alignment image output by the network; acquiring a target image corresponding to the training image, and acquiring first edge characteristic information of a predicted alignment image and second edge characteristic information of the target image; edge feature loss between the first edge feature information and the second edge feature information is calculated and parameter values of the image alignment network are updated based on the edge feature loss.

Optionally, in one embodiment of the present application, the transformation module 300 is specifically configured to: removing a direct current component in the sample data and a periodic sinusoidal signal caused by rotation of the wind turbine generator; substituting the sample data excluding the interference signals into a preset discrete Fourier transform formula to obtain the frequency spectrum data corresponding to each group of sample data.

Optionally, in one embodiment of the present application, the transformation module 300 is specifically configured to: obtaining a main component in the direct current spectrum data; the amplitude ratio and the phase difference of the principal components are used as the frequency domain characteristic parameters of the corresponding sample data.

Optionally, in one embodiment of the present application, the tracking analysis module 400 is specifically configured to perform a lateral vibration characteristic analysis of the lifting device by the following formula:

wherein f _n For the natural vibration frequency of the lifting equipment, T is the tension at two ends of the lifting equipment, L is the length of the steel wire suspension rope, n is the order of the natural frequency, n is 1, E is the elastic modulus of the steel wire suspension rope, I is the moment of inertia of the steel wire suspension rope, and ρ is the linear density of the steel wire suspension rope.

It should be noted that the foregoing explanation of the embodiment of the machine vision-based target tracking analysis method is also applicable to the system of this embodiment, and will not be repeated here.

In summary, in the machine vision-based target tracking analysis system according to the embodiment of the present application, vibration detection is performed on relevant devices of a wind turbine generator set by using machine vision, and a hardware module and a software processing module are designed and selected. The system is based on machine vision unit equipment vibration monitoring, can accurately acquire vibration signals of related unit equipment, and acquires various target parameters such as structural vibration parameters, strain parameters and the like. The method has the advantages that the hardware module used in the practical application is small in size, small in occupied area, stable in system operation, small in data acquisition error, accurate in monitoring result and low in misjudgment rate, the vibration monitoring requirement of related unit equipment is met to the maximum extent, and the method is beneficial to guaranteeing the safety in the operation process of the later unit equipment. In addition, the system is provided with the image preprocessing module, and the acquired unit equipment images are preprocessed, so that the definition of the acquired images is increased in the subsequent use process, and meanwhile, the processing efficiency and accuracy in the subsequent analysis processing process of the images are accelerated, so that the running speed of the whole tracking analysis process is improved.

In order to implement the above embodiments, the present application further proposes a non-transitory computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the machine vision-based target tracking analysis method according to any one of the above embodiments.

In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless explicitly defined otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.

Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.

Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. Although embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives, and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.

Claims (10)

1. The target tracking analysis method based on machine vision is characterized by comprising the following steps of:

installing a plurality of image information acquisition devices at target devices to be tracked and analyzed in a wind turbine generator, and acquiring a plurality of groups of image data of the target devices by adjusting each image information acquisition device after setting parameters;

normalizing the plurality of groups of image data, and aligning the normalized image data through a trained image alignment network;

selecting sample data with preset time length from each group of processed image data, performing fast Fourier transform on each group of sample data to obtain DC-removed frequency spectrum data corresponding to the sample data, and analyzing the frequency domain characteristics of each group of sample data;

Analyzing the vibration condition of the target equipment based on the DC-free frequency spectrum data, analyzing the transverse vibration characteristic of the target equipment based on the frequency domain characteristic, acquiring the structural vibration parameter and the strain parameter of the target equipment, and repeatedly carrying out tracking analysis on the target parameter according to a preset period, wherein the target parameter comprises the structural vibration parameter and the strain parameter.

2. The machine vision based target tracking analysis method of claim 1, wherein normalizing the plurality of sets of image data comprises:

for each group of image data, respectively calculating the average value of the image gray levels, and subtracting the average value from the gray level of each image;

and calculating the standard deviation of each group of image data, calculating the sum value of the standard deviation and a preset constant, and dividing the gray scale of each image by the sum value.

3. The machine vision based target tracking analysis method according to claim 1, wherein the aligning the normalized image data includes:

performing feature point detection and feature point rough matching on the normalized image data;

Combining the feature point rough matching result and an image alignment network trained based on an unsupervised deep learning algorithm to obtain a homography matrix for pixel mapping;

all pixel points in each orthogonal polarized image are mapped into a single polarized image based on the homography matrix.

4. The machine vision based target tracking analysis method of claim 1, wherein training the image alignment network comprises:

acquiring a training data set, and inputting training images in the training data set into a pre-built image alignment network to acquire a predicted alignment image output by the network;

acquiring a target image corresponding to the training image, and acquiring first edge characteristic information of the prediction alignment image and second edge characteristic information of the target image;

and calculating edge characteristic loss between the first edge characteristic information and the second edge characteristic information, and updating parameter values of an image alignment network based on the edge characteristic loss.

5. The machine vision based target tracking analysis method according to claim 1, wherein performing a fast fourier transform on each set of the sample data to obtain dc-removed spectrum data corresponding to the sample data, comprises:

Removing a direct current component in the sample data and a periodic sinusoidal signal caused by rotation of the wind turbine generator;

substituting the sample data excluding the interference signals into a preset discrete Fourier transform formula to obtain the frequency spectrum data corresponding to each group of sample data.

6. The machine vision based target tracking analysis method of claim 5, wherein said analyzing the frequency domain characteristics of each set of said sample data comprises:

acquiring a main component in the DC-removed frequency spectrum data;

and taking the amplitude ratio and the phase difference of the main components as frequency domain characteristic parameters of corresponding sample data.

7. The machine vision-based target tracking analysis method according to claim 1, wherein the target device comprises a lifting device of the wind turbine generator, and the lifting device is subjected to transverse vibration characteristic analysis by the following formula:

wherein f _n For the natural vibration frequency of the lifting equipment, T is the tension at two ends of the lifting equipment, L is the length of the steel wire suspension rope, n is the order of the natural frequency, n is 1, E is the elastic modulus of the steel wire suspension rope, I is the moment of inertia of the steel wire suspension rope, and ρ is the linear density of the steel wire suspension rope.

8. A machine vision-based target tracking analysis system, comprising the following modules:

The system comprises an acquisition module, a tracking module and a control module, wherein the acquisition module is used for installing a plurality of image information acquisition devices at target devices to be tracked and analyzed in a wind turbine, and acquiring a plurality of groups of image data of the target devices by adjusting each image information acquisition device after setting parameters;

the preprocessing module is used for carrying out normalization processing on the plurality of groups of image data and carrying out alignment processing on the normalized image data through a trained image alignment network;

the transformation module is used for selecting sample data with preset duration from each group of processed image data, performing fast Fourier transformation on each group of sample data to obtain DC-removed frequency spectrum data corresponding to the sample data, and analyzing the frequency domain characteristics of each group of sample data;

the tracking analysis module is used for analyzing the vibration condition of the target equipment based on the DC-free frequency spectrum data, analyzing the transverse vibration characteristic of the target equipment based on the frequency domain characteristic, acquiring the structural vibration parameter and the strain parameter of the target equipment, and repeatedly carrying out tracking analysis on the target parameter according to a preset period, wherein the target parameter comprises the structural vibration parameter and the strain parameter.

9. The machine vision-based target tracking analysis system of claim 8, wherein the preprocessing module is specifically configured to:

for each group of image data, respectively calculating the average value of the image gray levels, and subtracting the average value from the gray level of each image;

and calculating the standard deviation of each group of image data, calculating the sum value of the standard deviation and a preset constant, and dividing the gray scale of each image by the sum value.

10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the machine vision based target tracking analysis method according to any of claims 1-7.