CN110674697A - Filtering method and device and related product - Google Patents
Filtering method and device and related product Download PDFInfo
- Publication number
- CN110674697A CN110674697A CN201910815821.3A CN201910815821A CN110674697A CN 110674697 A CN110674697 A CN 110674697A CN 201910815821 A CN201910815821 A CN 201910815821A CN 110674697 A CN110674697 A CN 110674697A
- Authority
- CN
- China
- Prior art keywords
- correlation peak
- filtering
- correlation
- peak
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/15—Correlation function computation including computation of convolution operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
- G06F2218/10—Feature extraction by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a filtering method, a filtering device and a related product. The filtering method comprises the following steps: acquiring one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video, wherein the correlation peaks are frequency domain signals; determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak; and carrying out filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak. By selecting a proper filtering strategy for filtering according to the position of each correlation peak and the corresponding frequency band, noise information in each correlation peak can be reduced, the signal-to-noise ratio of a frequency signal obtained from a detection video is higher, and the running condition of the device to be detected obtained according to the detection video is more accurate.
Description
Technical Field
The invention relates to the technical field of detection, in particular to a filtering method, a filtering device and a related product.
Background
In the related art, vibration may reflect an operation status of some mechanical structures, a video of a vibrating device during operation may be captured, a vibration signal may be extracted from the video, and an operation status of the device may be obtained according to the vibration signal.
Disclosure of Invention
The embodiment of the invention provides a filtering method, a filtering device and a related product.
In a first aspect, an embodiment of the present invention provides a filtering method, including:
acquiring one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video, wherein the correlation peaks are frequency domain signals;
determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band of each correlation peak;
and carrying out filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak.
In some embodiments, the filtering policy includes a filtering bandwidth, and the determining the filtering policy corresponding to each correlation peak according to the position of each correlation peak in the detected video and the frequency band corresponding to the correlation peak includes:
determining an initial filtering bandwidth corresponding to each correlation peak according to the frequency band corresponding to each correlation peak;
determining a bandwidth adjustment coefficient corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video;
and taking the product of the initial filter bandwidth corresponding to each correlation peak and the bandwidth adjustment coefficient as the filter bandwidth corresponding to each correlation peak.
In some embodiments, the determining, according to the position of each correlation peak in the detection video, a bandwidth adjustment coefficient corresponding to the correlation peak includes:
and acquiring a preset bandwidth adjustment coefficient corresponding to a region where the corresponding position of each correlation peak in the detection video is located, and taking the preset bandwidth adjustment coefficient as a bandwidth adjustment coefficient corresponding to the correlation peak.
In some embodiments, before determining the filtering strategy corresponding to each correlation peak according to the position of each correlation peak in the detected video and the frequency band corresponding to the correlation peak, the filtering method further includes:
performing dimensionality reduction on each correlation peak by using a principal component decomposition method;
the determining the filtering strategy corresponding to each correlation peak according to the position of each correlation peak in the detection video and the frequency band corresponding to each correlation peak comprises:
and determining a filtering strategy corresponding to each correlation peak according to the position of each correlation peak after the dimension reduction processing in the detection video and the frequency band corresponding to the correlation peak.
In some embodiments, before performing the dimensionality reduction on each correlation peak by using the principal component decomposition method, the filtering method further includes:
comparing the peak value of each correlation peak with a preset peak value range, and respectively judging whether the peak value of each correlation peak is in the preset peak value range;
the dimension reduction processing of each correlation peak by using the principal component decomposition method comprises the following steps:
and carrying out dimension reduction processing on the correlation peak of which the peak value is within a preset peak value range.
In some embodiments, the filtering the correlation peaks according to the filtering strategy corresponding to the correlation peaks includes:
and carrying out filtering processing on each correlation peak in an interpolation filtering mode according to the filtering strategy corresponding to each correlation peak.
In a second aspect, the present invention further provides a filtering apparatus, including:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video, and the correlation peaks are frequency domain signals;
a filtering strategy determining module, configured to determine a filtering strategy corresponding to each correlation peak according to a position of each correlation peak in the detected video and a frequency band corresponding to the correlation peak;
and the filtering module is used for filtering each correlation peak according to the filtering strategy corresponding to each correlation peak.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor, a memory, and an information recommendation program stored on the memory and executable by the processor, where the information recommendation program, when executed by the processor, implements the instructions of the steps in the filtering method according to any one of the above embodiments.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a filtering program is stored, wherein the filtering program, when executed by a processor, implements the filtering method according to any one of the above.
In a fifth aspect, the present invention further provides a detection apparatus for detecting an operating condition of a device to be detected, where the detection apparatus includes the filter device of the foregoing embodiment or the electronic device of the foregoing embodiment.
The filtering method of the embodiment of the invention obtains one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video; determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak; and carrying out filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak. By selecting a proper filtering strategy for filtering according to the position of each correlation peak and the corresponding frequency band, noise information in each correlation peak can be reduced, the signal-to-noise ratio of a state change signal obtained from a detection video is higher, and the running condition of the device to be detected obtained according to the detection video is more accurate.
Drawings
Some drawings to which embodiments of the present invention relate will be described below.
FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the invention;
FIG. 2 is a flow chart illustrating a filtering method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a process of amplifying a detection video involved in detecting an operation status of a device to be detected by using the filtering method according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart illustrating a filtering method according to an embodiment of the present invention;
FIG. 5 is a schematic flow chart illustrating a filtering method according to an embodiment of the present invention;
fig. 6 is a schematic flow chart of a filtering method according to another embodiment of the invention.
Detailed Description
The embodiments of the present invention will be described below with reference to the drawings.
Referring to fig. 1, fig. 1 is a schematic diagram of a hardware structure of an electronic device 100 according to an embodiment of the invention. The electronic device 100 comprises a processor 101, a memory 102, an input output interface 103, and one or more programs stored in the memory 102 and configured to be executed by the processor 101, the programs comprising instructions to perform the steps of the filtering method of any of the following embodiments. The electronic apparatus 100 may be a server device or a terminal device. The memory 102 may be a high-speed RAM memory, or may be a non-volatile memory (e.g., a disk memory), and the memory 102 may optionally be a storage device independent of the processor 101. The input/output interface 103 may optionally include a USB interface, a standard wired interface, a wireless interface (e.g., WI-FI interface), etc.
Referring to fig. 2, fig. 2 is a flow chart illustrating a filtering method according to an embodiment of the present invention, which includes, but is not limited to, the following steps:
01. acquiring one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video, wherein the correlation peaks are frequency domain signals;
the filtering method of the embodiment of the invention can be used for detecting the vibration condition of the device to be detected, but is not limited to the detection. The vibration condition of the device to be detected may reflect the operating condition of the device to be detected. The detection video can be a video obtained by shooting the device to be detected by the imaging module. The vibration information of the device to be detected can be extracted from the detection video by analyzing the detection video, and then the vibration information is analyzed to obtain the running condition of the device to be detected.
In a preferred embodiment, the detection video is a vibration amplification video obtained by amplifying vibration in a video obtained by shooting the device to be detected by the imaging module by using an Euler algorithm. The devices to be detected include, but are not limited to, various types of mechanical equipment, building structures, and the like. After the detection video is obtained, phase correlation calculation is carried out on a frame sequence corresponding to the detection video to obtain a cross power spectrum, and the cross power spectrum comprises one or more correlation peaks. It can be understood that the correlation peaks are frequency domain signals, each correlation peak may reflect a state change intensity of a certain frequency band at a certain position in the detected video, and a cross power spectrum obtained by performing phase correlation calculation on a frame sequence corresponding to the detected video may be understood as extracting state change information in a video picture from the detected video. The state change information comprises vibration information and other noise information, and the vibration information can reflect the operation condition of the device to be detected. By the filtering method provided by the embodiment of the invention, each correlation peak in the cross power spectrum is subjected to filtering processing, so that noise information in each correlation peak is reduced, and the operation condition of the device to be detected obtained according to the detection video is more accurate.
02. Determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak;
the corresponding position of each correlation peak in the detection video can be understood as the position in the detection video corresponding to the state change reflected by the correlation peak. For example, the brightness of a position in the detected video periodically changes with time, and one or more correlation peaks obtained by performing phase correlation calculation on the corresponding frame sequence of the detected video include at least one correlation peak that can represent the brightness change of the position.
Due to the fact that in the detection video, the vibration at different positions has different effects on detecting the operation condition of the device to be detected. For example, the vibration at the edge position in the frame of the detection video has a smaller effect on detecting the operation condition of the device to be detected than the vibration at the middle position, and then the filtering strategy of each correlation peak can be determined based on the corresponding position of each correlation peak in the detection video. The frequency ranges (i.e. frequency bands) corresponding to the correlation peaks are different, and the filtering strategy of each correlation peak is determined according to the frequency band corresponding to the correlation peak, so that the filtering strategy is more adaptive to each correlation peak, and the extraction of useful state change signals from the correlation peaks is facilitated. The filtering strategy may include, for example, but not limited to, the bandwidth of the filter, the type of filter, and the like.
03. And carrying out filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak.
And after the filtering strategy of each correlation peak is obtained, filtering each correlation peak according to the corresponding filtering strategy so as to enable the signal-to-noise ratio of the signal obtained from the detection video to be higher. Preferably, the filtering process can be performed on each correlation peak in an interpolation filtering manner according to the filtering strategy corresponding to each correlation peak, so that the filtering effect is better.
The filtering method of the embodiment of the invention obtains one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video; determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak; and carrying out filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak. By selecting a proper filtering strategy for filtering according to the position of each correlation peak and the corresponding frequency band, noise information in each correlation peak can be reduced, the signal-to-noise ratio of a state change signal obtained from a detection video is higher, and the running condition of the device to be detected obtained according to the detection video is more accurate.
When detecting the operation status of the device to be detected, it is necessary to extract the vibration information from the detection video including the picture of the device during operation. As shown in fig. 3, the frame sequence in the detected video may be converted from RGB color space to YIQ color space, and the luminance information and the chrominance information of the video frame may be separated. The conversion relation between RGB and YIQ is as follows:
Y＝0.299*R+0.587*G+0.114*B；
I＝0.596*R–0.275*G–0.321*B；
Q＝0.212*R-0.523*G+0.311*B。
and then keeping I, Q channels unchanged, carrying out FFT operation on the Y channel, and amplifying the video data line by using an Euler motion amplification algorithm. The method specifically comprises the following steps: carrying out complex controllable pyramid spatial domain decomposition on the Y-channel image after FFT to obtain a pyramid structure consisting of a plurality of sub-images with different spatial resolutions; and performing time domain band-pass filtering processing on each sub-image in the plurality of sub-images in the pyramid structure to obtain a conversion signal corresponding to the target frequency band. It will be appreciated that in video pictures, vibrations may be reflected by the luminance of a sequence of video frames, and vibration information in the detected video may be obtained by analyzing the Y channel in the detected video. Then amplifying a conversion signal corresponding to the target frequency band obtained after time domain band-pass filtering, extracting interested motion information, and performing complex steerable pyramid reconstruction on the interested motion information to obtain an amplified Y-channel image; and finally, adding the reconstructed Y-channel image and the original I, Q-channel image, and converting the image into an RGB color space to obtain an output video.
And then, calculating the cross power spectrum among the frame sequences by adopting a phase correlation algorithm on the frame sequences after the video motion amplification processing. The phase correlation algorithm calculates the cross-cross power spectrum using the following formula.
In the above formula, Fa is the Fourier of the a-frame imageThe result of the transformation is a transformation,
for the conjugate signal of the fourier transform of the b frame image, the lower side of the divisor is the modulus of the correlation product of the two fourier transformed signals. And R is the cross-power spectrum (including frequency-domain noise) of the calculation result of the step.
Finally, according to the filtering method of the embodiment of the invention, each correlation peak in the cross power spectrum is subjected to filtering processing, so that the signal-to-noise ratio of the useful signal is improved, and the signal-to-noise ratio of the useful signal is higher due to the information extracted from the detection video containing the picture when the detection device operates.
Specifically, after the image frame sequence of the target vibration video is subjected to pyramid decomposition, a pyramid-shaped structure is obtained. The pyramid-shaped structure comprises a plurality of layers of sub-images, the resolution of the images is reduced from top to bottom, and the spatial frequency is reduced. The time domain band-pass filtering processing is carried out on each layer of image, so that a target frequency band is obtained, the resolution of the sub-image of the target frequency band can clearly express the motion characteristics of the image, and the overlarge calculation amount caused by the overhigh resolution is avoided. Therefore, the frequency bands of each sub-image at different spatial frequencies are obtained from bottom to top according to the pyramid structure, and are compared with the standard frequency bands according to the obtaining sequence, when the frequency band of a certain layer of sub-image is successfully matched with the standard frequency band, the frequency band obtaining and matching of the spatial frequencies of one or more layers of sub-images above the layer of sub-image are not needed, and the efficiency of time domain band-pass filtering is improved. In addition, the maximum number of levels of pyramidal decomposition determines: log2(min (xres, yres)), where xres is the width pixel value of the image and yres is the height pixel value of the image.
Further, performing time domain bandpass filtering processing on each sub-image in the plurality of sub-images in the pyramid structure to obtain a conversion signal corresponding to a target frequency band, specifically comprising:
acquiring the total layer number N of the pyramid structure, wherein the pyramid structure is sequentially increased from bottom to top, and the resolution of corresponding sub-images is sequentially increased;
grouping a plurality of sub-images in the pyramid structure, wherein the first group comprises sub-images corresponding to 2N +1 layers, the second group comprises sub-images corresponding to 2(N +1) layers, N is an integer greater than or equal to 0, and N is greater than or equal to max (2(N +1),2N + 1);
performing time domain band-pass filtering processing on the first group of sub-images by using a first processor according to the number of layers from small to large, performing time domain band-pass filtering processing on the second group of sub-images by using a second processor according to the number of layers from small to large, wherein the time domain band-pass filtering processing is to match frequency bands of different spatial frequencies corresponding to the sub-images with a standard frequency band, and the first processor and the second processor are independently operated processors;
when the first processor or the second processor determines that the frequency band is successfully matched with the standard frequency band, determining that the frequency band is a target frequency band;
acquiring a subimage corresponding to a target frequency band as a first target subimage, and acquiring a layer of subimage above the first target subimage as a second target subimage;
and determining the first target sub-image and the second target sub-image as transformation signals corresponding to the target frequency bands.
Specifically, after a frame sequence composed of multi-frame images of the target vibration video is subjected to spatial pyramid decomposition, the total number N of layers of the pyramid structure can be obtained, and the resolution of 1-N layers of corresponding sub-images of the pyramid structure is sequentially increased from bottom to top. Then, grouping the N layers of sub-images, including a base group and an even group, where the base group uses a first processor to perform time-domain bandpass filtering processing according to the sequence of 1,3,5 … 2N +1, the even group uses a second processor to perform time-domain bandpass filtering processing according to the sequence of 2,4,6, … 2(N +1), the two processors may start operating simultaneously or have a certain processing time interval, and in addition, the time-domain bandpass filtering processing is to acquire a target frequency band, and the acquisition method may be to match frequency bands of different spatial frequencies corresponding to the sub-images with a standard frequency band. And when the first processor or the second processor determines that the frequency band is successfully matched with the standard frequency band, determining the frequency band as a target frequency band.
And then, acquiring a sub image of a layer above the sub image corresponding to the target frequency band as a second target sub image, and performing subsequent amplification transformation by taking the first target sub image and the second target sub image as transformation signals corresponding to the target frequency band. Therefore, the frame image corresponding to the target vibration video can be amplified more accurately, more accurate motion information can be obtained, and meanwhile, the operation amount which is increased when the image with higher resolution ratio is amplified is avoided.
In the embodiment of the application, the sub-images corresponding to the pyramid structure are grouped, then the time domain band-pass filtering processing is performed on the sub-images in different groups through two independently operating processors, the filtering processing efficiency can be improved, meanwhile, after the sub-image in the layer corresponding to the target frequency band is obtained and used as the first target sub-image, the sub-image in the upper layer is obtained and used as the second target sub-image, and then the first target sub-image and the second target sub-image are used as the conversion signals corresponding to the target frequency band, so that more accurate amplification motion information can be obtained, and meanwhile, the operation amount which needs to be increased when the higher resolution image is amplified is avoided.
Referring to fig. 4, based on the foregoing embodiments, in some embodiments, the filtering strategy includes a filtering bandwidth, and the determining the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detected video and the frequency band corresponding to each correlation peak includes, but is not limited to, the following steps:
021. determining an initial filtering bandwidth corresponding to each correlation peak according to the frequency band corresponding to each correlation peak;
it can be understood that the correlation peak obtained by performing the phase correlation calculation on the frame sequence corresponding to the detected video includes a plurality of similar state change information of a plurality of pixel points with similar positions in the picture corresponding to the detected video, so that the state change of the positions of the plurality of pixel points can be reflected. Then, the state change conditions at different positions in the picture corresponding to the detected video are different, and the frequency distribution of the corresponding correlation peaks is also different. If the filtering is performed with the same filtering bandwidth, the filtering bandwidth needs to be set to a larger value in order to avoid filtering the useful signal, but this may result in the noise signal not being filtered. In the filtering method, the initial filtering bandwidth corresponding to each correlation peak is determined according to the frequency band corresponding to each correlation peak, the frequency band corresponding to each correlation peak is positively correlated with the initial filtering bandwidth, and the larger the frequency band corresponding to each correlation peak is, the wider the corresponding initial filtering bandwidth is, so that the filtering bandwidth can be better adapted to each correlation peak, not only can noise signals be effectively filtered, but also useful signals can be prevented from being filtered, and the filtering effect can be better.
For example, B × (1-a) may be used as the initial filtering bandwidth, B is the frequency band corresponding to each correlation peak, a may be determined according to actual requirements, a may be set to a larger value when a better drying effect needs to be obtained, and a may be set to a smaller value when more useful signals need to be obtained. Of course, in other embodiments, the setting method of the initial filtering bandwidth is not limited to the above example, and is not limited herein.
022. Determining a bandwidth adjustment coefficient corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video;
the effects of detecting the vibration at different positions in the video on detecting the operation condition of the device to be detected are different in size. The bandwidth adjustment factor corresponding to each correlation peak can be determined according to the corresponding position of each correlation peak in the detected video. And the initial filtering bandwidth is adjusted by using the bandwidth adjustment coefficient, so that the filtering bandwidth is more reasonable.
For example, the picture corresponding to the detection video may be divided into a plurality of regions according to the magnitude of the effect of the vibration of the corresponding position of the correlation peak in the detection video on the operation condition of the device to be detected, and a preset bandwidth adjustment coefficient may be set for each region, so that when the bandwidth adjustment coefficient is determined in step 022, the preset bandwidth adjustment coefficient corresponding to the region where the corresponding position of each correlation peak in the detection video is located may be directly obtained, and the preset bandwidth adjustment coefficient may be used as the bandwidth adjustment coefficient corresponding to the correlation peak. Therefore, the bandwidth adjustment coefficient can be obtained quickly and accurately.
Furthermore, a user can set a bandwidth weighting coefficient for each region in a picture corresponding to a detected video according to the structure of the device to be detected, and then the product of the preset bandwidth adjustment coefficient and the bandwidth weighting coefficient is used as the related bandwidth adjustment coefficient of the corresponding region, so that the obtained filtering bandwidth of each related peak is more reasonable. For example, the vibration of some key parts in the device to be detected can reflect the operation condition of the device to be detected more than other parts, and then the bandwidth weighting coefficient of the corresponding region of the key part in the picture of the detection video can be set to be a larger value, so that the filtering bandwidth is larger, and more useful information can be extracted.
023. And taking the product of the initial filter bandwidth corresponding to each correlation peak and the bandwidth adjustment coefficient as the filter bandwidth corresponding to each correlation peak.
The obtained filtering bandwidth comprehensively considers the corresponding position of the correlation peak in the detection video and the frequency band corresponding to the correlation peak, so that the filtering bandwidth is more adaptive to each correlation peak, and a useful state change signal is extracted from each correlation peak.
It should be noted that the method for calculating the filtering bandwidth is not limited to the above method, and in other embodiments, an appropriate calculation method may be selected according to the frequency band corresponding to each correlation peak and the corresponding position of each correlation peak in the detected video to determine the filtering bandwidth.
Referring to fig. 5, based on the above embodiments, in some embodiments, the filtering method further includes the steps of:
04. performing dimensionality reduction on each correlation peak by using a principal component decomposition method;
024. and determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak after the dimension reduction processing in the detection video and the frequency band corresponding to the correlation peak.
After the dimension of the related peak data is reduced, the related peaks after the dimension reduction processing are calculated to obtain a filtering strategy corresponding to each related peak, so that the calculated amount in the filtering process can be reduced, and the filtering efficiency is improved.
Further, referring to fig. 6, before step 04, the filtering method further includes:
05. comparing the peak value of each correlation peak with a preset peak value range, and respectively judging whether the peak value of each correlation peak is in the preset peak value range;
step 05 is performed after step 01 and before step 04. When the imaging module shoots a detection video of the device to be detected, the small change of some pixel points can also form a related peak, but the change degree of the frequency is not large enough, the peak value of the related peak is small, the signals have little or no effect on analyzing the operation condition of the device to be detected, and the signals can be understood as noise signals. Then, a preset peak value range can be preset, and the peak value of each correlation peak is compared with the preset peak value range during filtering, so that some common noise signals can be filtered.
041. and carrying out dimension reduction processing on the correlation peak of which the peak value is within a preset peak value range.
The correlation peak of the peak value in the preset peak value range is subjected to dimension reduction processing, and the correlation peak which is not in the preset peak value range is filtered, so that on one hand, the signal-to-noise ratio of the state change signal obtained from the detected video can be improved, more useful signals are obtained, and on the other hand, part of noise signals are removed, so that the subsequent filtering calculation amount can be reduced, and the filtering efficiency is improved.
Further, the vibration of the device to be detected is a periodic reciprocating motion, and the state change caused by the vibration is also periodic. Although a lot of noise information may cause the state change of each pixel in the detected video, the state change caused by the noise is often not periodic, for example, in an outdoor scene, the state change is caused by the brightness change of an outdoor optical fiber. And when the operation condition of the device to be detected is analyzed according to the vibration of the device to be detected, the periodic vibration can be used for reflecting the operation condition of the device to be detected, because the aperiodic vibration is often caused by the external environment instead of the device to be detected, and the part of aperiodic signals can not be used for analyzing the operation condition of the device to be detected. Then, after filtering processing is performed on each correlation peak according to a filtering strategy corresponding to each correlation peak, inverse fourier transform can be performed on the correlation peaks, the frequency domain signals are converted into time domain signals, a plurality of state change signals are obtained, and then non-periodic signals in the state change signals are removed, so that noise is further removed, and the operation condition of the device to be detected obtained according to the detection video is more accurate.
An embodiment of the present invention further provides a filtering apparatus, including:
the first acquisition module is used for acquiring one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video, wherein the correlation peaks are frequency domain signals;
the filtering strategy determining module is used for determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak;
and the filtering module is used for filtering each correlation peak according to the filtering strategy corresponding to each correlation peak.
The filtering device of the embodiment of the invention obtains one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video, wherein the correlation peaks are frequency domain signals; determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak; and carrying out filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak. By selecting a proper filtering strategy for filtering according to the position of each correlation peak and the corresponding frequency band, noise information in each correlation peak can be reduced, the signal-to-noise ratio of a state change signal obtained from a detection video is higher, and the running condition of the device to be detected obtained according to the detection video is more accurate. It should be noted that the explanations and technical effects of the embodiments of the filtering method described above are also applicable to the filtering apparatus of this embodiment, and are not described herein again to avoid redundancy.
In some embodiments, the filtering policy includes a filtering bandwidth filtering policy determination module comprising:
the initial filtering bandwidth determining unit is used for determining the initial filtering bandwidth corresponding to each correlation peak according to the frequency band corresponding to each correlation peak;
the bandwidth adjustment coefficient determining unit is used for determining the bandwidth adjustment coefficient corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video;
and the filter bandwidth determining unit is used for taking the product of the initial filter bandwidth corresponding to each correlation peak and the bandwidth adjusting coefficient as the filter bandwidth corresponding to each correlation peak.
In some embodiments, the bandwidth adjustment coefficient determining unit is configured to obtain a preset bandwidth adjustment coefficient corresponding to a region where a corresponding position of each correlation peak in the detected video is located, and use the preset bandwidth adjustment coefficient as the bandwidth adjustment coefficient corresponding to the correlation peak.
In some embodiments, the filtering apparatus further comprises:
the dimensionality reduction module is used for carrying out dimensionality reduction processing on each correlation peak by using a principal component decomposition method;
and the filtering strategy determining module is used for determining the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak after the dimension reduction processing in the detection video and the frequency band corresponding to the correlation peak.
In some embodiments, the filtering apparatus further comprises:
the judging module is used for comparing the peak value of each correlation peak with a preset peak value range and respectively judging whether the peak value of each correlation peak is in the preset peak value range or not;
and the dimension reduction module is used for carrying out dimension reduction processing on the correlation peak of the peak value in a preset peak value range.
In some embodiments, the filtering module is configured to perform filtering processing on each correlation peak by performing interpolation filtering according to a filtering strategy corresponding to each correlation peak.
The function implementation and technical effects of each module in the filtering apparatus correspond to each step in the filtering method embodiment, and the function and implementation processes are not described in detail herein.
The present invention further provides a computer-readable storage medium, on which a filter program is stored, wherein the filter program, when executed by a processor, implements the steps of the filtering method of any of the above embodiments.
The method for implementing the filtering program when executed and the corresponding technical effects may refer to various embodiments of the filtering method of the present invention, and are not described herein again.
The embodiment of the invention also provides detection equipment for detecting the running condition of the device to be detected, wherein the detection equipment comprises the filtering device or the electronic device of the embodiment.
The detection device of the embodiment of the invention obtains one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video; determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak; and carrying out filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak. By selecting a proper filtering strategy for filtering according to the position of each correlation peak and the corresponding frequency band, noise information in each correlation peak can be reduced, the signal-to-noise ratio of a state change signal obtained from a detection video is higher, and the running condition of the device to be detected obtained according to the detection video is more accurate.
When the detection equipment detects the running condition of the device to be detected, the imaging module can be used for shooting a detection video of the device to be detected when the device to be detected runs, and then state change information is extracted from the detection video containing pictures of the device to be detected when the device runs, wherein the state change information can reflect vibration information of the device to be detected, and the running condition of the device to be detected can be obtained according to the vibration information of the device to be detected. As shown in fig. 3, the frame sequence in the detected video may be converted from RGB color space to YIQ color space, and the luminance information and the chrominance information of the video frame may be separated. The conversion relation between RGB and YIQ is as follows:
Y＝0.299*R+0.587*G+0.114*B；
I＝0.596*R–0.275*G–0.321*B；
Q＝0.212*R-0.523*G+0.311*B。
and then keeping I, Q channels unchanged, carrying out FFT operation on the Y channel, and amplifying the video data line by using an Euler motion amplification algorithm. The method specifically comprises the following steps: and carrying out complex controllable pyramid space domain decomposition on the Y-channel image after the FFT. The images with different scales after the spatial domain decomposition of the Y channel are subjected to time domain band-pass filtering, so that it can be understood that in a video picture, vibration can be reflected by the brightness of a video frame sequence, and then vibration information in a detected video can be obtained by analyzing the Y channel in the detected video. (ii) a Then amplifying the interested motion information after time domain band-pass filtering, and performing complex steerable pyramid reconstruction on the interested motion information to obtain an amplified Y-channel image; and finally, adding the reconstructed Y-channel image and the original I, Q-channel image, and converting the image into an RGB color space to obtain an output video.
And then, calculating the cross power spectrum among the frame sequences by adopting a phase correlation algorithm on the frame sequences after the video motion amplification processing. The phase correlation algorithm calculates the cross-cross power spectrum using the following formula.
In the above equation, Fa is the fourier transform of the a-frame image,
for the conjugate signal of the fourier transform of the b frame image, the lower side of the divisor is the modulus of the correlation product of the two fourier transformed signals. And R is the cross-power spectrum (including frequency-domain noise) of the calculation result of the step. The cross-power spectrum comprises one or more correlation peaks, and the correlation peaks are frequency-domain signals. It will be appreciated that each correlation peak mayThe state change information in the video picture is extracted from the detection video by reflecting the state change of a certain position in the detection video and carrying out phase correlation calculation on a true sequence corresponding to the detection video to obtain a cross power spectrum. The state change information includes vibration information and other noise information. By the filtering method provided by the embodiment of the invention, each correlation peak in the cross power spectrum is subjected to filtering processing so as to reduce noise information in each correlation peak, so that the signal-to-noise ratio of a useful signal is higher in information extracted from a detection video containing a picture in the operation of a detection device.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., compact disk), or a semiconductor medium (e.g., solid state disk), among others. In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is merely a logical division, and the actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the indirect coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, and may be electrical or in other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage media may include, for example: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Claims (10)
1. A method of filtering, comprising:
acquiring one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video, wherein the correlation peaks are frequency domain signals;
determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band of each correlation peak;
and carrying out filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak.
2. The filtering method according to claim 1, wherein the filtering policy includes a filtering bandwidth, and the determining the filtering policy corresponding to each correlation peak according to the position of each correlation peak in the detected video and the frequency band corresponding to the correlation peak includes:
determining an initial filtering bandwidth corresponding to each correlation peak according to the frequency band corresponding to each correlation peak;
determining a bandwidth adjustment coefficient corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video;
and taking the product of the initial filter bandwidth corresponding to each correlation peak and the bandwidth adjustment coefficient as the filter bandwidth corresponding to each correlation peak.
3. The filtering method according to claim 2, wherein the determining the bandwidth adjustment coefficient corresponding to each correlation peak according to the corresponding position of the correlation peak in the detected video comprises:
and acquiring a preset bandwidth adjustment coefficient corresponding to a region where the corresponding position of each correlation peak in the detection video is located, and taking the preset bandwidth adjustment coefficient as a bandwidth adjustment coefficient corresponding to the correlation peak.
4. The filtering method according to claim 1, wherein before determining the filtering strategy corresponding to each correlation peak according to the position of each correlation peak in the detected video and the frequency band corresponding to each correlation peak, the filtering method further comprises:
performing dimensionality reduction on each correlation peak by using a principal component decomposition method;
the determining the filtering strategy corresponding to each correlation peak according to the position of each correlation peak in the detection video and the frequency band corresponding to each correlation peak comprises:
and determining a filtering strategy corresponding to each correlation peak according to the position of each correlation peak after the dimension reduction processing in the detection video and the frequency band corresponding to the correlation peak.
5. The filtering method according to claim 4, wherein before performing the dimensionality reduction processing on each correlation peak by using the principal component decomposition method, the filtering method further comprises:
comparing the peak value of each correlation peak with a preset peak value range, and respectively judging whether the peak value of each correlation peak is in the preset peak value range;
the dimension reduction processing of each correlation peak by using the principal component decomposition method comprises the following steps:
and carrying out dimension reduction processing on the correlation peak of which the peak value is within a preset peak value range.
6. The filtering method according to claim 1, wherein the filtering the correlation peaks according to the filtering strategy corresponding to the correlation peaks comprises:
and carrying out filtering processing on each correlation peak in an interpolation filtering mode according to the filtering strategy corresponding to each correlation peak.
7. A filtering apparatus, comprising:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring one or more correlation peaks obtained by performing phase correlation calculation on a frame sequence corresponding to a detected video, and the correlation peaks are frequency domain signals;
a filtering strategy determining module, configured to determine a filtering strategy corresponding to each correlation peak according to a position of each correlation peak in the detected video and a frequency band corresponding to the correlation peak;
and the filtering module is used for filtering each correlation peak according to the filtering strategy corresponding to each correlation peak.
8. An electronic device comprising a processor, a memory, and a filter program stored on the memory and executable by the processor, wherein the filter program, when executed by the processor, implements the steps of the filtering method of any one of claims 1 to 6.
9. A computer-readable storage medium, having a filter program stored thereon, wherein the filter program, when executed by a processor, implements the steps of the filtering method of any one of claims 1 to 6.
10. A detection device for detecting the operating condition of a device to be detected, characterized in that it comprises a filtering device according to claim 7 or an electronic device according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/105583 WO2021036661A1 (en) | 2019-04-26 | 2020-07-29 | Filtering method and apparatus, and related product |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2019103460141 | 2019-04-26 | ||
| CN201910346014 | 2019-04-26 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110674697A true CN110674697A (en) | 2020-01-10 |
| CN110674697B CN110674697B (en) | 2023-04-25 |
Family
ID=69075853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910815821.3A Active CN110674697B (en) | 2019-04-26 | 2019-08-30 | Filtering method, device and related product |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110674697B (en) |
| WO (1) | WO2021036661A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021036661A1 (en) * | 2019-04-26 | 2021-03-04 | 深圳市豪视智能科技有限公司 | Filtering method and apparatus, and related product |
| CN114504327A (en) * | 2021-12-28 | 2022-05-17 | 深圳大学 | Electroencephalogram noise processing method and device and computer equipment |
| CN117576091A (en) * | 2024-01-15 | 2024-02-20 | 深圳昱拓智能有限公司 | Cooling tower fan vibration detection method and system based on video detection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2187059A (en) * | 1986-02-20 | 1987-08-26 | British Broadcasting Corp | Motion detector for video signal prediction systems |
| CN103714522A (en) * | 2014-01-06 | 2014-04-09 | 海信集团有限公司 | Image denoising method and image denoising system |
| US20170238842A1 (en) * | 2016-02-19 | 2017-08-24 | Covidien Lp | Systems and methods for video-based monitoring of vital signs |
| CN107844779A (en) * | 2017-11-21 | 2018-03-27 | 重庆邮电大学 | A kind of video key frame extracting method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103024248B (en) * | 2013-01-05 | 2016-01-06 | 上海富瀚微电子股份有限公司 | The video image noise reducing method of Motion Adaptive and device thereof |
| CN110674697B (en) * | 2019-04-26 | 2023-04-25 | 深圳市豪视智能科技有限公司 | Filtering method, device and related product |
| CN110659583B (en) * | 2019-04-26 | 2023-11-28 | 深圳市豪视智能科技有限公司 | Signal processing method and device and related products |
-
2019
- 2019-08-30 CN CN201910815821.3A patent/CN110674697B/en active Active
-
2020
- 2020-07-29 WO PCT/CN2020/105583 patent/WO2021036661A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2187059A (en) * | 1986-02-20 | 1987-08-26 | British Broadcasting Corp | Motion detector for video signal prediction systems |
| CN103714522A (en) * | 2014-01-06 | 2014-04-09 | 海信集团有限公司 | Image denoising method and image denoising system |
| US20170238842A1 (en) * | 2016-02-19 | 2017-08-24 | Covidien Lp | Systems and methods for video-based monitoring of vital signs |
| CN107844779A (en) * | 2017-11-21 | 2018-03-27 | 重庆邮电大学 | A kind of video key frame extracting method |
Non-Patent Citations (1)
| Title |
|---|
| 轩建平等: "微小运动放大及其在超声波传播中的应用", 《华中科技大学学报(自然科学版)》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021036661A1 (en) * | 2019-04-26 | 2021-03-04 | 深圳市豪视智能科技有限公司 | Filtering method and apparatus, and related product |
| CN114504327A (en) * | 2021-12-28 | 2022-05-17 | 深圳大学 | Electroencephalogram noise processing method and device and computer equipment |
| CN114504327B (en) * | 2021-12-28 | 2024-05-17 | 深圳大学 | Electroencephalogram noise processing method and device and computer equipment |
| CN117576091A (en) * | 2024-01-15 | 2024-02-20 | 深圳昱拓智能有限公司 | Cooling tower fan vibration detection method and system based on video detection |
| CN117576091B (en) * | 2024-01-15 | 2024-04-09 | 深圳昱拓智能有限公司 | Cooling tower fan vibration detection method and system based on video detection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110674697B (en) | 2023-04-25 |
| WO2021036661A1 (en) | 2021-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110659583B (en) | Signal processing method and device and related products | |
| JP6461165B2 (en) | Method of inverse tone mapping of image | |
| US8692896B2 (en) | Image quality evaluation device, terminal device, image quality evaluation system, image quality evaluation method and computer-readable recording medium for storing programs | |
| US9824424B2 (en) | Image amplifying method, image amplifying device, and display apparatus | |
| US9324153B2 (en) | Depth measurement apparatus, image pickup apparatus, depth measurement method, and depth measurement program | |
| CN110674697B (en) | Filtering method, device and related product | |
| WO2021036663A1 (en) | Method for detecting anomaly of fixed screw for device, and related products | |
| WO2014125659A1 (en) | Image processing device, image capture device, filter generating device, image restoration method, and program | |
| JP5983373B2 (en) | Image processing apparatus, information processing method, and program | |
| US9319602B2 (en) | Method and apparatus for processing image | |
| JP2010218110A (en) | Noise reduction device, noise reduction method, noise reduction program, and recording medium | |
| US9153013B2 (en) | Image processing apparatus, image processing method and computer readable medium | |
| US20150093027A1 (en) | Image processing apparatus and image processing method | |
| US20100092086A1 (en) | Method and system for image deblurring | |
| US20220070417A1 (en) | Content adaptive lens shading correction method and apparatus | |
| JP5765893B2 (en) | Image processing apparatus, imaging apparatus, and image processing program | |
| JP2021043874A (en) | Image processing apparatus, image processing method, and program | |
| KR20100101463A (en) | Apparatus and method for eliminating noise | |
| JP3743389B2 (en) | Image compression apparatus and image compression program | |
| JP5810778B2 (en) | Image processing apparatus and image processing method | |
| JP7197210B2 (en) | Image correction system, image correction method, and program | |
| JP2004348293A (en) | Image quality computing device and method, program thereof, and recording medium | |
| CN114514743A (en) | Server device and program | |
| JP2021048522A (en) | Image processing apparatus and image processing method | |
| CN116391202A (en) | Image noise reduction method, device and chip |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |