WO2021036633A1 - Vibration detection method and related product - Google Patents

Abstract

A vibration detection method comprises: acquiring a first video sequence associated with a preset portion of a target object (101); extracting, according to the first video sequence, a first vibration parameter set generated from vibration at the preset portion (102); extracting, from the first vibration parameter set, a second vibration parameter set meeting a preset requirement (103); performing a position marking operation on the preset portion according to the second vibration parameter set, and obtaining at least one position mark (104); and displaying the at least one position mark in the first video sequence (105). In the method, a vibration parameter corresponding to a preset portion can be obtained by analyzing a video of the preset portion, and a corresponding position at which vibration is generated can be displayed. Also disclosed are a vibration detection device, an electronic apparatus, a computer readable storage medium, and a computer program product.

Description

Vibration detection methods and related products Technical field

This application relates to the field of image processing technology, and specifically relates to a vibration detection method and related products.

Background technique

All mechanical and motion systems produce various vibrations, some of which reflect the normal motion state of the system, while others reflect the abnormal motion state of the system (for example, equipment failure, unbalanced shaft connection, etc.) ). Therefore, to predictively maintain system equipment, vibration detection is an important part. Nowadays, most of the vibration detection uses accelerometer equipment. Although more accurate and reliable, the accelerometer requires a long preparation and installation time, requires physical contact with the system under test during the test (therefore, it will change the vibration response of the system under test), and can only test a limited number of discrete points, so , The problem of how to accurately display the vibration position needs to be solved urgently.

Summary of the invention

The embodiments of the present application provide a vibration detection method and related products, which can effectively display the vibration effect.

The first aspect of the embodiments of the present application provides a vibration detection method, including:

Acquiring a first video sequence for a preset part of the target object;

Extracting, according to the first video sequence, a first set of vibration parameters generated by the vibration of the preset part;

Extracting a second set of vibration parameters that meets a preset requirement from the set of first vibration parameters;

Marking the preset position according to the second vibration parameter set to obtain at least one position mark;

The at least one position marker is displayed in the first video sequence.

A second aspect of the embodiments of the present application provides a vibration detection device, including:

An acquiring unit, configured to acquire a first video sequence for a preset part of the target object;

An extraction unit, configured to extract a first set of vibration parameters generated by the vibration of the preset part according to the first video sequence; and extract a second set of vibration parameters that meets the preset requirements from the first set of vibration parameters;

A marking unit, configured to mark the position of the preset location according to the second set of vibration parameters to obtain at least one position mark;

The display unit is configured to display the at least one position marker in the first video sequence.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and are configured by the above Executed by the processor, and the foregoing program includes instructions for executing the steps in the first aspect of the embodiments of the present application.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the above-mentioned computer-readable storage medium stores a computer program for electronic data exchange, wherein the above-mentioned computer program enables a computer to execute Some or all of the steps described in one aspect.

In a fifth aspect, the embodiments of the present application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute as implemented in this application. Example part or all of the steps described in the first aspect. The computer program product may be a software installation package.

The implementation of the embodiments of this application has the following beneficial effects:

It can be seen that through the vibration detection method and related products of the embodiments of the present application, the first video sequence for the preset part of the target object is obtained, and the first vibration parameter set generated by the vibration of the preset part is extracted according to the first video sequence, and The second vibration parameter set that meets the preset requirements is extracted from the first vibration parameter set, and the preset position is marked according to the second vibration parameter set to obtain at least one position mark, and the at least one position mark is displayed in the first video sequence, Furthermore, the corresponding vibration parameters can be analyzed through the preset part of the video, and the corresponding vibration location can be displayed.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. A person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

FIG. 1A is a schematic flowchart of an embodiment of a vibration detection method provided by an embodiment of the present application;

FIG. 1B is a schematic diagram of an application scenario demonstration provided by an embodiment of the present application;

FIG. 1C is a schematic diagram of a motion amplification method provided by an embodiment of the present application;

2 is a schematic flowchart of an embodiment of another vibration detection method provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of an embodiment of a vibration detection device provided by an embodiment of the present application;

Fig. 4 is a schematic structural diagram of an embodiment of an electronic device provided by an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The terms "first", "second", "third" and "fourth" in the specification and claims of the application and the drawings are used to distinguish different objects, rather than describing a specific order . In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units, or optionally also includes Other steps or units inherent in these processes, methods, products or equipment.

Reference to "embodiments" herein means that a specific feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The display of the phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art clearly and implicitly understand that the embodiments described herein can be combined with other embodiments.

The electronic devices described in the embodiments of this application may include smart phones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, handheld computers, notebook computers, video matrixes, monitoring platforms, and mobile Internet devices (MID, Mobile Internet Devices). ), drones, DVs or wearable devices, etc. The above are only examples and not exhaustive, including but not limited to the above devices. Of course, the above electronic devices can also be servers.

Please refer to FIG. 1A, which is a schematic flowchart of an embodiment of a vibration detection method provided by an embodiment of this application. The vibration detection method described in this embodiment includes the following steps:

101. Acquire a first video sequence for a preset part of a target object.

Among them, the target object may be one of the following: mechanical equipment (numerically controlled machine tools), transportation tools, communication tools, medical equipment, buildings, people, animals, etc., which are not limited here. The first video sequence of the target object within a period of time may be acquired, and the first video sequence may be an RGB video image. The preset position can be set by the user or the system defaults. The embodiment of this application can be applied to the scene shown in FIG. 1B. The first video sequence can be captured by the camera to obtain a video sequence of the target object. The electronic device analyzes the video sequence to obtain the vibration parameters, and displays it on the display interface. Show vibration parameters.

In the embodiment of the present application, the electronic device may include at least one camera, and the camera may be at least one of the following: an infrared camera or a visible light camera. The visible light camera may be a normal camera or a light angle camera, etc., which is not limited herein. Of course, the electronic device may also include an ultrasonic sensor, and the first video sequence of the target object may be acquired through the ultrasonic sensor. The first video sequence may be a panoramic video image for the entire target object, or a video image of a designated part of the target object, or an internal operation image of the target object.

102. Extract, according to the first video sequence, a first set of vibration parameters generated by the vibration of the preset part.

The vibration parameter set may include at least one of the following vibration parameters: vibration frequency, vibration amplitude, number of vibrations, vibration time-domain waveform diagram, vibration period, vibration range, etc., which are not limited here.

Optionally, the above step 102, extracting the first vibration parameter set generated by the vibration of the preset part according to the first video sequence, may include the following steps:

21. Perform color space transformation on the first video sequence to obtain the transformed first video sequence;

22. Separate the transformed first video sequence to obtain luminance component information and chrominance component information;

23. Perform Fourier transform on the brightness component information to obtain frequency domain brightness component information;

24. Perform motion amplification processing on the frequency domain luminance component information to obtain the frequency domain luminance component after amplification processing;

25. Perform an inverse Fourier transform on the frequency-domain luminance component after the amplification process to obtain time-domain luminance component information;

26. Synthesize the temporal luminance information and the chrominance component information to obtain a synthesized video sequence;

27. Perform an inverse transformation corresponding to the color space transformation on the composite video sequence to obtain a second video sequence;

28. Determine a first set of vibration parameters generated by the vibration of the preset part according to the second video sequence.

The aforementioned color space conversion may include any of the following: RGB to YCbCr color space conversion, RGB to YIQ color space conversion, RGB to HIS color space conversion, RGB to YUV color space conversion, which is not limited here. In specific implementation, the first video sequence can be transformed into the color space to obtain the transformed first video sequence. In this way, the luminance component can be extracted, that is, the transformed first video sequence can be separated to obtain the luminance component information and the color space. Degree component information, for example, the camera shoots a piece of video containing a vibrating object under test, and then converts the sequence frame of the video from RGB color space to YIQ color space to separate the luminance component information and chrominance component information of the video frame , The subsequent steps are performed on the Y phase of the YIQ video, that is, the brightness information, which can reduce the amount of algorithm calculations and increase the running speed of the algorithm. The conversion relationship between RGB and YIQ is:

Y＝0.299*R+0.587*G+0.114*B;

I=0.596*R–0.275*G–0.321*B;

Q=0.212*R-0.523*G+0.311*B.

Further, Fourier transform can be performed on the brightness component information to obtain the frequency domain brightness component information, and the frequency domain brightness component information can be subjected to motion amplification processing to obtain the frequency domain brightness component after the amplification processing, and the frequency domain brightness after the amplification processing The components are subjected to inverse Fourier transform to obtain temporal luminance component information, the temporal luminance information and chrominance component information are synthesized to obtain a synthesized video sequence, and the inverse transformation corresponding to the color space transformation is performed on the synthesized video sequence to obtain the first In the second video sequence, the first vibration parameter set generated by the vibration of the preset location is determined according to the second video sequence. In this way, since the characteristics corresponding to the vibration are amplified, it is convenient to better extract the vibration parameters from the video sequence.

Further optionally, the above step 28, based on the first vibration parameter set generated by the vibration of the preset part of the second video sequence, may include the following steps:

281. Perform image segmentation on the second video sequence according to the preset location to obtain a third video sequence, where each video frame in the third video sequence includes only images corresponding to the preset location;

282. Determine a cross cross power spectrum between frame sequences corresponding to the third video sequence.

283. Perform interpolation filtering processing on the cross cross power spectrum to obtain the processed cross cross power spectrum.

284. Perform an inverse Fourier transform on the processed cross cross power spectrum, and compare phase by phase to obtain a first set of vibration parameters generated by the vibration of the preset position.

In a specific implementation, the second video sequence may be segmented according to a preset position to obtain a third video sequence. Each video frame in the third video sequence includes only images corresponding to the preset position, and furthermore, The frame sequence after the video motion amplification process uses the phase correlation algorithm to calculate the cross cross power spectrum between the frame sequences. Specifically, the phase correlation algorithm uses the following formula to calculate the cross cross power spectrum.

Wherein, _a Fourier F. A third frame image in a video sequence conversion,

Is the Fourier-transformed conjugate signal of the b-frame image in the third video sequence, and the lower side of the division formula is the modulus of the correlation product of the two Fourier-transformed signals. R crosses the cross power spectrum (including frequency domain noise) of the calculation result of this step.

Further, an adaptive filter bank can be used to reconstruct the motion signal, and the filter bank can be adaptively selected for filtering according to the position of the correlation peak of R. After filtering, the inverse Fourier transform is performed, and then the phase comparison (phase-by-phase comparison) is performed. At this time, the sliding window adaptive matching method can be used to estimate and extract the vibration parameters to obtain the cross cross power spectrum R′ after filtering the frequency domain noise, and then perform the inverse Fourier transform on the cross cross power spectrum to compare phase by phase , The calculation formula for the vibration information of the pixels in the video can be obtained as follows.

r=F ^-1 {R′}

Among them, F ^-1 represents the inverse Fourier transform of the cross cross power spectrum, R′ is the cross cross power spectrum after filtering the frequency domain noise, and r is the vibration information corresponding to the pixel in the video.

Wherein, the filter bank can be adaptively selected for filtering according to the position of the correlation peak of the cross cross power spectrum R, to obtain the filtered cross cross power spectrum R'.

Optionally, in the above step 282, performing interpolation filtering processing on the cross cross power spectrum to obtain the processed cross cross power spectrum may include the following steps:

2821. Acquire multiple state change signals corresponding to the cross cross power spectrum, where the state change signals are time-domain signals.

2822. Extract a target state change signal segment of a preset length from the plurality of state change signals to obtain a plurality of target state change signal segments, and obtain each target state change signal segment of the plurality of target state change signal segments The target frequency of, get multiple target frequencies;

2823. Set the corresponding sliding window according to the target frequency corresponding to each state change signal in the multiple state change signals to obtain multiple sliding windows, and send each state change signal in the multiple state change signals to the corresponding Sliding window

2824. Use the state change signal of the plurality of state change signals that cannot pass through the corresponding sliding window as an aperiodic signal to obtain at least one aperiodic signal;

2825. Remove the at least one non-periodic signal from the multiple state change signals to obtain a filtered cross cross power spectrum.

Among them, the cross cross power spectrum R is a frequency domain signal, which includes one or more correlation peaks. After the cross cross power spectrum R is subjected to inverse Fourier transform, the state change signal corresponding to each correlation peak can be obtained. Each state change signal can reflect the state change of a certain position in the amplified video. Therefore, a predetermined length of the target state change signal segment can be extracted from the multiple state change signals to obtain multiple target state change signal segments . The state change information includes vibration data and other noise information. For example, changes in illumination can also cause state changes in the video screen, and the vibration data can reflect the operating conditions of the object to be vibrated. The vibration of the equipment to be tested is periodic when it is running, and the state change caused by the vibration is also periodic. Although a lot of noise information will cause the state change of each pixel in the amplified video, the state change caused by noise is often not periodic, and according to the operating conditions, periodic vibration can be used to reflect the operating conditions of the equipment, because it is aperiodic. Sexual vibrations are often caused by the external environment, not by the equipment itself, and this part of the non-periodic signals cannot be used to analyze the operating conditions of the equipment. Acquire noise signals that are not caused by self-vibration by acquiring non-periodic signals in the state change signals. Since aperiodic signals are often signals that have little or no effect or even interference on the operating conditions of the analysis equipment, this part of the aperiodic signals can be removed to make the state change signals obtained from the amplified video more useful information. many.

Among them, after obtaining multiple state change signals, it can be judged whether each state change signal is a periodic signal. Specifically, a target state change signal segment with a preset length can be extracted first, the target frequency of the target change signal segment can be obtained, and then the state The frequencies of other parts in the change signal are compared with the target frequency. If the frequencies of other parts in the state change signal are not consistent with the target frequency, the state change signal can be regarded as a non-periodic signal. The preset length can be set by a user to a certain value, or it can be self-adapted according to the length of the signal during signal processing. For example, the preset length can be set to 1/10 of the length of the state change signal. After obtaining the target frequency of the target state transition signal segment, the window size of the sliding window can be set according to the target frequency. For example, the window size of the sliding window can be set to be consistent with the target frequency, so that only signals with the same frequency as the target frequency can pass through the sliding window. However, signals that are inconsistent with the target frequency cannot pass through the sliding window. If the state change signal cannot pass through the corresponding sliding window, it means that there is a signal segment whose frequency is inconsistent with the target frequency in the state change signal, that is, the state change signal is a non-periodic signal. In this way, by judging whether the frequencies of other parts of the state change signal are consistent with the target frequency by means of sliding windows, conclusions can be obtained conveniently and quickly, and the amount of calculation is smaller.

Further optionally, in the above step 24, performing motion amplification processing on the frequency domain luminance component information to obtain the frequency domain luminance component after the amplification processing may include the following steps:

2411. Perform feature point extraction on the frequency domain luminance component information to obtain a feature point set.

2412. Perform clustering on the feature point set to obtain multiple types of feature points.

2413. Perform an optical flow field interpolation operation on each pixel in the frequency domain luminance component information to obtain a dense optical flow field;

2414. Track the motion trajectory of each type of feature point in the multiple types of feature points to obtain multiple action layers, and each type of feature point corresponds to one action layer;

2415. Perform texture filling on the multiple action layers, and perform amplification processing on a preset layer of the multiple action layers after texture filling, to obtain the frequency domain brightness component after the amplification processing.

Among them, the feature point extraction can be at least one of the following methods: Harris corner detection, scale invariant feature transform (SIFT), SURF feature extraction algorithm, etc., which are not limited here. In specific implementation, you can Feature points are extracted from the frequency domain brightness component information to obtain a feature point set. The feature point set is clustered to obtain multiple types of feature points. The optical flow field interpolation operation is performed on each pixel in the frequency domain brightness component information to obtain a dense The optical flow field tracks the motion trajectory of each type of feature points in multiple types of feature points to obtain multiple action layers. Each type of feature point corresponds to an action layer. The multiple action layers are filled with texture and the texture is filled. The preset layer in the multiple action layers performs amplification processing to obtain the amplified frequency domain brightness component. The preset layer can be set by the user or defaulted by the system. In this way, the characteristics corresponding to the vibration can be amplified, and the subsequent analysis of the impact of the vibration can be facilitated.

Optionally, before step 241, the frequency domain luminance component information can be registered to obtain a registered image. Then, when step 241 is performed, feature point extraction can be performed on the registered image to obtain feature points. Set, in this way, can prevent the movement caused by camera shake.

In the specific implementation, considering that the tracking of feature points may be affected by other factors such as image darkness, occlusion, etc., which affects the tracking results, the accuracy of motion level segmentation is directly related to the final zoom effect, and this process can also be combined with manual intervention. That is, some motion layers are selected artificially, so that the tracking accuracy can be further improved.

Further optionally, in the above step 24, performing motion amplification processing on the frequency domain luminance component information to obtain the frequency domain luminance component after the amplification processing may include the following steps:

2421. Perform spatial domain decomposition on the frequency domain luminance component to obtain the frequency domain luminance component after decomposition.

2422. Perform time-domain filtering on the decomposed frequency-domain luminance component, and filter the frequency-domain luminance component;

2423. Perform amplification processing on the frequency domain brightness component to obtain the amplified frequency domain brightness component.

2424. Perform an inverse transform corresponding to the spatial domain decomposition on the frequency domain luminance component to obtain the frequency domain luminance component after the inverse transform and the amplification process.

Among them, the spatial decomposition may include at least one of the following algorithms: fast Fourier transform, contourlet transform, non-sampled contourlet transform (NSCT), wavelet transform, ridgelet transform, shear wave transform, etc. It is not limited here. For example, the spatial decomposition may specifically be FFT first, and then complex operable pyramid spatial decomposition. In specific implementation, the frequency domain brightness component can be decomposed in the space domain to obtain the decomposed frequency domain brightness component, and the decomposed frequency domain brightness component can be filtered in the time domain, and the frequency domain brightness component after the filtering can be performed on the frequency domain brightness component. Enlargement processing to obtain the amplified frequency domain luminance component, and perform the inverse transformation corresponding to the spatial domain decomposition of the frequency domain luminance component to obtain the frequency domain luminance component after the inverse transformation after the enlargement processing, which can add the deep details in the image Enlargement helps to accurately extract the key feature points in the image, and it can still achieve better tracking of the feature points when the feature points are affected by the shadow, occlusion, etc. of the image.

In the specific implementation, for example, as shown in Fig. 1C, the YIQ color space of the first video sequence is converted, keeping the I and Q channels unchanged, and the Y channel is spatially decomposed (such as FFT operation), and the FFT transformed The Y-channel image is decomposed in the complex and manipulable gold tower spatial domain, and the images of different scales after the Y-channel spatial domain decomposition are time-domain band-pass filtered, and the motion information of interest after the time-domain band-pass filtering is amplified, and the motion information of interest is performed Reconstruction of complex manipulable pyramids to obtain the enlarged Y channel image. Finally, the reconstructed Y channel image is added to the original I and Q channel images, and then converted into RGB color space to obtain the second video sequence. In this way, it can be accurately extracted The vibration parameters corresponding to the target object help to improve the accuracy of fault detection.

103. Extract a second set of vibration parameters that meets a preset requirement from the set of first vibration parameters.

Among them, the preset requirements can be set by the user or the system defaults. For example, the preset requirement may be the vibration parameter corresponding to the test point with the largest vibration amplitude. For another example, the preset requirement may be the vibration parameter corresponding to the preset test point, and the preset test point may be set by the user or the system defaults.

Optionally, the first vibration parameter set may include multiple vibration parameters; the foregoing step 103, extracting a second vibration parameter set that meets a preset requirement from the first vibration parameter set, may include the following steps:

31. Perform cluster analysis on the first vibration parameter set to obtain multiple classes, each of which corresponds to a test point;

32. Use the vibration parameters corresponding to the test points in the multiple classes as the preset test points as the second vibration parameter set.

Among them, the above-mentioned preset test points can be set by the user or the system defaults. In the specific implementation, cluster analysis is performed on the first set of vibration parameters to obtain multiple classes, each of which corresponds to a test point, and the corresponding test point in the multiple classes is the vibration parameter corresponding to the preset test point as the second The vibration parameter set, in turn, can filter the vibration parameters so as to accurately realize the fixed-point analysis.

104. Mark the position of the preset position according to the second vibration parameter set to obtain at least one position mark.

Wherein, the second vibration parameter set includes a large number of vibration parameters, and each vibration parameter can correspond to a point in the preset position. Furthermore, each point can include at least one vibration parameter. Based on this, the second vibration parameter set can be Mark the preset position to obtain at least one position mark. Of course, it is also possible to map the second set of vibration parameters to a preset position, divide the preset position into a plurality of squares, mark the squares with vibration parameters greater than the preset number in the squares, to obtain at least one position mark, Among them, the aforementioned preset number can be set by the user or the system defaults.

105. Display the at least one position marker in the first video sequence.

Among them, in a specific implementation, a video sequence of at least one location mark in a specified time period can be displayed, and the specified time period can be set by the user or the system defaults.

Optionally, the above step 105, displaying the at least one position mark in the first video sequence, may include the following steps:

51. Extract a target vibration parameter for a position mark i from the second vibration parameter set, where the position mark i is any position mark in the at least one position mark;

52. Process the target vibration parameters to obtain target characteristic values;

53. Determine the target display effect parameter corresponding to the target characteristic value according to the mapping relationship between the preset characteristic value and the display effect parameter;

54. Display the position marker i in the first video sequence according to the target display effect parameter.

The above-mentioned display effect parameter may be at least one of the following: display brightness, display color, display hue, whether to add a display frame, whether to add an arrow, etc., which are not limited here. The electronic device can also pre-store the mapping relationship between the preset feature value and the display effect parameter. In specific implementation, since different vibration parameters correspond to different marks, each mark can take a point as the center, within a certain range As the vibration parameter of the point, the target vibration parameter for the position marker i can be extracted from the second vibration parameter set, where the position marker i is any position marker in the at least one position marker, and the target vibration parameter is processed , Obtain the target feature value, determine the target display effect parameter corresponding to the target feature value according to the mapping relationship between the preset feature value and the display effect parameter, and mark the display position i in the first video sequence according to the target display effect parameter, so , Different markers can be displayed differentiated according to the vibration difference, which helps users to see the vibration effect quickly and accurately.

Further optionally, the above step 52, processing the target vibration parameter to obtain the target characteristic value, may include the following steps:

521. Obtain a time-domain waveform from the target vibration parameter.

522. Perform preprocessing on the time domain waveform to obtain a target time domain waveform.

523. Compare the target time domain waveform with a preset time domain waveform.

524. When the target time-domain waveform and the preset time-domain waveform are successfully compared, determine multiple extreme points of the target time-domain waveform;

525. Determine the target characteristic value according to the multiple extreme points.

Among them, because the vibration is always vibrating, for example, if the device is in motion, its components are also constantly rotating. The vibration parameters may include a large amount of vibration data. The above-mentioned preprocessing may include at least one of the following: signal amplification, noise reduction processing, etc. , It is not limited here. The preset time domain waveform can be the waveform corresponding to the fault state, and the preset time domain waveform can be set by the user or the system defaults. In specific implementation, the time-domain waveform can be obtained from the target vibration parameters, and then the target time-domain waveform can be preprocessed to obtain the target time-domain waveform. The target time-domain waveform is compared with the preset time-domain waveform, and the target time-domain waveform is compared with the preset time-domain waveform. When the time-domain waveform is successfully compared with the preset time-domain waveform, multiple extreme points of the target time-domain waveform are determined. The extreme points may include maximum points or minimum points, which may be based on the multiple extreme points. Point to determine the target characteristic value. Conversely, when the target time-domain waveform and the preset time-domain waveform fail to compare, it can indicate that the target object is operating normally, and the vibration display may not be performed, or it can prompt the user that the device is operating normally. Specifically, for example, the mean square error of the plurality of extreme value points may be used as the target feature value, or the number of the plurality of extreme value points may be used as the target feature value.

Optionally, the foregoing step 525, determining the target characteristic value according to the multiple extreme points, may include the following steps:

5251. Select an extreme point with an absolute value greater than a preset threshold from the multiple extreme points to obtain multiple target extreme points;

5252. Divide the multiple target extreme points into two categories, the first category includes multiple maximum points, and the second category includes multiple minimum points;

5253. Determine the first mean square error between the multiple maximum points in the first category.

5254. Determine a second mean square error between the multiple minimum values in the second category.

S255. Use the absolute value of the ratio between the first mean square error and the second mean square error as the target feature value.

Among them, the aforementioned preset threshold can be set by the user or the system defaults. In specific implementation, one can select extreme points whose absolute value is greater than a preset threshold from multiple extreme points to obtain multiple target extreme points. The multiple target extreme points are divided into two categories. The first category includes multiple Maximum points, the second category includes multiple minimum points, determine the first mean square deviation between multiple maximum points in the first category, and determine the first mean square difference between multiple minimum points in the second category Two-mean-square error, the absolute value of the ratio between the first mean square error and the second mean square deviation is used as the target feature value. In this way, the extreme points are firstly screened to obtain stable extreme points, which can be more analyzed The vibration condition of the target object. In addition, these extreme points are divided into two categories, maximum points and minimum points, and the mean square deviation is calculated respectively through these two categories, and then the absolute value of the final ratio is used as the target characteristic value. Since the target eigenvalue fully grasps the distribution of the amplitude in the upward and downward movement, such a target eigenvalue can accurately reflect the vibration trend and help improve the accuracy of the fault information.

For example, in the embodiment of the present application, the target object may be a mechanical device. Based on the above-mentioned vibration detection device, a video sequence of the mechanical device vibrating under working conditions can be obtained, and the vibration parameters are extracted according to the video sequence, and the mechanical equipment is analyzed according to the vibration parameters. Fault information.

For example, in the embodiment of the present application, the target object may be a person. Based on the above-mentioned vibration detection device, the ultrasonic sensor can obtain a user's video sequence of heart vibration, extract the vibration parameters from the video sequence, and analyze the human heart function based on the vibration parameters. As well as realizing health diagnosis, of course, pregnant women can also be tested to detect the growth of the baby.

Based on the above embodiments of the present application, in specific implementation, the electronic device can display the real image or video of the target part of the target object on the display interface according to user needs, where the real image or video is the electronic device based on any of the above vibration detection The method is obtained by processing the original video of the detected object. For the displayed image or video, the embodiment of the present application can further enhance the display processing, can analyze the position of the strongest vibration, and can also highlight the vibrating part of the region corresponding to the strongest vibration position. Specifically, a mark frame can be used. , Arrows, and color processing methods for marking. Of course, you can also mark the vibration location based on user settings or by yourself, and display the vibration analysis result data of the object synchronously, such as vibration parameters such as amplitude, frequency, and time domain waveform.

It can be seen that through the vibration detection method of the embodiment of the present application, the first video sequence for the preset part of the target object is obtained, and the first vibration parameter set generated by the vibration of the preset part is extracted according to the first video sequence, and the first vibration parameter set is obtained from the first vibration The second vibration parameter set that meets the preset requirements is extracted from the parameter set, and the preset position is marked according to the second vibration parameter set to obtain at least one position mark. At least one position mark is displayed in the first video sequence. Analyze the corresponding vibration parameters through the preset part of the video, and display the corresponding vibration location.

Consistent with the above, please refer to FIG. 2, which is a schematic flowchart of an embodiment of a vibration detection method provided by an embodiment of this application. The vibration detection method described in this embodiment includes the following steps:

201. Acquire a first video sequence for a preset part of a target object.

202. Extract, according to the first video sequence, a first set of vibration parameters generated by the vibration of the preset part.

203. Extract a second vibration parameter set that meets a preset requirement from the first vibration parameter set.

204. Mark the preset position according to the second vibration parameter set to obtain at least one position mark.

205. Extract a target vibration parameter for a position mark i from the second vibration parameter set, where the position mark i is any position mark in the at least one position mark.

206. Process the target vibration parameter to obtain a target characteristic value.

207. Determine the target display effect parameter corresponding to the target characteristic value according to the preset mapping relationship between the characteristic value and the display effect parameter.

208. Display the position marker i in the first video sequence according to the target display effect parameter.

For the vibration detection method described in steps 201 to 208 above, reference may be made to the corresponding steps of the vibration detection method described in FIG. 1A.

It can be seen that through the vibration detection method of the embodiment of the present application, the first video sequence for the preset part of the target object is obtained, and the first vibration parameter set generated by the vibration of the preset part is extracted according to the first video sequence, and the first vibration parameter set is obtained from the first vibration The second vibration parameter set that meets the preset requirements is extracted from the parameter set, and the preset position is marked according to the second vibration parameter set to obtain at least one position mark. At least one position mark is displayed in the first video sequence. Analyze the corresponding vibration parameters through the preset part of the video, and display the corresponding vibration location.

Consistent with the above, the following is a device for implementing the above vibration detection method, and the details are as follows:

Please refer to FIG. 3, which is a schematic structural diagram of an embodiment of a vibration detection device provided by an embodiment of the application. The vibration detection device described in this embodiment includes: an acquisition unit 301, an extraction unit 302, a marking unit 303, and a display unit 304, which are specifically as follows:

The acquiring unit 301 is configured to acquire a first video sequence for a preset part of the target object;

The extraction unit 302 is configured to extract a first vibration parameter set generated by the vibration of the preset part according to the first video sequence; and extract a second vibration parameter set that meets the preset requirements from the first vibration parameter set ；

The marking unit 303 is configured to mark the position of the preset part according to the second vibration parameter set to obtain at least one position mark;

The display unit 304 is configured to display the at least one position marker in the first video sequence.

It can be seen that through the vibration detection device of the embodiment of the present application, the first video sequence for the preset part of the target object is obtained, and the first vibration parameter set generated by the vibration of the preset part is extracted according to the first video sequence, and the first vibration The second vibration parameter set that meets the preset requirements is extracted from the parameter set, and the preset position is marked according to the second vibration parameter set to obtain at least one position mark. At least one position mark is displayed in the first video sequence. Analyze the corresponding vibration parameters through the preset part of the video, and display the corresponding vibration location.

Optionally, in the aspect of extracting the first vibration parameter set generated by the vibration of the preset part according to the first video sequence, the extracting unit 302 is specifically configured to:

Performing color space transformation on the first video sequence to obtain the transformed first video sequence;

Separating the transformed first video sequence to obtain luminance component information and chrominance component information;

Performing Fourier transform on the brightness component information to obtain frequency domain brightness component information;

Performing motion amplification processing on the frequency domain luminance component information to obtain the frequency domain luminance component after the amplification processing;

Performing an inverse Fourier transform on the frequency-domain luminance component after the amplification process to obtain time-domain luminance component information;

Synthesize the temporal luminance information and the chrominance component information to obtain a synthesized video sequence;

Performing an inverse transformation corresponding to the color space transformation on the composite video sequence to obtain a second video sequence;

A first set of vibration parameters generated by the vibration of the preset part is determined according to the second video sequence.

Optionally, the first vibration parameter set may include multiple vibration parameters;

In terms of extracting a second vibration parameter set that meets a preset requirement from the first vibration parameter set, the extracting unit 302 is specifically configured to:

Perform cluster analysis on the first vibration parameter set to obtain multiple classes, each of which corresponds to a test point;

The vibration parameter corresponding to the test point corresponding to the preset test point in the multiple classes is used as the second vibration parameter set.

Optionally, in the aspect of displaying the at least one position marker in the first video sequence, the displaying unit 304 is specifically configured to:

Extracting a target vibration parameter for a position mark i from the second vibration parameter set, where the position mark i is any position mark in the at least one position mark;

Processing the target vibration parameter to obtain a target characteristic value;

Determine the target display effect parameter corresponding to the target characteristic value according to the mapping relationship between the preset characteristic value and the display effect parameter;

The position marker i is displayed in the first video sequence according to the target display effect parameter.

Optionally, in terms of processing the target vibration parameter to obtain a target feature value, the display unit 304 is specifically configured to:

Acquiring a time-domain waveform from the target vibration parameter;

Preprocessing the time domain waveform to obtain a target time domain waveform;

Comparing the target time domain waveform with a preset time domain waveform;

Determining a plurality of extreme points of the target time domain waveform when the comparison between the target time domain waveform and the preset time domain waveform is successful;

The target feature value is determined according to the multiple extreme points.

It is understandable that the functions of each program module of the vibration detection device of this embodiment can be implemented according to the method in the above method embodiment, and the specific implementation process can be referred to the relevant description of the above method embodiment, which will not be repeated here.

Consistent with the above, please refer to FIG. 4, which is a schematic structural diagram of an embodiment of an electronic device provided by an embodiment of this application. The electronic device described in this embodiment includes: at least one input device 1000; at least one output device 2000; at least one processor 3000, such as a CPU; and a memory 4000, the aforementioned input device 1000, output device 2000, processor 3000, and The memory 4000 is connected through the bus 5000.

Wherein, the aforementioned input device 1000 may specifically be a touch panel, a physical button, or a mouse.

The aforementioned output device 2000 may specifically be a display screen.

The aforementioned memory 4000 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory. The foregoing memory 4000 is used to store a set of program codes, and the foregoing input device 1000, output device 2000, and processor 3000 are used to call the program codes stored in the memory 4000 to perform the following operations:

The above-mentioned processor 3000 is used for:

Acquiring a first video sequence for a preset part of the target object;

Extracting, according to the first video sequence, a first set of vibration parameters generated by the vibration of the preset part;

Extracting a second set of vibration parameters that meets a preset requirement from the set of first vibration parameters;

Marking the preset position according to the second vibration parameter set to obtain at least one position mark;

The at least one position marker is displayed in the first video sequence.

It can be seen that through the electronic device described in the embodiment of the present application, the first video sequence for the preset part of the target object is obtained, and the first vibration parameter set generated by the vibration of the preset part is extracted according to the first video sequence, and the first vibration parameter set is extracted from the first video sequence. A second vibration parameter set that meets the preset requirements is extracted from a vibration parameter set, the preset position is marked according to the second vibration parameter set to obtain at least one position mark, and at least one position mark is displayed in the first video sequence, and then , Can analyze the corresponding vibration parameters through the preset part of the video, and display the corresponding vibration location.

Optionally, in terms of extracting the first vibration parameter set generated by the vibration of the preset part according to the first video sequence, the processor 3000 is specifically configured to:

Performing color space transformation on the first video sequence to obtain the transformed first video sequence;

Separating the transformed first video sequence to obtain luminance component information and chrominance component information;

Performing Fourier transform on the brightness component information to obtain frequency domain brightness component information;

Performing motion amplification processing on the frequency domain luminance component information to obtain the frequency domain luminance component after the amplification processing;

Performing an inverse Fourier transform on the frequency-domain luminance component after the amplification process to obtain time-domain luminance component information;

Synthesize the temporal luminance information and the chrominance component information to obtain a synthesized video sequence;

Performing an inverse transformation corresponding to the color space transformation on the composite video sequence to obtain a second video sequence;

A first set of vibration parameters generated by the vibration of the preset part is determined according to the second video sequence.

Optionally, the first vibration parameter set may include multiple vibration parameters;

In terms of extracting a second vibration parameter set that meets a preset requirement from the first vibration parameter set, the processor 3000 is specifically configured to:

Perform cluster analysis on the first vibration parameter set to obtain multiple classes, each of which corresponds to a test point;

The vibration parameter corresponding to the test point corresponding to the preset test point in the multiple classes is used as the second vibration parameter set.

Optionally, in the aspect of displaying the at least one position marker in the first video sequence, the processor 3000 is specifically configured to:

Extracting a target vibration parameter for a position mark i from the second vibration parameter set, where the position mark i is any position mark in the at least one position mark;

Processing the target vibration parameter to obtain a target characteristic value;

Determine the target display effect parameter corresponding to the target characteristic value according to the mapping relationship between the preset characteristic value and the display effect parameter;

The position marker i is displayed in the first video sequence according to the target display effect parameter.

Optionally, in terms of processing the target vibration parameter to obtain a target feature value, the processor 3000 is specifically configured to:

Acquiring a time-domain waveform from the target vibration parameter;

Preprocessing the time domain waveform to obtain a target time domain waveform;

Comparing the target time domain waveform with a preset time domain waveform;

Determining a plurality of extreme points of the target time domain waveform when the comparison between the target time domain waveform and the preset time domain waveform is successful;

The target feature value is determined according to the multiple extreme points.

An embodiment of the present application further provides a computer storage medium, wherein the computer storage medium can store a program, and the program includes part or all of the steps of any vibration detection method recorded in the above method embodiment when the program is executed.

An embodiment of the present application further provides a computer program product, wherein the foregoing computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the foregoing computer program is operable to cause a computer to execute any of the foregoing Some or all of the steps described in the method. The computer program product may be a software installation package.

Although this application has been described in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art can understand and understand by viewing the drawings, the disclosure, and the appended claims in the process of implementing the claimed application. Implement other changes of the disclosed embodiment. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude a plurality. A single processor or other unit may implement several functions listed in the claims. Certain measures are described in mutually different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, devices (equipment), or computer program products. Therefore, this application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The computer program is stored/distributed in a suitable medium, provided with other hardware or as a part of the hardware, and can also be distributed in other forms, such as through the Internet or other wired or wireless telecommunication systems.

This application is described with reference to the flowcharts and/or block diagrams of the methods, devices (equipment) and computer program products of the embodiments of this application. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated It is a device that realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. The instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

Although the application has been described in combination with specific features and embodiments, it is obvious that various modifications and combinations can be made without departing from the spirit and scope of the application. Correspondingly, the specification and drawings are merely exemplary descriptions of the application as defined by the appended claims, and are deemed to cover any and all modifications, changes, combinations or equivalents within the scope of the application. Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.

Claims (13)

1. A vibration detection method, characterized in that it comprises:

   Acquiring a first video sequence for a preset part of the target object;

   Extracting, according to the first video sequence, a first set of vibration parameters generated by the vibration of the preset part;

   Extracting a second set of vibration parameters that meets a preset requirement from the set of first vibration parameters;

   Marking the preset position according to the second vibration parameter set to obtain at least one position mark;

   The at least one position marker is displayed in the first video sequence.
2. The method according to claim 1, wherein the extracting a first vibration parameter set generated by the vibration of the preset part according to the first video sequence comprises:

   Performing color space transformation on the first video sequence to obtain the transformed first video sequence;

   Separating the transformed first video sequence to obtain luminance component information and chrominance component information;

   Performing Fourier transform on the brightness component information to obtain frequency domain brightness component information;

   Performing motion amplification processing on the frequency domain luminance component information to obtain the frequency domain luminance component after the amplification processing;

   Performing an inverse Fourier transform on the frequency-domain luminance component after the amplification process to obtain time-domain luminance component information;

   Synthesize the temporal luminance information and the chrominance component information to obtain a synthesized video sequence;

   Performing an inverse transformation corresponding to the color space transformation on the composite video sequence to obtain a second video sequence;

   A first set of vibration parameters generated by the vibration of the preset part is determined according to the second video sequence.
3. The method according to claim 1 or 2, wherein the first set of vibration parameters may include multiple vibration parameters;

   The extracting a second vibration parameter set that meets a preset requirement from the first vibration parameter set includes:

   Perform cluster analysis on the first vibration parameter set to obtain multiple classes, each of which corresponds to a test point;

   The vibration parameter corresponding to the test point corresponding to the preset test point in the multiple classes is used as the second vibration parameter set.
4. The method according to any one of claims 1-3, wherein the displaying the at least one position marker in the first video sequence comprises:

   Extracting a target vibration parameter for a position mark i from the second vibration parameter set, where the position mark i is any position mark in the at least one position mark;

   Processing the target vibration parameter to obtain a target characteristic value;

   Determine the target display effect parameter corresponding to the target characteristic value according to the mapping relationship between the preset characteristic value and the display effect parameter;

   The position marker i is displayed in the first video sequence according to the target display effect parameter.
5. The method according to claim 4, wherein the processing the target vibration parameter to obtain the target characteristic value comprises:

   Acquiring a time-domain waveform from the target vibration parameter;

   Preprocessing the time domain waveform to obtain a target time domain waveform;

   Comparing the target time domain waveform with a preset time domain waveform;

   Determining a plurality of extreme points of the target time domain waveform when the comparison between the target time domain waveform and the preset time domain waveform is successful;

   The target feature value is determined according to the multiple extreme points.
6. A vibration detection device, characterized in that it comprises:

   An acquiring unit, configured to acquire a first video sequence for a preset part of the target object;

   An extraction unit, configured to extract a first set of vibration parameters generated by the vibration of the preset part according to the first video sequence; and extract a second set of vibration parameters that meets the preset requirements from the first set of vibration parameters;

   A marking unit, configured to mark the position of the preset location according to the second set of vibration parameters to obtain at least one position mark;

   The display unit is configured to display the at least one position marker in the first video sequence.
7. The device according to claim 6, wherein, in terms of extracting the first vibration parameter set generated by the vibration of the preset part according to the first video sequence, the extracting unit is specifically configured to:

   Performing color space transformation on the first video sequence to obtain the transformed first video sequence;

   Separating the transformed first video sequence to obtain luminance component information and chrominance component information;

   Performing Fourier transform on the brightness component information to obtain frequency domain brightness component information;

   Performing motion amplification processing on the frequency domain luminance component information to obtain the frequency domain luminance component after the amplification processing;

   Performing an inverse Fourier transform on the frequency-domain luminance component after the amplification process to obtain time-domain luminance component information;

   Synthesize the temporal luminance information and the chrominance component information to obtain a synthesized video sequence;

   Performing an inverse transformation corresponding to the color space transformation on the composite video sequence to obtain a second video sequence;

   A first set of vibration parameters generated by the vibration of the preset part is determined according to the second video sequence.
8. The device according to claim 7, wherein the first set of vibration parameters may include multiple vibration parameters;

   In terms of extracting a second vibration parameter set that meets a preset requirement from the first vibration parameter set, the extraction unit is specifically configured to:

   Perform cluster analysis on the first vibration parameter set to obtain multiple classes, each of which corresponds to a test point;

   The vibration parameter corresponding to the test point corresponding to the preset test point in the multiple classes is used as the second vibration parameter set.
9. The device according to any one of claims 6-8, wherein, in terms of displaying the at least one position marker in the first video sequence, the marking unit is specifically configured to:

   Extracting a target vibration parameter for a position mark i from the second vibration parameter set, where the position mark i is any position mark in the at least one position mark;

   Processing the target vibration parameter to obtain a target characteristic value;

   Determine the target display effect parameter corresponding to the target characteristic value according to the mapping relationship between the preset characteristic value and the display effect parameter;

   The position marker i is displayed in the first video sequence according to the target display effect parameter.
10. The device according to claim 9, wherein, in terms of processing the target vibration parameter to obtain a target characteristic value, the marking unit is specifically configured to:

    Acquiring a time-domain waveform from the target vibration parameter;

    Preprocessing the time domain waveform to obtain a target time domain waveform;

    Comparing the target time domain waveform with a preset time domain waveform;

    Determining a plurality of extreme points of the target time domain waveform when the comparison between the target time domain waveform and the preset time domain waveform is successful;

    The target feature value is determined according to the multiple extreme points.
11. An electronic device, characterized by comprising a processor and a memory, the memory is used to store one or more programs, and is configured to be executed by the processor, and the program includes a program for executing Instructions for the steps in any one of the methods.
12. A computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1-5.
13. A computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of claims 1-5 The method described.

Images