CN107492117B - Micro-vibration functional image imaging method based on video - Google Patents
Micro-vibration functional image imaging method based on video Download PDFInfo
- Publication number
- CN107492117B CN107492117B CN201710623457.1A CN201710623457A CN107492117B CN 107492117 B CN107492117 B CN 107492117B CN 201710623457 A CN201710623457 A CN 201710623457A CN 107492117 B CN107492117 B CN 107492117B
- Authority
- CN
- China
- Prior art keywords
- video
- image
- vibration
- micro
- iωt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 8
- 230000003321 amplification Effects 0.000 claims abstract description 21
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract description 21
- 238000004458 analytical method Methods 0.000 claims abstract description 17
- 230000009471 action Effects 0.000 claims abstract description 15
- 238000009499 grossing Methods 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims description 36
- 206010044565 Tremor Diseases 0.000 claims description 27
- 238000000034 method Methods 0.000 claims description 22
- 230000003595 spectral effect Effects 0.000 claims description 13
- 238000006073 displacement reaction Methods 0.000 claims description 11
- 238000001228 spectrum Methods 0.000 claims description 3
- 238000012800 visualization Methods 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims description 2
- 210000003205 muscle Anatomy 0.000 description 14
- 230000006870 function Effects 0.000 description 10
- 238000002604 ultrasonography Methods 0.000 description 8
- 230000000007 visual effect Effects 0.000 description 7
- 241000282414 Homo sapiens Species 0.000 description 6
- 201000000585 muscular atrophy Diseases 0.000 description 6
- 206010028289 Muscle atrophy Diseases 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000000354 decomposition reaction Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000020763 muscle atrophy Effects 0.000 description 3
- 238000011587 new zealand white rabbit Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 210000004204 blood vessel Anatomy 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000005284 excitation Effects 0.000 description 2
- 230000010363 phase shift Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000010998 test method Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 210000000689 upper leg Anatomy 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 241000283973 Oryctolagus cuniculus Species 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007728 cost analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 210000000887 face Anatomy 0.000 description 1
- 230000003387 muscular Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 238000007794 visualization technique Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/262—Analysis of motion using transform domain methods, e.g. Fourier domain methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Abstract
The invention discloses a video-based micro-vibration functional image imaging method. The imaging method comprises a video acquisition module, a signal preprocessing module, a signal analysis module and a micro-vibration functional image generation module; after the video of an object is collected, preprocessing such as denoising, interframe smoothing and super-resolution is carried out on the original video, action amplification and vibration track analysis are carried out on the preprocessed video, and finally a micro-vibration function image, a micro-vibration frequency image and a micro-vibration angle image are obtained. The invention can analyze and visualize the tiny vibration of sub-pixel level in the video.
Description
Technical Field
The invention relates to a video-based micro-vibration functional image imaging method, and belongs to the technical field of digital video processing.
Background
Video processing technology refers to a technology for analyzing and processing videos by using a computer to achieve a required result. Video processing generally refers to digital video processing, and digital video processing technology is mainly based on digital image processing technology and extraction and utilization of information related to continuous frames in video. The digital video refers to single-channel or multi-channel time sequence data based on a two-dimensional array obtained by shooting with a mobile phone camera, an industrial camera, a video camera, an ultrasonic device, a magnetic resonance device and the like, wherein elements of the two-dimensional array are called pixels, and values of the elements are called gray values. The digital image processing technology is a method and technology for processing an image by removing noise, enhancing, restoring, multi-scale transforming, matching, segmenting, extracting features and the like through a computer, and is widely applied to industries such as agriculture and animal husbandry, forestry, environment, military, industry, medicine and the like.
The perception ability of human eyes to tiny vibration of an object is limited, but the video processing technology can surpass the vibration recognition ability of human eyes under specific conditions. For example, the voice of a human speaking has an acoustic pressure that causes forced vibration of a stationary chair. We can see these forced vibrations through a high-speed ultra-high resolution camera, but the ordinary person is almost impossible to see with the naked eye. For most people, forced vibration caused by such sound pressure has never been noticed even. Not only can external excitation cause forced vibration of the object, but internal excitation of the object itself can also cause forced vibration of the object. For example, the heart beat of the human body drives blood vessels around the tissue, and these beats cause the muscle tissue around the blood vessels to generate forced vibration.
The function image is a visual image reflecting some characteristic of the object, for example, elasticity imaging in an ultrasonic image is to represent the magnitude of Young's modulus of each part of the object. The micro-vibration function is a visual image of vibration which is not perceivable to human eyes, and is convenient for human beings to observe vibration characteristics of all parts of the object, such as vibration energy, frequency, angle and the like, which are ignored before.
One always wants to know the situation of the micro-vibration of the object, but it faces two difficulties: firstly, the traditional video processing method cannot analyze the tiny vibration at the sub-pixel level, and the vibration with smaller analysis is more easily interfered by noise (for example, the invention patent application publication No. CN 104240250A of the invention, which is granted, of Zhao Hai et al in a rainshed vibration detection method of a high-speed railway station based on video and image registration); secondly, the conventional vibration visualization method cannot decompose the vibration into vibrations with different frequencies, and further cannot visualize sub-pixel level micro-vibration conditions of different frequencies of the object (for example, visual mechanical vibration detection system of people in Viffei et al, application publication No. CN 104568118A, in examination).
Disclosure of Invention
The invention provides a method for analyzing the micro-vibration at the sub-pixel level based on a video and obtaining a visual micro-vibration function image, which aims to overcome the defect that the traditional video processing method cannot analyze the micro-vibration at the sub-pixel level and further is difficult to visualize. The micro-vibration function image generated by the method can reflect the sub-pixel level vibration condition of the object on different frequency scales, so that the vibration characteristic of the object is evaluated.
The invention provides the following technical scheme for solving the technical problems:
1) the invention relates to a video-based micro-vibration functional image imaging method, which comprises the following steps:
A. video acquisition: acquiring video signals of an object to be detected in real time and obtaining a video V of the object to be detected0；
B. Video preprocessing: video V0Denoising, inter-frame smoothing and super-resolution processing are carried out to obtain a preprocessed video V;
C. signal analysis: dividing the preprocessed video V into m × n rectangular ROI (Region of interest) videos V with the same sizeiI is 1, 2, 3, …, m n, m and n are the number of horizontal and vertical ROIs respectively, the step size of the sliding window is set to L, and further, V is set for each videoiPerforming action amplification based on a certain frequency band range omega to obtain an amplified video Viω(ii) a Omega denotes a preset frequency range, and the unit is Hz; finally, vibration track analysis based on image matching is carried out to obtain each video ViωPSD (power spectral density distribution) corresponding to any angle thetaiωθ；
D. Generating a micro-tremor function image: for each ROI, selecting a PSD of the power spectral density in all of its respective frequency bandsiωθMiddle maximum energy EiCorresponding frequency ωiAngle thetaiAnd output Ei、ω*i、θ*iThese three parameters; finally, the parameters E corresponding to all the ROIs are fusediObtaining a micro-vibration energy image E; fusing the parameters omega corresponding to all ROIsiObtaining a micro-vibration frequency image omega; fusing the parameters theta corresponding to all ROIs respectivelyiObtaining a micro-vibration angle image theta; the micro-vibration energy image, the micro-vibration frequency image and the micro-vibration angle image belong to the micro-vibration function image.
2) In the step B, firstly V is treated0Adopts an entropy regularization denoising method and uses L1/2Measuring the norm; then selecting V0V is carried out on 1-4 frames adjacent to each other in the middle through multi-frame averaging0And performing interframe smooth denoising processing on the video image.
3) In the step C, the motion is releasedBig for ViEach frame image BijCarrying out pyramid decomposition of s scales with directivity to obtain s scale representation B of each frame imageijsTo B, pairijsTwo-dimensional Fourier transform is carried out to obtain corresponding amplitude distribution ABijsAnd phase distribution PBijs(ii) a And for each phase distribution PBijsExtraction of displacement characteristic phase distribution PB by phase shift methodijs(ii) a For PBijsA certain frequency sub-band omega adopts a preset corresponding gain coefficient KsCarrying out displacement characteristic phase amplification; further, inverse Fourier transform is carried out, and then multi-scale fusion in the corresponding direction is carried out to obtain a single-frame micro-vibration space motion amplification image Bijω(ii) a Finally, the images B amplified by the actions are fused in sequenceijωObtaining video V with amplified micro-vibration spatial motioniω；BijRepresents ViJ frame image in the video; s is a positive integer.
4) In the step C, the video V is processediωUsing video V in image matching based judder track analysisiωThe first frame of the picture is used as a reference frame, and other frames are subjected to image matching with the first frame, so that the transverse displacement U of the video picture changing along with time is obtainediωtAnd a longitudinal displacement ViωtT is time; then, the U of the orthogonal coordinate system is determinediωtAnd ViωtConverting to a polar coordinate system to obtain rhoiωtAnd thetaiωt(ii) a Final angle θiωtCorresponding rhoiωtCarrying out power spectrum estimation to obtain each video ViωPSD (power spectral density distribution) of flutter track corresponding to any angle thetaiωθ。
5) In said step D, all Ei、ω*i、θ*iNormalized to [0,1 ]]And then performing two-dimensional visualization.
The operational procedure of the present invention (see fig. 1 and 2): the method comprises the steps of firstly acquiring an original video of an object through a video acquisition module, then carrying out various preprocessing operations on the original video through a signal preprocessing module, then generating a final micro-vibration functional image through a signal analysis module and finally a micro-vibration functional image generation module.
Generally, the original video will have image noise, so the entropy regularization denoising method is selected in the signal preprocessing stage and used L1/2Norm measure L2Norm denoising, which does not preserve image edges well although smoothing the interior of image edges well, L1The norm, while good at preserving image edges, can exhibit a step effect that makes image edges unnatural1Norm sum L2L norm1/2And (4) norm.
The purpose of the inter-frame smoothing process is also denoising. When the frame rate of the video reaches a certain degree, the noise can be mutually offset by alternately superposing the multi-frame images, so that the signal-to-noise ratio of the video is improved, and useful signals are more highlighted.
The super-resolution processing uses an image super-resolution processing technology, and the time resolution is replaced by the space resolution, so that the resolution of the original video can be improved, and the analysis capability of the invention on the sub-pixel level micro vibration is improved.
The spatial frequency of the image is an index representing the intensity of the change of the gray level in the image, and is the gradient of the gray level on a plane space. Such as: a large area of desert is an area with slow gray level change in an image, and the corresponding frequency value is very low; and for the edge region with violent surface attribute transformation, the image is a region with violent gray scale change, and the corresponding frequency value is higher. For an image, the edge part of the image is a sudden change part which changes rapidly, so that the reaction is a high-frequency component in a frequency domain; the noise of the image is mostly a high frequency part; the gently changing part of the image is a low-frequency component.
And carrying out multi-scale image pyramid decomposition on the image to obtain images with different spatial frequencies. The low-frequency information of the image forms the basic gray level of the image, and the determining effect on the image structure is small; the intermediate frequency information of the image determines the basic structure of the image and forms the main edge structure of the image; the high frequency information of the image forms the edges and details of the image. The characteristics of the image can be observed by transforming the image from a gray-scale distribution to a frequency distribution by fourier transform.
The purpose of image matching is to obtain a "displacement-time" map of the vibration of a single image region over time, typically using time as the abscissa and displacement as the ordinate. Then, the invention performs fourier transform on the time series curve to obtain a power spectral density map of the time series curve. On the power spectral density map, the energy magnitudes corresponding to different frequencies can be seen.
Compared with the prior art, the invention has the advantages that:
the invention realizes the low-cost analysis of the micro-vibration condition of the non-contact object, and can amplify and analyze the sub-pixel level micro-vibration; the method realizes the visual micro-vibration function image of the sub-pixel level vibration condition of the object on the comprehensive different frequency scales, thereby being capable of evaluating the vibration mode of the object based on the three micro-vibration function images.
Drawings
FIG. 1 is a schematic diagram of a logic module of the present invention, with solid arrows indicating the direction of data flow.
FIG. 2 is a detailed diagram of the logic module of the present invention, and the solid arrows indicate the data flow direction. 1. A video acquisition module; 2. a signal preprocessing module; 3. a signal analysis module; 4. and a micro-vibration functional image generation module. 5. An action amplification module; 6. and a vibration track analysis module. 7. A denoising module; 8. an interframe smoothing module; 9. and a super-resolution module. 10. A multi-scale transformation module; 11. a two-dimensional Fourier transform module; 12. an amplification phase distribution module; 13. a two-dimensional inverse Fourier transform module; 14. a multi-scale inverse transform module; 15. a track obtaining module; 16. a coordinate transformation module; 17. acquiring a power spectral density module; 18. a maximum energy comparison module; 19. a fusion module; 20. a micro-vibration energy image generation module; 21. a micro-vibration frequency image generation module; 22. and a micro-vibration angle image generation module.
Figure 3 is an ultrasound video screenshot with a corresponding ultrasound video-based functional image of a tremor. FIG. 3(A) is a B-mode ultrasonic image of the affected side of severe muscular atrophy, and FIG. 3(B) is a B-mode ultrasonic image of the healthy muscle side; fig. 3(a.1) is a micro-tremor energy image obtained from B-mode ultrasound video processing of the affected side of severe muscle atrophy, and fig. 3(b.1) is a micro-tremor energy image obtained from B-mode ultrasound video processing of the healthy muscle side, with lighter gray areas representing higher vibration energy and darker gray areas representing lower vibration energy; fig. 3(a.2) is a micro-tremor frequency image obtained from B-mode ultrasound video processing on the affected side of severe muscle atrophy, and fig. 3(b.2) is a micro-tremor frequency image obtained from B-mode ultrasound video processing on the healthy muscle side, with lighter gray areas representing higher vibration frequencies and darker gray areas representing lower vibration frequencies; fig. 3(a.3) is a micro-tremor angle image obtained from B-mode ultrasound video processing of the affected side of severe muscle atrophy, fig. 3(b.3) is a micro-tremor angle image obtained from B-mode ultrasound video processing of the healthy muscle side, the pointing direction of the black arrow is the initial direction of the main angle of micro-vibration, and a longer black arrow represents a larger amplitude of vibration.
Fig. 4 is a screenshot of a cell phone video and a corresponding camera video-based micro-tremor function image. Fig. 4(C) is an original curtain video screenshot taken by the mobile phone, and fig. 4(D) is a curtain video screenshot after the action amplification processing; fig. 4(c.1) is a micro-tremor energy image obtained from the original curtain video processing, and fig. 4(d.1) is a micro-tremor energy image obtained from the curtain video processing after the motion amplification processing, in which the lighter gray areas represent higher vibration energy and the darker gray areas represent lower vibration energy; fig. 4(c.2) is a micro-tremor frequency image obtained from the original curtain video processing, and fig. 4(d.2) is a micro-tremor frequency image obtained from the curtain video processing after the action amplification processing, in which the lighter gray areas represent higher vibration frequencies and the darker gray areas represent lower vibration frequencies; fig. 4(c.3) is a micro-tremor angle image obtained from the original curtain video processing, and fig. 4(d.3) is a micro-tremor angle image obtained from the curtain video processing after the motion amplification processing, the black arrow points in the initial direction of the main angle of the micro-vibration, and a longer black arrow indicates a larger amplitude of the vibration.
Detailed Description
The invention will be further illustrated by the following specific examples in order to better understand the invention, without however being limited thereto. The experimental procedures used in the following examples are all conventional procedures unless otherwise specified. The video capture devices used in the following examples are commercially available, unless otherwise specified.
1. The conventional implementation flow is as follows:
1) acquisition of B-mode ultrasonic video V by using linear array probe of common ultrasonic equipment0The video frame rate is 30Hz, the duration is 5 seconds, the resolution is 400 x 400, and 150 frames of images are obtained in total;
2) for original video V0Denoising, interframe smoothing and super-resolution processing are carried out, a denoising selective entropy regularization method is adopted, and L is selected1/2Measuring the norm; selecting 3 frames adjacent to the reference frame by inter-frame smoothing; super-resolution processing selects the Maximum a Posteriori probability Method (MAP);
3) the signal analysis step is responsible for dividing the preprocessed video V into 40 by 40 rectangular ROI videos V with the same sizei( i 1, 2, 3, …,40 x 40; 40 and 40 for the number of transverse and longitudinal ROIs respectively; sliding window step size set to 40; further, V for each videoiAmplifying actions based on 0.1 Hz-1 Hz, 1 Hz-2 Hz, 2 Hz-3 Hz, … and 99 Hz-100 Hz to obtain 100 amplified videos Viω(ii) a Finally, flutter track analysis based on phase registration is carried out to obtain each video ViωPSD (Power spectral Density) distribution corresponding to 90 angles of 1 degree, 2 degrees, 3 degrees, … degrees and 90 degreesiωθ；
4) The step of generating the micro-dithering functional image is responsible for selecting, for each ROI, the PSD of the power spectral density in all the corresponding frequency bandsiωθMiddle maximum energy EiCorresponding frequency ωiAngle thetai(ii) a For example, in this video data, we find V23The energy of the ROI video in the 40 DEG direction of 67-68 Hz is maximum, the energy is 48, and then the ROI video is positioned at V23This ROI region outputs Ei＝48、ω*i＝67～68Hz、θ*iThree parameters of 40 degrees; finally, the parameters E corresponding to all the ROIs are fusediTo obtainMicro-tremor energy image E; fusing the parameters omega corresponding to all ROIsiObtaining a micro-vibration frequency image omega; fusing the parameters theta corresponding to all ROIs respectivelyiObtaining a micro-vibration angle image theta; normalizing to [0,1 ] for all Ei, ω i, θ i]After the interval, performing visualization;
5) the operation amplification processing flow is as follows: action amplification in Signal analysis Module for ViEach frame image Bij(BijRepresents ViJ is 150) in the jth frame image in the video, carrying out 10-scale pyramid decomposition with directionality to obtain 10-scale representation B of each frame imageijsTo B, pairijsTwo-dimensional Fourier transform is carried out to obtain corresponding amplitude distribution ABijsAnd phase distribution PBijs(ii) a And for each phase distribution PBijsExtraction of displacement characteristic phase distribution PB by phase shift methodijs(ii) a For PBijsThe frequency sub-band of 1-1.7Hz adopts a preset corresponding gain coefficient Ks(Ks10) carrying out displacement characteristic phase amplification; further, inverse Fourier transform is carried out, and then multi-scale fusion in the corresponding direction is carried out to obtain a single-frame micro-vibration space motion amplification image Bijω(ii) a Finally, the images B amplified by the actions are fused in sequenceijωObtaining video V with amplified micro-vibration spatial motioniω；
6) The power spectral density distribution of the microvibration trace is obtained by: signal analysis module to video ViωUsing video V in image matching based judder track analysisiωThe first frame of the picture is used as a reference frame, and other frames are subjected to image matching with the first frame, so that the transverse displacement U of the video picture changing along with time is obtainediωtAnd a longitudinal displacement ViωtT is time; then, the U of the orthogonal coordinate system is determinediωtAnd ViωtConverting to a polar coordinate system to obtain rhoiωtAnd thetaiωt(ii) a Final angle θiωtCorresponding rhoiωtCarrying out power spectrum estimation to obtain each video ViωPSD (power spectral density distribution) of flutter track corresponding to any angle thetaiωθ。
2. And (3) micro-vibration functional image test based on ultrasonic video:
the test procedure was as follows:
1) preparing a new Zealand white rabbit, and enabling the thigh on the left side of the rabbit to have severe muscular atrophy in an operation mode, wherein the corresponding thigh on the right side does not have any operation;
2) acquiring a B-type ultrasonic image video of the severe muscular atrophy side of the New Zealand white rabbit and a B-type ultrasonic image video of the healthy muscular side of the New Zealand white rabbit by using a linear array probe of common ultrasonic equipment in a longitudinal cutting mode;
3) the generation of the micro-tremor functional image was performed on a video of atrophic muscles and a video of healthy muscles using the steps and parameters in a conventional real-time procedure. In general, the vibration of the atrophied muscle should be more random and have no badge, and the vibration of the healthy muscle should be more regular. Comparing the experimental results (see fig. 3), we can find that the vibration energy distribution of the atrophy muscle is relatively random on the micro-tremor energy image, and in comparison, the vibration energy of the healthy muscle is concentrated and distributed in certain areas, which illustrates the feasibility of distinguishing different objects by the micro-tremor energy image; on the micro-tremor frequency image, the vibration frequencies of the atrophic muscles are distributed in a scattered manner, and in comparison, the vibration frequencies of the healthy muscles are distributed in certain areas in a concentrated manner, so that the feasibility of distinguishing different objects by using the micro-tremor frequency image is illustrated; in the micro-tremor angle image, the vibration angles of the atrophic muscles are relatively scattered and small in amplitude, and the vibration angles of the healthy muscles are relatively regular and large in amplitude, which illustrates the feasibility of distinguishing different objects by using the micro-tremor angle image.
3. The method comprises the following steps of (1) micro-vibration functional image test based on a mobile phone camera video:
the test procedure was as follows:
1) when no wind exists, a common mobile phone camera is aligned to a static curtain to shoot a curtain video with the length of 4 seconds;
2) performing action amplification processing on the original curtain video, wherein the frequency is selected to be 0.5-2 Hz, and obtaining the curtain video after action amplification;
3) and (3) generating a micro-vibration function image of the original curtain video and the curtain video after motion amplification by using steps and parameters in a conventional real-time process. It is common that a slight air flow causes the curtain to wobble slightly, which is difficult to detect by the naked eye. Comparing the experimental results (see fig. 4), we can find that: on the micro-vibration energy image, the vibration energy area of the curtain video after action amplification is matched with the result observed by naked eyes, and the vibration energy area of the original video shows almost no vibration, which illustrates the feasibility of reflecting the sub-pixel level vibration energy of an object in the video by the micro-vibration energy image; on the micro-tremor frequency image, the vibration frequency region of the curtain video after action amplification is matched with the result of visual observation, but the vibration frequency region of the original video shows almost no vibration, which illustrates the feasibility of reflecting the frequency of sub-pixel level vibration of an object in the video by the micro-tremor frequency image; on the micro-vibration angle image, the vibration direction of the curtain video after action amplification is matched with the result of visual observation, the vibration amplitude is large, the vibration direction of the original video is very random, and the vibration amplitude is small, so that the feasibility of reflecting the angle of sub-pixel level vibration of an object in the video by the micro-vibration angle image is demonstrated.
Claims (4)
1. A method for imaging a video-based micro-tremor function image, comprising the steps of:
A. video acquisition: acquiring video signals of an object to be detected in real time and obtaining a video V of the object to be detected0；
B. Video preprocessing: video V0Denoising, inter-frame smoothing and super-resolution processing are carried out to obtain a preprocessed video V;
C. signal analysis: dividing the preprocessed video V into m × n rectangular ROI (Region of interest) videos V with the same sizeiI is 1, 2, 3, …, m n, m and n are the number of horizontal and vertical ROIs respectively, the step size of the sliding window is set to L, and further, V is set for each videoiPerforming action amplification based on a certain frequency band range omega to obtain an amplified video Viω(ii) a Omega denotes a preset frequency range, and the unit is Hz; finally proceed based onAnalyzing the vibration track matched with the image to obtain each video ViωPSD (power spectral density distribution) corresponding to any angle thetaiωθ；
D. Generating a micro-tremor function image: for each ROI, selecting a PSD of the power spectral density in all of its respective frequency bandsiωθMiddle maximum energy EiCorresponding frequency ωiAngle thetaiAnd output Ei、ω*i、θ*iThese three parameters; finally, the parameters E corresponding to all the ROIs are fusediObtaining a micro-vibration energy image E; fusing the parameters omega corresponding to all ROIsiObtaining a micro-vibration frequency image omega; fusing the parameters theta corresponding to all ROIs respectivelyiObtaining a micro-vibration angle image theta; the micro-vibration energy image, the micro-vibration frequency image and the micro-vibration angle image belong to the micro-vibration function image.
2. The method of claim 1, wherein: in the step B, firstly V is treated0Adopts an entropy regularization denoising method and uses L1/2Measuring the norm; then selecting V0V is carried out on 1-4 frames adjacent to each other in the middle through multi-frame averaging0And performing interframe smooth denoising processing on the video image.
3. The method of claim 1, wherein: in the step C, the video V is processediωUsing video V in image matching based judder track analysisiωThe first frame of the picture is used as a reference frame, and other frames are subjected to image matching with the first frame, so that the transverse displacement U of the video picture changing along with time is obtainediωtAnd a longitudinal displacement ViωtT is time; then, the U of the orthogonal coordinate system is determinediωtAnd ViωtConverting to a polar coordinate system to obtain rhoiωtAnd thetaiωt(ii) a Final angle θiωtCorresponding rhoiωtCarrying out power spectrum estimation to obtain each video ViωThe power spectral density corresponding to the flutter track at any angle thetaCloth PSDiωθ。
4. The method of claim 1, wherein: in said step D, all Ei、ω*i、θ*iNormalized to [0,1 ]]And then performing two-dimensional visualization.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710623457.1A CN107492117B (en) | 2017-07-27 | 2017-07-27 | Micro-vibration functional image imaging method based on video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710623457.1A CN107492117B (en) | 2017-07-27 | 2017-07-27 | Micro-vibration functional image imaging method based on video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107492117A CN107492117A (en) | 2017-12-19 |
| CN107492117B true CN107492117B (en) | 2020-07-28 |
Family
ID=60644927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710623457.1A Active CN107492117B (en) | 2017-07-27 | 2017-07-27 | Micro-vibration functional image imaging method based on video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107492117B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108537785B (en) * | 2018-04-04 | 2021-04-30 | 北京大学人民医院 | Uterine ultrasound micro-peristalsis video processing method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1322305A (en) * | 1999-09-29 | 2001-11-14 | 奥林巴斯光学工业株式会社 | MIcrosope image observing system, method for controlling the same, and computer-readable recorded medium on which control program is recorded |
| CN103330576A (en) * | 2013-06-09 | 2013-10-02 | 西安交通大学 | Micro-elasticity imaging method based on tissue microbubble dynamics model |
| CN104240250A (en) * | 2014-09-16 | 2014-12-24 | 铁道第三勘察设计院集团有限公司 | High-speed rail station canopy vibration detection method based on video and image registration |
| CN104568118A (en) * | 2015-01-09 | 2015-04-29 | 江苏大学 | Visual mechanical vibration detecting system |
-
2017
- 2017-07-27 CN CN201710623457.1A patent/CN107492117B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1322305A (en) * | 1999-09-29 | 2001-11-14 | 奥林巴斯光学工业株式会社 | MIcrosope image observing system, method for controlling the same, and computer-readable recorded medium on which control program is recorded |
| CN103330576A (en) * | 2013-06-09 | 2013-10-02 | 西安交通大学 | Micro-elasticity imaging method based on tissue microbubble dynamics model |
| CN104240250A (en) * | 2014-09-16 | 2014-12-24 | 铁道第三勘察设计院集团有限公司 | High-speed rail station canopy vibration detection method based on video and image registration |
| CN104568118A (en) * | 2015-01-09 | 2015-04-29 | 江苏大学 | Visual mechanical vibration detecting system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107492117A (en) | 2017-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR100646715B1 (en) | Method for improving a quality of a 2-d ultrasound image by post-processing | |
| CN106127711A (en) | Shearlet conversion and quick two-sided filter image de-noising method | |
| US8852105B2 (en) | Ultrasound system and method of forming ultrasound images | |
| US20130165788A1 (en) | Ultrasonic diagnostic apparatus, medical image processing apparatus, and medical image processing method | |
| KR101610874B1 (en) | Module for Processing Ultrasonic Signal Based on Spatial Coherence and Method for Processing Ultrasonic Signal | |
| CN110101362A (en) | A method of the picture noise about OCT and OCTA removes | |
| CN107845079A (en) | 3D shearlet medicine CT video denoising methods based on compact schemes | |
| US10012619B2 (en) | Imaging apparatus, ultrasonic imaging apparatus, method of processing an image, and method of processing an ultrasonic image | |
| Magud et al. | Medical ultrasound image speckle noise reduction by adaptive median filter | |
| Hemmsen et al. | Tissue harmonic synthetic aperture ultrasound imaging | |
| Bhatnagar et al. | A new image fusion technique based on directive contrast | |
| CN109965905B (en) | Contrast region detection imaging method based on deep learning | |
| CN107492117B (en) | Micro-vibration functional image imaging method based on video | |
| Sharumathi et al. | A survey on various image enhancement techniques for underwater acoustic images | |
| Kumar et al. | Reduction of speckle noise from medical images using principal component analysis image fusion | |
| Gopi et al. | Capsule endoscopic colour image denoising using complex wavelet transform | |
| Wang et al. | 2-d fft for periodic noise removal on strain image & & | |
| CN111766306B (en) | Full-focus imaging method and device | |
| Magud et al. | An algorithm for medical ultrasound image enhancement by speckle noise reduction | |
| JP6274495B2 (en) | Image processing apparatus and ultrasonic diagnostic apparatus | |
| Haribabu et al. | Feature level based multimodal medical image fusion with Hadamard transform | |
| Tang et al. | Deep Learning Radar for High Fidelity Heart Sound Recovery in Real-World Scenarios | |
| Teku et al. | Contrast Enhanced Low-light Visible and Infrared Image Fusion. | |
| Balagalla et al. | A review on ultrasound image pre-processing, segmentation and compression for enhanced image storage and transmission | |
| JP2000229081A5 (en) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |