CN102819662A - Computing method of video fluid height - Google Patents

Abstract

The invention discloses a computing method of a video fluid height. The computing method comprises the following steps: firstly, initializing motion vectors of a fluid according to fluid motion features; utilizing LBM to compute the height of fluid particles in accordance with a result of the motion vectors; denoising and smoothing the height; utilizing LBM to perform recurrence on the height of continuous multiframes to obtain a height computing result with physical motion features of the fluid according to the continuity of the fluid motion; utilizing a linear interpolation of fluid particle distribution functions to obtain a height result of uniform changes of continuous motion fluids; and finally correcting the fluid height by using intermediate frames. The computing method provided by the invention is simple, convenient and fast. By using the computing method, a fluid height field of continuous changes can be obtained, and a real-time height result is produced. The computing method can be suitable for constructing virtual scenes of natural landscapes and has an application value; and with the adoption of the computing method, the real-time interaction of the fluid motion in the virtual scenes can be realized effectively, and the poor real-time capability in reconstruction based on a vision method is overcome.

Description

Method for calculating height of video fluid

Technical Field

The invention relates to a method for calculating the height of a video fluid.

Background

At present, in augmented reality research, preliminary results are obtained for research on rigid body virtual-real combination, and for scenes containing fluids such as water, smoke, cloud, fire and the like, because the fluids belong to strong textures, and the problems of shielding and reproduction exist in the motion process, the research on the augmented reality technology of the fluids has certain challenges, and the results obtained in the research are not significant enough. The fluid reconstruction technology is a key problem in research, how to quickly and accurately calculate the height of the fluid and establish an augmented reality scene of a fluid natural landscape with reality sense and interactivity still remains a very challenging subject in the field, and the research of the fluid reconstruction technology has important practical significance and application value.

With the development of image processing and computer vision technologies, some technologies and methods have emerged for the study of natural landscape scene height calculations. People use the characteristics of color, texture, shape, motion and the like in the image to research and obtain some achievements. The existing method calculates the height of the scene from the image texture, and obtains the height value of the scene through the variation characteristics of the size, the shape, the density and the like of the texture primitive.

Because the fluid belongs to strong texture, the attribute that the texture cannot be kept unchanged in the movement process, the height of the fluid is difficult to accurately calculate by utilizing the texture information; some researches are carried out by utilizing a color invariance rule and through the strength correlation of a local area, the height information of a three-dimensional scene is calculated, and therefore the scene is reconstructed; in the existing researches, some researches calculate space-time continuous motion information from different views to obtain corresponding points of two frames of images, and then combine the camera calibration, three-dimensional coordinate recovery and other technologies to obtain the height information of a scene; in other studies, the height of the object is calculated by extracting physical characteristics of the motion and recovering and calibrating internal and external parameters of the camera. The main problems in the height calculation method based on the motion characteristics are that: the determination of the motion characteristics of the object is difficult; the parameters of the camera are usually determined by adopting an offline batch processing method, and the methods have the problems of high complexity and large calculation amount and are difficult to complete by utilizing an online method, so that the real-time requirement cannot be met.

Systematic studies have been conducted on the reconstruction of fluids, and the typical result of this study is a method of shape recovery (SFS) based on darkness. The method estimates the normal direction of the object surface by using the intensity change of the object surface under different illumination conditions, thereby achieving the aim of reconstruction.

In recent years, some studies have been conducted in order to improve the accuracy and the sense of realism of reconstruction. The existing research comprises the steps of calculating the height information of a scene by using a method of combining cubic splines and SFS, and researching an algorithm for solving the SFS problem; people also use the SFS method to carry out smoothing processing on the height calculation result, and a better reconstruction result is obtained.

Some people use SFS and motion vectors of fluid to research, and realize height calculation of fluid based on the principle of mass conservation, which can obtain satisfactory height calculation results for non-transparent water areas, but in the case of strong light irradiation, such as water surfaces containing reflection or strong brightness, the accuracy of the height calculation results can be influenced.

Disclosure of Invention

In order to solve the above problems in the prior art, the present invention provides an effective method for calculating the height of a video fluid, which can obtain a height field of fluid motion reflecting the continuous motion characteristics of the fluid, and quickly and accurately calculate the fluid height.

The specific technical scheme of the invention is as follows:

a method for calculating the height of a video fluid comprises the following five steps:

1) calculating a fluid motion vector;

2) calculating the height of the combination of the motion vector and the LBM;

3) denoising and smoothing the height calculation result;

4) the constraint is performed by a continuity equation of the fluid motion.

5) And (6) correcting the height.

The specific technical details in the invention are as follows:

(1) calculation of fluid motion vectors

Since the neighborhood where the fluid particle is located has the property of keeping the intensity component constant before and after the movement. According to this feature, the fluid motion vector is initialized using the regional intensity correlation. And then clustering is carried out.

Defining a feature vector for clustering, wherein the form of vector = [ x, y, u, v, sigu, sigv ], wherein (x, y) is the position of a fluid particle, u and v respectively represent the components of the motion vector of the particle along the x direction and the y direction, sigu and sigv respectively represent the signs of u and v, and if the sign is positive, the sign is positive 1, if the sign is negative, and otherwise, the sign is 0. And finally, in order to obtain a dense motion vector field, adopting a shortest distance linear interpolation method, finding out the particle closest to each particle, and further obtaining a dense motion vector result by adopting a linear interpolation method.

The calculation of the fluid motion vector specifically includes:

1) and initializing the fluid motion vector by utilizing two continuous frames of images and adopting a region correlation operation mode.

2) Feature vector using motion vector = [ x, y, u, v, sigu, sigv = [ x, y, u, v, sigu, v = [ x, y, v ])]Clustering is performed, in particular, if the particle P_iAnd P_jThe feature vectors of the motion vector of (a) are respectively: vector_i=[x_pi,y_pi,u_pi，v_pi,sigu_pi,sigv_pi]And vector_j=[x_pj,y_pj,u_pj,v_pj,sigu_pj,sigv_pj]And they satisfy the formulae (1) to (4) at the same time, the particle P is considered to be_iAnd P_jHave similarities and group them into the same category.

$|\sqrt{{(x_{\mathrm{pi}}-x_{\mathrm{pj}})}^2+{(y_{\mathrm{pi}}-y_{\mathrm{pj}})}^2}<{\mathrm{threshold}}_{\mathrm{dis}}---\left(1\right)$

$\left|\sqrt{\mathrm{angle}({\mathrm{vec}}_{\mathrm{pi}},{\mathrm{vec}}_{\mathrm{pj}})}\right|<{\mathrm{threshold}}_{\mathrm{angle}}---\left(2\right)$

sigu_pi*sigu_pj≥0 （3）

sigv_pi*sigv_pj≥0 （4）

Among them, threshold_disRepresents a particle P_iAnd P_jA threshold value of the distance between; vec_piAnd vec_pjRespectively represent particles P_iAnd P_jMotion vector (u) of_pi,v_pi) And (u)_pj,v_pj)；threshold_angleA threshold value indicating the direction of their motion vectors.

3) And counting the scale of each category, and keeping the result of the category with larger scale in the clustering result. Let any one class D whose number of particles in class is D_numI.e. the scale of this class is D_numIf the motion vector satisfies the formula (5), the intra-class particle motion vector is considered to be more accurate, and the flag value of the particles is set to be 1; otherwise, setting the flag value of the particles to be 0;

D_num>threshold_D （5）

among them, threshold_DIs a threshold for the number of particles.

4) For any one flParticles P with an ag value of 1_mIn the surrounding area, a particle P having a flag value of 1 and closest to the selected particle P is selected_nAnd obtaining the motion vector results of the particles with the flag value of 0 on the connecting line of the two particles by adopting a linear interpolation method.

(2) Motion vector and LBM combined height calculation

LBM is a simplified microscopic model of fluid motion that represents the statistical results of microscopic thermal motion of a large number of fluid particles in the form of a particle distribution function that, through its evolution, further reflects the macroscopic motion of the fluid.

A standard orientation is defined. The nominal direction is the direction in which the central particle is subjected to forces from the surrounding 8 particles, which forces contribute to the high evolution of the central particle in three dimensions.

If the motion vector direction of the surrounding particle is the same as a certain standard direction, it will have a larger force on the central particle in that direction. According to the characteristic, the motion vectors of the surrounding particles are projected to the standard direction, and the effect of the surrounding particles on the central particles is determined. When the included angle between the motion direction component of the surrounding particles and the corresponding standard direction is greater than or equal to 90 degrees, the acting force generated on the central particles in the standard direction is zero.

The height of the fluid particles in three-dimensional space is f_iThe sum of (i ═ 1.., 8), the larger the sum, the larger the height value of the particle in three-dimensional space, and vice versa.

The height calculation combining the motion vector and the LBM specifically includes:

1) assume that any particle S (x, y) whose height has not been calculated has a surrounding particle T_i(x, y) (i ═ 1.., 8), their corresponding motion vectors are (u @_i,v_i). Particle T_iThe standard direction of the force of (x, y) on the particle S (x, y) is F_i. Set the standard direction F_iAnd vector Hor (u)_i0) angle of theta₁，F_iAnd vectorVer(0，v_i) Is theta₂. Calculation of the distribution function f of the particles S (x, y) by equation (6)_i(x，y，t)。

2) The height h of the particle S (x, y) in the three-dimensional space is calculated using equation (7).

<math> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>0</mn> </mrow> <mi>d</mi> </munderover> <msub> <mi>f</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>7</mn> <mo>)</mo> </mrow> </mrow> </math>

3) And judging whether the heights of all the particles are calculated, if so, turning to the step 4), and otherwise, turning to the step 1).

4) The algorithm ends.

(3) Denoising and smoothing height calculation results

Firstly, denoising a height calculation result by using the continuity of height change, and then smoothing the height calculation result in a local area. The method comprises the following specific steps:

1) for any unprocessed particle W (x, y), the second order difference of the height h (x, y) is calculated using equations (8) and (9) and is recorded as

And

<math> <mrow> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <mi>h</mi> </mrow> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> </mfrac> <mo>=</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>+</mo> <mn>1</mn> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <mn>2</mn> <mo>*</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>+</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>-</mo> <mn>1</mn> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>8</mn> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <mi>h</mi> </mrow> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> </mfrac> <mo>=</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> <mo>-</mo> <mn>2</mn> <mo>*</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>+</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>9</mn> <mo>)</mo> </mrow> </mrow> </math>

2) adopts the pair of the formula (10) and the formula (11)

And

and (6) judging. If (10) and (11) are satisfied simultaneously, setting the flagh value of the particle W (x, y) to be 1; otherwise, the height h (x, y) and flagh value of the particle W (x, y) are set to 0.

<math> <mrow> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <mi>h</mi> </mrow> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> </mfrac> <mo>|</mo> <mo>&lt;</mo> <mi>threshold</mi> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>10</mn> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <mi>h</mi> </mrow> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> </mfrac> <mo>|</mo> <mo>&lt;</mo> <mi>threshold</mi> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>11</mn> <mo>)</mo> </mrow> </mrow> </math>

3) Judging whether all the particles are processed, and if so, turning to the step 4); otherwise, turning to the step 1).

4) For any particle W (x, y) that has not been smoothed, the height is smoothed by equation (12) by taking the s × s region around it.

<math> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mi>s</mi> <mo>&times;</mo> <mi>s</mi> </mrow> </munderover> <mrow> <mo>(</mo> <mi>flagh</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>/</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mi>s</mi> <mo>&times;</mo> <mi>s</mi> </mrow> </munderover> <mi>flagh</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>12</mn> <mo>)</mo> </mrow> </mrow> </math>

Wherein, flagh (i) and h (i) are flagh value and height of ith particle in s × s neighborhood, respectively.

5) Judging whether all the particles are subjected to smoothing treatment, and if so, turning to the step 6); otherwise, go to step 4).

6) The algorithm ends.

(4) Recursion of fluid height using LBM

To satisfy the fluidThe actual requirement of real-time reconstruction adopts a recursive height method to generate the height value of the middle continuous frame. Using LBM method, using particle distribution function f_iTo realize the processes of collision and advection.

The height calculation using the LBM method includes processes of advection, collision, and boundary processing. The method specifically comprises the following steps:

1) in the advection process, after the initialization of the motion vector, the height of the current frame can be obtained, and then the particle distribution function in each direction of the next frame is calculated by using formula (13).

f_i(r+e_iδ_t,t+δ_t)=f_i(r,t) （13）

Wherein f is_i(r, t) is the particle distribution function at time t at r.

2) During the collision, the particle distribution function is calculated using equation (14), and the equilibrium distribution function is calculated using equation (15).

<math> <mrow> <msub> <mi>f</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>r</mi> <mo>+</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <msub> <mi>&delta;</mi> <mi>t</mi> </msub> <mo>,</mo> <mi>t</mi> <mo>+</mo> <msub> <mi>&delta;</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>-</mo> <msub> <mi>f</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>=</mo> <mo>-</mo> <mfrac> <mn>1</mn> <mi>&tau;</mi> </mfrac> <mo>[</mo> <msub> <mi>f</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>-</mo> <msubsup> <mi>f</mi> <mi>i</mi> <mi>eq</mi> </msubsup> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>]</mo> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>14</mn> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <msubsup> <mi>f</mi> <mi>i</mi> <mi>eq</mi> </msubsup> <mo>=</mo> <msub> <mi>&omega;</mi> <mi>i</mi> </msub> <mi>&rho;</mi> <mo>[</mo> <mn>1</mn> <mo>+</mo> <mfrac> <mrow> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>&CenterDot;</mo> <mi>c</mi> </mrow> <msubsup> <mi>e</mi> <mi>s</mi> <mn>2</mn> </msubsup> </mfrac> <mo>+</mo> <mfrac> <msup> <mrow> <mo>(</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>&CenterDot;</mo> <mi>c</mi> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mrow> <mn>2</mn> <msubsup> <mi>e</mi> <mi>s</mi> <mn>4</mn> </msubsup> </mrow> </mfrac> <mo>-</mo> <mfrac> <msup> <mi>c</mi> <mn>2</mn> </msup> <mrow> <mn>2</mn> <msubsup> <mi>e</mi> <mi>s</mi> <mn>2</mn> </msubsup> </mrow> </mfrac> <mo>]</mo> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>15</mn> <mo>)</mo> </mrow> </mrow> </math>

Wherein,is an equilibrium state distribution function along the i direction; omega_iRepresents a weight coefficient in the i direction; ρ represents the macroscopic density of the fluid; e.g. of the type_sIs a constant; e.g. of the type_iAnd c have the same meanings as described above.

The fluid height can be calculated using equation (7).

The boundary is processed by a periodic boundary method. The parameters in equation (15) are typically taken as:

$e_i=\left[\begin{array}{ccccccccc}0& 1& 0& -1& 0& 1& -1& -1& 1\\ 0& 0& 1& 0& -1& 1& 1& -1& -1\end{array}\right]$

<math> <mrow> <msub> <mi>&omega;</mi> <mi>i</mi> </msub> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mn>4</mn> <mo>/</mo> <mn>9</mn> <mo>,</mo> <msubsup> <mi>e</mi> <mi>i</mi> <mn>2</mn> </msubsup> <mo>=</mo> <mn>0</mn> </mtd> </mtr> <mtr> <mtd> <mn>1</mn> <mo>/</mo> <mn>9</mn> <mo>,</mo> <msubsup> <mi>e</mi> <mi>i</mi> <mn>2</mn> </msubsup> <mo>=</mo> <mn>1</mn> </mtd> </mtr> <mtr> <mtd> <mn>1</mn> <mo>/</mo> <mn>36</mn> <mo>,</mo> <msubsup> <mi>e</mi> <mi>i</mi> <mn>2</mn> </msubsup> <mo>=</mo> <mn>2</mn> </mtd> </mtr> </mtable> </mfenced> </mrow> </math> $e_s=\frac{1}{\sqrt{3}}$

(5) height correction

Assuming that the Tth frame and its previous heights have been generated by LBM recursion, the fluid height is corrected at the Kth frame in order to reduce the accumulated error in the recursion process. The height value of the Kth frame can be obtained by calculating the motion vector of the Kth frame, and is accurate. For the height of each frame between (T, K), the distribution function of each surrounding particle is calculated by adopting the interpolation method of the distribution function, and the height information of the central particle is further calculated by adopting the summation method of the distribution functions. The specific steps of fluid height correction are as follows:

1) for any frame X from the T th frame to the K th frame without height correction, the particle distribution function in the particle i direction at (X, y) is f_Ti(x, y, t) and f_Ki(x, y, t) which, if the difference between them is too great, i.e. the condition of equation (16) is not met, may lead to a situation in which the fluid level abruptly changes, for which purpose f is adjusted according to equation (17)_KiThe size of (2). Go to step 2).

<math> <mrow> <mo>|</mo> <mfrac> <mrow> <msub> <mi>f</mi> <mi>Ki</mi> </msub> <mo>-</mo> <msub> <mi>f</mi> <mi>Ti</mi> </msub> </mrow> <msub> <mi>f</mi> <mi>Ti</mi> </msub> </mfrac> <mo>|</mo> <mo>&le;</mo> <mi>factor</mi> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>16</mn> <mo>)</mo> </mrow> </mrow> </math>

f_Ki=f_Ti×(1+sig_factor×factor) （17）

In equations (16) and (17), the factor is used to measure the growth rate of the particle distribution function. Sig_factorIs f_Ki-f_TiIf the sign of (1) is positive, if the sign of (1) is negative, -1, otherwise, 0 is taken.

2) The particle distribution function of the Xth (X epsilon (T, K)) frame is obtained by linear interpolation according to the formula (18).

f_Xi=k×(f_Ki-f_Ti)/(K-T-1) （18）

Wherein K (K =1, 2 … …, K-T-1), f_XiRepresenting the particle distribution function in the i-direction of the X-th frame.

3) Judging whether a frame which is not subjected to height correction exists or not, and turning to the step 1) if the frame exists; otherwise go to step 4).

4) The algorithm ends.

The factor is calculated using equation (19).

$\mathrm{factor}=\frac{K-T-1}{f_{\mathrm{Ti}}+f_{\mathrm{Ki}}}---\left(19\right)$

After the height correction, a height calculation result consistent with the reality of the video fluid is obtained.

The height calculation method combining the motion vector and the LBM comprises the following steps:

in order to calculate the fluid height with the characteristics of maintaining the physical motion of the fluid and the continuity of the fluid in real time and obtain the fluid height result consistent with the reality of the video, the video is taken as one period every m +1 frames. Assume Seg for m +1 frames in a video sequence_i(i ═ k0, k 1.. km), Seg was first calculated_k0And Seg_kmAnd (3) calculating the height of the frame by using the result of the motion vector and combining the LBM, wherein the m +1 frame height is calculated by the following steps:

1) initialization of fluid motion vectors. For Seg_k0Frame and Seg_kmRespectively calculating dense motion vectors by the frames;

2) height calculation of the combination of motion vectors and LBM. Firstly, a frame Seg is calculated by using a height calculation method combining a motion vector and an LBM_k0And frame Seg_kmParticle distribution function of

And

further calculating the fluid height h of the two frames^(k0)(x, y, t) and h^(km)(x，y，t)；

3) And denoising and smoothing the height calculation result. Using the continuity of fluid motion to calculate the height h of the fluid^(k0)(x, y, t) and h^(km)(x, y, t) denoising and smoothing;

4) the height of the fluid is extrapolated using LBM. To be provided withAs an initial distribution function, successive num's are calculated (num ∈ [ k0, km)]) Height results of the frames.

5) And (6) correcting the height. And the height result is calibrated, so that a more accurate height calculation result is obtained.

When the method is used for calculating the ocean area with violent movement, the height calculation result can obviously distinguish different movement characteristics of higher waves and calm sea surfaces; when the height of a water area with smooth motion is calculated, the overall motion of the fluid is smooth, the calculated height of the fluid is not changed greatly, the details of waves are still reserved in the calculation result, and wave crests and wave troughs can be distinguished. The calculation result of the height of the motion vector has a feature of maintaining the fluid motion, and the calculated height coincides with the height at the time of the actual fluid motion. The algorithm of the invention can obtain the continuously changing fluid height field, and generate real-time height results, can be applied to the construction of natural landscape virtual scenes, and can effectively realize the real-time interaction of fluid movement in the virtual scenes, and can overcome the problem of poor real-time performance of reconstruction in a visual method, thereby having certain application value.

Drawings

FIG. 1 is a schematic diagram of the standard direction of force of a particle on a center particle;

fig. 2 is a diagram showing the initialization result of the motion vector of frame 61 in the embodiment "6482810" of the present invention.

Detailed Description

Examples

The invention is further described below with reference to the accompanying drawings.

The present embodiment uses frame 61 of "6482810" in the DynTex dynamic texture library to calculate the height of the fluid. The method is carried out under a Windows XP operating system on a PC, and the hardware configuration is 2.0GHz Intel Core (TM)2 Duo CPU and 2GB memory. For each 40 frames of video as a cycle, a fluid height of 20 frames is extrapolated using LBM, and correction of intermediate frames is performed. The method comprises the following specific steps:

(1) calculation of fluid motion vectors

Since the neighborhood where the fluid particle is located has the property of keeping the intensity component constant before and after the movement. According to this feature, the fluid motion vector is initialized using the regional intensity correlation. And clustering, and defining a feature vector for clustering, wherein the form of vector is [ x, y, u, v, sigu, sigv ], wherein (x, y) is the position of the fluid particle, u and v respectively represent the components of the motion vector of the particle along the x direction and the y direction, sigu and sigv respectively represent the signs of u and v, if the sign is positive, the sign is positive 1, if the sign is negative, and otherwise, the sign is 0. Performing clustering analysis on the defined characteristic vectors, optimizing according to the scale of a clustering result, and finally adopting a shortest distance linear interpolation method to obtain a dense motion vector field, finding out the closest particle to each particle, and further obtaining a dense motion vector result by adopting a linear interpolation method, wherein the method specifically comprises the following steps:

1) the fluid motion vector is initialized by using the 61 st and 62 nd frame images of '6482810' by using a region correlation operation.

2) Feature vector using motion vector ═ x, y, u, v, sigu, sigv]Clustering is performed, in particular, if the particle P_iAnd P_jThe feature vectors of the motion vector of (a) are respectively: vector_i=[x_pi,y_pi,u_pi,v_pi,sigu_pi,sigv_pi]And vector_j=[x_pj,y_pj,u_pj,v_pj,sigu_pj,sigv_pj]And, they satisfy the formulae (1) to (4) at the same time, we consider the particle P to be_iAnd P_jHave similarities and group them into the same category.

$|\sqrt{{(x_{\mathrm{pi}}-x_{\mathrm{pj}})}^2+{(y_{\mathrm{pi}}-y_{\mathrm{pj}})}^2}<{\mathrm{threshold}}_{\mathrm{dis}}---\left(1\right)$

$\left|\sqrt{\mathrm{angle}({\mathrm{vec}}_{\mathrm{pi}},{\mathrm{vec}}_{\mathrm{pj}})}\right|<{\mathrm{threshold}}_{\mathrm{angle}}---\left(2\right)$

sigu_pi*sigu_pj≥0 （3）

sigv_pi*sigv_pj≥0 （4）

Among them, threshold_disRepresents a particle P_iAnd P_jA threshold value of the distance between; vec_piAnd vec_pjRespectively represent particles P_iAnd P_jMotion vector (u) of_pi，v_pi) And (u)_pj,v_pj)；threshold_angleA threshold value indicating the direction of their motion vectors.

3) And counting the scale of each category, and keeping the result of the category with larger scale in the clustering result. Let any one class D whose number of particles in class is D_numI.e. the scale of this class is D_numIf the motion vector satisfies the formula (5), the intra-class particle motion vector is considered to be more accurate, and the flag value of the particles is set to be 1; otherwise, setting the flag value of the particles to be 0;

D_num>threshold_D （5）

threshold_Dis a threshold for the number of particles.

4) For any particle P with flag value 1_mIn the surrounding area, a particle P having a flag value of 1 and closest to the selected particle P is selected_nFor all flag values on the two particle connecting lines areThe particles of 0 adopt a linear interpolation method to obtain the motion vector result of the particles.

In this embodiment, threshold is used_disIs taken as 42, threshold_angleIs taken as

And threshold_DTaking the average of the number of samples of all classes

And (4) finishing. Through the four steps, the initialization result of the dense fluid motion vector is obtained.

(2) Motion vector and LBM combined height calculation

LBM is a simplified microscopic model of fluid motion that represents the statistical results of microscopic thermal motion of a large number of fluid particles in the form of a particle distribution function that, through its evolution, further reflects the macroscopic motion of the fluid.

A standard orientation is defined. The so-called standard orientation is F shown in FIG. 1_i(i = 1.., 8). The central particle is subjected to normal directional forces from the surrounding 8 particles, which contribute to the high degree of evolution of the central particle in three dimensions.

If the motion vector direction of the surrounding particle is the same as a certain standard direction, it will have a larger force on the central particle in that direction. According to this feature, the motion vectors of the surrounding particles are projected in the normal direction, so that the effect of the surrounding particles on the central particle can be determined. When the included angle between the motion direction component of the surrounding particles and the corresponding standard direction is greater than or equal to 90 degrees, the acting force generated on the central particles in the standard direction is considered to be zero.

The height of the fluid particles in three-dimensional space is f_iThe sum of (i ═ 1.., 8), the larger the sum, the larger the height value of the particle in three-dimensional space, and vice versa.

The specific calculation steps are as follows:

1) assume that any particle S (x, y) whose height has not been calculated has a surrounding particle T_i(x, y) (i ═ 1.., 8), their corresponding motion vectors are (u @_i,v_i). Particle T_iThe standard direction of the force of (x, y) on the particle S (x, y) is F_i. Set the standard direction F_iAnd vector Hor (u)_i0) angle of theta₁，F_iAnd vector Ver (0, v)_i) Is theta₂. Calculation of the distribution function f of the particles S (x, y) by equation (6)_i（x,y,t)。

2) The height h of the particle S (x, y) in the three-dimensional space is calculated using equation (7).

<math> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>0</mn> </mrow> <mi>d</mi> </munderover> <msub> <mi>f</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>7</mn> <mo>)</mo> </mrow> </mrow> </math>

3) And (4) judging whether the heights of all the particles are calculated, if so, turning to the step 4, otherwise, turning to the step 1.

4) The algorithm ends.

(3) Denoising and smoothing height calculation results

Firstly, denoising a height calculation result by using the continuity of height change, and then smoothing the height calculation result in a local area. The method comprises the following specific steps:

1) for any unprocessed particle W (x, y), the second order difference of the height h (x, y) is calculated using equations (8) and (9) and is recorded as

And

<math> <mrow> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <mi>h</mi> </mrow> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> </mfrac> <mo>=</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>+</mo> <mn>1</mn> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <mn>2</mn> <mo>*</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>+</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>-</mo> <mn>1</mn> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>8</mn> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <mi>h</mi> </mrow> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> </mfrac> <mo>=</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> <mo>-</mo> <mn>2</mn> <mo>*</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>+</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>9</mn> <mo>)</mo> </mrow> </mrow> </math>

2) adopts the pair of the formula (10) and the formula (11)

And

and (6) judging. If (10) and (11) are satisfied simultaneously, setting the flagh value of the particle W (x, y) to be 1; otherwise, the height h (x, y) and flagh value of the particle W (x, y) are set to 0.

<math> <mrow> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <mi>h</mi> </mrow> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> </mfrac> <mo>|</mo> <mo>&lt;</mo> <mi>threshold</mi> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>10</mn> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <mi>h</mi> </mrow> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> </mfrac> <mo>|</mo> <mo>&lt;</mo> <mi>threshold</mi> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>11</mn> <mo>)</mo> </mrow> </mrow> </math>

3) Judging whether all the particles are processed, if so, turning to step 4; otherwise, turning to step 1.

4) For any particle W (x, y) that has not been smoothed, the height is smoothed by equation (12) by taking the s × s region around it.

<math> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mi>s</mi> <mo>&times;</mo> <mi>s</mi> </mrow> </munderover> <mrow> <mo>(</mo> <mi>flagh</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mi>h</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>/</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mi>s</mi> <mo>&times;</mo> <mi>s</mi> </mrow> </munderover> <mi>flagh</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>12</mn> <mo>)</mo> </mrow> </mrow> </math>

Wherein, flagh (i) and h (i) are flagh value and height of ith particle in s × s neighborhood, respectively.

5) Judging whether all the particles are subjected to smoothing treatment, if so, turning to step 6; otherwise, go to step 4.

6) The algorithm ends.

The threshold in the formula (10) and the formula (11) is 0.5, and the region s in the formula (12) is 3, and continuous fluid height results are obtained after denoising and smoothing.

(4) Recursion of fluid height using LBM

In order to meet the practical requirement of real-time fluid reconstruction, the height value of the intermediate continuous frame is generated by a recursion height method. Using LBM method, using particle distribution function f_iTo realize the processes of collision and advection.

The height calculation using the LBM method includes processes of advection, collision, and boundary processing. The method specifically comprises the following steps:

1) in the advection process, after the initialization of the motion vector, the height of the current frame can be obtained, and then the particle distribution function in each direction of the next frame is calculated by using formula (13).

f_i(r+e_iδ_t，t+δ_t)＝f_i(r，t) (13)

Wherein f is_i(r, t) is the particle distribution function at time t at r.

2) During the collision, the particle distribution function is calculated using equation (14), and the formula equilibrium distribution function is calculated using equation (15).

<math> <mrow> <msub> <mi>f</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>r</mi> <mo>+</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <msub> <mi>&delta;</mi> <mi>t</mi> </msub> <mo>,</mo> <mi>t</mi> <mo>+</mo> <msub> <mi>&delta;</mi> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mo>-</mo> <msub> <mi>f</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>=</mo> <mo>-</mo> <mfrac> <mn>1</mn> <mi>&tau;</mi> </mfrac> <mo>[</mo> <msub> <mi>f</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>-</mo> <msubsup> <mi>f</mi> <mi>i</mi> <mi>eq</mi> </msubsup> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>]</mo> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>14</mn> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <msubsup> <mi>f</mi> <mi>i</mi> <mi>eq</mi> </msubsup> <mo>=</mo> <msub> <mi>&omega;</mi> <mi>i</mi> </msub> <mi>&rho;</mi> <mo>[</mo> <mn>1</mn> <mo>+</mo> <mfrac> <mrow> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>&CenterDot;</mo> <mi>c</mi> </mrow> <msubsup> <mi>e</mi> <mi>s</mi> <mn>2</mn> </msubsup> </mfrac> <mo>+</mo> <mfrac> <msup> <mrow> <mo>(</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>&CenterDot;</mo> <mi>c</mi> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mrow> <mn>2</mn> <msubsup> <mi>e</mi> <mi>s</mi> <mn>4</mn> </msubsup> </mrow> </mfrac> <mo>-</mo> <mfrac> <msup> <mi>c</mi> <mn>2</mn> </msup> <mrow> <mn>2</mn> <msubsup> <mi>e</mi> <mi>s</mi> <mn>2</mn> </msubsup> </mrow> </mfrac> <mo>]</mo> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>15</mn> <mo>)</mo> </mrow> </mrow> </math>

Wherein,is an equilibrium state distribution function along the i direction; omega_iRepresents a weight coefficient in the i direction; ρ represents the macroscopic density of the fluid; e.g. of the type_sIs a constant; e.g. of the type_iAnd c have the same meanings as described above. The relaxation time τ was taken to be 0.8 in recursion.

3) The fluid height can be calculated using equation (7).

The boundary is processed by a periodic boundary method. The parameters in equation (15) are typically taken as:

$e_i=\left[\begin{array}{ccccccccc}0& 1& 0& -1& 0& 1& -1& -1& 1\\ 0& 0& 1& 0& -1& 1& 1& -1& -1\end{array}\right]$

<math> <mrow> <msub> <mi>&omega;</mi> <mi>i</mi> </msub> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mn>4</mn> <mo>/</mo> <mn>9</mn> <mo>,</mo> <msubsup> <mi>e</mi> <mi>i</mi> <mn>2</mn> </msubsup> <mo>=</mo> <mn>0</mn> </mtd> </mtr> <mtr> <mtd> <mn>1</mn> <mo>/</mo> <mn>9</mn> <mo>,</mo> <msubsup> <mi>e</mi> <mi>i</mi> <mn>2</mn> </msubsup> <mo>=</mo> <mn>1</mn> </mtd> </mtr> <mtr> <mtd> <mn>1</mn> <mo>/</mo> <mn>36</mn> <mo>,</mo> <msubsup> <mi>e</mi> <mi>i</mi> <mn>2</mn> </msubsup> <mo>=</mo> <mn>2</mn> </mtd> </mtr> </mtable> </mfenced> </mrow> </math> $e_s=\frac{1}{\sqrt{3}}$

(5) height correction

To reduce the accumulated error in the recursion process, the fluid height is corrected at the K frame. The height value of the Kth frame can be obtained by calculating the motion vector of the Kth frame, and is accurate. For the height of each frame between (T, K), the distribution function of each surrounding particle is calculated by adopting the interpolation method of the distribution function, and the height information of the central particle is further calculated by adopting the summation method of the distribution functions. In this example, T is taken to be 20 and K is taken to be 40. The specific steps of fluid height correction are as follows:

1) for any frame X from the T th frame to the K th frame without height correction, the particle distribution function in the particle i direction at (X, y) is f_Ti(x, y, t) and f_Ki(x, y, t) which, if the difference between them is too great, i.e. the condition of equation (16) is not met, may lead to a situation in which the fluid level abruptly changes, for which purpose f is adjusted according to equation (17)_KiThe size of (2). And (6) turning to the step 2.

<math> <mrow> <mo>|</mo> <mfrac> <mrow> <msub> <mi>f</mi> <mi>Ki</mi> </msub> <mo>-</mo> <msub> <mi>f</mi> <mi>Ti</mi> </msub> </mrow> <msub> <mi>f</mi> <mi>Ti</mi> </msub> </mfrac> <mo>|</mo> <mo>&le;</mo> <mi>factor</mi> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>16</mn> <mo>)</mo> </mrow> </mrow> </math>

f_Ki=f_Ti×(1+sig_factor×factor) （17）

In the formula (16) and the formula (17), the factor is used to measure the particle distribution functionThe rate of increase in the number. Sig_factorIs f_Ki-f_TiIf the sign of (1) is positive, if the sign of (1) is negative, -1, otherwise, 0 is taken.

2) The particle distribution function of the Xth (X epsilon (T, K)) frame is obtained by linear interpolation according to the formula (18).

f_Xi＝k×(f_Ki-f_Ti)/(K-T-1) (18)

Wherein K (K ═ 1, 2 … …, K-T-1), f_XiRepresenting the particle distribution function in the i-direction of the X-th frame.

3) Judging whether a frame which is not subjected to height correction exists or not, and turning to the step 1 if the frame exists; otherwise, turning to step 4.

4) The algorithm ends.

The factor is calculated using equation (19).

$\mathrm{factor}=\frac{K-T-1}{f_{\mathrm{Ti}}+f_{\mathrm{Ki}}}---\left(19\right)$

After the height correction, a height calculation result consistent with the reality of the video fluid is obtained.

The height calculation method combining the motion vector and the LBM comprises the following steps:

in order to calculate the fluid height with the characteristics of maintaining the physical motion of the fluid and the continuity of the fluid in real time and obtain the fluid height result consistent with the reality of the video, taking each m +1 frame of the video as one weekIn the period, m is 40. Assume Seg for m +1 frames in a video sequence_i(i ═ k0, k 1.. km), Seg was first calculated_k0And Seg_kmAnd (3) calculating the height of the frame by using the result of the motion vector and combining the LBM, wherein the m +1 frame height is calculated by the following steps:

1) initialization of fluid motion vectors. For Seg_k0Frame and Seg_kmRespectively calculating dense motion vectors by the frames;

2) height calculation of the combination of motion vectors and LBM. Firstly, a frame Seg is calculated by using a height calculation method combining a motion vector and an LBM_k0And frame Seg_kmParticle distribution function ofAnd

further calculating the fluid height h of the two frames^(k0)(x, y, t) and h^(km)(x，y，t)；

3) And denoising and smoothing the height calculation result. Using the continuity of fluid motion to calculate the height h of the fluid^(k0)(x, y, t) and h^(km)(x, y, t) denoising and smoothing;

4) the height of the fluid is extrapolated using LBM. To be provided with

As an initial distribution function, successive num's are calculated (num ∈ [ k0, km)]) Height results of the frames.

5) And (6) correcting the height. And the height result is calibrated, so that a more accurate height calculation result is obtained.

The visualization result of the height calculation can prove that when the method is used for calculating the severely moving ocean area, the height calculation result can obviously distinguish different motion characteristics of high waves and calm sea surfaces; when the height of a water area with smooth motion is calculated, the overall motion of the fluid is smooth, the calculated height of the fluid is not changed greatly, the details of waves are still reserved in the calculation result, and wave crests and wave troughs can be distinguished. The calculation result of the height of the motion vector has a feature of maintaining the fluid motion, and the calculated height coincides with the height at the time of the actual fluid motion. Fig. 2 shows the initialization result of the motion vector of frame 61 of "6482810".

The invention is compared with the same algorithm.

To illustrate the effectiveness of the algorithm, the present invention is compared to existing methods. Three different types of scenes were used for experimental comparisons: one is a fluid scene containing reflections; a fluid scene with a stationary background area; and thirdly, a motion scene with local details.

1) When the fluid scene containing the reflection is calculated by the existing method, the height cannot be effectively calculated due to the dependence on the brightness of the scene. The calculated height is low because the brightness of the partial area covered by the reflection is dark, and the calculated height value is large for the bright area, but the height of the area is relatively flat in the actual scene, and the precision of the existing method is still to be improved when the existing method is used for processing the scene. When the method is used for calculation, the calculation result is more practical.

2) The background area is a fluid scene without movement. The experimental result proves that for a fluid scene with an immobile background area, the background area can be distinguished, the height of the fluid area is calculated, the calculation result is more accurate, and the error of the height calculation result is larger when the calculation is carried out by using the existing method.

3) A motion scene with local details. The experimental result proves that when the motion scenes with local details are compared, the height result which is relatively accordant with reality can be calculated by adopting the method; for the experiment of the existing method, the main problems are that when the water surface with high specular reflection is drawn, for example, the water surface with shadow or strong brightness, the precision is influenced in the height calculation of local details, and the obtained height result has a large difference from the real situation of the video. When the method is used for calculating, the height calculation result is more practical.

Compared with the similar algorithm, the comparison experiment shows that the method is more practical when the fluid height is calculated, particularly when a fluid scene containing a reflection, a fluid scene with a stationary background area and a motion scene with local details are calculated, and the effectiveness of the method can be further embodied.

In order to verify the accuracy of the present invention, the same view is used to display the frames of the original video and the result of the three-dimensional reconstruction, and in order to verify the accuracy, the average color of the two view results is used for comparison in the experiment, and the calculation method of the error is shown in formula (20).

<math> <mrow> <mi>error</mi> <mo>=</mo> <mfrac> <mn>1</mn> <mi>n</mi> </mfrac> <mo>&times;</mo> <mrow> <mo>(</mo> <mo>|</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>w</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <msub> <mi>r</mi> <mrow> <mn>2</mn> <mi>d</mi> <mo>_</mo> <mi>w</mi> </mrow> </msub> <mo>-</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>w</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <msub> <mi>r</mi> <mrow> <mn>3</mn> <mi>d</mi> <mo>|</mo> <mo>_</mo> <mi>w</mi> </mrow> </msub> <mo>|</mo> <mo>+</mo> <mo>|</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>w</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>m</mi> </munderover> <msub> <mi>g</mi> <mrow> <mn>2</mn> <mi>d</mi> <mo>_</mo> <mi>w</mi> </mrow> </msub> <mo>-</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>w</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <msub> <mi>g</mi> <mrow> <mn>3</mn> <mi>d</mi> <mo>_</mo> <mi>w</mi> </mrow> </msub> <mo>|</mo> <mo>+</mo> <mo>|</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>w</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <msub> <mi>b</mi> <mrow> <mn>2</mn> <mi>d</mi> <mo>_</mo> <mi>w</mi> </mrow> </msub> <mo>-</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>w</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <msub> <mi>b</mi> <mrow> <mn>3</mn> <mi>d</mi> <mo>_</mo> <mi>w</mi> </mrow> </msub> <mo>|</mo> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>20</mn> <mo>)</mo> </mrow> </mrow> </math>

Where n represents the total number of pixels in the image. r is_{2d_w}、g_{2d_w}And b_{2d_w}Three components of the w pixel color in the frame view of the original video, respectively. r is_{3d_w}、g_{3d_w}And b_{3d_w}Are the three components of the color of the w pixels in the three-dimensional view. The new and existing methods herein are used to perform height calculations on some videos in the dynamic texture library DynTex, and then the error is calculated separately using equation (20), and the error comparison results are shown in table 1.

TABLE 1

It is obvious from the error calculation results that in the experiment, the error obtained by using the method is small, and the highly calculated result is more practical, which further explains the accuracy and the effectiveness of the calculation result of the invention.

Time performance analysis of the present invention. To illustrate the highly calculated temporal performance of the present invention, the temporal performance of the present invention was tested using an average run time of 100 consecutive frames, and the calculation results are shown in table 2. As can be seen from the results in Table 2, the present invention has a lower run time. The average statistics is carried out on the running time of all videos, the time required for calculating the height of each frame is about 0.096787 seconds, namely the frame rate can reach 10.33 frames/second, and the result of the running time shows that the method needs less time for running and can meet the actual requirement of fluid three-dimensional reconstruction.

TABLE 2 average time per 100 frames (unit: seconds) of the new algorithm

Claims (1)

1. A method for calculating the height of a video fluid is characterized in that the method utilizes the calculation result of a fluid motion vector, obtains a distribution function of fluid particles by calculation according to the interaction of the particles during motion, and obtains the height of the fluid motion by calculation according to the distribution function, and specifically comprises the following steps:

(1) calculating a fluid motion vector;

(2) calculating the height of the combination of the motion vector and the LBM;

(3) denoising and smoothing the height calculation result;

(4) recursion of the height of the fluid using LBM;

(5) correcting the height; wherein:

calculation of the fluid motion vector: initializing a fluid motion vector by utilizing two continuous frames of images and adopting a region correlation operation mode; clustering the initialized result by using the feature vector of the defined motion vector, counting the scale of each category, reserving the main category, and finally performing linear interpolation by using the result of the fluid motion vector in the main category to obtain a motion vector calculation result; wherein the feature vector for clustering is defined as:

vector=[x,y,u,v,sigu,sigv] （1）

(x, y) is the position of the fluid particle, u and v respectively represent the components of the motion vector of the particle along the x direction and the y direction, sigu and sigv respectively represent the signs of u and v, if the sign is positive, 1 is taken if the sign is negative, and 0 is taken if the sign is negative; the specific calculation comprises the following steps:

1) initializing a fluid motion vector by using two continuous frames of images and adopting region correlation operation;

2) feature vector using motion vector ═ x, y, u, v, sigu, sigv]Clustering is performed, in particular, if the feature vectors of the motion vectors of the particles Pi and Pj are: vector_i=[x_pi,y_pi,u_pi，v_pi,sigu_pi,sigv_pi]And vector_j=[x_pj,y_pj,u_pj,v_pj,sigu_pj,sigv_pj]And they satisfy the formulas (2) to (5) at the same time, the particle P is considered to be_iAnd P_jHave similarities and group them into the same category;

$|\sqrt{{(x_{\mathrm{pi}}-x_{\mathrm{pj}})}^2+{(y_{\mathrm{pi}}-y_{\mathrm{pj}})}^2}<{\mathrm{threshold}}_{\mathrm{dis}}---\left(2\right)$

$\left|\sqrt{\mathrm{angle}({\mathrm{vec}}_{\mathrm{pi}},{\mathrm{vec}}_{\mathrm{pj}})}\right|<{\mathrm{threshold}}_{\mathrm{angle}}---\left(3\right)$

sigu_pi*sigu_pj≥0 （4）

sigv_pi*sigv_pj≥0 （5）

among them, threshold_disRepresents a particle P_iAnd P_jA threshold value of the distance between; vec_piAnd vec_pjRespectively represent particles P_iAnd P_jMotion vector (u) of_pi，v_pi) And (u)_pj,v_pj)；threshold_angleA threshold value representing the direction of their motion vectors;

3) counting the scale of each category, and keeping the result of the category with larger scale in the clustering result; let any one class D whose number of particles in class is D_numI.e. the scale of this class is D_numAnd if it satisfies formula (6)Setting the flag value of the particles to be 1; otherwise, setting the flag value of the particles to be 0;

D_num>threshold_D （6）

among them, threshold_DIs a threshold of the number of particles;

4) for any particle P with flag value 1_mIn the surrounding area, a particle P having a flag value of 1 and closest to the selected particle P is selected_nAdopting a linear interpolation method for all the particles with the flag value of 0 on the connecting line of the two particles to obtain the motion vector results of the particles;

the combined height calculation of the motion vector and the LBM comprises the following steps:

a) assume that any particle S (x, y) whose height has not been calculated has a surrounding particle T_i(x, y) (i ═ 1.., 8), their corresponding motion vectors are (u @_i,v_i) Particles T_iThe standard direction of the force of (x, y) on the particle S (x, y) is F_iSetting a standard direction F_iAnd vector Hor (u)_i0) angle of theta₁，F_iAnd vector Ver (0, v)_i) Is theta₂The distribution function f of the particles S (x, y) is calculated by_i（x,y,t)：

b) The height h of the particle S (x, y) in three-dimensional space is calculated using the following formula

<math> <mrow> <mi>h</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>0</mn> </mrow> <mi>d</mi> </munderover> <msub> <mi>f</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>r</mi> <mo>,</mo> <mi>t</mi> <mo>)</mo> </mrow> </mrow> </math>

c) Judging whether the heights of all the particles are calculated, if so, turning to the step d), otherwise, turning to the step a);

d) and finishing the calculation.

Images