CN103632392B - A kind of wave scene generating method based on video - Google Patents

Abstract

The invention discloses a kind of wave scene generating method based on video, first the method according to video flowing volume textures feature, calculates the height field of flow surface；Secondly, the result according to flow surface height field, utilize the method location of highl stratification to obtain the position of wave；In order to analyze each wave peak of wave further, use bivariate normal integral that wave heights is fitted；Then, use kmeans method to cluster at the unrestrained peak simulated, then, utilize the result of cluster that video flowing body region finally utilizes cluster result, use Graph-theoretical Approach to choose class Area generation wave scene.The present invention has simple, fast and easy feature, and can change the size generating scene according to demand, and the scene generated meets the demand of sense of reality and real-time.

Description

A kind of wave scene generating method based on video

Technical field

The present invention relates to wave scene generating method, especially a kind of wave scene generating method based on video.

Background technology

At present, in virtual reality research, for having been achieved for preliminary result based on the fluid three-dimensional reconstruction of video, but for the scene scale of above-mentioned reconstructed results, technology now is still limited by the size of input scene.Owing to fluid scene exists the feature blocked and reappear, and owing to fluid belongs to strong texture, this causes inputting for the fluid scene of random scale, be created that one realistic, the method simultaneously taking into account time performance still also exists certain challenge, and the achievement obtained in this on the one hand research is notable not enough.Key issue in random scale wave scene research, it is how to generate realistic fluid scene fast and efficiently, set up the technology according to user intention He the fluid wave scene with interactivity, being still problem extremely challenging in this field, its research has important practical significance and using value.

Along with the development of image processing techniques and computer vision technique, some technology and method are occurred in that for the research generated based on the random scale scene of wave scene.People utilize the features such as the color in image, texture, shape to study, achieve some achievements, existing method is started with from the feature of image texture, by input picture is divided into fraction, further according to the edge matching of zonule, is recombinated in each zonule, ultimately generate big scene.

Owing to wave fluid belongs to strong texture, motor process can not keep the attribute that texture is constant, and it does not have specific texture regular, cannot accurately generate the large scene with stronger sense of reality by contrasting the intensity dependence of regional area；In existing research, some researchs are devoted to the generation effect finding suitable area size to obtain the best, other researchs are by using the continuity Characteristics in fluid image texture, utilize markovian method, calculate fairly large fluid scene in conjunction with partial splice's field color optimization.Generating, based on Markov Chain method, the subject matter existed in extensive fluid scene method is: the large scale scene of generation has texture plyability；The scene generated does not have physical reality, and both fluid section did not meet hydrodynamics；Obvious splicing line is there is, i.e. discontinuity zone at region stitching position.Moreover, there is the problem that complexity is high, computationally intensive in traditional method, is difficult with completing at line method, therefore can not meet real-time demand.

For once carrying out the discussion of system based on the wave scene technology people of video, the typical achievement of this respect research is mainly based upon the method that height field repartitions scene areas.It is the distribution rule analyzing height field, in three-dimensional scenic, scene is divided into the method that some parts carry out splicing again.

In recent years, in order to improve accuracy and the sense of reality of reconstruction, people have carried out some researchs.The method that existing research includes utilizing cubic spline to be combined with horse department of pediatrics husband's random field generates wave scene, obtains good reconstructed results.

Also some personnel are had to utilize SFS to study scene generating method, the height of fluid is calculated based on flow surface shading value method, this method can obtain relatively satisfactory height calculation results for nontransparent waters, but when strong illumination, such as containing inverted image or the water surface of strong brightness, the accuracy of height calculation results will be affected.

Summary of the invention

A kind of wave scene generating method based on video provided for problems of the prior art is provided, the method has simple, fast and easy feature, and can changing the size generating scene according to demand, the scene generated meets the demand of sense of reality and real-time.

The object of the present invention is achieved like this:

A kind of wave scene generating method based on video, it includes:

1) calculating of flow surface height field；

2) identification in wave region；

3) matching at wave region Zhong Lang peak；

4) the kmeans cluster of wave peak fitting result；

5) wave class region, peak choose process.

Through above step, the class chosen is spliced in the scene being sized, calculates the large scene after being generated.The number of the class chosen, determines according to the scale of scene.

Ins and outs concrete in the present invention are as follows:

The calculating of () flow surface height field

Recover Method On Shape first with shading value and calculate the normal vector of flow surface, re-use Stokes model and calculate flow surface height；

First, following formula is used to be calculated flow surface particle method vector:

I_s=k_sI_pscosⁿ(N^T·H)(1)

Wherein I_sLight intensity for direct reflection；I_psFor incident intensity, k_sFor the specularity factor of body surface, n is constant, and it is relevant with body surface smoothness.N is the unit normal vector of incident direction, and H is under desirable surface conditions, the normal vector in observer direction.

Utilizing the reflection law of light, the direction of definition light source is N=(0,0,1)^T, owing to H is a unit vector, the N in formula (1)^TH is just for hz, and namely H is at the component of Z-direction, so can be effectively reduced amount of calculation and reach to simplify the effect of SFS method；Use formula (1) calculates and obtains hz.Finally according to calculated hz, the method for optical source colour-adjusting plate can be utilized to calculate the height field of flow surface.

(2) method of wave region recognition

First flow surface particle is pressed highl stratification, then judges that layer that connected component is maximum, use the method for least square of straight line to be fitted.Finally obtain the region of wave.

Wherein: the method judging connected component is algorithm of region growing, first chooses some the particle point in layering, it is judged that its contiguous position is with or without the particle existed, if it has, be then regarded as the proximate particle of first particle point.Connected component is defined as, in the region that a specific connected component comprises, and all of particle point proximate particle all each other.

(3) approximating method at wave region Zhong Lang peak

First in a wave region, choose particle point the highest highly labelling its for having chosen particle point.Centered by this particle point, utilizing bivariate normal integral iterative computation its peripheral region of matching, wherein distribution function is expressed as:

$h_z=\frac{1}{2{\mathrm{\π\σ}}^2}\exp (-\frac{{(x-{\mathrm{\μ}}_x)}^2+{(y-{\mathrm{\μ}}_y)}^2}{2{\mathrm{\σ}}^2})---\left(2\right)$

μ in formula_x,μ_yThe respectively coordinate figure of particle point position point, x, the span of y is S, i.e. x, y ∈ (μ_x,y-3σ,μ_x,y+ 3 σ).σ is the variance comprising particle point in the S of region.

The method using iteration calculates the size of σ of effective coverage, in order to reduce the iterations calculating σ, to initial value as σ of the half of 2 distances farthest in S, namely

$\mathrm{\σ}=\frac{max\left\{D_{ab}\right\}}{2}---\left(3\right)$

D in formula_abRepresent the distance between two difference a, b in S.

According to statistical theory, use Maximum Likelihood Estimation Method to estimate parameter σ, and use the center of gravity of spray point as μ_xAnd μ_yEstimator, namely

$\widehat{\mathrm{\σ}}=\frac{\underset{k\∈A}{\Σ}(L_k-L_a)}{n_a}---\left(4\right)$

In formula, n_aRepresent the quantity of point, L in the S of region_kRepresent the reconstruction height of kth point；L_aRepresent n_aThe average of the height of individual point.According to " 3 σ rule ", namely the span of normal variate is when (-∞, ∞), and its value nearly all drops in the interval of (μ-3 σ, μ+3 σ), and therefore, what take A ranges for (μ_x,y-3σ,μ_x,y+ 3 σ) interval.

For the size at Exact recovery wave peak, following formula iteration is utilized to try to achieve final σ value.

σ_i=σ_i-1+△σ_i-1(5)

△ σ in formula_i-1ForWith the error of σ, be expressed as

${\mathrm{\Δ\σ}}_i=\mathrm{\η}({\widehat{\mathrm{\σ}}}_i-{\mathrm{\σ}}_i)---\left(6\right)$

In formula, η is slack, η ∈ (0,1].Owing to the degree of accuracy of research is in Pixel-level, it is possible to the range of error of permission is at sub-pixel, and therefore the end condition of iteration is △ σ < 1.

Then labelling iteration completes the particle point in region for choose, and unselected particle point camber maximum of points is chosen in circulation, and performs aforesaid operations, until all particle points were all selected in region.

The fit procedure at wave peak is as follows:

) utilize apparent height to determine wave peak center point particle；

) use the method for Maximum-likelihood estimation to estimate the σ of spray regional area；

) use the value of method correction σ of iteration；

) circulation execution step) be entirely selected until particle in region.

(4) the kmeans cluster of wave peak fitting result

With the result of wave peak matching for element, use kmeans clustering.First, definition cluster numbers k isWherein n is the quantity at wave peak.Then, computed range cost function, distance cost function is represented by F (k), following formula calculate

$F\left(k\right)=\underset{i=1}{\overset{k}{\&Sigma;}}\left|\right|m_i-m_a\left|\right|+\underset{i=1}{\overset{k}{\&Sigma;}}\underset{x\&Element;C_i}{\&Sigma;}\left|\right|x-m_i\left|\right|-\mathrm{\&psi;}\underset{i=1}{\overset{k}{\&Sigma;}}|h_i-h_a|---\left(7\right)$

Wherein, m_aAnd m_iBeing the average of input sample and class number, x is class C_iBasic element；H_aIt is in same class i with h, the average of height field sample；ψ is the weight of addition Item；Finally use

$\mathrm{\&psi;}=\frac{max(L,W)}{h_a}---\left(8\right)$

Calculating ψ, wherein L and W is length and the width of input picture.Applicable unrestrained peak cluster numbers just can be calculated by iteration above step.

(5) wave class region, peak choose process

The process of choosing in wave class region, peak can be described as: uses class inner height difference σ as the build-in attribute of class, and inhomogeneous σ difference can be considered as the distance between class, therefore can use minimal spanning tree algorithm that each class is generated the structure of a tree, and use following steps to choose class；

) randomly select the node in tree as initial classes.

) randomly select the node class that initial classes connects.

If) step) node chosen is not a leaf node, iteration performs step)；Otherwise choose the node close to this node between class distance second.

) judge whether the node chosen is leaf node, if it is, randomly select node, meeting it is not step) choose the node being connected of node, if it is not, then perform step).

Accompanying drawing explanation

Fig. 1 is fluid three dimensional particles hierarchical specification diagram；

Fig. 2 is connected component schematic diagram；

Fig. 3 is quadratic fit procedure declaration figure；

Fig. 4 is that wave scene generates result figure.

Detailed description of the invention

Below in conjunction with accompanying drawing, the present invention is further described.

Embodiment

The present embodiment adopts the height that the 61st frame of " 6482810 " in DynTex dynamic texture storehouse carries out fluid to be calculated.Carrying out under WindowsXP operating system on PC, its hardware configuration is 3.4GHzIntelCore (TM) 2DuoCPU, 4GB internal memory.

(1) calculating of flow surface height field

The 61st two field picture first with " 6482810 " adopts shading value to recover the normal vector of Method On Shape calculating flow surface, re-uses Stokes model and calculates flow surface height.

Wherein, wave surface particle height method vector uses following formula to be calculated:

I_s=k_sI_pscosⁿ(N^T·H)(1)

Wherein I_sLight intensity for direct reflection；I_psFor incident intensity, k_sFor the specularity factor of body surface, n is constant, and it is relevant with body surface smoothness.N is the unit normal vector of incident direction, and H is under desirable surface conditions, the normal vector in observer direction.

Utilizing the reflection law of light, the direction of definition light source is N=(0,0,1)^T, owing to H is a unit vector, the N in formula (1)^TH is just for hz, and namely H is at the component of Z-direction, so can be effectively reduced amount of calculation and reach to simplify the effect of SFS method；Use formula (1) calculates and obtains hz；Finally according to calculated hz, the method for optical source colour-adjusting plate can be utilized to calculate the height field of flow surface.

(2) method of wave region recognition

First flow surface particle is pressed highl stratification, as shown in Figure 1.Judge that layer that connected component is maximum again, use the method for least square of straight line to be fitted.Finally obtain the region of wave.

Wherein: the method judging connected component is algorithm of region growing, first chooses some the particle point in layering, it is judged that its contiguous position is with or without the particle existed, if it has, be then regarded as the proximate particle of first particle point.Connected component is defined as, in the region that a specific connected component comprises, and all of particle point proximate particle all each other.In wave region in the connected component of particle such as Fig. 2 shown in square frame.

(3) approximating method at wave region Zhong Lang peak

First in a wave region, choose particle point the highest highly labelling its for having chosen particle point.Centered by this particle point, utilizing bivariate normal integral iterative computation its peripheral region of matching, wherein distribution function is expressed as:

$h_z=\frac{1}{2{\mathrm{\&pi;\&sigma;}}^2}\exp (-\frac{{(x-{\mathrm{\&mu;}}_x)}^2+{(y-{\mathrm{\&mu;}}_y)}^2}{2{\mathrm{\&sigma;}}^2})---\left(2\right)$

μ in formula_x,μ_yThe respectively coordinate figure of particle point position point, x, the span of y is S, i.e. x, y ∈ (μ_x,y-3σ,μ_x,y+ 3 σ).σ is the variance comprising particle point in the S of region.

The method using iteration calculates the size of σ of effective coverage, in order to reduce the iterations calculating σ, to initial value as σ of the half of 2 distances farthest in S, namely

$\mathrm{\&sigma;}=\frac{max\left\{D_{ab}\right\}}{2}---\left(3\right)$

D in formula_abRepresent the distance between two difference a, b in S.

According to statistical theory, use Maximum Likelihood Estimation Method to estimate parameter σ, and use the center of gravity of spray point as μ_xAnd μ_yEstimator, namely

$\widehat{\mathrm{\&sigma;}}=\frac{\underset{k\&Element;A}{\&Sigma;}(L_k-L_a)}{n_a}---\left(4\right)$

In formula, n_aRepresent the quantity of point, L in the S of region_kRepresent the reconstruction height of kth point；L_aRepresent n_aThe average of the height of individual point.According to " 3 σ rule ", namely the span of normal variate is when (-∞, ∞), and its value nearly all drops in the interval of (μ-3 σ, μ+3 σ), and therefore, what take A ranges for (μ_x,y-3σ,μ_x,y+ 3 σ) interval.

For the size at Exact recovery wave peak, following formula iteration is utilized to try to achieve final σ value.

σ_i=σ_i-1+△σ_i-1(5)

△ σ in formula_i-1ForWith the error of σ, be expressed as

${\mathrm{\&Delta;\&sigma;}}_i=\mathrm{\&eta;}({\widehat{\mathrm{\&sigma;}}}_i-{\mathrm{\&sigma;}}_i)---\left(6\right)$

In formula, η is slack, η ∈ (0,1].Owing to the degree of accuracy of research is in Pixel-level, it is possible to the range of error of permission is at sub-pixel, and therefore the end condition of iteration is △ σ < 1.

Then labelling iteration completes the particle point in region for choose, and unselected particle point camber maximum of points is chosen in circulation, and performs aforesaid operations, until all particle points were all selected in region.Fitting result corresponding to wave peak is as shown in Figure 3.

The fit procedure at wave peak is as follows:

) utilize apparent height to determine wave peak center point particle.

) use the method for Maximum-likelihood estimation to estimate the σ of spray regional area.

) use the value of method correction σ of iteration.

) circulation execution step) be entirely selected until particle in region.

(4) the kmeans cluster of wave peak fitting result

With the result of wave peak matching for element, use kmeans clustering.First, definition cluster numbers k isWherein n is the quantity at wave peak.Then, computed range cost function, distance cost function is represented by F (k), following formula calculate

$F\left(k\right)=\underset{i=1}{\overset{k}{\&Sigma;}}\left|\right|m_i-m_a\left|\right|+\underset{i=1}{\overset{k}{\&Sigma;}}\underset{x\&Element;C_i}{\&Sigma;}\left|\right|x-m_i\left|\right|-\mathrm{\&psi;}\underset{i=1}{\overset{k}{\&Sigma;}}|h_i-h_a|---\left(7\right)$

Wherein, m_aAnd m_iBeing the average of input sample and class number, x is class C_iBasic element；H_aIt is in same class i with h, the average of height field sample；ψ is the weight of addition Item；Finally use

$\mathrm{\&psi;}=\frac{max(L,W)}{h_a}---\left(8\right)$

Calculating ψ, wherein L and W is length and the width of input picture.Applicable unrestrained peak cluster numbers just can be calculated by iteration above step.

(5) wave class region, peak choose process

The process of choosing in wave class region, peak can be described as: uses class inner height difference σ as the build-in attribute of class, and inhomogeneous σ difference can be considered as the distance between class, therefore can use minimal spanning tree algorithm that each class is generated the structure of a tree, and use following steps to choose class；

) randomly select the node in tree as initial classes.

) randomly select the node class that initial classes connects.

If) step) node chosen is not a leaf node, iteration performs step)；Otherwise choose the node close to this node between class distance second.

) judge whether the node chosen is leaf node, if it is, randomly select node, meeting it is not step) choose the node being connected of node, if it is not, then perform step).

Through above step, the class chosen is spliced in a certain size scene, calculates the large scene after being generated.The number of the class chosen, determines according to the scale of scene.

The result of wave scene, as shown in Figure 4, it is possible to when proving to utilize the method for the present invention that sea area is generated, the result of generation can retain the primary characteristic of wave to a great extent；When surging wave scene is generated, the seam between concatenation module is difficult to identification, and the generation result of scene still remains the physical characteristic of wave, and result still remains the details of wave, it is possible to distinguish crest and trough.

The present invention is compared with existing method.The result of horse department of pediatrics husband's random field and the generation result of the present invention is utilized to carry out contrasting and utilizing the reasonability of the current physical distribution of LBM its result of methods analyst.

The time performance analysis of the present invention.In order to the time performance of high computational of the present invention is described, adopting the average operating time of continuous 50 frames to test the time performance of the present invention, the result of calculation step by step of time performance is shown in Table 1.From the results shown in Table 1, the present invention has the relatively low operation time.Be averaged statistics to operation time of all videos, obtaining normal frames scene and generate the about 0.2 second time needed, the process of key frame is about 2 seconds, can be seen that from these results running the time, the present invention runs the needs less time, it is possible to meet being actually needed of fluid three-dimensional reconstruction.

The average time (unit: millisecond) of every 50 frames of table 1

Claims (1)

1. the wave scene generating method based on video, it is characterised in that the method includes:

(1) calculating of flow surface height field；

(2) identification in wave region；

(3) matching at wave region Zhong Lang peak；

(4) the kmeans cluster of wave peak fitting result；

(5) wave class region, peak choose process；

Through above step, the class chosen is spliced in the scene being sized, calculates the large scene after being generated；The number of the class chosen, determines according to the scale of scene；Wherein:

The matching at Zhong Lang peak, described wave region chooses process with wave class region, peak, utilizes the physics law that fluid moves to generate realistic wave scene；

The calculating of described flow surface height field:

Recover Method On Shape first with shading value and calculate the normal vector of flow surface, re-use Stokes model and calculate flow surface height；

First, following formula is used to be calculated flow surface particle method vector:

I_s=k_sI_pscosⁿ(N^T·H)(1)

I in formula_sLight intensity for direct reflection；I_psFor incident intensity, k_sFor the specularity factor of body surface, n is constant, relevant with body surface smoothness；N is the unit normal vector of incident direction, and H is under desirable surface conditions, the normal vector in observer direction；

Utilizing the reflection law of light, the direction of definition light source is N=(0,0,1)^T, owing to H is a unit vector, the N in formula (1)^TH is just for hz, and namely H is at the component of Z-direction, uses formula (1) to calculate and obtains hz；Finally according to calculated hz, the method for optical source colour-adjusting plate is utilized to calculate the height field of flow surface；

The identification in described wave region:

First flow surface particle is pressed highl stratification, then judges that layer that connected component is maximum, use the method for least square of straight line to be fitted, finally obtain the region of wave；

Wherein: judge that connected component is algorithm of region growing, some the particle point in layering is first chosen, it is judged that its contiguous position is with or without the particle existed, if it has, be then regarded as the proximate particle of first particle point；Connected component is defined as, in the region that a specific connected component comprises, and all of particle point proximate particle all each other；

The matching at Zhong Lang peak, described wave region:

First in a wave region, choose highly the highest particle point labelling its for choose particle point, centered by this particle point, utilize bivariate normal integral iterative computation its peripheral region of matching, wherein distribution function is expressed as:

$h_z=\frac{1}{2{\mathrm{\&pi;\&sigma;}}^2}\exp (-\frac{{(x-{\mathrm{\&mu;}}_x)}^2+{(y-{\mathrm{\&mu;}}_y)}^2}{2{\mathrm{\&sigma;}}^2})---\left(2\right)$

μ in formula_x,μ_yThe respectively coordinate figure of particle point position point, x, the span of y is S, i.e. x, y ∈ (μ_x,y-3σ,μ_x,y+ 3 σ)；σ is the variance comprising particle point in the S of region；

The method using iteration calculates the size of σ of effective coverage, in order to reduce the iterations calculating σ, to initial value as σ of the half of 2 distances farthest in S, namely

$\mathrm{\&sigma;}=\frac{max\left\{D_{ab}\right\}}{2}---\left(3\right)$

D in formula_abRepresent the distance between two difference a, b in S；

According to statistical theory, use Maximum Likelihood Estimation Method to estimate parameter σ, and use the center of gravity of spray point as μ_xAnd μ_yEstimator, namely

$\widehat{\mathrm{\&sigma;}}=\frac{\underset{k\&Element;A}{\&Sigma;}(L_k-L_a)}{n_a}---\left(4\right)$

In formula, n_aRepresent the quantity of point, L in the S of region_kRepresent the reconstruction height of kth point；L_aRepresent n_aThe average of the height of individual point；According to " 3 σ rule ", namely the span of normal variate is when (-∞, ∞), and its value nearly all drops in the interval of (μ-3 σ, μ+3 σ), and therefore, what take A ranges for (μ_x,y-3σ,μ_x,y+ 3 σ) interval；

For the size at Exact recovery wave peak, following formula iteration is utilized to try to achieve final σ value；

σ_i=σ_i-1+△σ_i-1(5)

△ σ in formula_i-1ForWith the error of σ, be expressed as

${\mathrm{\&Delta;\&sigma;}}_i=\mathrm{\&eta;}({\widehat{\mathrm{\&sigma;}}}_i-{\mathrm{\&sigma;}}_i)---\left(6\right)$

In formula, η is slack, η ∈ (0,1]；

Then labelling completes the particle point in region for choose, and unselected particle point camber maximum of points is chosen in circulation, until all particle points were all selected in region；

The fit procedure at wave peak is as follows:

A) apparent height is utilized to determine wave peak center point particle；

B) method using Maximum-likelihood estimation estimates the σ of spray regional area；

C) value of the method correction σ of iteration is used；

D) circulation performs step a) until particle was selected entirely in region；

The kmeans cluster of described wave peak fitting result is:

With the result of wave peak matching for element, use kmeans clustering；First, definition cluster numbers k isWherein n is the quantity at wave peak；Then, computed range cost function, distance cost function is represented by F (k), following formula calculate

$F\left(k\right)=\underset{i=1}{\overset{k}{\&Sigma;}}\left|\right|m_i-m_a\left|\right|+\underset{i=1}{\overset{k}{\&Sigma;}}\underset{x\&Element;C_i}{\&Sigma;}\left|\right|x-m_i\left|\right|-\mathrm{\&psi;}\underset{i=1}{\overset{k}{\&Sigma;}}|h_i-h_a|---\left(7\right)$

Wherein, m_aAnd m_iBeing the average of input sample and class number, x is class C_iBasic element；H_aIt is in same class i with h, the average of height field sample；ψ is the weight of addition Item；Finally use

$\mathrm{\&psi;}=\frac{max(L,W)}{h_a}---\left(8\right)$

Calculating ψ, wherein L and W is length and the width of input picture, draws wave peak cluster numbers k by calculating the minima of F (k)；

Described wave class region, peak choose process:

Use class inner height difference σ as the build-in attribute of class, and inhomogeneous σ difference is considered as the distance between class, use minimal spanning tree algorithm that each class is generated the structure of a tree, and use following steps to choose class；

A) node in tree is randomly selected as initial classes；

B) the node class that initial classes connects is randomly selected；

If c) node that step b) chooses is not a leaf node, perform step b)；Otherwise choose from the nearest node of this node between class distance；

D) judge whether the node chosen is leaf node, if it is, randomly select node, perform step c) and choose the node being connected of node, if not performing step b).