CN101819462B - Image texture haptic representation system based on force/haptic interaction equipment - Google Patents

Abstract

The invention discloses an image texture tactile representation system based on force/haptic interaction equipment for virtual reality human-computer interaction, which is characterized in that when the virtual proxy of the force/haptic interaction equipment slides on a texture surface of a virtual object in a virtual environment, the surface height of the object texture corresponding to the contact point and a coefficient of kinetic friction for reflecting the rough degree of the contact point are firstly obtained on the basis of an image processing method, a continuous normal contact force model reflecting the concave-convex degree of the contact point and a tangential friction model reflecting the rough degree of the contact point are respectively established, and finally the texture contact force is fed back to an operator in real time through the force/haptic interaction equipment so as to realize the force haptic express and reappear when fingers slide over the surface texture of the virtual object. The feedback continuous change normal force not only enables the human-computer interaction to be more real, but also enables an interaction system to be more stable, and the feedback friction related to the rough degree of the contact point also further enhances the sense of reality when the texture reappears.

Description

Power tactile sense reproduction system based on the image texture of power/haptic interaction equipment

Technical field

The present invention relates to a kind of enhancing power haptic model, relate in particular to a kind of power tactile sense reproduction system that is used for virtual reality human-computer interaction based on the image texture of power/haptic interaction equipment.

Background technology

Haptic interaction is the state-of-the-art technology of field of human-computer interaction, and the texture sense of touch obtains more and more researchists' concern and attention as a pith of haptic interaction.In virtual reality system, the texture sense of touch not only can strengthen the sense of reality of haptic interaction, also helps remote control or carry out Machine Design in the Virtual Space, and the blind person obtains image information etc.

The modeling method of texture tactile sense reproduction mainly contains 3 kinds at present: 1. realize that based on true texture the sense of touch texture reproduces.Simulate the texture space information of dummy object by making up a real grain surface, thereby obtain the reproduction of sense of touch texture, the sand paper system of people such as typical devices such as Minsky invention.Yet real texture kind is limited after all, can not all virtual texture spatial informations of simulation.2. based on shaking principle design texture tactile representation device to reach the texture tactile sense reproduction.This method produces pressure at skin surface statically or dynamically mainly by special array apparatus is set, thereby produces the texture sensation.Typical pin type contact array as people such as Japanese Yasushi Ikei development adopts piezoelectric excitation mode driving needle to beat, thereby the position and the amplitude of the vibrations of control pin are simulated superficial makings information.This method is convenient to control, but the texture features that reproduces is relevant with the diameter and the distribution density of pin, and the concavo-convex of texture that can simulate is limited in scope.The gaseous tension tactile representation device of people such as the Naoya Asamura development of Tokyo agricultural science and technology university is by gaseous tension chafe receptor, this playback system is convenient to realize, but control inaccurately, can only simulate soft and hard two states, do not reach the simulate effect of Z-Correct bump mapping Z-correct.3. produce force feedback based on existing force sense-reproducing device and realize the texture tactile sense reproduction.This method mainly according to the spatial information of texture image, produces the contact force of corresponding texture and in conjunction with sport dynamics, in real time the texture contact force is fed back to staff by device for force feedback at contact point, thereby reaches the three-D grain information on reconstruction of objects surface.People such as Choi make a concerted effort to be modeled as the cycle sinusoidal signal that the roughness with article surface vein changes with texture contact, and body surface is coarse more, and the amplitude of cyclical signal is just big more, and texture is fine and closely woven more, and then the frequency of periodic signal is high more.This method can reflect the characteristic of texture to a certain extent preferably, but the texture kind that deterministic models can be expressed is limited, only can make force feeling that tangible discrimination is arranged, but lacks the sense of reality, can not texture tactility that is virtually reality like reality.

From above analysis as can be known, guaranteeing to realize that fast, in real time reproducing texture force touch simultaneously, optimized Algorithm is rationally carried out modeling to the texture contact force, and the sense of reality that improves the texture tactile sense reproduction is the matter of utmost importance that current virtual image texture tactile sense reproduction system needs to be resolved hurrily.

Summary of the invention

The present invention proposes a kind of power tactile sense reproduction system of the image texture based on power/haptic interaction equipment, and uses it for the perception of the 2 d texture picture of virtual reality human-computer interaction.This method need not special image texture haptic rendering device, need not the micro-profile on specialized equipment Measuring Object surface, can accurately obtain the three-dimensional spatial distribution information of dummy object texture, and set up contact force model, texture contact force by power/haptic interaction equipment feedback contact point, thereby the texture features of object under the reproduction true environment, the sense of reality and the stability of raising haptic interaction.

The present invention adopts following technical scheme:

A kind of power tactile sense reproduction system of the image texture based on power/haptic interaction equipment is characterized in that the key step of this method is as follows:

The texture picture of the step 1 pair outside dummy object that imports is preserved through row;

Step 2 system carries out Flame Image Process to the texture picture of the dummy object that calls, obtains the three-dimensional microcosmic geometric data of dummy object grain surface from the 2 d texture picture.Because in the texture picture, the space distribution of the space distribution of brightness and the displacement of texture microcosmic has correlativity, therefore, texture picture is analyzed and is handled, and just can obtain the three-dimensional spatial distribution information of dummy object superficial makings.Picture is carried out the surface elevation that spectrum analysis can obtain each pairing object texture in pixel summit.Specific algorithm is as follows:

(1) according to the body surface 2 d texture picture that imports, suppose that this 2 d texture picture size for m * n pixel, promptly laterally has m pixel and n pixel vertically arranged, the formula below utilizing obtains the brightness value of each pixel:

$l_{i,j}=\frac{R+G+B}{3},---\left(1\right)$

Wherein i ∈ 0,1 ... m-1}, j ∈ 0,1 ..., n-1} is the coordinate figure of pixel corresponding in the texture picture, R, G and B are respectively three color components of red, green, blue of this pixel, the luminance picture L that obtains _{M * n},

(2) luminance picture is carried out spectrum analysis and spatial filtering, the design Gaussian filter carries out spatial filtering to image, with picture breakdown is low frequency and high frequency two parts, low frequency part is used to reflect profile, tendency and the shape of object, HFS is used for the degree of roughness and the concavo-convex degree of reflection surface, and adopting the micro-profile information of high frequency imaging information as the reflection article surface vein, the high frequency imaging information that obtains is H _{M * n},

Concrete steps are as follows:

1. to luminance picture L _{M * n}Do discrete Fourier transformation, obtain frequency domain value F (k, v),

$F(k,v)=\frac{1}{\mathrm{mn}}\underset{a=0}{\overset{m-1}{\mathrm{\&Sigma;}}}\underset{b=0}{\overset{n-1}{\mathrm{\&Sigma;}}}{l}_{a,b}\exp (-\mathrm{<figure-callout\; id="2"\; label="j"\; filenames="HSA00000049785900021.png"\; state="\{\{state\}\}">j</figure-callout>}2\mathrm{\&pi;}(\frac{\mathrm{ka}}{m}+\frac{\mathrm{vb}}{n})),---\left(2\right)$

Wherein k ∈ 0,1 ... m-1}, v ∈ 0,1 ..., n-1} is the coordinate figure of pixel in frequency domain corresponding in the texture picture,

A ∈ 0,1 ... m-1}, b ∈ 0,1 ..., n-1}, exp () they are the exponential function of standard,

2. to luminance picture L _{M * n}In frequency domain, carry out gaussian filtering, the filtering high-frequency signal, (k v) is the gaussian filtering equation, promptly to G

The width of Gaussian filter (determining level and smooth degree) is characterized by parameter σ, and σ is big more, and the frequency band of Gaussian filter is just wide more, and level and smooth degree is just good more, S (k v) be an image through each pixel behind the gaussian filtering in the frequency domain scope correspondence value

S(k，v)＝G(k，v)F(k，v) (3)

3. (k v) carries out Fourier inversion, obtains reflecting the low-frequency information w of profile, tendency and the shape of object to S _{A, b},

$w_{a,b}=\underset{k=0}{\overset{m-1}{\mathrm{\&Sigma;}}}\underset{v=0}{\overset{n-1}{\mathrm{\&Sigma;}}}G(k,v)\exp \left(\mathrm{<figure-callout\; id="2"\; label="j"\; filenames="HSA00000049785900021.png"\; state="\{\{state\}\}">j</figure-callout>}2\mathrm{\&pi;}(\frac{\mathrm{ka}}{m}+\frac{\mathrm{vb}}{n})\right)---\left(4\right)$

Wherein a ∈ 0,1 ... m-1}, b ∈ 0,1 ..., n-1},

4. the high frequency imaging information that reflects article surface vein deducts the low-frequency information that reflects contour of object and trend for each pixel brightness value, promptly

h _i，j＝l _i，j-w _i，j (5)

Wherein i ∈ 0,1 ... m-1}, j ∈ 0,1 ..., n-1},

(3) to the high frequency imaging information H of resulting reflection article surface vein _{M * n}, carry out standardization processing, obtain the surface elevation Q of the pairing object texture of each pixel _{M * n},

Its computing formula is as follows:

$\left\{\begin{array}{c}{q}_{i,j}=h_{i,j}-\stackrel{\&OverBar;}{h}\\ \stackrel{\&OverBar;}{h}=\frac{\underset{i=0}{\overset{m-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{n-1}{\mathrm{\&Sigma;}}}{h}_{i,j}}{m\&times;n}\end{array}\right.,---\left(6\right)$

Wherein

The two-dimensional array matrix H _{M * n}The mean value of middle all elements,

(4) adopt regional averaging method to calculate the surface elevation D of each pairing object texture in pixel summit _{(m+1) * (n+1)}:

Wherein, each pixel summit P _{S, t}Expression, s ∈ 0,1 ... m}, t ∈ 0,1 ..., n}, summit P _{S, t}The surface elevation of pairing object texture is d _{S, t},

Described regional averaging method is: with each pixel summit P _{S, t}Around the mean value of surface elevation of neighborhood pixels area relative object texture be used as the surface elevation d of the pairing object texture on this summit _{S, t}, as shown in Figure 8, according to the surface elevation Q of the pairing object texture of each pixel _{M * n}Obtain d _{S, t}Specifically be calculated as follows:

$\left\{\begin{array}{c}{d}_{s+1,t+1}=\frac{q_{s,t}+q_{s+1,t}+q_{s,t+1}+q_{s+1,t+1}}{4},s\&Element;\{\mathrm{0,1},...,m-2\},t\&Element;\{\mathrm{0,1},...,n-2\}\\ {\mathrm{<figure-callout\; id="0"\; label="d"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">d</figure-callout>}}_{0,t}=\frac{q_{0,t}+q_{0,t-1}}{2},t\&Element;\{1,...,n-1\}\\ d_{t,0}=\frac{q_{s,0}+q_{s-\mathrm{1,0}}}{2},s\&Element;\{1,...,m-1\}\\ d_{m,t}=\frac{q_{m-1,t}+q_{m-1,t-1}}{2},t\&Element;\{1,...,n-1\}\\ d_{s,n}=\frac{q_{s,m}+q_{s-1,m}}{2},s\&Element;\{1,...,m-1\}\\ d_{\mathrm{0,0}}=q_{\mathrm{0,0}},d_{m,0}=q_{m-\mathrm{1,0}},{\mathrm{<figure-callout\; id="0"\; label="d"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">d</figure-callout>}}_{0,n}=q_{0,n-1},d_{m,n}=q_{m-1,n-1}\end{array}\right.$

Step 3 pair virtual scene carries out initialization, zone { 0≤x≤X in virtual environment coordinate system Oxyz, 0≤y≤Y, load the texture picture of dummy object among the z=0}, as shown in Figure 9, texture picture aligns about the position in the Virtual Space, and consistency from top to bottom, initialization virtual protocol in virtual environment, the i.e. power/agency of haptic interaction equipment in virtual environment;

Step 4 initialization power haptic interaction equipment;

Step 5 operator moves haptic interaction equipment by the lever of power/haptic interaction equipment end;

Step 6 utilize position detecting module in real time with the positional information of power/haptic interaction equipment output (Px, Py, Pz) be converted into the virtual protocol under the virtual environment coordinate (Mx, My, Mz).Concrete coordinate transformation relation is as follows:

Mx＝9*Py，My＝14*Pz，Mz＝Px*5-5， (7)

(Px wherein, Py, Pz) obtaining by the dhdGetPosition () function that calls in power/haptic interaction equipment standard api function storehouse of coordinate position realizes that unit is m, and it can obtain the positional information of lever in three dimensions of power/haptic interaction equipment end;

Step 7 judges whether current virtual protocol collides the superficial makings of dummy object, because the residing position of virtual texture picture is the regional Ω={ 0≤x≤X under the virtual environment coordinate system Oxyz, 0≤y≤Y, therefore z=0} can judge whether to collide the dummy object grain surface according to the positional information that virtual protocol moves;

Step 8 through exertin/sense of touch computing module, calculates the texture contact force size and Orientation that current location need be exported according to the result and the current coordinate of virtual protocol of collision detection, if do not collide the texture of dummy object, then power output is 0; If bump, then texture power calculating concrete steps are as follows:

(1) since texture picture be arranged in virtual environment space coordinates Oxyz the zone 0≤x≤X, 0≤y≤Y, z=0}, the size of texture picture is a m * n pixel, each pixel shared plane sizes under virtual environment is:

$\mathrm{\&Delta;X}\&times;\mathrm{\&Delta;Y}=\frac{X}{m}\&times;\frac{Y}{n}$

Along unidirectional diagonal line each pixel zone of texture picture is divided into two triangles, as Fig. 9 institute not,

(2) positional information of the virtual protocol of exporting according to position detecting module obtains the triangle at the subpoint place of the point of impingement on face Oxy, supposes that subpoint is at triangle P _{S, t}P _{S+1, t}P _{S, t+1}In, P _{S, t}(Δ X*s, Δ Y*t, 0), P _{S+1, t}(Δ X* (s+1), Δ Y*t, 0), P _{S, t+1}(Δ X*s, Δ Y* (t+1), 0), suppose the coordinate of the point of impingement under virtual environment space coordinates Oxyz for (x, y, z), promptly (x, y, z)=(Mx, My.Mz), then the projection coordinate of the point of impingement on face Oxy be (x, y),

(3) suppose triangle P _{S, t}P _{S+1, t}P _{S, t+1}The surface of pairing object texture is spatial triangle Z _{S, t}Z _{S+1, t}Z _{S, t+1}, based on resulting triangle P in the texture picture pre-service _{S, t}P _{S+1, t}P _{S, t+1}The surface elevation value of three pairing object texture in summit obtains triangle Z _{S, t}Z _{S+1, t}Z _{S, t+1}Surface equation, the surface elevation value that substitution subpoint coordinate obtains the pairing object texture of subpoint is d _{X, y}, as shown in figure 10,

$d_{x,y}=\frac{d_{s+1,t}-d_{s,t}}{\mathrm{\&Delta;X}}x+\frac{d_{s,t+1}-d_{s,t}}{\mathrm{\&Delta;Y}}y+(1+s+t)d_{s,t}-s{d}_{s+1,t}-t{d}_{s,t+1}---\left(8\right)$

When subpoint when the Ω territory changes continuously in face Oxy, the surface elevation value of the object texture that it is corresponding is d _{X, y}Change continuously,

(4) contacting the computing method of making a concerted effort at point of impingement texture is:

${\stackrel{\&RightArrow;}{F}}_c(x,y)={\stackrel{\&RightArrow;}{F}}_n(x,y)+{\stackrel{\&RightArrow;}{F}}_f(x,y),---\left(9\right)$

At the force modeling of contact point as shown in figure 11, wherein:

The computing method of normal force are as follows:

F _n(x，y)＝kd _x，y， (10)

Wherein k is the stiffness coefficient of article surface vein, and the value of k is relevant with the body surface attribute, stiffness coefficient be illustrated in article surface vein collided and the extruding situation under deformation extent, when being made as 0 is full flexible, be 1 o'clock be perfect rigidity, scope is 0～1, d _{X, y}Be the height value of contact point correspondence, if d _{X, y}For just, represent that then contact point is protruding, normal force is perpendicular to face Oxy and point to z axle positive dirction; If d _{X, y}For negative, represent that then contact point is recessed, normal force is perpendicular to face Oxy and point to z axle negative direction,

The computing method of friction force are as follows:

F _f(x，y)＝μ(x，y)|F _n(x，y)|sign(V)， (11)

(x y) is the kinetic friction coefficient of contact point to μ, and V is the relative velocity of virtual protocol on the virtual texture surface, and sign (V) is the sign function of standard, promptly

$\mathrm{sign}\left(V\right)=\left\{\begin{array}{c}1,V>0\\ 0,V=0\\ -1,V<0\end{array}\right.,---\left(12\right)$

The direction of friction force is parallel with face Oxy, and opposite with direction of motion,

Kinetic friction coefficient is designed to μ (x, y)=and ρ r (x, y), wherein ρ is the coefficient of roughness of grain surface material, surface roughness coefficient is more little, then the surface is smooth more, and the scope of ρ is 0～1, r (x, y) be the local roughness coefficient of contact point, relevant with the local concavo-convex degree of this contact point, r (x, y) available following model representation:

r(x，y)＝|d _s，t-d _x，y|+|d _s+1，t-d _x，y|+|d _s+1，t+1-d _x，y|+|d _s，t+1-d _x，y|， (13)

D wherein _{S, t}, d _{S+1, t}, d _{S+1, t+1}, d _{S, t+1}Height value for the pairing object texture surface, four summits of pixel region at point of impingement subpoint place on face Oxy, if subpoint drops on the intersection of two (or four) pixel regions, then this subpoint is belonged to these two (or the four) pixel regions and (13) formula of pressing respectively and calculate the local roughness coefficient of this point, get the local roughness coefficient that average is this point at last;

The texture contact force information power of transferring to/haptic interaction equipment that the needs that step 9 produces power/sense of touch computing module by PCI integrated circuit board and PCI integrated circuit board connecting line by power/tactile data feedback module are exported comes the acting force that setting power/haptic interaction equipment need be exported by the dhdSetForce () that calls in power/haptic interaction equipment standard api function storehouse;

If step 10 virtual protocol collides the grain surface of dummy object, then virtual protocol no longer moves (consideration be the rigidity contact), still draws virtual protocol in situ, and figure refreshes then; If virtual protocol does not collide the grain surface of dummy object, then should in virtual environment, upgrade the position of virtual protocol, promptly detect to such an extent that virtual protocol is drawn in the position that makes new advances at position-based, carry out figure and refresh;

Step 11 feeds back to the operator to power/tactile data that requires to upgrade and visual information by power/haptic interaction equipment, display, makes it not only can see and can feel the variation of power/tactile data.And then get back to step 5.

Compared with prior art, advantage of the present invention:

(1) compare with texture playback system in the past, this texture reproduces the texture expression device that need not design specialized, utilizes general power/haptic interaction equipment can simulate the sense of touch that virtual hand slips over article surface vein;

(2) this method can not only realize force modeling and real-time texture expression fast, what is more important, and the contact force that feeds back to the operator is directed to true texture, has improved the sense of reality that texture reproduces greatly;

(3) normal force of the concavo-convex degree of the true grain surface of reflection is a continually varying power, this continually varying power not only makes when carrying out man-machine interaction truer, and make interactive system more stable, when virtual protocol slips over the texture of body surface, for obtaining the continually varying normal force, at first obtain the superficial makings height value of the pairing dummy object of contact point based on image process method, guarantee that simultaneously this height value is continuous variable quantity, then, the stiffness coefficient that this height value be multiply by body surface obtains the normal force in the required output of contact point;

(4) friction force of the true grain surface degree of roughness of reflection multiply by kinetic friction coefficient for the normal force size, its kinetic friction coefficient is not only relevant with the material coefficient of roughness of grain surface, also relevant with the local roughness degree of current contact point, its local roughness coefficient is based on image processing method and obtains, the more realistic staff of the modeling method of this power slips over the power model that the inconsistent texture of rough surface is fed back, and has improved the sense of reality that texture reproduces;

(5) this playback system can be used for remote control or carry out Machine Design in the Virtual Space, and virtual operation emulation, tele-medicine, blind person obtain fields such as image information.

Description of drawings

Fig. 1 is a system chart;

Fig. 2 is the circuit system schematic diagram;

Fig. 3 is based on the power tactile sense reproduction system software process flow diagram of the image texture of power/haptic interaction equipment;

The importing of Fig. 4 texture image, storage and pretreatment process figure;

The position probing of Fig. 5 power/haptic interaction equipment and collision detection process flow diagram;

Fig. 6 is based on the texture contact force algorithm flow chart of texture image surface geometry feature extraction;

Fig. 7 figure refresh flow figure;

Each pixel coordinate point of Fig. 8 texture picture and each pixel summit graph of a relation;

Load texture picture figure among Fig. 9 virtual environment coordinate system Oxyz;

The surface height map of the subpoint pairing object texture of Figure 10 point of impingement in face Oxy;

Figure 11 staff slips over the power touch feeling model building figure of article surface vein;

The texture height map of Figure 12 texture picture after based on Flame Image Process, Fig. 4 (a) is the stone texture picture, Fig. 4 (b) is the surface height map through every pairing object texture after the Flame Image Process;

The virtual environment that Figure 13 makes up, virtual bead and stone texture reciprocal process rendering effect figure.

Embodiment:

A kind of power tactile sense reproduction system of the stone image texture based on the DELTA hand controller,

The operation platform of this example is Windows 2000, and development language is VC++6.0, and based on the OpenGL shape library.Power/haptic interaction equipment is 6DOF DELTA hand controller, its work space scope is translation Φ 360mm * L300mm and rotation ± 20deg/axis, can realize along X, Y, the acting force tactile feedback of Z axle, handle by DELTA hand controller end can touch, perception and the virtual bead of control slide at virtual stone grain surface, and the texture force touch information that produces in the reciprocal process is fed back to the operator in real time

Described power tactile sense reproduction system comprises:

The step 1 pair outside stone texture picture (shown in Figure 12 a) that imports is preserved through row,

Step 2 system carries out Flame Image Process to the stone texture picture that calls, and obtains the three-dimensional microcosmic geometric data of dummy object grain surface from the 2 d texture picture.Because in the texture picture, the space distribution of the space distribution of brightness and the displacement of texture microcosmic has correlativity, therefore, texture picture is analyzed and is handled, and just can obtain the three-dimensional spatial distribution information (shown in Figure 12 b) of dummy object superficial makings.Picture is carried out the surface elevation that spectrum analysis can obtain each pairing object texture in pixel summit.Specific algorithm is as follows:

(1) according to the body surface 2 d texture picture that imports, this texture picture size is 128 * 128 pixels, m=128 pixel is promptly laterally arranged and n=128 pixel vertically arranged, and the formula below utilizing obtains the brightness value of each pixel:

$l_{i,j}=\frac{R+G+\mathrm{<figure-callout\; id="3"\; label="B"\; filenames="HSA00000049785900021.png"\; state="\{\{state\}\}">B</figure-callout>}}{3},---\left(1\right)$

Wherein i ∈ 0,1 ... m-1}, j ∈ 0,1 ..., n-1} is the coordinate figure of pixel corresponding in the texture picture, R, G and B are respectively three color components of red, green, blue of this pixel, the luminance picture L that obtains _{M * n}:

(2) luminance picture is carried out spectrum analysis and spatial filtering, the design Gaussian filter carries out spatial filtering to image, with picture breakdown is low frequency and high frequency two parts, low frequency part is used to reflect profile, tendency and the shape of object, HFS is used for the degree of roughness and the concavo-convex degree of reflection surface, and adopting the micro-profile information of high frequency imaging information as the reflection article surface vein, the high frequency imaging information that obtains is H _{M * n},

Concrete steps are as follows:

1. to luminance picture L _{M * n}Do discrete Fourier transformation, obtain frequency domain value F (k, v),

$F(k,v)=\frac{1}{\mathrm{mn}}\underset{a=0}{\overset{m-1}{\mathrm{\&Sigma;}}}\underset{b=0}{\overset{n-1}{\mathrm{\&Sigma;}}}{l}_{a,b}\exp (-\mathrm{<figure-callout\; id="2"\; label="j"\; filenames="HSA00000049785900021.png"\; state="\{\{state\}\}">j</figure-callout>}2\mathrm{\&pi;}(\frac{\mathrm{ka}}{m}+\frac{\mathrm{vb}}{n})),---\left(2\right)$

Wherein k ∈ 0,1 ... m-1}, v ∈ 0,1 ..., n-1} is the coordinate figure of pixel in frequency domain corresponding in the texture picture,

A ∈ 0,1 ... m-1}, b ∈ 0,1 ..., n-1}, exp () they are the exponential function of standard,

2. to luminance picture L _{M * n}In frequency domain, carry out gaussian filtering, the filtering high-frequency signal, (k v) is the gaussian filtering equation, promptly to G

σ=10, (k v) is an image to S through the corresponding value in the frequency domain scope of each pixel behind the gaussian filtering

S(k，v)＝G(k，v)F(k，v) (3)

3. (k v) carries out Fourier inversion, obtains reflecting the low-frequency information w of profile, tendency and the shape of object to S _{A, b},

$w_{a,b}=\underset{k=0}{\overset{m-1}{\mathrm{\&Sigma;}}}\underset{v=0}{\overset{n-1}{\mathrm{\&Sigma;}}}G(k,v)\exp \left(\mathrm{<figure-callout\; id="2"\; label="j"\; filenames="HSA00000049785900021.png"\; state="\{\{state\}\}">j</figure-callout>}2\mathrm{\&pi;}(\frac{\mathrm{ka}}{m}+\frac{\mathrm{vb}}{n})\right)---\left(4\right)$

Wherein a ∈ 0,1 ... m-1}, b ∈ 0,1 ..., n-1},

4. the high frequency imaging information that reflects article surface vein deducts the low-frequency information that reflects contour of object and trend for each pixel brightness value, promptly

h _i，j＝l _i，j-w _i，j (5)

Wherein i ∈ 0,1 ... m-1}, j ∈ 0,1 ..., n-1},

(3) to the high frequency imaging information H of resulting reflection article surface vein _{M * n}, carry out standardization processing, obtain the surface elevation Q of the pairing object texture of each pixel _{M * n},

Its computing formula is as follows:

$\left\{\begin{array}{c}{q}_{i,j}=h_{i,j}-\stackrel{\&OverBar;}{h}\\ \stackrel{\&OverBar;}{h}=\frac{\underset{i=0}{\overset{m-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{n-1}{\mathrm{\&Sigma;}}}{h}_{i,j}}{m\&times;n}\end{array}\right.,---\left(6\right)$

Wherein

The two-dimensional array matrix H _{M * n}The mean value of middle all elements,

(4) adopt regional averaging method to calculate the surface elevation D of each pairing object texture in pixel summit _{(m+1) * (n+1)}:

Wherein, each pixel summit P _{S, t}Expression, s ∈ 0,1 ... m), t ∈ O, 1 ..., n), summit p _{S, t}The surface elevation of pairing object texture is d _{S, t},

Described regional averaging method is: with each pixel summit P _{S, t}Around the mean value of surface elevation of neighborhood pixels area relative object texture be used as the surface elevation d of the pairing object texture on this summit _{S, t}, as shown in Figure 8, according to the surface elevation Q of the pairing object texture of each pixel _{M * n}, obtain d _{S, t}Specifically be calculated as follows:

$\left\{\begin{array}{c}{d}_{s+1,t+1}=\frac{q_{s,t}+q_{s+1,t}+q_{s,t+1}+q_{s+1,t+1}}{4},s\&Element;\{\mathrm{0,1},...,m-2\},t\&Element;\{\mathrm{0,1},...,n-2\}\\ {\mathrm{<figure-callout\; id="0"\; label="d"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">d</figure-callout>}}_{0,t}=\frac{{\mathrm{<figure-callout\; id="0"\; label="q"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">q</figure-callout>}}_{0,t}+{\mathrm{<figure-callout\; id="0"\; label="q"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">q</figure-callout>}}_{0,t-1}}{2},t\&Element;\{1,...,n-1\}\\ d_{t,0}=\frac{q_{s,0}+q_{s-\mathrm{1,0}}}{2},s\&Element;\{1,...,m-1\}\\ d_{m,t}=\frac{q_{m-1,t}+q_{m-1,t-1}}{2},t\&Element;\{1,...,n-1\}\\ d_{s,n}=\frac{q_{s,m}+q_{s-1,\mathrm{<figure-callout\; id="2"\; label="m"\; filenames="HSA00000049785900021.png"\; state="\{\{state\}\}">m</figure-callout>}}}{2},s\&Element;\{1,...,m-1\}\\ {\mathrm{<figure-callout\; id="0"\; label="d"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">\; <figure-callout\; id="0"\; label="d"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">d</figure-callout>\; </figure-callout>}}_{\mathrm{0,0}}={\mathrm{<figure-callout\; id="0"\; label="q"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">\; <figure-callout\; id="0"\; label="q"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">q</figure-callout>\; </figure-callout>}}_{\mathrm{0,0}},d_{m,0}=q_{m-\mathrm{1,0}},{\mathrm{<figure-callout\; id="0"\; label="d"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">d</figure-callout>}}_{0,n}={\mathrm{<figure-callout\; id="0"\; label="q"\; filenames="HSA00000049785900062.png"\; state="\{\{state\}\}">q</figure-callout>}}_{0,n-1},d_{m,n}=q_{m-1,n-1}\end{array}\right.$

Step 3 pair virtual scene carries out initialization, the regional Ω in virtual environment coordinate system Oxyz={ 0≤x≤200,0≤y≤200, load the texture picture of dummy object among the z=0}, texture picture aligns about the position in the Virtual Space, and consistency from top to bottom, initialization virtual protocol in virtual environment, (represent that with virtual bead its initial coordinate is (100,100,20), radius is 5.), as shown in figure 13

Step 4 initialization Delta hand controller,

Step 5 operator is that the lever of Delta hand controller end moves haptic interaction equipment by power/haptic interaction equipment 3,

Step 6 utilize position detecting module 243 with power/haptic interaction equipment 3 be in real time the output of Delta hand controller positional information (Px, Py, Pz) be converted into the virtual protocol under the virtual environment coordinate (Mx, My, Mz).Concrete coordinate transformation relation is as follows:

Mx＝9*Py，My＝14*Pz，Mz＝Px*5-5， (7)

(Px wherein, Py, Pz) coordinate position to obtain by calling power/haptic interaction equipment 3 be that dhdGetPosition () function in the Delta hand controller standard A PI function library is realized, unit is m, it can obtain the positional information of lever in three dimensions that power/haptic interaction equipment 3 is Delta hand controller end

Step 7 judges whether current virtual protocol collides the superficial makings of dummy object.Because the residing position of virtual texture picture is { 0≤x≤200, zone under the virtual environment coordinate system Oxyz, 0≤y≤200, z=0}, because the radius of virtual protocol bead is 5, if (Mx, My Mz) are positioned at following regional δ and then collide the coordinate of position probing output, otherwise do not bump, wherein

δ=(x, y, z) | x ∈ [0,200], and y ∈ [0,200] and z ∈ [5,5] }

Step 8 through exertin/sense of touch computing module 245, calculates the texture contact force size and Orientation that current location need be exported according to the result and the current coordinate of virtual protocol of collision detection, if do not collide the texture of dummy object, then power output is 0; If bump, then texture power calculating concrete steps are as follows:

(1) since texture picture be arranged in the regional Ω of virtual environment space coordinates Oxyz=each pixel shared plane sizes under virtual environment is for 0≤x≤200,0≤y≤200, z=0}:

$\mathrm{\&Delta;X}\&times;\mathrm{\&Delta;Y}=\frac{200}{128}\&times;\frac{200}{128}$

Along unidirectional diagonal line each pixel zone of texture picture is divided into two triangles, as shown in Figure 9,

(2) positional information of the virtual protocol of exporting according to position detecting module obtains the triangle at the subpoint place of the point of impingement on face Oxy, supposes subpoint P in triangle _{S, t}P _{S+1, t}P _{S, t+1}, P _{S, t}(Δ X*s, Δ Y*t, 0), P _{S+1, t}(Δ X* (s+1), Δ Y*t, 0), P _{S, t+1}(Δ X*s, Δ Y* (t+1), 0), suppose the coordinate of the point of impingement under virtual environment space coordinates Oxyz for (x, y, z), promptly (x, y, z)=(Mx, My.Mz), then the projection coordinate on face Oxy be (x, y),

(3) suppose triangle P _{S, t}P _{S+1, t}P _{S, t+1}The surface of pairing object texture is spatial triangle Z _{S, t}Z _{S+1, t}Z _{S, t+1}, based on resulting triangle P in the texture picture pre-service _{S, t}P _{S+1, t}P _{S, t+1}The surface elevation value of three pairing object texture in summit obtains triangle Z _{S, t}Z _{S+1, t}Z _{S, t+1}Surface equation, the surface elevation value that substitution subpoint coordinate obtains the pairing object texture of subpoint is d _{X, y},

$d_{x,y}=\frac{d_{s+1,t}-d_{s,t}}{\mathrm{\&Delta;X}}x+\frac{d_{s,t+1}-d_{s,t}}{\mathrm{\&Delta;Y}}y+(1+s+t)d_{s,t}-s{d}_{s+1,t}-t{d}_{s,t+1}---\left(8\right)$

When subpoint when the Ω territory changes continuously in face Oxy, its correspondence the surface elevation value of object texture be d _{X, y}Change continuously,

(4) contacting the computing method of making a concerted effort at point of impingement texture is:

${\stackrel{\&RightArrow;}{F}}_c(x,y)={\stackrel{\&RightArrow;}{F}}_n(x,y)+{\stackrel{\&RightArrow;}{F}}_f(x,y),---\left(9\right)$

Wherein:

The computing method of normal force are as follows:

F _n(x，y)＝kd _x，y， (10)

Wherein k is the stiffness coefficient of stone grain surface, k=0.8, d _{X, y}Be the height value of contact point correspondence, if d _{X, y}For just, represent that then contact point is protruding, normal force is perpendicular to face Oxy and point to z axle positive dirction; If d _{X, y}For negative, represent that then contact point is recessed, normal force is perpendicular to face Oxy and point to z axle negative direction,

The computing method of friction force are as follows:

F _f(x，y)＝μ(x，y)|F _n(x，y)|sign(V)， (11)

(x y) is the kinetic friction coefficient of contact point to μ, and V is the relative velocity of virtual protocol on the virtual texture surface, and sign (V) is the sign function of standard, promptly

$\mathrm{sign}\left(V\right)=\left\{\begin{array}{c}1,V>0\\ 0,V=0\\ -1,V<0\end{array}\right.,---\left(12\right)$

The direction of friction force is parallel with face Oxy, and opposite with direction of motion,

Kinetic friction coefficient be designed to μ (x, y)=ρ r (x, y), wherein ρ is the coefficient of roughness of stone grain surface, ρ=0.4, (x y) is the local roughness coefficient of contact point to r, relevant with the local concavo-convex degree of this contact point, r (x, y) available following model representation:

r(x，y)＝|d _s，t-d _x，y|+|d _s+1，t-d _x，y|+|d _s+1，t+1-d _x，y|+|d _s，t+1-d _x，y|， (13)

D wherein _{S, t}, d _{S+1, t}, d _{S+1, t+1}, d _{S, t+1}Height value for the pairing object texture surface, four summits of pixel region at point of impingement subpoint place on face Oxy, if subpoint drops on the intersection of two (or four) pixel regions, then this subpoint is belonged to these two (or the four) pixel regions and (13) formula of pressing respectively and calculate the local roughness coefficient of this point, get the local roughness coefficient that average is this point at last

The texture contact force information power of transferring to/haptic interaction equipment 3 that step 9 is exported the needs that power/sense of touch computing module 245 produces by PCI integrated circuit board 23 and PCI integrated circuit board connecting line by power/tactile data feedback module 246 is Delta hand controller Delta hand controller, by calling power/haptic interaction equipment 3 is that dhdSetForce () in the Delta hand controller standard A PI function library sets the acting force that the Delta hand controller need be exported

If step 10 virtual protocol collides the grain surface of dummy object, then virtual protocol no longer moves (consideration be the rigidity contact), still draws virtual protocol in situ, and figure refreshes then; If virtual protocol does not collide the grain surface of dummy object, then should in virtual environment, upgrade the position of virtual protocol, promptly detect to such an extent that virtual protocol is drawn in the position that makes new advances at position-based, carry out figure and refresh,

Step 11 is that Delta hand controller, display 1 feed back to the operator to power/tactile data that requires to upgrade and visual information by power/haptic interaction equipment 3, makes it not only can see and can feel the variation of power/tactile data, and then gets back to step 5,

For verifying implementation result of the present invention, the operator is by the handle of DELTA hand controller end, and the virtual bead of touch, perception and control slides at virtual stone grain surface, and the texture force touch information that produces in the reciprocal process is fed back to the operator in real time.In reciprocal process, the operator can perceive the power/tactile data of staff when slipping over the stone grain surface in real time, truly, mutual nature, and comfortable, the power tactile sensation is steady.Experimental result shows: this image texture power tactile sense reproduction system not only realizes simply, and can guarantee that texture contact force and calculating have degree of precision, the texture tactile sense reproduction have the higher sense of reality.

Claims (1)

1. based on the power tactile sense reproduction system of the image texture of power/haptic interaction equipment, it is characterized in that:

(1) according to the body surface 2 d texture picture that imports, suppose that this 2 d texture picture size for m * n pixel, promptly laterally has m pixel and n pixel vertically arranged, the formula below utilizing obtains the brightness value of each pixel:

$l_{i,j}=\frac{R+G+B}{3},---\left(1\right)$

Wherein i ∈ 0,1 ... m-1}, j ∈ 0,1 ..., n-1} is the coordinate figure of pixel corresponding in the texture picture, R, G and B are respectively three color components of red, green, blue of this pixel, the luminance picture L that obtains _{M * n}

(2) luminance picture is carried out spectrum analysis and spatial filtering, the design Gaussian filter carries out spatial filtering to image, with picture breakdown is low frequency and high frequency two parts, low frequency part is used to reflect profile, tendency and the shape of object, HFS is used for the degree of roughness and the concavo-convex degree of reflection surface, and adopting the micro-profile information of high frequency imaging information as the reflection article surface vein, the high frequency imaging information that obtains is H _{M * n},

Concrete steps are as follows:

1. to luminance picture L _{M * n}Do discrete Fourier transformation, obtain frequency domain value F (k, v),

$F(k,v)=\frac{1}{\mathrm{mn}}\underset{a=0}{\overset{m-1}{\mathrm{\&Sigma;}}}\underset{b=0}{\overset{n-1}{\mathrm{\&Sigma;}}}{l}_{a,b}\exp (-j2\mathrm{\&pi;}(\frac{\mathrm{ka}}{m}+\frac{\mathrm{vb}}{n})),---\left(2\right)$

Wherein k ∈ 0,1 ... m-1}, v ∈ 0,1 ..., n-1} is the coordinate figure of pixel in frequency domain corresponding in the texture picture, a ∈ 0,1 ... m-1}, b ∈ 0,1 ..., n-1}, exp () they are the exponential function of standard,

2. to luminance picture L _{M * n}In frequency domain, carry out gaussian filtering, the filtering high-frequency signal, (k v) is the gaussian filtering equation to G, and (k v) is an image to S through the corresponding value in the frequency domain scope of each pixel behind the gaussian filtering

S(k，v)＝G(k，v)F(k，v) (3)

3. (k v) carries out Fourier inversion, obtains reflecting the low-frequency information w of profile, tendency and the shape of object to S _{A, b},

$w_{a,b}=\underset{k=0}{\overset{m-1}{\mathrm{\&Sigma;}}}\underset{v=0}{\overset{n-1}{\mathrm{\&Sigma;}}}S(k,v)\exp \left(j2\mathrm{\&pi;}(\frac{\mathrm{ka}}{m}+\frac{\mathrm{vb}}{n})\right)---\left(4\right)$

4. the high frequency imaging information H that reflects article surface vein _{M * n}For each pixel brightness value deducts the low-frequency information that reflects contour of object and trend, promptly

h _i，j＝l _i，j-w _i，j (5)

Wherein i ∈ 0,1 ... m-1}, j ∈ 0,1 ..., n-1},

(3) to the high frequency imaging information H of resulting reflection article surface vein _{M * n}, carry out standardization processing, obtain the surface elevation Q of the pairing object texture of each pixel _{M * n},

Its computing formula is as follows:

$\left\{\begin{array}{c}{q}_{i,j}=h_{i,j}-\stackrel{\&OverBar;}{h}\\ \stackrel{\&OverBar;}{h}=\frac{\underset{i=0}{\overset{m-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{n-1}{\mathrm{\&Sigma;}}}{h}_{i,j}}{m\&times;n}\end{array}\right.,---\left(6\right)$

Wherein

It is the two-dimensional array matrix H _{M * n}The mean value of middle all elements,

(4) adopt regional averaging method to calculate the surface elevation D of each pairing object texture in pixel summit _{(m+1) * (n+1)}:

Wherein, each pixel summit P _{S, t}Expression, s ∈ 0,1 ... m}, t ∈ 0,1 ..., n}, summit P _{S, t}The surface elevation of pairing object texture is d _{S, t},

Described regional averaging method is: with each pixel summit P _{S, t}Around the mean value of surface elevation of neighborhood pixels area relative object texture be used as the surface elevation d of the pairing object texture on this summit _{S, t},

Described position detecting module (243) is used for, with the coordinate position of power/haptic interaction equipment (3) output (Px, Py, Pz) be converted into coordinate under the virtual environment (Mz), concrete coordinate transformation relation is as follows for Mx, My:

Mx＝9*Py，My＝14*Pz，Mz＝Px*5-5， (7)

Wherein (Pz) obtaining of coordinate position is to realize that by power/haptic interaction equipment it can obtain the positional information of lever in three dimensions of power/haptic interaction equipment end for Px, Py,

Described collision detection module (244) is used for, and judges whether the current power/agency of haptic interaction equipment (3) in virtual environment collides the superficial makings of dummy object,

Described power/sense of touch computing module (245) is used to calculate the texture contact force of current output, if do not collide the texture of dummy object, then power output is 0; If bump, then texture power calculating concrete steps are as follows:

(1) supposes that the texture picture of dummy object is arranged in the regional Ω of virtual environment space coordinates Oxyz={ 0≤x≤X, 0≤y≤Y, z=0}, texture picture aligns about the position in the Virtual Space, and consistency from top to bottom, along unidirectional diagonal line each pixel zone of texture picture is divided into two triangles

(2) positional information of the virtual protocol of exporting according to position detecting module, suppose the coordinate of the point of impingement under virtual environment for (x, y, z), obtain the subpoint of the point of impingement on face Oxy (x, the y) triangle at place,

(3) based on the surface elevation value of the pairing object texture on resulting each pixel summit in the texture picture pretreatment module (242), the surface elevation value that obtains the pairing object texture of subpoint is d _{X, y},

(4) contacting the computing method of making a concerted effort at point of impingement texture is:

${\stackrel{\&RightArrow;}{F}}_c(x,y)={\stackrel{\&RightArrow;}{F}}_n(x,y)+{\stackrel{\&RightArrow;}{F}}_f(x,y),---\left(8\right)$

Wherein:

The computing method of normal force are as follows:

F _n(x，y)＝kd _x，y， (9)

Wherein k is the stiffness coefficient on object texture surface, d _{X, y}Be the height value of contact point correspondence, if d _{X, y}For just, represent that then contact point is protruding, normal force is perpendicular to face Oxy and point to z axle positive dirction; If d _{X, y}For negative, represent that then contact point is recessed, normal force is perpendicular to face Oxy and point to z axle negative direction,

The computing method of friction force are as follows:

F _f(x，y)＝μ(x，y)|F _n(x，y)|sign(V)， (10)

(x y) is the kinetic friction coefficient of contact point to μ, and V is the relative velocity of virtual protocol on the virtual texture surface, and sign (V) is the sign function of standard, promptly

$\mathrm{sign}\left(V\right)=\left\{\begin{array}{c}1,V>0\\ 0,V=0\\ -1,V<0\end{array}\right.,---\left(11\right)$

The direction of friction force is parallel with face Oxy, and opposite with direction of motion,

Kinetic friction coefficient be designed to μ (x, y)=ρ r (x, y), wherein ρ is the coefficient of roughness of grain surface material, r (x y) is the local roughness coefficient of contact point, and is relevant with the local concavo-convex degree of this contact point, r (x, the model representation below y) available:

r(x，y)＝|d _s，t-d _x，y|+|d _s+1，t-d _x，y|+|d _s+1，t+1-d _x，y|+|d _s，t+1-d _x，y|， (12)

D wherein _{S, t}, d _{S+1, t}, d _{S+1, t+1}, d _{S, t+1}Height value for the pairing object texture surface, four summits of pixel region at point of impingement subpoint place on face Oxy, if subpoint drops on the intersection of two or four pixel regions, then this subpoint is belonged to these two or four pixel regions respectively and calculate the local roughness coefficient of this point by (12) formula, get the local roughness coefficient that average is this point at last

The texture contact force information power of transferring to/haptic interaction equipment (3) that the needs that described power/tactile data feedback module (246) produces power/sense of touch computing module (245) by PCI integrated circuit board (23) and PCI integrated circuit board connecting line are exported, come the acting force that setting power/haptic interaction equipment need be exported by power/haptic interaction equipment

Described figure refresh module (247) is used for the shift position that experimenter's Real Time Observation on display arrives virtual protocol,

Described display card (21), PCI integrated circuit board (23), hard disk (24) are all realized transmitted in both directions by bus (22),

Described display (1) is connected with display card (21) by the VGA connecting line.

Images