CN112506339A - Virtual hand force sense synthesis method and system for wearable touch sense interaction - Google Patents
Virtual hand force sense synthesis method and system for wearable touch sense interaction Download PDFInfo
- Publication number
- CN112506339A CN112506339A CN202011377646.3A CN202011377646A CN112506339A CN 112506339 A CN112506339 A CN 112506339A CN 202011377646 A CN202011377646 A CN 202011377646A CN 112506339 A CN112506339 A CN 112506339A
- Authority
- CN
- China
- Prior art keywords
- virtual hand
- configuration information
- pose
- virtual
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to a virtual hand force sense synthesis method and system for wearable touch sense interaction. The method comprises the steps of constructing a virtual hand model according to the physiological structure of the hand; constructing a virtual object; determining collision pairs corresponding to different collision areas of the virtual hand model by adopting a collision detection algorithm according to the virtual object and the virtual hand model; determining a pose optimization model of a corresponding collision area according to the collision pair of each collision area, and determining configuration information of the pose of the virtual hand figure which is not penetrated by the corresponding collision area and the virtual object according to the pose optimization model; and determining interaction force information required in the interaction process between the corresponding collision area in the virtual hand model and the virtual object according to the configuration information of the virtual hand graph pose of each collision area which is not penetrated by the virtual object and the configuration information of the physical pose of the corresponding collision area of the virtual hand model. The invention improves the computational efficiency of the virtual hand force sense synthesis and ensures the non-penetration simulation of the interaction between the virtual hand and the virtual object.
Description
Technical Field
The invention relates to the field of human-computer interaction of touch sense, in particular to a virtual hand force sense synthesis method and system for wearable touch sense interaction.
Background
As a typical wearable force tactile feedback device, a force feedback glove allows a user to touch and/or grab a virtual object by controlling a virtual hand avatar, and obtain force feedback at a fingertip or the whole hand, compared with desktop type tactile interaction, the wearable type tactile interaction method enables the user to touch and operate a remote/virtual object more naturally and directly, can significantly enhance the immersion and interactivity with a virtual scene, and is a research hotspot in the field of current tactile man-machine interaction.
The haptic feedback device and the corresponding haptic synthesis algorithm are two important factors for promoting the prosperous development of haptic interaction technology. The rapid development of force feedback gloves is facilitated by the urgent need for wearable interactive applications (e.g., virtual fitting, virtual surgery, high-risk work environment simulation, professional skill training, etc.), and the development of immersive wearable haptic interactive application scenarios also requires high-fidelity virtual hand force synthesis/rendering algorithms for support.
In order to meet the requirement of wearable haptic interaction, researchers at home and abroad develop related researches on a virtual hand force sense synthesis algorithm, and the researches are gradually transited from a heuristic strategy which only supports specific gestures at first to a force sense synthesis method based on rigid virtual hand modeling and a force sense synthesis method of rigid-flexible coupling virtual hand modeling considering human hand physiological structures. Zachmann et al uses the distribution of contact between the phalanges of the fingers, the phalanges of the thumb, and the palm to distinguish between the types of grasp, and then maintains the grasp by considering the relative position and orientation of the object with respect to the palm. Borst et al, effectively simulated force feedback on the fingers and palm using linear and torsional spring damper systems to couple the real hand configuration with the virtual hinged hand model. Garre et al have constructed a deformed virtual hand model containing bones and muscles, have modeled the contact between the deformed hand and the object using a constraint-based approach, and have improved the stability of force feedback by extending a virtual coupling algorithm.
Although the existing virtual hand force sense synthesis methods can generate relatively natural interactive simulation, the methods only support the interactive simulation of the virtual hand and the rigid object in a simple contact state, and the realistic presentation of the virtual hand interactive force in a complex contact state is difficult to realize. Complex contact mainly refers to the situation that contact points between a virtual hand and a virtual object to be operated are distributed in a complex topology, such as multi-point multi-area contact, contact point sliding and the like. The topological distribution of contact points between a virtual hand and an operated object in a complex contact state is complex and changeable, and great challenges are brought to efficient and stable synthesis of virtual hand interaction force in the complex contact state, while the calculation process of the existing method is only tens of hertz, and researches on how to improve the calculation efficiency of virtual hand force perception synthesis are urgently needed so as to realize that the update frequency of all calculation modules is more than 1kHz, and meanwhile, ensure non-penetration simulation of interaction between the virtual hand and the virtual object.
Disclosure of Invention
The invention aims to provide a virtual hand force sense synthesis method and system for wearable haptic interaction, which improve the calculation efficiency of virtual hand force sense synthesis.
In order to achieve the purpose, the invention provides the following scheme:
a virtual hand force sensation synthesis method for wearable haptic interaction, comprising:
constructing a virtual hand model according to the physiological structure of the hand; the virtual hand model comprises a palm and fingers; the fingers comprise a thumb, an index finger, a middle finger, a ring finger and a little finger; the finger comprises three knuckles;
constructing a virtual object;
determining collision pairs corresponding to different collision areas of the virtual hand model by adopting a collision detection algorithm according to the virtual object and the virtual hand model; the collision region comprises a palm, a thumb, an index finger, a middle finger, a ring finger and a little finger in the virtual hand model;
determining a pose optimization model of a corresponding collision area according to the collision pair of each collision area, and determining configuration information of the pose of the virtual hand figure which is not penetrated by the corresponding collision area and the virtual object according to the pose optimization model; the configuration information comprises attitude data and six-degree-of-freedom position information;
determining interaction force information required in the interaction process between the corresponding collision area in the virtual hand model and the virtual object according to the configuration information of the virtual hand graphic pose of each collision area which is not penetrated by the virtual object and the configuration information of the physical pose of the corresponding collision area of the virtual hand model; the interaction force information includes a feedback force and a feedback moment for each impact zone.
Optionally, the constructing a virtual hand model according to the physiological structure of the hand specifically includes:
constructing a palm in the virtual hand model by adopting a ball tree model according to the physiological structure of the hand;
and constructing fingers in the virtual hand model by adopting a cylindrical hinge model according to the physiological structure of the hand.
Optionally, the determining, according to the collision pair of each collision region, a pose optimization model of the corresponding collision region, and determining, according to the pose optimization model, configuration information of a pose of a virtual hand figure, which is not penetrated by the virtual object, of the corresponding collision region specifically includes:
judging whether the collision area has a palm in a virtual hand model;
if the palm exists, determining a pose optimization model of the palm according to the collision pair of the palm;
determining configuration information of a virtual hand graphic pose which is not penetrated by the palm and the virtual object at the current moment by using an active set method according to the pose optimization model of the palm;
if the virtual hand graphic pose does not exist, determining the configuration information of the palm in the virtual hand graphic pose at the previous moment as the configuration information of the palm at the current moment;
judging whether fingers in the virtual hand model exist in the collision area or not;
if the finger collision pair exists, sequentially determining pose optimization models of three knuckles in the finger from the knuckle connected with the palm according to the finger collision pair;
according to the pose optimization model of the fingers, sequentially determining configuration information of the pose of the virtual hand graph which is not penetrated by the fingers and the virtual object at the current moment from the knuckle connected with the palm by using an active set method;
and if the virtual hand graphic pose does not exist, determining the configuration information of the finger in the virtual hand graphic pose at the previous moment as the configuration information of the finger at the current moment.
Optionally, the determining, according to the collision pair of each collision region, a pose optimization model of the corresponding collision region, and determining, according to the pose optimization model, configuration information of a pose of a virtual hand figure that the corresponding collision region and the virtual object do not penetrate through, further includes:
and acquiring the configuration information of the physical pose of the corresponding collision area of the virtual hand model by using a hand pose measuring device.
Optionally, the determining, according to configuration information of a virtual hand figure pose of each collision region and the virtual object, which is not penetrated through by the virtual object, and configuration information of a physical pose of a collision region corresponding to the virtual hand model, interaction force information required in an interaction process between the corresponding collision region in the virtual hand model and the virtual object specifically includes:
determining the configuration information difference value of the corresponding collision area according to the configuration information of the virtual hand figure pose of each collision area and the virtual object non-penetration and the configuration information of the physical pose of the corresponding collision area of the virtual hand model;
and determining the interaction force information of the corresponding collision area according to the configuration information difference value, the rigidity coefficient and the torsion spring coefficient of each collision area.
A virtual hand force sensation synthesis system for wearable haptic interaction, comprising:
the virtual hand model building module is used for building a virtual hand model according to the physiological structure of the hand; the virtual hand model comprises a palm and fingers; the fingers comprise a thumb, an index finger, a middle finger, a ring finger and a little finger; the finger comprises three knuckles;
the virtual object constructing module is used for constructing a virtual object;
a collision pair determining module, configured to determine, according to the virtual object and the virtual hand model, a collision pair corresponding to different collision regions of the virtual hand model by using a collision detection algorithm; the collision region comprises a palm, a thumb, an index finger, a middle finger, a ring finger and a little finger in the virtual hand model;
the configuration information determining module is used for determining a pose optimization model of a corresponding collision area according to the collision pair of each collision area and determining configuration information of the pose of the virtual hand figure which is not penetrated by the corresponding collision area and the virtual object according to the pose optimization model; the configuration information comprises attitude data and six-degree-of-freedom position information;
the interaction force information determining module is used for determining interaction force information required in the interaction process between the corresponding collision area in the virtual hand model and the virtual object according to the configuration information of the virtual hand graphic pose of each collision area which is not penetrated by the virtual object and the configuration information of the physical pose of the corresponding collision area of the virtual hand model; the interaction force information includes a feedback force and a feedback moment for each impact zone.
Optionally, the virtual hand model building module specifically includes:
the palm constructing unit is used for constructing a palm in the virtual hand model by adopting a ball tree model according to the physiological structure of the hand;
and the finger building unit is used for building the fingers in the virtual hand model by adopting a cylindrical hinge model according to the physiological structure of the hand.
Optionally, the configuration information determining module specifically includes:
the first judgment unit is used for judging whether the collision area has a palm in a virtual hand model;
the palm posture optimization model determining unit is used for determining a palm posture optimization model according to the collision pair of the palm if the palm exists;
the first configuration information determining unit corresponding to the palm is used for determining configuration information of a virtual hand graphic pose which is not penetrated by the palm and the virtual object at the current moment by using an active set method according to the pose optimization model of the palm;
the second configuration information determining unit corresponding to the palm is used for determining the configuration information of the palm in the virtual hand graphic pose at the previous moment as the configuration information of the palm at the current moment if the second configuration information does not exist;
the second judgment unit is used for judging whether fingers in the virtual hand model exist in the collision area or not;
if the model determining unit exists, sequentially determining pose optimization models of three knuckles in the finger from the knuckle connected with the palm according to the collision pair of the finger;
the first determination unit of the configuration information corresponding to the finger is used for sequentially determining the configuration information of the virtual hand graphic pose which is not penetrated by the finger and the virtual object at the current moment from a knuckle connected with a palm by using an active set method according to the pose optimization model of the finger;
and the second configuration information determining unit corresponding to the finger is used for determining the configuration information of the finger in the virtual hand graphic pose at the previous moment as the configuration information of the finger at the current moment if the configuration information does not exist.
Optionally, the method further includes:
and the physical pose configuration information acquisition module is used for acquiring the physical pose configuration information of the corresponding collision area of the virtual hand model by using the hand pose measurement device.
Optionally, the interaction force information determining module specifically includes:
the configuration information difference value determining unit is used for determining the configuration information difference value of the corresponding collision area according to the configuration information of the virtual hand figure pose of each collision area and the virtual object non-penetration and the configuration information of the physical pose of the corresponding collision area of the virtual hand model;
and the interactive force information determining unit is used for determining the interactive force information of the corresponding collision area according to the configuration information difference value, the rigidity coefficient and the torsion spring coefficient of each collision area.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
according to the virtual hand force sense synthesis method and system for wearable touch sense interaction, in the interaction process of the virtual hand model and the virtual object, collision detection algorithms are adopted to determine collision pairs corresponding to different collision areas of the virtual hand model, namely the contact areas of the virtual hand model and the virtual object are divided. Further, a pose optimization model of each collision area of the virtual hand model is determined, and configuration information of virtual hand graph poses, which are not penetrated by the virtual object, of each collision area is determined; and determining interaction force information required in the interaction process between the corresponding collision area in the virtual hand model and the virtual object according to the configuration information of the virtual hand graph pose of each collision area which is not penetrated by the virtual object and the configuration information of the physical pose of the corresponding collision area of the virtual hand model. The virtual hand force sense synthesis method decomposes complex contact processing problems, reduces the difficulty of virtual hand posture optimization, and can improve the calculation efficiency of the whole virtual hand force sense synthesis process. And moreover, the calculation efficiency of the whole virtual hand force sense synthesis process can be further effectively improved by adopting a parallel calculation mode.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flow chart of a virtual hand force sensation synthesis method for wearable haptic interaction according to the present invention;
FIG. 2 is a schematic diagram of a virtual hand force sense synthesis principle provided by the present invention;
FIG. 3 is a schematic diagram of the intersection of a finger and a virtual object cylinder-sphere model in the virtual hand model provided by the present invention;
FIG. 4 is a flow chart of multi-thread force sense synthesis of a virtual hand model provided by the present invention;
FIG. 5 is a schematic diagram of virtual hand graphical pose optimization provided by the present invention;
fig. 6 is a schematic flow chart of a virtual hand force sensation synthesis system for wearable haptic interaction according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a virtual hand force sense synthesis method and system for wearable haptic interaction, which improve the calculation efficiency of virtual hand force sense synthesis.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a schematic flow chart of a virtual hand force sensation synthesis method for wearable haptic interaction provided by the present invention, fig. 2 is a schematic diagram of a principle of virtual hand force sensation synthesis provided by the present invention, and as shown in fig. 1 and fig. 2, a virtual hand force sensation synthesis method for wearable haptic interaction provided by the present invention includes:
s101, constructing a virtual hand model according to the physiological structure of the hand; the virtual hand model comprises a palm and fingers; the fingers comprise a thumb, an index finger, a middle finger, a ring finger and a little finger; the finger includes three knuckles.
S101 specifically comprises the following steps:
and constructing the palm in the virtual hand model by adopting a ball tree model according to the physiological structure of the hand.
And constructing fingers in the virtual hand model by adopting a cylindrical hinge model according to the physiological structure of the hand.
S102, constructing a virtual object.
S103, determining collision pairs corresponding to different collision areas of the virtual hand model by adopting a collision detection algorithm according to the virtual object and the virtual hand model; the collision regions include a palm, a thumb, an index finger, a middle finger, a ring finger, and a little finger in the virtual hand model.
The collision detection algorithm comprises a ball-ball collision detection algorithm or a ball-cylinder collision detection algorithm.
As a specific embodiment, the collision pair determination method includes:
and performing collision detection by taking the whole virtual hand model as an object to obtain all collision pairs, and then obtaining the collision pairs of collision regions (contact points/regions) of all parts (namely the palm, the thumb, the index finger, the middle finger, the ring finger and the little finger) according to the division mode of the virtual hand.
Collision detection is performed on each part (i.e., palm, thumb, index finger, middle finger, ring finger, and little finger) of the virtual hand model, and collision pairs of collision regions (contact points/regions) corresponding to each part are obtained.
As a specific embodiment, a ball tree model is used to construct a virtual object, and a ball-ball collision detection algorithm is used to detect collision information between a palm in a virtual hand model and the virtual object, and whether a collision occurs is determined according to whether a distance between a small ball on the palm and a center of a small ball on the virtual object is smaller than a sum of radii, and if so, the two collision occurs, and the specific determination formula is as follows:
(xa-xb)+(ya-yb)+(za-zb)＜(ra+rb)2。
wherein (x)a,ya,za) And (x)b,yb,zb) Respectively representing the central coordinates, r, of the virtual palm and of the small sphere on the virtual objectaAnd rbRespectively the radius of the pellet.
As a specific embodiment, a ball tree model is adopted to construct a virtual object, and then a ball-cylinder collision detection algorithm is adopted to detect collision information between fingers and the virtual object in a virtual hand model, wherein the collision information is obtained according to different intersection modes (complete intersection, incomplete intersection, normal intersection and oblique intersection) of the finger cylinder model and the virtual object ball modelDifferent judgment bases are adopted for judgment. The intersection of the finger cylinder model and the virtual object sphere model in the virtual hand model is shown in fig. 3, where O denotes the center of the sphere, A, B denotes the focus of the single knuckle cylinder model and the virtual object sphere model, respectively, C denotes the location of the foot from the center of the virtual object sphere model to the central axis of the cylinder model, D and E denote the two end points of the cylinder, (x)D,yD,zD) And (x)E,yE,zE) Respectively, the coordinates of the point D and the point E.
If the intersection mode is complete intersection, judging the relationship between the distance | OC | from the center of the sphere to the central axis of the cylinder and the radius R of the sphere and the radius R of the cylinder, and if the following formula is satisfied, judging that the two are collided:
wherein the content of the first and second substances,
B＝(yD-yO)(zE-zD)-(yE-yD)(zD-zO)，C＝(zD-zO)(xE-xD)-(zE-zD)(xD-xO)，D＝(xD-xO)(yE-yD)-(xE-xD)(yD-yO)。
if the intersection mode is completely disjoint, judging whether the conditions in the following formula are met at the same time, and if so, judging that the two are collided:
wherein F ═ xO-xE)(xD-xE)+(yO-yE)(yD-yE)+(zO-zE)(zD-zE)。
If the intersection mode is normal intersection or oblique intersection, judging whether the following formula is met, and if so, judging that the two are collided:
wherein the content of the first and second substances,
and
the distances from the center of the ball on the virtual object to two end points D and E of the finger cylinder are respectively.
As a specific example, collision detection is performed on each part (i.e., palm, thumb, index finger, middle finger, ring finger, and little finger) of the virtual hand model, and collision pairs of contact points/regions of each part are obtained. Specifically, the collision pair of the virtual hand palm contacting the virtual object is obtained by using the ball-ball collision detection algorithm described above, and the collision pair of the finger contacting the virtual object in the virtual hand module is obtained by using the ball-cylinder collision detection algorithm described above.
S104, determining a pose optimization model of a corresponding collision area according to the collision pair of each collision area, and determining configuration information of the pose of the virtual hand figure which is not penetrated by the corresponding collision area and the virtual object according to the pose optimization model; the configuration information includes attitude data and six degree of freedom position information.
As a specific embodiment, a pose optimization model is respectively constructed for each collision region of the virtual hand model, and the palm of the virtual hand graph pose is determined by a six-dimensional variable in space, namely
Wherein
Representing the translation amount of the virtual hand figure pose along the coordinate axes X, Y and Z at the current moment,
and representing the rotation amount of the graphic pose of the virtual tool at the current moment along the coordinate axes X, Y and Z. Similarly, the palm of the virtual hand physical pose is represented in space as
For the solution of the virtual hand palm graphic pose, the following optimization model is constructed:
where G is a 6 x 6 diagonal stiffness matrix including the translational and torsional stiffness of the tool. i represents the ith collision ball pair (N collision pairs in total) on the virtual palm, CiThe constraint condition of the ith collision ball pair, namely the sum of the radii of the object ball and the palm ball, is that the virtual palm is not embedded into the virtual object, and the constraint condition is expressed as follows:
(xi-xOj)2+(yi-yOj)2+(zi-zOj)2≥(ri-rOj)2。
wherein (x)Oj,yOj,zOj) Is the sphere center coordinate of the jth collision ball on the virtual object. r isi,rOjThe radii of the small ball on the virtual palm and the small ball on the virtual object are respectively.
Simulating fingers in a virtual hand module by adopting a cylindrical hinge model, solving the poses of each knuckle (cylindrical model) of five fingers of the virtual hand, and constructing the following optimization model:
wherein the content of the first and second substances,
the corner of the metacarpophalangeal joint, the corner of the proximal phalangeal joint and the corner of the distal phalangeal joint of the single finger figure pose are respectively represented. In the same way as above, the first and second,
Grrepresenting a hessian matrix. CjThe constraint condition that the jth collision pair (M collision pairs in total) represents that the virtual finger is not embedded into the virtual object, namely the distance between the center of the sphere of the object and the distance between the central axes of the cylinders of the fingers are greater than the sum of the radii of the sphere and the cylinders, is expressed as follows:
wherein (x)j,yj,zj) Is the sphere center coordinate of the jth collision ball on the virtual object. r iss,rcThe radii of the object sphere and the hand cylinder of the virtual hand model, respectively.
S104 specifically comprises the following steps:
and judging whether the collision area has a palm in the virtual hand model.
And if so, determining a pose optimization model of the palm according to the collision pair of the palm.
And determining configuration information of the virtual hand graphic pose which is not penetrated by the palm and the virtual object at the current moment by using an active set method according to the pose optimization model of the palm.
And if the virtual hand graphic pose does not exist, determining the configuration information of the palm in the virtual hand graphic pose at the previous moment as the configuration information of the palm at the current moment.
And judging whether the finger in the virtual hand model exists in the collision area.
And if so, sequentially determining pose optimization models of three knuckles in the fingers from the knuckle connected with the palm according to the collision pair of the fingers.
And according to the pose optimization model of the finger, sequentially determining configuration information of the pose of the virtual hand graph which is not penetrated by the finger and the virtual object at the current moment from a knuckle connected with the palm by using an active set method.
And if the virtual hand graphic pose does not exist, determining the configuration information of the finger in the virtual hand graphic pose at the previous moment as the configuration information of the finger at the current moment.
And if the collision pair obtained by collision detection comprises the contact between the thumb and the virtual object, sequentially optimizing the poses of the three knuckles of the thumb from the knuckle connected with the palm according to the configuration information of the palm in the virtual hand figure pose to obtain the optimal configuration information of the thumb in the virtual hand figure pose, and if the collision pair obtained by collision detection does not comprise the contact between the thumb and the virtual object, assigning the configuration information of the thumb in the virtual hand figure pose at the previous moment to the configuration information of the thumb at the current moment.
And sequentially optimizing the poses of the index finger, the middle finger, the ring finger and the little finger respectively to finally obtain the optimal configuration information of the pose of the whole virtual hand graph.
The method for optimizing the graphic pose of the palm and each finger in the virtual hand model is shown in fig. 5, and in some embodiments, a progressive solution idea is adopted, so that the palm is firstly subjected to constraint optimization, and then five fingers are respectively subjected to constraint optimization on the basis, and further the graphic tool pose of the hand is solved. The virtual hand figure pose optimization idea for progressive solving has the advantages that: each solving variable is few, and a feasible solution can be quickly found; moreover, because the solving processes of the fingers are independent, even if one finger does not obtain a feasible solution, the solving results of the rest fingers are not influenced.
In S104, the pose solution of each finger is not interfered with each other, and high-speed parallel computation is supported. Specifically, by utilizing a physiological characteristic that motions between fingers are basically independent from each other, palm force sense synthesis and five-finger force sense synthesis are respectively placed in six threads, each thread relates to collision detection, graphic pose optimization and interactive force (including feedback force and moment) calculation of a virtual palm/finger and a virtual object, and the parallel computing mode can effectively improve the computing efficiency of the whole virtual hand force sense synthesis process so as to meet the application requirement of force sense feedback of 1 kHz.
After S104, further comprising:
and acquiring the configuration information of the physical pose of the corresponding collision area of the virtual hand model by using a hand pose measuring device.
In a specific embodiment, the posture data of the hand is captured by using a nocateva Hi5 motion capture glove, and the six-degree-of-freedom position information of the hand is obtained by tracking through an HTC Tracker 2.0.
S105, determining interaction force information required in the interaction process between the corresponding collision area in the virtual hand model and the virtual object according to the configuration information of the virtual hand graph pose of each collision area which is not penetrated by the virtual object and the configuration information of the physical pose of the corresponding collision area of the virtual hand model; the interaction force information includes a feedback force and a feedback moment for each impact zone.
S105 specifically comprises the following steps:
and determining the configuration information difference value of the corresponding collision area according to the configuration information of the virtual hand figure pose of each collision area and the virtual object non-penetration and the configuration information of the physical pose of the corresponding collision area of the virtual hand model.
And determining the interaction force information of the corresponding collision area according to the configuration information difference value, the rigidity coefficient and the torsion spring coefficient of each collision area.
The virtual hand force sense synthesis method for wearable haptic interaction provided by the invention converts a complex virtual hand force sense synthesis problem into a plurality of sub-problems by adopting a divide-and-conquer strategy, and has the following advantages:
(1) the difficulty in optimizing and solving the series of sub-problems is greatly reduced, the optimal configuration of the virtual hand graphical tool is facilitated, and the virtual hand is ensured not to penetrate through a virtual object in the interaction process.
(2) In the optimization solving process of the subproblems, the palm, the thumb, the index finger, the middle finger, the ring finger and the little finger in the virtual hand graphical tool are respectively processed, so that the pose optimization of each part of the virtual hand is not interfered with each other, the poses of three knuckles of each finger are sequentially optimized, and the improvement of the naturalness of the interactive gestures of the virtual hand is facilitated.
(3) Compared with the method of finishing the force sense synthesis by taking the virtual hand as a whole, the method has the advantages that the difficulty in solving the neutron problem in the divide-and-conquer strategy is reduced, and the calculation efficiency of the virtual hand force sense synthesis can be improved.
(4) In addition, six force sense synthesis subproblems (including collision detection, pose optimization and interactive force calculation) related to the palm, the thumb, the index finger, the middle finger, the ring finger and the little finger are respectively processed in a parallel computing mode, and the computing efficiency of virtual hand force sense synthesis can be further improved.
Fig. 6 is a schematic flow chart of a virtual hand force sensation synthesis system for wearable haptic interaction according to the present invention, and as shown in fig. 6, the virtual hand force sensation synthesis system for wearable haptic interaction according to the present invention includes: a virtual hand model construction module 601, a virtual object construction module 602, a collision pair determination module 603, a configuration information determination module 604, and an interaction force information determination module 605.
The virtual hand model building module 601 is used for building a virtual hand model according to the physiological structure of the hand; the virtual hand model comprises a palm and fingers; the fingers comprise a thumb, an index finger, a middle finger, a ring finger and a little finger; the finger includes three knuckles.
The virtual object build module 602 is used to build virtual objects.
The collision pair determining module 603 is configured to determine, according to the virtual object and the virtual hand model, a collision pair corresponding to different collision regions of the virtual hand model by using a collision detection algorithm; the collision regions include a palm, a thumb, an index finger, a middle finger, a ring finger, and a little finger in the virtual hand model.
The configuration information determining module 604 is configured to determine a pose optimization model of each collision region according to the collision pair of each collision region, and determine configuration information of a virtual hand figure pose which is not penetrated by the corresponding collision region and the virtual object according to the pose optimization model; the configuration information includes attitude data and six degree of freedom position information.
The interaction force information determining module 605 is configured to determine, according to configuration information of a virtual hand figure pose of each collision region and the virtual object, which is not penetrated through by the virtual object, and configuration information of a physical pose of a collision region corresponding to the virtual hand model, interaction force information required in an interaction process between the corresponding collision region in the virtual hand model and the virtual object; the interaction force information includes a feedback force and a feedback moment for each impact zone.
The virtual hand model building module 601 specifically includes: a palm construction unit and a finger construction unit.
The palm constructing unit is used for constructing a palm in the virtual hand model by adopting a ball tree model according to the physiological structure of the hand.
The finger building unit is used for building the fingers in the virtual hand model by adopting a cylindrical hinge model according to the physiological structure of the hand.
The configuration information determining module 604 specifically includes: the device comprises a first judgment unit, a palm pose optimization model determination unit, a palm corresponding configuration information first determination unit, a palm corresponding configuration information second determination unit, a second judgment unit, a finger pose optimization model determination unit, a finger corresponding configuration information first determination unit and a finger corresponding configuration information second determination unit.
The first judging unit is used for judging whether the collision area has a palm in the virtual hand model.
And the palm pose optimization model determining unit is used for determining a palm pose optimization model according to the collision pair of the palm if the palm exists.
The first configuration information determining unit corresponding to the palm is used for determining configuration information of a virtual hand graphic pose which is not penetrated by the palm and the virtual object at the current moment by using an active set method according to the pose optimization model of the palm;
the second configuration information determining unit corresponding to the palm is used for determining the configuration information of the palm in the virtual hand graphic pose at the previous moment as the configuration information of the palm at the current moment if the second configuration information does not exist;
the second judgment unit is used for judging whether the collision area has fingers in the virtual hand model;
if the model determining unit exists, sequentially determining pose optimization models of three knuckles in the finger from the knuckle connected with the palm according to the collision pair of the finger;
the first determination unit of the configuration information corresponding to the finger is used for sequentially determining the configuration information of the virtual hand graphic pose which is not penetrated by the finger and the virtual object at the current moment from a knuckle connected with a palm by using an active set method according to the pose optimization model of the finger;
and the second configuration information determining unit corresponding to the finger is used for determining the configuration information of the finger in the virtual hand graphic pose at the previous moment as the configuration information of the finger at the current moment if the configuration information does not exist.
The invention provides a virtual hand force sense synthesis system for wearable haptic interaction, which further comprises: and a configuration information acquisition module of the physical pose.
And the physical pose configuration information acquisition module is used for acquiring the physical pose configuration information of the corresponding collision area of the virtual hand model by using the hand pose measurement device.
The interactive force information determining module 605 specifically includes: a configuration information difference value determining unit and an interaction force information determining unit.
The configuration information difference value determining unit is used for determining the configuration information difference value of the corresponding collision area according to the configuration information of the virtual hand figure pose of each collision area and the virtual object non-penetration and the configuration information of the physical pose of the corresponding collision area of the virtual hand model.
The interactive force information determining unit is used for determining the interactive force information of the corresponding collision area according to the configuration information difference value, the rigidity coefficient and the torsion spring coefficient of each collision area.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (10)
1. A virtual hand force sensation synthesis method for wearable haptic interaction, comprising:
constructing a virtual hand model according to the physiological structure of the hand; the virtual hand model comprises a palm and fingers; the fingers comprise a thumb, an index finger, a middle finger, a ring finger and a little finger; the finger comprises three knuckles;
constructing a virtual object;
determining collision pairs corresponding to different collision areas of the virtual hand model by adopting a collision detection algorithm according to the virtual object and the virtual hand model; the collision region comprises a palm, a thumb, an index finger, a middle finger, a ring finger and a little finger in the virtual hand model;
determining a pose optimization model of a corresponding collision area according to the collision pair of each collision area, and determining configuration information of the pose of the virtual hand figure which is not penetrated by the corresponding collision area and the virtual object according to the pose optimization model; the configuration information comprises attitude data and six-degree-of-freedom position information;
determining interaction force information required in the interaction process between the corresponding collision area in the virtual hand model and the virtual object according to the configuration information of the virtual hand graphic pose of each collision area which is not penetrated by the virtual object and the configuration information of the physical pose of the corresponding collision area of the virtual hand model; the interaction force information includes a feedback force and a feedback moment for each impact zone.
2. The method according to claim 1, wherein the constructing a virtual hand model according to the physiological structure of the hand specifically comprises:
constructing a palm in the virtual hand model by adopting a ball tree model according to the physiological structure of the hand;
and constructing fingers in the virtual hand model by adopting a cylindrical hinge model according to the physiological structure of the hand.
3. The virtual hand force sensation synthesis method for wearable haptic interaction according to claim 1, wherein the determining a pose optimization model of each collision region according to the collision pair of each collision region, and determining configuration information of the virtual hand graphic pose of each collision region and the virtual object non-penetrating according to the pose optimization model specifically comprises:
judging whether the collision area has a palm in a virtual hand model;
if the palm exists, determining a pose optimization model of the palm according to the collision pair of the palm;
determining configuration information of a virtual hand graphic pose which is not penetrated by the palm and the virtual object at the current moment by using an active set method according to the pose optimization model of the palm;
if the virtual hand graphic pose does not exist, determining the configuration information of the palm in the virtual hand graphic pose at the previous moment as the configuration information of the palm at the current moment;
judging whether fingers in a virtual hand model exist in the collision area or not;
if the finger collision pair exists, sequentially determining pose optimization models of three knuckles in the finger from the knuckle connected with the palm according to the finger collision pair;
according to the pose optimization model of the fingers, sequentially determining configuration information of the pose of the virtual hand graph, which is not penetrated by the fingers and the virtual object at the current moment, from the knuckle connected with the palm by using an active set method;
and if the virtual hand graphic pose does not exist, determining the configuration information of the finger in the virtual hand graphic pose at the previous moment as the configuration information of the finger at the current moment.
4. A virtual hand force sensation synthesis method for wearable haptic interaction according to claim 1, wherein the determining a pose optimization model of the respective collision region according to the collision pair of each collision region and determining configuration information of the respective collision region and the virtual hand graphic pose that is not penetrated by the virtual object according to the pose optimization model, then further comprises:
and acquiring the configuration information of the physical pose of the corresponding collision area of the virtual hand model by using a hand pose measuring device.
5. The virtual hand force sensation synthesis method for wearable haptic interaction according to claim 1, wherein the determining interaction force information required during interaction between a corresponding collision region in the virtual hand model and the virtual object according to configuration information of a virtual hand graphic pose of each collision region and the virtual object, and configuration information of a physical pose of the corresponding collision region in the virtual hand model, specifically comprises:
determining the configuration information difference value of the corresponding collision area according to the configuration information of the virtual hand figure pose of each collision area and the virtual object non-penetration and the configuration information of the physical pose of the corresponding collision area of the virtual hand model;
and determining the interaction force information of the corresponding collision area according to the configuration information difference value, the rigidity coefficient and the torsion spring coefficient of each collision area.
6. A virtual hand force sensation synthesis system for wearable haptic interaction, comprising:
the virtual hand model building module is used for building a virtual hand model according to the physiological structure of the hand; the virtual hand model comprises a palm and fingers; the fingers comprise a thumb, an index finger, a middle finger, a ring finger and a little finger; the finger comprises three knuckles;
the virtual object constructing module is used for constructing a virtual object;
a collision pair determining module, configured to determine, according to the virtual object and the virtual hand model, a collision pair corresponding to different collision regions of the virtual hand model by using a collision detection algorithm; the collision region comprises a palm, a thumb, an index finger, a middle finger, a ring finger and a little finger in the virtual hand model;
the configuration information determining module is used for determining a pose optimization model of a corresponding collision area according to the collision pair of each collision area and determining configuration information of the pose of the virtual hand figure which is not penetrated by the corresponding collision area and the virtual object according to the pose optimization model; the configuration information comprises attitude data and six-degree-of-freedom position information;
the interaction force information determining module is used for determining interaction force information required in the interaction process between the corresponding collision area in the virtual hand model and the virtual object according to the configuration information of the virtual hand graphic pose of each collision area which is not penetrated by the virtual object and the configuration information of the physical pose of the corresponding collision area of the virtual hand model; the interaction force information includes a feedback force and a feedback moment for each impact zone.
7. A virtual hand force sensation synthesis system for wearable haptic interaction according to claim 6, characterized in that the virtual hand model construction module comprises in particular:
the palm constructing unit is used for constructing a palm in the virtual hand model by adopting a ball tree model according to the physiological structure of the hand;
and the finger building unit is used for building the fingers in the virtual hand model by adopting a cylindrical hinge model according to the physiological structure of the hand.
8. A virtual hand force sensation synthesis system for wearable haptic interaction according to claim 6, characterized in that the configuration information determination module comprises in particular:
the first judgment unit is used for judging whether the collision area has a palm in a virtual hand model;
the palm posture optimization model determining unit is used for determining a palm posture optimization model according to the collision pair of the palm if the palm exists;
the first configuration information determining unit corresponding to the palm is used for determining configuration information of a virtual hand graphic pose which is not penetrated by the palm and the virtual object at the current moment by using an active set method according to the pose optimization model of the palm;
the second configuration information determining unit corresponding to the palm is used for determining the configuration information of the palm in the virtual hand graphic pose at the previous moment as the configuration information of the palm at the current moment if the second configuration information does not exist;
the second judgment unit is used for judging whether fingers in the virtual hand model exist in the collision area or not;
if the model determining unit exists, sequentially determining pose optimization models of three knuckles in the finger from the knuckle connected with the palm according to the collision pair of the finger;
the first determination unit of the configuration information corresponding to the finger is used for sequentially determining the configuration information of the virtual hand graphic pose which is not penetrated by the finger and the virtual object at the current moment from a knuckle connected with a palm by using an active set method according to the pose optimization model of the finger;
and the second configuration information determining unit corresponding to the finger is used for determining the configuration information of the finger in the virtual hand graphic pose at the previous moment as the configuration information of the finger at the current moment if the configuration information does not exist.
9. A virtual hand force sensation synthesis system for wearable haptic interaction according to claim 6, further comprising:
and the physical pose configuration information acquisition module is used for acquiring the physical pose configuration information of the corresponding collision area of the virtual hand model by using the hand pose measurement device.
10. A virtual hand force sensation synthesis system for wearable haptic interaction according to claim 6, characterized in that the interaction force information determination module comprises in particular:
the configuration information difference value determining unit is used for determining the configuration information difference value of the corresponding collision area according to the configuration information of the virtual hand figure pose of each collision area and the virtual object non-penetration and the configuration information of the physical pose of the corresponding collision area of the virtual hand model;
and the interactive force information determining unit is used for determining the interactive force information of the corresponding collision area according to the configuration information difference value, the rigidity coefficient and the torsion spring coefficient of each collision area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011377646.3A CN112506339A (en) | 2020-11-30 | 2020-11-30 | Virtual hand force sense synthesis method and system for wearable touch sense interaction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011377646.3A CN112506339A (en) | 2020-11-30 | 2020-11-30 | Virtual hand force sense synthesis method and system for wearable touch sense interaction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112506339A true CN112506339A (en) | 2021-03-16 |
Family
ID=74968569
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011377646.3A Pending CN112506339A (en) | 2020-11-30 | 2020-11-30 | Virtual hand force sense synthesis method and system for wearable touch sense interaction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112506339A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115019591A (en) * | 2022-08-05 | 2022-09-06 | 上海华模科技有限公司 | Operation simulation method, operation simulation device and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105955458A (en) * | 2016-04-21 | 2016-09-21 | 北京天人同达软件科技有限公司 | Method and device for generating feedback force |
| CN109147057A (en) * | 2018-08-28 | 2019-01-04 | 北京航空航天大学 | A kind of virtual hand collision checking method towards wearable haptic apparatus |
-
2020
- 2020-11-30 CN CN202011377646.3A patent/CN112506339A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105955458A (en) * | 2016-04-21 | 2016-09-21 | 北京天人同达软件科技有限公司 | Method and device for generating feedback force |
| CN109147057A (en) * | 2018-08-28 | 2019-01-04 | 北京航空航天大学 | A kind of virtual hand collision checking method towards wearable haptic apparatus |
Non-Patent Citations (2)
| Title |
|---|
| 刘贤梅等: "《基于VR/AR的智能制造技术》", 华中科技大学出版社 * |
| 郭园等: "融合多元触觉和沉浸式视觉的可移植VR软件框架"", 《系统仿真学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115019591A (en) * | 2022-08-05 | 2022-09-06 | 上海华模科技有限公司 | Operation simulation method, operation simulation device and storage medium |
| CN115019591B (en) * | 2022-08-05 | 2022-11-04 | 上海华模科技有限公司 | Operation simulation method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yang et al. | Gesture interaction in virtual reality | |
| Alimanova et al. | Gamification of hand rehabilitation process using virtual reality tools: Using leap motion for hand rehabilitation | |
| Ong et al. | Augmented assembly technologies based on 3D bare-hand interaction | |
| Lee et al. | Model-based analysis of hand posture | |
| CN109116992B (en) | Collision response system for virtual hand force feedback interaction | |
| CN108983978A (en) | virtual hand control method and device | |
| Abdel-Malek et al. | Towards understanding the workspace of human limbs | |
| CN107272908A (en) | A kind of gesture identifying device, system and gesture identification method | |
| Zubrycki et al. | Using integrated vision systems: three gears and leap motion, to control a 3-finger dexterous gripper | |
| Peña Pitarch | Virtual human hand: Grasping strategy and simulation | |
| Liu et al. | Development of a virtual maintenance system with virtual hand | |
| Li et al. | Glove-based virtual hand grasping for virtual mechanical assembly | |
| CN112506339A (en) | Virtual hand force sense synthesis method and system for wearable touch sense interaction | |
| Kwon et al. | Electromyography-based decoding of dexterous, in-hand manipulation of objects: Comparing task execution in real world and virtual reality | |
| Tong et al. | Configuration-based optimization for virtual hand haptic simulation | |
| Teleb et al. | Data glove integration with 3d virtual environments | |
| WO2021195916A1 (en) | Dynamic hand simulation method, apparatus and system | |
| Amatya et al. | Real time kinect based robotic arm manipulation with five degree of freedom | |
| Caldwell et al. | Dextrous exploration of a virtual world for improved prototyping | |
| Mahdikhanlou et al. | Object manipulation and deformation using hand gestures | |
| Sofronia et al. | Haptic devices in engineering and medicine | |
| WO2023197485A1 (en) | Contact processing method and system for virtual hand force sense interaction | |
| JP5083992B1 (en) | Gripping posture generation apparatus, holding posture generation method, and holding posture generation program | |
| Kavakli | Gesture recognition in virtual reality | |
| Fuchs et al. | Holistic animation and functional solution for digital manikins in context of virtual assembly simulations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |