WO2023197485A1 - 一种用于虚拟手力觉交互的接触处理方法及系统 - Google Patents
一种用于虚拟手力觉交互的接触处理方法及系统 Download PDFInfo
- Publication number
- WO2023197485A1 WO2023197485A1 PCT/CN2022/111749 CN2022111749W WO2023197485A1 WO 2023197485 A1 WO2023197485 A1 WO 2023197485A1 CN 2022111749 W CN2022111749 W CN 2022111749W WO 2023197485 A1 WO2023197485 A1 WO 2023197485A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- hand
- virtual
- contact
- graphic
- ball
- Prior art date
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 93
- 238000003672 processing method Methods 0.000 title claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 51
- 238000004088 simulation Methods 0.000 claims abstract description 11
- 238000001514 detection method Methods 0.000 claims description 39
- 210000001145 finger joint Anatomy 0.000 claims description 33
- 238000000034 method Methods 0.000 claims description 27
- 230000004044 response Effects 0.000 claims description 26
- 238000004364 calculation method Methods 0.000 claims description 21
- 238000005457 optimization Methods 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 16
- 101100291369 Mus musculus Mip gene Proteins 0.000 claims description 14
- 101150116466 PALM gene Proteins 0.000 claims description 14
- 210000000811 metacarpophalangeal joint Anatomy 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 11
- 230000009466 transformation Effects 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 abstract description 8
- 230000000149 penetrating effect Effects 0.000 abstract description 7
- 238000012800 visualization Methods 0.000 abstract description 5
- 210000003811 finger Anatomy 0.000 description 72
- 238000004422 calculation algorithm Methods 0.000 description 16
- 230000002452 interceptive effect Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 238000010276 construction Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 210000003813 thumb Anatomy 0.000 description 5
- 238000013519 translation Methods 0.000 description 5
- 210000000236 metacarpal bone Anatomy 0.000 description 4
- 238000013507 mapping Methods 0.000 description 3
- 230000035515 penetration Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 210000004247 hand Anatomy 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 239000002775 capsule Substances 0.000 description 1
- 239000000306 component Substances 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 210000002478 hand joint Anatomy 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011017 operating method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
- 238000001308 synthesis method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
Definitions
- the present invention relates to the technical field of tactile force interaction, and in particular to a contact processing method and system for virtual hand force interaction.
- the contact processing method for virtual hand force-sense interaction mainly needs to solve the following two key problems: (1) quickly and accurately determine the contact state between the virtual hand and the virtual object; (2) quickly and accurately calculate the virtual hand motion without penetrating the virtual object. Hand incarnation and contact force.
- the solution to the above two problems requires a virtual hand model to describe the three-dimensional shape of the virtual hand, the degree of freedom of movement, and the geometric constraints of the palm and multiple fingers to meet the needs of motion mapping, collision detection, collision response, contact force calculation, and Graphics rendering needs.
- virtual hand geometry modeling simplified models (such as capsule models representing fingers or sampling some points/areas on the surface of the virtual hand model) are usually used to cope with the demand for high update frequency in force-sense interactions. These simplified models can also be combined with other geometric models. The models are combined to represent the virtual hand.
- the purpose of the present invention is to provide a contact processing method and system for virtual hand force sense interaction, using a truncated cone hinge model to simulate a real hand, ensuring the visualization of the virtual hand is realistic, and utilizing the contact constraints between the sphere and the truncated cone to ensure that the virtual fingers There is no virtual object embedded in the side of the hand, thus effectively solving the problem of inconsistent visual and tactile feedback caused by the large gap between the simplified virtual hand model and the real hand in the existing technology.
- the graphic hand and the tactile hand are difficult to match during the interaction process. .
- using a circular cone to simulate the knuckles of a real hand reduces the number of contact constraints and enables parallel processing, thus effectively solving the problem of contact processing efficiency in the existing technology.
- the present invention provides the following solutions:
- a contact processing method for virtual hand force interaction includes:
- a circular cone hinge model is used to simulate a real hand to obtain a virtual hand;
- the circular cone hinge model includes several circular cones, each of which simulates each knuckle of the real hand; the metacarpophalangeal joint and the joint between two knuckles are Sphere simulation;
- the virtual hand also includes several hemispheres, each hemisphere is located on a round table simulating the end knuckles, and the hemispheres are used to simulate the fingertips of the real hand;
- the ball tree model to simulate objects to obtain virtual objects;
- the virtual objects include several numbered spheres;
- a tactile hand Using the posture of the real hand to drive the virtual hand is called a tactile hand;
- the collision ball information that is in contact with the side of the graphic hand finger is obtained;
- the collision ball information includes the number of the collision ball , the coordinates of the center of the collision ball and the radius of the sphere of the collision ball;
- the collision ball is the sphere in the virtual object that comes into contact with the graphic hand;
- the side of the finger of the graphic hand is the side of the truncated cone knuckle of the finger;
- the graphic hand posture information is used for graphic display;
- N is the number of spheres on the virtual object that are in contact with the fingers of the graphic hand; according to the non-embedded contact constraints between the circular cone and the sphere, for each finger of the graphic hand, its three joint angles and As a whole, it is optimized under the non-embedded contact constraint set, and the graphical hand finger joint angle information without embedded virtual objects is obtained;
- the graphic hand posture is the graphic hand posture at the previous moment
- the graphical hand obtained through the above changes is used to determine the contact between the graphical hand and the virtual object, and the collision ball obtained according to the ball-cone contact judgment condition and ball-ball contact judgment condition is used to construct the non-embedded contact constraint set.
- each distal phalanx of a finger when constructing the non-embedded contact constraints At the same time, it also includes:
- the method further includes:
- a contact processing system for virtual hand force interaction includes:
- a circular cone hinge model is used to simulate a real hand to obtain a virtual hand;
- the virtual hand includes several circular cones, each of which simulates each knuckle of the real hand;
- the metacarpophalangeal joint and the joint between the two knuckles are spheres Simulation;
- the virtual hand also includes several hemispheres, each hemisphere is located on a circular table simulating the distal knuckles, and each hemisphere simulates each fingertip of the real hand;
- the ball tree model to simulate objects to obtain virtual objects;
- the virtual objects include several numbered spheres;
- a tactile hand Using the posture of the real hand to drive the virtual hand is called a tactile hand;
- Ball-cone collision detection module used for:
- the collision ball information that is in contact with the side of the graphic hand finger is obtained;
- the collision ball information includes the number of the collision ball , the coordinates of the center of the collision ball and the radius of the sphere of the collision ball;
- the collision ball is the sphere in the virtual object that comes into contact with the graphic hand;
- the side of the finger of the graphic hand is the side of the truncated cone knuckle of the finger;
- the graphic hand posture information is used for graphic display;
- Sphere-cone collision response module used for:
- N is the number of spheres on the virtual object that are in contact with the virtual fingers of the graphic hand; according to the non-embedded contact constraints between the circular cone and the sphere, for each finger of the graphic hand, its three joint angles and As a whole, it is optimized under the non-embedded contact constraint set, and the graphical hand finger joint angle information without embedded virtual objects is obtained;
- the ball-cone collision detection module also includes the following features:
- the graphic hand posture is the graphic hand posture at the previous moment
- the graphical hand obtained through the above changes is used to determine the contact between the graphical hand and the virtual object, and the collision ball obtained according to the ball-cone contact judgment condition and ball-ball contact judgment condition is used to construct the non-embedded contact constraint set.
- an endpoint contact processing module used for:
- a non-embedded constraint between the sphere and the hemisphere is constructed: Add the contact constraint to the non-embedded contact constraint set for optimization of graphic finger joint angles;
- the method further includes:
- the palm pose information of the graphic hand is first optimized, and the finger pose information of the graphic hand is obtained through matrix transformation, and then the finger pose information of the graphic hand and the pose of the virtual object are Information, the ball-cone collision detection module, ball-cone collision response module, and endpoint contact processing module are used to optimize and solve the graphical hand finger joint angle information without embedded virtual objects.
- the present invention discloses the following technical effects:
- the present invention provides a contact processing method and system for virtual hand force sense interaction.
- the fingers are formed into a circular cone. express.
- the ball tree model is used to construct virtual objects, and a ball-cone contact processing system is proposed for force-sense interaction between virtual hands and virtual objects.
- This system achieves stable, non-penetrating and more realistic virtual hand force-sense interaction. While simulating, it can achieve higher (higher than 1kHz) contact processing efficiency.
- the circular cone hinge model is used to simulate the real hand, which ensures the visualization of the virtual hand is realistic.
- the small amount of contact constraints between the sphere and the circular cone is used to reduce the computational complexity while ensuring that the sides of the virtual fingers are not embedded with virtual objects, and can be processed in parallel. Improve computing efficiency, thereby effectively solving the problem of inconsistent visual and tactile feedback caused by the large gap between the simplified virtual hand model and the real hand in the existing technology. The problem of low efficiency.
- FIG. 1 is a flow chart of a contact processing method for virtual hand force sense interaction provided by Embodiment 1 of the present invention.
- FIG. 2 is a schematic diagram of a contact processing system for virtual hand force sense interaction provided in Embodiment 2 of the present invention.
- Figure 3 is a hybrid virtual hand model based on a truncated cone hinge and different levels of ball trees provided in Embodiment 2 of the present invention.
- FIG. 4 is a schematic diagram of the contact state of the ball and the circular cone for discrete collision detection of the ball and the circular cone according to the second embodiment of the present invention.
- Figure 5 shows that the contact constraints between the ball tree model and the truncated cone model provided in Embodiment 2 of the present invention have endpoint optimization anomaly problems.
- FIG. 6 is a schematic diagram of the contact state between the object ball and the fingertip hemisphere (including the knuckle cone) used to solve the end point optimization problem provided in Embodiment 2 of the present invention.
- FIG. 7 is a schematic diagram of the finger-object interactive contact processing process based on contact constraint prediction provided in Embodiment 2 of the present invention.
- FIG. 8 is a schematic diagram of the contact force model (partial contact) between a virtual finger and a virtual object provided in Embodiment 2 of the present invention.
- Figure 9 is a flow chart of a virtual hand force sense interaction system provided by Embodiment 2 of the present invention.
- the purpose of the present invention is to provide a contact processing method and system for virtual hand force sense interaction, using a circular cone hinge model to simulate a real hand, ensuring the fidelity of virtual hand visualization, and utilizing contact constraints between a sphere and a circular cone to ensure virtual
- the sides of the fingers are not embedded with virtual objects, thus effectively solving the problem of difficulty in matching the graphic hand and the tactile hand during the interaction caused by the large difference between the simplified virtual hand model and the real hand in the existing technology, resulting in inconsistent visual and tactile feedback. question.
- the purpose of the present invention is to overcome the shortcomings of the existing technology and provide a contact processing method and system for virtual hand force sense interaction, with the goal of achieving efficient and non-penetrating contact processing, starting from the virtual hand structure and taking into account human
- the joints of the fingers are different in thickness, so the fingers are represented by a circular cone.
- the ball tree model is used to construct virtual objects, and a ball-cone contact processing system is proposed for force-sense interaction between virtual hands and virtual objects. This system achieves stable, non-penetrating and more realistic virtual hand force-sense interaction. While simulating, it can achieve high (up to 1kHz) contact processing efficiency.
- the incarnation of the user's hand in the virtual environment is called a haptic hand, and its posture is the position and posture of the user's real hand in the virtual world (OpenGL or existing engines, such as Unity, Unreal, etc. . Direct mapping in C++, c#, etc.).
- the visual incarnation of the tactile hand is defined as the graphic hand.
- the technical solution of the present invention is: a contact processing system for virtual hand force sense interaction.
- the system consists of a virtual hand building module, a ball-cone collision detection module, a ball-cone collision response module, an endpoint contact processing module and a contact force Computing module composition.
- the virtual hand building module establishes a truncated cone hinge virtual hand model based on the specific parameters of the fingers (including the length of each knuckle of each finger and the cross-sectional diameter of both ends of each knuckle);
- the ball-cone collision detection module is used to determine whether the virtual finger collides with the virtual object and returns collision information
- the ball-cone collision response module is used to quickly and accurately calculate the reasonable posture of the graphic hand to ensure that the graphic hand is not embedded in the virtual object;
- the endpoint contact processing module is used to deal with the problem of hanging or embedded fingertips that may occur when virtual fingertips come into contact with virtual objects;
- the contact force calculation module calculates the contact force between the virtual hand and the virtual object based on the collision information obtained by the collision detection module and the optimization results of the collision response module. The following will start from the construction of the virtual hand model and introduce the above modules in detail.
- the virtual hand force sense interactive contact processing system proposed by the present invention is suitable for different virtual hand interaction applications, such as virtual hands grabbing objects with one hand, operating objects with both hands, multi-person collaborative operation of objects, etc.
- the method proposed by the present invention supports Multi-object, multi-hand operation, multi-person collaboration.
- the focus of the present invention is on the modeling and contact processing of virtual hand fingers.
- the virtual hand palm can be modeled in a variety of ways, such as a ball tree model and a triangular grid model.
- the virtual hand palm is constructed using a ball tree model.
- the contact force calculation module proposed by the present invention can calculate the positive pressure at all contact points during the interaction between the virtual object and the virtual hand.
- the contact force calculation module can be calculated according to the force feedback forms that different force feedback devices can provide (such as fingertips).
- Sharp force feedback designs different feedback force calculation models, so that users can experience different feedback force modes through different force feedback devices.
- this embodiment provides a contact processing method for virtual hand force interaction, which method includes:
- the circular cone hinge model to simulate a real hand to obtain a virtual hand;
- the circular cone hinge model includes several circular cones, and each circular cone simulates each knuckle of the real hand;
- the virtual hand also includes several hemispheres, Each hemisphere is positioned on a circular table simulating a distal knuckle, said hemisphere being used to simulate a fingertip of said real hand;
- the tactile hand is displayed in the virtual scene to obtain a graphic hand, and the virtual object is displayed in the virtual scene; in the constrained space, if The posture of the tactile hand is directly used in the graphical hand, and the graphical hand will be embedded in the virtual object.
- the graphical hand maintained on the surface of the virtual object is obtained through the following contact processing steps.
- the collision ball information that is in contact with the side of the fingers of the graphic hand according to the graphic hand pose information, the virtual object pose information, and the contact judgment condition between the sphere and the circular cone;
- the collision ball information includes the collision ball number, the center coordinate of the collision ball and the sphere radius of the collision ball;
- the collision ball is the sphere in the virtual object that comes into contact with the graphic hand;
- the side of the finger of the graphic hand is the side of the truncated cone knuckle of the finger Edge;
- the graphic hand posture information is used for graphic display;
- N is the number of spheres on the virtual object that are in contact with the fingers of the graphic hand; according to the non-embedded contact constraints between the circular cone and the sphere, for each finger of the graphic hand, its three joint angles and As a whole, it is optimized under the non-embedded contact constraint set, and the graphical hand finger joint angle information without embedded virtual objects is obtained;
- the graphic hand posture is the graphic hand posture at the previous moment
- the graphical hand obtained through the above changes is used to determine the contact between the graphical hand and the virtual object, and the collision ball obtained according to the ball-cone contact judgment condition and ball-ball contact judgment condition is used to construct the non-embedded contact constraint set.
- the above provides the preferred implementation mode of the present application.
- the present application can be implemented in many different forms, such as the construction of virtual hand models, virtual hand posture optimization methods, and interaction force calculation methods, etc., and is not limited to what is described herein. implementation.
- this embodiment provides a contact processing system for virtual hand force interaction.
- the system includes:
- a circular cone hinge model is used to simulate a real hand to obtain a virtual hand;
- the virtual hand includes several circular cones, each of which simulates each knuckle of the real hand; the metacarpophalangeal joint and the joint between the two knuckles are spheres Simulation;
- the virtual hand also includes several hemispheres, each hemisphere is located on a circular table simulating the distal knuckles, and each hemisphere simulates each fingertip of the real hand.
- the ball tree model is used to simulate objects to obtain virtual objects; the virtual objects include several numbered spheres.
- a tactile hand Using the posture of the real hand to drive the virtual hand is called a tactile hand;
- the virtual hand construction module uses a circular cone hinge model to construct virtual fingers, that is, a circular cone (upper and lower cross-sectional diameters are different) is used to represent each joint of the virtual finger, and a sphere is used to connect two adjacent knuckles to represent the joints.
- the tip is represented by a hemisphere.
- other models such as the ball-tree model
- the present invention focuses on describing the construction model of the fingers of the virtual hand.
- the triangular mesh model, the ball tree model, etc. can be used to construct the palm of the virtual hand, which is within the protection scope of the present invention.
- the use of ball-tree models to construct virtual tools and virtual objects is effective in improving the computational efficiency and non-penetrating simulation of the tool-based six-degree-of-freedom force synthesis method.
- an octagonal tree to construct a virtual hand
- the knuckles of a real human hand can be approximated as a circular cone, and a circular cone hinge model can be used to represent the fingers of a virtual hand.
- a circular cone represents one knuckle of a finger.
- the number of circular cones used for virtual hand modeling is less than the number of spheres required for the ball tree model.
- using the circular cone hinge model to represent the fingers is more conducive to efficient execution of virtual hand force sense. Interactive contact processing.
- each virtual knuckle constructed with a circular cone there are three key parameters, namely the length of the knuckle and the diameter of the upper and lower end surfaces of the knuckle.
- the above three parameters can be obtained by measuring the length of the knuckles of the user's real hand and the diameter of the cross-section at each joint.
- a sphere was used to represent the joint between adjacent knuckles, and a hemisphere was connected to a frustum of the distal knuckle to represent the fingertip.
- the human palm is a complex object composed of five metacarpal bones.
- the metacarpal bone of the thumb has rotational freedom and the remaining four metacarpal bones have very little movement.
- the metacarpal area of the thumb is treated as a phalanx and represented using a frustum model like the other phalanges. Therefore, each finger of the virtual hand is modeled using a hinge model consisting of three circular cones. Considering that the range of motion of the other four metacarpal bones is small, the palm except the thumb metacarpal area can be used as a rigid body, and the rigid body can further be represented by an octagonal tree model.
- Figure 3 shows a virtual hand model based on the truncated cone hinge, in which the palm adopts octagonal tree models of different levels.
- Sphere-cone collision response module used for:
- the finger posture information of the graphic hand and the collision ball information are obtained according to the contact constraints between the sphere and the circular cone; the collision ball information includes the number of the collision ball, the coordinates of the center of the sphere and the radius of the sphere; the collision ball is the same as The fingers of the graphic hand come into contact with the sphere.
- the collision ball information that is in contact with the side of the graphic hand finger is obtained;
- the collision ball information includes the number of the collision ball , the coordinates of the center of the collision ball and the radius of the sphere of the collision ball;
- the collision ball is the sphere in the virtual object that comes into contact with the graphic hand;
- the side of the finger of the graphic hand is the side of the truncated cone knuckle of the finger;
- the graphic hand posture information is used for graphic display.
- the ball-cone collision detection module uses a ball-cone collision detection algorithm to determine whether the virtual finger collides with the virtual object on the basis that the virtual finger is represented by a cone and the virtual object is represented by a ball tree model. This algorithm is fully Considering the different radii of each section of the circular cone virtual finger, the contact state of the virtual finger at different positions with the object can be accurately determined.
- the purpose of collision detection is to determine whether the virtual hand avatar comes into contact with a virtual object in the virtual scene at a certain moment, and in what manner.
- collision detection algorithms for virtual hand interaction need to be computationally efficient.
- This embodiment proposes a ball-cone collision detection algorithm to determine the contact state between the virtual finger and the virtual object.
- the circle is the sphere on the virtual object, and the trapezoid represents the circular cone that simulates the finger.
- Point C is the vertical foot from the center O of the sphere to the central axis l of the circular cone.
- D and E represent the centers of the two circular bases of the truncated cone, and their coordinates are (x D , y D , z D ) and (x E , y E , z E ) respectively.
- the contact state between the virtual object and the finger can be determined by comparing the current
- can be calculated with the following formula:
- ⁇ of the collision detection condition can be calculated by the following formula:
- R is the radius of the sphere
- ⁇ is the angle between the side of the circular cone and its central axis
- k represents the position of the center axis of the center of the virtual sphere O on the central axis l of the virtual finger (i.e. and ), the calculation formula is as follows:
- Sphere-cone collision response module used for:
- N is the number of spheres on the virtual object that are in contact with the virtual fingers of the graphic hand; according to the non-embedded contact constraints between the circular cone and the sphere, for each finger of the graphic hand, its three joint angles are and As a whole, it is optimized under the non-embedded contact constraint set, and the graphical hand finger joint angle information without embedded virtual objects is obtained.
- the ball-cone collision response module uses the ball-cone collision response algorithm to solve the reasonable virtual hand posture based on the collision information obtained by the ball-cone collision detection algorithm.
- This algorithm constructs a virtual hand that is suitable for different radius sections of the virtual finger.
- the ball-cone contact constraint can optimally solve the virtual finger pose without embedded virtual objects on the side.
- Collision response is one of the core components of force-sensing rendering.
- This module requires the information obtained by the collision detection module as its input, and then performs collision response related calculations.
- the contact processing of virtual hand force sense interaction relied on predefined prior information of hand and/or object contact, and performed collision response based on the proximity between the actual operation behavior and the predefined operation behavior. It performs well in the defined operations, but in actual applications, unrealistic contact (such as mutual penetration) between the virtual hand and the manipulated object will occur.
- this embodiment adopts a constraint-based method to perform contact processing during the virtual hand force sense interaction process.
- an optimization target is established and a virtual hand is established based on the collision detection results.
- the non-penetration constraints of the interaction between the hand and the virtual object are finally solved to obtain a reasonable graphic pose of the virtual hand that does not penetrate the virtual object.
- the direct mapping of the user's hand in the virtual environment is called the haptic hand, and its posture is obtained by converting the posture of the real hand (for example, HTC tracker obtains the six-degree-of-freedom position information of the hand,
- the noisy data glove obtains the bending angle of each finger knuckle) and directly maps it into the virtual world.
- the visual incarnation of the haptic hand is defined as a graphic hand for graphic display. In order to achieve realistic and natural reproduction of virtual hand interaction, it is necessary to ensure that the graphic hand does not penetrate the virtual object during the interaction.
- the virtual hand fingers adopt a cone hinge model, and it is necessary to construct ball-cone contact constraints to calculate the finger pose information (for display) of the graphic hand during the interaction with the virtual object, where and represent the metacarpophalangeal joint angle, proximal phalangeal joint angle, and distal phalangeal joint angle of each finger respectively.
- these three joint angles are optimized as a whole.
- the optimization goal can be written as:
- G r is a 3 ⁇ 3 diagonal torsional stiffness matrix.
- P D/E refers to the coordinates of D or E in the global coordinate system
- the local coordinates of finger D or E determined by the length of the virtual finger knuckles.
- M s ( ⁇ s ) is the rotation matrix of the abduction motion of the metacarpal joint
- ⁇ s is the abduction angle.
- M s ( ⁇ s ) is the rotation matrix of the abduction motion of the metacarpal joint
- ⁇ s is the abduction angle.
- is the homogeneous translation and rotation matrix of the metacarpal joint with respect to the metacarpal joint coordinate system is the translation rotation matrix of the proximal finger joint relative to the metacarpal joint
- C j represents the jth constraint
- M is the number of balls on the virtual object that are in contact with the circular cone.
- Equation (8) the contact constraints between the ball and the cone in Equation (8) can be expressed as:
- j is the distance from the center O (x Oj , y Oj , z Oj ) of the j-th sphere that collides with the circular cone to the central axis l of the circular cone
- R j is the radius of the sphere
- k j represents the center of the sphere
- the position of the vertical foot of O(x Oj ,y Oj ,z Oj ) on the central axis l, that is Formula (8) is the unified writing method of constraints
- formula (9) is the calculation method of constraints.
- equation (9) can be expressed by the parameters to be optimized. After linearizing the contact constraints using Taylor expansion, the effective set method is used to iteratively optimize and solve to obtain the joint angle information of the five fingers of the graphic hand ( and ).
- Endpoint contact processing module for:
- This module determines the contact state between the virtual hand's fingers and the virtual object ball by determining the position of the virtual object ball on the central axis of the finger cone, and determines based on this state whether to use ball-cone constraint or ball-ball constraint when solving collision response constraints. .
- this embodiment constructs an endpoint contact processing module.
- the contact state between the virtual finger tip and the ball on the object is shown in Figure 6.
- ball-ball collision detection is used to determine the contact state between the virtual finger tip and the virtual object, that is, through Compare whether the distance between the centers of the two balls on the fingertip and the virtual object is less than the sum of the radii of the two balls.
- the specific formula is as follows:
- the state of the colliding ball is also saved.
- the contact constraint of the ball is established as a ball-ball contact constraint, Otherwise, the contact constraint of the ball is established as the ball-cone contact constraint expressed by Equation (8):
- R represents the radius of the sphere that collides with the circular cone.
- the ball-ball contact constraint means that the distance between the centers of the two balls is greater than or equal to the sum of the radii of the two balls, which is obtained according to the ball-ball collision detection condition of Equation (10). Each pair of balls is treated in the same way, and there is no need to distinguish which ball is which.
- Formula (11) is obtained based on the geometric relationship between the object ball and the fingertip hemisphere in Figure 6, as well as the object ball and the finger circular cone.
- k -R*sin ⁇ /
- is the line connecting the center of the object sphere and the bottom edge of the finger circular table perpendicular to the side of the circular table.
- the haptic hand when the haptic hand is embedded deeply into the virtual object, there will be a problem of abnormal graphics hand optimization, that is, the phenomenon of virtual fingers being embedded in the virtual object. This is mainly due to the use of haptic hand as the collision subject.
- the result of the collision detection is the object ball that is in contact with the haptic hand at the current moment.
- these balls may not be on the surface of the virtual object, resulting in the inability to establish surface contact constraints; on the other hand, when the haptic hand penetrates the virtual object, the results of the collision detection are basically invalid, resulting in the inability to establish appropriate ball-round table and ball- Ball contact constraints. This makes it impossible to optimize the appropriate graphic hand pose.
- this embodiment uses the graphic hand as the main body of collision detection to detect whether the circular cone and ball, the finger components of the graphic hand, collide with the object ball. Collision, the collision results are used for the construction of ball-cone and ball-ball contact constraints.
- the idea of contact prediction is used to alleviate the problem of missed detection.
- the black trapezoid represents the knuckle of the graphic hand at time t-1. The radius of each section of the knuckle at time t-1 is increased by ⁇ r to obtain a dotted trapezoid.
- the palm pose information of the graphic hand is first optimized, and the finger pose information of the graphic hand is obtained through matrix transformation, and then the finger pose information of the graphic hand and the virtual object are The pose information is obtained by using the ball-cone collision detection module, ball-cone collision response module, and endpoint contact processing module to optimize and solve the graphical hand and finger joint angle information that is not embedded in virtual objects.
- This module calculates the contact force information between the virtual finger and the virtual object during the interaction based on the contact information obtained by the collision detection module and the optimization results of the collision response module.
- the fingertip normal contact force can be calculated for a force feedback glove that can only provide fingertip feedback force, and is also within the scope of the present invention.
- FIG 8 shows the partial contact situation during the interaction between the virtual finger and the virtual object.
- the gray trapezoid represents the cross-section of a virtual knuckle (cone), and the black circle represents part of the sphere on the surface of the virtual object that is in contact with the virtual finger.
- the contact state between the virtual hand and the virtual object shown in Figure 8 is obtained by using a series of contact processing methods such as collision detection algorithm and collision response algorithm, and is finally solved through optimization.
- the virtual fingers are not embedded in the virtual object.
- G i is a 3 ⁇ 3 stiffness matrix.
- is the length of the line segment between the point Pi on the graphic hand and the corresponding point Pi ' on the haptic hand.
- the friction force fi between the i-th ball and the virtual finger can be calculated by the positive contact pressure calculated by the present invention. If relative friction occurs between the two, and the friction coefficient of the virtual object surface is u, then a method of calculating f i is:
- n i is the normal vector of the relative movement between the i-th ball and the virtual finger.
- the contact processing method and system for virtual hand force sense interaction can be used not only for single-finger single-knuckle interaction, but also for single-finger multi-knuckle interaction and multi-finger interaction.
- multi-finger interaction is involved, Using parallel processing to improve the efficiency of contact processing is within the scope of the present invention.
- interaction includes operations such as virtual hand touching, grabbing virtual objects, and virtual hands sliding on the surface of objects, as well as single-person two-hand operation and multi-person collaboration.
- the contact processing method and system for virtual hand force sense interaction not only supports the virtual hand to perform interactive operations with a virtual object in the virtual scene using the above interaction method, but also supports the virtual hand and a variety of different shapes of virtual objects in the virtual scene. The interaction between them is also within the protection scope of the present invention.
- the various objects here not only refer to virtual objects with regular geometric shapes such as spheres, cylinders, and cubes, but also include virtual objects containing complex geometric features (such as shapes with sharp features, concave shapes).
- the virtual object can be a rigid body or a deformable body, which are all within the protection scope of the present invention.
- the circular cone model used in the virtual hand model in this system is conducive to more realistic visualization of the virtual hand, and facilitates the execution of the collision detection algorithm and collision response algorithm between the virtual finger and the virtual object, achieving efficient and stable contact processing;
- the proposed ball-cone collision detection algorithm can accurately and efficiently determine the contact status between the side of the virtual finger and the virtual object, and can adapt to complex contact types, such as multi-point/multi-region sliding contact, concave contact, etc.;
- the proposed ball-cone collision response algorithm can accurately constrain the sides of the optimized graphic virtual finger to the surface of the virtual object, effectively prevent the virtual finger from being embedded in the virtual object, and can well adapt to complex contacts (multi-point/multiple points). Constrained optimization of areas of sliding contact, concave contact, etc.);
- the proposed endpoint contact processing method can effectively avoid the virtual fingertips from being suspended for a certain distance on the surface of the object or embedded in the surface of the virtual object during the interaction process, and well supports fingertip operations, such as fingertips sliding on the object surface and two fingertips facing each other. pinch etc;
- the proposed contact processing method for virtual hand force interaction can effectively calculate the contact force of multiple contact types (single point contact, multi-point contact, etc.) involved in virtual hand interaction, including normal contact force and tangential contact force.
- directional contact force directional contact force
- the proposed contact processing method for virtual hand force interaction can use parallel processing to accelerate the collision detection, collision response, endpoint contact processing and contact force calculation methods involved in contact processing based on the anatomical structure of the human hand. , improve the efficiency of virtual hand force interaction;
- the proposed contact processing method and system for virtual hand force sense interaction can obtain a force sense interaction update frequency of 1 kHz without using graphics card acceleration, and can be used in low-frequency virtual hand interaction application systems such as games and entertainment. cost development.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
本发明涉及一种用于虚拟手力觉交互的接触处理方法及系统，从虚拟手结构出发，考虑到人类手指各个关节粗细不同，将手指用圆台表示。在此基础上，采用球树模型构建虚拟物体，提出用于虚拟手和虚拟物体力觉交互的球-圆台接触处理系统，该系统在实现稳定、非穿透且较为真实的虚拟手力觉交互仿真的同时，能够达到较高(高达1kHz)的接触处理效率。利用圆台铰链模型模拟真实手，确保了虚拟手的可视化真实，利用球体与圆台之间的接触约束确保虚拟手指的侧边不嵌入虚拟物体，从而有效解决了现有技术中简化的虚拟手模型与真实手的外形差距较大导致的交互过程中图形手和触觉手难以匹配，而导致视触觉反馈不一致的问题。
Description
本申请要求于2022年04月13日提交中国专利局、申请号为202210384860.4、发明名称为“一种用于虚拟手力觉交互的接触处理方法及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
本发明涉及触力觉交互技术领域，特别是涉及一种用于虚拟手力觉交互的接触处理方法及系统。
为了模拟使用工具操作对象的交互体验，国内外研究者已开发了一系列用于虚拟工具力觉交互的接触处理方法(3-DOF(Degree of Freedom)/6-DOF)与系统，推动了桌面式力觉交互装置的广泛应用，例如将PhantomPremium用于牙科手术模拟器。日常生活中，人们通常直接用手触摸和操作物体。为了模拟这种直观的交互方式，以增强与虚拟世界交互的沉浸感(例如虚拟装配、虚拟手术、高危工作环境模拟、职业技能培训、娱乐、游戏等)，除了需要研发用于虚拟手交互的力反馈设备，例如岱仕科技的Dexmo力反馈手套，还需要用于虚拟手力觉交互的高保真接触处理方法及系统作支撑。
与虚拟工具力觉交互的接触处理方法相比，虚拟手力觉交互的接触处理方法的实现更具挑战，这主要是由于虚拟手力觉交互涉及的操作自由度较多、操作方式多样化、接触点/区域较多，使得用于虚拟手力觉交互的接触处理方法所涉及的计算更加复杂、耗时。用于虚拟手力觉交互的接触处理方法主要需要解决以下两个关键问题：(1)快速准确地确定虚拟手与虚拟物体的接触状态；(2)快速准确地计算不穿透虚拟物体的虚拟手化身及接触力。
以上两个问题的解决均需要涉及虚拟手模型，用来描述虚拟手的三维形状、运动自由度以及手掌和多个手指的几何约束，以满足运动映射、碰撞检测、碰撞响应、接触力计算以及图形渲染的需求。在虚拟手几何建模中，通常采用简化模型(如胶囊模型表示手指或者在虚拟手模型表面采样 部分点/区域)来应对力觉交互对高更新频率的需求，这些简化模型也可以与其他几何模型相结合来表示虚拟手。
虽然通过现有方法可以在一定程度上提高虚拟手力觉交互的接触处理性能，但由于简化的虚拟手模型与真实手的外形差距较大，在虚拟手力觉交互的过程中，会出现图形手和触觉手难以匹配而导致视触觉反馈不一致的问题，例如当图形手嵌入虚拟物体时未能执行有效的接触处理，用户会无法感受到力反馈。此外，受限于接触约束模型的复杂度，现有接触处理方法仅能实现数十赫兹的更新频率，无法满足力觉交互对高更新频率(例如1kHz)的需求。
因此，本领域亟需一种在确保虚拟手可视化真实的情况下，实现快速稳定力反馈的同时确保虚拟手与虚拟物体交互的非穿透模拟技术方案。
发明内容
本发明的目的是提供一种用于虚拟手力觉交互的接触处理方法及系统，利用圆台铰链模型模拟真实手，确保了虚拟手的可视化真实，利用球体与圆台之间的接触约束确保虚拟手指的侧边不嵌入虚拟物体，从而有效解决了现有技术中简化的虚拟手模型与真实手的外形差距较大导致的交互过程中图形手和触觉手难以匹配，而导致视触觉反馈不一致的问题。此外，用圆台模拟真实手的指节，减少了接触约束的数量，可采用并行处理方式，从而有效解决了现有技术接触处理效率地的问题。
为实现上述目的，本发明提供了如下方案：
一种用于虚拟手力觉交互的接触处理方法，所述方法包括：
利用圆台铰链模型模拟真实手，得到虚拟手；所述圆台铰链模型包括若干个圆台，每个所述圆台模拟所述真实手的每一个指节；掌指关节及两个指节间的关节采用球体模拟；所述虚拟手还包括若干个半球，每个半球位于模拟末端指节的圆台上，所述半球用于模拟所述真实手的指尖；
利用球树模型模拟物体，得到虚拟物体；所述虚拟物体包括若干个带有编号的球体；
获取用户真实手的姿态；
利用所述真实手的姿态驱动虚拟手，称为触觉手；
将所述触觉手在虚拟场景中显示，得到图形手，并将所述虚拟物体在所述虚拟场景中显示；
构建所述球体与所述圆台之间的接触判断条件：
和
其中，|OC|
j为虚拟物体上编号为j的球体的球心到图形手中圆台的中心轴线l的距离，R
j为虚拟物体上编号为j的球体的半径，M为虚拟物体上有待检测的球体个数，
C
j表示虚拟物体上编号为j的球体的球心在中心轴线l上的垂足，E和D分别表示图形手中圆台上下底面的圆心，α表示图形手中圆台侧边与其中心轴线的夹角，r
2和r
1分别表示图形手中圆台上下底面的半径；
根据图形手位姿信息、虚拟物体位姿信息，以及所述球体与圆台之间的接触判断条件得到与所述图形手手指侧边接触的碰撞球信息；所述碰撞球信息包括碰撞球的编号、碰撞球的球心坐标和碰撞球的球体半径；所述碰撞球是所述虚拟物体中与所述图形手发生接触的球体；所述图形手手指侧边是手指圆台指节的侧边；所述图形手位姿信息用于图形显示；
利用
和
分别代表图形手每个手指的掌指关节角、近指关节角和远指关节角，根据所述图形手位姿信息，以及图形手与虚拟物体接触的碰撞球信息，构建图形手手指的侧边与虚拟物体的非嵌入接触约束
其中N为虚拟物体上与图形手手指接触的球体个数；根据所述圆台与球体的非嵌入接触约束，对于图形手的每个手指，将其三个关节角
和
作为一个整体在非嵌入接触约束集下进行优化，求解得到不嵌入虚拟物体的图形手手指关节角信息；
根据所述图形手手指关节角信息和公式F
i＝G
i·n·|P
ig-P
ih|计算第i个碰撞球与对应的虚拟手指接触的正压力F
i；其中，G
i为3×3刚度矩阵，n表示
的单位向量，O
i表示第i个碰撞球的球心，P
i表示第i个碰撞球与虚拟手指侧边的接触点，|P
ig-P
ih|为图形手上点P
i与触觉手上相应位置点P
i'之间的线段长度；
在一些实施例中，在图形手与虚拟物体的接触判断时：
所述图形手位姿是前一时刻的图形手位姿；
所述图形手的每一个指节圆台截面半径均增加Δr；
所述图形手的每一个指节半球半径均增加Δr；
通过以上改变得到的图形手用于图形手与虚拟物体的接触判断，根据所述球-圆台接触判断条件和球-球接触判断条件得到的碰撞球用于非嵌入接触约束集的构建。
在一些实施例中，对于每个手指的远端指节，在构建所述非嵌入接触约束
的同时，还包括：
如果碰撞球与模拟指尖的球体有接触，则构建球体与半球之间的非嵌入约束：
将该接触约束添加到所述非嵌入接触约束集中用于图形手指节关节角的优化。
在一些实施例中，在所述根据公式F
i＝G
i·n·|P
ig-P
ih|计算第i个碰撞球与对应的虚拟指节接触的正压力F
i之后，还包括：
利用公式f
i＝u·n
i·|F
i|计算虚拟手滑动过程中的摩擦力；其中，f
i表示在第i个碰撞球处的摩擦力，u表示动摩擦系数，n
i为第i个碰撞球与虚拟手指之间相对运动的法向量。
一种用于虚拟手力觉交互的接触处理系统，所述系统包括：
虚拟手构建模块，用于：
利用圆台铰链模型模拟真实手，得到虚拟手；所述虚拟手包括若干个圆台，每个所述圆台模拟所述真实手的每一个指节；掌指关节及两个指节间的关节采用球体模拟；所述虚拟手还包括若干个半球，每个半球位于模拟远端指节的圆台上，每个半球模拟所述真实手的每一个指尖；
利用球树模型模拟物体，得到虚拟物体；所述虚拟物体包括若干个带有编号的球体；
获取用户真实手的姿态；
利用所述真实手的姿态驱动虚拟手，称为触觉手；
将所述触觉手在虚拟场景中显示，得到图形手，并将所述虚拟物体在所述虚拟场景中显示；
球-圆台碰撞检测模块，用于：
构建所述球体与所述圆台之间的接触判断条件：
和
其中，|OC|
j为虚拟物体上编号为j的球体的球心到图形手中圆台的中心轴线l的距离，R
j为虚拟物体上编号为j的球体的半径，M为虚拟物体上有待检测的球体个数，
C
j表示虚拟物体上编号为j的球体的球心在中心轴线l上的垂足，E和D分别表示图形手中圆台上下底面的圆心，α表示图形手中圆台侧边与其中心轴线的夹角，r
2和r
1分别表示图形手中圆台上下底面的半径；
根据图形手位姿信息、虚拟物体位姿信息，以及所述球体与圆台之间的接触判断条件得到与所述图形手手指侧边接触的碰撞球信息；所述碰撞球信息包括碰撞球的编号、碰撞球的球心坐标和碰撞球的球体半径；所述碰撞球是所述虚拟物体中与所述图形手发生接触的球体；所述图形手手指侧边是手指圆台指节的侧边；所述图形手位姿信息用于图形显示；
球-圆台碰撞响应模块，用于：
利用
和
分别代表图形手每个手指的掌指关节角、近指关节角和远指关节角，根据所述图形手位姿信息，以及图形手与虚拟物体接触的碰撞球信息，构建图形手手指的侧边与虚拟物体的非嵌入接触约束
其中N为虚拟物体上与图形手虚拟手指接触的球体个数；根据所述圆台与球体的非嵌入接触约束，对于图形手的每个手指，将其三个关节角
和
作为一个整体在非嵌入接触约束集下进行优化，求解得到不嵌入虚拟物体的图形手手指关节角信息；
接触力计算模块，用于：
根据所述图形手手指关节角信息和公式F
i＝G
i·n·|P
ig-P
ih|计算第i个碰撞球与对应的虚拟手指接触的正压力F
i；其中，G
i为3×3刚度矩阵，n表示
的单位向量，O
i表示第i个碰撞球的球心，P
i表示第i个碰撞球与虚拟手指侧边的接触点，|P
ig-P
ih|为图形手上点P
i与触觉手上相应位置点P
i'之间的线段长度；
在一些实施例中，在图形手与虚拟物体的接触判断时：
球-圆台碰撞检测模块，还包括以下特点：
所述图形手位姿是前一时刻的图形手位姿；
所述图形手的每一个指节圆台截面半径均增加Δr；
所述图形手的每一个指节半球半径均增加Δr；
通过以上改变得到的图形手用于图形手与虚拟物体的接触判断，根据所述球-圆台接触判断条件和球-球接触判断条件得到的碰撞球用于非嵌入接触约束集的构建。
在一些实施例中，还包括：端点接触处理模块，用于：
对于每个手指的远端指节，在构建所述非嵌入接触约束
的同时；
若碰撞球与模拟指尖的球体有接触，则构建球体与半球之间的非嵌入约束：
将该接触约束添加到所述非嵌入接触约束集中用于图形手指关节角的优化；
若碰撞球与模拟指尖的球体没有接触，则仅构建球体与圆台之间的接触约束。
在一些实施例中，在所述根据公式F
i＝G
i·n·|P
ig-P
ih|计算第i个碰撞球与对应的虚拟指节接触的正压力F
i之后，还包括：
利用公式f
i＝u·n
i·|F
i|计算虚拟手滑动过程中的摩擦力；其中，f
i表示在第i个碰撞球处的摩擦力，u表示动摩擦系数，n
i为第i个碰撞球与虚拟手指之间相对运动的法向量。
在一些实施例中，当涉及手掌交互时，先优化图形手的手掌位姿信息，通过矩阵变换得到图形手的手指位姿信息后，再根据图形手的手指位姿信息、虚拟物体的位姿信息，采用所述球-圆台碰撞检测模块、球-圆台碰撞响应模块、端点接触处理模块优化求解不嵌入虚拟物体的图形手手指关节角信息。
在一些实施例中，当处理多手指交互时，采用并行处理方式。
根据本发明提供的具体实施例，本发明公开了以下技术效果：
本发明提供了一种用于虚拟手力觉交互的接触处理方法及系统，以实现准确、高效的接触处理为目标，从虚拟手结构出发，考虑到人类手指各个关节粗细不同，将手指用圆台表示。在此基础上，采用球树模型构建虚 拟物体，提出用于虚拟手和虚拟物体力觉交互的球-圆台接触处理系统，该系统在实现稳定、非穿透且较为真实的虚拟手力觉交互仿真的同时，能够达到较高(高于1kHz)的接触处理效率。利用圆台铰链模型模拟真实手，确保了虚拟手的可视化真实，利用球体与圆台之间的少量接触约束在降低计算复杂度的同时确保虚拟手指的侧边不嵌入虚拟物体，并可通过并行处理方式提高计算效率，从而有效解决了现有技术中简化的虚拟手模型与真实手的外形差距较大导致的交互过程中图形手和触觉手难以匹配，而导致视触觉反馈不一致的问题，以及接触处理效率低的问题。
说明书附图
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例一提供的用于虚拟手力觉交互的接触处理方法的流程图。
图2为本发明实施例二提供的用于虚拟手力觉交互的接触处理系统的示意图。
图3为本发明实施例二提供的基于圆台铰链和不同层级球树的一种混合虚拟手模型。
图4为本发明实施例二提供的用于球-圆台离散碰撞检测的球-圆台接触状态示意图。
图5为本发明实施例二提供的球树模型与圆台模型接触约束存在端点优化异常问题。
图6为本发明实施例二提供的用于解决端点优化问题的物体球-指尖半球(包含指节圆台)接触状态示意图。
图7为本发明实施例二提供的基于接触约束预测的手指-物体交互接触处理过程示意图。
图8为本发明实施例二提供的虚拟手指与虚拟物体的接触力模型(部分接触)示意图。
图9为本发明实施例二提供的虚拟手力觉交互系统流程图。
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明的目的是提供一种用于虚拟手力觉交互的接触处理方法及系统，利用圆台铰链模型模拟真实手，确保了虚拟手可视化的逼真度，利用球体与圆台之间的接触约束确保虚拟手指的侧边不嵌入虚拟物体，从而有效解决了现有技术中简化的虚拟手模型与真实手的外形差距较大导致的交互过程中图形手和触觉手难以匹配，而导致视触觉反馈不一致的问题。
本发明目的在于克服现有技术的缺陷，提供一种用于虚拟手力觉交互的接触处理方法及系统，以实现高效、非穿透的接触处理为目标，从虚拟手结构出发，考虑到人类手指各个关节粗细不同，将手指用圆台表示。在此基础上，采用球树模型构建虚拟物体，提出用于虚拟手和虚拟物体力觉交互的球-圆台接触处理系统，该系统在实现稳定、非穿透且较为真实的虚拟手力觉交互仿真的同时，能够达到较高(高达1kHz)的接触处理效率。
在虚拟手触觉模拟中，将用户的手在虚拟环境中的化身称为触觉手(haptic hand)，其姿态为用户真实手的位姿在虚拟世界(OpenGL或者现有引擎，例如Unity、Unreal等。C++、c#等)中的直接映射。在图形显示中，将触觉手的视觉化身定义为图形手(graphic hand)。
本发明的技术方案是：一种用于虚拟手力觉交互的接触处理系统，该系统由虚拟手构建模块、球-圆台碰撞检测模块、球-圆台碰撞响应模块、端点接触处理模块及接触力计算模块构成。其中，
虚拟手构建模块根据手指具体参数(包括每个手指各指节的长度和各指节两端的截面直径)建立圆台铰链虚拟手模型；
球-圆台碰撞检测模块用于确定虚拟手指是否与虚拟物体发生了碰撞，并返回碰撞信息；
球-圆台碰撞响应模块用于快速准确地计算图形手的合理位姿，以确保图形手不嵌入虚拟物体；
端点接触处理模块用于处理虚拟手指尖与虚拟物体接触可能出现的指尖悬空或者嵌入问题；
接触力计算模块根据碰撞检测模块得到的碰撞信息以及碰撞响应模块的优化结果来计算虚拟手与虚拟物体之间的接触力。以下将从虚拟手模型的构建出发，详细介绍上述模块。
值得说明以下三个问题：
(1)本发明提出的虚拟手力觉交互接触处理系统适用于不同的虚拟手交互应用，例如虚拟手单手抓取物体、双手操作物体、多人协作操作物体等，本发明提出的方法支持多物体、多手操作、多人协作。
(2)本发明重点在于虚拟手手指的建模与接触处理，虚拟手手掌可以有多种建模方式，例如球树模型、三角网格模型，本实施例虚拟手手掌采用球树模型构建。
(3)本发明提出的接触力计算模块能计算出虚拟物体与虚拟手交互过程中所有接触点处的正压力，在此基础上，可根据不同力反馈设备能够提供的力反馈形式(例如指尖力反馈)设计不同的反馈力计算模型，从而能使用户通过不同的力反馈设备体验不同的反馈力模式。
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图和具体实施方式对本发明作进一步详细的说明。
实施例一：
如图1所示，本实施例提供了一种用于虚拟手力觉交互的接触处理方法，该方法包括：
S1、利用圆台铰链模型模拟真实手，得到虚拟手；所述圆台铰链模型包括若干个圆台，每个所述圆台模拟所述真实手的每一个指节；所述虚拟手还包括若干个半球，每个半球位于模拟末端指节的圆台上，所述半球用于模拟所述真实手的指尖；
S2、利用球树模型模拟物体，得到虚拟物体；所述虚拟物体包括若干个带有编号的球体；
S3、获取用户真实手的姿态；
S4、利用所述真实手的姿态驱动虚拟手，称为触觉手；
S5、将所述触觉手在虚拟场景中显示，得到图形手，并将所述虚拟物体在所述虚拟场景中显示。
在自由空间(即虚拟手与虚拟物体未接触)中，将所述触觉手在虚拟场景中显示，得到图形手，并将所述虚拟物体在所述虚拟场景中显示；在约束空间中，如果将触觉手的位姿直接用于图形手，图形手会嵌入虚拟物体，通过以下接触处理步骤得到保持在虚拟物体表面的图形手。
S6、构建所述球体与所述圆台之间的接触判断条件：
和
其中，|OC|
j为虚拟物体上编号为j的球体的球心到图形手中圆台的中心轴线l的距离，R
j为虚拟物体上编号为j的球体的半径，M为虚拟物体上有待检测的球体个数，
C
j表示虚拟物体上编号为j的球体的球心在中心轴线l上的垂足，E和D分别表示图形手中圆台上下底面的圆心，α表示图形手中圆台侧边与其中心轴线的夹角，r
2和r
1分别表示图形手中圆台上下底面的半径；
S7、根据图形手位姿信息、虚拟物体位姿信息，以及所述球体与圆台之间的接触判断条件得到与所述图形手手指侧边接触的碰撞球信息；所述碰撞球信息包括碰撞球的编号、碰撞球的球心坐标和碰撞球的球体半径；所述碰撞球是所述虚拟物体中与所述图形手发生接触的球体；所述图形手手指侧边是手指圆台指节的侧边；所述图形手位姿信息用于图形显示；
S8、利用
和
分别代表图形手每个手指的掌指关节角、近指关节角和远指关节角，根据所述图形手位姿信息，以及图形手与虚拟物体接触的碰撞球信息，构建图形手手指的侧边(即圆台)与虚拟物体(即球体)的非嵌入接触约束
其中N为虚拟物体上与图形手手指接触的球体个数；根据所述圆台与球体的非嵌入接触约束，对于图形手的每个手指，将其三个关节角
和
作为一个整体在非嵌入接触约束集下进行优化，求解得到不嵌入虚拟物体的图形手手指关节角信息；
S9、根据所述图形手手指关节角信息和公式F
i＝G
i·n·|P
ig-P
ih|计算第i个碰撞球与对应的虚拟手指接触的正压力F
i；其中，G
i为3×3刚度矩阵，n表示
的单位向量，O
i表示第i个碰撞球的球心，P
i表示第i个碰撞球与虚拟手指侧边的接触点，|P
ig-P
ih|为图形手上点P
i与触觉手上相应位置点P
i'之间的线段长度。
在所述球-圆台接触判断条件和球-球接触判断条件中用到的图形手及其位姿，还包括以下特点：
所述图形手位姿是前一时刻的图形手位姿；
所述图形手的每一个指节圆台截面半径均增加Δr；
所述图形手的每一个指节半球半径均增加Δr；
通过以上改变得到的图形手用于图形手与虚拟物体的接触判断，根据所述球-圆台接触判断条件和球-球接触判断条件得到的碰撞球用于非嵌入接触约束集的构建。
对于每个手指的远端指节，在构建所述非嵌入接触约束
的同时，还包括：
如果碰撞球与模拟指尖的球体有接触，则构建球体与半球之间的非嵌入约束：
将该接触约束添加到所述非嵌入接触约束集中用于图形手指节关节角的优化。
在所述根据公式F
i＝G
i·n·|P
ig-P
ih|计算第i个碰撞球与对应的虚拟指节接触的正压力F
i之后，还包括：
利用摩擦计算公式，例如虚拟手指在虚拟物体表面滑动，利用公式f
i＝u·n
i·|F
i|计算虚拟手滑动过程中的摩擦力；其中，f
i表示在第i个碰撞球处的摩擦力，u表示动摩擦系数，n
i为第i个碰撞球与虚拟手指之间相对运动的法向量。
上述给出了本申请的较佳实施方式，本申请可以以许多不同的形式来实现，例如虚拟手模型的构建、虚拟手位姿优化方法以及交互力的计算方法等，并不限于本文所描述的实施方式。
实施例二：
如图2所示，本实施例提供了一种用于虚拟手力觉交互的接触处理系统，该系统包括：
虚拟手构建模块，用于：
利用圆台铰链模型模拟真实手，得到虚拟手；所述虚拟手包括若干个圆台，每个所述圆台模拟所述真实手的每一个指节；掌指关节及两个指节间的关节采用球体模拟；所述虚拟手还包括若干个半球，每个半球位于模拟远端指节的圆台上，每个半球模拟所述真实手的每一个指尖。
利用球树模型模拟物体，得到虚拟物体；所述虚拟物体包括若干个带有编号的球体。
获取用户真实手的姿态；
利用所述真实手的姿态驱动虚拟手，称为触觉手；
在自由空间中，将所述触觉手在虚拟场景中显示，得到图形手，并将所述虚拟物体在所述虚拟场景中显示；在约束空间中，如果将触觉手的位姿直接用于图形手，图形手会嵌入虚拟物体，通过以下功能模块的接触处理得到保持在虚拟物体表面的图形手。
本实施例中，虚拟手构建模块，采用圆台铰链模型构建虚拟手指，即采用圆台(上下截面直径大小不同)表示虚拟手指的各个关节，用一个球体连接两个相邻指节以表示关节，指尖采用半球表示。为了更好的模拟大拇指，可以采用其他模型(例如球树模型)构建大拇指的掌骨关节。
值得注意地是，本发明重点描述了虚拟手的手指构建模型，可以使用三角网格模型、球树模型等来构建虚拟手的手掌，均在本发明的保护范围内。
具体来讲，采用球树模型构建虚拟工具与虚拟物体，在提高tool-based的六自由度力觉合成方法的计算效率及非穿透模拟方面卓有成效。然而，采用八叉球树构建虚拟手，在实现虚拟手与虚拟物体交互过程中，随着交互/接触手指及接触数量增多，接触处理耗时也增多。人类真实手的指节可以近似成圆台，能够使用圆台铰链模型来表示虚拟手的手指，一个圆台代表手指的一个指节。直观上，用于虚拟手建模的圆台数量比球树模型所需球体的数量要少，与多层球树模型相比，使用圆台铰链模型来表示手指 更有利于高效地执行虚拟手力觉交互的接触处理。
对于每一个用圆台构建的虚拟指节而言，包含三个关键参数，即指节的长度和指节上下两个端面的直径。上述三个参数，可以通过测量用户真实手的指节长度和每个关节处截面的直径获得。此外，为了视觉显示及交互逼真性的实现，采用一个球体来代表相邻指节之间的关节，并将一个半球连接到远端指节的圆台来代表指尖。
人类的手掌是由五个掌骨组成的复杂物体。大拇指掌骨有旋转自由度，其余四个掌骨的活动非常小，将大拇指的掌骨区域视为指骨，并像其他指骨一样使用圆台模型来表示。因此，虚拟手的每一根手指都使用包括三个圆台的铰链模型来建模。考虑到其他四个掌骨的运动范围较小，可将除拇指掌骨区域外的手掌作为刚体，进一步地可用八叉球树模型表示该刚体。图3给出了基于圆台铰链的一种虚拟手模型，其中手掌采用不同层级的八叉球树模型。
球-圆台碰撞响应模块，用于：
构建所述球体与所述圆台之间的接触判断条件(此处为手指侧边与虚拟物体的接触检测条件)：
和
其中，|OC|
j为虚拟物体上编号为j的球体的球心到图形手中圆台的中心轴线l的距离，R
j为虚拟物体上编号为j的球体的半径，M为虚拟物体上有待检测的球体个数，
C
j表示虚拟物体上编号为j的球体的球心在中心轴线l上的垂足，E和D分别表示图形手中圆台上下底面的圆心，α表示图形手中圆台侧边与其中心轴线的夹角，r
2和r
1分别表示图形手中圆台上下底面的半径；
根据所述球体与圆台之间的接触约束得到所述图形手的手指位姿信息和碰撞球信息；所述碰撞球信息包括碰撞球的编号、球心坐标和球体半径；所述碰撞球是与所述图形手的手指发生接触的球体。
根据图形手位姿信息、虚拟物体位姿信息，以及所述球体与圆台之间 的接触判断条件得到与所述图形手手指侧边接触的碰撞球信息；所述碰撞球信息包括碰撞球的编号、碰撞球的球心坐标和碰撞球的球体半径；所述碰撞球是所述虚拟物体中与所述图形手发生接触的球体；所述图形手手指侧边是手指圆台指节的侧边；所述图形手位姿信息用于图形显示。
本实施例中球-圆台碰撞检测模块，在虚拟手指采用圆台表示、虚拟物体采用球树模型表示的基础上，采用球-圆台碰撞检测算法确定虚拟手指是否与虚拟物体发生了碰撞，该算法充分考虑圆台虚拟手指各个截面半径不同的特点，能准确确定虚拟手指不同位置与物体的接触状态。
具体来讲：碰撞检测的目的是判断虚拟手化身在某一时刻是否与虚拟场景中的虚拟物体发生了接触，以及接触方式是怎样的。为了支持实时的接触处理，用于虚拟手交互的碰撞检测算法需要有较高的计算效率。同时，需要准确计算虚拟手与虚拟物体接触状态的相关参数，从而为碰撞响应提供精确的接触信息。本实施例提出球-圆台碰撞检测算法确定虚拟手指与虚拟对象的接触状态。
如图4所示，圆为虚拟物体上的球体，梯形表示模拟手指的圆台。点C是从球体球心O到圆台中心轴线l的垂足。D和E表示圆台两个圆底面的中心，其坐标分别为(x
D,y
D,z
D)和(x
E,y
E,z
E)。虚拟物体和手指之间的接触状态可以通过比较当前的|OC|(即物体球的中心O到虚拟手指中心轴线l的距离)和球体与圆台侧面相切时的阈值|OC|
Γ(图4(a))来确定。如果满足|OC|≤|OC|
Γ，则可以确定圆台与球发生了碰撞，|OC|可以用下式计算:
其中l
1,l
2,l
3和l
4通过以下公式计算：
假设圆台上下底部的半径分别为r
2和r
1(r
2＜r
1)，则可通过下式计算碰撞检测条件的阈值|OC|
Γ：
其中R为球面半径，α为圆台侧边与其中心轴线的夹角，k表示虚拟球体的球心O在虚拟手指中心轴线l上的垂足C所处中心轴线的位置(即
与
的比值)，计算公式如下:
其中，l
5＝(x
O-x
E)(x
D-x
E)+(y
O-y
E)(y
D-y
E)+(z
O-z
E)(z
D-z
E)。
如图4(b)所示，虽然满足条件|OC|≤|OC|
Γ，但灰色虚线球体并未与圆台接触。更具体地说，当k＞1+R/|DE|或k＜-R/|DE|时，球体将不会与圆台接触。当k＝1+R/|DE|时，球面与圆台的下底部(延伸)相切(即图4(c)中的黑色虚线球体O
1)；当k＝-R/|DE|时，球体与圆台的上底部(延伸)相切(即图4(c)中的黑色虚线球体O
2)；当k＝0时，垂足C与端点E重合；当k＝1时，垂足C与端点D重合。总的来说，判断球体和圆台发生碰撞还需要满足另一个条件，即-R/|DE|＜k＜1+R/|DE|。因此，当满足以下条件可判定虚拟手指与虚拟物体发生了碰撞。
球-圆台碰撞响应模块，用于：
利用
和
分别代表图形手每个手指的掌指关节角、近指关节角和远指关节角，根据所述图形手位姿信息，以及图形手与虚拟物体接触的碰撞球信息，构建图形手手指的侧边(即圆台)与虚拟物体(即球体)的非嵌入接触约束
其中N为虚拟物体上与图形手虚拟手指接触的球体个数；根据所述圆台与球体的非嵌入接触约束，对于图形手的每个手指，将其三个关节角
和
作为一个整体在非嵌入接触约束集下进行优化，求解得到不嵌入虚拟物体的图形手手指关节角信息。
本实施例中球-圆台碰撞响应模块，在球-圆台碰撞检测算法所得碰撞信息的基础上，采用球-圆台碰撞响应算法求解合理的虚拟手位姿，该算法构造适应虚拟手指不同半径截面的球-圆台接触约束，能优化求解出侧边不嵌入虚拟物体的虚拟手指位姿。
具体来讲：碰撞响应是力觉渲染的核心组件之一，该模块需要碰撞检测模块所得信息作为其输入，然后执行碰撞响应相关计算。在早期，虚拟手力觉交互的接触处理依赖于预先定义的手和/或物体接触的先验信息，根据实际操作行为和预定义操作行为之间的接近程度来执行碰撞响应，该方法在预定义的操作中表现良好，但在实际应用中，会出现虚拟手和被操纵对象之间不真实的接触(如相互穿透)。为了实现逼真的虚拟手交互，本实施例采用基于约束的方法来执行虚拟手力觉交互过程中的接触处理，具体而言是针对构建的虚拟手模型，建立优化目标并根据碰撞检测结果建立虚拟手与虚拟物体交互的非穿透约束，最终求解得到不穿透虚拟物体的虚拟手合理图形位姿。
在虚拟手触觉交互仿真中，将虚拟环境中用户手的直接映射化身称为 触觉手(haptic hand)，其姿态是通过将真实手的姿态(例如，HTC tracker获取手部六自由度位置信息，诺亦腾数据手套获取各手指指节的弯曲角度)直接映射到虚拟世界中得到的。将haptic hand的视觉化身定义为图形手(graphic hand)，用于图形显示。为了虚拟手交互的逼真自然再现，需要确保交互过程中graphic hand不会穿透虚拟对象。
本实施例虚拟手手指采用圆台铰链模型，需要构造球-圆台接触约束来计算与虚拟物体交互过程中图形手的手指位姿信息(用于显示)，其中
和
分别代表每个手指的掌指关节角、近指关节角和远指关节角。对于每一个手指而言，将这三个关节角度作为一个整体进行优化，优化目标可以写成：
其中，
表示触觉手的手指关节角信息(即由硬件设备(例如数据手套)采集所得)，
和
的定义与
和
相同。G
r为3×3对角抗扭刚度矩阵。
在已知图形手的手掌位姿信息
(3DoF(自由度)平移和3oF旋转)和手指各关节角信息(
和
)后，通过以下矩阵变换得到圆台两底面圆心D和E在全局坐标系下的坐标(x
D,y
D,z
D)和(x
E,y
E,z
E)：
其中，P
D/E指D或E的在全局坐标系下的坐标，
指D或E的局部坐标，由虚拟手指指节长度确定。
为包含
中六维变量的齐次平移旋转矩阵。M
s(θ
s)为掌骨关节外展运动的旋转矩阵，θ
s为外展角度。
为掌骨关节对于掌腕关节坐标系的齐次平移旋转矩阵，
为近指关 节相对于掌骨关节的平移旋转矩阵，
为远指关节相对于近指关节的平移旋转矩阵。
因此，圆台与球面接触约束可表示为：
其中，C
j表示第j个约束条件，M为虚拟物体上与圆台接触的球的数目。
根据球-圆台碰撞检测模块中所述的球-圆台碰撞检测条件(式(5))，式(8)中球与圆台之间的接触约束可表示为:
其中，|OC|
j为与圆台碰撞的第j个球体的球心O(x
Oj,y
Oj,z
Oj)到圆台中心轴线l的距离，R
j为该球体的半径，k
j表示球心O(x
Oj,y
Oj,z
Oj)在中心轴线l上的垂足的位置，即
公式(8)是约束的统一写法，公式(9)是约束的计算方法。
对于一个特定的手指，圆台的参数r
1,r
2和α是恒定的。将
和
代入式(2)和式(4)，可以用待优化的参数表示式(9)。使用Taylor展开将接触约束线性化后，使用有效集方法迭代优化求解得到图形手的五个手指的关节角信息(
和
)。
端点接触处理模块，用于：
处理虚拟手指尖与虚拟物体接触可能出现的指尖悬空或者嵌入的问题。该模块通过确定虚拟物体球在手指圆台中心轴线上的位置，确定虚拟手手指与虚拟物体球的接触状态，并根据该状态确定碰撞响应约束优化求解时是采用球-圆台约束还是球-球约束。
具体来讲，采用球-圆柱碰撞响应算法执行虚拟手交互过程中，会出现手指指尖位姿信息优化异常的问题，如图5所示，(a)为图形手手指指尖悬空(即图形手与虚拟物体分离)的案例，(b)为图形手手指嵌入(即图形手穿透虚拟物体)的案例。
为了解决上述端点异常问题，本实施例构建了端点接触处理模块。虚拟手指指尖与物体上的球接触状态如图6所示，当连接在圆台底部的半球与物体球发生碰撞时，采用球-球碰撞检测确定虚拟手指尖与虚拟物体的接触状态，即通过比较指尖与虚拟物体上两球的球心距是否小于两球半径之和，具体如下式：
|OC|
2≤(R+r
2)
2 (10)
其中，|OC|
2＝(x
T-x
O)
2+(y
T-y
O)
2+(z
T-z
O)
2，(x
T,y
T,z
T)和(x
O,y
O,z
O)分别表示虚拟手指尖半球与虚拟物体球在全局坐标系中的球心坐标，R表示虚拟物体球的半径。
具体而言，根据球-圆台的碰撞检测条件，在保存碰撞球本身信息(包括球号、球心坐标、球半径)的同时，保存碰撞球的状态。在构建约束时，如果碰撞球的状态为其球心在圆台中心轴线上的位置满足式(11)条件下同时满足式(10)，则将该球的接触约束建立为球-球接触约束，否则将该球的接触约束建立为式(8)所表示的球-圆台接触约束：
其中，R表示与圆台碰撞的球体的半径。球-球接触约束是指两球的球心距大于等于两球半径之和，是根据式(10)球-球碰撞检测条件得到的。每对球处理方式相同，不需要区分第几个球体。
公式(11)是根据图6中物体球和指尖半球，以及物体球与手指圆台的几何关系得到的。
k＝-R*sinα/|DE|是物体球圆心与手指圆台上底面边缘的连线垂直于圆台侧边。
此外，当haptic hand嵌入虚拟物体较深时，会出现graphic hand优化异常的问题，即虚拟手指嵌入虚拟物体的现象。这主要是由于采用haptic hand作为碰撞主体，当haptic hand嵌入虚拟物体较深(甚至穿过)时，碰撞检测所得结果是当前时刻与haptic hand接触的物体球。一方面，这些球有可能不在虚拟物体表面，导致无法建立表面接触约束；另一方面，当haptic hand穿透虚拟物体时，碰撞检测所得结果基本无效，导致无法建立恰当的球-圆台及球-球接触约束。从而使得无法优化得到合适的graphic hand位姿。
通过对上述haptic hand嵌入虚拟物体较深时存在的graphic hand优化异常问题的分析，本实施例采用graphic hand作为碰撞检测的主体，检测graphic hand的手指构成元素—圆台和球是否与物体球发生了碰撞，碰撞结果用于球-圆台及球-球接触约束的构建。此外，采用接触预测的思想缓解漏检问题，如图7所示，黑色梯形代表图形手在t-1时刻的指节，将t-1时刻的指节各截面半径均增加Δr，得到虚线梯形，然后使用第2节中描述的碰撞检测算法得到圆台-球接触对，再根据公式(8)建立相交圆台和球的接触约束，最后在此约束下便，采用Taylor展开将接触约束线性化后，使用有效集方法迭代优化求解虚拟手指的无穿透关节角度，图7所示的梯形t为计算出来的虚拟手手指。
作为一种可选的实施方式，当涉及手掌交互时，先优化图形手的手掌位姿信息，通过矩阵变换得到图形手的手指位姿信息后，再根据图形手的 手指位姿信息、虚拟物体的位姿信息，采用所述球-圆台碰撞检测模块、球-圆台碰撞响应模块、端点接触处理模块优化求解不嵌入虚拟物体的图形手手指关节角信息。
接触力计算模块，用于：
计算虚拟手与虚拟物体交互过程中的接触力信息。该模块根据碰撞检测模块得到的接触信息以及碰撞响应模块的优化结果来计算交互过程中虚拟手指与虚拟物体之间的接触力信息。
值得注意地是，除了本发明提到的接触点及接触区域所涉及的接触力计算方法，根据本发明提出的碰撞检测模块得到的接触信息以及碰撞响应模块的优化结果来计算更为丰富的反馈力，例如可为仅能提供指尖反馈力的力反馈手套计算指尖法向接触力，也在本发明的保护范围内。
具体来讲，除了高效准确的碰撞检测、碰撞响应算法，虚拟手指与虚拟物体交互过程中的接触力计算是实现连续、稳定力反馈的又一关键环节。图8展示了虚拟手指与虚拟物体交互过程中的部分接触情况，其中灰色梯形表示一个虚拟指节(圆台)的截面，黑色圆表示与虚拟手指接触的虚拟物体表面上的部分球体。图8展示的虚拟手与虚拟物体的接触状态是采用碰撞检测算法及碰撞响应算法等一系列接触处理方法，最终通过优化求解所得。具体而言，在虚拟手与虚拟物体交互过程中，虚拟手指不嵌入虚拟物体。
假设虚拟物体表面有n个球与虚拟手指发生了接触，以图8为例，假设第i个球(球心为O
i)与虚拟手指侧边的接触点(即切点)为P
i，交互过程中虚拟手指受到的正压力(灰色箭头)为F
i，其方向与虚拟手指侧边垂直，即与向量
方向相同。球心O
i的全局坐标
已知，计算F
i的方向需 要先求解P
i的全局坐标
接下来介绍P
i的全局坐标
的一种计算方法。令
的延长线与l的交点为
通过式(3)可计算
进而在已知α及C
i在中轴线l上的位置表征参数k
i(式(4))的前提下，可计算出Q
i的全局坐标
及
与
的比值，从而计算出P
i的全局坐标
将其记作P
ig，即graphic hand上接触位置的坐标。采用同样的方式根据haptic hand上相应指节两端点的局部坐标可计算出与P
i相对应位置P
i'处的全局坐标，将其记作P
ih，则可通过下式计算第i个球与相应虚拟指节接触的正压力F
i：
F
i＝G
i·n·|P
ig-P
ih| (12)
其中，G
i为3×3刚度矩阵。
表示
的的单位向量。|P
ig-P
ih|为graphic hand图形手上点P
i与haptic hand上相应位置点P
i'之间的线段长度。
进一步地，如果第i个球与虚拟手指之间有相对运动的趋势或者发生了相对运动，通过本发明计算的接触正压力可计算出第i个球与虚拟手指之间的摩擦力f
i。如果两者之间发生了相对摩擦，虚拟物体表面的摩擦系数为u，则一种计算f
i的方法为：
f
i＝u·n
i·|F
i| (13)
其中，n
i为第i个球与虚拟手指之间相对运动的法向量。
以上计算所得均为虚拟物体表面上某一个球与虚拟手指接触所产生的正压力和摩擦力。考虑到所用力反馈设备能够提供的反馈力形式及在手指上的施力位置，可进一步计算出力反馈设备所需提供的反馈力。本实施例具体流程图如图9所示。
所述用于虚拟手力觉交互的接触处理方法与系统，不仅可以用于单指 单指节交互，还可以用于单指多指节交互以及多指交互，当涉及多指交互时，可采用并行处理方式来提高接触处理的效率，均在本发明的保护范围内。
值得注意地是，此处交互包括虚拟手触碰、抓取虚拟物体以及虚拟手在物体表面滑动等操作，还包括单人双手操作及多人协作。
所述用于虚拟手力觉交互的接触处理方法与系统，不仅支持虚拟手采用上述交互方式与虚拟场景中的一个虚拟物体执行交互操作，还支持虚拟手与虚拟场景中多种不同形状虚拟物体之间的交互，也在本发明的保护范围内。
值得注意地是，此处多种物体不仅指球体、圆柱体、立方体等具有规则几何形状的虚拟物体，还包括包含复杂几何特征(例如具有尖锐特征的形状、凹面形状)的虚拟物体。此外，虚拟物体可以是刚体还可以是变形体，均在在本发明的保护范围内。
本实施例提出的用于虚拟手力觉交互的接触处理系统具有以下优点：
1.该系统中虚拟手模型所采用的圆台模型有利于更加真实的虚拟手可视化，且方便执行虚拟手指与虚拟物体的碰撞检测算法和碰撞响应算法，实现高效稳定的接触处理；
2.提出的球-圆台碰撞检测算法，能够准确高效地确定虚拟手指侧边与虚拟物体的接触状态，且能适应复杂的接触类型，例如多点/多区域的滑动接触、凹面接触等；
3.提出的球-圆台碰撞响应算法，能够准确地将优化求解的图形虚拟手指侧边约束在虚拟物体的表面，有效避免虚拟手指嵌入虚拟物体，且能良好地适应复杂接触(多点/多区域的滑动接触、凹面接触等)的约束优化；
4.提出的端点接触处理方法，能够有效避免交互过程中虚拟手指尖悬空于物体表面一段距离或者嵌入虚拟物体表面，良好地支持指尖操作，例如指尖在物体表面滑动、两指指尖对捏等；
5.提出的用于虚拟手力觉交互的接触处理方法，能够有效计算虚拟手交互所涉及的多种接触类型(单点接触、多点接触等)的接触力，包括法向接触力和切向接触力；
6.提出的用于虚拟手力觉交互的接触处理方法，能够根据人手的解剖 结构特点，采用并行处理方式对接触处理涉及的碰撞检测、碰撞响应、端点接触处理及接触力计算等方法进行加速，提高虚拟手力觉交互的效率；
7.提出的用于虚拟手力觉交互的接触处理方法与系统能够在不采用图形显卡加速的情况下获得1kHz的力觉交互更新频率，能用于游戏、娱乐等虚拟手交互应用系统的低成本开发。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。
提供以上实施例仅仅是为了描述本发明的目的，而并非要限制本发明的范围。本发明的范围由所附权利要求限定。不脱离本发明的精神和原理而做出的各种等同替换和修改，均应涵盖在本发明的范围之内。
Claims (10)
- 一种用于虚拟手力觉交互的接触处理方法，其特征在于，所述方法包括：利用圆台铰链模型模拟真实手，得到虚拟手；所述圆台铰链模型包括若干个圆台，每个所述圆台模拟所述真实手的每一个指节；掌指关节及两个指节间的关节采用球体模拟；所述虚拟手还包括若干个半球，每个半球位于模拟末端指节的圆台上，所述半球用于模拟所述真实手的指尖；利用球树模型模拟物体，得到虚拟物体；所述虚拟物体包括若干个带有编号的球体；获取用户真实手的姿态；利用所述真实手的姿态驱动虚拟手，称为触觉手；将所述触觉手在虚拟场景中显示，得到图形手，并将所述虚拟物体在所述虚拟场景中显示；构建所述球体与所述圆台之间的接触判断条件：和
其中，|OC| j为虚拟物体上编号为j的球体的球心到图形手中圆台的中心轴线l的距离，R j为虚拟物体上编号为j的球体的半径，M为虚拟物体上有待检测的球体个数，
C j表示虚拟物体上编号为j的球体的球心在中心轴线l上的垂足，E和D分别表示图形手中圆台上下底面的圆心，α表示图形手中圆台侧边与其中心轴线的夹角，r 2和r 1分别表示图形手中圆台上下底面的半径；
根据图形手位姿信息、虚拟物体位姿信息，以及所述球体与圆台之间的接触判断条件得到与所述图形手手指侧边接触的碰撞球信息；所述碰撞球信息包括碰撞球的编号、碰撞球的球心坐标和碰撞球的球体半径；所述碰撞球是所述虚拟物体中与所述图形手发生接触的球体；所述图形手手指侧边是手指圆台指节的侧边；所述图形手位姿信息用于图形显示；利用和
分别代表图形手每个手指的掌指关节角、近指关节角和远指关节角，根据所述图形手位姿信息，以及图形手与虚拟物体接触的碰撞球信息，构建图形手手指的侧边与虚拟物体的非嵌入接触约束
其中N为虚拟物体上与图形手虚拟手指接触的球体个数；根据所述圆台与球体的非嵌入接触约束，对于图形手的每个手指，将其三个关节角
和
作为一个整体在非嵌入接触约束集下进行优化，求解得到不嵌入虚拟物体的图形手手指关节角信息；
根据所述图形手手指关节角信息和公式F i＝G i·n·|P ig-P ih|计算第i个碰撞球与对应的虚拟手指接触的正压力F i；其中，G i为3×3刚度矩阵，n表示的单位向量，O i表示第i个碰撞球的球心，P i表示第i个碰撞球与虚拟手指侧边的接触点，|P ig-P ih|为图形手上点P i与触觉手上相应位置点P i'之间的线段长度。
- 根据权利要求1所述的用于虚拟手力觉交互的接触处理方法，其特征在于，在图形手与虚拟物体的接触判断时：所述图形手位姿是前一时刻的图形手位姿；所述图形手的每一个指节圆台截面半径均增加Δr；所述图形手的每一个指节半球半径均增加Δr；通过以上改变得到的图形手用于图形手与虚拟物体的接触判断，根据所述球-圆台接触判断条件和球-球接触判断条件得到的碰撞球用于非嵌入接触约束集的构建。
- 根据权利要求1所述的用于虚拟手力觉交互的接触处理方法，其特征在于，对于每个手指的远端指节，在构建所述非嵌入接触约束的同时，还包括：
如果碰撞球与模拟指尖的球体有接触，则构建球体与半球之间的非嵌入约束：将该接触约束添加到所述非嵌入接触约束集中用于图形手指关节角的优化。
- 根据权利要求1所述的用于虚拟手力觉交互的接触处理方法，其特征在于，在所述根据公式F i＝G i·n·|P ig-P ih|计算第i个碰撞球与对应的虚拟指节接触的正压力F i之后，还包括：利用公式f i＝u·n i·|F i|计算虚拟手滑动过程中的摩擦力；其中，f i表示在第i个碰撞球处的摩擦力，u表示动摩擦系数，n i为第i个碰撞球与虚拟 手指之间相对运动的法向量。
- 一种用于虚拟手力觉交互的接触处理系统，其特征在于，所述系统包括：虚拟手构建模块，用于：利用圆台铰链模型模拟真实手，得到虚拟手；所述虚拟手包括若干个圆台，每个所述圆台模拟所述真实手的每一个指节；掌指关节及两个指节间的关节采用球体模拟；所述虚拟手还包括若干个半球，每个半球位于模拟远端指节的圆台上，每个半球模拟所述真实手的每一个指尖；利用球树模型模拟物体，得到虚拟物体；所述虚拟物体包括若干个带有编号的球体；获取用户真实手的姿态；利用所述真实手的姿态驱动虚拟手，称为触觉手；将所述触觉手在虚拟场景中显示，得到图形手，并将所述虚拟物体在所述虚拟场景中显示；球-圆台碰撞检测模块，用于：构建所述球体与所述圆台之间的接触判断条件：和
其中，|OC| j为虚拟物体上编号为j的球体的球心到图形手中圆台的中心轴线l的距离，R j为虚拟物体上编号为j的球体的半径，M为虚拟物体上有待检测的球体个数，
C j表示虚拟物体上编号为j的球体的球心在中心轴线l上的垂足，E和D分别表示图形手中圆台上下底面的圆心，α表示图形手中圆台侧边与其中心轴线的夹角，r 2和r 1分别表示图形手中圆台上下底面的半径；
根据图形手位姿信息、虚拟物体位姿信息，以及所述球体与圆台之间的接触判断条件得到与所述图形手手指侧边接触的碰撞球信息；所述碰撞球信息包括碰撞球的编号、碰撞球的球心坐标和碰撞球的球体半径；所述碰撞球是所述虚拟物体中与所述图形手发生接触的球体；所述图形手手指侧边是手指圆台指节的侧边；所述图形手位姿信息用于图形显示；球-圆台碰撞响应模块，用于：利用和
分别代表图形手每个手指的掌指关节角、近指关节角和远指关节角，根据所述图形手位姿信息，以及图形手与虚拟物体接触的碰撞球信息，构建图形手手指的侧边与虚拟物体的非嵌入接触约束
其中N为虚拟物体上与图形手虚拟手指接触的球体个数；根据所述圆台与球体的非嵌入接触约束，对于图形手的每个手指，将其三个关节角
和
作为一个整体在非嵌入接触约束集下进行优化，求解得到不嵌入虚拟物体的图形手手指关节角信息；
接触力计算模块，用于：根据所述图形手手指关节角信息和公式F i＝G i·n·|P ig-P ih|计算第i个碰撞球与对应的虚拟手指接触的正压力F i；其中，G i为3×3刚度矩阵，n表示的单位向量，O i表示第i个碰撞球的球心，P i表示第i个碰撞球与虚拟手指侧边的接触点，|P ig-P ih|为图形手上点P i与触觉手上相应位置点P i'之间的线段长度。
- 根据权利要求5所述的用于虚拟手力觉交互的接触处理系统，其特征在于，在图形手与虚拟物体的接触判断时：球-圆台碰撞检测模块，还包括以下特点：所述图形手位姿是前一时刻的图形手位姿；所述图形手的每一个指节圆台截面半径均增加Δr；所述图形手的每一个指节半球半径均增加Δr；通过以上改变得到的图形手用于图形手与虚拟物体的接触判断，根据所述球-圆台接触判断条件和球-球接触判断条件得到的碰撞球用于非嵌入接触约束集的构建。
- 根据权利要求5所述的用于虚拟手力觉交互的接触处理系统，其特征在于，还包括：端点接触处理模块，用于：对于每个手指的远端指节，在构建所述非嵌入接触约束的同时；
若碰撞球与模拟指尖的球体有接触，则构建球体与半球之间的非嵌入 约束：将该接触约束添加到所述非嵌入接触约束集中用于图形手指关节角的优化；
若碰撞球与模拟指尖的球体没有接触，则仅构建球体与圆台之间的接触约束。 - 根据权利要求5所述的用于虚拟手力觉交互的接触处理系统，其特征在于，在所述根据公式F i＝G i·n·|P ig-P ih|计算第i个碰撞球与对应的虚拟指节接触的正压力F i之后，还包括：利用公式f i＝u·n i·|F i|计算虚拟手滑动过程中的摩擦力；其中，f i表示在第i个碰撞球处的摩擦力，u表示动摩擦系数，n i为第i个碰撞球与虚拟手指之间相对运动的法向量。
- 根据权利要求5所述的用于虚拟手力觉交互的接触处理系统，其特征在于，当涉及手掌交互时，先优化图形手的手掌位姿信息，通过矩阵变换得到图形手的手指位姿信息后，再根据图形手的手指位姿信息、虚拟物体的位姿信息，采用所述球-圆台碰撞检测模块、球-圆台碰撞响应模块、端点接触处理模块优化求解不嵌入虚拟物体的图形手手指关节角信息。
- 根据权利要求5所述的用于虚拟手力觉交互的接触处理系统，其特征在于，当处理多手指交互时，采用并行处理方式。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210384860.4 | 2022-04-13 | ||
| CN202210384860.4A CN115268623A (zh) | 2022-04-13 | 2022-04-13 | 一种用于虚拟手力觉交互的接触处理方法及系统 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023197485A1 true WO2023197485A1 (zh) | 2023-10-19 |
Family
ID=83758633
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2022/111749 WO2023197485A1 (zh) | 2022-04-13 | 2022-08-11 | 一种用于虚拟手力觉交互的接触处理方法及系统 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115268623A (zh) |
| WO (1) | WO2023197485A1 (zh) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102339129A (zh) * | 2011-09-19 | 2012-02-01 | 北京航空航天大学 | 一种基于语音和手势的多通道人机交互方法 |
| CN103869983A (zh) * | 2014-03-26 | 2014-06-18 | 南京信息工程大学 | 一种用于力触觉人机交互的柔性物体变形仿真方法 |
| US20170212589A1 (en) * | 2016-01-27 | 2017-07-27 | Tactai, Inc. | Providing fingertip tactile feedback from virtual objects |
| CN109116992A (zh) * | 2018-08-31 | 2019-01-01 | 北京航空航天大学 | 一种用于虚拟手力反馈交互的碰撞响应系统 |
-
2022
- 2022-04-13 CN CN202210384860.4A patent/CN115268623A/zh active Pending
- 2022-08-11 WO PCT/CN2022/111749 patent/WO2023197485A1/zh unknown
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102339129A (zh) * | 2011-09-19 | 2012-02-01 | 北京航空航天大学 | 一种基于语音和手势的多通道人机交互方法 |
| CN103869983A (zh) * | 2014-03-26 | 2014-06-18 | 南京信息工程大学 | 一种用于力触觉人机交互的柔性物体变形仿真方法 |
| US20170212589A1 (en) * | 2016-01-27 | 2017-07-27 | Tactai, Inc. | Providing fingertip tactile feedback from virtual objects |
| CN109116992A (zh) * | 2018-08-31 | 2019-01-01 | 北京航空航天大学 | 一种用于虚拟手力反馈交互的碰撞响应系统 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115268623A (zh) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiang et al. | Gesture recognition based on skeletonization algorithm and CNN with ASL database | |
| Santos et al. | Reported anatomical variability naturally leads to multimodal distributions of Denavit-Hartenberg parameters for the human thumb | |
| CN109116992B (zh) | 一种用于虚拟手力反馈交互的碰撞响应系统 | |
| CN105278685B (zh) | 一种基于eon的辅助教学系统及教学方法 | |
| Blaga et al. | Too hot to handle: An evaluation of the effect of thermal visual representation on user grasping interaction in virtual reality | |
| Kim et al. | Computationally efficient techniques for real time surgical simulation with force feedback | |
| Zhang et al. | Collaborative virtual laboratory environments with hardware in the loop | |
| Tong et al. | Survey on hand-based haptic interaction for virtual reality | |
| Liu et al. | Development of a virtual maintenance system with virtual hand | |
| Wong et al. | Virtual 3d sculpting | |
| Blaga et al. | Freehand grasping: An analysis of grasping for docking tasks in virtual reality | |
| WO2023197485A1 (zh) | 一种用于虚拟手力觉交互的接触处理方法及系统 | |
| Teleb et al. | Data glove integration with 3d virtual environments | |
| Tong et al. | Configuration-based optimization for virtual hand haptic simulation | |
| WO2021195916A1 (zh) | 动态手部仿真方法、装置和系统 | |
| Zhang | Computer-assisted human-computer interaction in visual communication | |
| Shi et al. | Grasping 3d objects with virtual hand in vr environment | |
| Shi et al. | A mixed-depth visual rendering method for bleeding simulation | |
| Zhang et al. | Shadow-driven 4d haptic visualization | |
| CN112506339A (zh) | 一种用于穿戴式触觉交互的虚拟手力觉合成方法及系统 | |
| Mahdikhanlou et al. | Object manipulation and deformation using hand gestures | |
| Liu et al. | COMTIS: Customizable touchless interaction system for large screen visualization | |
| Han et al. | A framework for virtual hand haptic interaction | |
| CN109189206A (zh) | 一种三维手势生成方法 | |
| Yin | A hierarchical approach to continuous gesture analysis for natural multi-modal interaction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22937123 Country of ref document: EP Kind code of ref document: A1 |