CN112306234B - Hand operation identification method and system based on data glove - Google Patents

Hand operation identification method and system based on data glove Download PDF

Info

Publication number
CN112306234B
CN112306234B CN202011020659.5A CN202011020659A CN112306234B CN 112306234 B CN112306234 B CN 112306234B CN 202011020659 A CN202011020659 A CN 202011020659A CN 112306234 B CN112306234 B CN 112306234B
Authority
CN
China
Prior art keywords
hand
target object
virtual
data glove
action
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011020659.5A
Other languages
Chinese (zh)
Other versions
CN112306234A (en
Inventor
张军
金淼
郭志伟
赖国书
黄天富
叶强
雷民
陈习文
陈卓
卢冰
汪泉
王斯琪
王旭
聂高宁
周玮
付济良
齐聪
郭子娟
余雪芹
刘俊
郭鹏
朱赤丹
吴志武
王春光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Corp of China SGCC
China Electric Power Research Institute Co Ltd CEPRI
State Grid Fujian Electric Power Co Ltd
Marketing Service Center of State Grid Fujian Electric Power Co Ltd
Original Assignee
State Grid Corp of China SGCC
China Electric Power Research Institute Co Ltd CEPRI
State Grid Fujian Electric Power Co Ltd
Marketing Service Center of State Grid Fujian Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Corp of China SGCC, China Electric Power Research Institute Co Ltd CEPRI, State Grid Fujian Electric Power Co Ltd, Marketing Service Center of State Grid Fujian Electric Power Co Ltd filed Critical State Grid Corp of China SGCC
Priority to CN202011020659.5A priority Critical patent/CN112306234B/en
Publication of CN112306234A publication Critical patent/CN112306234A/en
Application granted granted Critical
Publication of CN112306234B publication Critical patent/CN112306234B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014Hand-worn input/output arrangements, e.g. data gloves
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • G06V40/28Recognition of hand or arm movements, e.g. recognition of deaf sign language

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a hand operation identification method and a hand operation identification system based on data gloves, wherein the method comprises the following steps: determining a core area and a buffer area of a target object; establishing an interactive animation model of the virtual imitation hand and the data glove; judging the action type of the data glove when the positioning of the data glove enters the core area of the target object; when the target object performs a first type of action, the virtual imitation hand and the target object are kept in a connection state; when the target object cannot move, the virtual imitation hand is kept in a connected state with the target object and does not move; when the target object moves, the target object moves along with the positioning of the virtual imitation hand; when the target object performs a second type of action and when the target object moves after contact with the virtual simulated hand, then the target object moves following the positioning of the virtual simulated hand.

Description

Hand operation identification method and system based on data glove
Technical Field
The invention relates to the technical field of virtual reality, in particular to a hand operation identification method and system based on data gloves.
Background
The collision problem in the virtual reality environment is mostly to set a rigid structure for an object or to further perform collision processing according to all structures and part shapes in a scene, however, the hand operation collision problem of the data glove cannot be well solved by the method in the prior art.
The data glove can be interacted with the virtual environment to establish a simulated gesture model, so that the operation behaviors of a person are reproduced in the virtual environment.
Therefore, a technique is needed to implement a data glove based hand operation collision recognition method and system.
Disclosure of Invention
The technical scheme of the invention provides a hand operation collision recognition method and a hand operation collision recognition system based on data gloves, which are used for solving the problem of how to recognize hand operation collision based on the data gloves.
In order to solve the above problems, the present invention provides a hand operation recognition method based on data glove, the method comprising:
setting and positioning for the virtual imitation hand and the data glove according to the set distance;
determining a core area of a target object, wherein the core area is a space area with the distance between the positioning of a virtual imitation hand and the target object as a radius when the virtual imitation hand can touch the target object;
determining a buffer area of a target object, wherein the buffer area is a spatial range of the core area which extends outwards and is based on the positioning of the virtual imitation hand and the distance between the core area and the target object;
establishing an interactive animation model of the virtual imitation hand and the data glove, and determining the position change of the data glove according to the animation of the data glove based on the interactive animation model;
judging the action type of the data glove when the positioning of the data glove enters the core area of the target object;
when the target object performs a first type of action, the virtual imitation hand and the target object are kept in a connection state; when the target object cannot move, the virtual mimicking hand remains connected to the target object and does not move; when the target object moves, the target object moves along with the positioning of the virtual imitation hand;
when the target object performs a second type of action and when the target object moves after being contacted with the virtual simulated hand, the target object moves along with the positioning of the virtual simulated hand; or when the target object does not move after contacting with the virtual simulated hand, judging that the virtual simulated hand contacts with the target object;
when the data glove performs a first type of action in the core area and the positioning of the data glove leaves the core area of the target object, and when the positioning of the data glove is at the boundary of the core area, the virtual imitation glove restores to the real action of the data glove after performing the action of releasing the target object, the virtual imitation hand follows the positioning position of the data glove, so that the virtual imitation hand coincides with the positioning position of the data glove at the boundary of the buffer area, and the movement speed of the virtual imitation hand in the buffer area is as follows:
V hand with a handle =[(D 1,2 +40)/40]*V Fixing device
wherein ,VHand with a handle V for the movement speed of hands in a virtual reality system Fixing device For the speed of movement of the data glove positioned in the buffer zone, D 1,2 To virtually mimic the spatial distance of the location of the hand's location from the location of the data glove.
Preferably, the first type of action is an action that virtually mimics a hand with a strong correlation to the target object;
the second type of action is a virtual simulated hand that is not associated with the action of the target object.
Preferably, the positioning of the virtual simulated hand is within the core region when the target object performs a first type of action.
Preferably, the method further comprises: determining an external area of a target object, wherein the external area is a space range outside a buffer area of the target object;
the virtual simulated hand is consistent with the motion of the data glove when the virtual simulated hand enters the buffer region from the outer region or when the virtual simulated hand enters the buffer region from the core region.
Preferably, the method further comprises: the virtual simulated hand is consistent with the motion of the data glove when the virtual simulated hand is located in the outer region and is not in contact with the target object.
Based on another aspect of the present invention, the present invention provides a hand operation recognition system based on data glove, the system comprising:
the initial unit is used for setting and positioning the virtual imitation hand and the data glove according to the set distance; determining a core area of a target object, wherein the core area is a space area with the distance between the positioning of a virtual imitation hand and the target object as a radius when the virtual imitation hand can touch the target object; determining a buffer area of a target object, wherein the buffer area is a spatial range of the core area which extends outwards and is based on the positioning of the virtual imitation hand and the distance between the core area and the target object; establishing an interactive animation model of the virtual imitation hand and the data glove, and determining the position change of the data glove according to the animation of the data glove based on the interactive animation model;
the execution unit is used for judging the action type of the data glove when the positioning of the data glove enters the core area of the target object;
when the target object performs a first type of action, the virtual imitation hand and the target object are kept in a connection state; when the target object cannot move, the virtual mimicking hand remains connected to the target object and does not move; when the target object moves, the target object moves along with the positioning of the virtual imitation hand;
when the target object performs a second type of action and when the target object moves after being contacted with the virtual simulated hand, the target object moves along with the positioning of the virtual simulated hand; or when the target object does not move after contacting with the virtual simulated hand, judging that the virtual simulated hand contacts with the target object;
when the data glove performs a first type of action in the core area and the positioning of the data glove leaves the core area of the target object, and when the positioning of the data glove is at the boundary of the core area, the virtual imitation glove restores to the real action of the data glove after performing the action of releasing the target object, the virtual imitation hand follows the positioning position of the data glove, so that the virtual imitation hand coincides with the positioning position of the data glove at the boundary of the buffer area, and the movement speed of the virtual imitation hand in the buffer area is as follows:
V hand with a handle =[(D 1,2 +40)/40]*V Fixing device
wherein ,VHand with a handle V for the movement speed of hands in a virtual reality system Fixing device For the speed of movement of the data glove positioned in the buffer zone, D 1,2 Positioning of data glove for virtually mimicking the position of a handIs a spatial distance of the position of (c).
Preferably, the first type of action is an action that virtually mimics a hand with a strong correlation to the target object;
the second type of action is a virtual simulated hand that is not associated with the action of the target object.
Preferably, the positioning of the virtual simulated hand is within the core region when the target object performs a first type of action.
Preferably, the method further comprises: determining an external area of a target object, wherein the external area is a space range outside a buffer area of the target object;
the virtual simulated hand is consistent with the motion of the data glove when the virtual simulated hand enters the buffer region from the outer region or when the virtual simulated hand enters the buffer region from the core region.
Preferably, the method further comprises: the virtual simulated hand is consistent with the motion of the data glove when the virtual simulated hand is located in the outer region and is not in contact with the target object.
In the technical scheme of the invention, a Hi5 VR Glove data Glove is taken as an example, wherein the data Glove consists of a positioner fixed on a wrist and a plurality of knuckle sensors on fingers. Wherein the positioner determines the position of the hand and the knuckle sensor determines the posture of the hand.
Drawings
Exemplary embodiments of the present invention may be more completely understood in consideration of the following drawings:
FIG. 1 is a flow chart of a method for identifying hand operations based on a data glove according to a preferred embodiment of the present invention;
fig. 2 is a schematic view illustrating the division of an external space region of a target object according to a preferred embodiment of the present invention;
FIG. 3 is a schematic view of target object core region scoping in accordance with a preferred embodiment of the present invention;
FIG. 4 is a schematic diagram of a first type of action in accordance with a preferred embodiment of the present invention;
FIG. 5 is a schematic diagram of a second type of action in accordance with a preferred embodiment of the present invention;
FIG. 6 is a schematic diagram of the interaction flow of a target object in a core region according to a preferred embodiment of the present invention;
FIG. 7 is a schematic diagram of interaction rules of a target object in a core region according to a preferred embodiment of the present invention;
fig. 8 is a core area movement diagram of a target object according to a preferred embodiment of the present invention;
FIG. 9 is a diagram of a core region grabbing action according to a preferred embodiment of the present invention;
FIG. 10 is a schematic view of a locator just out of the core area according to a preferred embodiment of the invention;
FIG. 11 is a schematic illustration of the change in velocity off core area in accordance with a preferred embodiment of the present invention;
FIG. 12 is a schematic diagram of a simultaneous de-buffers according to a preferred embodiment of the present invention;
FIG. 13 is a schematic illustration of palm and ball sizes in accordance with a preferred embodiment of the present invention;
FIG. 14 is a schematic diagram of interactions under a first type of action in accordance with a preferred embodiment of the present invention;
FIG. 15 is a schematic diagram of a sphere motion with traction according to a preferred embodiment of the present invention;
FIG. 16 is a two-dimensional coordinate diagram of hand versus positioner motion velocity in accordance with a preferred embodiment of the present invention;
FIG. 17 is a two-dimensional coordinate diagram of a hand versus positioner motion velocity relationship in accordance with a preferred embodiment of the present invention;
FIG. 18 is a two-dimensional coordinate diagram of hand versus positioner motion velocity in accordance with a preferred embodiment of the present invention; and
figure 19 is a block diagram of a hand operation recognition system based on a data glove in accordance with a preferred embodiment of the present invention.
Detailed Description
The exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, however, the present invention may be embodied in many different forms and is not limited to the examples described herein, which are provided to fully and completely disclose the present invention and fully convey the scope of the invention to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the accompanying drawings is not intended to be limiting of the invention. In the drawings, like elements/components are referred to by like reference numerals.
Unless otherwise indicated, terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. In addition, it will be understood that terms defined in commonly used dictionaries should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense.
Fig. 1 is a flow chart of a hand operation recognition method based on a data glove according to a preferred embodiment of the present invention. The invention provides a real-time modeling method for a hand and an impenetrable contact object in real time, which solves the problem of operation authenticity of the hand and the contact object. In a virtual reality environment of a partially real-time operation training class, a handle cannot be used to complete a refined operation, and therefore, a data glove is required to be used for operation in the virtual reality environment.
As shown in fig. 1, the present invention provides a hand operation recognition method based on a data glove, the method comprising:
preferably, in step 101: and setting positioning for the virtual imitation hand and the data glove according to the set distance.
Preferably, at step 102: and determining a core area of the target object, wherein the core area is a space area with the distance between the positioning of the virtual simulated hand and the target object as a radius when the virtual simulated hand can touch the target object.
Preferably, in step 103: a buffer area of the target object is determined, and the buffer area is a spatial range which is extended outwards by taking the core area as a basis of the positioning of the virtual imitation hand and the distance between the target object.
The virtual reality collision detection method under the operation of the data glove divides the outer space of the impenetrable object into three areas, namely an outer area, a buffer area and a core area.
The invention divides the region of the virtual reality space into three blocks, and the dividing method is based on the process of the data glove during the operation in the virtual space and the collision with the object.
The core area is an area where a hand may collide with an impenetrable object. The extension is 20cm outwards based on an impenetrable object (typically the locator on the wrist is no more than 20cm from the fingertip). That is, if the locator on the wrist does not enter the core area, the finger tip is not likely to come into contact with the impenetrable object furthest.
The buffer zone was set such that the core zone extended outwardly by a further 40cm. The purpose of the buffer area is to express an area that will separate from the impenetrable object after the hand leaves the core area, provided for a visually realistic experience.
The outer region, which is farther from the object, is the region of the glove after it has completely left the cushioning region. In the outer region, the position of the hand is fully synchronized with the actual motion.
Preferably, at step 104: establishing an interactive animation model of the virtual imitation hand and the data glove, and determining the position change of the data glove according to the animation of the data glove based on the interactive animation model.
Preferably, in step 105: judging the action type of the data glove when the positioning of the data glove enters the core area of the target object; preferably, the first type of action is an action that virtually mimics a hand with a target object; the second type of action is an action that virtually mimics a hand with a target object.
Preferably, at step 106: when the target object performs a first type of action, the virtual imitation hand and the target object are kept in a connection state; when the target object cannot move, the virtual imitation hand is kept in a connected state with the target object and does not move; when the target object moves, the target object moves along with the positioning of the virtual imitation hand; preferably, the positioning of the virtual simulated hand is within the core region when the target object performs the first type of action.
Preferably, in step 107: when the target object performs a second type of action and when the target object moves after contacting the virtual simulated hand, the target object moves along with the positioning of the virtual simulated hand; or when the target object does not move after contacting with the virtual simulated hand, judging that the virtual simulated hand contacts with the target object.
In the present invention, after the positioning of the data glove into the core region, penetration of the hand and the impenetrable object may occur. Actions expressed by the finger sensors are classified into two types: 1. the hand is strongly associated with the impenetrable object. 2. The hand is not associated with an impenetrable object. For example: the actions of grabbing, pinching, holding and the like belong to the first class; pushing, beating, throwing, placing and the like belong to the second category.
In the core area, the hands are connected to the impenetrable object if a first type of action is performed. As long as the locator position does not leave the core area, the hand is always connected with the impenetrable object. If the impenetrable object is not movable, the state of motion is maintained in the virtual reality system regardless of the motion of the positioner. When the hand performs a releasing action, the position of the locator in the virtual space leaves the core area, and then the hand synchronously performs the releasing action and leaves the impenetrable object in the virtual reality system. The specific exit steps are illustrated in the buffer area interaction rules.
If the impenetrable object can be moved, such as rotated, moved, etc. The movement of the positioner, if it is the movable direction of the impenetrable object, the hand moves with the impenetrable object. The core region follows the change with the movement of the impenetrable object.
In the core area, if the second type of motion is performed, in the virtual reality system, the position of the locator does not enter the core area, and the contact between the hand and the impenetrable object is considered to be generated in a logical relationship. If the impenetrable object is movable by such contact, the core region follows the change in movement of the impenetrable object, and the position of the hand does not change. If the impenetrable object is unable to move through such contact, only the hand is judged to be in contact with the impenetrable object.
Preferably, at step 108: when the data glove performs a first type of action in the core area and the positioning of the data glove leaves the core area of the target object, and when the positioning of the data glove is at the boundary of the core area, the virtual imitation glove restores to the real action of the data glove after performing the action of releasing the target object, the virtual imitation hand follows the positioning position of the data glove, so that the virtual imitation hand coincides with the positioning position of the data glove at the boundary of the buffer area, and the movement speed of the virtual imitation hand in the buffer area is as follows:
V hand with a handle =[(D 1,2 +40)/40]*V Fixing device
wherein ,VHand with a handle V for the movement speed of hands in a virtual reality system Fixing device For the speed of movement of the data glove positioned in the buffer zone, D 1,2 To virtually mimic the spatial distance of the location of the hand's location from the location of the data glove.
The buffer area is configured to allow the hand to be separated from the impenetrable object during the first type of operation from the core area, because in the virtual reality system, the hand is located beside the impenetrable object, and the positioner is located away from the core area, the sense of realism perceived by the operator is greatly reduced if the positioner is directly switched out. For this case, a buffer area is provided, and the corresponding operation is performed to make the feeling of the operator more realistic.
When the locator leaves the core area, the hands are separated from the impenetrable object after a releasing action is performed by the hands in the virtual reality system. At this time, the positioner is already at the junction of the buffer region and the core region, set as d1= { x 1 ,y 1 ,z 1 In a virtual reality system, the hand position is near the impenetrable object, set as d2= { x 2 ,y 2 ,z 2 }. The width of the buffer area is40cm. The positioner may move from one end of the buffer zone to the other, while in a virtual reality system, the hand position is from near the impenetrable object to the end of the buffer zone bordering the outer zone.
At this time, in the virtual reality system, the movement speed of the hand is:
V hand with a handle =[(D 1,2 +40)/40]*V Fixing device
wherein ,VHand with a handle V for the movement speed of hands in a virtual reality system Fixing device D for the movement speed of the positioner in the buffer zone 1,2 Is the spatial distance between the position of the locator in the virtual reality system and the position of the actual locator.
Preferably, the method further comprises: determining an external area of the target object, wherein the external area is a space range outside a buffer area of the target object;
the virtual simulated hand remains consistent with the motion of the data glove when the virtual simulated hand enters the buffer area from the outer area or when the virtual simulated hand enters the buffer area from the core area.
Preferably, the method further comprises: when the virtual simulated hand is located in the outer region and is not in contact with the target object, the virtual simulated hand remains consistent with the movements of the data glove.
The invention describes the situation that a hand leaves the core area and enters the buffer area after being strongly associated with an impenetrable object from the core area. In addition to this, other cases such as: when a hand enters the buffer area from the outer area, leaves the buffer area from the core area in the second type of motion, and the like, the operation of the buffer area is identical to the operation of the outer area. And according to the position coincidence of the normal hand and the positioner, displaying the action represented by the finger sensor in the virtual reality.
In the invention, the outer area is not contacted with the impenetrable object by hands, so that the area coincides with the position of the locator according to the normal hands, and the action represented by the finger sensor is displayed in virtual reality. That is, the hand represented by the data glove is fully coincident with the hand represented in the virtual reality system.
When modeling, the touchable object in the virtual environment needs to establish a hand model interactive animation established with the glove. In the event of a collision, the displayed collision picture needs to include animation such as grasping, holding, lifting, throwing, rotating, and the like. And establishing a position coordinate updating relation between the animation and the virtual object, and updating the position information of the virtual object in real time according to the operation in the animation so as to ensure that the operation behavior finished through the animation process after the virtual operation is finished can be correctly reflected through the position change of the virtual object. After the hand enters the core area, the interaction mode is changed into a collision mode using the handle, so that the image keeps true.
The present invention provides a study of a collision handling method for a data glove for hand movement and an impenetrable contact object. The operation in the virtual reality environment in the prior art is extremely easy to occur the phenomenon that the hand penetrates through the object. According to the invention, better visual feedback effect can be obtained through space region division.
The invention is illustrated by taking a hand-held sphere as an example:
the virtual space is provided with a sphere smaller than the palm of the hand. As shown in fig. 13. The core region is constructed according to the method, the buffer region is constructed, and other regions in the virtual space are external regions, such as partial spheres on fig. 14.
In fig. 14, the upper half shows a virtual system and the lower half is a data glove actually worn. When the ball is to be grasped, the hand enters the buffer area from the external area and then enters the core area, and if the first type of operation action is not made, the locator at the wrist cannot enter the core area in the virtual reality system. In a virtual reality system, a finger can touch the edge of a sphere. However, if the positioner continues to advance, in the virtual reality system, the gesture is changed only along with the return action of the finger sensor, and the wrist does not enter the core area. This avoids the problem of hand passing through the ball. I.e. the condition that the finger touches the sphere is determined by the position of the locator.
In a virtual reality environment, after a hand touches a ball, the hand in the virtual reality system does not advance any more even if the hand is still advancing in the real environment.
If a first type of operation is made, the hand grabs the sphere, the positioner enters the core area, the sphere moves with the hand movement, and if the grabbing is maintained, the sphere moves with the hand. The core region, the buffer region and the outer region will all change as the sphere moves. The core area, the buffer area and the outer area are integrated with the sphere, and the set parameters are aligned on the periphery of the sphere model.
As shown in fig. 15, if the sphere cannot move along with the hand for some reasons (rope traction, etc.), and the hand is always in a gripping state, the positioner leaves the core area during the movement of the hand, and after the hand performs a releasing action in the virtual reality system, the movement is performed according to the aforementioned strategy. The movement speed in the virtual system is slightly faster than that of the positioner, if the virtual system moves towards the outer area, the virtual system merges at the junction of the outer area and the buffer area, and if the virtual system moves again like the core area in the buffer area and performs the gripping action, the hand grips the sphere again at the junction of the buffer area and the core area.
The invention relates to a method for dividing a space region when a data glove is used in a virtual reality system and a hand interacts with an impenetrable object. The invention is characterized in that the core area is an impenetrable object and extends outwards for 20cm, and the buffer area is the core area and extends outwards for 40cm. Embodiments of the present invention include, but are not limited to, methods of setting up protection zones, values, and design considerations.
In the core area, the locator can enter the core area only when the first type of actions such as grabbing, holding and the like are performed, and the grabbing process of the hand and the impenetrable object is completed, at the moment, the hand can always grab the impenetrable object as long as the locator does not leave the core area, and can move along with the motion direction of the object; the hand does not change when moving in the direction in which the object cannot move. Until the positioner leaves the core region.
In the buffer area, when the locator leaves the core area, the locator can be released. This action is to make the motion of the hand display more realistic, and without this action the hand would penetrate the object in the virtual reality system. It appears unrealistic.
In the buffer area, when the positioner moves in the buffer area after leaving the core area, the relationship between the movement speed of the hand and the movement speed of the positioner in the virtual reality system. For example, if the two-dimensional coordinates are represented as the following straight lines (as shown in fig. 16) in the formula, other curves are shown in fig. 17 and 18.
Figure 19 is a block diagram of a hand operation recognition system based on a data glove in accordance with a preferred embodiment of the present invention. As shown in fig. 19, the present invention provides a hand operation recognition system based on data glove, the system comprising:
an initial unit 1901 for setting a position for the virtual simulated hand and the data glove, respectively, at a set distance; determining a core area of a target object, wherein the core area is a space area with the distance between the positioning of the virtual simulated hand and the target object as a radius when the virtual simulated hand can touch the target object; determining a buffer area of the target object, wherein the buffer area is a space range which is formed by extending outwards the core area and is based on the positioning of the virtual imitation hand and the distance between the target object; establishing an interactive animation model of the virtual imitation hand and the data glove, and determining the position change of the data glove according to the animation of the data glove based on the interactive animation model.
The virtual reality collision detection method under the operation of the data glove divides the outer space of the impenetrable object into three areas, namely an outer area, a buffer area and a core area.
The invention divides the region of the virtual reality space into three blocks, and the dividing method is based on the process of the data glove during the operation in the virtual space and the collision with the object.
The core area is an area where a hand may collide with an impenetrable object. The extension is 20cm outwards based on an impenetrable object (typically the locator on the wrist is no more than 20cm from the fingertip). That is, if the locator on the wrist does not enter the core area, the finger tip is not likely to come into contact with the impenetrable object furthest.
The buffer zone was set such that the core zone extended outwardly by a further 40cm. The purpose of the buffer area is to express an area that will separate from the impenetrable object after the hand leaves the core area, provided for a visually realistic experience.
An execution unit 1902, configured to determine a type of motion of the data glove when the positioning of the data glove enters a core area of the target object; preferably, the first type of action is an action that virtually mimics a hand with a target object; the second type of action is an action that virtually mimics a hand with a target object.
When the target object performs a first type of action, the virtual imitation hand and the target object are kept in a connection state; when the target object cannot move, the virtual imitation hand is kept in a connected state with the target object and does not move; when the target object moves, the target object moves along with the positioning of the virtual imitation hand; preferably, the positioning of the virtual simulated hand is within the core region when the target object performs the first type of action.
When the target object performs a second type of action and when the target object moves after contacting the virtual simulated hand, the target object moves along with the positioning of the virtual simulated hand; or when the target object does not move after contacting with the virtual imitation hand, judging that the virtual imitation hand contacts with the target object;
when the data glove performs a first type of action in the core area and the positioning of the data glove leaves the core area of the target object, and when the positioning of the data glove is at the boundary of the core area, the virtual imitation glove restores to the real action of the data glove after performing the action of releasing the target object, the virtual imitation hand follows the positioning position of the data glove, so that the virtual imitation hand coincides with the positioning position of the data glove at the boundary of the buffer area, and the movement speed of the virtual imitation hand in the buffer area is as follows:
V hand with a handle =[(D 1,2 +40)/40]*V Fixing device
wherein ,VHand with a handle V for the movement speed of hands in a virtual reality system Fixing device For the speed of movement of the data glove positioned in the buffer zone, D 1,2 To virtually mimic the spatial distance of the location of the hand's location from the location of the data glove.
In the present invention, after the positioning of the data glove into the core region, penetration of the hand and the impenetrable object may occur. Actions expressed by the finger sensors are classified into two types: 1. the hand is strongly associated with the impenetrable object. 2. The hand is not associated with an impenetrable object. For example: the actions of grabbing, pinching, holding and the like belong to the first class; pushing, beating, throwing, placing and the like belong to the second category.
In the core area, the hands are connected to the impenetrable object if a first type of action is performed. As long as the locator position does not leave the core area, the hand is always connected with the impenetrable object. If the impenetrable object is not movable, the state of motion is maintained in the virtual reality system regardless of the motion of the positioner. When the hand performs a releasing action, the position of the locator in the virtual space leaves the core area, and then the hand synchronously performs the releasing action and leaves the impenetrable object in the virtual reality system. The specific exit steps are illustrated in the buffer area interaction rules.
If the impenetrable object can be moved, such as rotated, moved, etc. The movement of the positioner, if it is the movable direction of the impenetrable object, the hand moves with the impenetrable object. The core region follows the change with the movement of the impenetrable object.
In the core area, if the second type of motion is performed, in the virtual reality system, the position of the locator does not enter the core area, and the contact between the hand and the impenetrable object is considered to be generated in a logical relationship. If the impenetrable object is movable by such contact, the core region follows the change in movement of the impenetrable object, and the position of the hand does not change. If the impenetrable object is unable to move through such contact, only the hand is judged to be in contact with the impenetrable object.
When the locator leaves the core area, the hands are separated from the impenetrable object after a releasing action is performed by the hands in the virtual reality system. At this time, the positioner is already at the junction of the buffer region and the core region, set as d1= { x 1 ,y 1 ,z 1 In a virtual reality system, the hand position is near the impenetrable object, set as d2= { x 2 ,y 2 ,z 2 }. The buffer zone width was 40cm. The positioner may move from one end of the buffer zone to the other, while in a virtual reality system, the hand position is from near the impenetrable object to the end of the buffer zone bordering the outer zone.
At this time, in the virtual reality system, the movement speed of the hand is:
V hand with a handle =[(D 1,2 +40)/40]*V Fixing device
wherein ,VHand with a handle V for the movement speed of hands in a virtual reality system Fixing device D for the movement speed of the positioner in the buffer zone 1,2 Is the spatial distance between the position of the locator in the virtual reality system and the position of the actual locator.
Preferably, the method further comprises: determining an external area of the target object, wherein the external area is a space range outside a buffer area of the target object;
the virtual simulated hand remains consistent with the motion of the data glove when the virtual simulated hand enters the buffer area from the outer area or when the virtual simulated hand enters the buffer area from the core area.
Preferably, the method further comprises: when the virtual simulated hand is located in the outer region and is not in contact with the target object, the virtual simulated hand remains consistent with the movements of the data glove.
The invention describes the situation that a hand leaves the core area and enters the buffer area after being strongly associated with an impenetrable object from the core area. In addition to this, other cases such as: when a hand enters the buffer area from the outer area, leaves the buffer area from the core area in the second type of motion, and the like, the operation of the buffer area is identical to the operation of the outer area. And according to the position coincidence of the normal hand and the positioner, displaying the action represented by the finger sensor in the virtual reality.
The outer region, which is farther from the object, is the region of the glove after it has completely left the cushioning region. In the outer region, the position of the hand is fully synchronized with the actual motion.
In the invention, the outer area is not contacted with the impenetrable object by hands, so that the area coincides with the position of the locator according to the normal hands, and the action represented by the finger sensor is displayed in virtual reality. That is, the hand represented by the data glove is fully coincident with the hand represented in the virtual reality system.
When modeling, the touchable object in the virtual environment needs to establish a hand model interactive animation established with the glove. In the event of a collision, the displayed collision picture needs to include animation such as grasping, holding, lifting, throwing, rotating, and the like. And establishing a position coordinate updating relation between the animation and the virtual object, and updating the position information of the virtual object in real time according to the operation in the animation so as to ensure that the operation behavior finished through the animation process after the virtual operation is finished can be correctly reflected through the position change of the virtual object. After the hand enters the core area, the interaction mode is changed into a collision mode using the handle, so that the image keeps true.
The invention has been described with reference to a few embodiments. However, as is well known to those skilled in the art, other embodiments than the above disclosed invention are equally possible within the scope of the invention, as defined by the appended patent claims.
Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise therein. All references to "a// the [ means, component, etc ]" are to be interpreted openly as referring to at least one instance of means, component, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

Claims (10)

1. A method of hand operation recognition based on data glove, the method comprising:
setting and positioning for the virtual imitation hand and the data glove according to the set distance;
determining a core area of a target object, wherein the core area is a space area with the distance between the positioning of a virtual imitation hand and the target object as a radius when the virtual imitation hand can touch the target object;
determining a buffer area of a target object, wherein the buffer area is a spatial range of the core area which extends outwards and is based on the positioning of the virtual imitation hand and the distance between the core area and the target object;
establishing an interactive animation model of the virtual imitation hand and the data glove, and determining the position change of the data glove according to the animation of the data glove based on the interactive animation model;
judging the action type of the data glove when the positioning of the data glove enters the core area of the target object;
when the target object performs a first type of action, the virtual imitation hand and the target object are kept in a connection state; when the target object cannot move, the virtual mimicking hand remains connected to the target object and does not move; when the target object moves, the target object moves along with the positioning of the virtual imitation hand;
when the target object performs a second type of action and when the target object moves after being contacted with the virtual simulated hand, the target object moves along with the positioning of the virtual simulated hand; or when the target object does not move after contacting with the virtual simulated hand, judging that the virtual simulated hand contacts with the target object;
when the data glove performs a first type of action in the core area and the positioning of the data glove leaves the core area of the target object, and when the positioning of the data glove is at the boundary of the core area, the virtual imitation glove restores to the real action of the data glove after performing the action of releasing the target object, the virtual imitation hand follows the positioning position of the data glove, so that the virtual imitation hand coincides with the positioning position of the data glove at the boundary of the buffer area, and the movement speed of the virtual imitation hand in the buffer area is as follows:
wherein ,for the movement speed of the hand in the virtual reality system, < >>For the speed of movement of the data glove positioned in the buffer zone,/->To virtually mimic the spatial distance of the location of the hand's location from the location of the data glove, d 1 To virtually mimic the location of the hand, d 2 Is the location of the data glove.
2. The method of claim 1, the first type of action being a virtual mimicking hand's action having a strong association with the target object;
the second type of action is a virtual simulated hand that is not associated with the action of the target object.
3. The method of claim 1, the positioning of the virtual simulated hand within the core region when the target object performs a first type of action.
4. The method of claim 1, further comprising: determining an external area of a target object, wherein the external area is a space range outside a buffer area of the target object;
the virtual simulated hand is consistent with the motion of the data glove when the virtual simulated hand enters the buffer region from the outer region or when the virtual simulated hand enters the buffer region from the core region.
5. The method of claim 4, further comprising: the virtual simulated hand is consistent with the motion of the data glove when the virtual simulated hand is located in the outer region and is not in contact with the target object.
6. A data glove-based hand operation recognition system, the system comprising:
the initial unit is used for setting and positioning the virtual imitation hand and the data glove according to the set distance; determining a core area of a target object, wherein the core area is a space area with the distance between the positioning of a virtual imitation hand and the target object as a radius when the virtual imitation hand can touch the target object; determining a buffer area of a target object, wherein the buffer area is a spatial range of the core area which extends outwards and is based on the positioning of the virtual imitation hand and the distance between the core area and the target object; establishing an interactive animation model of the virtual imitation hand and the data glove, and determining the position change of the data glove according to the animation of the data glove based on the interactive animation model;
the execution unit is used for judging the action type of the data glove when the positioning of the data glove enters the core area of the target object;
when the target object performs a first type of action, the virtual imitation hand and the target object are kept in a connection state; when the target object cannot move, the virtual mimicking hand remains connected to the target object and does not move; when the target object moves, the target object moves along with the positioning of the virtual imitation hand;
when the target object performs a second type of action and when the target object moves after being contacted with the virtual simulated hand, the target object moves along with the positioning of the virtual simulated hand; or when the target object does not move after contacting with the virtual simulated hand, judging that the virtual simulated hand contacts with the target object;
when the data glove performs a first type of action in the core area and the positioning of the data glove leaves the core area of the target object, and when the positioning of the data glove is at the boundary of the core area, the virtual imitation glove restores to the real action of the data glove after performing the action of releasing the target object, the virtual imitation hand follows the positioning position of the data glove, so that the virtual imitation hand coincides with the positioning position of the data glove at the boundary of the buffer area, and the movement speed of the virtual imitation hand in the buffer area is as follows:
wherein ,for the movement speed of the hand in the virtual reality system, < >>For the speed of movement of the data glove positioned in the buffer zone,/->To virtually mimic the spatial distance of the location of the hand's location from the location of the data glove, d 1 To virtually mimic the location of the hand, d 2 Is the location of the data glove.
7. The system of claim 6, the first type of action being a virtual mimicking hand's action having a strong correlation with the target object;
the second type of action is a virtual simulated hand that is not associated with the action of the target object.
8. The system of claim 6, the virtual mimicking hand positioning within the core area when the target object performs a first type of motion.
9. The system of claim 6, further comprising: determining an external area of a target object, wherein the external area is a space range outside a buffer area of the target object;
the virtual simulated hand is consistent with the motion of the data glove when the virtual simulated hand enters the buffer region from the outer region or when the virtual simulated hand enters the buffer region from the core region.
10. The system of claim 9, further comprising: the virtual simulated hand is consistent with the motion of the data glove when the virtual simulated hand is located in the outer region and is not in contact with the target object.
CN202011020659.5A 2020-09-25 2020-09-25 Hand operation identification method and system based on data glove Active CN112306234B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011020659.5A CN112306234B (en) 2020-09-25 2020-09-25 Hand operation identification method and system based on data glove

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011020659.5A CN112306234B (en) 2020-09-25 2020-09-25 Hand operation identification method and system based on data glove

Publications (2)

Publication Number Publication Date
CN112306234A CN112306234A (en) 2021-02-02
CN112306234B true CN112306234B (en) 2023-10-31

Family

ID=74488632

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011020659.5A Active CN112306234B (en) 2020-09-25 2020-09-25 Hand operation identification method and system based on data glove

Country Status (1)

Country Link
CN (1) CN112306234B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108983978A (en) * 2018-07-20 2018-12-11 北京理工大学 virtual hand control method and device
CN109460150A (en) * 2018-11-12 2019-03-12 北京特种机械研究所 A kind of virtual reality human-computer interaction system and method
KR20200051938A (en) * 2018-11-06 2020-05-14 한길씨앤씨 주식회사 Method for controlling interaction in virtual reality by tracking fingertips and VR system using it

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9552673B2 (en) * 2012-10-17 2017-01-24 Microsoft Technology Licensing, Llc Grasping virtual objects in augmented reality

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108983978A (en) * 2018-07-20 2018-12-11 北京理工大学 virtual hand control method and device
KR20200051938A (en) * 2018-11-06 2020-05-14 한길씨앤씨 주식회사 Method for controlling interaction in virtual reality by tracking fingertips and VR system using it
CN109460150A (en) * 2018-11-12 2019-03-12 北京特种机械研究所 A kind of virtual reality human-computer interaction system and method

Also Published As

Publication number Publication date
CN112306234A (en) 2021-02-02

Similar Documents

Publication Publication Date Title
JP5087101B2 (en) Program, information storage medium, and image generation system
US6597347B1 (en) Methods and apparatus for providing touch-sensitive input in multiple degrees of freedom
Ha et al. WeARHand: Head-worn, RGB-D camera-based, bare-hand user interface with visually enhanced depth perception
EP3275514A1 (en) Virtuality-and-reality-combined interactive method and system for merging real environment
Oprea et al. A visually realistic grasping system for object manipulation and interaction in virtual reality environments
CN112105486B (en) Augmented reality for industrial robots
CN113710432A (en) Method for determining a trajectory of a robot
Boonbrahm et al. Assembly of the virtual model with real hands using augmented reality technology
KR20200051938A (en) Method for controlling interaction in virtual reality by tracking fingertips and VR system using it
Nainggolan et al. User experience in excavator simulator using leap motion controller in virtual reality environment
CN109669538B (en) Object grabbing interaction method under complex motion constraint in virtual reality
CN112306234B (en) Hand operation identification method and system based on data glove
US11500453B2 (en) Information processing apparatus
CN104239119A (en) Method and system for realizing electric power training simulation upon kinect
Aleotti et al. Trajectory reconstruction with nurbs curves for robot programming by demonstration
WO2021195916A1 (en) Dynamic hand simulation method, apparatus and system
Barber et al. Sketch-based robot programming
Spanogianopoulos et al. Human computer interaction using gestures for mobile devices and serious games: A review
Shi et al. Grasping 3d objects with virtual hand in vr environment
Qingchao et al. The application of leap motion in astronaut virtual training
US20210232289A1 (en) Virtual user detection
Kim et al. Direct hand manipulation of constrained virtual objects
Figueiredo et al. Bare hand natural interaction with augmented objects
TW202144983A (en) Method of interacting with virtual creature in virtual reality environment and virtual object operating system
Nikolakis et al. A mixed reality learning environment for geometry education

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant