CN117251058B - Control method of multi-information somatosensory interaction system - Google Patents
Control method of multi-information somatosensory interaction system Download PDFInfo
- Publication number
- CN117251058B CN117251058B CN202311506619.5A CN202311506619A CN117251058B CN 117251058 B CN117251058 B CN 117251058B CN 202311506619 A CN202311506619 A CN 202311506619A CN 117251058 B CN117251058 B CN 117251058B
- Authority
- CN
- China
- Prior art keywords
- finger
- hand
- matrix
- angle
- joint
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000003993 interaction Effects 0.000 title claims abstract description 27
- 230000003238 somatosensory effect Effects 0.000 title claims abstract description 23
- 239000011159 matrix material Substances 0.000 claims abstract description 75
- 238000001514 detection method Methods 0.000 claims abstract description 22
- 210000003811 finger Anatomy 0.000 claims description 71
- 210000004247 hand Anatomy 0.000 claims description 21
- 230000001133 acceleration Effects 0.000 claims description 16
- 230000009471 action Effects 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 11
- 230000033001 locomotion Effects 0.000 claims description 10
- 230000000694 effects Effects 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 239000000463 material Substances 0.000 claims description 8
- 238000004088 simulation Methods 0.000 claims description 8
- 230000005489 elastic deformation Effects 0.000 claims description 7
- 210000004932 little finger Anatomy 0.000 claims description 6
- 210000003813 thumb Anatomy 0.000 claims description 6
- 244000060701 Kaempferia pandurata Species 0.000 claims description 5
- 235000016390 Uvaria chamae Nutrition 0.000 claims description 5
- 210000000988 bone and bone Anatomy 0.000 claims description 5
- 230000005484 gravity Effects 0.000 claims description 4
- 230000004044 response Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000009877 rendering Methods 0.000 claims description 2
- 230000000875 corresponding effect Effects 0.000 description 14
- 241000220225 Malus Species 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 7
- 238000005452 bending Methods 0.000 description 6
- 210000001145 finger joint Anatomy 0.000 description 6
- 230000002452 interceptive effect Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000036544 posture Effects 0.000 description 4
- 210000000707 wrist Anatomy 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000001276 controlling effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000010172 mouse model Methods 0.000 description 3
- 235000021016 apples Nutrition 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000007654 immersion Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000012530 fluid Substances 0.000 description 1
- 210000002478 hand joint Anatomy 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
Abstract
The invention provides a control method of a multi-information somatosensory interaction system, which relates to the technical field of control methods of multi-information somatosensory interaction systems, and comprises the following steps: acquiring an initial three-dimensional coordinate of a hand model; acquiring a rotation deflection angle alpha of the micro brushless motor through magnetic field detection 0 The method comprises the steps of carrying out a first treatment on the surface of the Rotation deflection angle alpha based on miniature brushless motor 0 And the diameter d of a single joint of the finger, determining the rotation angle of the single joint; constructing a first matrix T describing a specific shape of the hand model based on the rotation angle of the single joint; the moment provided by each joint is finally calculated.
Description
Technical Field
The invention relates to the technical field of control methods of multi-information somatosensory interaction systems, in particular to a control method of a multi-information somatosensory interaction system.
Background
In the virtual reality or augmented reality technology, a person in a physical space needs to control an object in the virtual space by manipulating a controller, and the fineness and feedback capability of the controller directly affect the fineness and specific feeling of the control of the person in the physical space. When a person in the physical space is manipulating, if the fineness is insufficient, a larger error can occur in the corresponding action when the virtual space object is interacted, and if the feedback capability is insufficient, the person in the physical space cannot accurately judge the action of the object in the virtual space, so that correct reflection can be made. Therefore, it is necessary for such controllers to meet both of these requirements to achieve good results. Meanwhile, in the interaction, if the user cannot timely feed back the command output by the controller to the controlled terminal, the interaction experience is seriously affected, which means that the specific use method of the controller must be simple enough and have an efficient algorithm to judge the specific command of the controller, so that the controlled terminal can respond quickly. On the other hand, if feedback is required to exist at the control end, the specific feedback algorithm is also required to be accurate and simple so as to achieve the purpose of quick feedback.
At present, the control scheme of the hands in the augmented reality technology is mainly divided into three schemes of a handle, a gesture and a glove, wherein the existing handle is mainly columnar to hold, and the accurate control of each finger of the hands in the virtual space cannot be achieved; the gesture mainly achieves the control effect on the capturing of the hand motions through the camera, and although each finger can be accurately controlled, the capturing motions are easy to make mistakes and cannot be accurately fed back to the hands in reality, so that the immersion experience is achieved; the glove scheme can achieve accurate feedback and fine control at the same time. However, in the existing scheme, the problem that the feedback form of the hand is single exists, for example, the purpose of feedback is achieved by only vibrating whether an object is touched or not, and the hand can penetrate through the object in the virtual space, so that finer feedback cannot be achieved; or the hand feedback is careful but difficult to realize and cannot bring substantial feedback sense, for example, simulation of the surface texture and temperature of an object is realized, a large number of sensors and feedback elements are required to be mounted on the glove, but good experience cannot be brought about on the simulated holding sense, and the holding sense directly influences the judgment of an operator on details such as the shape, the volume and the like of the object, so that the operator cannot perform finer operation.
In the existing gesture reading method of the virtual reality technology on the hand, a visual sensor is generally adopted to read the gesture of the user. The scheme has the problems that the gesture reading is inaccurate, a large number of sensors are needed to determine the gestures, the palm position calculating speed is low, and the like, for example, the old can have the problem that the gestures cannot be operated because the actions of the old are not in place when the old uses some gestures. On the other hand, the existing scheme cannot adapt to gestures of different people, and gesture reading accuracy cannot be gradually accurate along with using time when a user uses the gesture for a long time.
Disclosure of Invention
The invention aims to provide a control method of a multi-information somatosensory interaction system. In order to solve the technical problems existing in the background art.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the control method of the multi-information somatosensory interaction system comprises a glove body, a force feedback device, a vibration motor, an acceleration sensor, an inclination angle sensor and a microprocessor; the feedback device comprises a plurality of miniature brushless motors, worms, force feedback right-angle elements and magnetic field detection elements;
the method comprises the following steps:
acquiring an initial three-dimensional coordinate of a hand model;
through magnetic field detection, the rotation deflection angle alpha of the brushless motor is obtained 0 ；
Rotation deflection angle alpha based on brushless motor 0 And the diameter d of a single joint of the finger, determining the rotation angle of the single joint;
constructing a first matrix T describing a specific shape of the hand model based on the rotation angle of the single joint;
based on the first matrix T, determining the coordinates of each joint relative to the palm center through a first preset algorithm;
based on the data acquired by the acceleration sensor and the inclination sensor, determining the three-dimensional acceleration of the hand model and the inclination of the hand model around the three axes;
determining a real-time position of the hand model based on the three-dimensional acceleration of the hand model and the inclination angle of the hand model around the three axes;
determining a parameter information matrix M based on the coordinates of each joint relative to the palm center and the real-time position of the hand model;
based on the parameter information matrix M, predicting the motion of the finger to obtain a predicted parameter information matrix Mpt;
obtaining a real-time hand model based on the prediction parameter information matrix Mpt or the parameter information matrix M;
performing collision detection based on the real-time hand model, and extracting a position parameter and an inclination angle in a predicted parameter information matrix Mpt or a parameter information matrix M during collision detection so as to generate two hands in the form at corresponding positions in a virtual space;
extracting position, form and material data of an object in a preset virtual space, and rendering the virtual object according to the data to obtain a hand situation;
responding to the hand situation to meet a first preset condition or a second preset condition, and determining the actual force action of the fingers on the force generated by each motor based on a second preset method; and calculating the moment provided by each joint based on a Newton-Euler recursion dynamics algorithm.
In some embodiments, determining the rotation angle of the individual joint based on the rotation offset angle α0 of the brushless motor and the diameter d of the individual condyle of the finger comprises:
rotation deflection angle alpha based on brushless motor 0 Determining the offset kα of the force feedback right angle element relative to the initial position of the connection of the bottom end of the worm 0 ；
Based on the offset kα 0 The method comprises the steps of carrying out a first treatment on the surface of the Determining the rotation angle of the force feedback right-angle element by combining the diameter d of a single condyle of the finger;
the rotation angle of the individual joint is determined based on the rotation angle of the force feedback right angle element.
In some embodiments, the first matrix T is represented as follows:
；
wherein, the horizontal subscript indicates the order from thumb to little finger, and the vertical subscript indicates the order from finger root to fingertip;
the determining, based on the first matrix and through a first preset algorithm, coordinates of each joint relative to the palm center includes:
establishing a three-dimensional coordinate system by taking a palm center as an origin, comprising: for an index finger, establishing the three-dimensional coordinate system at each knuckle;
taking out a second column matrix in the T matrix, wherein the second column matrix has two degrees of freedom from the root to the second knuckle, and data beta 24 and beta 23 are needed, wherein beta 24 is a rotation parameter around the z axis, and beta 23 is a rotation parameter around the x axis;
based on euler angle rotation formula 1 and euler angle rotation formula 2:
（1）
（2）
substituting beta 24 into formula 1 and substituting beta 23 into formula 2 to obtain two rotation matrices:
rotating matrix 1:rotating matrix 2: />；
Wherein P is the coordinate of the origin of the coordinate system on the knuckle under the coordinate system of the finger root;
thereby, a conversion matrix can be obtained；
Coordinates of the second knuckle relative to the palm =The second knuckle corresponds to the root coordinate;
other transformation matrices and thus coordinates to the relative palm of each joint can be obtained in the same way.
In some embodiments, the coordinates of each joint are sequentially connected in the prediction parameter information matrix Mpt or the parameter information matrix M, and the connection line is a condyle of the finger, so as to determine a specific bone posture, and after the parameters such as the finger and the finger diameter of the person are estimated, the model of the opponent can be rendered, so that a real-time hand model is obtained.
In some embodiments, the first preset condition is that the operator, while gripping the object in the virtual space, the two-hand model collides with the model in the virtual space, the collision being a collision of the model that progressively detects each condyle.
In some embodiments, under the first preset condition:
the whole hand of operator's right hand holds the object, because the elastic deformation of object this moment, people's hand can receive reaction force in reality and make the finger unable to continue crooked, consequently, the brushless motor of each joint of gloves also will rotate drive worm and take power feedback right angle component to make the component reverse motion, hinder each condyle to continue to remove, thereby prevent the crooked of finger, reach the same effect with the reality condition, because gravity effect, little finger and the ring finger of holding will receive bigger force, consequently the driving force that relevant brushless motor provided also will be bigger, make the simulation finely go into a little, let the operator feel the concrete weight of object.
In some embodiments, under the first preset condition: when the force applied by an operator exceeds the elastic deformation range of the object, the feedback object is damaged, all brushless motors on the glove holding the hand are stopped, and at the moment, the hands can naturally move without being blocked.
In some embodiments, the second preset condition is that the hand information of the user is the same as the preset action.
In some embodiments, under the second preset condition:
and generating a real-time visible two-hand model for representing the real-time states of the two hands of the operator in the virtual space according to the predicted parameter information matrix Mpt or the parameter information matrix M, and generating an invisible predicted two-hand model, wherein the model does not collide with the real-time model, and the other collisions are the same as the real-time model.
In some embodiments, the method further comprises:
transmitting a command for the real-time model and a command for the virtual model to the microprocessor unit in response to the hand condition satisfying the first preset condition or the second preset condition, and then the microprocessor unit operating the motor of the glove in accordance with the command;
in response to the parameter information matrix M of the glove being the same as the last predicted parameter information matrix Mpt, the last pre-stored predicted command is directly executed,
responding to the fact that the parameter information matrix M of the glove is different from the previous predicted parameter information matrix Mpt, and calling a command transmitted in real time;
the predicted commands in each detection cycle will be stored, and only one predicted command is stored in total, so the predicted commands are updated in a constantly replaced form.
Meanwhile, the invention also discloses a control method device of the multi-information somatosensory interaction system, which comprises a processor and a memory; the memory is configured to store instructions that, when executed by the processor, cause the apparatus to implement a method of controlling a multi-information somatosensory interaction system according to any one of the above.
Meanwhile, the invention also discloses a computer readable storage medium, wherein the storage medium stores computer instructions, and when the computer reads the computer instructions in the storage medium, the computer runs the control method of the multi-information somatosensory interaction system.
Compared with the prior art, the invention has the remarkable advantages that:
the scheme of the invention can be used for realizing real-time detection of the hands of the user and feedback of the behaviors. The glove and the interaction mode thereof can feed back quickly and accurately, and can feed back different forces to each finger joint so as to achieve accurate simulation, and the calculation scheme can be accurate and quick only by a small number of sensors. Such gloves may be used in scenes requiring immersive interactions for remote manipulation, virtual reality interactions, and the like.
Meanwhile, the method for judging whether force feedback is needed or not according to the scheme of the invention is that a specific hand model is generated and a collision volume is added for interaction. Compared with the traditional method for generating hand coordinates and measuring and calculating whether force feedback is needed through distance, the method can cope with all shapes, the material of collision volume between models can be modified according to the needs, and meanwhile, compared with the traditional method, the method is more accurate.
The scheme of the invention pre-stores the instruction by utilizing the prediction information, and compared with the instruction directly input from a computer, the time for transmitting the instruction from computer software to a microprocessor unit is saved, so that the glove is more efficient to use, and the delay of control equipment during input and output is effectively controlled.
Drawings
FIG. 1 is a schematic diagram of the present embodiment directed to a multi-information somatosensory interactive glove system for a virtual reality system;
FIG. 2 is a schematic diagram of a force feedback device according to the present embodiment;
FIG. 3 is a flow chart illustrating a control method of the multi-information somatosensory interaction system according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a feedback force calculation method according to an embodiment of the invention.
Description of the drawings: the device comprises a 1-miniature brushless motor, a 2-worm, a 3-force feedback right-angle element, a 4-magnetic field detection element, a 5-power line, a 6-communication module, a 7-earphone interface, an 8-loudspeaker, a 9-vibration motor, a 10-power key, an 11-pressure sensor and a 12-angle sensor.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
On the contrary, the application is intended to cover any alternatives, modifications, equivalents, and variations as may be included within the spirit and scope of the application as defined by the appended claims. Further, in the following detailed description of the present application, specific details are set forth in order to provide a more thorough understanding of the present application. The present application will be fully understood by those skilled in the art without a description of these details.
A control method of the multi-information somatosensory interaction system according to the embodiments of the present application will be described in detail with reference to fig. 1 to 4. It is noted that the following examples are only for explaining the present application and are not limiting of the present application.
As shown in fig. 1, the multi-information somatosensory interactive glove system for a virtual reality system comprises a somatosensory interactive glove system structure and an intelligent feedback interactive control module.
The interactive glove system structure comprises a glove body, a force feedback device (comprising a miniature brushless motor 1, a worm 2, a force feedback right-angle element 3 and a magnetic field detection element 4), a power line 5, a communication module 6 (such as a Bluetooth module), an earphone interface 7 (3.5 mm earphone interface), a loudspeaker 8, a vibration motor 9 (such as a linear vibration motor), a power key 10, a pressure sensor 11, an angle sensor 12, an acceleration sensor, an inclination sensor and a microprocessor unit (positioned at the center of the back of the hand) can realize detection and feedback of information such as the shape, position, speed, acceleration, inclination and the like of an opponent, and touch and operation on a virtual object.
The intelligent feedback interaction control module comprises rotation deflection angle detection of a brushless motor, rotation angle and relative position calculation of each joint, space relative position and inclination angle detection of hands, data preprocessing and two-hand model modeling.
As shown in fig. 2, the design of the force feedback device comprises the following structure:
in some embodiments, the force feedback device is composed of a plurality of micro brushless motors 1, a worm 2, a force feedback right angle element 3 and a magnetic field detection element 4. The miniature brushless motors are positioned on the back of the glove finger joints, and each joint is provided with a corresponding miniature brushless motor and is connected with a worm for driving a corresponding force feedback right-angle element. The worm is connected with a driving rod of the miniature brushless motor.
In some embodiments, the long end of the force feedback right angle element is clung to the inner side of the finger, when any joint of the finger moves, the force feedback right angle element can rotate and drive the micro brushless motor to rotate. The two ends of the force feedback right angle element are fixed on the glove, and the force feedback right angle element will rotate around the fixed points of the two ends. The worm produces a lateral offset to the upper end of the force-feedback right angle element, and the telescoping rod stretches while the upper end traverses, converting the lateral displacement into rotation of the long end of the force-feedback right angle element.
In some embodiments, the length of the long end of each force feedback right angle element is the same as the length of the corresponding knuckle of the finger.
In some embodiments, the magnetic field detecting element is located at the bottom end of the micro brushless motor, when the brushless motor rotates, the north and south poles of the internal magnetic field rotate simultaneously, the magnetic field changes, and the magnetic field detecting element can detect the deflection angle of the motor only by detecting the change of the magnetic poles, so that the rotation angle of the finger joint is detected.
In some embodiments, the arrangement of the multi-source sensor includes the following:
in some embodiments, an acceleration sensor and an inclination sensor are located at the center of the back of the hand, and are used for detecting the relative acceleration and inclination of the palm, and the speed and inclination so as to calculate the specific position, speed and other data of the palm, and generate a two-hand model in the virtual space by combining the bending angles of the joints.
In some embodiments, the communication module is fixed on the side of the palm and is used for connecting with a computer, transmitting the data of the glove to the software for processing and inputting instructions into the glove through Bluetooth after the software processing of the computer is finished.
In some embodiments, a pressure sensor, external to the glove's fingertip, is used to facilitate detection of whether the finger is in contact with an object in reality.
In some embodiments, the microprocessor unit is secured at the center of the back of the hand. The microprocessor unit is connected with each miniature brushless motor, the vibration motor and the communication module, and each sensor is connected through a data line, and data is input to the microprocessor unit through the data line.
In some embodiments, the vibration motor is closely attached to the palm for simulating the pressure of the palm, and by the vibration motor, the specific position of the palm touching the object can be simulated and the scene which cannot be simulated by the finger joint force feedback element can be supplemented. When commanded by the microprocessor unit, the vibration motor will generate a corresponding vibration.
In some embodiments, the earphone interface is located outside the wrist of the left hand glove for connecting to a wired earphone; the power line is positioned below the earphone interface and is used for providing power for the glove; the speaker and microphone are located inside the wrist for giving acoustic feedback and as acoustic input elements for the user; the power key is positioned at the innermost side of the wrist and is used for basically controlling the equipment to be turned on and off.
In some embodiments, the tilt sensor and the acceleration sensor are three-axis sensors, placed in close proximity to the microprocessor unit at the center of the back of the palm. The glove is internally provided with an infrared device for wearing and detecting. An angle sensor is present at the base of each knuckle to detect the declination of the lateral movement of the finger.
As shown in FIG. 3, the invention also discloses a control method of the multi-information somatosensory interaction system. Can be realized based on the multi-information somatosensory interaction system.
As shown in fig. 3, the method includes:
s1: by pressing the power key for a long time, the glove is started, and at the moment, the user needs to initialize the glove positions according to the requirements shown by the corresponding operated software, so that the initial positions of the two hands are known. In this embodiment, the head-mounted display of the common virtual reality device is taken as an example, the initialization action is taken as an example of holding the head by both hands, and at this time, the software recognizes that both hands are at the preset initial position, that is, the position when the both hands hold the head, and at this time, the model with the hand shape obtained can be placed at the corresponding positions on both sides of the head-mounted display in the computer software. After this, the hand model will have initial three-dimensional coordinates relative to the spatial coordinate system within the virtual space, completing the initialization position.
S2: after starting up, the glove detects whether or not to wear, if the glove is worn, each sensor is started, the magnetic field of the micro brushless motor is detected by the magnetic field detecting element, and the rotation deflection angle of the micro brushless motor, namely, the deflection angle of the rotation change of the magnetic field is detected by the magnetic field detecting element due to the characteristics of the micro brushless motor, so the rotation deflection angle alpha of the micro brushless motor can be detected by the magnetic field detecting element 0 . Through the deflection angle alpha 0 Calculating the specific rotation angle alpha of the worm 0 And obtaining the offset kα of the force feedback right angle element relative to the initial position at the connection of the bottom end of the worm 0 The rotation angle of the force feedback right angle element can be calculated by combining the diameter d of the single condyle of the finger. The position of the finger when unfolded is taken as the initial position, and the force feedback right-angle element rotatesThe rotation angle is the rotation angle beta of the single joint of the finger.
；
As can be seen from fig. 1, each joint of the finger has a corresponding micro brushless motor, so the matrix parameter T of 3*5 can be obtained in total to reflect the specific shape of the hand, and note that at this time, the T matrix cannot reflect other parameters describing the hand, such as the position of the hand, the inclination angle of the palm, etc. The horizontal subscript indicates the order from the thumb to the little finger, and the vertical subscript indicates the order from the root of the finger to the four joints of the fingertip (e.g., β 32 Representing the angle of the second joint from bottom to top), at which time there is:
at this time, since the human finger has only two degrees of freedom, all parameters (e.g., β 14 Representing the angle detected by the angle sensor at the root of the index finger, i.e. the angle at which the index finger rotates horizontally):
the specific calculation scheme is that a three-dimensional coordinate system is established by taking the palm center as an origin, taking an index finger as an example, taking out a second column matrix in the T matrix, and two degrees of freedom are provided from the finger root to a second knuckle, wherein data beta is needed 24 And beta 23 Wherein beta is 24 Beta, the rotation parameter around the z-axis 23 For the rotation parameter around the x-axis, the expression 1 and the expression 2 are rotated by using euler angles (α and γ represent the rotation angle for the z-axis and the rotation angle for the x-axis, respectively): equation 1:
；
equation 2:
；
substituting beta 24 into formula 1 and substituting beta 23 into formula 2 to obtain two rotation matrices:
conversion matrix ；
Conversion matrix；
Wherein P is the coordinate of the origin of the coordinate system on the knuckle under the finger root coordinate system.
(x 2, y2, 0) is the coordinates of the root in the palm coordinate system.
Thereby, a conversion matrix can be obtained；
Coordinates of the second knuckle relative to the palm =The second knuckle is opposite to the root coordinate.
Other transformation matrices and thus coordinates to the relative palms of the individual joints can be obtained in the same way as part of the parameter information matrix.
The space relative position of the hand and the inclination angle of the hand can be detected through the acceleration sensor and the inclination angle sensor, and the three-dimensional acceleration a of the hand model is obtained x 、a y 、a z Inclination angle sigma around three axes x 、σ y 、σ z . Assuming the initial position is (x 0, y0, z 0), then the real-time positionThe calculation method of (1) is as follows:
；
then there isAs another part of the parameter information matrix of the glove.
Combining with the parameters to obtain a complete parameter information matrix.
In the parameter information matrix M t Transmitting the data into a microprocessor unit, capturing the data stream in the previous clock period, and predicting the finger motion to obtain a predicted parameter information matrix M pt . After the processing is finished, the microprocessor unit inputs the real-time palm data and the predicted data into the computer software through the Bluetooth module.
S3: in computer software, the coordinates of all joints are sequentially connected according to an M matrix, and the joints are the condyles of the fingers, so that the specific bone gestures of the fingers of the human can be obtained, the model of the hand can be rendered after the parameters such as the fingers and the finger thickness of the human are estimated, the specific shape of the hand is obtained, and the model of the hand can perform collision detection after the collision volume is added. And then, extracting the position parameters and the inclination angle in the M matrix, namely generating the hands in the form at the corresponding positions of the virtual space, and generating the hand model.
S4: meanwhile, the computer software extracts data such as the position, the shape, the material and the like of the object in the preset virtual space, renders the virtual object according to the data, and generates a model of the object.
This time will be divided into two cases:
s5: the first case is: high priority case: when a user operates the glove, when the two-hand model generated in the virtual space collides with the virtual object, the computer software outputs the material of the virtual object and requires the corresponding brushless motor on the glove to rotate or stop, so that simulation is realized. For example, when an operator grasps an apple in a virtual space, a two-hand model in computer software collides with the model in the virtual space, and the collision is to gradually detect the collision of the model of each condyle, that is, when the thumb and the index finger have already grasped the apple and other fingers have not collided, the software only reflects the condyle where the thumb and the index finger correspondingly collide, and only operates the brushless motor to be simulated. The whole hand of the operator is supposed to hold the apple, at this moment, because the elastic deformation of the apple, people's hand can receive reaction force in reality and make the finger unable to continue bending, therefore the miniature brushless motor of each joint of gloves also will rotate the drive worm and take power feedback right angle component to make the component reverse motion, hinder each condyle to continue moving, thereby prevent the bending of finger, reach the same effect with the actual condition, because in reality gravity effect, little finger and the ring finger of holding will receive bigger force, consequently the driving force that relevant miniature brushless motor provided also will be bigger, make the simulation finely go into a little, let the operator feel the concrete weight of apple. If the operator continues to hold the apple, the assumption force is enough, the apple exceeds elastic deformation, then the apple is cracked, all the miniature brushless motors on the glove holding the hand are stopped, no force is applied, and therefore the hands can move naturally and are not blocked, and the glove holding device is the same as reality. The above example of holding an apple is only considered, and the developer can program more carefully the specific feedback of different objects in the virtual space.
S6: the second case is: if the hands in the form are in the preset postures, the software extracts a glove parameter matrix generating instruction in the preset postures and returns the preset instruction. For example, at this time, the right hand is in a position of lying over the mouse, and the left glove micro brushless motor will stop working, i.e. in a weak state; in the virtual space, a mouse model is generated under the right hand according to the prone holding gesture, the right hand joint model is completely contacted with the mouse model at the moment, the right hand thumb and the ring finger collide with the mouse model when an operator continues bending, the mouse is made of an inelastic material, all the miniature brushless motors of the fingers above are reversely rotated to prevent the fingers from continuing to move, namely, each joint cannot be bent under the action of the miniature brushless motors; when the index finger and the middle finger of the right hand continue to bend, the first joint and the second joint of the index finger and the middle finger can continue to bend due to the keys of the mouse, and the root joint of the finger can not bend due to the action of the brushless motor. When the bending is continued beyond the limit of the key, the first two-joint motors of the index finger and the middle finger start to rotate reversely, so that the fingers cannot be bent continuously, the vibration motor of the palm of the hand vibrates to prompt the user that the key is started. By above operation, the touch of mouse clicking and holding when lying prone in the reality will be simulated completely to the gloves, reaches the requirement of this patent immersive operation. The mouse examples above are for reference only, and the developer may make other preset action settings through software.
According to the matrix Mt and the matrix Mtt, the computer software generates a real-time visible two-hand model for representing the states of the two hands of an operator in the virtual space in real time, and generates an invisible predicted two-hand model, wherein the model does not collide with the real-time model, and the other collisions are the same as the real-time model. Based on both of the foregoing, the software transmits commands for the real-time model and commands for the virtual model to the microprocessor unit, which will then operate the glove's micro brushless motor in accordance with the commands. If the matrix Mt of the data glove is the same as the previous predicted matrix, the previous pre-stored predicted command is directly executed, so that the time for transmitting the command into the glove from the computer software is saved. If the commands are different, calling the commands which are transmitted in real time. The predicted commands in each detection cycle will be stored, and only one predicted command is stored in total, so the predicted commands are updated in a constantly replaced form.
In the two cases, the calculation formula of the actual moment action of the finger according to the force generated by each micro brushless motor is as follows:
τ = M（Θ）Θ’’ + V(Θ,Θ’) + G(Θ)
where τ is the joint moment, M (Θ) is the n mass matrix, V (Θ, Θ) ’ ) G (Θ) is an n×1 gravity vector, which is an n×1 centrifugal force and a coriolis force vector. The above equation is the state space equation of the force feedback device. Note that Θ is a matrix of all angle parameters of the device, i.e. an n x 1 matrix for the n angles. Theta (theta) ’ Is the derivative of Θ with respect to time.
The simplified force feedback device according to fig. 4 is a schematic diagram for calculating joint moment, where m is the mass of the rod end, l is the length of the rod, θ is the angle, τ corresponds to the jointMoment, subscripts denote their corresponding levers. The force feedback device can be obtained through Newton-Euler recursion dynamics algorithm: (sin and cos functions will be abbreviated as s in the following formulas, c.e. c 2 Represents cos theta 2 Specific values of (2)
；
；
；
；
From the foregoing, it can be seen that the angle of the motor as a function of the joint angle, and thus Θ (here, θ 1 And theta 2 A structured 2 x 1 matrix) in particular as a function of time t, and Θ ’ ,Θ ’’ Is carried into a formula to calculate the moment tau of two joints 1 And τ 2 。
The pressure of each joint to the specific finger joint can be calculated, and the driving force of the miniature brushless motor can be regulated and controlled to control the force feedback effect of the glove.
S7: if not, all the micro brushless motors and the vibrating motors are not started by default, namely, the micro brushless motors do not provide any resistance, the hands of an operator change the posture randomly, and the vibrating motors do not vibrate.
S8: the power key is pressed for a long time, the system stops supplying power, the microprocessor unit clears the stored data, and the computer software deletes the hand model. The operation is ended.
Meanwhile, the invention also discloses a control method device of the multi-information somatosensory interaction system, which comprises a processor and a memory; the memory is configured to store instructions that, when executed by the processor, cause the apparatus to implement a control method system for a multiple information somatosensory interaction system as described in any one of the above.
Meanwhile, the invention also discloses a computer readable storage medium, wherein the storage medium stores computer instructions, and after the computer reads the computer instructions in the storage medium, the computer runs the control method system of the multi-information somatosensory interaction system.
In summary, in the prior art, due to the structural reason, it is impossible to have a separate driving element for each finger knuckle, so as to achieve accurate feedback. This patent is because every joint all is equipped with corresponding force feedback element, and gloves can all accurate control to the dynamics of every condyle of finger. Therefore, feedback of force of the simulation scene can be maximized, so that the glove can be suitable for the scene of force action caused by all the behaviors of actively or passively causing finger bending, such as using a virtual mouse, controlling a virtual rocker, taking virtual articles, simulating hand weight and the like.
Meanwhile, in the prior art, the element for detecting the rotation angle of the finger joint needs to be separated from the force feedback element, and the force feedback element is large in size, so that the structure is complex and the weight is large. Because the miniature brushless motor is adopted to carry out force feedback operation on the joint, the specific rotation angle of the miniature brushless motor can be detected only by the magnetic field detection element, so that the whole force feedback device occupies small space and is light. Meanwhile, as the magnetic field detection element is directly connected with the miniature brushless motor, the miniature brushless motor can quickly change the force, so that the vibration which cannot be simulated by the traditional motor is simulated, and the forces in different forms such as multistage force feedback, linear force feedback and the like are simulated. In addition, the miniature brushless motor can generate a force which always has the same large reverse direction as the force when the operator holds the object, so that the force which has the same large reverse direction as the force applied to the hand when the operator holds the object in reality can be simulated.
In the force feedback device, the material used at the position where the force feedback right-angle element is in direct contact with the finger is elastic plastic, so that the force generated by the maximum elastic deformation can not damage the finger, and accidents such as finger fracture and the like can be effectively avoided when a machine is in fault.
The large-area vibration motor used in the palm center of the invention is a linear motor, and the linear vibration motor is different from the rotor vibration motor, and can accurately vibrate at a certain point and simulate the touch feeling of water flowing through the palm center. With such a vibration motor, an operator will feel force feedback when the hand touches the virtual object while operating, and reflect to a certain extent that the object is fluid or solid. Meanwhile, the vibration motor can effectively prompt an operator.
The invention is provided with the loudspeaker at the wrist, and the loudspeaker can reflect the sound generated when certain hands operate objects, such as the broken sound of apples when the apples are held in the embodiment, thereby enhancing the immersion of operators and meeting the requirements of virtual reality sensory simulation.
The microprocessor unit of the invention adopts a machine learning algorithm for predicting the behavior of the user, so that the glove has self-adaptability and can be continuously attached to the habit of the user, thereby achieving the effect of customization.
According to the invention, as each joint is provided with the corresponding force feedback element, unlike the existing gesture recognition based on image recognition, the recognition mode can calculate the hand motion and specific gestures more accurately by directly calculating the coordinates of each joint by using the Euler angle and the transformation matrix T, and is not interfered by factors such as light rays.
The method for calculating the position can be achieved only by using the acceleration sensor, the calculation mode is simple and rapid, and compared with the traditional scheme, fewer sensors are needed, and parameters reflecting coordinates are more direct.
The specific hand inclination angle of the invention is directly reflected by the inclination angle sensor, so that the whole inclination angle of the hand can be accurately fed back.
The method for judging whether force feedback is needed or not comprises the steps of generating a specific hand model and adding a collision volume to interact. Compared with the traditional method for generating hand coordinates and measuring and calculating whether force feedback is needed through distance, the method can cope with all shapes, the material of collision volume between models can be modified according to the needs, and meanwhile, compared with the traditional method, the method is more accurate.
The invention pre-stores the instruction by using the prediction information, and compared with the instruction directly input from the computer, the time for transmitting the instruction from the computer software to the microprocessor unit is saved, so that the glove is more efficiently used, and the delay of the control equipment during input and output is effectively controlled.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
Claims (8)
1. The control method of the multi-information somatosensory interaction system is characterized in that the multi-information somatosensory interaction system comprises a glove body, a force feedback device, a vibration motor, an acceleration sensor, an inclination angle sensor and a microprocessor; the feedback device comprises a plurality of miniature brushless motors, worms, force feedback right-angle elements and magnetic field detection elements;
the method comprises the following steps:
acquiring an initial three-dimensional coordinate of a hand model;
through magnetic field detection, the rotation deflection angle alpha of the brushless motor is obtained 0 ；
Rotation deflection angle alpha based on brushless motor 0 And the diameter d of a single joint of the finger, determining the rotation angle of the single joint;
constructing a first matrix T describing a specific shape of the hand model based on the rotation angle of the single joint;
based on the first matrix T, determining the coordinates of each joint relative to the palm center through a first preset algorithm;
based on the data acquired by the acceleration sensor and the inclination sensor, determining the three-dimensional acceleration of the hand model and the inclination of the hand model around the three axes;
determining a real-time position of the hand model based on the three-dimensional acceleration of the hand model and the inclination angle of the hand model around the three axes;
determining a parameter information matrix M based on the coordinates of each joint relative to the palm center and the real-time position of the hand model;
based on the parameter information matrix M, predicting the motion of the finger to obtain a predicted parameter information matrix Mpt;
obtaining a real-time hand model based on the prediction parameter information matrix Mpt or the parameter information matrix M;
performing collision detection based on the real-time hand model, and extracting a position parameter and an inclination angle in a predicted parameter information matrix Mpt or a parameter information matrix M during collision detection so as to generate two hands in the form at corresponding positions in a virtual space;
extracting position, form and material data of an object in a preset virtual space, and rendering the virtual object according to the data to obtain a hand situation;
responding to the hand situation to meet a first preset condition or a second preset condition, and determining the actual force action of the fingers on the force generated by each motor based on a second preset algorithm; calculating the provided value of each joint based on Newton-Euler recursion dynamics algorithm;
the first preset condition is that an operator collides the models in the virtual space when grasping the object in the virtual space, and the collision is that the models of each bone segment are gradually detected; the second preset condition is that the hand information of the user is the same as the preset action.
2. The method of claim 1, wherein determining the rotation angle of the individual joint based on the rotation angle α0 of the brushless motor and the diameter d of the individual condyle of the finger comprises:
rotation deflection angle alpha based on brushless motor 0 Determining the offset kα of the relative initial position of the connection of the force-feedback right-angle element and the bottom end of the worm 0 ；
Based on the offset kα 0 The method comprises the steps of carrying out a first treatment on the surface of the Determining the rotation angle of the force feedback right-angle element by combining the diameter d of a single condyle of the finger;
the rotation angle of the individual joint is determined based on the rotation angle of the force feedback right angle element.
3. The method according to claim 2, characterized in that the first matrix T is represented as follows:
；
wherein beta is the rotation angle of the force feedback right-angle element, namely the rotation angle of a single joint of the finger; the horizontal subscript indicates the order from the thumb to the little finger, and the vertical subscript indicates the order from the root of the finger to the fingertip;
the determining, based on the first matrix and through a first preset algorithm, coordinates of each joint relative to the palm center includes:
establishing a three-dimensional coordinate system by taking a palm center as an origin, comprising: for an index finger, establishing the three-dimensional coordinate system at each knuckle; two degrees of freedom exist from the root to the second knuckle; expanding the T matrix to
Taking out a second column matrix in the T matrix, wherein beta 24 is a rotation parameter around the z axis, and beta 23 is a rotation parameter around the x axis;
based on euler angle rotation formula 1 and euler angle rotation formula 2:
（1）
（2）
substituting beta 24 into formula 1 and substituting beta 23 into formula 2 to obtain two rotation matrices:
rotating matrix 1:rotating matrix 2: />；
Wherein P is the coordinate of the origin of the coordinate system on the knuckle under the coordinate system of the finger root;
thereby, a conversion matrix can be obtained；
Coordinates of the second knuckle relative to the palm =The second knuckle corresponds to the root coordinate;
other transformation matrices and thus coordinates to the relative palm of each joint can be obtained in the same way.
4. A method according to claim 3, wherein coordinates of each joint are sequentially connected in the prediction parameter information matrix Mpt or the parameter information matrix M, and the connection lines are bone segments of the finger, so as to determine a specific bone posture, and after the parameters of the finger and the finger diameter of the person are estimated, the model of the opponent can be rendered, and a real-time hand model is obtained.
5. The method according to claim 1, characterized in that, under said first preset conditions:
the whole hand of operator's right hand holds the object, because the elastic deformation of object this moment, people's hand can receive reaction force in reality and make the finger unable to continue crooked, consequently, the brushless motor of each joint of gloves also will rotate drive worm and take power feedback right angle component to make the component reverse motion, hinder each condyle to continue to remove, thereby prevent the crooked of finger, reach the same effect with the reality condition, because gravity effect, little finger and the ring finger of holding will receive bigger force, consequently the driving force that relevant brushless motor provided also will be bigger, make the simulation finely go into a little, let the operator feel the concrete weight of object.
6. The method of claim 5, wherein, under the first preset condition: when the force applied by an operator exceeds the elastic deformation range of the object, the feedback object is damaged, all brushless motors on the glove holding the hand are stopped, and at the moment, the hands can naturally move without being blocked.
7. The method of claim 6, wherein, under the second preset condition:
and generating a real-time visible two-hand model for representing the real-time states of the two hands of the operator in the virtual space according to the predicted parameter information matrix Mpt or the parameter information matrix M, and generating an invisible predicted two-hand model, wherein the model does not collide with the real-time model, and the other collisions are the same as the real-time model.
8. The method of claim 6, wherein the method further comprises:
transmitting a command for the real-time model and a command for the virtual model to the microprocessor unit in response to the hand condition satisfying the first preset condition or the second preset condition, and then the microprocessor unit operating the motor of the glove in accordance with the command;
in response to the parameter information matrix M of the glove being the same as the last predicted parameter information matrix Mpt, the last pre-stored predicted command is directly executed,
responding to the fact that the parameter information matrix M of the glove is different from the previous predicted parameter information matrix Mpt, and calling a command transmitted in real time;
the predicted commands in each detection cycle will be stored, and only one predicted command is stored in total, so the predicted commands are updated in a constantly replaced form.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311506619.5A CN117251058B (en) | 2023-11-14 | 2023-11-14 | Control method of multi-information somatosensory interaction system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311506619.5A CN117251058B (en) | 2023-11-14 | 2023-11-14 | Control method of multi-information somatosensory interaction system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117251058A CN117251058A (en) | 2023-12-19 |
| CN117251058B true CN117251058B (en) | 2024-01-30 |
Family
ID=89129766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311506619.5A Active CN117251058B (en) | 2023-11-14 | 2023-11-14 | Control method of multi-information somatosensory interaction system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117251058B (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000132305A (en) * | 1998-10-23 | 2000-05-12 | Olympus Optical Co Ltd | Operation input device |
| JP2004213350A (en) * | 2002-12-27 | 2004-07-29 | Seiko Epson Corp | Inner force sense presenting device and image correcting method |
| KR20070061255A (en) * | 2005-12-08 | 2007-06-13 | 한국전자통신연구원 | Hand interface device using miniaturized absolute position sensors, system for hand interface using it |
| CN105677036A (en) * | 2016-01-29 | 2016-06-15 | 清华大学 | Interactive type data glove |
| CN105818129A (en) * | 2016-04-12 | 2016-08-03 | 华南理工大学 | Humanoid hand control system based on data glove |
| CN106527738A (en) * | 2016-12-08 | 2017-03-22 | 东北大学 | Multi-information somatosensory interaction glove system and method for virtual reality system |
| CN108196686A (en) * | 2018-03-13 | 2018-06-22 | 北京无远弗届科技有限公司 | A kind of hand motion posture captures equipment, method and virtual reality interactive system |
| CN108214452A (en) * | 2016-12-21 | 2018-06-29 | 深圳市肯綮科技有限公司 | A kind of power joint arrangement for ectoskeleton |
| CN108958471A (en) * | 2018-05-17 | 2018-12-07 | 中国航天员科研训练中心 | The emulation mode and system of virtual hand operation object in Virtual Space |
| CN109669532A (en) * | 2018-11-01 | 2019-04-23 | 深圳岱仕科技有限公司 | The control method and system of hand mechanical exoskeleton force feedback |
| CN110605714A (en) * | 2019-08-06 | 2019-12-24 | 华中科技大学 | Hand-eye coordination grabbing method based on human eye fixation point |
| CN111552383A (en) * | 2020-04-24 | 2020-08-18 | 南京爱奇艺智能科技有限公司 | Finger identification method and system of virtual augmented reality interaction equipment and interaction equipment |
| CN111645093A (en) * | 2020-05-22 | 2020-09-11 | 北京工业大学 | Force sense feedback data glove for teleoperation |
| CN115016645A (en) * | 2022-06-15 | 2022-09-06 | 哈尔滨工业大学 | Multi-degree-of-freedom acquired data glove for cooperative control of artificial fingers |
| CN115761787A (en) * | 2022-09-27 | 2023-03-07 | 西北工业大学 | Hand gesture measuring method with fusion constraints |
| CN116442283A (en) * | 2023-05-26 | 2023-07-18 | 南京信息工程大学 | Finger joint angle detection mechanism and object three-dimensional model construction method |
| CN116610211A (en) * | 2023-04-17 | 2023-08-18 | 清华大学 | Finger state sensing glove based on magneto-sensitive sensor |
-
2023
- 2023-11-14 CN CN202311506619.5A patent/CN117251058B/en active Active
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000132305A (en) * | 1998-10-23 | 2000-05-12 | Olympus Optical Co Ltd | Operation input device |
| JP2004213350A (en) * | 2002-12-27 | 2004-07-29 | Seiko Epson Corp | Inner force sense presenting device and image correcting method |
| KR20070061255A (en) * | 2005-12-08 | 2007-06-13 | 한국전자통신연구원 | Hand interface device using miniaturized absolute position sensors, system for hand interface using it |
| CN105677036A (en) * | 2016-01-29 | 2016-06-15 | 清华大学 | Interactive type data glove |
| CN105818129A (en) * | 2016-04-12 | 2016-08-03 | 华南理工大学 | Humanoid hand control system based on data glove |
| CN106527738A (en) * | 2016-12-08 | 2017-03-22 | 东北大学 | Multi-information somatosensory interaction glove system and method for virtual reality system |
| CN108214452A (en) * | 2016-12-21 | 2018-06-29 | 深圳市肯綮科技有限公司 | A kind of power joint arrangement for ectoskeleton |
| CN108196686A (en) * | 2018-03-13 | 2018-06-22 | 北京无远弗届科技有限公司 | A kind of hand motion posture captures equipment, method and virtual reality interactive system |
| CN108958471A (en) * | 2018-05-17 | 2018-12-07 | 中国航天员科研训练中心 | The emulation mode and system of virtual hand operation object in Virtual Space |
| CN109669532A (en) * | 2018-11-01 | 2019-04-23 | 深圳岱仕科技有限公司 | The control method and system of hand mechanical exoskeleton force feedback |
| CN110605714A (en) * | 2019-08-06 | 2019-12-24 | 华中科技大学 | Hand-eye coordination grabbing method based on human eye fixation point |
| WO2021023315A1 (en) * | 2019-08-06 | 2021-02-11 | 华中科技大学 | Hand-eye-coordinated grasping method based on fixation point of person's eye |
| CN111552383A (en) * | 2020-04-24 | 2020-08-18 | 南京爱奇艺智能科技有限公司 | Finger identification method and system of virtual augmented reality interaction equipment and interaction equipment |
| CN111645093A (en) * | 2020-05-22 | 2020-09-11 | 北京工业大学 | Force sense feedback data glove for teleoperation |
| CN115016645A (en) * | 2022-06-15 | 2022-09-06 | 哈尔滨工业大学 | Multi-degree-of-freedom acquired data glove for cooperative control of artificial fingers |
| CN115761787A (en) * | 2022-09-27 | 2023-03-07 | 西北工业大学 | Hand gesture measuring method with fusion constraints |
| CN116610211A (en) * | 2023-04-17 | 2023-08-18 | 清华大学 | Finger state sensing glove based on magneto-sensitive sensor |
| CN116442283A (en) * | 2023-05-26 | 2023-07-18 | 南京信息工程大学 | Finger joint angle detection mechanism and object three-dimensional model construction method |
Non-Patent Citations (2)
| Title |
|---|
| Unity game engine: interactive software design using digital glove for virtual reality baseball pitch training;Juan Zhou;《TECHNICAL PAPER》;全文 * |
| 基于Kinect体感交互技术的虚拟装配实验系统开发;廖宏建;曲哲;;实验技术与管理(第07期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117251058A (en) | 2023-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10534431B2 (en) | Tracking finger movements to generate inputs for computer systems | |
| CN110096131B (en) | Touch interaction method and device and touch wearable equipment | |
| KR101548156B1 (en) | A wireless exoskeleton haptic interface device for simultaneously delivering tactile and joint resistance and the method for comprising the same | |
| RU187548U1 (en) | VIRTUAL REALITY GLOVE | |
| RU179301U1 (en) | VIRTUAL REALITY GLOVE | |
| KR100934391B1 (en) | Hand-based Grabbing Interaction System Using 6-DOF Haptic Devices | |
| Sanfilippo et al. | A low-cost multi-modal auditory-visual-tactile framework for remote touch | |
| US20230142242A1 (en) | Device for Intuitive Dexterous Touch and Feel Interaction in Virtual Worlds | |
| KR20200051938A (en) | Method for controlling interaction in virtual reality by tracking fingertips and VR system using it | |
| RU2670649C9 (en) | Method of manufacturing virtual reality gloves (options) | |
| Zhu et al. | TapeTouch: A handheld shape-changing device for haptic display of soft objects | |
| CN117251058B (en) | Control method of multi-information somatosensory interaction system | |
| Evreinova et al. | From kinesthetic sense to new interaction concepts: Feasibility and constraints | |
| CN117572965A (en) | Multi-information somatosensory interactive glove system for virtual reality system | |
| CN109960404A (en) | A kind of data processing method and device | |
| Covaciu et al. | Development of a virtual reality simulator for a lower limb rehabilitation robot | |
| WO2023236416A1 (en) | Fingertip texture tactile feedback device based on airbag driving | |
| Jyothi et al. | Haptic technology-a sense of touch | |
| US11003247B1 (en) | Deployable controller | |
| Shanmugam et al. | A Comprehensive Review of Haptic Gloves: Advances, Challenges, and Future Directions | |
| RU2673406C1 (en) | Method of manufacturing virtual reality glove | |
| JP6905384B2 (en) | Force presenter and virtual force presenter | |
| KR101605740B1 (en) | Method for recognizing personalized gestures of smartphone users and Game thereof | |
| Efraimidis et al. | Wireless embedded system on a glove for hand motion capture and tactile feedback in 3d environments | |
| CN212032115U (en) | Virtual reality sensing equipment and virtual reality system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |