CN112256136A

CN112256136A - Tactile feedback device and method, electronic equipment and man-machine interaction system

Info

Publication number: CN112256136A
Application number: CN202011270136.6A
Authority: CN
Inventors: 魏磊; 张东胜; 潘汤杰; 陈科
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-11-13
Filing date: 2020-11-13
Publication date: 2021-01-22
Anticipated expiration: 2040-11-13
Also published as: CN112256136B

Abstract

The application discloses a tactile feedback device, a tactile feedback method, electronic equipment and a man-machine interaction system. The tactile feedback device includes a gripping mechanism and a tactile feedback mechanism. The tactile feedback mechanism comprises a torque simulation component, an impulse simulation component, a friction simulation component or a vibration simulation component, and the force feedback modes corresponding to the simulation components are different; the holding mechanism is used for providing a holding position for a user. The tactile feedback mechanism is arranged on the corresponding holding position based on the force feedback modality, so that the user receives the tactile feedback mechanism and sends out corresponding tactile feedback based on the operation action of the user, and the multi-modal tactile feedback process is realized. The tactile feedback device can comprehensively simulate various force perceptions possibly received by the lower arm of a user, and improves the completeness of tactile feedback through limited simulation component combinations.

Description

Tactile feedback device and method, electronic equipment and man-machine interaction system

Technical Field

The present application relates to the field of electronic technologies, and in particular, to a tactile feedback device, a tactile feedback method, an electronic device, and a human-computer interaction system.

Background

With the continuous progress of science and technology, technologies such as virtual reality, augmented reality and the like make great breakthrough; the vision related technology is relatively mature, and the telepresence of the user can be further enhanced only by matching the force technology with the vision technology. The tactile feedback device in the force sense equipment can enable a user to better interact with the virtual world. By providing haptic feedback in response to a user's actions, the user's perception in some scenarios in the virtual world may be simulated.

In the process of the user interacting with the virtual world, the user performs tactile feedback on some actions, such as moment simulation through a tactile feedback device when the user swings an arm, and the feedback is performed.

However, in a specific interaction process, the user's actions are complex, the above-mentioned haptic feedback device cannot perform haptic feedback in different force feedback modalities to simulate various perceptions received by the user in a virtual scene, and the above-mentioned haptic feedback device has the problems of too large volume and device redundancy, so that it is not possible to perform relatively complete simulation on human perception.

Disclosure of Invention

In view of the above, the present application provides a haptic feedback device that improves the completeness of haptic feedback through a limited combination of analog components.

The first aspect of the present application provides a haptic feedback device, which may be applied in a human-computer interaction scenario, specifically including:

a holding mechanism and a tactile feedback mechanism;

the tactile feedback mechanism comprises a torque simulation component, an impulse simulation component, a friction simulation component and a vibration simulation component, wherein the torque simulation component, the impulse simulation component, the friction simulation component and the vibration simulation component are different in corresponding force feedback modes;

the holding mechanism is used for providing a holding position for a target user, and the tactile feedback mechanism is arranged on the corresponding holding position based on the force feedback modality, so that the target user receives the tactile feedback mechanism and sends out corresponding tactile feedback based on the operation action of the target user.

Optionally, in some possible implementations of the present application, the torque simulation module includes:

the device comprises a frame, a first motor, a transmission shaft and a momentum wheel;

the first motor and the momentum wheel are connected through the transmission shaft, and the transmission shaft penetrates through the support part of the rack, so that the first motor and the momentum wheel are arranged on two sides of the rack;

the first motor drives the momentum wheel to rotate through the transmission shaft so as to generate torque feedback.

Optionally, in some possible implementations of the present application, the impulse simulation component includes:

the first screw rod, the sliding block, the first bearing seat and the second motor;

the first bearing seat is fixed on the inner surface of the holding mechanism, the first screw rod is fixed inside the holding mechanism through the first bearing seat, the sliding block is connected with the first screw rod, and the first screw rod is connected with the second motor;

the second motor drives the sliding block to move through the first screw rod so as to generate impulse feedback.

Optionally, in some possible implementations of the present application, the impulse simulation component further includes:

a slide rail;

the slide rail is fixed inside the holding mechanism and corresponds to the first screw rod;

the slide rail is used for supporting the slide block to move.

Optionally, in some possible implementations of the present application, the friction simulation module includes:

the sliding piece, the moving part, the second bearing seat and the third motor;

the slide sheet is connected with the moving part, the moving part is fixed in the holding mechanism through the second bearing seat, and the moving part is connected with the third motor;

the third motor drives the sliding sheet to move through the moving part so as to generate friction feedback.

Optionally, in some possible implementations of the present application, the moving part includes:

the second screw rod, the nut and the sliding piece connector;

the second screw rod is fixed on the holding mechanism through the second bearing seat, the nut is connected with the second screw rod, the nut is connected with the slip sheet connector through a moving groove in the holding mechanism, and the slip sheet connector is connected with the slip sheet;

the third motor drives the nut to move through the second lead screw, and the nut drives the sliding sheet to move so as to generate the friction feedback.

Optionally, in some possible implementations of the present application, the moving part includes one of a linear motor, a spring, a timing belt, or a gear device.

Optionally, in some possible implementations of the present application, the friction simulation assembly includes a plurality of sliding pieces, and the sliding pieces are configured based on the holding position;

different sliding sheets are connected with each other, so that the moving part drives the sliding sheets to move simultaneously.

Optionally, in some possible implementations of the present application, the vibration simulation module includes:

at least one vibration unit;

wherein the vibration unit is disposed outside the holding mechanism, the vibration unit being in contact with the target user;

the vibration unit is used for responding to the position information of the target user holding the tactile feedback device and emitting vibration of a target frequency to generate vibration feedback.

Optionally, in some possible implementation manners of the present application, the vibration units are arranged in a target array, and the target array wakes up the vibration units in different quantities based on the position information to generate the vibration feedback.

Optionally, in some possible implementations of the present application, the haptic feedback device further includes:

a routing limit component;

the routing limiting part is arranged on the holding mechanism;

the routing limiting part is used for limiting a circuit in the tactile feedback device so as to avoid the influence of the tactile feedback mechanism and the circuit.

Optionally, in some possible implementations of the present application, the torque simulation component is disposed at an end point of the holding mechanism, the impulse simulation component is disposed inside the holding mechanism, the friction simulation component is disposed outside the holding mechanism for performing a tactile feedback, and the vibration simulation component is disposed at a position corresponding to a palm of a hand of the target user when the target user holds the holding mechanism.

A second aspect of the present application provides a haptic feedback method comprising:

acquiring a first target action of a target user;

acquiring a first virtual scene;

generating a multi-modal first feedback instruction according to the first target action and the first virtual scene;

controlling the haptic feedback device of any one of claims 1-12 for haptic feedback according to the first feedback instruction.

Optionally, in some possible implementations of the present application, the method further includes:

acquiring a second target action of the target user;

generating a second virtual scene according to the first target action and the first virtual scene;

generating a multi-modal second feedback instruction according to the second target action and the second virtual scene;

controlling the haptic feedback device of any one of claims 1-12 for haptic feedback according to the second feedback instruction.

Optionally, in some possible implementations of the present application, wherein:

the first target action is a waving action, the waving action corresponds to a chopping action in the first virtual scene and is not chopped into a target virtual object, and the first feedback instruction comprises a moment parameter and a friction parameter determined based on the first target action;

the second target action is used as the waving action, the waving action corresponds to a chopping action in the second virtual scene and is chopped to a target virtual object, and the second feedback instruction comprises vibration parameters determined based on the second stage action.

Optionally, in some possible implementations of the present application, the generating a multi-modal first feedback instruction according to the first target motion and the first virtual scene includes:

determining impulse parameters according to the first virtual scene and the lifting action;

acquiring an impulse threshold corresponding to the first virtual scene;

if the impulse parameter is larger than the impulse threshold, determining a vibration parameter corresponding to the impulse parameter according to the impulse parameter;

and generating the first feedback instruction based on the impulse parameter and the vibration parameter corresponding to the impulse parameter.

acquiring a plurality of first target actions in a preset time period to obtain a target action set;

determining a set of feedback instructions in the first virtual scene based on the set of target actions;

determining a first sub-feedback and a second sub-feedback based on the set of feedback instructions;

sequentially controlling the haptic feedback device of any one of the above-mentioned aspects to perform the first sub-feedback and the second sub-feedback.

Optionally, in some possible implementations of the present application, the sequentially controlling the haptic feedback device according to any one of the above-mentioned aspects to perform the first sub-feedback and the second sub-feedback includes:

detecting a feedback parameter in the first sub-feedback execution process;

and if the feedback parameter is larger than a preset value, controlling the haptic feedback device of any one of the above aspects to execute the second sub-feedback.

A third aspect of the present application provides an electronic device comprising: an interaction device, an input/output (I/O) interface, a processor, and a memory, the memory having stored therein program instructions, the processor for executing the program instructions stored in the memory;

the interaction device is used for acquiring an operation instruction input by a user; wherein the interaction means comprises a haptic feedback device as described in any one of the above aspects.

A fourth aspect of the present application provides a human-computer interaction system, comprising: the device comprises an operating device, a processor, an input/output (I/O) interface, a memory and an interaction controller, wherein the interaction controller and the operating device are respectively connected with the memory and the processor through the input/output (I/O) interface, and the operating device is used for controlling the movement of the interaction controller;

the operating device is used for acquiring an operating instruction input by a user, wherein the operating device comprises the tactile feedback device according to any one of the above aspects.

According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the haptic feedback apparatus provided in the first aspect or the various alternative implementations of the first aspect.

According to the technical scheme, the embodiment of the application has the following advantages:

the tactile feedback device includes a gripping mechanism and a tactile feedback mechanism. The tactile feedback mechanism comprises a torque simulation component, an impulse simulation component, a friction simulation component or a vibration simulation component, and the force feedback modes corresponding to the simulation components are different; the holding mechanism is used for providing a holding position for a target user. The tactile feedback mechanism is arranged on the corresponding holding position based on the force feedback modality, so that the target user receives the tactile feedback mechanism and sends out corresponding tactile feedback based on the operation action of the target user. The haptic feedback corresponds to a force feedback modality, thereby implementing a multi-modal haptic feedback process. Because the simulation components of the four force feedback modes are adopted for haptic feedback, the haptic feedback device can comprehensively simulate various force senses possibly received by the forearm of a user, and the completeness of the haptic feedback is improved through the limited combination of the simulation components.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a diagram of a network architecture for operation of a human-computer interaction system;

FIG. 2 is a schematic structural diagram of a haptic feedback device according to an embodiment of the present application;

FIG. 3 is a schematic diagram of another haptic feedback device provided in accordance with an embodiment of the present application;

FIG. 4 is a schematic diagram of another haptic feedback device provided in accordance with an embodiment of the present application;

FIG. 5 is a schematic diagram of another haptic feedback device provided in accordance with an embodiment of the present application;

FIG. 6 is a schematic diagram of another haptic feedback device provided in accordance with an embodiment of the present application;

FIG. 7 is a flow chart of another haptic feedback method provided in embodiments of the present application;

fig. 8 is a schematic view of a scene of a haptic feedback method according to an embodiment of the present application;

FIG. 9 is a schematic view of another embodiment of a haptic feedback method according to the present disclosure;

FIG. 10 is a schematic view of another embodiment of a haptic feedback method according to the present disclosure;

fig. 11 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

The embodiment of the application provides a tactile feedback device, a tactile feedback method, electronic equipment and a human-computer interaction system, and can be applied to a human-computer interaction scene. The haptic feedback device includes a grip mechanism and a haptic feedback mechanism. The tactile feedback mechanism comprises a torque simulation component, an impulse simulation component, a friction simulation component or a vibration simulation component, and the force feedback modes corresponding to the simulation components are different; the holding mechanism is used for providing a holding position for a target user. The tactile feedback mechanism is arranged on the corresponding holding position based on the force feedback modality, so that the target user receives the tactile feedback mechanism and sends out corresponding tactile feedback based on the operation action of the target user. The haptic feedback corresponds to a force feedback modality, thereby implementing a multi-modal haptic feedback process. Because the simulation components of the four force feedback modes are adopted for haptic feedback, the haptic feedback device can comprehensively simulate various force senses possibly received by the forearm of a user, and the completeness of the haptic feedback is improved through the limited combination of the simulation components.

The terms first, second, third and fourth in the description and in the claims of the present application and the accompanying drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so described are interchangeable under appropriate circumstances such that the embodiments described herein are capable of operation in other sequences than illustrated or otherwise described herein. Furthermore, unless otherwise indicated, the terms "comprise" and "correspond to" and any variations thereof are intended to cover open-ended inclusion relationships. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements specifically listed, but may include other steps or elements not specifically listed at the same time.

The tactile feedback device provided by the application can be applied to a human-computer interaction system. As shown in fig. 1 by way of example, it is a network architecture diagram of the operation of the man-machine interaction system. As can be seen from the figure, the human-computer interaction system can provide a tactile feedback process with multiple information sources, that is, a user interacts with a terminal through a tactile feedback device to generate a corresponding action command, and uploads the action command to a server for analysis, so as to obtain a corresponding action feedback, and responds based on the tactile feedback device. It is understood that fig. 1 illustrates various terminal devices, the terminal devices may be computer devices, and in an actual scene, there may be more or less types of terminal devices participating in the process of haptic feedback, and the specific number and types are determined by the actual scene and are not limited herein. In addition, fig. 1 shows only one server, but in an actual scenario, a plurality of servers may participate, and the specific number of servers depends on the actual scenario.

In this embodiment, the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a CDN, a big data and artificial intelligence platform, and the like. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through a wired or wireless communication manner, and the terminal and the server may be connected to form a block chain network, which is not limited herein.

It is understood that the human-computer interaction system can be operated in a personal mobile terminal, a server, and a third-party device for providing tactile feedback. The specific human-computer interaction system may be operated in the above-mentioned device in the form of a program, may also be operated as a system component in the above-mentioned device, and may also be used as one of cloud service programs, and the specific operation mode is determined by an actual scene, and is not limited herein.

It should be noted that the human-computer interaction system provided in the embodiment of the present application can be applied to various different usage scenarios, for example: a human-computer interaction controller in a Virtual Reality (VR) technology or the operation force of a remote end or a Virtual world is fed back to a user at proper time, so that the force sense and the telepresence of an operator are enhanced. The present embodiment is not limited to this.

Further, the device provided by the embodiment of the present application can also be used in the field of Artificial Intelligence (AI). The AI technique is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. The artificial intelligence technology is a comprehensive subject and relates to the field, the technology of the hardware level and the technology of the software level exist, wherein an operation/interaction system is one of the basic technologies of the artificial intelligence, and in the hardware level, the tactile feedback device provided by the embodiment of the application can be applied to the man-machine interaction of an AI system. Artificial intelligence can make a machine have the functions of perception, reasoning and decision making. The machine can generate a haptic feedback decision by itself through the AI and feed back to the user through the haptic feedback device provided by the embodiment of the application, so that the AI system can interact with the user at a hardware level.

Haptic feedback technology (Haptic feedback) can reproduce the tactile sensation for a user through a series of actions such as force, vibration, etc. This reproduced tactile sensation may be applied to aid in the creation and control of virtual scenes or virtual objects in computer simulations, as well as to enhance remote manipulation of machinery and equipment.

With the continuous progress of science and technology, technologies such as virtual reality, augmented reality and the like make great breakthrough, wherein vision related technologies are relatively mature. Only if the force technology is matched with the vision technology, the telepresence of the user can be further enhanced. The tactile feedback device in the force sense equipment not only enables a user to better interact with the virtual world, but also can endow the user with more real operation sense of the remote control robot. By providing haptic feedback in response to a user's actions, the user's perception in some scenarios in the virtual world may be simulated.

In order to solve the above problems, the present application proposes a haptic feedback device. The haptic feedback device includes a grip mechanism and a haptic feedback mechanism. The holding mechanism is used for providing a holding position for a target user, namely, the user controls the tactile feedback device through hand holding or other holding modes. The tactile feedback mechanism is arranged on the corresponding holding position based on the force feedback modality, so that the target user receives the tactile feedback mechanism and sends out corresponding tactile feedback based on the operation action of the target user. The haptic feedback corresponds to a force feedback modality, for example, when the force feedback modality is a friction force, the corresponding gripping positions are both sides of the gripping mechanism.

Specifically, the tactile feedback mechanism includes at least two of a torque simulation component, an impulse simulation component, a friction simulation component, and a vibration simulation component, that is, the tactile feedback mechanism may be a combination of any two or more of the above simulation components. The following description will be given taking a tactile feedback mechanism as an example of a combination of all the above analog components.

The torque simulation assembly, the impulse simulation assembly, the friction simulation assembly and the vibration simulation assembly are different in corresponding force feedback modes. The torque simulation component simulates the force feedback effect of the torque of the wrist of the user, the impulse simulation component simulates the force feedback effect of impulse and gravity center movement, the friction simulation component simulates the force feedback effect of friction, and the vibration simulation component simulates the force feedback effect of vibration with spatial resolution.

It can be understood that, when the user obtains the feedback perception of the virtual scene, the user mainly perceives the holding of the tactile feedback device by the hand, that is, the user perceives in a contact manner; and the small arm is used as moment to sense, namely the sensing of motion. Specifically, the feedback mode corresponding to the contact sensing process mainly includes friction and vibration; the feedback mode corresponding to the motion perception is mainly set based on the motion direction, the motion direction can be analyzed into a transverse variable and a longitudinal variable, the feedback mode corresponding to the transverse variable can be simulated through impulse feedback, and the feedback mode corresponding to the longitudinal variable can be simulated through moment feedback. Therefore, in the present embodiment, the above-mentioned various possible sensing modes are integrally designed, so that the combination of the above-mentioned four components comprehensively simulates the mechanical sensing of the user's forearm and the following parts, the possible sensing forms of the user are analyzed into several modes, and the simulation is respectively performed by the corresponding mode simulation mechanisms, thereby perceptually and accurately simulating the combined force feedback effect generated when the user performs the actions of chopping, stabbing, hitting, blocking, pulling, etc. by using the sword, the stick, the sports equipment, etc.

It should be noted that the feedback perception corresponding to the above actions is only an example, and other actions that can generate the above feedback perception can also be applied in the present embodiment; in addition, the above-mentioned multiple actions may be combined to perform a composite feedback process of multiple feedback perceptions, for example, for a virtual scene of splitting a trunk, in addition to torque (moment feedback), momentum for removing hands (momentum feedback), friction (friction feedback) during waving, and even vibration (vibration feedback) after splitting the trunk may be simulated together.

The tactile feedback mechanism is illustrated in the following with reference to the accompanying drawings. Fig. 2 is a schematic structural diagram of a haptic feedback device according to an embodiment of the present application. The internal structure of the haptic feedback device is shown, including a gripping mechanism 100, a torque simulation assembly 200, an impulse simulation assembly 300, and a friction simulation assembly 400.

The torque simulator assembly 200 is located in one embodiment at an end of the handle mechanism 100 remote from the handle position. Setting the torque simulator assembly 200 in this position, i.e., the position shown in fig. 2, can increase the perceived intensity of torque feedback by increasing the distance, thereby increasing the energy utilization efficiency, while consuming the same amount of energy. However, in practical applications, the torque simulation assembly 200 may be disposed at other positions of the holding mechanism 100, such as the middle portion of the holding mechanism 100, for other reasons.

The impulse simulation module 300 is shown as being disposed inside the gripping mechanism 100, and in an actual scene, the impulse simulation module 300 is disposed at another part of the gripping mechanism 100, which is not limited herein. However, considering that there is a movement process of the object (for example, sliding of the slider) during the impulse simulation of the impulse simulation module 300, the impulse simulation module 300 needs to be placed in a closed environment to perform the movement process of the object, so as to ensure that the impulse simulation is not interfered by external factors, and to prevent the user from being hit by the moving object during the holding process, so that the impulse simulation module 300 can be placed inside the holding mechanism 100 to achieve the above-mentioned purposes.

Further, the friction simulating assemblies 400 are shown on both sides of the handle mechanism 100, which is positioned so that the user will often apply more pressure to the handle mechanism 100 when holding it. The friction force can be increased by increasing the contact area, the effect of saving electric energy under the condition of generating the friction force feedback with the same strength is achieved, namely the friction force works the most under the condition that the friction simulation assembly 400 performs the same displacement, and the friction feedback of the tactile feedback device to a user is obvious. In other practical applications, the friction simulation assembly 400 may be disposed on the holding mechanism 100 at other positions where the user contacts the holding mechanism.

In other embodiments, more or fewer friction simulation assemblies 400 may be provided in the haptic feedback device, depending on the actual scene.

The vibration simulation module 500 is explained below. Fig. 3 is a schematic structural diagram of a haptic feedback device according to another embodiment of the present application. A vibration simulating assembly 500 is shown disposed on the housing of the gripping mechanism 100. Placing the vibration simulator assembly 500 within the housing of the gripping mechanism 100 allows closer proximity to the user and less attenuation of the vibration effect, ensuring the strength of the vibratory force feedback that the user can feel. In practical applications, the vibration simulation assembly 500 may be disposed at other positions, such as integrated inside the holding mechanism 100, for other reasons, which are not limited herein.

Next, a possible composition manner of each analog component is described, and as shown in fig. 4, a schematic structural diagram of another haptic feedback device provided in the embodiments of the present application is provided.

For one embodiment of the torque simulation assembly 200, comprising:

a frame 201, a first motor 204, a transmission shaft 203 and a momentum wheel 202; wherein the first motor 204 and the momentum wheel 202 are connected by a transmission shaft 203, and the transmission shaft 203 passes through the support portion of the frame 201, so that the first motor 204 and the momentum wheel 202 are disposed at both sides of the frame 201.

Specifically, a "T" shaped structure may be adopted for the frame 201, that is, the supporting portion is located on a connecting plane of the frame 201 and the holding mechanism 100, and the connecting plane may be a housing of the holding mechanism 100, for example, a housing at the top end of the holding mechanism 100; it may be a support surface of the frame 201, for example, a planar member (support surface) screwed to the frame 201 of the holding mechanism 100. In addition, the supporting part can be designed in a rectangular shape or a circular shape, and the specific shape is determined by the actual scene.

During operation, the first motor 204 rotates the momentum wheel 202 via the drive shaft 203 to generate torque feedback. For example, when a user performs a chopping action, the momentum wheel 202 is driven by the first motor 204 through the transmission shaft 203 to rotate in a direction opposite to the chopping action, so as to generate a reverse torque, thereby simulating a torsion force applied to a wrist of the user during the chopping action.

It is understood that other analog components that can perform force feedback in a vertical direction with respect to the user's forearm may also be applied in the present embodiment, and are not limited herein.

For one embodiment of the impulse simulation assembly 300, it comprises a first lead screw 301, a slider 302, a first bearing housing 305, and a first motor 303; the first bearing seat 305 is fixed on the inner surface of the holding mechanism 100, the first lead screw 301 is fixed inside the holding mechanism 100 through the first bearing seat 305, the slider 302 is connected with the first lead screw 301, and the first lead screw 301 is connected with the first motor 303; specifically, in order to ensure that the sliding block 302 is not interfered during the sliding process, the first lead screw 301 may be disposed on a transverse center line of an internal space of the holding mechanism 100, so as to increase the sliding space of the sliding block 302, and in order to save the internal space of the holding mechanism 100, the first motor 303 may be disposed outside the holding mechanism 100, and may be disposed at an end point of the holding mechanism 100 by using a connection manner such as a threaded connection, a snap connection, and the like, and the end point and the top end where the frame 201 is disposed are different ends of the holding mechanism 100.

In the feedback process, the first motor 303 drives the sliding block 302 to move through the first lead screw 301, so as to generate impulse feedback. For example, when a user performs a pulling action, the first motor 303 drives the sliding block 302 to move in the opposite direction of the pulling direction through the first lead screw 301, so as to simulate feedback of pulling a heavy object. In addition, the moving direction of the slider 302 may also be the same as the moving direction of the object in the virtual scene, for example, in a scene formed by a pet, a user needs to pull the virtual pet, and in order to simulate the pulling of the virtual pet in the pulling process, the first motor 303 may be controlled to drive the slider 302 to move in the same direction as the moving direction of the virtual pet through the first lead screw 301, so as to simulate the pulling feeling of the virtual pet.

Optionally, the impulse simulation module 300 further includes: a slide rail 304; the slide rail 304 is fixed inside the holding mechanism 100, and corresponds to the first screw rod 301; the slide rail 304 is used for supporting the slider 302 for movement. Therefore, the motion stability of the sliding block 302 is improved, the accuracy of impulse feedback is ensured, and the influence on the simulation of other forces due to the shaking of the sliding block 302 is avoided.

Optionally, in order to ensure the stability of the power transmission between the first motor 303 and the first lead screw 301, the impulse simulation assembly 300 further includes a first coupler 306. The first coupling 306 is a device for coupling two shafts or a shaft and a rotating member to rotate together during power transmission and not to be separated under normal conditions. Can be used as a safety device for preventing the coupled machine parts from bearing excessive load, and plays a role of overload protection.

It is understood that other devices capable of performing power transmission can also be applied to the impulse simulation assembly 300 to ensure the stability of the power transmission between the first motor 303 and the first lead screw 301, which is not limited herein.

For one embodiment of the friction simulation assembly 400, comprising: a sliding piece 401, a moving part, a second bearing seat 404 and a second motor 403; the slide sheet 401 is connected with a moving part, the moving part is fixed in the holding mechanism through a second bearing pedestal 404, and the moving part is connected with a second motor 403;

in the friction feedback process, the second motor 403 drives the slide 401 to move through the moving component, so as to generate friction feedback.

Specifically, the moving parts shown in the figures include a second lead screw 402, a nut 405, and a slip sheet connector 406; the second lead screw 402 is fixed on the holding mechanism 100 through a second bearing block 404, a nut 405 is connected with the second lead screw 402, the nut 405 is connected with a sliding piece connector 406 through a moving groove on the holding mechanism 100, and the sliding piece connector 406 is connected with the sliding piece 401; in this embodiment, the second lead screw 402 is fixed close to the inner surface of the holding mechanism 100 by the second bearing housing 404, which facilitates the connection between the slide connector 406 and the slide 401 of the second lead screw 402, and saves the inner space of the holding mechanism 100.

In the friction feedback process, the second motor 403 drives the nut to move through the second lead screw 402, the nut drives the sliding piece 401 to move, and the sliding piece moves relative to the palm or other parts of the user to generate friction feedback. For example, in a fishing scene, a user needs to wave a fishing rod, and at this time, in order to simulate the feeling that the fishing rod is out of hand, the second motor 403 may be controlled to drive the nut to move through the second lead screw 402, so that the nut drives the slider 401 to move along the lateral component of the waving direction, and the user may feel the friction force of the slider 401 on the lateral component, thereby simulating the feeling that the fishing rod is out of hand.

Optionally, the composition of the moving part is an example, and other composition manners include, but are not limited to, a linear motor, a spring, a synchronous belt or a gear device, and the specific manner depends on an actual scene.

Optionally, in order to improve the strength of the friction feedback, a plurality of friction simulation components may be set, or one friction simulation component may drive a plurality of sliding pieces 401 to perform the friction feedback, and the sliding pieces 401 are configured based on the holding position. Different sliding pieces 401 are connected with each other, so that the moving part drives the sliding pieces 401 to move simultaneously. That is, the second motor 403 drives the sliding piece 401 to move, and the sliding piece 401 drives other sliding pieces to move, so that the friction feedback is improved by increasing the friction area.

Optionally, in order to ensure the stability of the power transmission between the second motor 403 and the second lead screw 402, the friction simulation assembly 40 further includes a second coupling 407. The second coupling 407 is a device for coupling two shafts or a shaft and a rotating member to rotate together during the transmission of motion and power without being separated under normal conditions. Can be used as a safety device for preventing the coupled machine parts from bearing excessive load, and plays a role of overload protection.

For one embodiment of a vibration simulation assembly 500, a schematic diagram of another haptic feedback device is provided for an embodiment of the present application, as shown in fig. 5. The vibration simulation assembly 500 is shown to contain at least one vibration unit 501; wherein the vibration unit 501 is placed outside the gripping mechanism and the vibration unit 501 is in contact with the target user.

In the vibration feedback process, the vibration unit 501 is used to emit vibrations of a target frequency in response to position information of a target user holding the haptic feedback device to generate vibration feedback. For example, in a virtual scene in which a user performs chopping of wood, it is necessary to simulate vibration feedback after chopping of wood. According to the relation between the moment and the acting force, the moment is smaller the farther the user holds the position of the tactile feedback device, the acting force after the wood is chopped is weaker, the corresponding vibration feedback is smaller, and the vibration unit 501 is controlled to vibrate at a low frequency; and when the position where the user holds the tactile feedback device is closer, the moment is larger, the acting force after the user splits the wood is stronger, the corresponding vibration feedback is stronger, and the vibration unit 501 is controlled to vibrate at high frequency, so that different vibration feedbacks are simulated.

In addition, the vibration units 501 are arranged by using a target array, and the target array wakes up different numbers of vibration units 501 based on the position information to generate vibration feedback, so that the vibration feedback has spatial resolution, that is, vibration effects in different spaces have obvious differences.

It is understood that other components capable of generating a vibration force can be applied to the present embodiment, and are not limited herein.

Specifically, fig. 5 also shows a housing 102 of the momentum wheel and a motor housing 103 of the gripping mechanism 100, which are used for packaging and protecting the related components of the torque simulation module 200, the momentum simulation module 300 and the friction simulation module 400.

Optionally, the holding mechanism 100 may further include a walking line limiting component, as shown in fig. 6, which is a schematic structural diagram of another tactile feedback device provided in the embodiments of the present application. The figure shows a first routing limiting component 601, a second routing limiting component 602, a third routing limiting component 603 and a fourth routing limiting component 604; the first routing limiting part 601, the second routing limiting part 602, the third routing limiting part 603 and the fourth routing limiting part 604 are arranged in the holding mechanism 100;

specifically, the first routing limiting component 601, the second routing limiting component 602, the third routing limiting component 603, and the fourth routing limiting component 604 are used for limiting a line in the tactile feedback device, so as to avoid the influence of the tactile feedback mechanism and the line.

In a possible scene, through the tactile feedback device combining all the simulation components, the force feedback effect on the wrist torque can be realized through acceleration and deceleration of the momentum wheel; and through the movement of the copper slide block, the force feedback effect of impulse and gravity center movement is realized; the effect of friction force feedback can be realized by moving the two sliding sheets, and the effect of force feedback of vibration with spatial resolution can also be realized by the vibration motor array.

It can be understood that one or more of the above feedback modes can be analyzed when a user performs an action, thereby implementing a comprehensive force sensing simulation process. Specifically, for the analysis process of the motion, the horizontal component (impulse feedback) and the vertical component (moment feedback) of the current motion in the virtual scene are determined; it is then determined whether the lateral component and the longitudinal component correspond to contact conduction of friction or vibration, so that the above-mentioned dimension determination of force perception is made on the basis of the determination result and the corresponding force perception dimension is simulated.

As is clear from the above embodiments, the tactile feedback device includes the gripping mechanism and the tactile feedback mechanism. The tactile feedback mechanism comprises a torque simulation component, an impulse simulation component, a friction simulation component or a vibration simulation component, and the force feedback modes corresponding to the simulation components are different; the holding mechanism is used for providing a holding position for a target user. The tactile feedback mechanism is arranged on the corresponding holding position based on the force feedback modality, so that the target user receives the tactile feedback mechanism and sends out corresponding tactile feedback based on the operation action of the target user. The haptic feedback corresponds to a force feedback modality, thereby implementing a multi-modal haptic feedback process. Because the simulation components of the four force feedback modes are adopted for haptic feedback, the haptic feedback device can comprehensively simulate various force senses possibly received by the forearm of a user, and the completeness of the haptic feedback is improved through the limited combination of the simulation components.

The following describes a process of parameter analysis by the haptic feedback device. Referring to fig. 7, fig. 7 is a flowchart of a haptic feedback method according to an embodiment of the present application, where the embodiment of the present application includes at least the following steps:

701. a first target action of a target user is acquired.

In this embodiment, referring to the flow shown in fig. 8, fig. 8 is a schematic view of a scenario of a haptic feedback method provided in this embodiment, that is, an action of a target user is obtained through a sensor.

It is understood that the first target action in this embodiment may be obtained by a motion sensor in the haptic feedback device. Specifically, the motion sensor may include, for example, accelerometer sensors that detect acceleration in various directions (typically three axes), detect gravity at rest and detect gravity in various directions, and may be used in applications that recognize the attitude of the haptic feedback device (e.g., hand-held attitude switching; related games, magnetometer attitude calibration), vibration recognition related functions (e.g., pedometer, tapping), and the like; in addition, the tactile feedback device may further include other sensors such as a gyroscope and an infrared sensor, which are not described herein again.

702. A first virtual scene is obtained.

In this embodiment, the first virtual scene is a scene executed by the first target action. Specifically, the starting scene of the VR interactive game may be, or any scene in the progress of the VR interactive game may be, which is not limited herein.

703. And generating a multi-modal first feedback instruction according to the first target action and the first virtual scene.

In this embodiment, the determination process of the first feedback instruction is performed based on the first virtual scene and the first target action, that is, firstly determining which modalities of force feedback may exist in the first virtual scene, then determining interaction with the first virtual scene according to the first target action, and determining specific feedback forms and combinations from the modalities of force feedback. Specifically, for the process of determining a specific feedback form according to the analysis action, reference may be made to the examples in the embodiments shown in fig. 2 to fig. 6, that is, the corresponding feedback component is controlled according to the direction and form of the action, and the specific action form depends on the actual scene, and details are not repeated here.

In some embodiments, the virtual scene may be constantly changing. With the adjustment/change of the virtual scene, different feedback instructions can be analyzed based on the same target action. The adjustment/change of the virtual scene may be naturally generated or may be caused by a target action. In the latter case, first a first target action and a first virtual scene are obtained, a first sub-feedback instruction is generated according to the first target action and the first virtual scene, and tactile feedback is generated according to the first feedback instruction; and then generating a second virtual scene according to the first action and the first virtual scene, acquiring a second target action, generating a second feedback instruction according to the second action and the second virtual scene, and generating tactile feedback according to the second feedback instruction. Therefore, the matching degree of the target action in the virtual scene is improved.

In a possible scene, the first target action is a waving action, the corresponding chopping action of the target user in the first virtual scene is not chopped into a target virtual object, and the obtained first feedback instruction comprises a moment parameter and a friction parameter determined based on the first target action; then, a second virtual scene (a hit target) is generated in the first virtual scene due to the first target action, namely the second target action is used as a waving action, and the target virtual object is hacked by the hacking action of the target user in the corresponding second virtual scene, so that the second feedback instruction contains vibration parameters (namely, vibration after the hit object is simulated) determined based on the second stage action.

It is understood that the first feedback instruction may be sent to the haptic feedback device immediately after being generated, which is an instant haptic feedback process; the first sub-feedback instruction and the second sub-feedback instruction may also be generated and then sent together in a package, and the specific order is determined by the decision logic of the virtual scene or user setting, which is not limited herein.

In another possible scenario, the determination of the feedback instruction may further perform a threshold determination, for example, if the first target motion is a lifting motion, the impulse parameter may be determined according to the first virtual scenario and the lifting motion; then, acquiring an impulse threshold of a virtual object in the first virtual scene; if the impulse parameter is greater than the impulse threshold, determining a vibration parameter corresponding to the impulse parameter according to the impulse parameter; and then determining a first feedback instruction based on the impulse parameter and the vibration parameter corresponding to the impulse parameter, so as to simulate the tactile feedback of lifting the immobile object.

704. And controlling the tactile feedback device to perform tactile feedback according to the first feedback instruction.

In this embodiment, generally, the haptic feedback is performed in a multi-modal composition. For example, for a chopping process in a waving process, a waving action (first target action) may be performed first at the time of a first virtual scene; determining a moment parameter and a friction parameter according to the first virtual scene and the waving action; a first feedback command is then determined based on the torque parameter, the friction parameter, and the vibration parameter. That is, when the user swings the tactile feedback device, in order to simulate the feeling of hands-off during the process of swinging the weapon in the virtual scene, the torque feedback, the friction feedback and the impulse feedback are generated at the same time. Further, after the weapon cuts the middle target in the virtual scene (triggering a second virtual scene), stopping torque feedback, friction feedback and impulse feedback; and determining vibration parameters according to the second virtual scene and the waving action, and further performing vibration feedback in order to simulate the vibration feeling after the target is chopped.

In some scenarios, when the user lifts the haptic feedback device, friction feedback and impulse feedback (impulse parameters) may be generated simultaneously, as well as vibration feedback by a virtual character in the virtual scene when it cannot bear the weight (impulse parameters greater than impulse threshold), in order to simulate the downward pressure of the weight.

In some scenes, when a user uses the tactile feedback device to perform a blocking action, in order to simulate impact force and impact feeling in the blocking process, vibration feedback, friction feedback and impulse feedback are generated simultaneously, and when a virtual character in a virtual scene cannot bear the impact force, namely the blocking fails, a blocking object flies out, and at the moment, torque feedback, vibration feedback and friction feedback are generated simultaneously, so that the user can sense the feeling that the blocking object flies out of hand.

In some scenes, the feedback of a single modality can also comprehensively simulate real tactile feedback, for example, when a virtual scene is a tug-of-war scene, only friction feedback is triggered; the number of modes for feedback depends on the actual scene.

Optionally, the feedback of these modalities may also be feedback performed after collecting multiple actions, as shown in fig. 9, which is a schematic view of another haptic feedback method provided for the embodiment of the present application. The method comprises the steps of acquiring a plurality of first target actions in a preset time period to obtain a target action set; then determining a feedback instruction set in the first virtual scene based on the target action set; and then determining a first sub-feedback and a second sub-feedback based on the feedback instruction set, and then sequentially executing the first sub-feedback and the second sub-feedback, so as to simulate the sequence of the acting force, for example, in a game like a ninja, a user needs to firstly 'tie mark' (execute a plurality of actions within a preset time period to obtain a target action set), and then can start generation of a feedback instruction (determine a feedback instruction set) in a virtual scene, for example, generate a move for generating an explosion effect, and at this time, in order to simulate the perception of explosion impact, the feedback instruction set may include instructions for friction feedback, impulse feedback, and vibration feedback. Specifically, these multi-modal feedbacks may occur simultaneously (the first sub-feedback and the second sub-feedback are performed simultaneously), or may be performed sequentially. The specific execution sequence depends on the requirements of a specific scenario, and is not limited herein.

Optionally, in the above haptic feedback process based on the target action set, a feedback parameter may also be used to indicate a feedback result. That is, the feedback parameter in the first sub-feedback execution process is detected first, and if the feedback parameter is greater than a preset value, the second sub-feedback is executed. For example, in the game scene of "ninja", when the vibration feedback generated by explosion reaches a certain intensity (feedback parameter), the impulse feedback is triggered, that is, when the simulated explosion impact force is large enough, the character displacement is generated. In another possible scenario, fig. 10 is a schematic diagram of another scenario of a haptic feedback method provided in an embodiment of the present application. When a user waves the tactile feedback device, friction feedback is generated, if the feedback strength reaches a preset value, a vibration warning is sent to prompt that a weapon is about to fall off, and therefore interactivity of the tactile feedback device in various virtual scenes is improved.

It is understood that the above scenario description is an example, and the specific feedback mode may be any combination of the above mentioned modalities, and is not limited herein.

Through multi-mode combined simulation and sequential judgment, a user can realize various tactile feedbacks through the tactile feedback device in the application, the tactile feedback simulation is accurate, the association relationship can be set, the fit degree with an interactive scene is ensured, no extra devices are needed, namely, the tactile feedback device realizes comprehensive simulation of various force perceptions possibly received by the forearm of the user, and the completeness of the tactile feedback is improved through limited simulation component combination.

Referring to fig. 11, fig. 11 is a schematic structural diagram of a server provided in this embodiment, where the server 1100 may have large differences due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 1122 (e.g., one or more processors) and a memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) storing an application program 1142 or data 1144. Memory 1132 and storage media 1130 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 1130 may include one or more modules (not shown), each of which may include a series of instruction operations for the server. Still further, the central processor 1122 may be provided in communication with the storage medium 1130 to execute a series of instruction operations in the storage medium 1130 on the server 1100.

The server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input-output interfaces 1158, and/or one or more operating systems 1141, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so forth.

The steps performed by the management apparatus in the above-described embodiment may be based on the server configuration shown in fig. 11.

Further, an electronic device is provided in an embodiment of the present application, which includes an interaction device, an input/output (I/O) interface, a processor, and a memory, where the memory stores program instructions, and the processor is configured to execute the program instructions stored in the memory, where the program instructions may be a VR game interface, a remote control program, or an AI program; the interaction device is used for acquiring an operation instruction input by a user; wherein the interaction means comprises a haptic feedback device as described above in any one of figures 2 to 6.

Further, the embodiment of the present application also provides a robot system, and the system includes: the system comprises an operating device, a processor, an input/output (I/O) interface, a memory and an interaction controller, wherein the interaction controller and the operating device are respectively connected with the memory and the processor through the input/output (I/O) interface, and the operating device is used for controlling the movement of the interaction controller; the operating device is used for acquiring an operating instruction input by a user, wherein the operating device comprises the tactile feedback device shown in any one of the above figures 2 to 6.

In this embodiment, the memory and the processor may be connected to the interactive controller and the operation device through one input/output (I/O) interface, or may be connected to the interactive controller and the operation device through two interfaces. The operating means comprise the aforementioned tactile feedback means, in the course of which the memory reads and stores the relevant data.

In this embodiment, in the human-computer interaction system, the mechanical perception of the user's forearm and the following portions is simulated in all directions. I.e. the haptic feedback device is instructed to perform haptic feedback by analysis of multiple modalities.

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. Those of skill would further appreciate that the elements of the examples described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described above generally in terms of their functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a haptic feedback device, or a network device) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. A haptic feedback device, comprising:

a holding mechanism and a tactile feedback mechanism;

the holding mechanism is used for providing a holding position for a target user, and the tactile feedback mechanism is used for sending out corresponding tactile feedback according to the operation action of the target user, wherein the tactile feedback corresponds to the force feedback modality.

2. The apparatus of claim 1, wherein the torque simulation component comprises:

3. The apparatus of claim 1, wherein the impulse simulation component comprises:

4. The apparatus of claim 3, wherein the impulse simulation component further comprises:

a slide rail;

the slide rail is used for supporting the slide block to move.

5. The device of claim 1, wherein the friction simulation assembly comprises:

6. The apparatus of claim 5, wherein the moving member comprises:

the second screw rod, the nut and the sliding piece connector;

7. The device of claim 5, wherein the moving member comprises one of a linear motor, a spring, a timing belt, or a gear arrangement.

8. The apparatus of claim 5, wherein the friction simulating assembly includes a plurality of the slides, the slides being configured based on the holding position;

9. The apparatus of claim 1, wherein the vibration simulating assembly comprises:

at least one vibration unit;

10. The apparatus of claim 9, wherein the vibratory units are arranged in a target array that wakes up different numbers of the vibratory units based on the location information to generate the vibratory feedback.

11. The device of any of claims 1-10, wherein the haptic feedback device further comprises:

a routing limit component;

the routing limiting part is arranged on the holding mechanism;

12. The device according to claim 1, wherein the torque simulation means is disposed at an end of the holding means, the impulse simulation means is disposed inside the holding means, the friction simulation means is disposed outside the holding means for tactile feedback, and the vibration simulation means is disposed at a position corresponding to a palm of a hand of the target user when the target user holds the holding means.

13. A haptic feedback method, comprising:

acquiring a first target action of a target user;

acquiring a first virtual scene;

14. The method of claim 13, further comprising:

acquiring a second target action of the target user;

15. The method of claim 14, wherein:

16. The method of claim 13, wherein the first target motion is a lifting motion, and wherein generating a first feedback instruction for multiple modalities based on the first target motion and the first virtual scene comprises:

acquiring an impulse threshold corresponding to the first virtual scene;

17. An electronic device, characterized in that the electronic device comprises: an interaction device, an input/output (I/O) interface, a processor, and a memory, the memory having stored therein program instructions, the processor for executing the program instructions stored in the memory;

the interaction device is used for acquiring an operation instruction input by a user; wherein the interaction device comprises a haptic feedback device according to any one of claims 1-12.

18. A human-computer interaction system, characterized in that the system comprises: the device comprises an operating device, a processor, an input/output (I/O) interface, a memory and an interaction controller, wherein the interaction controller and the operating device are respectively connected with the memory and the processor through the input/output (I/O) interface, and the operating device is used for controlling the movement of the interaction controller;

the operating device is used for acquiring an operating instruction input by a user, wherein the operating device comprises the tactile feedback device according to any one of claims 1-12.