WO2020224566A1

WO2020224566A1 - Hand operation method and apparatus for virtual reality, augmented reality, and merged reality

Info

Publication number: WO2020224566A1
Application number: PCT/CN2020/088563
Authority: WO
Inventors: 于毅欣
Original assignee: 于毅欣
Priority date: 2019-05-06
Filing date: 2020-05-03
Publication date: 2020-11-12
Also published as: CN111897413A

Abstract

A hand operation method and apparatus that may be used for virtual reality, augmented reality, and merged reality. The apparatus acquires hand information in real time by means of a structured light sensor module, and is characterized in comprising: an information processing module, a battery module, the structured light sensor module, a network communication module, a gyroscope, an infrared detection module, and a comfortable strap. Compared to traditional virtual reality gloves, the described apparatus is simpler and more convenient and is suitable for use over long periods of time.

Description

Virtual reality, augmented reality, and fusion reality hand operation method and device

Technical field

The invention relates to the technical field of virtual reality systems and the field of digital control, in particular to a data glove device based on virtual reality simulation.

Background technique

The traditional virtual reality gloves are slightly redundant, and the gloves that cover the whole hand are less convenient to wear, and are relatively cumbersome to wear, are easier to increase hand sweat, are more easily damaged, and are more difficult to wear for a long time. Starting from simplicity, the present invention provides a simpler and relatively labor-saving virtual reality hand-based interaction method and device, which is more suitable for long-term use. The present invention can also be used in control methods and devices based on hand movements.

Summary of the invention

The purpose of the present disclosure is to provide a hand operation method and device that can be used in virtual reality, augmented reality, and fusion reality, which can be worn more concisely, easier to operate, and more suitable for longer-term use.

According to one aspect of the present disclosure, there is provided a hand operation device that can be used in virtual reality, augmented reality, and fusion reality, which is characterized by comprising: an information processing module, a battery module, a structured light sensor module, a network communication module, and a gyroscope (Optional), infrared detection module (optional), more comfortable belt.

As an optional implementation manner, the information processing module is responsible for processing the structured light information received by the structured light sensor module, and obtains information such as fingers, palms or bones through processing, and can interact with network connected devices through the network module Module.

Preferably, the information processing module can be inlaid or placed in a more comfortable belt (referred to as a hand belt), and connected to the structured light sensor module on the hand belt through a line (which can be built into the hand belt) to obtain power and exchange data Wait.

Preferably, the structured light sensor module may include one or more structured light sensor devices.

Preferably, the battery can be arranged at the back of the hand and connected to the information processing module to provide power.

As an optional embodiment, the battery can be detached from the hand strap.

Preferably, the network module is connected to the information processing module, and can synchronize data to the device connected to the network of the device.

Preferably, the method of establishing the network connection may be wired or wireless.

Optionally, if it is equipped with a gyroscope module, the gyroscope module is connected to the information processing module and is preferentially arranged on the back of the hand, such as the back of the hand, which can better track the direction of the entire hand and feedback information.

Optionally, if an infrared detection module is equipped in an optional embodiment, the infrared detection module is connected to the information processing module to complete the purpose of transmitting infrared detection information to the information processing module to assist in identifying the position of the hand .

Preferably, the main deployment position of the structured light sensor module in the hand device is under the palm or the base of the thumb, refer to the schematic diagram of FIG. 1 and the schematic diagram of FIG. 2.

As an optional embodiment, the hand strap can be turned into a hand clip or other simple shape that can be relatively fixed on the hand to meet the needs.

According to one aspect of the present disclosure, there is provided a hand operation method that can be used in virtual reality, augmented reality, and fusion reality, which is characterized in that it includes: a structured light module collects information, and transmits the information to an information processing module. After processing the information, the processing result is sent to the glasses or the device connected to the device network through the network module.

Preferably, the image collected by the structured light module recognizes and locates the information of the fingers and the palm according to the characteristic information of the hand, and calculates the bone and other information according to the information of the fingers and the palm.

Optionally, when the infrared detection module is equipped, the data of the corresponding area in the depth information of the structured light sensor corresponding to the area in the image whose temperature is close to the hand temperature can be preferentially processed according to the data in the infrared detection module.

Optionally, a special gyroscope module can be added to capture related information such as hand rotation.

As an optional implementation manner, the data of the structured light sensor module can be processed in the information processing module, and information such as bones can be identified, and sent to the glasses through the network module or receiving equipment for processing hand information.

As an optional implementation manner, the information processing module can simplify or compress the data of the structured light sensor, and send it to the glasses or the device that accepts the processing of hand information through the network module for processing or identification.

Preferably, the device connected to the hand device of the present invention can set the specific finger information of interest through an instruction or a programming interface to adjust or affect the calculation amount and the information of interest of the information processing module, so as to increase the refresh frequency or decrease the refresh rate when necessary. Data transfer volume and other purposes, such as focusing only on the information of the index finger.

Preferably, a virtual reality system can be equipped with multiple hand operating devices for an operator, and each hand operating device can function at the same time or only partially, and can set a certain operation authority for each hand operating device .

Preferably, the hand operation device of the present invention may have a pause\resume button to pause or resume the processing, recognition, and sending functions of the hand operation.

Optionally, the pause\resume button can be deployed on the hand strap or other convenient one-handed operation position.

Optionally, a gyroscope module can be added to assist in obtaining data such as the movement of the arm.

Preferably, the hand operating device of the present invention sends the identified bone information to the glasses or the receiving processing device, and the glasses or receiving processing device recognizes further intentions or operations to affect the virtual reality environment according to various forms of the bones.

Preferably, the method can be combined with glasses to assist the positioning of the hands, but the method does not rely on the positioning of the hands of the glasses.

Optionally, when the gyroscope is equipped, the gyroscope parameter information can be checked for some reason when the device is used for the first time.

Preferably, the meaning of different hand shapes or hand movements can be set at the information processing end connected to the hand device as required.

Preferably, when the structured light sensor module obtains incomplete data, the hand device or the connected information processing device of the present invention can calculate the position according to the experience database or prompt the user to adjust the wearing position.

Preferably, the sensitivity of finger movement recognition of the device of the present invention can be set and/or adjusted through a development interface in a virtual reality application.

As an optional implementation manner, the battery can be arranged on the arm and connected to the device of the present invention through a wire, so as to reduce the total weight of the hand-worn device of the device of the present invention.

Description of the drawings

Fig. 1 is a schematic diagram of a hand surface schematically showing a structured light sensor deployed near the palm.

Fig. 2 is a schematic diagram schematically showing a structured light sensor deployed on the back of the hand near the palm.

FIG. 3 is a schematic diagram of a hand surface schematically showing a structured light sensor deployed near the thumb.

Fig. 4 is a schematic diagram schematically showing a structured light sensor deployed on the back of the hand near the thumb.

The above schematic diagram is only used as a schematic display of a possible implementation position of the structured light sensor module of the present invention, which is convenient for showing or showing the rationality of a deployment situation of the present invention, and is not used to limit the present invention.

The above are specific embodiments of the present invention, but the protection scope of the present invention should not be limited thereto. Any changes, substitutions or simplifications that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention shall be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention shall be defined by the claims. The limited protection scope shall prevail.

Claims

A hand data acquisition device that can be used in virtual reality, augmented reality, and fusion reality, which is characterized by comprising: an information processing module, a battery module, a structured light sensor module, a network communication module, a gyroscope (optional), and infrared detection Module (optional), more comfortable strap, pause and resume buttons;

The information processing module is used to process and/or sort the acquired sensor information;

The battery module is used to store and supply power, and can allow the battery to be removed from the power supply interface where it is placed;

The structured light sensor module is used to obtain hand depth information;

The network communication module is responsible for synchronizing data to glasses or equipment that needs to use the hand device of the present invention for data interaction through a wired or wireless network;

The gyroscope (optional) is used to assist in obtaining information such as the direction of the hand;

The infrared detection module (optional) is used to obtain hand infrared data;

The more comfortable belt is a belt suitable for a longer time wearing, in which there are necessary circuits to complete information transmission or power supply, and can support, accommodate or carry batteries and sensors, and can be provided with soft cushions;

The pause and resume buttons are used to pause or resume the current operation, and are mainly deployed on a more comfortable belt or other convenient one-handed operation position.
A hand operation method that can be used in virtual reality, augmented reality, and fusion reality, which is characterized in that it includes:

Step 1: The structured light module works and collects information;

Step 2: Identify information such as fingers and palms from the information collected by the structured light module;

Step 3: Abstract specific information based on the recognized finger and palm information, including but not limited to bone information, and send the specified data to the target receiving device. If there is gyroscope information, it can also be sent at this time;

Step 4: The target receiving device will recognize the corresponding meaning from the settable gesture library according to the received bone and other information, and perform corresponding expressions or perform corresponding functions.
The "step one" of claim 2, wherein the structured light module works and collects information, including:

Obtain information through structured light modules;

The information acquired by the structured light module should be as wide as possible to exceed 60 degrees, or the information collected by multiple structured light devices can be combined into a wider angle of more than 60 degrees;

The acquired information is transmitted to the information processing module for processing.
The "step two" according to claim 2, characterized in that identifying information such as fingers and palms from the information collected by the structured light module includes:

Identify fingers and palms from structured light data based on the characteristics of fingers and palms;

For faster recognition, you can record the previous recognition results for comparison.
The "step three" according to claim 2, characterized in that specific information is abstracted based on the information of the identified fingers and palms, including but not limited to bone information, and the designated data is sent to the target receiving device, if there is a gyroscope Information can also be sent at this time, including:

Abstract the orientation of the palm from the structured light information, or calculate the orientation of the fingers based on this, and abstract the bones and other required information based on the finger information;

As an optional implementation, when a gyroscope is equipped, the recognition result of the hand information can be verified or corrected in combination with the gyroscope data, and a limited verification or correction can be performed;

Send finger bones and palm information or intermediate data to the glasses or the module responsible for hand data processing, and send gyroscope information if there is any;

As an optional implementation, here the hand device can be set by the glasses or the module responsible for hand data processing to only send partial information in exchange for more frequent identification or interaction speed, for example, only the index finger information is sent;

As an optional implementation manner, if an infrared detection module is deployed, the data of the corresponding area in the depth information of the structured light sensor corresponding to the area in the image whose temperature is close to the hand temperature can be preferentially processed according to the data in the infrared detection module.
The "step four" according to claim 2, characterized in that the target receiving device will recognize the corresponding meaning from the settable gesture library based on the received bone and other information, and perform corresponding expressions or perform corresponding functions, including :

The target receiving device is usually glasses or a computer;

The target receiving device will recognize the meaning of the current gesture according to the preset gesture library;

The target receiving device can also recognize the current hand movement in combination with the previously received information, and can determine its meaning together with the current gesture.