WO2018219279A1

WO2018219279A1 - Virtual touch system, method and device

Info

Publication number: WO2018219279A1
Application number: PCT/CN2018/088915
Authority: WO
Inventors: 史俊
Original assignee: 青岛海尔股份有限公司; 北京一数科技有限公司
Priority date: 2017-05-31
Filing date: 2018-05-30
Publication date: 2018-12-06
Also published as: CN107272892B; CN107272892A

Abstract

Disclosed in the embodiments of the present invention are a virtual touch system, method and device, the virtual touch system comprising: a head-mounted display device and an intelligent wrist strap, wherein the intelligent wrist strap is used for obtaining touch information of a user and sending the touch information to the head-mounted display device, and the head-mounted display device is used for receiving the touch information and responding to a touch command corresponding to the touch information. Since the use of a depth camera is not required, the cost of virtual touch is greatly reduced.

Description

Virtual touch system, method and device

The present application claims the priority of the Chinese patent application entitled "A Virtual Touch System, Method and Apparatus" on May 31, 2017, the application number is 201710399191.7, the entire contents of which are incorporated herein by reference. In the application.

Technical field

The present invention relates to the field of virtual touch technologies, and in particular, to a virtual touch system, method and device.

Background technique

In recent years, display technologies such as virtual reality, reality enhancement, and mixed reality have developed rapidly, and most of them are displayed through head-mounted display devices. In order to facilitate the user's manipulation of the head-mounted display device, virtual touch technology came into being.

In the existing virtual touch technology, a virtual camera is combined with object recognition to implement virtual touch. Generally, a plurality of depth cameras are mounted on the head mounted display device, and the shooting range of the plurality of depth cameras is a virtual touch detectable range of the head mounted display device. That is to say, after the user wears the head-mounted display device, the user can issue touch gestures such as clicking and sliding within the range detectable by the depth camera, and the depth camera can shoot the user's touch gesture, and the head-mounted display device can be used. The touch command corresponding to the touch gesture can be recognized, and then the touch command is responded to.

However, with the above virtual touch method, since a plurality of depth cameras are used, the head-mounted device has high power consumption and high cost.

Summary of the invention

The embodiment of the invention discloses a virtual touch system, method and device for reducing virtual touch cost. The technical solutions are as follows:

An embodiment of the present invention provides a virtual touch system, where the system includes: a head mounted display device and a smart wristband, wherein

The smart wristband is configured to acquire touch information of the user, and send the touch information to the head mounted display device;

The head mounted display device is configured to receive the touch information and respond to the touch instruction corresponding to the touch information.

Optionally, the smart wristband is specifically configured to acquire a motion speed, an acceleration, a vibration frequency, and a motion time, and determine a touch motion according to the motion speed, the acceleration, the vibration frequency, and the motion time, and the touch The action information of the control action is sent to the head mounted display device as the touch information.

Optionally, the system further includes:

a sensing ring for transmitting electromagnetic waves and transmitting a time series of transmission of the electromagnetic waves to the smart wristband;

The smart wristband is further configured to receive the electromagnetic wave through a skin, receive the transmission time sequence through a wireless network, and determine the sensing ring according to the transmitting time sequence and receiving a receiving time sequence of the electromagnetic wave. Moving a track and transmitting the motion track as the touch information to the head mounted display device;

The head-mounted display device is further configured to receive the motion track, determine a touch command corresponding to the motion track according to the motion track and the current display content, and respond to the touch command.

Optionally, the smart wristband is further configured to record a forming time of the motion track, and determine, according to the motion track and the forming time, whether the touch gesture corresponding to the motion track is valid, and if yes, Transmitting the motion track as the touch information to the head mounted display device;

or,

The smart wristband is further configured to record a formation time of the motion track, and send the formation time to the head mounted display device; the head mounted display device is further configured to receive the formation time, and Determining, according to the forming time, whether the touch instruction corresponding to the motion track is valid, and if yes, responding to the touch instruction.

The embodiment of the present invention further provides a virtual touch method, which is applied to a smart wristband in a virtual touch system, wherein the virtual touch system includes: the smart wristband and a head mounted display device, and the method include:

Obtain the touch information of the user;

Sending the touch information to the head mounted display device, so that the head mounted display device receives the touch information and responds to the touch command corresponding to the touch information.

Optionally, the method further includes:

Receiving electromagnetic waves transmitted by the sensing ring through the skin;

Receiving, by the wireless network, a transmission time sequence of the electromagnetic wave sent by the sensing ring;

Determining a motion trajectory of the sensing finger according to the transmitting time sequence and receiving a receiving time sequence of the electromagnetic wave;

Sending the motion track as the touch information to the head mounted display device, so that the head mounted display device: receives the motion track, and determines the motion track according to the motion track and the current display content. Corresponding touch commands and responding to the touch commands.

Optionally, the step of sending the motion track as the touch information to the head mounted display device comprises: recording a formation time of the motion track; according to the motion track and the forming time, Determining whether the touch gesture corresponding to the motion track is valid; if yes, sending the motion track as the touch information to the head mounted display device.

or,

Transmitting the motion trajectory as the touch information to the head mounted display device, so that the head mounted display device: receiving the motion trajectory, determining, according to the motion trajectory and current display content, The step of the touch command corresponding to the motion track and the step of responding to the touch command includes: recording a formation time of the motion track; and transmitting the formation time and the motion track as the touch information to the head The display device is configured to: the head-mounted display device: receiving the motion track and the forming time, determining, according to the motion track and the current display content, a touch command corresponding to the motion track; And determining whether the touch instruction corresponding to the motion track is valid; if yes, responding to the touch instruction.

The embodiment of the invention further provides a virtual touch device, which is applied to a smart wristband in a virtual touch system, wherein the virtual touch system comprises: the smart wristband and the head mounted display device, the device include:

The touch information acquiring module is configured to acquire touch information of the user;

The touch information sending module is configured to send the touch information to the head mounted display device, so that the head mounted display device receives the touch information and responds to the touch command corresponding to the touch information.

The embodiment of the invention further provides a smart wristband, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

The processor is configured to implement the method steps described above when executing a program stored on the memory.

The embodiment of the invention further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the method steps are implemented.

In the solution provided by the embodiment of the present invention, the virtual touch system includes: a head-mounted display device and a smart wristband, wherein the smart wristband is used to acquire touch information of the user, and send the touch information to the head-mounted display device. The head mounted display device is configured to receive the touch information and respond to the touch command corresponding to the touch information. Since the depth camera is not required, the cost of virtual touch is greatly reduced.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.

1 is a schematic structural diagram of a virtual touch system according to an embodiment of the present invention;

2 is a schematic structural diagram of another virtual touch system based on the embodiment shown in FIG. 1;

FIG. 3 is a flowchart of a virtual touch method according to an embodiment of the present invention;

4 is a flow chart of another virtual touch method based on the embodiment shown in FIG. 3;

FIG. 5 is a specific flowchart of step S504 in the embodiment shown in FIG. 4;

FIG. 6 is a schematic structural diagram of a virtual touch device according to an embodiment of the present disclosure;

FIG. 7 is a schematic structural diagram of a smart wristband according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

In order to reduce the virtual touch cost, the embodiment of the invention provides a virtual touch system, method, device, smart wristband and storage medium.

A virtual touch system provided by an embodiment of the present invention is first introduced.

As shown in FIG. 1 , a virtual touch system provided by an embodiment of the present invention includes a head mounted display device 110 and a smart wrist band 120 . The head-mounted display device 110 is a device that is worn on the user's head for displaying virtual reality, reality enhancement, mixed reality, and the like. The smart wristband 120 is an electronic device worn on a user's wrist, arm, or the like.

Specifically, the smart wristband 120 is configured to acquire the touch information of the user, and send the touch information to the head mounted display device 110. The head mounted display device 110 is configured to receive the touch information and respond to the touch information. Corresponding touch commands. It can be understood that when the user uses the head mounted display device 110, in order to control the display content of the head mounted display device 110, a touch gesture is generated, and the smart wristband 120 acquires touch information corresponding to the touch gesture of the user. . The smart wristband 120 establishes a communication connection with the wearable display device 110 to transmit data. For convenient use, the communication connection may be a wireless network connection, such as a WIFI, a WLAN, a Bluetooth connection, etc., and is not specifically limited herein.

For example, the smart wristband 120 is worn on the user's arm. When the user views the video using the head mounted display device 110, the user may need to perform operations such as pause, fast forward, etc., at which time the user can wear the arm of the smart wristband 120. Make virtual touches, for example, you can swing your arms, vibrate your arms, and more. At this time, the movement of the arm will drive the smart wristband 120 to move together, and the smart wristband 120 can obtain the touch information corresponding to the virtual touch gesture of the user by detecting the motion information of the user, and then send the touch information to the head mounted display device. After receiving the touch information, the head-mounted display device 110 can respond to the touch command corresponding to the touch information. For example, if the touch command corresponding to the touch information is a fast forward command, the head mounted display device 110 performs fast forward processing on the video being played to meet the user's needs.

It can be seen that, in the solution provided by the embodiment of the present invention, the virtual touch system includes: a head-mounted display device and a smart wristband, wherein the smart wristband is used to acquire touch information of the user, and send the touch information to the head-mounted display device. The head mounted display device is configured to receive the touch information and respond to the touch command corresponding to the touch information. Since the depth camera is not required, the cost of virtual touch is greatly reduced.

As an implementation manner of the embodiment of the present invention, the smart wristband 120 can be specifically configured to acquire its own motion speed, acceleration, vibration frequency, and motion time, and determine the touch according to the motion speed, acceleration, vibration frequency, and motion time. The action is controlled, and the action information of the touch action is sent to the head mounted display device 110 as the touch information.

In one embodiment, the smart wristband 120 can acquire its own motion speed and acceleration through an internal inertial measurement unit (IMU). The IMU is a device that measures the three-axis attitude angle (or angular rate) and acceleration of an object. In general, an IMU consists of three single-axis accelerometers and three single-axis gyros. The accelerometer detects objects in the carrier coordinate system. The acceleration signal of the axis, and the gyro detection carrier measures the angular velocity and acceleration of the object in three-dimensional space with respect to the angular velocity signal of the navigation coordinate system, and calculates the posture of the object according to the angular velocity and acceleration of the object in three-dimensional space.

At the same time, the smart wristband 120 can record its own movement time and measure its own vibration frequency through a built-in acceleration sensor or other sensor with related functions. Further, the smart wristband 120 can use the obtained motion speed, acceleration, vibration frequency, and exercise time to derive corresponding touch actions, for example, palm flip, thumb and middle finger cooperate with the finger, thumb and index finger cooperate with the finger, etc. Control action. It should be noted that, in this embodiment, the specific algorithm for estimating the touch action is not limited. Computational models that can achieve the same or similar effects in the art can be combined under the overall technical solution of the present embodiment.

It should be noted that, since the smart wristband 120 is worn on the wrist, the arm, and the like of the user, the motion of the user wearing the smart wristband 120 causes the smart wristband 120 to move along with the speed, acceleration, and vibration frequency of the smart wristband 120. The exercise time is almost the same, so the speed, acceleration, vibration frequency and exercise time of the smart wristband 120 itself reflect the motion state of the part where the user wears the smart wristband 120, and therefore, the touch action determined by the above manner This is the touch action that the user sends.

Further, the smart wristband 120 can be used to send the action information of the touch action to the head mounted display device 110 as touch information. The action information of the touch action may be preset identification information, keyword information, and the like corresponding to the touch action. For example, if the touch action is a palm flip, the action information may be page turning.

It should be noted that, in order to enable the head mounted display device 110 to obtain the touch command accurately and quickly, the head mounted display device 110 may pre-store the corresponding relationship between the action information and the touch command, such that the head mounted display device 110 receives After the touch information, the corresponding touch command can be found, and in response, the touch command found is responded to. Assuming that the action information of the touch action is: the finger, the head-mounted display device 110 can find the touch command corresponding to the touch command according to the corresponding relationship between the preset action information and the touch command, and if the finger corresponds to the preset touch The control command is: the current interface is closed, and the head-mounted display device 110 can respond to the touch command, that is, close the current interface to complete the virtual touch.

It can be seen that the smart wristband can obtain its own motion speed, acceleration, vibration frequency and movement time, and determine the touch action according to this, and can accurately determine the touch action in the ambient light, even in the dark state, and ensure the head wear display. The device responds correctly to the user's touch commands.

As an embodiment of the embodiment of the present invention, as shown in FIG. 2, the system may further include: a sensing finger 130 for transmitting electromagnetic waves, and transmitting a transmission time sequence of the electromagnetic waves to the smart wristband 120. . The sensing finger 130 is generally worn on a finger portion of the user to facilitate obtaining a touch gesture of the user's finger.

In practical applications, the sensing finger 130 can emit electromagnetic waves through a biological waveguide device that can be internally mounted to emit electromagnetic waves, and the sensing finger 130 can be used to record the emission time sequence of the electromagnetic waves while emitting electromagnetic waves. It can be understood that since the emission of electromagnetic waves is a continuous process, its transmission time is a time series including a plurality of transmission times.

In general, the sensing ring 130 and the smart wristband 120 can establish a communication connection over the wireless network to transmit the above-described transmission time series to the smart wristband 120 over the wireless network. The wireless network may be a WIFI, a WLAN, a Bluetooth connection, etc., and is not specifically limited herein.

Correspondingly, the smart wristband 120 is further configured to receive electromagnetic waves transmitted by the sensing ring 130 through the skin, receive the transmission time sequence through the wireless network, and determine the motion of the sensing finger 130 according to the transmission time sequence and the receiving time sequence of receiving the electromagnetic waves. The trajectory is sent to the head mounted display device 110 as a touch information.

The smart wristband 120 can be used to record the receiving time sequence of receiving electromagnetic waves when receiving the electromagnetic waves sent by the sensing finger 130, so that the smart wristband 120 can be transmitted according to the transmission time series, the receiving time series, and the electromagnetic waves in the skin. The rate, the position of the sensing finger 130 during the movement is calculated, and the position of the sensing finger 130 during the movement is determined to determine the motion trajectory of the sensing finger 130. It can be understood that the sensing ring 130 is The motion track is the motion track of the user to make a touch gesture. The smart wristband 120 can further be used to send the determined motion trajectory to the head mounted display device 110.

It should be noted that, in order for the smart wristband 120 to accurately determine the motion trajectory of the sensing finger 130, the smart wristband 120 can receive electromagnetic waves through a plurality of positions, so that the time difference of electromagnetic waves propagated to the plurality of positions can be accurately determined. The position of the sensing finger 130 at any time is determined to accurately determine the motion trajectory of the sensing finger 130.

It should be further noted that, in order to prevent the smart wristband 120 from recognizing the motion trajectory of all the touch gestures as one point, the user needs to wear the sensing ring 130 and the smart wristband 120 when they are worn with relative displacement. Two positions, for example, the smart wristband 120 is worn on the wrist of one of the user's hands, so that the sensing finger 130 can be worn on a finger of the user's other hand, so that the user wears the sensing When the finger of the finger 130 performs a handwriting or other touch gesture on the back or arm of the hand of the wrist wearing the smart wristband 120, the sensing finger 130 and the smart wristband 120 are relatively displaced, and the smart wristband 120 receives the electromagnetic wave. The receiving time series is varied, and the smart wristband 120 is also capable of determining the motion trajectory of the sensing finger 130 based on the transmission time sequence and the receiving time sequence of the received electromagnetic waves.

The head mounted display device 110 is further configured to receive the motion track, determine a touch command corresponding to the motion track according to the motion track and the current display content, and respond to the touch command.

After the head-mounted display device 110 receives the motion track sent by the smart wristband 120, the head-mounted display device 110 can determine the touch command corresponding to the motion track according to the motion track and the current display content. For example, if the current display content of the head mounted display device 110 is a WeChat message input interface, and the motion track sent by the smart wristband 120 is a horizontal line from left to right, then the head mounted display device 110 can determine the corresponding motion track. The touch command is: inputting the Chinese character "one"; assuming that the current display content of the head mounted display device 110 is a picture browsing interface, and the motion track sent by the smart wrist band 120 is a horizontal line from left to right, then the head mounted display device 110 The touch command corresponding to the motion track can be determined as: viewing the previous picture.

It can be seen that, by the cooperation of the sensing ring and the smart wristband, the virtual touch system provided by the embodiment of the present invention can accurately recognize the user's handwriting input and the like, the accuracy is obviously improved, the user operation is convenient, and the user experience is improved.

In order to ensure that the touch commands responded to by the head-mounted display device 110 are correct touch commands to avoid erroneous response, in one embodiment, the smart wristband 120 can also be used to record the formation time of the motion track. It is determined whether the touch gesture corresponding to the motion track is valid according to the motion track and the formation time. If yes, the motion track is sent to the head mounted display device 110 as touch information.

In some cases, the user may perform some misoperations. In order to avoid the impact of the erroneous operation on the virtual touch, the smart wristband 120 may be used to record the formation time of the motion trajectory, and determine the motion trajectory according to the motion trajectory and the formation time. Whether the touch gesture is valid. In one possible case, the motion trajectory is an irregular messy trajectory, and is not any Chinese character. The motion trajectory is likely caused by a user's misoperation. At this time, the user does not actually issue any need to wear the display device 110. The touch command is executed, and then the smart wristband 120 can determine that the touch gesture corresponding to the motion track is invalid, and then the motion track is not sent to the head mounted display device 110 as touch information.

In another possible case, the formation time of the motion trajectory is too long or too short, and it is no longer within a preset time period. The motion trajectory is likely caused by a user's misoperation, and the user does not actually issue any need. When the touch command is executed by the display device 110, the smart wristband 120 can determine that the touch gesture corresponding to the motion track is invalid, and the motion track is not sent as the touch information to the head mounted display device. 110.

In another embodiment, the smart wristband 120 can also be used to record the formation time of the motion trajectory and send the formation time to the head mounted display device 110; the head mounted display device 110 can also be used to receive the formation time, and According to the forming time, it is determined whether the touch command corresponding to the motion track is valid, and if yes, responding to the touch command.

That is to say, after the smart wristband 120 records the formation time of the motion track, the formation time can be sent to the head mounted display device 110, and the head mounted display device 110 determines whether the touch command corresponding to the motion track is valid. The specific determination manner is the same as the manner in which the smart wristband 120 determines whether the touch gesture corresponding to the motion track is valid. For details, refer to the description of the manner in which the smart wristband 120 determines whether the touch gesture corresponding to the motion track is valid. Let me repeat.

Corresponding to the foregoing system embodiment, the embodiment of the present invention further provides a virtual touch method. The following describes a virtual touch method provided by the embodiment of the present invention.

As shown in FIG. 3, a virtual touch method is applied to a smart wristband in a virtual touch system, wherein the virtual touch system includes: the smart wristband and the head mounted display device, and the head mounted display device That is, it is a device worn on the user's head for displaying virtual reality, reality enhancement, mixed reality, etc., and the smart wristband is an electronic device worn on a user's wrist, arm, or the like. The method includes:

S301. Acquire user touch information.

When the user wears the display device, in order to control the display content of the display device, a touch gesture is generated, and the smart wristband acquires the touch information corresponding to the touch gesture of the user.

For example, the smart wristband can be worn on the user's arm. When the user uses the head-mounted display device to watch the video, the user may need to pause, fast forward, etc., and the user can virtualize by wearing the arm of the smart wristband. Touch, for example, can swing the arm, vibrate the arm, etc., the movement of the arm will drive the smart wristband to move together, and the smart wristband can obtain the touch information corresponding to the virtual touch gesture of the user by detecting the motion information of the arm.

S302, the touch information is sent to the head mounted display device, so that the head mounted display device receives the touch information and responds to the touch instruction corresponding to the touch information.

It is to be understood that the smart wristband establishes a communication connection with the head-mounted display device to transmit data. The communication connection is generally a wireless network connection, and may be, for example, a WIFI, a WLAN, a Bluetooth connection, or the like, which is not specifically limited herein.

After the smart wristband obtains the touch information of the user, the touch information can be sent to the head mounted display device, and the touch display device can receive the touch information and respond to the touch command corresponding to the touch information.

For example, if the touch command corresponding to the touch information sent by the smart wristband is a fast forward command, the head mounted display device performs fast forward processing on the currently playing video to meet the user's needs.

It can be seen that, in the solution provided by the embodiment of the present invention, the smart wristband acquires the touch information of the user, and sends the touch information to the head mounted display device, and the head mounted display device receives the touch information, and responds to the touch information. Touch command. Since the depth camera is not required, the cost of virtual touch is greatly reduced.

As an implementation manner of the embodiment of the present invention, the step of acquiring the touch information of the user may include:

Obtaining its own motion speed, acceleration, vibration frequency and motion time; determining the touch motion according to the motion speed, acceleration, vibration frequency and motion time.

Since the smart wristband is worn on the user's wrist, arm, etc., the movement of the user wearing the smart wristband will cause the smart wristband to move, and the speed, acceleration, vibration frequency and movement time of the two are almost the same. Therefore, when the user performs virtual touch by wearing the smart wristband, the movement of the smart wristband reflects the user's touch gesture, and the smart wristband can record its own exercise time, and is obtained by the IMU installed inside. Its own speed and acceleration, and its own vibration frequency is measured by a built-in acceleration sensor or other sensor with related functions.

After the smart wristband obtains its own motion speed, acceleration, vibration frequency and motion time, it can calculate the corresponding touch action according to the motion speed, acceleration, vibration frequency and motion time, for example, the palm flip, the thumb and the middle finger cooperate. Use a finger, thumb and index finger to match the touch action. It should be noted that, in this embodiment, the specific algorithm for estimating the touch action is not limited. Computational models that can achieve the same or similar effects in the art can be combined under the overall technical solution of the present embodiment.

As an implementation manner of the embodiment of the present invention, the step of sending the touch information to the head mounted display device may include:

Sending the action information of the touch action as the touch information to the head mounted display device.

After the smart wristband determines the touch action, the action information of the touch action can be sent to the head mounted display device as touch information. The action information of the touch action may be preset identification information, keyword information, and the like corresponding to the touch action. For example, if the touch action is a palm flip, the action information may be page turning.

It should be noted that, in order to enable the head-mounted display device to obtain the touch command accurately and quickly, the head-mounted display device may pre-store the corresponding relationship between the action information and the touch command, so that the head-mounted display device receives the touch. After the information, the corresponding touch command can be found, and in response, the touch command found is responded to. It is assumed that the action information of the touch action is: the ring finger, and the head-mounted display device can find the corresponding touch command according to the corresponding relationship between the preset action information and the touch command, and if the finger corresponds to the preset touch The command is: close the current interface, then the head-mounted display device can respond to the touch command, that is, close the current interface to complete the virtual touch.

As an implementation manner of the embodiment of the present invention, as shown in FIG. 4, the foregoing method may further include:

S401: Receive electromagnetic waves sent by the sensing ring through the skin;

The virtual touch system may further include a sensing finger ring, which is generally worn on a finger portion of the user, so as to obtain a touch gesture of the user's finger. In practical applications, the sensing ring can emit electromagnetic waves through a biological waveguide device that can be internally mounted to emit electromagnetic waves. The smart wristband can receive the electromagnetic waves sent by the sensing ring through the skin. It should be noted that in order to enable the smart wristband to accurately determine the motion track of the sensing ring, the smart wristband can receive electromagnetic waves through multiple positions, so that The time difference of electromagnetic wave propagation to the plurality of positions accurately determines the position of the sensing finger at any time, thereby accurately determining the motion trajectory of the sensing finger ring.

S402. Receive a transmission time sequence of the electromagnetic wave sent by the sensing ring through a wireless network.

While transmitting the electromagnetic wave, the sensing finger can record the time series of emission of the electromagnetic wave and send the time series of the electromagnetic wave to the smart wristband. It can be understood that since the emission of electromagnetic waves is a continuous process, its transmission time is a time series including a plurality of transmission times. In general, the sensing ring and the smart wristband can establish a communication connection through the wireless network to transmit the above-mentioned transmission time series to the smart wristband through the wireless network. The wireless network may be a WIFI, a WLAN, a Bluetooth connection, etc., and is not specifically limited herein.

S403. Determine a motion trajectory of the sensing finger according to the transmitting time sequence and receiving a receiving time sequence of the electromagnetic wave.

When receiving the electromagnetic wave sent by the sensing ring, the smart wristband can record the receiving time sequence of receiving the electromagnetic wave, so that the smart wristband can be calculated according to the transmission time sequence, the receiving time series, and the propagation rate of the electromagnetic wave in the skin. The position of the ring during the movement is determined, and all the positions of the sensing ring during the movement are determined to determine the movement track of the sensing ring. It can be understood that the motion track of the sensing ring is the motion track of the user to make a touch gesture.

It should be noted that, in order to prevent the smart wristband from recognizing the motion trajectory of all the touch gestures as one point, when the user wears the sensing ring and the smart wristband, the two are required to be worn in two positions with relative displacement. For example, if the smart wristband is worn on the wrist of one of the user's hands, the sensing ring can be worn on one of the fingers of the user's other hand, so that the user wears the smart finger using the sensing ring. When the hand strap or the arm of the hand of the wristband performs handwriting or other touch gestures, the sensing ring and the smart wristband are relatively displaced, and the receiving time sequence of the electromagnetic wristband receiving the electromagnetic wave is changed, and the smart wristband is changed. In other words, the motion trajectory of the sensing finger can be determined according to the transmission time sequence and the receiving time sequence of the received electromagnetic wave.

S404: Send the motion track as the touch information to the head mounted display device, so that the head mounted display device receives the motion track, and determine the motion according to the motion track and the current display content. The track corresponds to the touch command and responds to the touch command.

Further, after the smart wristband determines the motion track of the sensing ring, the motion track can be sent to the head mounted display device. After the head-mounted display device receives the motion track sent by the smart wristband, the head-mounted display device can determine the touch command corresponding to the motion track according to the motion track and the current display content.

For example, if the current display content of the head-mounted display device is a WeChat message input interface, and the motion track sent by the smart wristband is a horizontal line from left to right, then the head-mounted display device can determine the touch corresponding to the motion track. The instruction is: input Chinese character "one"; assuming that the current display content of the head-mounted display device is a picture browsing interface, and the motion track sent by the smart wristband is a horizontal line from left to right, then the head-mounted display device can determine the motion track The corresponding touch command is: view the previous picture.

In order to ensure that the touch commands responded by the head-mounted display device are correct touch commands, to avoid erroneous response, in one embodiment, as shown in FIG. 5, the motion track is used as the touch The step of sending control information to the head mounted display device may include:

S501, recording a formation time of the motion track;

In some cases, the user may perform some misoperations. In order to avoid the impact of the misoperation on the virtual touch, the smart wristband can record the formation time of the motion trajectory of the sensing ring when receiving the electromagnetic wave emitted by the sensing ring. The subsequent process can determine whether the touch gesture corresponding to the motion track is valid according to the formation time of the motion track.

S502, determining, according to the motion trajectory and the forming time, whether the touch gesture corresponding to the motion track is valid, and if yes, executing step S503; if not, performing no operation;

In one possible case, the motion trajectory is an irregular messy trajectory, and is not any Chinese character. The motion trajectory is likely caused by a user's misoperation. At this time, the user does not actually issue any need to wear the display device. The touch command, then the smart wristband can determine that the touch gesture corresponding to the motion track is invalid, and no operation can be performed.

In another possible case, the formation time of the motion trajectory is too long or too short, and it is no longer within a preset time period. The motion trajectory is likely caused by a user's misoperation, and the user does not actually issue any need. When the touch command is executed by the display device, the smart wristband can determine that the touch gesture corresponding to the motion track is invalid, and no operation can be performed.

In other cases, for example, when the motion track is a Chinese character or a regular graphic, and the motion time is within a preset time period, the smart wristband can determine that the touch gesture corresponding to the motion track is valid, and continue to perform step S503.

S503. Send the motion track as the touch information to the head mounted display device.

After the smart wristband determines that the touch gesture corresponding to the motion track is valid, the motion track can be sent to the head-mounted display device as the touch information, and the head-mounted display device can receive the touch information according to the motion. The track and the current display content determine a touch command corresponding to the motion track, and further respond to the touch command.

In another embodiment, the step of sending the motion track as the touch information to the head mounted display device may include:

Recording a formation time of the motion trajectory; transmitting the formation time and the motion trajectory as the touch information to the head mounted display device.

That is to say, after the smart wristband records the formation time of the motion track, the formation time can be sent to the head mounted display device, and the head mounted display device determines whether the touch command corresponding to the motion track is valid.

Correspondingly, the step of the head-mounted display device responding to the touch command may include:

Receiving the forming time; determining, according to the forming time, whether the touch instruction corresponding to the motion track is valid; if yes, responding to the touch instruction.

When the head-mounted display device receives the formation time of the smart wristband transmission, it can determine whether the touch command corresponding to the motion track is valid according to the formation time, and if yes, respond to the touch command, if no, no operation is performed.

The method for determining whether the touch command corresponding to the motion track is valid is the same as the manner in which the touch gesture corresponding to the smart wristband determines the motion track is valid. For details, refer to the above-mentioned smart wristband to determine the touch corresponding to the motion track. The description of the way in which the gesture is valid is not repeated here.

Corresponding to the above method embodiment, the embodiment of the present invention further provides a virtual touch device. The following describes a virtual touch device provided by the embodiment of the present invention.

As shown in FIG. 6 , a virtual touch device is applied to a smart wristband in a virtual touch system, wherein the virtual touch system includes: the smart wristband and a head mounted display device, and the device includes :

The touch information acquiring module 610 is configured to acquire touch information of the user.

The touch information sending module 620 is configured to send the touch information to the head mounted display device, so that the head mounted display device receives the touch information and responds to the touch command corresponding to the touch information.

As an implementation manner of the embodiment of the present invention, the touch information acquiring module 610 may include:

An information acquisition unit (not shown in FIG. 6) for its own motion speed, acceleration, vibration frequency, and exercise time;

a touch action determining unit (not shown in FIG. 6 ) configured to determine a touch action according to the motion speed, the acceleration, the vibration frequency, and the motion time;

The touch information sending module 620 can include:

The first touch information sending unit (not shown in FIG. 6 ) is configured to send the action information of the touch action as the touch information to the head mounted display device.

As an implementation manner of the embodiment of the present invention, the device may further include:

An electromagnetic wave receiving module (not shown in FIG. 6) for receiving electromagnetic waves transmitted by the sensing ring through the skin;

a time sequence receiving module (not shown in FIG. 6), configured to receive, by the wireless network, a transmission time sequence of the electromagnetic wave sent by the sensing ring;

a motion trajectory determining module (not shown in FIG. 6), configured to determine a motion trajectory of the sensing finger according to the transmitting time sequence and receiving a receiving time sequence of the electromagnetic wave;

a motion track sending module (not shown in FIG. 6), configured to send the motion track as the touch information to the head mounted display device, so that the head mounted display device receives the motion track, according to The motion track and the current display content determine a touch command corresponding to the motion track, and respond to the touch command.

As an implementation manner of the embodiment of the present invention, the motion track sending module may include:

a first formation time recording unit (not shown in FIG. 6) for recording a formation time of the motion trajectory;

a touch gesture determining unit (not shown in FIG. 6 ), configured to determine, according to the motion track and the forming time, whether the touch gesture corresponding to the motion track is valid;

The second touch information sending unit (not shown in FIG. 6 ) is configured to send the motion track as the touch information to the head mounted display device when the touch gesture is valid.

a second formation time recording unit (not shown in FIG. 6) for recording a formation time of the motion trajectory;

Forming a time transmitting unit (not shown in FIG. 6) for transmitting the forming time and the motion track as the touch information to the head mounted display device to cause the head mounted display device to receive Determining the formation time, and determining whether the touch instruction corresponding to the motion track is valid according to the forming time, and if yes, responding to the touch instruction.

The embodiment of the present invention further provides a smart wristband, as shown in FIG. 7, including a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 pass through a communication bus. 740 completes communication with each other;

a memory 730, configured to store a computer program;

The processor 710 is configured to perform the following steps when executing the program stored on the memory 730:

Obtain the touch information of the user;

Sending the touch information to the head mounted display device, so that the head mounted display device receives the touch information and responds to the touch instruction corresponding to the touch information.

The step of obtaining the touch information of the user may include:

Get your own speed, acceleration, vibration frequency and exercise time;

Determining a touch action according to the motion speed, acceleration, vibration frequency, and exercise time;

The step of sending the touch information to the head mounted display device may include:

The above method may further include:

Sending the motion track as the touch information to the head mounted display device, so that the head mounted display device receives the motion track, and determining, according to the motion track and the current display content, the motion track corresponding to the motion track a touch command and respond to the touch command.

The step of sending the motion track as the touch information to the head mounted display device may include:

Recording the formation time of the motion trajectory;

Determining, according to the motion trajectory and the forming time, whether the touch gesture corresponding to the motion track is valid;

If so, the motion trajectory is sent to the head mounted display device as the touch information.

Recording the formation time of the motion trajectory;

Transmitting the forming time and the motion track as the touch information to the head mounted display device;

The step of the head mounted display device responding to the touch command includes:

Receiving the formation time;

Determining, according to the forming time, whether the touch instruction corresponding to the motion track is valid;

If yes, respond to the touch command.

The communication bus mentioned in the above smart wristband may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the above electronic device and other devices.

The memory may include a random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; or may be a digital signal processing (DSP), dedicated integration. Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.

The embodiment of the invention further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the following steps are implemented:

Obtain the touch information of the user;

The step of obtaining the touch information of the user may include:

Get your own speed, acceleration, vibration frequency and exercise time;

The above method may further include:

Recording the formation time of the motion trajectory;

Receiving the formation time;

If yes, respond to the touch command.

It can be seen that, in the solution provided by the embodiment of the present invention, when the computer program is executed by the processor, the smart wristband acquires the touch information of the user, and sends the touch information to the head mounted display device, and the head mounted display device receives the touch information. And responding to the touch command corresponding to the touch information. Since the depth camera is not required, the cost of virtual touch is greatly reduced.

For the device, the smart wristband and the storage medium embodiment, since it is basically similar to the system and method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the system and method embodiments.

It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the term "comprises" or "comprises" or "comprises" or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

The various embodiments in the present specification are described in a related manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments.

The above is only the preferred embodiment of the present invention and is not intended to limit the scope of the present invention. Any modifications, equivalents, improvements, etc. made within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims

A virtual touch system, the system comprising: a head mounted display device and a smart wristband, wherein

The smart wristband is configured to acquire touch information of the user, and send the touch information to the head mounted display device;

The head mounted display device is configured to receive the touch information and respond to the touch instruction corresponding to the touch information.
The system of claim 1 wherein:

The smart wristband is specifically configured to acquire a motion speed, an acceleration, a vibration frequency, and a motion time, and determine a touch motion according to the motion speed, the acceleration, the vibration frequency, and the motion time, and perform the motion of the touch motion. Information is sent to the head mounted display device as the touch information.
The system of claim 1 wherein said system further comprises:

a sensing ring for transmitting electromagnetic waves and transmitting a time series of transmission of the electromagnetic waves to the smart wristband;

The smart wristband is further configured to receive the electromagnetic wave through a skin, receive the transmission time sequence through a wireless network, and determine the sensing ring according to the transmitting time sequence and receiving a receiving time sequence of the electromagnetic wave. Moving a track and transmitting the motion track as the touch information to the head mounted display device;

The head-mounted display device is further configured to receive the motion track, determine a touch command corresponding to the motion track according to the motion track and the current display content, and respond to the touch command.
The system of claim 3 wherein:

The smart wristband is further configured to record a forming time of the motion track, and determine, according to the motion track and the forming time, whether a touch gesture corresponding to the motion track is valid, and if yes, the motion Transmitting the track as the touch information to the head mounted display device;

or,

The smart wristband is further configured to record a formation time of the motion track, and send the formation time to the head mounted display device; the head mounted display device is further configured to receive the formation time, and Determining, according to the forming time, whether the touch instruction corresponding to the motion track is valid, and if yes, responding to the touch instruction.
A virtual touch method is applied to a smart wristband in a virtual touch system, wherein the virtual touch system includes: the smart wristband and a head mounted display device, and the method includes:

Obtain the touch information of the user;

Sending the touch information to the head mounted display device, so that the head mounted display device receives the touch information and responds to the touch command corresponding to the touch information.
The method of claim 5, wherein the method further comprises:

Receiving electromagnetic waves transmitted by the sensing ring through the skin;

Receiving, by the wireless network, a transmission time sequence of the electromagnetic wave sent by the sensing ring;

Determining a motion trajectory of the sensing finger according to the transmitting time sequence and receiving a receiving time sequence of the electromagnetic wave;

Sending the motion track as the touch information to the head mounted display device, so that the head mounted display device: receives the motion track, and determines the motion track according to the motion track and the current display content. Corresponding touch commands and responding to the touch commands.
The method of claim 6 wherein:

The step of transmitting the motion track as the touch information to the head mounted display device includes: recording a formation time of the motion track; determining the motion according to the motion track and the forming time Whether the touch gesture corresponding to the track is valid; if yes, the motion track is sent to the head mounted display device as the touch information.

or,

Transmitting the motion trajectory as the touch information to the head mounted display device, so that the head mounted display device: receiving the motion trajectory, determining, according to the motion trajectory and current display content, The step of the touch command corresponding to the motion track and the step of responding to the touch command includes: recording a formation time of the motion track; and transmitting the formation time and the motion track as the touch information to the head The display device is configured to: the head-mounted display device: receiving the motion track and the forming time, determining, according to the motion track and the current display content, a touch command corresponding to the motion track; And determining whether the touch instruction corresponding to the motion track is valid; if yes, responding to the touch instruction.
A virtual touch device is applied to a smart wristband in a virtual touch system, wherein the virtual touch system includes: the smart wristband and the head mounted display device, and the device includes:

The touch information acquiring module is configured to acquire touch information of the user;

The touch information sending module is configured to send the touch information to the head mounted display device, so that the head mounted display device receives the touch information and responds to the touch command corresponding to the touch information.
A smart wristband, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

The method of any one of claims 5-7 when the processor is configured to execute a program stored on the memory.
A computer readable storage medium, wherein the computer readable storage medium stores a computer program, the computer program being executed by a processor to implement the method steps of any one of claims 5-7.