CN115857696A

CN115857696A - Device interaction method, wearable device and storage medium

Info

Publication number: CN115857696A
Application number: CN202211659449.XA
Authority: CN
Inventors: 张亚东; 赵东方
Original assignee: Hubei Xingji Shidai Technology Co Ltd
Current assignee: Hubei Xingji Shidai Technology Co Ltd
Priority date: 2022-06-30
Filing date: 2022-12-22
Publication date: 2023-03-28

Abstract

The embodiment of the application discloses a device interaction method, wearable devices and a storage medium, wherein the method comprises the following steps: determining a control signal characteristic in response to a limb movement of the wearer; determining control operation corresponding to the control signal characteristics according to a preset action instruction mapping relation; transmitting the control operation to a controlled device to control the controlled device to perform a specified function. The problem that the controlled device is high in control difficulty is solved, the interaction difficulty of the controlled device is improved, the convenience degree of control of equipment is improved, and therefore user experience is improved.

Description

Device interaction method, wearable device and storage medium

Technical Field

The embodiment of the application relates to the technical field of computer application, in particular to a device interaction method, wearable devices and a storage medium.

Background

With the development of internet technology, the era of intelligent internet gradually comes into being, the intelligent internet is based on a logistics network, takes platform type intelligent hardware as a carrier, and can pass through an agreed communication protocol and a data interaction standard, and the intelligent internet is an intelligent network for information acquisition, processing, analysis and application between an intelligent terminal and a user. The interaction channel of the user and the intelligent device becomes an important infrastructure of the intelligent internet.

At present, interaction between a user and a smart device often depends on a button, a touch screen, a touch pad or a controller connected with the smart device, but the interaction often has problems in practical application, for example, button operation arranged on the surface of the smart device is often limited by the shape of the smart device, and particularly in smart glasses type wearable devices, the button is difficult to be visually seen by the user, so that the user is difficult to remember the operation mode, the operation position and the like, and misoperation is easily generated; some intelligent wearable devices are also provided with external controllers such as remote controllers and remote control levers, so that the intelligent wearable devices are large in size and difficult to carry, and can only be used in indoor fixed places.

Disclosure of Invention

The embodiment of the application provides an equipment interaction method, wearable equipment and a storage medium, so that a controlled device can be conveniently operated, and the use experience of a user is improved.

According to an aspect of an embodiment of the present application, there is provided a device interaction method, where the method includes:

determining a control signal characteristic in response to a limb movement of the wearer;

determining control operation corresponding to the control signal characteristics according to a preset action instruction mapping relation;

transmitting the control operation to a controlled device to control the controlled device to perform a specified function.

According to another aspect of embodiments of the present application, there is provided a wearable device, wherein the wearable device includes:

the control module, the connecting belt and at least one motion monitoring sensor;

the connecting band is used for wearing the control module and the motion monitoring sensor on a target part with a motion monitoring signal;

the control module and the action monitoring sensor are arranged on the connecting belt and are electrically connected;

the motion monitoring sensor is configured to acquire a motion monitoring signal of the target site;

the control module is configured to receive the action monitoring signals acquired by the action monitoring sensor and trigger control operation according to a stored preset action instruction mapping relation and control signal characteristics of the action monitoring signals.

According to another aspect of the embodiments of the present application, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the device interaction method according to any one of the embodiments of the present application when the computer instructions are executed.

According to the technical scheme, the control signal characteristics are determined through the limb actions based on the wearer, the control operation corresponding to the control information characteristics is determined according to the preset action instruction mapping relation, the control operation is transmitted to the controlled device to achieve the designated function, the problem that the controlled device is high in control difficulty is solved, the interaction difficulty with the controlled device can be reduced, the convenience degree of equipment control is improved, and therefore the use experience of the user is improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they necessarily limit the scope of the embodiments of the present application. Other features of the embodiments of the present application will become readily apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a flowchart of a device interaction method provided according to an embodiment of the present application;

FIG. 2 is a flow chart of another method of device interaction provided in accordance with an embodiment of the present application;

FIG. 3 is an exemplary diagram of a basic action provided in accordance with an embodiment of the present application;

fig. 4 is a schematic structural diagram of a wearable device provided in an embodiment of the present application;

fig. 5 is a schematic structural diagram of another wearable device provided in an embodiment of the present application;

FIG. 6 is a schematic view of a connection band worn according to an embodiment of the present application;

FIG. 7 is a diagram illustrating an example of an arrangement of electromyographic sensor electrodes provided according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a low-pass filter provided according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a high-pass filter according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of an inverting amplifier provided according to an embodiment of the present application;

FIG. 11 is a block diagram of an inverse adder according to an embodiment of the present application;

FIG. 12 is a muscle surface electrical signal detection diagram provided in accordance with an embodiment of the present application.

Detailed Description

In order to make the embodiments of the present application better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments, but not all embodiments, of the embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without any creative effort shall fall within the protection scope of the embodiments in the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of the embodiments of the present application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In the following embodiments, operations and controls of the controlled device by the wearable device are described, and such operations and controls correspond to adjustment of one or more functions of the controlled device, such as zooming in and out of a screen, increasing and decreasing of volume, increasing and decreasing of brightness, confirming and canceling of options, and even turning on and off of the controlled device.

In the following embodiments, the concept of smart glasses is described, which refers to glasses interacting with user-generated functions through their own data processing capability or interacting with user-generated functions through data communication with tablet, mobile phone, computer, etc., including but not limited to AR, VR, MR, bluetooth glasses, etc.

The smart glasses may or may not have a display capability.

In the following embodiments, the term electrical connection refers to a coupling connection implemented by a wireless network, a wired network, and/or any combination of a wireless network and a wired network. The network may include a local area network, the Internet, a telecommunications network, an Internet of Things (Internet of Things) based on the Internet and/or a telecommunications network, and/or any combination thereof, and/or the like. The wired network may communicate by using twisted pair, coaxial cable, or optical fiber transmission, for example, and the wireless network may use 3G/4G/5G mobile communication network, bluetooth, zigbee, or Wi-Fi, for example.

In the following embodiments, the processor may be a Central Processing Unit (CPU), a field programmable logic array (FPGA), a single chip Microcomputer (MCU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Graphics Processing Unit (GPU), or other logical operation device with data processing capability and/or program execution capability.

In the embodiments described below, the memory may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, solid state disk, hard disk, and the like.

Fig. 1 is a flowchart of a device interaction method provided in an embodiment of the present application, where the present embodiment is applicable to a case of terminal device control, and the method may be performed by a device interaction apparatus, where the device interaction apparatus may be implemented in the form of hardware and/or software, and the device interaction apparatus may be configured in a wearable device. As shown in fig. 1, the method includes:

the control signal characteristic is determined in response to a limb movement of the wearer, step 110.

The control signal characteristics may be characteristics for generating signals for controlling the controlled device to execute functions, and the control signal characteristics may have a corresponding relationship with the limb actions, in some embodiments, the control signal characteristics may include electromyographic signals of the wearer under different limb actions, and in other embodiments, the control signal characteristics may include configuring a movement direction of the wearer under different limb actions, and the like. The control signal characteristic may be determined by one or more sensors collecting the wearer's limb movements.

In the embodiment of the application, the wearable device can be worn by a wearer, the wearable device can collect one or more data signals to identify the state of the wearer when the wearer makes various limb actions for controlling a controlled device, and the control signal characteristics can be determined through analysis of the one or more data signals, and the control signal characteristics can be used for determining the control operation for controlling the controlled device. In some embodiments, the wearable device may include an electromyographic sensor that may be used to collect electromyographic signals of the wearer under different limb movements, and the electromyographic signal strength may be determined as a control signal characteristic.

And step 120, determining control operation corresponding to the control signal characteristics according to the preset action instruction mapping relation.

The preset action instruction mapping relationship may be a corresponding relationship including control signal characteristics and control operations, and one or more control signal characteristics in the preset action instruction mapping relationship may be mapped to one control operation. In some embodiments, the preset action instruction mapping relationship may exist in the form of a database table or a configuration file, and the preset action instruction mapping relationship may include a relationship pair of a control signal characteristic and a control operation. The control operation may be an operation that controls a function performed by the controlled device, may include a basic action operation, such as a single click, a double click, a long press, a slide up and down, a slide left and right, a pinch, a grab, a place, a push away, a pull in, a six Degree Of Freedom (DOF) motion, and the like, and in some embodiments may be a combination Of basic action operations, and the control operation may be generated by one or more combinations Of basic action operations.

In the embodiment of the application, a preset action instruction mapping relationship configured in advance may be extracted, where the preset action instruction mapping relationship may specifically include a database table or a configuration file, and a control operation having a corresponding relationship with the determined control signal characteristic may be determined according to the preset action instruction mapping relationship. In some embodiments, the control operation of the associative storage may be looked up according to a database table embodying a preset action instruction mapping relation according to the control signal characteristics.

Step 130, transmitting the control operation to the controlled device to control the controlled device to execute the specified function.

The designated function may be a characteristic function that the wearer controls the controlled device to perform, and there may be a corresponding relationship with the control operation, where the corresponding relationship may be preconfigured by the wearable device or may be preconfigured by the controlled device.

In this application embodiment, wearable equipment can establish communication connection in controlled device, and this communication connection can be based on bluetooth, WIFI, cellular network, 5G network and realize, and wearable equipment can transmit control operation to controlled device through this communication connection, and controlled device can carry out appointed function according to received control operation.

According to the embodiment of the application, the control signal characteristics are determined through the limb actions based on the wearer, the control operation corresponding to the control information characteristics is determined according to the preset action instruction mapping relation, the control operation is transmitted to the controlled device to realize the designated function, the problem that the controlled device is high in control difficulty is solved, the interaction difficulty with the controlled device can be reduced, the convenience degree of equipment control is improved, and therefore the use experience of the user is improved.

Fig. 2 is a flowchart of another device interaction method provided in an embodiment of the present application, which is embodied on the basis of the embodiment of the present application, and referring to fig. 2, the method provided in the embodiment of the present application specifically includes the following steps:

and step 210, constructing a preset action instruction mapping relation between the control signal characteristics and the control operation.

In this embodiment of the application, the wearable device may establish a preset action instruction mapping relationship between the control signal characteristic and the control operation, and the establishment process may be set by the wearable device before leaving a factory or set by a user before wearing the wearable device. In some embodiments, the wearable device may provide an example of a standard limb action, the wearer may perform a movement according to the standard limb action after wearing the wearable device, and the wearable device may collect control signal characteristics under different standard limb actions and establish a preset action instruction mapping relationship between the control signal characteristics and a control operation corresponding to the standard limb action.

And step 220, acquiring surface electromyographic signals corresponding to the limb actions according to the electromyographic sensor of the wearable device.

The electromyographic sensor can measure muscle activity by detecting electric potential so as to identify the limb movement of the wearer, and the surface electromyographic signal can be information generated by the electromyographic sensor by collecting the limb movement of the wearer.

In the embodiment of the application, an electromyographic sensor can be configured in the wearable device, and when a wearer wears the wearable device to perform limb actions, the electromyographic sensor can acquire surface electromyographic signals of the wearer under different limb actions.

And step 230, acquiring the motion direction of the limb action according to the motion direction sensor of the wearable device.

The motion direction sensor can be a sensor for collecting the motion direction of the wearer, and the motion direction sensor can comprise an accelerometer, a gyroscope and the like.

In this embodiment, a motion direction sensor may be configured in the wearable device, and when a wearer performs a limb action, the motion direction sensor may acquire a motion direction of the limb action of the wearer, where the motion direction may include a motion direction on a two-dimensional plane and a motion direction in a three-dimensional space.

And 240, extracting the signal intensity range of the surface electromyogram signal and/or the movement direction of the limb movement as a control signal characteristic.

The signal intensity range may be a value reflecting an upper limit and/or a lower limit of the signal intensity of the surface electromyogram signal.

In the embodiment of the application, the upper limit and/or the lower limit of the signal intensity of the collected surface electromyography information can be determined as the component of the control signal characteristic, and the movement direction of the limb movement collected by the movement direction sensor can be obtained as the control signal characteristic.

And step 250, determining control operation corresponding to the control signal characteristics according to the preset action instruction mapping relation.

Specifically, the corresponding control operation may be searched in the preset action instruction mapping relationship according to the control signal characteristics, and different control signal characteristics may correspond to different control operations.

Further, in some embodiments, the execution sequence between the control operations may also be limited, and the preset action instruction mapping relationship may also include the execution sequence between different control operations.

Step 260, transmitting the control operation to the controlled device according to the preset communication connection with the controlled device.

Wherein, preset communication connection can be the data communication link that wearable equipment and controlled device established, and this preset communication connection can include WIFI communication connection, bluetooth communication connection, 5G communication connection, cellular network communication connection etc..

In this embodiment of the application, the wearable device may establish a preset communication connection with the controlled device in advance, data exchange between the wearable device and the controlled device may be achieved, and the control operation determined by the wearable device may be transmitted to the controlled device through the preset communication connection.

And step 270, controlling the controlled device to execute a designated function corresponding to the control operation in the use scene.

The usage scenario may be a scenario in which the controlled device executes a service function, and may include camera preview, video watching, video call, and the like.

In this embodiment of the application, the controlled device acquires the control operation, may determine a specific function corresponding to the control operation according to the current usage scenario of the controlled device, and the controlled device may execute the execution function, thereby implementing execution of the controlled operation.

In some application embodiments, the specific function corresponding to the control operation in the usage scenario includes at least one of:

in a camera preview scene, the designated function corresponding to the single click operation in the control operation comprises focus selection; in a camera preview scene, the designated function corresponding to the pinch operation in the control operation comprises photographing; in a text reading scene, the designated function corresponding to the single-click operation in the control operation comprises text highlighting; in a text reading scene, the designated function corresponding to the pinch-in operation in the control operation comprises field deletion; in a video watching scene, the designated function corresponding to the push-in operation and push-out operation of the control operation comprises picture zooming-out; in a video watching scene, the designated function corresponding to the upward sliding operation in the control operation comprises the improvement of the brightness of the picture; under the scene of information prompt, the appointed function corresponding to the push-away operation in the control operation comprises reply information; and under the scene of information prompt, the designated function corresponding to the upward sliding operation in the control operation comprises information omission.

According to the embodiment of the application, the preset action instruction mapping relation between the control signal characteristics and the control operation is established, the surface electromyographic signals corresponding to the limb actions are collected according to the electromyographic sensors of the wearable device, the movement directions of the limb actions are collected through the movement direction sensors, the signal intensity range and/or the movement directions of the surface electromyographic signals are/is used as the control signal characteristics, the corresponding control operation is searched in the preset action instruction mapping relation according to the control signal characteristics, the control operation is transmitted to the controlled device, the controlled device is controlled to execute the specified function according to the use scene, the problem that the control difficulty of the controlled device is large is solved, the interaction difficulty of the controlled device can be reduced, the convenience degree of the control of the device is improved, and therefore the use experience of a user is improved.

In some exemplary embodiments, the preset action instruction mapping relationship comprises at least one of:

mapping the characteristics of the control signal of the single-time upward and downward movement of the limb into single-click operation;

the control signal characteristic of the limb moving from top to bottom twice continuously in the first threshold time is mapped into double-click operation;

the limb moves from top to bottom once, and the control signal characteristic which keeps the limb movement unchanged at the end position within the threshold time is mapped into long-time pressing operation;

the control information characteristic of the limb which continuously moves upwards after the short hovering is mapped into an upward sliding operation;

the control information characteristic of the limb which continuously moves downwards after the limb is suspended for a short time is mapped into a gliding operation;

the control information characteristic of the limb which moves to the right after the limb is suspended for a short time is mapped into right sliding operation;

the control information characteristic of the limb continuously moving to the left after the limb is suspended for a short time is mapped into left sliding operation;

after at least two fingers are opened and are suspended for a short time, the characteristic mapping of the control signal of the movement of continuously reducing the opening angle is kneading operation;

and after the at least two fingers are opened and are suspended for a short time, mapping control information for continuing the opposite-direction movement of the at least two fingers into rotation operation.

In the embodiment of the present application, the limb state and the movement direction may be determined by characteristics of the control signal, different combinations of the limb state and the movement direction may be mapped to different control operations, and the preset action instruction mapping relationship may include mapping the limb state and/or the movement direction to a basic action or a combination of basic actions in a two-dimensional plane, where the basic action may include clicking, double clicking, long pressing, up-down sliding, left-right sliding, kneading, expanding, rotating, and the like. The preset action instruction mapping relationship may include:

the control signal characteristic of the single up-to-down movement of the limb is mapped to a single click operation, the control signal characteristic may include the movement direction of the single up-to-down movement of the limb, and in some embodiments, the limb may be specific to a specific limb action, such as a single finger or a fist, in which case, the control signal characteristic may further include the signal intensity range of the surface electromyogram signal.

The control signal characteristic of the limb moving from top to bottom twice within the first threshold time is mapped to a double-click operation, and the control signal may include the moving direction of the limb moving from top to bottom twice within the first threshold time. Further, in some embodiments, the control signal characteristic further comprises a signal strength range of the surface electromyography signal for distinguishing the limb as a designated limb movement.

And mapping the control signal characteristics of the limb moving from top to bottom in a single time and keeping the limb movement unchanged at the end position within the threshold time into long-press operation, wherein the control signal characteristics can comprise the moving direction from top to bottom and the moving direction within the threshold time.

Mapping the control signal characteristics of the limb moving in different directions after the limb is suspended for a short time to be sliding operation in the corresponding direction; the direction may include up, down, left, right, etc.

After at least two fingers are opened and are suspended for a short time, the control signal characteristics of the movement of continuously reducing the opening angle are mapped into the kneading operation, and the control signal characteristics can comprise the movement direction of the movement of continuously reducing the opening angle after the fingers are opened and suspended and the signal intensity range of the surface myoelectric signals of which the opening angles of the fingers are reduced. In some embodiments, the limb action may also be replaced by reducing the distance between the palms after hovering over the palms at a threshold distance.

After at least two fingers are opened and hover for a short time, control information of the opposite-direction movement of the at least two fingers is mapped into rotation operation, and the control signal characteristics can include the movement direction of the opened fingers, the movement direction of the short hover and the movement direction of the fingers in the opposite direction, and can also include the signal strength of surface electromyographic signals of the opened fingers, the short hover and the fingers in the opposite directions.

In other embodiments, the preset mapping relationship of the action command includes:

after at least two fingers are opened and are suspended for a short time, the characteristic mapping of a control signal for continuously reducing the opening angle motion is taken as a grabbing operation;

under the condition that the preorder control operation is grabbing operation, mapping the characteristics of control signals of the opening motion of at least two fingers into placing operation;

after the at least two fingers are opened and are suspended for a short time, continuing mapping control information of the same-direction movement of the at least two fingers into push-off operation;

after the at least two fingers are opened and are suspended for a short time, continuing mapping control information of the movement of the at least two fingers in opposite directions into zooming-in operation;

the control information of the limb doing six-degree-of-freedom motion in the space by taking any motion as a reference is mapped into the corresponding direction degree-of-freedom motion operation.

In this embodiment of the application, the limb state and the movement direction may be determined by characteristics of the control signal, and the preset action instruction mapping relationship may include mapping a combination of different limb states and movement directions to different control operations, where the mapping relationship of the preset action instruction may include mapping the limb state and/or the movement direction to a basic action or a combination of basic actions in a three-dimensional space, and the basic actions may include grabbing, placing, pushing away, pulling close, and 6-degree-of-freedom movement, and the like. The preset action instruction mapping relationship may include:

representing that at least two fingers are opened and are suspended temporarily through the movement direction and the signal intensity range of the myoelectric surface signal, and then continuing to reduce the opening angle movement, wherein the control signal characteristic including the movement direction and the signal intensity range of the myoelectric surface signal is mapped into a grabbing action;

the method comprises the following steps of taking a signal intensity range representing the opening movement direction of at least two fingers and an electromyographic surface signal and taking a preorder control operation as a control signal characteristic, and mapping the control signal characteristic as a placing operation;

mapping the control signal characteristics to push-off operation by representing that the motion directions of the motion of the at least two fingers in the same direction and the signal intensity range of the muscle surface signal are used as the control signal characteristics after the at least two fingers are opened and are hovered for a short time;

mapping the control signal characteristics to zoom-in operation by representing that the motion directions of the motion of at least two fingers in opposite directions and the signal intensity range of the muscle surface signal are used as the control signal characteristics after the at least two fingers are opened and are hovered for a short time;

the motion direction of the limbs doing six-degree-of-freedom motion in the space by taking any motion as a reference and the signal intensity range of the muscle surface signal are taken as control signal characteristics, and the control signal characteristics are mapped into the motion operation of the degrees of freedom in the corresponding direction.

In some application embodiments, the method further comprises: adjusting monitoring information of the limb action corresponding to the control signal characteristic based on user operation habit data, wherein the monitoring information comprises at least one of the hovering time of the short hovering, the opening angle of the opening and the threshold time.

In the embodiment of the application, parameters such as the short hovering time, the opening angle and the threshold time for detecting and using the limb actions during the control signal characteristic acquisition can be adjusted according to the user operation habit data, and specific values of the parameters are increased or reduced, so that the use experience of the wearable device is improved.

In some exemplary embodiments, the limb comprises at least one of: fingers, wrists, palms, arms, heads, legs, feet.

In this embodiment of the application, the wearer may control the controlled device through the movement of the fingers, wrists, palms, arms, heads, legs, feet, and the like, and the control signal characteristics determined in the preset action instruction mapping relationship may be realized by collecting the movement directions of the fingers, wrists, palms, arms, heads, legs, feet, and the like of the wearer and the surface myoelectric signals.

In some exemplary embodiments, the wearable device may control the controlled device through the electromyographic signals, and may use the electromyographic sensors in the wearable device to perform one-to-one mapping with the interaction in the two-dimensional plane, see fig. 3, and the basic actions in the two-dimensional plane may include clicking, double clicking, long pressing, up-and-down sliding, left-and-right sliding, double finger pinching/expanding, and double finger rotating. After the myoelectric action is recognized, the interaction operation between different myoelectric actions and the two-dimensional plane is mapped as follows:

1) Single-finger single-time myoelectric action movement from top to bottom is mapped into single-click operation;

2) Two consecutive top-down movements of a single finger within a certain time threshold are mapped to a double click operation. The time threshold may be determined by system default settings, such as 1 second; the time length which is best matched with the operation habit of the user can be automatically optimized through the training and learning of a software system;

3) After a single-time top-down movement of the single finger, the finger action is kept unchanged at the end position within a certain time threshold, and the action is mapped to long-time pressing operation. The time threshold may be determined by system default settings, such as 1 second; the time length which is best matched with the operation habit of the user can be automatically optimized through the training and learning of a software system;

4) After the single finger or the multiple fingers are hovered for a short time, the operation is continued to be carried out upwards, and the mapping is the upglide operation. The hover time may be determined by system default settings, such as 0.5 seconds; the time length which is best matched with the operation habit of the user can be automatically optimized through the training and learning of a software system;

5) After the single finger or the multiple fingers are hovered for a short time, the operation is continued to be performed downwards and is mapped into the operation of sliding down. The hover time may be determined by system default settings, such as 0.5 seconds; the time length which is best matched with the operation habit of the user can be automatically optimized through the training and learning of a software system;

6) After the single finger or the multiple fingers are hovered for a short time, the operation is continued to the right, and the operation is mapped into the right sliding operation. The hover time may be determined by system default settings, such as 0.5 seconds; the time length which is best matched with the operation habit of the user can be automatically optimized through the training and learning of a software system;

7) After the single finger or the multiple fingers are hovered for a short time, the operation is continued to the left, and the operation is mapped into the left sliding operation. The hover time may be determined by system default settings, such as 0.5 seconds; the time length which is best matched with the operation habit of the user can be automatically optimized through the training and learning of a software system;

8) After the two fingers are opened at a certain angle and are suspended for a short time, the operation of reducing the opening angle is continued, and the two fingers are mapped into the two-finger kneading. The initial opening angle and hovering time can be determined by default settings of the system, such as 50 degrees (included angle formed by extension lines of two pointing wrist ends) and 0.5 second; the initial opening angle and the time length which are best matched with the operation habits of the user can be automatically optimized through the training and learning of a software system;

9) After the two fingers are opened at a certain angle and are suspended temporarily, the operation of increasing the opening angle is continued, and the expansion is mapped as the expansion of the two fingers. The initial opening angle and hovering time can be determined by default settings of the system, such as 20 degrees (included angle formed by extension lines of the directions of the two finger tips) and 0.5 second; the initial opening angle and the time length which are best matched with the operation habits of the user can be automatically optimized through the training and learning of a software system;

10 The two fingers open at a certain angle and hover for a short time, and then move to the opposite directions (front and back, left and right, etc.) of the two fingers in the space, and the movement is mapped into the rotation of the two fingers. The initial opening angle and hovering time can be determined by default settings of the system, such as 20 degrees (included angle formed by extension lines of the directions of the two finger tips) and 0.5 second; the initial opening angle and the time length which are best matched with the operation habits of the user can be automatically optimized through the training and learning of the software system. The discrimination condition of the motion in the opposite directions of the two fingers is distinguished from the aforementioned expansion/kneading of the two fingers in particular in that whether the motion of the two fingers is the rotation of the two fingers is determined by the strength and the directional characteristic of the electromyographic signal.

It is understood that the above mapping operation is performed by way of example only and not limitation, and other limb actions are also within the scope of the embodiments of the present application, such as three-finger kneading or palm action sliding; while different limb movements may be mapped to the same basic operation, for example, palm left-sliding, head left-shaking, single finger left-sliding may all be mapped to left-sliding operations.

In other application embodiments, the wearable device may control the controlled device through the electromyographic signals, one-to-one mapping may be performed between the electromyographic sensors in the wearable device and the interaction in the three-dimensional space, and the basic actions in the three-dimensional space may include grabbing, placing, pushing away, pulling close, 6DOF movement, and the like. After the myoelectric action is recognized, the interaction operation between different myoelectric actions and the two-dimensional plane is mapped as follows:

1) After the two fingers or the multiple fingers are opened at a certain angle and are suspended temporarily, the operation of reducing the opening angle is continued, and the operation is mapped as the grabbing operation. The initial opening angle and the hovering time can be determined by default settings of the system, such as 50 degrees (the maximum value of the included angle formed by the extension lines of the two finger wrist end directions) and 0.5 second; or the initial opening angle and the time length which are matched with the operation habit of the user are optimized automatically through the training and learning of the software system

2) The placing action is only on the grabbing action occurs. If the preorder action is grabbing and the opening angle of two or more fingers is increased, the action is mapped to a placing operation. The threshold value of the increase of the field angle can be determined by default settings of the system, for example, the 50 degrees (the maximum change value of the included angle formed by the extension lines of the directions of the two finger ends) can also be automatically optimized into a plurality of field angle change values which are most matched with the operation habits of the user through the training and learning of the software system

3) And after the palm is unfolded and hovered for a short time, continuing to perform a forward pushing action, and mapping the forward pushing action into a pushing-away operation. The hovering time can be determined by default settings of the system, such as 0.5 second, and can be automatically optimized to the initial opening angle and the time length which are best matched with the operation habits of the user through the training and learning of the software system

4) And after the palm is opened and hovered for a short time, the operation is continued to be pulled backwards, and the operation is mapped into a pulling-in operation. The hovering time can be determined by default settings of the system, such as 0.5 second, and can be automatically optimized to the initial opening angle and the time length which are best matched with the operation habits of the user through the training and learning of the software system

5) And the single finger or the multiple fingers take any motion as a reference, keep the motion unchanged, do 6DOF motion in space and map the motion into the 6DOF motion corresponding to the elements in the 3D scene. The following are exemplary: and 5, slightly gathering to form a sphere, keeping the action, and performing a turning action in a vertical plane, so that the operation can be mapped to the top-bottom turning operation of a certain object in the 3D scene. The push-away/pull-up operation described above, in some embodiments, may be a special case of a 6DOF motion in space.

In the embodiment of the present application, the above mapping is only exemplary and not limiting, and other limb movements are also within the scope of the embodiment of the present application, such as fist making, fist loosing, wrist rotation, etc.; while different limb motions may be mapped to the same basic operation, for example, both palm pushing and punch-breaking motions may be mapped to push-away operations in three-dimensional space.

Fig. 4 is a schematic structural diagram of a wearable device according to an embodiment of the present application, where the wearable device of this embodiment is applicable to a control device, and a controlled device includes, but is not limited to, smart glasses, a smart phone, a laptop computer, a desktop computer, a tablet computer, and the like, and may be, for example, smart glasses. Referring to fig. 1, the wearable device includes: a control module 10, a connecting belt 9 and at least one motion monitoring sensor 11; the connecting belt 9 is used for wearing the control module 10 and the motion monitoring sensor 11 on a target part with a motion monitoring signal; the control module 10 and the motion monitoring sensor 11 are arranged on the connecting belt, and the control module 10 and the motion monitoring sensor 11 are connected; the motion monitoring sensor 11 is configured to acquire a motion monitoring signal of the target site; the control module 10 is configured to receive the motion monitoring signal collected by the motion monitoring sensor 11, and trigger a control operation according to the stored preset motion instruction mapping relation and the control signal characteristic of the motion monitoring signal.

In some embodiments, the control module 10 may store a signal threshold value which may be used for comparison with the collected surface electromyographic signals of the motion monitoring sensor 11, which may be implemented by hardware circuitry or by a processor.

In other embodiments, a comparator circuit may be disposed in the control module 10, and may compare the electromyographic signal input by the electromyographic sensor with a signal corresponding to a signal threshold, and may trigger different preset action control signals according to the comparison result. It can be understood that different preset motion control signals can trigger the controlled object to perform different motions, and the preset dynamic control signal can be implemented by using signals with different levels. For another example, the control module 10 may be provided with a processor, may perform analog-to-digital conversion on the electromyographic signal, may compare the electromyographic signal with a signal threshold in a digital signal form, and may trigger different preset action control signals according to the comparison result. The motion monitoring sensor 11 can measure muscle activity through detecting electric potential, and the motion monitoring sensor 11 and the control module 10 can be wearable through the connecting band 9 realization for wear the motion monitoring sensor 11 and the control module 10 in user's health, and the connecting band 9 can be elastic material or non-elastic material, and the connecting band 9 is worn the motion monitoring sensor 11 and the control module 10 in user's health with the form of binding or constraint.

In some embodiments, the attachment strap 9 may have an adhesive to attach the motion monitoring sensor 11 and the control module 10 to the user's body.

In some embodiments, the connection band 9 may also be part of the smart wearable device, for example, the connection band 9 may be a wrist band of a smart watch, and the control module 10 and the motion monitoring sensor 11 may be integrated in the smart watch. It should be understood that fig. 4 only shows an illustration of the connection relationship between the connection belt 9 and the motion monitoring sensor 11 and the control module 10, the number of the connection belts 9 may be multiple, and the myoelectric sensor 11 and the control module 10 may be worn on the body of the user by different connection belts 9.

In an exemplary embodiment, the connection band 9 is any one of a lace, a wrist band, a hair band, and a watch band.

In some embodiments, referring to fig. 5, the control module 10 includes a memory 12, the memory 12 being configured to store signal thresholds for determining characteristics of the control signal, the signal thresholds including at least one set of upper and/or lower limits for a surface electromyographic signal corresponding to the target site, the upper and/or lower limits for each of the surface electromyographic signals corresponding to different states of muscle action at the target site.

In some embodiments, the motion monitoring sensor 11 includes at least one of a myoelectric sensor and a motion direction sensor.

In the embodiment of the present application, one or more sensors may be configured in the wearable device as motion monitoring sensors for determining the motion of the limb of the wearer, and the motion monitoring sensors 11 may include one or more of myoelectric sensors and motion direction sensors.

In the embodiment of the present application, the control module 10 is provided with a memory 12, the control module 10 may generate signals of corresponding levels according to signal thresholds stored in the memory 12, so as to implement comparison with the electromyographic signals, it is understood that the target portion may be a wearing portion of the wearable device, the signal thresholds may be one or more sets, each set of signal thresholds may include a maximum value or a minimum value of a muscle-skin surface current of a body of the user in different muscle movement states, and the signal thresholds may be determined through a large amount test or obtained by the user according to a specified motion measurement. For example, the motion monitoring sensor 11 may collect a surface electromyographic signal of the user's arm as a signal threshold.

Illustratively, a simple way is that the wearer can make four different actions as flag states while wearing the wearable device, four actions being bending upwards: the palm is upward and bent up and down; bending downwards: the palm is downward and bent up and down; bending to the left: the palm is bent leftwards and rightwards; bending to the right: the palm is bent to the right and left. It is understood that the above four actions are only examples, and other actions such as making a fist, opening a fist, etc. are also within the scope of the embodiments of the present application. The embodiment of the application can collect the muscle skin surface current corresponding to the unbending and the limit bending of the user in the four actions, and can store the upper limit and the lower limit of the muscle skin surface current as the signal threshold. It can be understood that the above four actions can be preset actions for controlling the controlled object, and the user can control the controlled object to execute different functions when making different preset actions.

In some embodiments, the upper and/or lower limits of the at least one set of muscle-skin surface currents in memory 12 are obtained from acquiring corresponding muscle action states of the user, or from corresponding averages of the user.

Specifically, the upper limit and/or the lower limit of the muscle-skin surface current in the memory 12 may be the skin surface current value acquired by the user in the specified muscle action state, or may be an average value of the upper limit and the lower limit of the skin surface current value of a certain number of sample users in the specified muscle action state in the current gender of the user in the current age group.

In some embodiments, the electromyographic sensor comprises at least 1 set of bipolar electrodes, which are uniformly arranged on the connecting band 9. The electromyographic sensor comprises 4 groups or 6 groups of bipolar electrodes which are arranged at equal intervals, and at least one group of bipolar electrodes faces the target part in the longitudinal direction.

In the embodiment of the present application, at least 4 sets of bipolar electrodes, for example, 4 or 6 sets of bipolar electrodes, may be used as the electromyographic sensor in the motion monitoring sensor 11, and the electromyographic sensor 11 is disposed on the connecting band 9, and each set of bipolar electrodes may be uniformly distributed on the connecting band 9. In an exemplary embodiment, referring to fig. 6, the electromyographic sensor may be disposed in a wrist band. The arrangement of the electrodes may be such that at least one set of bipolar electrodes is facing the target site in the longitudinal direction, i.e., the arms are laid with the palms flattened to the horizontal direction, and a set of bipolar electrodes is placed in the vertical and horizontal direction, as shown in fig. 7.

In some embodiments, as shown in fig. 6, the connecting band is provided at a position 3-5cm away from the elbow (the thickest position of the forearm) in the direction close to the palm.

In some application embodiments, referring to fig. 5, the wearable device further includes at least one electromyographic signal processing circuit 14, two ends of the electromyographic signal processing circuit 14 are respectively connected to the control module 10 and the bipolar electrode, and the electromyographic signal processing circuit is configured to filter and amplify the surface electromyographic signal.

Specifically, the electromyographic signal processing circuit 14 may be connected to a bipolar electrode of an electromyographic sensor in the motion monitoring sensor 11, and may process the acquired electromyographic signal, and the electromyographic signal processing circuit 14 may be configured to improve accuracy of the electromyographic signal, reduce an influence of an environmental factor on the electromyographic signal, and perform processing such as filtering and amplification on the electromyographic signal through the electromyographic signal processing circuit 14. The electromyographic signal processing circuitry 14 may also be connected to the control module 10 for transmitting the processed electromyographic signals to the control module 10.

In an exemplary embodiment, the electromyographic signal processing circuit 14 includes a differential amplifier, a low-pass filter, a high-pass filter, an inverse amplifier and an inverse adder, which are connected in sequence, the electromyographic signal collected by the bipolar electrode is used as an input of the differential amplifier, the electromyographic signal is amplified by the differential amplifier, then clutter is filtered by the low-pass filter and the high-pass filter in sequence, the amplification ratio of the electromyographic signal is adjusted by the inverse amplifier, and finally the electromyographic signal is input into the inverse adder to be pulled up to a positive voltage.

Specifically, each bipolar electrode of the electromyographic sensor in the motion monitoring sensor 11 may be connected to a differential amplifier, the electromyographic signal collected by the bipolar electrode may be input to the differential amplifier, the differential amplifier may amplify the electromyographic signal, the amplified electromyographic signal may sequentially flow into a low-pass filter shown in fig. 8 and a high-pass filter shown in fig. 9, clutter in the electromyographic signal is filtered by the low-pass filter and the high-pass filter, then the electromyographic signal is further amplified by a reverse amplifier shown in fig. 10, and finally the amplified electromyographic signal is input to a reverse adder shown in fig. 11, so as to increase the level of the electromyographic signal and pull up the electromyographic signal to a positive voltage. As an example, the differential amplifier can use INA-128UA of Texas instrument, and the low-pass filter and the high-pass filter respectively adopt 450Hz and 10Hz specifications to meet the overlapping of main energy frequency bands (50-100 Hz) of Electromyogram (EMG) and overcome 60Hz daily environmental electric power signal interference.

Referring to fig. 5 in some exemplary embodiments, the wearable device further comprises: a sensor start module 15, the sensor start module 15 being connected to the motion detection sensor 11, the sensor start module 15 being configured to trigger the motion detection sensor 11 to be powered on in response to detecting the motion signal.

Specifically, the wearable device may be provided with a sensor starting module 15 for reducing power consumption, and the sensor starting module 15 may determine that a starting point of a preset action occurs, where the starting point may include setting different ways for different types of action monitoring sensors, for example, a motion direction sensor belonging to the action monitoring sensors may collect a severe direction change as a starting point, and an electromyographic sensor belonging to the action monitoring sensors may set a sudden fluctuation of an electrical signal on a muscle surface as a starting point. The sensor starting module 15 may start the motion monitoring sensor when monitoring the starting point, and it is understood that the motion monitoring sensor includes multiple types of sensors, and the sensor starting module 15 starts all types of motion monitoring sensors when monitoring the starting point of one of the types of sensors. In an exemplary embodiment, the start detection point shown in fig. 12, in one embodiment, although it is possible to rely solely on the sudden fluctuation cycle of the muscle surface electrical signal as the start detection point, the myoelectric sensor is required to perform continuous detection, resulting in an increase in power consumption of the wearable device. In another embodiment, a sensor starting module 15 may be disposed in the wearable device, and may use a starting point for detecting a preset action, and trigger the electromyographic sensor to power on when the starting point is detected. The sensor starting module 15 may be specifically an accelerometer or a gyroscope to detect a starting point of a preset action and trigger the electromyographic sensor to power on when the starting point is detected.

In one exemplary embodiment, the sensor activation module includes at least an accelerometer and a gyroscope.

Specifically, the wearable device determines that the muscle action of the user changes through the accelerometer and the gyroscope together, so as to activate the action monitoring sensor 11. The accelerometer and the gyroscope can also be arranged in the wearable device in an integrated inertial sensor IMU mode, the inertial sensor usually integrates the accelerometer, the gyroscope and the magnetometer, and when the IMU detects that a person turns over corresponding preset actions, such as arm movement, arm rotation and the like, the myoelectric sensor is started to detect the electric signals on the surface of the muscle.

In an exemplary embodiment, the wearable device serves as a non-contact control device for controlling a head-mounted smart device such as an AR or VR, and it is understood that the controlled device is not limited to the AR or VR head-mounted smart device, and the controlled device of the wearable device includes but is not limited to smart glasses, a smart phone, a laptop, a desktop computer, a tablet computer, and the like.

In some embodiments, the wearable device comprises a sensor and a control module, wherein the sensor comprises an electromyography sensor and a first processor, the electromyography sensor is worn on the arm of the user and is used for acquiring an electric signal of the surface of muscle of the arm of the user; the control module group comprises a second processor and a wireless communication element, the control module group is used for acquiring an electric signal of the sensor and translating the electric signal into an operation intention of a user according to a preset signal mode, and the wireless communication element is used for sending an operation instruction sent by the second processor to paired head-mounted intelligent equipment in a Bluetooth, WIFI and other wireless modes.

In some embodiments, the first processor and the second processor are the same processor, which may be programmed to perform the functions of the first and second processors simultaneously or non-simultaneously.

The wearable device provided by the embodiment of the application can be independent, such as an EMG bracelet; also can be integrated, for example in the watchband of wearable smart machine such as intelligent wrist-watch, intelligent bracelet, can multiplexing wireless communication modules such as wearable smart machine's treater and bluetooth, wi-Fi, only need in the watchband medial surface of wrist-watch integrated flesh electric sensor can to simplify the design, reduce cost.

In the embodiment of the application, the operation intention of the user can be judged by only acquiring the surface electromyographic signals of the arm of the user. For example, a simple way is that the user makes four different actions as four sign states (upward bending: palm upward, palm bending up and down, downward bending: palm downward, palm bending up and down, leftward bending: palm leftward, palm bending left and right, rightward bending: palm bending rightward, palm bending right and palm bending left and right), and the different actions correspond to different operation intentions, such as sound increase reduction, image brightness increase reduction, and the like. This is only an example and other preset actions may be provided, such as making a fist and opening, etc.

The principle of the wearable device works by storing a set of detection thresholds in a memory of a control module, wherein the set of detection thresholds correspond to the muscle skin surface currents of the upper limit and the lower limit of different action states of the muscle, for example, corresponding values are set as On and Off points respectively, the On and Off points serve as the upper limit and the lower limit of the surface current caused by muscle activity, and the signals detected by the electromyographic sensor are separated from the original signals, so that whether the set action state of the muscle occurs or not is accurately judged.

In order to improve the detection accuracy and realize the arrangement on a wrist strap or a watch strap of a smart watch, the electromyographic sensor usually uses 6 groups or 4 groups of bipolar electrodes, for example, 6 groups of bipolar electrodes are adopted, the electrodes are arranged in a manner that a palm is flattened and opened downwards as a starting point, a position 3-5cm away from the thickest position of the forearm is used as the starting point, and the electrodes are sequentially arranged at equal intervals. For the acquisition of the On and Off thresholds, there may be many ways to acquire the On and Off thresholds, for example, after the user wears the device, the user first performs a standard action corresponding to a preset action, for example, the palm is restored to the initial state from the initial state to the limit state, the values of the upper and lower limits of the process are recorded, for example, the palm faces downward to the desktop in the 0 th second, the palm faces downward to the desktop in the 1 st second, the palm faces downward to the desktop in the 2 nd second, the palm bends downward to the limit position in the 2 nd second according to the preset action, the 3 rd second stays at the limit position, the 4 th second is restored to the initial state from the limit position, the 5 th second keeps the initial state, and the 6 th second keeps the initial state. The upper and lower limits of the process are stored as On, off, and so On, and the values of several preset actions are recorded in the memory. The average values of On and Off under the conditions of different age groups, different genders and the like can be counted in advance by finding a large number of users in a counting mode, and when the equipment is started for the first time, the users can select the conditions of age, gender and the like, so that the preset average values are directly applied.

In the embodiment of the application, the wearable device uses the myoelectric sensor to judge the position of a starting detection point existing in wrist movement, and the simple sudden fluctuation cycle of the electric signal on the surface of the muscle can be used as the starting detection point, but the monitoring mode needs the myoelectric sensor to be always turned on and continuously judges whether the signal appears or not, so that the power consumption and the operation pressure are increased. Therefore, an Inertial sensor (IMU) may be further integrated in the sensor in the wearable device, and when it is detected that the human is turning over the corresponding preset action, such as movement of the arm, rotation of the arm, and the like, the electromyographic sensor is started to detect the electrical signal on the surface of the muscle, and usually the electromyographic signal is collected continuously for 6 seconds.

In the embodiment of the application, in order to improve the accuracy of the electromyographic signals and reduce the influence of the environment, necessary elements can be added in the circuit of the wearable device, the electromyographic sensor uses bipolar electrodes, each bipolar electrode is used as the input of a differential amplifier, amplified signals are filtered out of noise waves by a low-pass filter and a high-pass filter in sequence, the amplification factor is further adjusted by a reverse amplifier, and the amplified signals are input to a reverse adder to improve the signal level and are pulled to positive voltage, so that the amplified signals are converted into digital signals through an analog-to-digital converter and are output to a processor. The differential amplifier can use INA-128UA of Texas instruments, and the low-pass filter and the high-pass filter respectively adopt the specifications of 450Hz and 10Hz so as to meet the overlapping of main energy frequency bands (50-100 Hz) of EMG and overcome the interference of 60Hz daily environment electric power signals. The second processor of the wearable device compares the collected signals with data stored in the memory in advance, judges whether the user sends the intention of operating the VR or AR headset according to the operation mode, and sends the intention to the headset through the wireless communication element, and accordingly corresponding operation is generated.

In some application embodiments, the device interaction method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as a memory unit. In some application embodiments, part or all of the computer program may be loaded and/or installed on the wearable device via the ROM and/or the communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the device interaction method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the device interaction method by any other suitable means (e.g., by way of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Computer programs for implementing the methods provided by embodiments of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.

In the context of embodiments of the present application, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described herein may be implemented on a wearable device, which may have: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the wearable device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the embodiments of the present application may be executed in parallel, may be executed sequentially, or may be executed in different orders, as long as the desired results of the technical solutions of the embodiments of the present application can be achieved, which is not limited herein.

The above detailed description does not limit the scope of the embodiments of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the embodiments of the present application, are intended to be included within the scope of the embodiments of the present application.

Claims

1. A device interaction method is applied to a wearable device, and comprises the following steps:

2. The method of claim 1, further comprising:

and constructing a mapping relation between the control signal characteristics and the preset action instruction of the control operation.

3. The method of claim 1, wherein determining the control signal characteristic in response to a limb movement of the wearer comprises:

acquiring surface electromyographic signals corresponding to the limb actions according to an electromyographic sensor of the wearable device;

acquiring the motion direction of the limb action according to a motion direction sensor of the wearable device;

and extracting the signal intensity range of the surface electromyogram signal and/or the movement direction of the limb movement as the control signal characteristic.

4. The method of claim 3, wherein the preset action command mapping relationship comprises at least one of:

5. The method of claim 3, wherein the preset action command mapping relationship comprises:

6. The method of claim 4 or 5, further comprising:

adjusting monitoring information of the limb action corresponding to the control signal characteristic based on user operation habit data, wherein the monitoring information comprises at least one of the hovering time of the short hovering, the opening angle of the opening and the threshold time.

7. The method of claim 4 or 5, wherein the limb comprises at least one of: fingers, wrists, palms, arms, heads, legs, feet.

8. The method of claim 1, wherein the transmitting the control operation to a controlled device to control the controlled device to perform a specified operation comprises:

transmitting the control operation to the controlled device according to a preset communication connection with the controlled device;

and controlling the controlled device to execute the designated function corresponding to the control operation under the use scene.

9. The method according to claim 8, wherein the designated function corresponding to the control operation in the usage scenario comprises at least one of:

in a camera preview scene, the designated function corresponding to the single click operation in the control operation comprises focus selection;

in a camera preview scene, the designated function corresponding to the pinch-in operation in the control operation comprises photographing;

in a text reading scene, the designated function corresponding to the single-click operation in the control operation comprises text highlighting;

in a text reading scene, the designated function corresponding to the pinch-in operation in the control operation comprises field deletion;

in a video watching scene, the designated function corresponding to the push-in operation and push-out operation of the control operation comprises picture zooming-out;

in a video watching scene, the designated function corresponding to the upward sliding operation in the control operation comprises the improvement of the brightness of the picture;

under the scene of information prompt, the appointed function corresponding to the push-away operation in the control operation comprises reply information;

and under the scene of information prompt, the designated function corresponding to the upward sliding operation in the control operation comprises information omission.

10. A wearable device, characterized in that the wearable device comprises: the control module, the connecting belt and at least one motion monitoring sensor;

the control module and the action monitoring sensor are arranged on the connecting belt and connected with each other;

11. The wearable device according to claim 10, wherein the control module comprises a memory configured to store signal thresholds determining the control signal characteristics, the signal thresholds comprising at least one set of upper and/or lower limits of surface electromyographic signals corresponding to the target site, each of the upper and/or lower limits of surface electromyographic signals corresponding to a different state of muscle action at the target site.

12. The wearable device of claim 10, comprising a communication module coupled to the control module for transmitting the preset motion control signal to a controlled device.

13. The wearable device of claim 10, wherein the controlled device comprises smart glasses.

14. The wearable device according to claim 10, wherein the motion monitoring sensor comprises at least one of a myoelectric sensor, a motion direction sensor.

15. The wearable device according to claim 14, wherein the electromyographic sensor comprises at least 1 set of bipolar electrodes, the bipolar electrodes being uniformly disposed on the connection band.

16. The wearable device according to claim 14, wherein the electromyographic sensors comprise 4 or 6 equally spaced sets of bipolar electrodes, at least one set facing the target site in a longitudinal direction.

17. The wearable device according to claim 15, further comprising at least one electromyographic signal processing circuit, wherein two ends of the electromyographic signal processing circuit are respectively connected to the control module and the bipolar electrode, and the electromyographic signal processing circuit is configured to filter and amplify the surface electromyographic signal.

18. The wearable device according to claim 14, wherein the electromyographic signal processing circuit comprises a differential amplifier, a low-pass filter, a high-pass filter, an inverting amplifier and an inverting adder, the electromyographic signal collected by the bipolar electrode is used as an input of the differential amplifier, the electromyographic signal is amplified by the differential amplifier and then filtered by the low-pass filter and the high-pass filter in sequence to filter clutter, the inverting amplifier adjusts the amplification factor of the electromyographic signal, and finally the electromyographic signal is input into the inverting adder to be pulled up to a positive voltage.

19. The wearable device of claim 10, further comprising: a sensor start module connected with the motion monitoring sensor, the sensor start module configured to trigger the motion monitoring sensor to power up in response to detecting a motion signal.

20. The wearable device of claim 19, wherein the sensor activation module comprises at least an accelerometer and a gyroscope.

21. The wearable device of claim 10, wherein the connection band is any one of a lace, a wrist band, a hair band, and a watch band, and the target site comprises a wrist of a user wearing the wearable device.

22. A computer-readable storage medium storing computer instructions for causing a processor to perform the device interaction method of any one of claims 1-9 when executed.