TWI722735B - Method, device and system for using fitness equipment to recognize limb movement and calculate limb movement power - Google Patents

Abstract

A method for using a fitness equipment to recognize a limb movement is provided. The method includes: obtaining a three-axis acceleration signal and a three-axis angular velocity signal of a limb movement in a space rectangular coordinate system by an inertial measurement unit (IMU) mounted on a fitness equipment; obtaining a known motion track of the limb motion; dividing the three-axis acceleration signal and the three-axis angular velocity signal into a plurality of signal segments according to the known motion track; and recognizing an action of the limb movement according to the signal segments.

Description

Method, device and system for recognizing limb movement and calculating limb movement power using fitness equipment

The present disclosure generally relates to methods, devices, and systems for recognizing limb movement actions and calculating limb movement power, and more specifically relates to a method, device, and system for recognizing limb movement actions and calculating limb movement power using fitness equipment.

In recent years, with the development of sensor technology, the application of motion recognition has become a focus of attention. Simple motion recognition technologies, such as pedometers, have been widely used in wearable devices such as smart bracelets. However, the motion recognition function of this type of device is relatively single, which is far from being able to meet the needs of recognizing more motions in physical training.

The current mainstream motion recognition technologies include image sensor-based and motion sensor-based motion recognition technologies.

The motion recognition technology based on image sensors mainly collects visual images through CCD or CMOS sensors, analyzes and processes the collected images, and realizes feature extraction and motion recognition. The main disadvantage of this type of method is that during the action collection process, the actual position of the sensor and its relative position to the human body are relatively fixed, and it is not suitable for portable devices that can be carried around. The amount of data generated by the sensor is very large. The image processing process often requires powerful DSPs and FPGAs to complete, which is not suitable for low-cost, low-power application scenarios.

The motion recognition technology based on motion sensors is mainly based on inertial sensors (for example, gyroscopes and other devices), but the gyroscope itself has the problem of drift, which is easy to cause displacement and trajectory recognition errors.

Therefore, there is a need for a method, device, and system for recognizing the movement of the limbs and calculating the movement power of the limbs by using fitness equipment to solve the above-mentioned problems.

The content disclosed below is only exemplary, and is not meant to be restricted in any way. In addition to the illustrative aspects, embodiments, and features, other aspects, embodiments, and features will also be apparent by referring to the drawings and the following specific embodiments. That is, the content disclosed below is provided to introduce the concepts, key points, benefits, and novel and non-obvious technical advantages described herein. The selected, not all, examples will be described in further detail as follows. Therefore, the content disclosed below is not intended to be an essential feature of the claimed subject matter, nor is it intended to be used in determining the scope of the claimed subject matter.

Therefore, the main purpose of the present disclosure is to provide a method, device and system for recognizing the movement of the limbs and calculating the movement power of the limbs using fitness equipment, so as to improve the above-mentioned shortcomings.

This disclosure proposes a method for using fitness equipment to identify movement of limbs, including: obtaining the movement of a limb in a rectangular coordinate system by an Inertial Measurement Unit (IMU) installed on a fitness equipment A three-axis acceleration signal and a three-axis angular velocity signal; obtain a known motion trajectory of the limb movement; divide the three-axis acceleration signal and the three-axis angular velocity signal into plural signal segments according to the known motion trajectory; and The above-mentioned plural signal segmentally recognizes an action of the above-mentioned limb movement.

The present disclosure proposes a device for recognizing body movements using fitness equipment, including: one or more processors; and one or more computer storage media for storing computer readable instructions, wherein the processor uses the computer storage media to Implementation: Obtain a three-axis acceleration signal and a three-axis angular velocity signal of a limb movement in a spatial rectangular coordinate system by an Inertial Measurement Unit (IMU) installed on a fitness equipment; obtain the above limb A known motion trajectory of the movement; divide the three-axis acceleration signal and the three-axis angular velocity signal into a plurality of signal segments according to the known motion trajectory; and identify an action of the limb movement based on the plurality of signal segments.

This disclosure proposes a system that uses fitness equipment to recognize the movement of limbs, including: a fitness equipment, including: an inertial sensor, which obtains a three-axis acceleration signal and a three-axis acceleration signal of a limb movement in a spatial orthogonal coordinate system Angular velocity signal; a camera that shoots the fitness equipment to obtain a known movement track of the limb movement; and an electronic device connected to the fitness equipment to receive the three-axis acceleration signal and the three-axes transmitted by the fitness equipment The angular velocity signal and the known motion trajectory transmitted by the camera; the three-axis acceleration signal and the three-axis angular velocity signal are divided into multiple signal segments based on the known motion trajectory; and the limb motion is identified based on the multiple signal segments One action.

This disclosure proposes a method for calculating the power of a limb using a fitness equipment, which includes: obtaining the movement of a limb in a rectangular coordinate system by an Inertial Measurement Unit (IMU) installed on a fitness equipment A three-axis acceleration signal and a three-axis angular velocity signal; to obtain a known motion trajectory of the above-mentioned limb movement; according to the above-mentioned known motion trajectory, the above-mentioned three-axis acceleration signal and the above-mentioned three-axis angular velocity signal are divided into plural signal segments to obtain Corresponding to a number and a time of the above-mentioned limb movement; performing quaternion calculation and quaternion rotation on the above-mentioned three-axis acceleration signal and the above-mentioned three-axis angular velocity signal to obtain a quaternion conversion result; removing the above through a band-pass filter The quaternion conversion result is one of gravity; and the quaternion conversion result after the gravity is removed, the number of times, and the time required to obtain the power consumption of one of the limb movements.

The present disclosure proposes a device for calculating the power of a limb using fitness equipment, including: one or more processors; and one or more computer storage media storing computer readable instructions, wherein the processor uses the computer storage media to Implementation: Obtain a three-axis acceleration signal and a three-axis angular velocity signal of a limb movement in a spatial rectangular coordinate system by an Inertial Measurement Unit (IMU) installed on a fitness equipment; obtain the above limb A known motion trajectory of the movement; according to the known motion trajectory, the three-axis acceleration signal and the three-axis angular velocity signal are divided into multiple signal segments to obtain a number and a time corresponding to the movement of the limb; The acceleration signal and the three-axis angular velocity signal perform quaternion calculation and quaternion rotation to obtain a quaternion conversion result; remove one of the gravity of the quaternion conversion result through a band-pass filter; and remove the gravity according to The subsequent quaternion conversion result, the above-mentioned number of times and the above-mentioned time obtain one of the power consumption of the above-mentioned limb movements.

This disclosure proposes a system that uses fitness equipment to calculate the power of the limbs, including: a fitness equipment, including: an inertial sensor, which obtains a three-axis acceleration signal and a three-axis movement in a spatial orthogonal coordinate system Angular velocity signal; a camera that shoots the fitness equipment to obtain a known movement track of the limb movement; and an electronic device connected to the fitness equipment to receive the three-axis acceleration signal and the three-axes transmitted by the fitness equipment The angular velocity signal and the known motion trajectory transmitted by the camera; according to the known motion trajectory, the three-axis acceleration signal and the three-axis angular velocity signal are divided into multiple signal segments to obtain a number and one corresponding to the motion of the limb Time; perform quaternion calculation and quaternion rotation on the three-axis acceleration signal and the three-axis angular velocity signal to obtain a quaternion conversion result; remove one of the gravity of the quaternion conversion result through a band-pass filter; And according to the quaternion conversion result after the above-mentioned gravity is removed, the above-mentioned number of times and the above-mentioned time, the power consumption of one of the above-mentioned limb movements is obtained.

Hereinafter, various aspects of the present disclosure will be described more fully with reference to the accompanying drawings. However, the present disclosure can be embodied in many different forms and should not be construed as being limited to any specific structure or function presented throughout the present disclosure. On the contrary, the provision of these aspects will make this disclosure comprehensive and complete, and this disclosure will fully convey the scope of the disclosure to those skilled in the art. Based on the content taught in this article, those skilled in the art should realize that no matter whether it is implemented alone or in combination with any other aspect of this disclosure, the scope of this disclosure is intended to cover any aspect disclosed in this article. For example, it can be implemented using any number of devices or execution methods proposed herein. In addition, in addition to the various aspects of the present disclosure set forth herein, the scope of the present disclosure is intended to cover devices or methods implemented using other structures, functions, or structures and functions. It should be understood that any aspect disclosed herein can be embodied by one or more elements in the scope of the patent application.

The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any aspect of this disclosure or a design described herein as "exemplary" is not necessarily construed as being preferred or superior to other aspects of this disclosure or design. In addition, the same number indicates the same element in all the several figures, and unless otherwise specified in the description, the articles "a" and "above" include plural references.

It will be understood that when an element is referred to as being “connected” or “coupled” to another element, the element can be directly connected or coupled to the other element or intervening elements may be present. Conversely, when the element is said to be "directly connected" or "directly coupled" to another element, there are no intervening elements. Other words used to describe the relationship between elements should be interpreted in a similar way (for example, "between" and "directly between", "adjacent" and "directly adjacent", etc.).

Fig. 1 shows an exemplary schematic diagram of a system 100 that uses fitness equipment to recognize movement of limbs according to an embodiment of the present invention. The system 100 may include a fitness equipment 110 and an electronic device 130 connected to the network 120.

The fitness equipment 110 may at least include an Inertial Measurement Unit (IMU) (not shown in the figure) and a light ball 112 (also referred to as a Marker or Optical). The inertial sensor can be installed inside the fitness equipment 110, and it can include an accelerometer and a gyroscope for detecting motion, and is used to detect the three-axis acceleration signal and the three-axis angular velocity signal of the subject using the fitness equipment 110 to perform limb movement.

The light ball 112 can be installed on the outside of the fitness equipment 110 for easy disassembly. The user can use a high-resolution camera (not shown in the figure) to photograph and record the movement track of the detected object using the fitness equipment 110 to perform limb movement. In more detail, infrared emitters are set around these high-resolution cameras, and high-frequency infrared flashes that are invisible to the naked eye are projected. The light ball 112 reflects infrared light spots. After receiving the light spot through a high-resolution camera, the light spot information can be transmitted to the electronic device 130 for calculation to obtain the three-dimensional X, Y, and Z displacement movement of the light ball 112 in the physical space, so as to further obtain the fitness equipment 110 Movement trajectory.

The electronic device 130 can receive the three-axis acceleration signal and the three-axis angular velocity signal transmitted by the inertial sensor through the network 120 or receive the motion track transmitted by the camera that shoots the photosphere 112 through the network 120. The types of the electronic device 130 range from small handheld devices (for example, mobile phones/portable computers) to large host systems (for example, large computers). Examples of portable computers include personal digital assistants (PDAs), notebook computers, and other devices. The network can be any type of network familiar to those skilled in the art, and it can use any of various communication protocols available to support data communication, including but not limited to TCP/IP and so on. For example, the network can be a local area network (LAN), such as an Ethernet network, etc., a virtual network, including but not limited to a virtual private network (Virtual Private Network, VPN). ), the Internet, wireless networks and/or any combination of these and/or other networks.

It is worth noting that although the fitness equipment 110 uses dumbbells as an example in Figure 1, and limb exercises are based on upper limb exercises as an example in Figure 1, the present invention should not be limited to this.

It should be understood that the electronic device 130 shown in FIG. 1 is an example of the architecture of a system 100 that uses fitness equipment to recognize movement of limbs. Each element shown in Figure 1 can be implemented by any type of computing device, such as the computing device 1200 described with reference to Figure 12, as shown in Figure 12.

FIG. 2 shows a flowchart of a method 200 of using fitness equipment to recognize movement of limbs according to an embodiment of the present disclosure. This method can be executed by the electronic device 130 shown in FIG. 1.

In step S205, the electronic device obtains a three-axis acceleration signal and a three-axis angular velocity of a limb movement in a spatial orthogonal coordinate system through an inertial measurement unit (IMU) installed on a fitness equipment Signal. In one embodiment, the above-mentioned limb movement is an upper limb movement. In another embodiment, the aforementioned fitness equipment is a dumbbell.

In step S210, the electronic device obtains a known motion trajectory of the above-mentioned limb movement, wherein the above-mentioned known motion trajectory is obtained by photographing a light ball installed on the above-mentioned fitness equipment.

Next, in step S215, the electronic device divides the three-axis acceleration signal and the three-axis angular velocity signal into multiple signal segments according to the known motion trajectory. In more detail, before the electronic device divides the three-axis acceleration signal and the three-axis angular velocity signal into multiple signal segments, the three-axis acceleration signal and the three-axis angular velocity signal can be "analyzed the axis of motion" to determine which axis The acceleration amplitude in the direction is the largest. Then, the electronic device can perform band-pass filtering on the three-axis acceleration signal based on the peaks and valleys of the known motion trajectory obtained by the photosphere, so that the electronic device can correct the peaks and valleys of the three-axis acceleration signal, so as to correct the three-axis acceleration signal. The acceleration signal and the aforementioned three-axis angular velocity signal are divided into multiple signal segments. In other words, the crest/valley is the turning point of the centripetal/centrifugal action, so every two troughs can be regarded as an action cycle, and each signal segment is an action cycle. The electronic device can obtain the number of movements and time of the above-mentioned limb movement according to each signal segment. In other embodiments, the electronic device performs band-pass filtering on the three-axis angular velocity, or the three-axis angular velocity and the three-axis acceleration based on the peaks and troughs of the known motion trajectory obtained by the photosphere, so as to filter the above-mentioned three-axis acceleration signal and/ Or the above-mentioned three-axis angular velocity signal is divided into multiple signal segments. Furthermore, in step S220, the electronic device segmentally recognizes one of the above-mentioned limb movements according to the above-mentioned complex signal.

FIGS. 3A to 3C are schematic diagrams showing the division of the three-axis acceleration signal and the three-axis angular velocity signal according to the motion trajectory of the light ball according to an embodiment of the present disclosure. As shown in Fig. 3A, Fig. 310 is the trajectory of the photosphere on the X, Y, and Z axes, Fig. 320 is the three-axis acceleration signal, and Fig. 320 is the three-axis angular velocity signal. Compare the motion trajectory of the light ball on the X, Y, and Z axes with the three-axis acceleration signal and the three-axis angular velocity signal. As shown in Figure 3A, first analyze the three-axis acceleration signal and the three-axis angular velocity signal. It is found that the Y-axis amplitude of the acceleration signal is the largest. From the perspective of the Z axis, although the peaks and valleys of the photosphere are more synchronized with the position of the angular velocity signal. However, because there are too many possibilities for the angular velocity signal to change during movement, if the angular velocity is used as the basis for dividing the signal segmentation, it may cause misjudgment. Therefore, the operation of dividing the signal segmentation still needs to consider the acceleration signal of the Y axis, such as As shown in Figure 3B. Then, the acceleration signal of the Y axis can be band-pass filtered to filter the non-motion information to synchronize with the trajectory of the light ball on the Z axis, as shown in Figure 3C. In Figure 3C, the band-pass filtered Y-axis acceleration signal (shown by the dashed line) can be made smoother to facilitate segmentation of the signal and estimate the eccentric/centripetal action time.

FIGS. 4A to 4C are schematic diagrams showing that the three-axis acceleration signal and the three-axis angular velocity signal are divided into signal segments according to the motion trajectory of the light ball according to an embodiment of the disclosure. Figures 4A to 4C are divided into signal segments corresponding to one motion cycle by means of Figures 3A to 3C. As shown in Fig. 4A, Fig. 410 and Fig. 412 are the signal segments of the three-axis acceleration signal and the three-axis angular velocity signal of the user using dumbbells for biceps training. As shown in Figure 4B, Figures 420 and 422 are signal segments of the three-axis acceleration signal and the three-axis angular velocity signal of the user using dumbbells to perform triceps training. As shown in Fig. 4C, Fig. 430 and Fig. 432 are the signal segments of the three-axis acceleration signal and the three-axis angular velocity signal of the user using dumbbells for dumbbell shoulder fly training.

The following will describe in detail how in step S220, the electronic device recognizes one of the above-mentioned limb movements according to the above-mentioned complex signal. Regarding the part of recognizing the movement of the limbs, this disclosure proposes a method for recognizing the movement of the limbs by calculating the time domain features with an electronic device.

其中 R函式唯一自相關函數，而class=1~3可分別表示不同的動作。舉例來說，Class=1表示二頭彎舉動作、Class=2表示三頭屈伸動作、Class=3表示肩飛鳥動作。第6圖係顯示根據本揭露一實施例所述之電子裝置根據辨識模型識別肢體運動的動作之示意圖。如圖所示，透過辨識模型可清楚識別不同肢體運動的動作。例如，藍點區域係表示Class=1、綠點區域係表示Class=2、紅點區域係表示Class=3。 FIG. 5 shows a flow chart of the method 500 for the electronic device to recognize the movement of the limbs by calculating the time-domain features according to an embodiment of the present disclosure. Before executing the method 500, the electronic device may calculate in advance the time-domain characteristic value of each axis (for example, the average value of the acceleration x-axis) and the time-domain characteristic value between the axes (for example, the phase difference between the x-axis and the y-axis of angular velocity) of each action. , And create and store a recognition model in the memory of the electronic device to recognize the movement of the limbs. In one embodiment, the aforementioned identification model can be expressed by a formula as follows:

Among them, the R function is the only autocorrelation function, and class=1~3 can respectively indicate different actions. For example, Class=1 means two-head curling action, Class=2 means three-head flexion and extension action, and Class=3 means shoulder bird action. FIG. 6 is a schematic diagram of the electronic device according to an embodiment of the disclosure that recognizes the movement of the limbs according to the recognition model. As shown in the figure, through the recognition model, the movements of different limbs can be clearly recognized. For example, the blue dot area indicates Class=1, the green dot area indicates Class=2, and the red dot area indicates Class=3.

In step S505, the electronic device calculates a time domain feature based on the complex signal segment obtained in step S215 in FIG. 2. To explain in more detail, take the signal segment in Figs. 4A to 4C as an example. The electronic device can find out the repetition pattern of the signal by using the autocorrelation (also called sequence correlation) of the signal segments in Figures 4A to 4C. Figures 7A-7C are schematic diagrams showing signal segmentation after autocorrelation according to an embodiment of the present disclosure. As shown in Fig. 7A, Figs. 710 and 712 are the signal segments of the three-axis acceleration signal and the three-axis angular velocity signal of the Biceps curl training exercise after autocorrelation. As shown in Fig. 7B, Figs. 720 and 722 are the signal segments of the three-axis acceleration signal and the three-axis angular velocity signal of the overhead triceps extension training exercise after autocorrelation. As shown in Fig. 7C, Figs. 730 and 732 are the signal segments of the three-axis acceleration signal and the three-axis angular velocity signal of the Shoulder fly training action after autocorrelation. According to the signal segmentation after autocorrelation in Figures 7A to 7C, time-domain features can be obtained, which can help distinguish the overhead triceps extension training actions. In step S510, the electronic device inputs the above-mentioned temporal features into a recognition model to recognize the above-mentioned movement of the above-mentioned limb movement.

FIG. 8 is a flowchart of a method 800 of using fitness equipment to obtain power of a limb according to an embodiment of the present disclosure. This method can be executed by the electronic device 130 shown in FIG. 1. In step S805, the electronic device obtains a three-axis acceleration signal and a three-axis angular velocity of a limb movement in a spatial orthogonal coordinate system through an inertial measurement unit (IMU) installed on a fitness equipment Signal. In one embodiment, the above-mentioned limb movement is an upper limb movement. In another embodiment, the aforementioned fitness equipment is a dumbbell.

In step S810, the electronic device obtains a known motion trajectory of the limb movement, wherein the known motion trajectory is obtained by photographing a light ball installed on the fitness equipment. Then, in step S815, the electronic device divides the three-axis acceleration signal and the three-axis angular velocity signal into multiple signal segments according to the known motion trajectory to obtain a number and a time corresponding to the limb movement. It should be noted that the actions of steps S805 to S815 are the same as the actions of steps S205 to S215 in Figure 2, so the details will not be repeated here.

In step S820, the electronic device performs quaternion calculation and quaternion rotation on the three-axis acceleration signal and the three-axis angular velocity signal to obtain a quaternion conversion result. In step S825, the electronic device removes one of the above-mentioned quaternion conversion results through a band-pass filter. In step S830, the electronic device obtains the power consumption of one of the above-mentioned limb movements according to the quaternion conversion result after the above-mentioned gravity is removed, the above-mentioned number of times, and the above-mentioned time. In more detail, the electronic device can then obtain the three-axis acceleration data and the three-axis angular velocity data of the absolute coordinates through a quaternion rotation matrix. The electronic device can deduct the acceleration caused by gravity. The three-axis acceleration data and the three-axis angular velocity data after the deduction of gravity are the actual force exerted on the fitness equipment by the user. In this way, the mass of the fitness equipment and the three-axis acceleration data can be determined as well. Corresponding to the number of times and time obtained by the signal segment, the power consumed during limb movement is calculated.

Figures 9 to 11 further illustrate how the electronic device obtains the power consumed by the limb movement according to the process in Figure 8. FIG. 9 is a schematic diagram showing a three-axis acceleration signal and a three-axis angular velocity signal in a right-angled spatial coordinate system of a limb movement according to an embodiment of the present disclosure. As shown in Figure 9, there are gravity factors in the six-axis raw data. Although the fitness equipment is relatively unaffected in the static state, the gravity of the fitness equipment will act on the three axes in the state of limb movement (moving, rotating).

FIG. 10A is a schematic diagram showing the quaternion calculation of the signal according to an embodiment of the present disclosure. FIG. 10B is a schematic diagram showing the signal after a quaternion rotation according to an embodiment of the present disclosure. As shown in Figure 10A, after calculating with quaternion, it can be proved that gravity affects the condition of each axis during exercise. As shown in Figure 10B, after quaternion rotation (reverse conversion), the fitness equipment can be converted back to an unrotated angle. The quaternion conversion result shows that gravity only acts on the Z axis.

FIG. 11 is a schematic diagram showing the high-pass filtering of the quaternion conversion result according to an embodiment of the disclosure. As shown in Figure 11, the quaternion conversion result has removed gravity. The electronic device can use a function theorem (Work-Energy Theorem) to obtain one of the power consumption of the limb movement according to the quaternion conversion result, the number of times and the time after the gravity is removed.

As described above, the method, device, and system for recognizing limb movement and calculating limb movement power by using fitness equipment of the present disclosure can avoid the problem of drift of the inertial sensor itself and improve the track recognition error.

For the described embodiments of the present invention, an exemplary operating environment in which the embodiments of the present invention can be implemented is described below. Specifically, referring to FIG. 12, FIG. 12 shows an exemplary operating environment for implementing an embodiment of the present invention, which can be generally regarded as a computing device 1200. The computing device 1200 is only an example of a suitable computing environment, and is not intended to imply any limitation on the use or functional scope of the present invention. The computing device 1200 should not be interpreted as having any dependency or requirement related to any one or combination of the illustrated elements.

The present invention can be executed in computer program code or machine using instructions. The instructions can be computer executable instructions of program modules, and the program modules are executed by computers or other machines, such as personal digital assistants or other portable devices. . Generally speaking, program modules include routines, programs, objects, components, data structures, etc. Program modules refer to program codes that perform specific tasks or implement specific abstract data types. The present invention can be implemented in various system configurations, including portable devices, consumer electronic products, general-purpose computers, more professional computing devices, and the like. The invention can also be implemented in a distributed computing environment to process devices connected by a communication network.

Refer to Figure 12. The computing device 1200 includes a bus 1210, a memory 1212, one or more processors 1214, one or more display elements 1216, an input/output (I/O) port 1218, an input/output (I/O) port 1218, a memory 1212, one or more processors 1214 that are directly or indirectly coupled to the following devices Output (I/O) element 1220 and illustrative power supply 1222. The bus 1210 represents a component that can be one or more buses (for example, an address bus, a data bus, or a combination thereof). Although the blocks in Figure 12 are shown with lines for brevity, in fact, the boundaries of the various components are not specific. For example, the presentation components of the display device can be regarded as I/O components; the processor can have a memory. .

The computing device 1200 generally includes various computer readable media. The computer-readable medium may be any available medium that can be accessed by the computing device 1200, and the medium includes both volatile and non-volatile media, removable and non-removable media. For example, but not limited to, computer-readable media may include computer storage media and communication media. Computer-readable media also includes volatile and non-volatile media implemented in any method or technology used to store information such as computer-readable instructions, data structures, program modules or other data, etc. Removable and non-removable media. Computer storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical disc storage devices, floppy disk, floppy disk, floppy disk storage A device or other magnetic storage device, or any other medium that can be used to store required information and that can be accessed by the computing device 1200. The computer storage medium itself does not include the signal.

Communication media generally include computer-readable instructions, data structures, program modules, or other data in the form of modular data signals such as carrier waves or other transmission mechanisms, and include any information transmission media. The term "modular data signal" refers to a signal that has one or more feature sets or is modified in one of the ways that information is encoded in the signal. For example, but not limited to, communication media include wired media and wireless media such as wired networks or direct wired connections, such as audio, radio frequency, infrared, and other wireless media. The combination of the above media is included in the range of computer readable media.

The memory 1212 includes computer storage media in the form of volatile and non-volatile memory. The memory can be removable, non-movable, or a combination of the two. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, and the like. The computing device 1200 includes one or more processors that read data from various entities such as the memory 1212 or the I/O element 1220. The display element 1216 displays data instructions to the user or other devices. Exemplary display elements include display devices, speakers, printing elements, vibration elements, and the like.

The I/O port 1218 allows the computing device 1200 to be logically connected to other devices including I/O components 1220, some of which are built-in devices. Exemplary components include microphones, joysticks, gaming stations, satellite dish receivers, scanners, printers, wireless devices, etc. The I/O element 1220 can provide a natural user interface for processing user-generated gestures, sounds, or other physiological inputs. In some instances, these inputs can be sent to an appropriate network component for further processing. The computing device 1200 may be equipped with a depth camera, such as a stereo camera system, an infrared camera system, an RGB camera system, and a combination of these systems, to detect and identify objects. In addition, the computing device 1200 may be equipped with sensors (for example, radar, LiDAR) to periodically sense the surrounding environment within a sensing range, and generate sensor information indicating that it is associated with the surrounding environment. Furthermore, the computing device 1200 may be equipped with an accelerometer or a gyroscope to detect motion. The output of the accelerometer or gyroscope can be provided to the computing device 1200 for display.

In addition, the processor 1214 in the computing device 1200 can also execute programs and instructions in the memory 1212 to present the actions and steps described in the above embodiments, or other descriptions in the specification.

Any specific sequence or hierarchical steps of the procedure disclosed herein is purely an example. Based on design preferences, it must be understood that any specific sequence or hierarchical steps in the procedure can be rearranged within the scope disclosed in this document. The accompanying method claims present elements of the various steps in an exemplary order, and therefore should not be limited by the specific order or hierarchy shown here.

The use of ordinal numbers such as "first", "second", and "third" used to modify the elements in the scope of the patent application does not imply any priority, priority, order between elements, or execution of methods The order of the steps is only used as an identification to distinguish different elements with the same name (with different ordinal numbers).

Although this disclosure has been disclosed as above with an implementation example, it is not intended to limit the case. Anyone familiar with this technique can make some changes and modifications without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this case should be Subject to the definition of the scope of patent application attached hereto.

100: System 110: fitness equipment 112: Light Ball 120: Network 130: electronic device 200: method S205, S210, S215, S220: steps 310, 320, 330: Figure 410, 412, 420, 422, 430, 432: Figure 500: method S505, S510: steps 710, 712, 720, 722, 730, 732: Figure 800: method S805, S810, S815, S820, S825, S830: steps 1200: computing device 1210: bus 1212: memory 1214: processor 1216: display element 1218: I/O port 1220: I/O components 1222: power supply

Fig. 1 shows an exemplary schematic diagram of a system for recognizing the movement of limbs by using fitness equipment according to an embodiment of the present invention. Fig. 2 shows a flowchart of a method for recognizing body movements by using fitness equipment according to an embodiment of the present disclosure. FIGS. 3A to 3C are schematic diagrams showing the division of the three-axis acceleration signal and the three-axis angular velocity signal according to the motion trajectory of the light ball according to an embodiment of the present disclosure. FIGS. 4A to 4C are schematic diagrams showing that the three-axis acceleration signal and the three-axis angular velocity signal are divided into signal segments according to the motion trajectory of the light ball according to an embodiment of the disclosure. FIG. 5 shows a flowchart of a method for the electronic device to recognize body movement by calculating time domain features according to an embodiment of the present disclosure. FIG. 6 is a schematic diagram of the electronic device according to an embodiment of the disclosure that recognizes the movement of the limbs according to the recognition model. Figures 7A-7C are schematic diagrams showing signal segmentation after autocorrelation according to an embodiment of the present disclosure. FIG. 8 is a flow chart showing a method of using fitness equipment to power the limbs according to an embodiment of the present disclosure. FIG. 9 is a schematic diagram showing a three-axis acceleration signal and a three-axis angular velocity signal in a right-angled spatial coordinate system of a limb movement according to an embodiment of the present disclosure. FIG. 10A is a schematic diagram showing the quaternion calculation of the signal according to an embodiment of the present disclosure. FIG. 10B is a schematic diagram showing the signal after a quaternion rotation according to an embodiment of the present disclosure. FIG. 11 is a schematic diagram showing the high-pass filtering of the quaternion conversion result according to an embodiment of the disclosure. Figure 12 shows an exemplary operating environment for implementing embodiments of the present invention.

200: method

S205, S210, S215, S220: steps

Claims (19)

A method of using fitness equipment to recognize the movement of limbs includes: obtaining a three-axis of the movement of a limb in a rectangular coordinate system by an Inertial Measurement Unit (IMU) installed on a fitness equipment Acceleration signal and a three-axis angular velocity signal; obtain a known motion trajectory of the above-mentioned limb movement; divide the above-mentioned three-axis acceleration signal and the above-mentioned three-axis angular velocity signal into multiple signal segments according to the above known motion trajectory; Calculate a time domain feature; and input the time domain feature to an identification model to identify the known motion trajectory of the limb movement, wherein the identification model includes an autocorrelation function. As described in the first item of the scope of patent application, the method of using fitness equipment to recognize the movement of limbs, wherein the known motion track is obtained by photographing a light ball installed on the fitness equipment. As described in the first item of the scope of patent application, the method of using fitness equipment to recognize the movement of the limbs, wherein the above-mentioned limb movement is an upper limb movement. As described in the first item of the scope of patent application, the method of using fitness equipment to recognize the movement of limbs, wherein the fitness equipment is a dumbbell. A device for recognizing the movement of limbs using fitness equipment includes: one or more processors; and one or more computer storage media storing computer readable instructions, wherein the processor uses the computer storage media to execute: Obtain a three-axis acceleration signal and a three-axis angular velocity signal of a limb movement in a spatial orthogonal coordinate system by an Inertial Measurement Unit (IMU) installed on a fitness equipment; obtain the above-mentioned limb movement A known motion trajectory; according to the known motion trajectory, the three-axis acceleration signal and the three-axis angular velocity signal are divided into complex signal segments; a time domain feature is calculated based on the complex signal segment; and the time domain feature is input to a An identification model is used to identify the known movement track of the above-mentioned limb movement, wherein the identification model includes an autocorrelation function. As described in item 5 of the scope of patent application, the device for recognizing the movement of the limbs using fitness equipment, wherein the known motion track is obtained by photographing a light ball installed on the fitness equipment. As described in item 5 of the scope of patent application, a device for recognizing the movement of limbs by using fitness equipment, wherein the above-mentioned limb movement is an upper limb movement. As described in item 5 of the scope of patent application, a device for recognizing the movement of limbs by using fitness equipment, wherein the fitness equipment is a dumbbell. A system for recognizing the movement of a limb using fitness equipment, including: a fitness equipment, including: an inertial sensor, which obtains a three-axis acceleration signal and a three-axis angular velocity signal of a limb movement in a spatial orthogonal coordinate system; A camera that shoots the fitness equipment to obtain a known movement track of the limb movement; and an electronic device connected to the fitness equipment to receive the three-axis acceleration signal and the three-axis angular velocity signal transmitted by the fitness equipment, and Transmitted by the above camera The said known motion trajectory; the said three-axis acceleration signal and the said three-axis angular velocity signal are divided into complex signal segments according to the said known motion trajectory; a time domain feature is calculated according to the said complex signal segment; and the said time domain feature is input to An identification model to identify the known motion trajectory of the above-mentioned limb movement, wherein the identification model includes an autocorrelation function. As described in item 9 of the scope of patent application, the system for recognizing the movement of limbs using fitness equipment, wherein the system further includes: a light ball, wherein the known motion track is obtained by photographing the light mounted on the fitness equipment The ball is made. As described in item 9 of the scope of patent application, the system for recognizing the movement of limbs by using fitness equipment, wherein the step of recognizing the movement of the movement of the limbs by the electronic device according to the complex signal further includes: calculating according to the complex signal. A time domain feature; and input the time domain feature to a recognition model to recognize the movement of the limb movement. As described in item 9 of the scope of patent application, the system for recognizing the movement of limbs by using fitness equipment, wherein the above-mentioned limb movement is an upper limb movement. As described in item 9 of the scope of patent application, the system for recognizing the movement of limbs by using fitness equipment, wherein the fitness equipment is a dumbbell. A method of using fitness equipment to calculate the power of the limbs includes: obtaining a three-axis of a limb movement in a spatial rectangular coordinate system by an Inertial Measurement Unit (IMU) installed on a fitness equipment Acceleration signal and a three-axis angular velocity signal; Obtain a known motion trajectory of the above-mentioned limb movement; divide the above-mentioned three-axis acceleration signal and the above-mentioned three-axis angular velocity signal into plural signal segments according to the above-mentioned known motion trajectory, so as to obtain a number and a time corresponding to the above-mentioned limb movement; The three-axis acceleration signal and the three-axis angular velocity signal perform quaternion calculation and quaternion rotation to obtain a quaternion conversion result; remove one of the gravity of the quaternion conversion result through a band-pass filter; and according to the shift The quaternion conversion result after dividing the above-mentioned gravity, the above-mentioned number of times, and the above-mentioned time obtain one of the power consumption of the above-mentioned limb movements. The method of using fitness equipment to calculate the exercise power of limbs as described in item 14 of the scope of patent application, wherein the known motion trajectory is obtained by photographing a light ball installed on the fitness equipment. A device for using fitness equipment to calculate the exercise power of limbs includes: one or more processors; and one or more computer storage media for storing computer readable instructions, wherein the processor uses the computer storage media to execute: An Inertial Measurement Unit (IMU) installed on a fitness equipment obtains a three-axis acceleration signal and a three-axis angular velocity signal of a limb movement in a spatial orthogonal coordinate system; obtains one of the above-mentioned limb movements Known motion trajectory; according to the known motion trajectory, divide the three-axis acceleration signal and the three-axis angular velocity signal into multiple signal segments to obtain a number and a time corresponding to the motion of the limb; Perform quaternion calculation and quaternion rotation on the three-axis acceleration signal and the three-axis angular velocity signal to obtain a quaternion conversion result; remove one of the gravity of the quaternion conversion result through a band-pass filter; and according to The quaternion conversion result after removing the above-mentioned gravity, the above-mentioned number of times, and the above-mentioned time obtain one of the power consumption of the above-mentioned limb movements. As described in item 16 of the scope of patent application, the device for calculating the exercise power of the limbs using fitness equipment, wherein the known motion trajectory is obtained by photographing a light ball installed on the fitness equipment. A system that uses fitness equipment to calculate the power of limb movement, including: a fitness equipment, including: an inertial sensor, which obtains a three-axis acceleration signal and a three-axis angular velocity signal of a limb movement in a spatial orthogonal coordinate system; A camera that shoots the fitness equipment to obtain a known movement track of the limb movement; and an electronic device connected to the fitness equipment to receive the three-axis acceleration signal and the three-axis angular velocity signal transmitted by the fitness equipment, and The known motion trajectory transmitted by the camera; according to the known motion trajectory, the three-axis acceleration signal and the three-axis angular velocity signal are divided into multiple signal segments to obtain a number and a time corresponding to the motion of the limb; The three-axis acceleration signal and the three-axis angular velocity signal perform quaternion calculation and quaternion rotation to obtain a quaternion conversion result; remove one of the gravity of the quaternion conversion result through a band-pass filter; and according to the shift The quaternion conversion result after dividing the above-mentioned gravity, the above-mentioned number of times, and the above-mentioned time obtain one of the power consumption of the above-mentioned limb movements. As described in item 18 of the scope of patent application, the system for calculating the exercise power of limbs using fitness equipment, wherein the system further includes: a light ball, wherein the known motion trajectory is obtained by photographing the light installed on the fitness equipment. The ball is made.

Images