RU2813471C1 - Methods and systems for identifying user action - Google Patents

Abstract

FIELD: computer engineering.

SUBSTANCE: group of inventions can be used to identify user actions. Method comprises steps of obtaining user action data collected from a plurality of measurement points on the user, wherein data correspond to unknown user action; obtaining a plurality of sets of reference data on action, wherein each set of reference action data corresponds to at least one reference action; performing a two-level screening operation on a plurality of reference action data sets based on the user action data, wherein the two-level screening operation comprises a combination of a difference-based screening operation and a probability-based screening operation; determining the reference action corresponding to the target reference action data as the target action; making a decision that the user action contains a target action, based on the result of the two-level screening operation; and sending the user information associated with the target action.

EFFECT: high accuracy of recognizing user actions.

10 cl, 9 dwg

Description

Field of technology to which the invention relates

The present disclosure relates to the field of wearable technology and, in particular, to methods and systems for identifying user activity.

State of the art

Given people's attention to evidence-based exercise and physical health, the development of motion control devices for fitness has gained momentum. At present, the main development direction of motion control devices for monitoring user action is to analyze user action data based on reference action data when the type of user action is known, so as to control whether the user action is standardized. Therefore, in a practical application scenario, the user must inform the motion control device about the fitness action type in advance before exercise, so that the motion control device can select action reference data for the action type to accurately monitor the user's action. Before each user's fitness action, the motion control device must be informed of the type of action, which leads to inconvenience for the user. In addition, existing motion control devices monitor the user's action not in real time, which results in the user being able to receive information related to the fitness action after completing the fitness action, which also leads to inconvenience for the user.

Therefore, it is necessary to provide a method and system for identifying a user's fitness activity in real time without requiring the user to input the activity type in advance.

Disclosure of the invention

The present disclosure provides a method for identifying a user action. In accordance with one approach of the present disclosure, the method may comprise the following step. User action data collected from a plurality of measurement locations on the user may be obtained, and the user action data may correspond to an unknown user action. The user action comprises a target action, wherein the acquisition of the user action data may be identified based on at least one set of target action reference data, and the at least one set of target action reference data may correspond to the target action. Information associated with the target action can be sent to the user.

In some embodiments, identifying that a user action contains a target action may comprise the following operations. A plurality of possible action reference data sets may be obtained, with each action reference data set corresponding to at least one action reference data. A two-level screening operation may be performed on a plurality of candidate action reference data sets based on the user action data, wherein the two-level screening procedure may comprise a combination of a degree-of-dissimilarity screening operation and a probability-based screening operation. In this way, a user action can be defined that contains a target action based on the result of the two-level elimination operation.

In some embodiments, identifying that a user action contains a target action may comprise the following operations. A plurality of action reference data sets may be obtained, with each action reference data set corresponding to at least one action reference data. Each set of action reference data from the plurality of action reference data sets may in turn be selected as candidate action reference data. At least one degree of difference may be determined by segment-by-segment comparison of at least one action identification subdata segment of the candidate action reference data with a corresponding user action subdata segment. The composite degree of difference can be determined by weighing and summing at least one degree of difference.

In some embodiments, each set of action reference data may contain M pieces of action reference subdata, where each piece of action reference subdata may contain at least one action identification subdata segment, where M is an integer greater than 1. Action identification subdata M The action reference subdata fragments may form integral action identification data, and each action identification subdata segment may correspond to at least a portion of the action reference at at least one measurement location of the plurality of measurement locations.

In some embodiments, determining at least one degree of difference by segment-by-segment comparison of at least one action identification subdata segment of the candidate action reference data with a corresponding user action subdata segment may comprise the following operations. A sliding window of a given length may be selected on each piece of activity identification subdata, where the sliding window may comprise a data segment of user action data collected in a given time interval. For a sliding window, the degree of difference between the data segment and the corresponding action identification subdata can be determined at the current moment.

In some embodiments, identifying that a user action contains a target action may further comprise the following operations. It may be determined that the complex difference value is greater than the first predetermined value. The sliding window can be moved to the next data segment with a given step size and the comparison can be repeated.

In some embodiments, the duration of data collection corresponding to a data segment in a sliding window may be negatively correlated with the speed of the user's action.

In some embodiments, a given step size may satisfy one or more of the following conditions. A given step size can be positively correlated with the value of the complex degree of difference at the previous moment. A given step size can be positively correlated with the trend of the complex dissimilarity value.

In some embodiments, a data segment may contain multiple data points about a user action. Determining at least one degree of difference by comparing at least one segment of possible action identification subdata from among the possible action reference data with a corresponding segment of user action subdata may comprise the following operations. The target data interval for comparison may be selected from the action identification subdata, wherein the target data interval for comparison comprises a plurality of identification data points. A data segment corresponding to a plurality of scales may be adjusted to obtain a plurality of adjusted data segments. The degree of difference between the action identification subdata and each adjusted data segment of the plurality of adjusted data segments can be determined accordingly. A minimum degree of difference between the action identification subdata and the data segment may be determined.

In some embodiments, determining at least one degree of difference by segment-by-segment comparison of at least one action identification subdata segment from the possible action reference data with a corresponding user action subdata segment may comprise the following operations. A distance matrix [D _ij ] may be defined, where D _ij denotes the distance between the i-th data point of the target comparison data interval and the j-th data point of the data segment. The shortest distance of the distance matrix can be determined, where the shortest distance can satisfy the following operations. The starting point of the shortest distance path can be in the first row of the matrix [D _ij ], two adjacent points at the shortest distance can be adjacent in the distance matrix, the next point at the shortest distance can be to the right, below or right below the previous point, the end point of the shortest distance path may be in the last row of [D _ij ], the shortest distance path may have the smallest regularization cost, where the regularization cost is determined by the distances between points on the corresponding shortest distance path of the distance matrix, and the degree of difference may be related to the regularization cost.

In some embodiments, if the first data point of a data segment can be determined to be the data point at which a user action begins, the starting point of the shortest distance path may be the distance D ₁₁ between the first point of the data segment and the first point of the target comparison data interval.

In some embodiments, if the last data point of a data segment can be defined as the data point at which the user's action ends, then the end point of the shortest distance path may be the distance D _mn between the last point of the data segment and the last point of the target comparison data interval.

In some embodiments, identifying that a user action contains a target action may further comprise the following operations. The N pieces of possible second-level action reference data may be selected from a plurality of sets of action reference data. The complex degree of difference value of the second-level candidate action reference data may be less than the first predetermined value, and N may be an integer greater than 1. N distances between the user action data and N pieces of the second-level candidate action reference data may be calculated accordingly. N probability values can be respectively calculated based on N distances. Candidate second-level action references whose probability value is greater than the second predetermined value may be selected as target user action references. A reference action corresponding to the target action reference data may be defined as a target action.

Another approach of the present disclosure discloses a system for identifying user action. The user action identification system may contain the following operations. The at least one storage medium may store at least one set of commands for obtaining data about a user's action while the user is moving. The at least one processor communicates with the at least one medium and, when the system is operating, the at least one processor can read the at least one set of instructions and perform the above method and the target action identification method disclosed in the embodiments.

Brief description of drawings

The present disclosure may be further described by way of exemplary embodiments, which may be described in detail through the use of the accompanying drawings. These embodiments are not limiting and in these embodiments the same reference numerals refer to the same structures, where:

fig. 1 illustrates an application scenario for a traffic control system consistent with certain embodiments of the present disclosure;

fig. 2 illustrates exemplary hardware and/or software components of a wearable device corresponding to some embodiments of the present disclosure;

fig. 3 illustrates exemplary hardware and/or software components of a computing device corresponding to some embodiments of the present disclosure;

fig. 4 is an exemplary block diagram of a wearable device consistent with some embodiments of the present disclosure;

fig. 5 is a flowchart of an exemplary process for determining a target action in accordance with some embodiments of the present disclosure;

fig. 6 is an exemplary coordinate system during user movement, corresponding to some embodiments of the present application;

fig. 7A illustrates an example action identification data segment in the action reference data and a segment curve of the user action subdata collected by a sliding window in the time axis user action data in accordance with some embodiments of the present disclosure;

fig. 7B is a distance matrix and shortest distance path from the upper left corner to the lower right corner of the distance matrix, corresponding to some embodiments of the present disclosure;

fig. 7C is a diagram for determining a complex degree of difference by a sliding window when the user action data contains a plurality of user action subdata, corresponding to some embodiments of the present disclosure.

Carrying out the invention

To more clearly illustrate the technical solutions of the embodiments of the present disclosure, drawings illustrating the embodiments are briefly presented below. The drawings in the following illustrations are merely some examples or embodiments of the present disclosure. Those skilled in the art can apply the present disclosure to other similar scenarios in accordance with the drawings without much creative effort. Unless the context clearly indicates or the context indicates otherwise, the same reference numeral in the drawings refers to the same structure or operation.

It should be understood that the terms "system", "device", "unit" and/or "module" as used herein are a way of distinguishing between different components, elements, parts, sections or assemblies of different levels. However, if the same purpose can be achieved by other expressions, such words may be replaced by other expressions.

As used in the disclosure and appended claims, the singular forms include the plural forms unless the content clearly dictates otherwise. In general, the terms “comprising” and “including” only suggest steps and elements that are explicitly defined, and these steps and elements do not constitute an exclusive list. The methods or apparatus may also contain other steps or elements.

Flowcharts are used in the present disclosure to explain operations performed by a system in accordance with some embodiments of the present disclosure. It should be understood that the preceding or subsequent operations may not necessarily be performed in the exact order specified. Instead, each step can be performed in reverse order or simultaneously. At the same time, other operations may also be added to the procedures, or a step or several steps may be removed from the procedures.

The present disclosure provides a system for targeting an action. A set of commands stored on a storage medium in the target action detection system may be executed to obtain data about a user's action while the user is moving. The target action detection system may be applicable to a wearable device (eg, wearable, wristband, and helmet), medical testing device (eg, electromyographic tester (EMG)), fitness device, etc. After the user wears the device, sensors on the device can be attached to a plurality of measurement locations on the user's body and thus data about the user's activity can be collected by the device's sensors. After the sensors have collected data about the user's action, the processor in the target action detection system can communicate with the data storage medium to access or read commands stored on the medium, etc. When the target action determination system is running, the processor may access action reference data stored on a medium with known action content. Based on the action reference data from this known action content, the system can perform target action identification on the action data of a user whose action content is unknown. After determining the target action, the system can send the user content related to the target action.

In some embodiments of the present disclosure, the system may perform target action identification on the user action data immediately or within a predetermined time, where the predetermined time may be a short time such as 0.1 second or 0.5 second. In this way, the system can realize real-time identification of user action data, and the user can receive corresponding action materials immediately after performing the action.

In some embodiments of the present disclosure, user activity data may also be obtained by other means without being collected by sensors on devices, such as a wearable device (e.g., clothing, bracelet, helmet), medical detection device (e.g., EMG tester), fitness device, etc. For example, images of a user in video mode can be analyzed by an artificial intelligence algorithm to obtain action data at multiple measurement locations on the user's body. In short, as long as user action data can be obtained in real time, the method and system according to the present disclosure can be configured to determine the target action.

The storage medium may include a propagating data signal having a driver program contained therein, for example, in a baseband or as part of a carrier wave. The propagating signal can take various forms, including electromagnetic, optical, etc. or an appropriate combination thereof. The computer storage medium can be any computer-readable medium, other than a computer-readable medium that can communicate, distribute, or transmit a program for use by communicating with an instruction execution system, hardware, or device. The control program recorded on a computer medium may be transmitted via any suitable medium, including radio, cable, fiber optic cable, radio frequency medium, or the like. or a combination of them. Specifically, the storage medium can be random-access memory (RAM), read-only memory (ROM), etc. Examples of ROMs may include mask ROM (MROM), programmable ROM (PROM), erasable programmable ROM (PEROM), electrically erasable programmable ROM (EEPROM), compact ROM disk (compact disc ROM, CD-ROM), ROM on a digital versatile disk, etc. Examples of RAM may include dynamic RAM (DRAM), double rate synchronous dynamic RAM (SDRAM DDR), static RAM (SRAM), thyristor RAM (T-RAM), RAM with zero capacity (zero capacitance, Z-RAM), etc.

As an example, the processor may be a multi-core processor, a single-core processor, a distributed processor, a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction processor, ASIP), graphics processor (GPU), physical processor (PPU), digital signal processor (DSP), field programmable gate array (FPGA), programmable logic circuit (programmable logic circuit (PLD), controller, microcontroller module, reduced instruction set computer (RISC), microprocessor device, etc. or any combination thereof.

In fig. 1 is a schematic representation of an application scenario for a traffic control system consistent with certain embodiments of the present disclosure. As shown in FIG. 1, the target action detection system 100 (or system 100) may include a processing device 110, a network 120, a wearable device 130, and a mobile terminal device 149. System 100 may receive user action data (e.g., EMG signal, posture signal, stress data, and physiological monitoring data such as ECG signal, respiratory rate signal, etc.) provided to characterize the user's action and determine whether which target action the user's training action belongs to according to the user's action data.

For example, the system 100 may identify a user action performed by the user while exercising. When a user wears the wearable device 130 to perform fitness exercises, the wearable device 130 may receive data about the user's activity. The processing device 110 or the mobile terminal device 140 may receive and analyze user action data to identify the user's fitness action, such as whether the user's fitness action is bench press, curl or squat, etc., so as to send the user content associated with a target action, in which a certain identified action of the user (for example, bench press, curl, squat, etc.) is the target action. Specifically, the content associated with a target action may include the name of the target action as prompted by voice or video, the type of action, the number of actions, the duration of the action, information about the physiological parameters of the user when the user performs the action, etc. Additionally, the system 100 can generate a feedback signal to the user's fitness action based on the result of analyzing the user's fitness action data, such as whether the user's fitness action is standard, etc., to guide the user's fitness.

As another example, system 100 may identify a user action performed by the user while exercising. For example, when the user wears the wearable device 130 to perform running, the system 100 may receive the user's running action data and determine that the user's current movement is running based on the action reference data. When the user runs for too long or when the running action is incorrect, the fitness device can feedback the motion state to the user to prompt the user to correct the running action or running timing.

In some embodiments, the processing device 110 may be configured to process information and/or data associated with the user's movement. For example, the processing device 110 may receive a user action signal (for example, an EMG signal, a posture signal, an ECG signal, a respiratory rate signal, etc.) and further extract feature information (for example, feature information corresponding to the EMG signal or feature information corresponding to the pose signal in the action signal) corresponding to the action signal. In some embodiments, processing device 110 may perform special signal processing on the EMG signal or body signal collected by wearable device 130, such as signal segmentation, signal pre-processing (e.g., signal correction process, filtering process, etc.), etc. .P. In some embodiments, processing device 110 may also determine whether the user action is correct based on the user action signal. For example, the processing device 110 may determine whether the user's action is correct based on feature information (eg, amplitude information, frequency information, etc.) corresponding to the EMG signal. As another example, the processing device 110 can determine whether the user's action is correct based on attribute information (eg, angular velocity, angular velocity direction, angular velocity acceleration, angle, displacement information, voltage, etc.) corresponding body signal. As a further other example, the processing device 110 can determine whether the user's action is correct based on the feature information corresponding to the EMG signal and the feature information corresponding to the body signal. In some embodiments, the processing device 110 may also determine whether the physiological parameter information of the user while the user is moving meets a medical standard. In some embodiments, processing device 110 may also issue appropriate commands to feedback the user's movement situation. For example, when the user is running, the system 100 monitors that the user's running time is too long. In this case, the processing device 110 may instruct the mobile terminal device 140 to prompt the user to adjust the running time. It should be noted that the feature information corresponding to the body signal is not limited to the above-mentioned angular speed, angular speed direction, angular speed acceleration, angle, displacement information, voltage, etc., but may also be other feature information. Any parameter information that may reflect the relative movement of the user's body may be feature information corresponding to a body motion signal. For example, when the posture sensor is a strain gauge, the bending angle and bending direction at the user's joint can be obtained by measuring the amount of resistance in the strain gauge, which varies proportionally to the stretch length.

In some embodiments, processing device 110 may be local or remote. For example, processing device 110 may, through network 120, access information and/or data stored in wearable device 130 and/or mobile terminal device 140. In some embodiments, processing device 110 may connect directly to wearable device 130 and/or mobile terminal device 140 to access information and/or data stored therein. For example, the processing device 110 may be located in the wearable device 130 and communicate with the mobile terminal device 140 via the network 120. As another example, the processing device 110 may be located in the mobile terminal device 140 and communicate with the wearable device 130 via the network. . In some embodiments, processing device 110 may be implemented on a cloud platform.

In some embodiments, processing device 110 may process data and/or information related to activity monitoring to perform one or more of the functions described herein. In some embodiments, the processing device 110 may receive an action signal while the user is moving received by the wearable device 130. In some embodiments, the processing device 110 may send a control command to the wearable device 130 or to the mobile terminal device 140. The control command may control states of switches of the wearable device 130 and sensors of the wearable device 130, and may also control the mobile terminal device 140 to send information in a timely manner. In some embodiments, processing device 110 may include one or more auxiliary processing devices (eg, a single-core processor device or a multi-core processor device).

The network 120 may facilitate the exchange of data and/or information from the traffic control system 100 . In some embodiments, one or more components of the motion control system 100 may send data and/or information to other components of the motion control system 100 through the network 120. For example, an action signal received by the wearable device 130 may be transmitted to the processing device 110 through the network 120. As another For example, the confirmation result of an action signal determined by processing device 110 may be transmitted to mobile terminal device 140 via network 120. In some embodiments, network 120 may be any type of wired or wireless network.

Wearable device 130 may refer to an item of clothing or device with a wearable function. In some embodiments, wearable device 130 may include, but is not limited to, outerwear device 130-1, trouser device 130-2, bracelet device 130-3, shoe device 130-4, and the like. In some embodiments, wearable device 130 may include M sensors, where M is an integer greater than one. The sensors can receive various action signals (eg, EMG signal, position signal, temperature information, heart rate, electrocardiogram signal, etc.) generated while the user is moving. In some embodiments, the sensors may comprise, but are not limited to, one or more of an EMG sensor, a position sensor, a temperature sensor, a humidity sensor, an electrocardiogram sensor, a blood oxygen saturation sensor, a Hall sensor, a skin electricity sensor, a rotation sensor, etc. P. For example, the outerwear device 130-1 may include an EMG sensor located at a muscle location (e.g., biceps brachii, triceps brachii, latissimus dorsi, trapezius, etc.) of the human body, and the EMG sensor may correspond to the skin user and receive the EMG signal while the user is moving. As another example, the clothing device 130-1 may include an electrocardiogram sensor located near the left pectoral muscle of the human body, and the electrocardiogram sensor may receive an electrocardiographic signal from the user. As yet another example, the trouser device 130-2 may include a posture sensor located at a muscle location (e.g., gluteus maximus, quadriceps femoris, vastus medialis, gastrocnemius, etc.) of the human body and a posture sensor can receive a signal about the user's posture. In some embodiments, the wearable device 130 may also provide feedback on the user's actions. For example, when the action of a certain body region during the user's movement is not in accordance with the standard, the EMG sensor corresponding to that region can generate a stimulation signal (eg, electric stimulation or electric shock signal) to remind the user.

It should be noted that the wearable device 130 is not limited to the outerwear device 130-1, trouser device 130-2, bracelet device 130-3 or shoe device 130-4 shown in FIG. 1. The wearable device 130 may also include other devices used for motion control, such as a helmet device, a knee pad device, etc., which are not limited to. Any device that can use the motion control method presented in this disclosure is within the scope of protection of this disclosure.

In some embodiments, mobile terminal device 140 may obtain information or data from system 100. In some embodiments, mobile terminal device 140 may receive user action data processed by processing device 110 and return back an action record based on the processed action data. Exemplary feedback modes may include, but are not limited to, speech prompt, video prompt, video presentation, text prompt, and the like. In some embodiments, the user, through the mobile terminal device 140, may receive a recording of an activity made while he or she is moving. For example, mobile terminal device 140 may be connected to wearable device 130 via network 120 (eg, wired connection, wireless connection). The user may, through the mobile terminal device 140, receive an action record while the user is moving, and the action record may be transmitted to the processing device 110 through the mobile terminal device 140. In some embodiments, the mobile terminal device 140 may include a mobile device 140-1, a tablet computer 140-2, a laptop computer 140-3, and the like. or any combination thereof. In some embodiments, the mobile device 140-1 may include a cellular phone, a home smart device, a mobile smart device, a virtual reality device, an augmented reality device, etc. or any combination thereof. In some embodiments, a smart home device may include a smart device control device, a smart control device, a smart TV, a smart camera, etc. or any combination thereof. In some embodiments, the mobile smart device may include a smartphone, a personal digital assistant (PDA), a gaming device, a navigation device, a POS device, etc. or any combination thereof. In some embodiments, the virtual reality device and/or augmented reality device may comprise a virtual reality headset, virtual reality glasses, virtual reality glasses, an augmented reality headset, augmented reality glasses, augmented reality glasses, etc. or any combination thereof.

In some embodiments, the motion control system 100 may also include an action data reporting system 160. The action data presentation system 160 may be configured to process and display information and/or data associated with a user action. For example, what kind of movement the user is making may be displayed, or information and/or data may be combined with a virtual symbol and intuitively displayed on the user interface of the mobile terminal device 140 to make viewing easier for the user. For example, the action data system 160 may receive data about a user action. For example, the user action data may include an action signal such as an EMG signal, a posture signal, an electrocardiogram signal, a respiratory rate signal, etc. As another example, the user action data may contain feature information (eg, feature information corresponding to the EMG signal, feature information corresponding to the body signal in the action signal) obtained by the processing device 110 performing feature processing on the action signal. As yet another example, the user action data may include a signal obtained after the processing device 110 performs specific signal processing, such as signal segmentation, signal pre-processing (e.g., signal correction process, filtering process, etc.) etc. The action data representation system 160 may compare the user action data with the action reference data, combine the comparison result with the virtual symbol to generate an animation of the virtual symbol, and send the generated animation to the mobile terminal device 140 for display. Reference data about the user action may be described in detail in subsequent descriptions. For example, when the user performs a biceps curl, the action data system 160 may receive data about the user's action while performing a biceps curl, such as a biceps brachii EMG signal, a trapezius muscle EMG signal, a forearm swing posture, etc. The action data system 160 may compare the user's action data with a plurality of sets of action reference data stored in the motion control system 100 to determine that the user is performing a biceps brachii action. Additionally, the action data system 160 may display a virtual symbol that demonstrates a biceps curl, and the user can clearly and intuitively view the user's action data or the difference between the action data and reference action data (e.g., the difference in the position and size of muscle forces , difference in action posture, etc.) by animating the virtual symbol to adjust the action during movement.

In some embodiments, the action data reporting system 160 may be integrated into the processing device 110. In some embodiments, the action data reporting system 160 may also be integrated into the mobile terminal device 140. In some embodiments, the action data reporting system 160 may also exist independent of the processing device 110 and the mobile terminal device 140. The action data reporting system 160 may communicate with the processing device 110, the wearable device 130, and the mobile terminal device 140 to transmit and exchange information and/or data. In some embodiments, the action reporting system 160 may access information and/or data stored in the processing device 110, the wearable device 130, and/or the mobile terminal device 140 via the network 120. In some embodiments, the wearable device 130 may connect directly to the processing device 110 and/or to the mobile terminal device 140 to access information and/or data stored therein. For example, the action reporting system 160 may be located in the processing device 110 and communicates with the wearable device 130 and the mobile terminal device 140 via the network 120. As another example, the action reporting system 160 may be located in the mobile terminal device 140 and communicate with the processing device 110 and the wearable device 130 via the network. In some embodiments, the action reporting system 160 may be implemented on a cloud platform and communicate with the processing device 110, the wearable device 130, and the mobile terminal device 140 via a network.

For convenience of presentation, in the following descriptions, the action data reporting system 160 located in the mobile terminal device 140 may be taken as an example for description.

In some embodiments, the action data presentation system 160 may process data and/or information associated with the action data presentation to perform one or more of the functions described herein. In some embodiments, the action reporting system 160 may receive data about a user's action while the user is moving, such as an action signal while the user is moving received by the wearable device 130, or data received after an action signal received while the user is moving. user of the wearable device 130 will be processed by the processing device 110. In some embodiments, the action reporting system 160 may send a control command to the mobile terminal device 140 to control the display of the user interface of the mobile terminal device 140.

In some embodiments, system 100 may also include a database. The database may store data (eg, an initial threshold condition, etc.) and/or commands (eg, feedback commands). In some embodiments, the database may store data received from the wearable device 130 and/or from the mobile terminal device 140. In some embodiments, the database may store information and/or instructions for execution or use by processing device 110 to perform exemplary methods described in the present disclosure. In some embodiments, the database may be coupled to the network 120 to communicate with one or more components of the system 100 (eg, processing device 110, wearable device 130, mobile terminal device 140, etc.). One or more components of the system 100 may access data or commands stored in the database through the network 120. In some embodiments, the database may connect or communicate directly with one or more components in the system 100. In some embodiments, the database may be part of the processing device 110.

In fig. 2 illustrates exemplary hardware and/or software components of a wearable device in accordance with some embodiments of the present disclosure. As shown in FIG. 2, the wearable device 130 may include a receiving module 210, a processing module 220 (also referred to as a processor), a control module 230 (also referred to as a main controller, MCU, controller), a communication module 240, a power supply module 250, and an input/output module 260.

The receiving module 210 may be configured to receive an action signal while the user is moving. In some embodiments, acquisition module 210 may include a sensor, and the sensor may be configured to receive one or more action signals while the user is moving. In some embodiments, the sensor may comprise, but is not limited to, one or more of an EMG sensor, a posture sensor, an electrocardiogram sensor, a respiration sensor, a temperature sensor, a humidity sensor, an inertial sensor, a blood oxygen saturation sensor, a Hall sensor, an electrodermal sensor, rotation sensor, etc. In some embodiments, the action signal may comprise one or more of an EMG signal, a posture signal, an electrocardiogram signal, a respiratory rate signal, a temperature signal, a humidity signal, and the like. The sensor may be placed at various locations on the wearable device 130 depending on the type of action signal to be received. For example, in some embodiments, an EMG sensor (also referred to as an electrode) may be located at a human muscle location, and the EMG sensor may be configured to receive an EMG signal while the user is moving. The EMG signal and corresponding function information (eg, frequency information, amplitude information, etc.) can indicate the state of the muscle during the user's movement. The body motion sensor may be installed at various locations of the human body (eg, at locations corresponding to the torso, limbs, and joints in the wearable device 130), and the body motion sensor may be configured to receive a body motion signal while the user is moving. The body motion signal and corresponding feature information (eg, angular velocity direction, angular velocity value, angular velocity acceleration value, angle, displacement information, voltage, etc.) may reflect the user's body motion. The ECG sensor may be located at a location around the chest of a person, and the ECG sensor may be configured to acquire ECG data while the user is moving. The respiration sensor may be located at a location around the chest of a human body, and the respiration sensor may be configured to obtain respiration data (eg, respiration rate, respiration amplitude, etc.) while the user is moving. The temperature sensor may be configured to obtain temperature data (eg, surface body temperature) while the user is moving. The humidity sensor may be configured to obtain environmental humidity data while the user is moving.

The processing module 220 may process data received from the receiving module 210, the control module 230, the communication module 240, the power supply module 250, and/or the input/output module 260. For example, the processing module 220 may process the action signal while the user is moving from the receiving module 210. In some embodiments, processing module 220 may pre-process an action signal (eg, EMG signal, body signal) received by receiving module 210. For example, processing module 220 may perform a process of segmenting an EMG signal or a body motion signal while the user is moving. As another example, the processing module 220 may perform pre-processing (eg, a filtering process, a signal correction process) on the EMG signal while the user is moving, improving the quality of the EMG signal. As yet another example, processing module 220 may determine feature information corresponding to the motion signal based on the motion signal while the user is moving. In some embodiments, processing module 220 may process a command or operation received from input/output module 260. In some embodiments, the processed data may be stored in memory or on a hard disk. In some embodiments, processing module 220 may transmit processed data to one or more components of motion control system 100 via communication module 240 or network 120. For example, processing module 220 may send a user motion control result to control module 230 and control module 230 may perform subsequent operations or commands according to the result of the action definition.

The control module 230 may communicate with other modules of the wearable device 130. In some embodiments, the control module 230 may control the operating states of other modules of the wearable device 130. For example, the control module 230 may control the state of the power source (e.g., normal mode, power saving mode), operating time of the power source, etc. power supply module 250. As another example, the control module 230 may control the input/output module 260 in accordance with the user action determination result to control the mobile terminal device 140 to send the user motion feedback result to the user. If there is a problem with the user's action (for example, the action does not meet the standard) while the user is moving, the control module 230 may control the input/output module 260 to control the mobile terminal device 140 to send a feedback signal to the user so that the user can know your own motion state in real time and adjust the action. In some embodiments, the control module 230 may also control one or more sensors of the receiving module 210 or other modules to provide feedback to the human body. For example, if a certain muscle exhibits too much force during a user's movement, the control module 230 may control the electrode module at a location on the muscle to electrically stimulate the user to prompt the user to correct the action in time.

In some embodiments, communication module 240 may be configured to exchange information or data. In some embodiments, the communication module 240 may be configured to communicate between components of the wearable device 130. For example, the receiving module 210 may send a user action signal (e.g., an EMG signal, a body signal, etc.) to the communication module 240 and the module 240 communication may send an action signal to processing module 220. In some embodiments, communication module 240 may also be configured to communicate between wearable device 130 and other components in system 100. For example, communication module 240 may send state information (e.g., switch state) of wearable device 130 to processing device 110 and the processing device 110 may control the wearable device 130 based on the status information. Communication module 240 can accommodate wired, wireless, and wired/wireless hybrid technologies.

In some embodiments, power module 250 may provide power to other components in system 100.

The input/output module 260 can receive, transmit, and send a signal. The input/output module 260 may interface or communicate with other components in the system 100. Other components in the motion control system 100 may interface or communicate through the input/output module 260.

It should be noted that the above descriptions of the system 100 and its modules are made merely for convenience of description and are not intended to limit one or more embodiments of the present disclosure by the scope of the embodiments presented. It should be understood that those skilled in the art, after understanding the operating principle of the system, can arbitrarily combine various modules or form a subsystem for connection with other modules or remove one or more modules from it. For example, the receiving module 210 and the processing module 220 may be combined into one module, which may have functions of receiving and processing the user action signal. As another example, the processing module 220 may not be provided in the wearable device 130, but be integrated into the processing device 110. Such modifications are within the scope of protection of one or more embodiments of the present disclosure.

In fig. 3 illustrates exemplary hardware and/or software components of a computing device 300 corresponding to some embodiments of the present disclosure. In some embodiments, processing device 110 and/or mobile terminal device 140 may be implemented on computing device 300. In some embodiments, action data reporting system 160 may be implemented on computing device 300. As shown in FIG. 3, computing device 300 may include an internal communication bus 310, a processor 320, a read-only memory 330, a random access memory 340, a communication port 350, an input/output interface 360, a hard disk 370, and a user interface 380.

Internal communication bus 310 may allow data to be transferred among various components of computing device 300. For example, processor 320 may, via internal communication bus 310, send data to memory or other hardware components such as input/output interface 360.

The processor 320 may execute computer commands (control program) and perform the functions of the motion control system 100 described herein. Computer instructions may comprise a program, object, component, data structure, procedure, module, and function (function refers to a specific function described in the disclosure). For example, the processor 320 may process an action signal (e.g., an EMG signal, a body motion signal) received from the wearable device 130 and/or from the mobile terminal device 140 of the motion control system 100 while the user is moving, and monitor the user's action in accordance with the action signal while user movements. For illustration purposes only, in computing device 300 in FIG. 3 shows only one processor, but it should be noted that the computing device 300 in the present disclosure may also include multiple processors.

Memory (eg, read-only memory (ROM) 330, random access memory (RAM) 340, hard drive 370, etc.) of computing device 300 may store data received from any other component of motion control system 100. In some embodiments, the memory of the computing device 300 may be located in the wearable device 130 as well as in the processing device 110.

The input/output interface 360 may be configured to input or output signal, data, or information. In some embodiments, the input/output interface 360 may allow a user to interact with the motion control system 100.

The hard disk 370 may be configured to store information and data generated by or received from the processing device 110. For example, the hard drive 370 may store user confirmation information. In some embodiments, the hard drive 370 may be provided in the processing device 110 or in the wearable device 130. The user interface 380 may allow interaction and exchange of information between the computing device 300 and the user. In some embodiments, the user interface 380 may be configured to present motion recordings generated by the motion monitoring system 100 to the user. In some embodiments, user interface 380 may include a physical display, such as a speaker display, liquid crystal display, LED display, OLED display, electronic ink (E-Ink) display, and the like.

For example, wearable device 130 in system 100 may adopt any structure. For example, wearable device 130 may adopt the structure of wearable device 400 shown in FIG. 4. To describe the wearable device 130, the wearable device 400 in FIG. 4 can use clothing as an example. As shown in FIG. 4, wearable device 400 may include an outer garment 410. The outerwear item 410 may include an outerwear item 4110, one or more outerwear processing modules 4120, one or more outerwear feedback modules 4130, one or more outerwear receiving modules 4140, and the like. The outerwear body 4110 may refer to a garment worn on the upper part of the human body. In some embodiments, the outerwear body 4110 may include a short sleeve T-shirt, a long sleeve T-shirt, a shirt, a jacket, or the like. One or more outerwear processing modules 4120 and one or more outerwear receiving modules 4140 may be located on the outerwear body 4110 in areas that correspond to different areas of the human body. One or more outerwear feedback modules 4130 may be located anywhere on the outerwear body 4110, and one or more outerwear feedback modules 4130 may be configured to return information about a motion state of a wearer's upper body. Exemplary feedback methods may include, but are not limited to, speech prompt, text prompt, pressure prompt, electrical stimulation, and the like. In some embodiments, one or more outerwear acquisition modules 4140 may include, but are not limited to, one or more sensors including a body motion sensor, an ECG sensor, an EMG sensor, a temperature sensor, a humidity sensor, an acid-base sensor, an audio transducer waves, etc. The sensors in the outerwear acquisition module 4140 may be placed at various locations on the user's body in accordance with various signals to be measured. For example, when the body motion sensor is configured to receive a body motion signal while the wearer is moving, the body motion sensor may be placed at locations of the garment body 4110 corresponding to the torso, arms, and knuckles. As another example, when the EMG sensor is configured to receive an EMG signal while the user is moving, the EMG sensor may be located near the muscles that are to be measured by the user. In some embodiments, the body motion sensor may include, but is not limited to, a three-axis acceleration sensor, a three-axis angular velocity sensor, a magnetic force sensor, etc. or any combination thereof. For example, the body motion sensor may include a three-axis acceleration sensor and a three-axis angular velocity sensor. In some embodiments, the body motion sensor may also include a strain gauge. A strain gauge may be a sensor whose operation may be based on the deformation generated by the force and deformation of the object being measured. In some embodiments, the load cell may include, but is not limited to, one or more of a load cell load cell, a load cell pressure sensor, a load cell torque sensor, a load cell displacement sensor, a load cell acceleration sensor, and etc. For example, a strain gauge sensor can be installed at a joint location of a user, and by measuring the resistance of the strain gauge sensor, which changes with increasing length, the bending angle and bending direction of the user's joint can be obtained. It should be noted that in addition to the above-mentioned outerwear base 4110, outerwear processing unit 4120, outerwear feedback unit 4130, and outerwear receiving unit 4140, outerwear 410 may also include other modules such as a power supply module , communication module, input/output module, etc. The outerwear processing module 4120 may be similar to the processing module 220 in FIG. 2 and the outerwear receiving module 4140 may be similar to the receiving module 210 in FIG. 2. For a further description of the various modules of the outerwear item 410, refer to FIG. 2 and its accompanying descriptions in this disclosure, which are not repeated here.

In fig. 5 is a flowchart of an exemplary target action determination process consistent with some embodiments of the present disclosure. In some embodiments, process 500 may be implemented by system 100. For example, memory in processing device 110 and/or mobile device 140 may store one or more sets of action analysis and identification commands. An instruction set can contain multiple commands. The processor of the processing device 110 and/or the mobile device 140 may, at run time, read and execute a plurality of instructions from the instruction set and execute the process 500 in accordance with the plurality of instructions. Process 500 may complete in real time, or each operation may complete at varying times. Process 500 may comprise the following steps.

At step 510, user action data may be obtained while the user is moving. The user action data may correspond to the action of an unknown user.

Processing device 110 and/or mobile device 140 may measure the aforementioned action data at multiple measurement locations on the user, for example, may obtain raw data while the user is moving. Namely, the processing device 110 and/or the mobile device 140 may be communicated with the wearable device 130 directly or through a network 120. The wearable device 130 may have a plurality of sensors. When a user wears the wearable device 130, a plurality of sensors may be attached to a plurality of locations on the user's body. Therefore, the processing device 110 and/or the mobile device 140 can obtain measurement results from a plurality of sensors attached to the user at a plurality of measurement locations through respective acquisition modules to obtain data about the user's action.

User action data may refer to action data generated based on information about human body parameters while the user is moving. In some embodiments, the human body parameter information may comprise, but is not limited to, one or more signals of an EMG signal, a body movement signal, an electrocardiographic signal, a temperature signal, a humidity signal, a blood oxygen concentration, a respiratory rate, and the like. .

Because a user's action is the result of the coordination of multiple muscles and joints, accordingly, the user's action data may also comprise data collected by a plurality of sensors at M locations on the user's body, where M is an integer greater than 1. In some embodiments, the data collected by each individual sensor can be considered as a piece of subdata about the action. For example, in some embodiments, multiple sensors in wearable device 130 may receive signals from multiple parts of the body as the user moves. The combination of body motion signals from multiple body parts can reflect the relative motion situation between different parts of the human body. For example, an EMG sensor in wearable device 130 may receive an EMG signal while the user is moving, a body motion sensor in wearable device 130 may receive a body motion signal while the user is moving, and an angle sensor and angular velocity sensor in wearable device 130 may receive an angle and angular velocity for each joint as the user moves. The signal from each of the above sensors is recorded as a piece of action subdata. All subdata about a user's action can be combined to form action data.

For example, when a person performs the action of raising an arm, the corresponding action data may include angle data and angular velocity data of the shoulder, angle data and angular velocity data between the shoulder and forearm, EMG data of the biceps brachii muscle, EMG data of the deltoid muscle, EMG data trapezius muscle and back muscle group data, etc., measured by the wearable device 130. As another example, when the user performs a seated chest press fixation, the EMG sensor in the wearable device 130 corresponding to the position of the human body's pectoral muscle, latissimus dorsi muscle, etc. .d., can receive the EMG signal for the corresponding muscle position of the user. As a further other example, when the user performs a squat, the EMG sensor in the wearable device 130 corresponding to the position of the gluteus maximus muscle, quadriceps muscle of the person, etc. can receive the EMG signal of the corresponding muscle position of the user and the nearby angular sensor and angular velocity sensor can obtain the angle and angular velocity between the thigh and calf. The data received by each single sensor can be considered as a piece of action subdata. Therefore, the user's arm-raising action data may also include a plurality of user-action sub-data corresponding to data collected at multiple locations on the user's body when the user performs the arm-raising action.

When the user wears the wearable device 130, the wearable device 130 can start measuring the user's motion at any time, and the user can start exercising or start resting at any time. Thus, the processing device 110 and/or the mobile device 140 does not know whether the wearable device 130 receives a piece of random data about the user's action or the aforementioned fitness action, nor does the processing device 110 and/or the mobile device 140 know when the fitness action started in the received data. Therefore, what the wearable device 130 receives is action data, the content of which is unknown. That is, the system 100 is in an unknown state regarding what type of movement the user is performing (that is, whether the user's action contains multiple known target actions and the start time of each target action) and when the user begins to exercise. Therefore, the action data also does not have any identifier indicating the content of the action. Just for convenience of description, the present disclosure may determine whether the action data contains an action identifier as an example to describe whether the content of the action data is known.

At step 520, based on one or more sets of possible action reference data whose contents are known, by performing action identification on the user action data, it can be determined whether the user action contains a reference action (i.e., a target action) corresponding to one or more sets of possible action reference data.

The target action may refer to a specific action among the actual actions performed by the user, such as curls, bench press, squats, punches, push-ups, deadlifts, abdominal strengthening, etc. A reference action may refer to a standard action with action content corresponding to a specific action performed by a specialist (such as a trainer, etc.). In some embodiments, identification of the target action may be performed immediately or within a specified time after the user action data is received. The specified time can be a very short time, for example within one hour or 0.1, 0.5 seconds. Identification of the target action can also be performed in real time by obtaining data about the user's action. For example, when a user performs an action, the processor of the processing device 110 and/or the mobile device 140 may obtain data about the user's action while at the same time stretching or compressing the user's action data to a different scale and then comparing it with reference action data, thus , while simultaneously performing action identification on the user action data. The comparison method described above may be described in other parts of the present disclosure.

The user action reference data may comprise multiple sets of action reference data measured by a plurality of sensors. When obtaining action reference data similar to the user activity data set, a plurality of sensors may also be attached to M measurement locations on the practitioner, for example, the practitioner may wear a wearable device 130. The practitioner may then perform the specific action and data acquisition device (e.g., device 110 processing and/or mobile device 140 or other devices) may receive corresponding user action data through a corresponding acquisition module in the wearable device 130. Therefore, possible action reference data may be action data for a reference action whose content is known and measured at M locations measurements on the specialist, such as reference data about the user's action, marked with the content of the action. For example, reference data for a shoulder lift action may contain data collected from the same set of locations as the fitness trainer. The specialist may be a person who is used as a model to obtain reference data about the activity, such as a fitness trainer. For the data acquired by each sensor, the user's arm-raising action reference data may comprise M pieces of action reference subdata, each of which corresponds to data acquired at one of the M locations on the body when the practitioner performs the arm-raising action. In some embodiments, the M measurement locations on the practitioner may have a one-to-one correspondence with the M measurement locations on the user when the user's action data is acquired. Accordingly, the M pieces of user action subdata in the user action data may have a one-to-one correspondence with the M pieces of reference action subdata.

Action reference data may also be generated in other ways. For example, action data of a virtual person can be obtained through computer simulation, and reference action data can be obtained by artificially adjusting the virtual person in a video through artificial intelligence technologies (eg, artificially adjusting Mr. Olympia's action in a demonstration video). As long as certain action data can standardly represent a certain action and the content of the certain action data is known, certain action data can be used in the present disclosure as reference action data. As mentioned above, just for convenience of description, the present disclosure may take user action data tagged with content as an example to describe action data with known content.

At step 520, processing device 110 and/or mobile device 140 may access the action reference database. The action reference database may contain multiple sets of action reference data. A plurality of action reference data sets may correspond to a plurality of reference actions. For example, each action reference data set may correspond to one action reference data, or each action reference data set may correspond to multiple action reference data, or each action reference data set may correspond to multiple action reference data sets.

When performing target action identification on user action data, processing device 110 and/or mobile device 140 may identify user action data by sequentially comparing the user action data with each of a plurality of action reference data sets in the reference action database using a two-level operation dropouts. The two-level sifting operation can sift through multiple sets of action reference data through two different sifting operations and finally determine which reference action contains the user action. For example, a two-level screening operation may comprise a combination of a degree-based screening operation and a probability-based screening operation.

Namely, in the first-level screening, the processing device 110 and/or the mobile device 140 may select a set of candidate action references as the candidate action references of the first level, and then use the selected candidate user action references to determine the degree of difference of the action with user action data to determine whether the difference in data values between the user action data and possible first-level reference action data is small enough. If the degree of difference between the user action data and the determined candidate action reference data of the first level is less than a predetermined value, the candidate action reference data of the first level may shift to the candidate action reference data of the second level. Each possible second-level action reference data corresponds to one or more advanced reference actions, that is, second-level reference actions.

Attrition at the second level may be probabilistic attrition. In second-level screening, processing device 110 and/or mobile device 140 may determine the probability that the user action contains an advanced reference action (second-level reference action) and then determine whether the user action contains a second-level reference action. Whether the user action contains a target action corresponding to the target action reference data can be determined based on the result of the second level screening. The specific steps are described below.

At step 521, each action reference data set may be selected, in turn, from the plurality of action reference data sets as candidate first-level action reference data.

At step 522, the degree of difference between the possible first-level action reference data and the user action data may be determined.

At step 523, a judgment may be made as to whether the degree of difference value is less than the first predetermined value. If the difference degree value is greater than or equal to the first specified value, the overall difference between the user action and the possible first level action reference data may be considered relatively large. Then, operation 521 may be performed, that is, from the plurality of sets of action reference data in the action reference database, the next set of reference data may be selected as possible first-level action reference data, and the next set of reference data may again be compared with the action reference data. user action at the data value level. If the difference degree value of the next set of reference data is less than the first specified value, the overall difference between the user action and the possible first level action reference data can be considered small. Next, step 524 may be performed, that is, the candidate action data of the first layer can be determined as the candidate action data of the second layer, and then target identification of the next layer action data can be performed.

At step 525, a distance between the user action data and each of several sets of possible second-level action reference data may be determined.

At step 526, the probability that the user action data contains a target action corresponding to the possible second-level action reference data can be determined based on each distance.

At step 527, a decision is made whether the maximum value among the probability values is greater than the second predetermined value. If the maximum value does not exceed the second predetermined value, step 529 may be performed, that is, it is determined that the user action does not contain a reference action corresponding to the possible second-level action reference data. If the maximum value is greater than the second predetermined value, step 528 may be executed, that is, it is decided that the user action contains a reference action corresponding to the possible reference action data of the second level with the highest probability value, and the reference action is the target action.

The basis for selecting possible first-level action reference data may be random and sequential, or may be selected according to a specific rule, which is not limited to in the present disclosure. For example, all action reference data may be numbered in advance in the action reference database, and then the processing device 110 and/or mobile device 140 may select the object action data reference position as the possible first-level action reference data position corresponding to the number.

When comparing possible first-level action reference data with user action data, a comparison method using a sliding window may be adopted. For example, processing device 110 and/or mobile device 140 may move a sliding window of user action data along a time axis and select a segment of user action data in the sliding window. Since M pieces of user action subdata in the user action data are collected in parallel, the sliding window can act on each piece of user action subdata simultaneously and slide over each piece of user action subdata in parallel. The sliding window may correspond to a specified time interval (such as 0.1 seconds, 0.5 seconds, 1 second, etc.). Therefore, for M user action subdata pieces, the sliding window may contain M data segments for user action data collected in a given time interval. Processing apparatus 110 and/or mobile device 140 may respectively compare the M action data segments with some or all of the M action reference subdata pieces corresponding to the possible first-level action reference data to obtain one or more comparison subresults and then determine a complex degree differences by weighing and summing one or more subresults of the comparison. The complex degree of difference indicates the difference between the user action data and the reference action data. The smaller the value of the complex degree of difference, the smaller the difference, which indicates that the closer the user action data segment is to the reference action data, the closer the user action corresponding to the user action data segment is to the reference action, and the processing device 110 and/or or the mobile device 140 may decide that the user action data contains a reference action. For example, when a user performs a biceps curl while the user is moving, the user action data may include a corresponding user action data segment. When the processing device 110 and/or the mobile device 140 compares the user action data segment corresponding to the biceps curl with the reference action data corresponding to the biceps curl, the value of the complex degree of difference may be very small. On the other hand, a smaller complex difference value may indicate that the position of the user action data segment in the user action data is closer to the location of the target action in the user action data, that is, the user action corresponding to the user action data segment is closer in time by the time the user completes the target action.

Specifically, processing device 110 110 and/or mobile device 140 may use a sliding window of a given width to slide user activity data along a time axis with a given step size and select a segment of user activity data from the sliding window each time. For example, processing device 110 and/or mobile device 140 may sequentially select, with a given step size, a segment of continuous data with a given data length from the user action data. Considering that the speed of a user performing a target action may differ from the speed of a standard action performed by a specialist, the length of the sliding window may be negatively correlated with the speed of the use action to compensate for the difference. That is, when the user's action speed is faster, the usable sliding window length is shorter, and when the user's action speed is slower, the usable sliding window length is longer.

The specified step size can be a constant value. Since the complex difference value also denotes the time distance between the user action corresponding to the current user action data segment and the target action, the specified step size can also be adjusted based on the complex difference value. For example, to improve the efficiency of identifying a target action, a given step size can be positively correlated with the value of the complex degree of difference at the previous moment. A positive correlation may indicate that the given size is proportional to the value of the complex degree of difference at the previous moment, or it may be that the current step size at a particular step is chosen in a manner based on the value of the complex degree of difference at the previous moment, or it may be that the step size for the current moment greater than the step size for the previous moment by a constant amount, etc., which is not limited here. A given step size may also be positively correlated with the trend of the complex dissimilarity value. For example, if the difference between the complex value of the degree of difference at the current moment and the complex value of the degree of difference at the previous moment is greater than 0, i.e. the variation trend of the complex difference value increases, which means that the user action corresponding to the current user action data segment becomes more distant and moves away from the target action in time. In this case, the processing device 110 and/or the mobile device 140 may increase the step size. If the difference between the complex difference degree value at the current moment and the complex difference degree value at the previous moment is less than 0, it means that the user action corresponding to the current user action data segment is getting closer and closer to the target action over time. In this case, the processing device 110 and/or the mobile device 140 may reduce the step size. If the difference between the complex value of the degree of difference at the current moment and the complex value of the degree of difference at the previous moment is 0, the step size can remain unchanged.

Since the width of the sliding window is specified, the length of the data segment cut from the user action data can also be specified. Therefore, the user action data segment can correspond to all possible first-level action reference data. The user action data segment may also correspond to a portion of possible first-level action reference data. In some embodiments, the action reference data may comprise one or more action identification data segments. The action identification data may be action data (eg, angular velocity data, speed data, etc.) corresponding to at least a characteristic action portion of the reference action, which is substantially configured to represent the characteristic action. A reference action may be unique to a reference action. The reference action may be defined through a characteristic action portion, or all data may be defined as action reference data through the action identification data, so that when a data segment similar to the action identification data appears in the user action data, the user action can be recognized as containing corresponding target action. Meanwhile, the user action identification data may exist only in the action reference subdata portion of the action reference data, or the action identification data may exist in each action reference subdata. Action identification data existing in the action reference subdata may be referred to as action identification subdata.

For a sliding window at a current time, processing device 110 and/or mobile device 140 may compare M segments of user action data with corresponding M pieces of action identification subdata, respectively, to obtain corresponding M degrees of difference. The M degrees of difference can be weighted and summed to produce a complex degree of difference. If the complex difference degree is smaller than the first set value, the candidate action reference data of the first level that has passed the first-level screening can be determined, and the candidate action reference data of the first level can be selected as the candidate action reference data of the second level. If the complex degree of difference value is greater than the first specified value, the sliding window may slide to the next segment of user action data with a given step size, and then the comparison may be repeated until the complex degree of difference value is less than the first specified value, or until the sliding window will not advance to the end of the user action data.

Specifically, when comparing a certain segment of the action identification subdata of a certain piece of possible first-level action reference data with its corresponding user action subdata, the following operations may be performed.

For a specific activity identification subdata segment in the possible first-level activity reference data, the data is two-dimensional. For example, possible first-level action reference data for raising an arm may include action identification subdata at various times for the angle of forearm relative to the shoulder. The action identification subdata may contain a plurality of angle values and a plurality of corresponding times since the action identification subdata is two-dimensional data. For such action identification subdata for a single parameter, when a segment of action identification subdata is included in one piece of possible first-level action reference data, the specific process for obtaining the complex degree of difference is as follows. In fig. 7A shows an example action data identification segment in the action reference data and a curve of a user action subdata segment collected by a sliding window in the time axis user action data in accordance with some embodiments of the present disclosure. The action identification subdata in the action reference data may contain a plurality of data fragments {a _j } = {a ₁ , a ₂ , a ₃ , …a _n }, where n is an integer greater than 1, each data fragment corresponds to a timestamp and As j increases, the time instant corresponding to the timestamp of each data a _j increases sequentially. That is, the data points in the vector {aj} can be arranged in chronological order. A user action subdata segment may contain multiple data fragments {b _i } = {b ₁ , b ₂ , a ₃ , ..., b _m }, where m is an integer greater than 1, each data fragment corresponds to a timestamp, and when i increases, the time point corresponding to the timestamp of each data bi is incrementally increased. That is, the data points in the vector {b _i } can be arranged in chronological order. Generally speaking, the duration corresponding to the sliding window is less than the duration corresponding to the action identification subdata, and the amount of data corresponding to the sliding window is less than the amount of data corresponding to the automatic identification data, that is, m < n.

Assuming that the sampling rates and action rates of the action reference data and the user action data are the same for the same time interval, the number of data points in the action reference data and the number of user action data points may be the same. Thus, the segment {b _i } of the user action subdata can correspond to data of the same length in the action identification subdata {a _j }. That is, every data point in { b _i } can correspond to a data point in { a _j }. When determining the degree of difference between {b _i } and {a _j }, {b _i } only needs to be taken along the time axis t, the distance of one data point can slide each time, and one determination of the degree of difference between a data point and the corresponding data point in { a _j } can be determined each time. However, assuming that the sampling rates and/or action rates in the action reference data and the user action data are inconsistent, there is no one-to-one correspondence between the user action data points and the reference action data points. In this case, the time-action relation for {b _i } must be adjusted according to different time scales to create an adjusted action-time relation compatible with the action-time relation for the action reference data. For example, if the sampling rates of action reference data and user action data are the same, but the action rates are inconsistent, the time taken by the user to perform the action is different from the time spent by the corresponding reference action. For example, if the hardware sampling rate is 100 data points per second, the joint flexion speed of a reference action changes from 0° to 90° in 1 second, but the user action speed can only change from 0° to 45° in 1 second for 100 points data, reference action data corresponds to an angle change of 90°, and user action data corresponds to an angle change of 45°. Thus, the time interval {b _i } must be adjusted according to different time scales, i.e. {b _i } must expand or contract according to different scales, then distinguish the degree of difference between the processed data point and the corresponding data point at {a _j } once for each scale, until all degrees of difference corresponding to all scales are determined. Specific operations can be described as follows.

First, the processing device 110 and/or the mobile device 140 may calculate the distance D _ji between any point b _i in the user action subdata and any point a _j in the action identification subdata by placing segment {bi} of the user action subdata and subdata {a _j } identifying an action on the same time axis, and defining a matrix D _{of m×n} distances m×n, as shown in FIG. 7B. Each element in the distance matrix denotes the distance from the i-th (i≤m) point of the user action subdata segment to the j-th (i ≤ n) point of the action identification subdata in the possible first-level action reference data. Taking a fitness action such as bicep curls as an example of a user action, the user action subdata {b _i } may contain the angles between the shoulder and the forearm, and the corresponding user action identification subdata {a _j } in the reference data may also contain the angles between the shoulder and the forearm . Thus, the distance D _ji can indicate the difference D _ji = | a _j -b _i | between the angle between the user's shoulder and forearm and the angle represented by the action identification subdata. The distance D ₅₆ between a ₅ and b ₆ and the distance D ₅₃ between a ₅ and b ₃ is shown in FIG. 7A. Of course, the distance can also be a distance defined in other ways. For example, the distance between any point b _i in the action subdata and any point a _j in the action identification subdata can be Euclidean distance, Manhattan distance, P-parameter distance, cosine distance, Chebyshev distance, Marxian distance, edit distance, Jackard distance, or any different correlation distance. Thus, the _m×n distance matrix D contains point-to-point distances between all points in the user action subdata segment {b _i } at all scales and all points in the action identification subdata {a _j }.

Secondly, the processing device 110 and/or the mobile device 140 can determine the shortest distance path P _min in the _m×n distance matrix D, i.e. lowest regularization price. The distance can be expressed by the vector P = {p _k } = {p ₁ , p ₂ , … …, p _l }, which is a sequence consisting of some elements of the _m×n distance matrix D, where l is the number of elements at distance P. The distance P may contain a set of numbers, each of which is an element (i.e., a distance) in D _m×n . Any two adjacent numbers are two adjacent elements of the _m×n distance matrix D, and the position of the next number in the sequence in _{the m×n} distance matrix must be lower right or lower right of the corresponding position of the previous number in the sequence in _{the m×n} distance matrix D. Since the time corresponding to the user action subdata segment in the sliding window is shorter than the time corresponding to the action identification subdata, the two ends of the shortest distance path P can correspond to the first data point b ₁ and the last data point b _m from the number {b _i }, then is, the first number in the sequence is p ₁ =D _1x and the last number is p _l =D _my , where x <n, y <n, x and y are the positions of the corresponding data points in {a _j }, respectively. The shortest distance path P _min is the distance with the smallest sum of all elements among all paths satisfying the above conditions, that is, .

Assuming that the sliding window can be set with different durations and different data sampling rates, the number of data points in the user action data segment in the time window may differ. This may cause the value of the shortest distance path P to differ depending on the data sampling rate and the length of the sensors' time window. Taking these factors into account, the degree f of the difference can be defined as , Where may be a weighted average of the elements of the shortest distance path P _min , W = {w ₁ , w ₂ , … w _l } may be a weight vector consisting of m elements with 1 row and l columns, and is a scalar. For example, the degree of difference can be defined as the average distance, that is, all elements in W are equal to 1/l.

Through the above calculation, the processing apparatus 110 and/or the mobile device 140 can complete the operations of determining the degree of difference between each of the plurality of scaled data segments and the action identification subdata, respectively, and determining the smallest degree of difference among the degrees of difference between the data segment and the action identification subdata.

In some embodiments, to reduce the amount of computation and determine the scale of adjustment of the user action subdata, before determining the degree of difference of a segment of the user action subdata within a sliding window, the processing device 110 and/or the mobile device 140 may determine a target data interval for comparison based on action identification subdata, and compare only the target comparison data interval with the user action subdata segment to obtain the degree of difference. The target data interval for comparison may contain multiple action subdata points. The target data interval for comparison can be determined based on the following relationship between the user action data and the action identification subdata selected by the sliding window.

1) The beginning and end of the user action data segment selected by the sliding window exactly correspond to the beginning and end of the action identification subdata, that is, the user action data segment is exactly the complete distribution of the action identification subdata. In this case, the processing device 110 and/or the mobile device 140 may first determine that the first data point and the last data point of the user segment of the user action data selected by the sliding window correspond to the first data point and the last data point of the corresponding action identification subdata, respectively. . The target data interval for comparison covers all action identification subdata. A distance constraint P may contain the following: each number in the sequence is an element of D _m×n , any two adjacent numbers are two adjacent elements in the distance matrix D _m×n , and the position of the next number in the sequence in the distance matrix D _m×n is at the bottom right, or below to the right of the corresponding position of the previous number in the matrix D _{of m×n} distances. The two ends of the shortest distance path P can correspond to p ₁ =D ₁₁ and p _l =D _mn . That is, the shortest distance path P of the distance matrix is the shortest distance from the upper left corner to the lower right corner of the distance matrix.

2) The starting point of the user action subdata selected by the sliding window corresponds to the starting point of the action identification subdata, and the ending point of the user action subdata selected by the sliding window corresponds to a certain data point in the action identification subdata. That is, the user action subdata corresponds to a segment in the action identification subdata after scaling, and this segment is located at the starting position of the action identification subdata. In this case, the processing device 110 and/or the mobile device 140 may first decide that the first data point of the user action data segment selected by the sliding window corresponds to the first data point corresponding to the action identification subdata. Then, the target data interval for comparison may cover all user identification subdata. The distance constraint P may contain the following: each number in the sequence is an element in D _m×n ; any two adjacent numbers are two adjacent elements of the _m×n distance matrix D, and the position of the next number in the sequence of the _m×n distance matrix is the position to the right down or down to the right of the corresponding position of the previous number in _{the m×n} distance matrix Both ends of the shortest distance path P can correspond to p ₁ =D ₁₁ and p _l =D _my . That is, the shortest distance path P of the distance matrix is the distance from the upper left corner of the distance matrix to a certain point in the last row along the downward right direction.

3) The starting point of the user action subdata selected by the sliding window corresponds to the defined action identification subdata data, and the ending point of the user action subdata selected by the sliding window corresponds to the end point of the action identification subdata. That is, after scaling, the user action subdata may be a data segment in the final position of the action identification subdata. In this case, the processing device 110 and/or the mobile device 140 may first decide that the last data point of the user action data segment selected by the sliding window corresponds to the last data point of the corresponding action identification subdata. The target data interval for comparison may then cover all of the action identification subdata. A distance constraint P may contain the following: each number from the sequence is an element in D _m×n , any two adjacent numbers are two adjacent elements of the D _m×n distance matrix, and the position of the next number from the sequence in the D _m×n distance matrix is the down-right position or down to the right to the corresponding position of the previous number in the D _m×n distance matrix. Both ends of the shortest distance path P can correspond to p ₁ =D _1x and p _l =D _mn . That is, the shortest distance path P of the distance matrix is the shortest distance path from a certain point in the first row of the distance matrix to the lower right corner.

4) The starting point and ending data point of the user action subdata selected by the sliding window correspond to the two intermediate data points of the action identification subdata, respectively, and not the first data point and not the last data point. That is, the starting point of the user action subdata selected by the sliding window cannot be the beginning of the action identification subdata, and the ending point of the user action subdata selected by the sliding window also cannot be the end of the action identification subdata. After scaling, the user action subdata may be a data segment of the action identification subdata, and the segment may be located at a certain position in the middle of the action identification subdata. "Intermediate data" of a data segment can refer to data at any position other than the start point and end point of the data. In this case, the processing device 110 and/or the mobile device 140 may first decide that the first data point and the last data point of the user action subdata segment selected by the sliding window cannot be the first data point and the last data point of the corresponding action identification subdata. The target data interval for comparison may then cover all action identification subdata except the first data point and the last data point. The distance constraint P may contain the following: each number from the sequence is an element in D _m×n , any two adjacent numbers are two adjacent elements of the _m×n distance matrix D, and the position of the next number from the sequence in the _m×n distance matrix D is the down-right position or down to the right from the corresponding position of the previous number on the D _m×n distance matrix. The two ends of the shortest distance path P can correspond to p _l =D _1x and pl=D _my , where . That is, the shortest distance path P of the distance matrix starts from a certain point in the middle of the first row of the distance matrix, goes to the lower right corner and ends at a certain point in the last row.

In some embodiments, the action identification subdata may correspond to a start action or an end action of the reference action. In this case, whether a certain point in the user action subdata is the starting point or the ending point of the action can be determined by changing the angular velocities before the point and after the point. For example, when a user action data point shows that the angular velocity of the corresponding user action is 0, and the angular velocity of the point after the user action data point is not 0, it can be proven that the user starts a certain fitness action from the user action data point, so that it can be determined that a user action data point is the starting point of the user action. As another example, when the angular velocity of the user's action point is 0 and the angular velocity of the previous point is not 0, it can be proven that the user has stopped the action at that point, that is, the user's action point can be determined to be the end point of the action.

It should be understood that performing target action identification on user action data may involve one of the following two cases. 1) The action identification subdata may be compared with the user action data subsegment to obtain a complex degree of difference value. When the complex degree of difference value is smaller than the first set value, the processing device 110 and/or the mobile device 140 may decide that the first level candidate action reference data is the second layer candidate action reference data. 2) The user identification data may be compared with the user action data segment to obtain a complex degree of difference value. When the complex degree of difference value is greater than the first predetermined value, the processing device 110 and/or the mobile device 140 may move the sliding window to the next data segment with a predetermined step size and then repeat the comparison. The first predetermined value may be a criterion for determining whether the distance between the user action data and the action corresponding to the possible first-level action reference data is sufficiently small. Therefore, when the value of the complex degree of difference is smaller than the first specified value, it can be proven that the distance between the user action data and the possible reference action data of the first level is relatively small (that is, the similarity is very high), the user action data can be treated as containing target action data corresponding to possible first-level reference action data. In this case, the candidate action reference data of the first level can be defined as the candidate action reference data of the second level. When the complex degree of difference value is larger than the first set value, the similarity between the user action data and the candidate action reference data of the first level may be very low, and the user action data may be determined to not contain the target action data corresponding to the reference action.

The above presents a method for determining the degree of difference between a piece of user action subdata and the corresponding action identification subdata. In fig. 7C schematically illustrates a process for determining a complex degree of difference when the user action data contains a plurality of pieces of user action subdata corresponding to some embodiments of the present disclosure.

If the action data for a particular action contains measurement data of M parameters and M is an integer greater than 1, the action data for said action contains M parallel measurement subdata fragments. Therefore, the user action data may contain M pieces of user action subdata. The possible first-level action reference data may also contain M pieces of possible first-level action reference subdata. Each piece of possible first-level action reference subdata corresponds to local action data in the general action data obtained by measuring parameters, and contains at least an independent action identification subdata segment. All action identification subdata together constitute the action identification data for the reference action.

When the candidate first-level action reference subdata corresponding to a specific piece of user action subdata contains an action identification subdata segment, a comparison sub-result between the user action subdata and the candidate first-level action reference subdata can be obtained based on the following operations. Processing apparatus 110 and/or mobile device 140 may select a user action data segment from each of the M user action subdata fragments using a sliding window, and sequentially compare the data segments with the corresponding M action identification subdata segments in the M action reference subdata fragments for determining degrees of difference so that sub-results of comparison can be obtained. Then, the M sub-results of the comparison can be weighted and summed to obtain a complex degree of difference, and whether the user action data contains a target action can be determined based on the complex degree of difference and the first predetermined value. For each piece of M pieces of user action subdata, the processing device 110 and/or mobile device 140 can obtain the degree of difference between the user action subdata and the action identification subdata based on the method described above.

Specifically, for each piece of the M user action subdata pieces, the processing device 110 and/or the mobile device 140 may collect a user action subdata segment through a sliding window. Sliding windows corresponding to M pieces of user action subdata can be linked or act independently when sliding. Each sliding window may have the same width, that is, the user action subdata segment corresponding to each sliding window may equally contain d (d is an integer greater than 1) data points and, as described above, each data point from number d of data points corresponds to a time stamp. Of course, the width of different sliding windows may also differ, and the amount of data contained in the user action subdata segment in each sliding window may also differ. The processing device 110 and/or the mobile device 140 may determine the total distance between each data point of the user action data subdata segment and each action data point of the action identification subdata according to the method described above. The processing device 110 and/or the mobile device 140 may obtain the minimum total distance regularization cost in the above manner and then determine the degree of difference between the user action subdata segment and the action identification subdata. Since there are M pieces of user action subdata, according to the above method, the processing device 110 and/or the mobile device 140 can obtain all M degrees of difference. Finally, processing device 110 and/or mobile device 140 may perform a weighted summation on the M comparison sub-results to obtain a complex degree of difference.

In some embodiments of the present disclosure, after receiving the complex degree of difference value, the processing device 110 and/or the mobile device 140 can directly determine whether the user action data contains possible first-level action reference data. For example, processing device 110 and/or mobile device 140 may set a first set value for the action identification data, and when the complex degree of difference value is greater than the first set value, processing device 110 and/or mobile device 140 may shift each of the M sliding windows to next data segment with a given step size and then the comparison can be repeated. If the complex degree of difference value is less than the first set value, the processing device 110 and/or the mobile device 140 may consider that the user action subdata set may contain possible first-level action reference data, and thus the above loop may be considered complete.

When the possible first-level action reference subdata corresponding to a piece of user action subdata contains a plurality of segments (for example, p segments, where p is an integer greater than 1) of the action identification subdata, the result of the comparison between the user action subdata and the possible reference subdata The first level action can be obtained based on the following method. For each fragment of the M user action subdata fragments, the processing device 110 and/or the mobile device 140 may select a user action subdata segment from each user action subdata fragment through a sliding window. Sliding windows corresponding to M pieces of user action subdata can be linked or act independently when sliding. Each sliding window may have the same width, that is, the user action subdata segment corresponding to each sliding window may uniformly contain d (d is an integer greater than 1) data points, and, as described above, each d data point from data points correspond to the timestamp. Of course, the width of different sliding windows may also differ, and the amount of data contained in the user action subdata segment in each sliding window may also differ. Processing apparatus 110 and/or mobile device 140 may calculate p integral distances between the user action subdata segment and p action identification subdata segments, respectively, based on the method described above. The processing device 110 and/or the mobile device 140 may determine the minimum regularization cost p of the total distances using the above method as a sub-result of the comparison between the user action subdata segment and the action identification subdata. Since there are M pieces of user action subdata, the processing device 110 and/or the mobile device 140 can obtain a total of M comparison sub-results based on the above method. Finally, processing device 110 and/or mobile device 140 may perform a weighted summation of the M comparison sub-results to obtain a complex degree of difference value.

In some embodiments of the present disclosure, after receiving the complex degree of difference value, the processing device 110 and/or the mobile device 140 may directly determine whether the user action subdata set contains possible first-level action reference data. For example, the processing device 110 and/or the mobile device 140 may set a first set value for the action identification data, and when the complex degree of difference value is greater than the first set value, the processing device 110 and/or the mobile device 140 may shift each of the M sliding windows M to the next data segment with a given step size and then the comparison can be repeated. If the value of the complex degrees of difference is less than the first set value, the processing device 110 and/or the mobile device 140 may decide that the possible first-level action reference data is the possible second-level action reference data, and thus the above loop can be finished.

In some embodiments, after determining possible second-level action reference data, processing device 110 and/or mobile device 140 may further confirm whether the user action contains a reference action corresponding to the possible second-level action reference data.

In some embodiments of the present disclosure, a second target value may also be set, and the second target value may be a probability related target value. Assuming that through the above process, the processing device 110 and/or the mobile device 140 can finally decide that N (N is an integer greater than 1) sets of possible first-level action reference data are possible second-level action reference data. Namely, N distances (complex comparison results) between user action data and N sets of possible reference second-level action data can be calculated, respectively, by comparison, and then N probability values can be calculated, respectively, through N distances, respectively. The maximum value among the N probability values may be compared with the second predetermined value, and it may be determined whether the user action data contains possible second-level action reference data corresponding to the maximum probability value. The probability that the user action data contains a target action corresponding to the i-th probable reference action data of the second level can be expressed as follows:

where D _j is the distance (eg, the aforementioned complex regularization cost or complex degree of dissimilarity) between the user action data and the j-th possible second-level reference action data. The smaller the distance value between the user action data and the possible second-level action reference data, the higher the probability that the user action data may contain a target action corresponding to the possible second-level action reference data may be higher. For example, by comparing the user action data with three possible second-level action reference data (that is, taking N=3, the number of three possible second-level action reference data is 1, 2 and 3, respectively), it can be obtained that the distance between the user action data 1 and the possible second-level action reference data is D1, it can be obtained that the distance between the user action data 2 and the possible second-level action reference data is D2, and it can be obtained that the distance between the user action data 3 about the user action and possible reference data about the second level action is equal to D3. The smaller the distance value between the user action data and the possible second-level action reference data, the higher the likelihood that the user action data may contain a target action corresponding to the possible second-level action reference data. For example, the probability that the user action data contains a target action corresponding to the second level candidate action reference data 1 can be determined as follows:

The probability that the user action data contains a target action corresponding to action reference data 2 can be determined as follows:

The probability that the user action data contains a target action corresponding to reference action data 3 can be determined as follows:

In this case, the calculated maximum value of the three probability values can be compared with the second target value. When the maximum probability value is greater than the second predetermined value, it is decided that the user action data contains a target action corresponding to the possible second-level action reference data n, and the possible second-level action reference data n is the possible second-level action reference data , corresponding to the maximum probability value. When the maximum probability value is less than the second level predetermined value, it is decided that the user action data does not contain a target action corresponding to the reference action database.

At step 530, after the target action is determined, content associated with the target action may be sent to the user.

Specifically, after the user's action is identified, the action of the user's movement can be monitored and monitored information can be sent to the user. Monitoring the action of the user's movement may comprise monitoring information associated with the user's action. In some embodiments, information associated with a user action may include one or more types of user action, number of actions, quality of actions (e.g., whether the user action meets a standard), time of action, and the like. The action type may refer to fitness actions that the user performs while exercising. In some embodiments, the action type may comprise, but is not limited to, one or more of the following actions: seated chest press, squat, deadlift, plank, running, swimming, and the like. The amount of action may refer to the number of actions performed while the user is moving. For example, a user can perform 10 seated chest presses while the user is moving and these 10 times is the number of actions. Action quality may refer to the degree of standardness of a fitness action performed by a user relative to a standard fitness action. For example, when the user performs the action of squatting, the processing device 110 may determine the action type for the user's action based on the feature information of the action signal (EMG signal and body movement signal) corresponding to a certain muscle position (gluteus maximus, quadriceps femoris, etc. .), and determine the action quality of the user's crouch action based on the action signal for the standard crouch action. The action time may refer to the time corresponding to each of one or more types of user action or to the total time of the movement process.

To summarize, the target action determination method and system 100 presented in the present disclosure can obtain action data while the user is moving, and then the action data can be compared with reference action data labeled with the content of the action so that it can be determined whether user movement is a target action that coincides with a reference action. The method and system can perform target action identification on user action data without knowing (whether the user performed an annotation type action and when the annotation type action was performed) what action the user performed, and send content associated with the target action to the user after determining target action. Using the above-mentioned technical solutions, the present methods and systems have higher intelligence than traditional methods and systems and improve the user experience.

Having thus described the basic concepts, it should be readily apparent to those skilled in the art upon reading this detailed disclosure that the preceding detailed disclosure is intended to be presented by way of example and not limitation. Various changes, improvements and modifications may occur and are intended for those skilled in the art, although not expressly stated herein. These changes, improvements, and modifications are intended to be suggested by this disclosure and are within the spirit and scope of the exemplary embodiments of this disclosure.

In addition, certain terminology has been used to describe embodiments of the present disclosure. For example, the terms “one embodiment,” “an embodiment,” and/or “certain embodiments” mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present disclosure . It is therefore emphasized and understood that two or more references to “an embodiment,” “one embodiment,” or “alternative embodiment” in different parts of the present specification do not necessarily all refer to the same embodiment. Additionally, specific features, structures, or characteristics may be combined as appropriate in one or more embodiments of the present disclosure.

Claims (65)

1. A method for identifying a user action, containing the steps of: obtaining user action data collected from a plurality of measurement locations on the user, wherein the user action data corresponds to an unknown user action; identifying that the user action contains a target action when data about the user action is obtained based on at least one set of target action reference data, wherein the at least one set of target action reference data corresponds to the target action, Moreover, at the stage of identifying that the user action contains the target action: obtaining a plurality of sets of action reference data, each set of action reference data corresponding to at least one reference action; performing a two-level screening operation on a plurality of action reference data sets based on the user action data, wherein the two-level screening operation comprises a combination of a degree-of-difference screening operation and a probability-based screening operation and includes the steps of: selecting each set of action reference data in turn from the plurality of sets of action reference data as possible action reference data; determining at least one degree of difference by segment-by-segment comparison of at least one action identification subdata segment from the possible action reference data with a corresponding user action subdata segment; determine the complex degree of difference by weighing and summing the specified at least one degree of difference; selecting N pieces of possible second-level action reference data from a plurality of sets of action reference data, wherein the value of the complex degree of difference of the possible second-level action reference data is less than the first specified value, where N is an integer greater than 1; calculating N distances between the user action data and N pieces of possible second-level action reference data, respectively; calculating N probability values based on the N distances, respectively; selecting possible second-level action references whose probability value is greater than the second predetermined value as target action references; And defining a reference action corresponding to the target reference action data as a target action; And deciding that the user action contains the target action based on the result of the two-level screening operation; And send the user information related to the target action. 2. The method according to claim 1, in which each action reference data set contains M action reference subdata fragments, each action reference subdata fragment containing at least one action identification subdata segment, where M is an integer greater than 1; The action identification subdata for the M pieces of action reference subdata form integral action identification data, each segment of the action identification subdata corresponding to at least a portion of the action reference at at least one measurement location of the plurality of measurement locations. 3. The method of claim 1, wherein in the step of determining at least one degree of difference by segment-by-segment comparison of at least one action identification subdata segment from the possible action reference data with a corresponding user action subdata segment: selecting a sliding window with a predetermined duration on each piece of user identification subdata, the sliding window comprising a data segment of user action data collected in a predetermined time interval; And for the sliding window, the degree of difference between the data segment and the corresponding action identification subdata is currently determined. 4. The method according to claim 3, wherein the step of identifying that the user action contains the target action further comprises the steps of: determining that the value of the complex degree of difference is greater than the first predetermined value; And move the sliding window to the next segment with the specified step size and repeat the comparison. 5. The method according to claim 4, in which the duration of data collection corresponding to the data segment in the sliding window is negatively correlated with the speed of the user's action, wherein the specified step size satisfies one or more of the following conditions: the given step size is positively correlated with the value of the complex degree of difference at the previous moment; And a given step size is positively correlated with the trend of change in the value of the complex degree of difference. 6. The method of claim 3, wherein the data segment contains a plurality of data points about a user action; wherein at the step of determining at least one degree of difference by segment-by-segment comparison of at least one action identification subdata segment from the possible action reference data with a corresponding user action subdata segment: selecting a target data interval for comparison from the action identification subdata, wherein the target data interval for comparison contains a plurality of identification data points, adjusting the data segment according to a plurality of scales to obtain a plurality of adjusted data segments, determining the degree of difference between the action identification subdata and each adjusted data segment of the plurality of adjusted data segments, respectively, and determining a minimum degree of difference among the degrees of difference between the action identification subdata and the plurality of adjusted data segments. 7. The method of claim 3, wherein in the step of determining at least one degree of difference by segment-by-segment comparison of at least one action identification subdata segment from the possible action reference data with a corresponding user action subdata segment: defining a distance matrix [D _ij ], where D _ij denotes the distance between the i-th data point of the target data interval for comparison and the j-th data point of the data segment; determine the shortest distance path of the distance matrix, and the shortest distance path satisfies the following conditions: the starting point on the shortest distance path is in the first line of [D _ij ], two adjacent points on the shortest distance path are adjacent in the distance matrix, the next point on the shortest distance path is to the right of, below, or below the right of the previous point, the end point on the shortest distance path is in the last line of [D _ij ], and the shortest distance path has the lowest regularization cost, and the regularization cost is determined by the distances of points on the corresponding shortest distance path of the distance matrix; And the degree of difference is related to the cost of regularization. 8. The method of claim 7, wherein if the first data point of the data segment is defined as the data point at which the user action begins, the starting point on the shortest distance path is the distance D ₁₁ between the first point of the data segment and the first point of the target data interval for comparisons. 9. The method of claim 7, wherein if the last data point of the data segment is defined as the data point at which the user action ends, the end point on the shortest distance path is the distance D _mn between the last point of the data segment and the last point of the target data interval for comparisons. 10. User action identification system, containing: at least one storage medium, wherein the at least one storage medium stores at least one set of commands for obtaining data about a user's action while the user is moving; And at least one processor coupled to said at least one storage medium, wherein, when the system is operating, said at least one processor reads said at least one set of instructions and executes a method comprising the steps of: obtaining user action data collected from a plurality of measurement locations on the user, wherein the user action data corresponds to an unknown user action; identifying that the user action contains a target action when the user action data is obtained based on at least one set of target action reference data, wherein the at least one set of target action reference data corresponds to the target action; Moreover, at the stage of identifying that the user action contains the target action: obtaining a plurality of sets of action reference data, each set of action reference data corresponding to at least one reference action; performing a two-level screening operation on a plurality of action reference data sets based on the user action data, wherein the two-level screening operation comprises a combination of a degree-of-difference screening operation and a probability-based screening operation and includes the steps of: selecting each set of action reference data in turn from the plurality of sets of action reference data as possible action reference data; determining at least one degree of difference by segment-by-segment comparison of at least one action identification subdata segment from the possible action reference data with a corresponding user action subdata segment; determine the complex degree of difference by weighing and summing the specified at least one degree of difference; selecting N pieces of possible second-level action reference data from a plurality of sets of action reference data, wherein the value of the complex degree of difference of the possible second-level action reference data is less than the first specified value, where N is an integer greater than 1; calculating N distances between the user action data and N pieces of possible second-level action reference data, respectively; calculating N probability values based on the N distances, respectively; selecting possible second-level action references whose probability value is greater than the second predetermined value as target action references; And defining a reference action corresponding to the target reference action data as a target action; And deciding that the user action contains the target action based on the result of the two-level screening operation; And send the user information related to the target action.