WO2023040449A1 - Déclenchement d'une instruction d'opération de client à l'aide d'une action de fitness - Google Patents

Déclenchement d'une instruction d'opération de client à l'aide d'une action de fitness Download PDF

Info

Publication number
WO2023040449A1
WO2023040449A1 PCT/CN2022/106220 CN2022106220W WO2023040449A1 WO 2023040449 A1 WO2023040449 A1 WO 2023040449A1 CN 2022106220 W CN2022106220 W CN 2022106220W WO 2023040449 A1 WO2023040449 A1 WO 2023040449A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
fitness
action
exercise
actions
Prior art date
Application number
PCT/CN2022/106220
Other languages
English (en)
Chinese (zh)
Inventor
蒋晨之
宝鹏庆
王维强
Original Assignee
支付宝(杭州)信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 支付宝(杭州)信息技术有限公司 filed Critical 支付宝(杭州)信息技术有限公司
Publication of WO2023040449A1 publication Critical patent/WO2023040449A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks

Definitions

  • the embodiments of this specification relate to the field of computer applications, and in particular to a method and device for triggering client operation instructions by using fitness actions.
  • the client also known as the client, refers to an application that corresponds to the server and provides local services for users, and needs to cooperate with the server to run.
  • the client is generally installed on the user terminal, and the user generally triggers an operation command of the client by touching the screen or pressing a specific button on the mobile terminal.
  • the current interaction method is too single, which affects the user experience.
  • the embodiments of this specification provide a method and device for triggering an operation command of a client terminal by using a fitness action.
  • a method for triggering client operation instructions by using fitness actions including: determining the correspondence between preset operation instructions and fitness actions; wherein each operation instruction corresponds to at least one Fitness action, the fitness action corresponding to any two operation instructions does not overlap; obtain the user's current exercise posture data; identify the fitness movement corresponding to the acquired exercise posture data; determine and execute according to the corresponding relationship between the preset operation instruction and fitness action Operation instructions corresponding to fitness actions.
  • a device for triggering client operation instructions by using fitness actions including: a correspondence determination module for determining the correspondence between preset operation instructions and fitness actions; wherein each operation instruction Corresponding to at least one fitness action, the fitness actions corresponding to any two operation instructions do not overlap; the exercise posture data acquisition module is used to obtain the user's current exercise posture data; the fitness action recognition module is used to identify the acquired exercise posture data corresponding to Fitness action; an operation instruction execution module, configured to determine and execute the operation instruction corresponding to the exercise action according to the preset corresponding relationship between the operation instruction and the exercise action.
  • a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, the above-mentioned method for triggering an operation instruction of a client by using a fitness action is implemented.
  • an electronic device including: a processor; a memory for storing processor-executable instructions; wherein, the processor executes the executable instructions to realize the above-mentioned utilization of fitness Action triggers the method of client operation instruction.
  • the embodiment of this specification by acquiring the user's exercise posture data and identifying the fitness action corresponding to the exercise posture data, according to the corresponding relationship between the preset operation instruction and the fitness movement, determine the operation instruction corresponding to the fitness movement, and execute the corresponding operating instructions.
  • the embodiment of this specification utilizes fitness actions to trigger operation instructions, increases the interaction mode between the user and the client, and improves user experience.
  • Fig. 1 is a flow chart of a method for triggering an operation instruction of a client by using a fitness action according to an exemplary embodiment of the present specification.
  • Fig. 2 is a schematic diagram of identifying a user's fitness action according to an embodiment of this specification.
  • Fig. 3 is a block diagram of an apparatus for triggering an operation instruction of a client by using a fitness action according to an exemplary embodiment of the present specification.
  • Fig. 4 is a hardware structural diagram of a computer device where an apparatus for triggering an operation command of a client terminal by using a fitness action according to an exemplary embodiment of this specification is located.
  • the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification.
  • the method may include more or less steps than those described in this specification.
  • a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step in other embodiments describe.
  • the user for the client software installed on the mobile terminal, the user generally interacts with the client software by touching the screen or pressing a specific button on the mobile terminal, in other words, the user triggers the client software through the above-mentioned method operating instructions to complete the interaction with the client software.
  • the client software installed on other user terminals such as computers, TVs, etc.
  • the user generally completes the interaction with the client software through auxiliary tools (such as mouse keyboard or remote control, etc.). Tool to select a specific button on the page to trigger an operation command and complete the interaction.
  • the current interaction method is relatively single, which affects the user experience.
  • the fitness action can be combined with the trigger operation command, so that the user can exercise while using the client.
  • the embodiment of this specification utilizes fitness actions to trigger operation instructions, increases the interaction mode between the user and the client, and improves user experience.
  • this specification provides a method for using fitness actions to trigger client operation instructions, including the following steps: Step 101, determine the correspondence between preset operation instructions and fitness actions; where each operation instruction corresponds to At least one fitness movement, and the fitness movements corresponding to any two operation instructions do not overlap.
  • the execution body of the method may be client software installed on the mobile terminal, such as applied to smart phones, tablet computers, smart watches and other wearable devices. In some cases, it can also be applied to client software installed on non-mobile user terminals, such as personal computers, TVs, etc. When users use the client, they can freely choose to use the traditional touch method. Trigger the client's operation instructions, or trigger the client's operation instructions through fitness actions.
  • the user terminal where the client is located has a speaker or can connect to a wireless headset, it can also remind the user through voice, such as reminding the user of the action standard or reminder The placement of the user's mobile phone, etc.; if the picture of the user terminal where the client is located can be seen by the user in fitness, the user can also be reminded through the picture.
  • the user enables the operation command triggered by fitness action, which can be set on the setting page of the client; it can also be controlled by a specific gesture, such as pulling down the current page to the bottom to enable the operation command triggered by fitness action, or on the page Draw a big triangle on the top to trigger operation instructions through fitness actions; if the client or user terminal has a built-in voice assistant, the user can also enable the activation of exercise actions to trigger operation instructions by talking to the voice assistant; if the user terminal In the case of at least one button on the keyboard, it is also possible to trigger an operation command through a fitness action by pressing a specified button or a specified combination of buttons. Of course, the specified button or the combination of specified buttons has not been given other functions in advance. This specification of the user terminal does not limit the method of triggering the operation instruction of the client through fitness actions.
  • the user enables triggering operation commands through fitness actions, which can be enabled in the entire client to trigger operation commands through fitness actions, or only in a certain module of the client. For example, in the client Open it in an embedded applet, or open it in some interface of the client. For example, for an electronic wallet client, it can be opened on a certain carbon saving interface of the client. Of course, users can choose which interfaces to open.
  • the client After the user triggers the operation command through the fitness action, in order to determine what kind of operation command the user is currently doing to trigger the fitness action, the client needs to perform step 101 to determine the corresponding relationship between the preset operation command and the fitness action , so as to determine the operation instruction through the corresponding relationship.
  • a fitness action corresponds to multiple operation instructions, it will be impossible to determine which operation instruction the user wants to trigger when the user performs the exercise action. Therefore, a fitness action will only correspond to one operation instruction. In addition, in order to further improve user experience, one operation instruction can be selected to correspond to multiple fitness actions.
  • each operation instruction corresponds not only to a fitness movement, but also to a standard of the fitness movement.
  • the fitness action standard can be the execution frequency of the fitness action, or the range of motion, etc. For example, if the fitness action is a high leg raise, you can set the frequency of the fitness action execution to 2 times per second. If the fitness action is a squat, You can set the range of motion for squatting.
  • the standard of fitness actions can be set according to the user's age and physical condition. For older users, the frequency of fitness actions should not be too high to prevent users from accidents during the completion of fitness actions; For the best users, the frequency and range of fitness movements should not be too high to prevent accidents.
  • the user When the user performs a fitness movement, but the frequency or amplitude of the fitness movement does not meet the standard of the corresponding fitness movement, the user may be prompted by voice, vibration or other forms that the current fitness movement is not standard.
  • the client detects that the user has performed a certain fitness action, it judges whether the fitness action performed by the user meets the standard of the fitness action, and if it does not meet the standard of the fitness action, through a method To remind the user that the current fitness action is not standard.
  • Step 103 acquiring the current motion posture data of the user.
  • Step 105 identifying the fitness action corresponding to the acquired exercise posture data.
  • step 103-step 105 will be uniformly described.
  • the method of acquiring the user's motion posture data and identifying the user's fitness action this manual provides the following two acquisition methods. Of course, these two methods do not represent limitations to this manual. If other methods can also obtain motion posture If the data is recognized and the fitness action is recognized, it is also possible to use other acquisition methods for acquisition and recognition.
  • the first method is to acquire through the camera module of the user terminal, and use a machine learning method to identify the fitness action performed by the user.
  • step 103 is specifically: acquiring motion posture data captured by the camera module of the user terminal.
  • step 105 is specifically: obtaining a pre-trained feature extraction model and a pre-trained action recognition model; inputting the motion gesture data of the user into the feature extraction model to extract the action features of the motion gesture data; Input the action recognition model to determine the fitness action corresponding to the user's exercise posture data.
  • the camera module of the user terminal may be a self-contained camera module of the user terminal, or may be an external camera module of the user terminal.
  • the user can be reminded to stand at the position where the camera module can capture the whole body through sound, vibration or image.
  • the range that the module can capture can also remind the user that it has exceeded the range through sound, vibration, or text or images on the display interface, and prompt the user to move to enter the shooting range of the camera module.
  • the user since the user interacts with the client through fitness actions, if the camera module of the mobile terminal is used, the user may be too far away from the mobile terminal in order to capture the whole body of the user, and cannot see the image on the mobile terminal. The content cannot be well interacted. Therefore, when the screen of the mobile terminal is small, after the user activates the fitness action to trigger the operation command, the user may be prompted to use the screen projection method to project the content of the mobile terminal client to a larger screen; or the user may be prompted to use Take pictures with an external camera; or prompt the user to stand in front of the mirror, make the screen face the user, and use the rear camera of the mobile terminal to capture the mirrored action of the user in the mirror.
  • the user can also see the content on the screen of the mobile terminal, so that the user can have a better experience.
  • the acquired motion posture data is the real-time motion posture data of the user, not the motion posture data recorded in the past.
  • a live body detection function when obtaining motion posture data.
  • live body detection can be used while shooting data to detect whether the currently captured video is the user's Real-time motion posture data.
  • the current data is not analyzed, and the user is prompted that there is no living body currently by voice or other methods.
  • the feature extraction of the user's motion posture data can be realized by various methods, and the extracted action features can also include various forms, such as, action features, which are the positional features of human body key points in each frame image, and the human body key points It is used to identify human actions.
  • action features which are the positional features of human body key points in each frame image
  • human body key points It is used to identify human actions.
  • the extracted action features can be shown in FIG. 2 .
  • the key points of the human body can include joint points such as ankles, knees, shoulders, elbows, hip bones, and elbows, as well as key points used to identify facial movements.
  • Identifying key points can be identified by gesture detection models MoveNet, BlazePose or PoseNet.
  • key points can also be detected through auxiliary tools, for example, special stickers can be pasted on key points, and the user's posture can be determined by the position of the special sticker on the device.
  • the user terminal is a mobile terminal or a wearable device
  • the user's motion can also be detected according to the sensor on the mobile terminal to obtain the user's motion posture data .
  • step 103 includes: obtaining a measurement sequence of a motion sensor of the mobile terminal within a preset time period, and using the measurement sequence as motion posture data; the motion sensor is a sensor for detecting dynamic movement of the mobile terminal.
  • step 105 includes: acquiring a pre-trained action recognition model; inputting the measurement sequence into the action recognition model, and determining an action identifier corresponding to the measurement sequence.
  • the motion sensor may be an acceleration sensor, a gyroscope, and other sensors already installed in the mobile terminal itself.
  • the motion sensor may be an acceleration sensor, a gyroscope, and other sensors already installed in the mobile terminal itself.
  • the acceleration sensor a gyroscope
  • the motion trajectory of the mobile terminal or wearable device can be obtained through the displacement formula.
  • the measurement sequence can be the measured value of the sensor obtained directly, or It is the motion trajectory obtained according to the velocity formula and displacement formula.
  • the client In the case that the user terminal is a mobile terminal, in order to enable the motion sensor to obtain a measurement sequence, the client needs to be fixed on a certain part of the body, such as the user's forearm, forearm or thigh. If it is fixed, it can be fixed by a strap. If the mobile terminal is not large, you can also put the mobile terminal in the user's tight pocket (to prevent it from shaking in the pocket and affect the final measurement result). Of course, you can also place the mobile terminal The mobile terminal is held in the hand for measurement.
  • the method provided in this specification is a method for the user to interact with the client, and when the motion sensor on the mobile terminal or wearable device is used to measure, the user cannot see the motion of the mobile terminal or wearable device.
  • the display interface in order to enable the user to interact with the client, the user can be prompted to cast the screen, the display interface of the mobile terminal or wearable device can be displayed on the screen, or the visual interaction can be replaced by voice broadcast to complete the user and Client interaction.
  • the action recognition model is a pre-trained neural network model used to identify fitness actions. Different users have different actions. For each user, in order to enable the action recognition model to successfully identify the specific fitness actions of the user, the user can be prompted to record in advance before the recognition. Videos/measurement sequences of users doing different fitness actions, and the action/sensor sequence recognition model is updated through these videos, so that the action recognition model of each user's client can be more suitable for the user.
  • Step 107 according to the preset correspondence between the operation instruction and the exercise action, determine and execute the operation instruction corresponding to the exercise action.
  • the user may also be prompted by voice that the operation instruction has been completed, so that the user can get feedback.
  • some rewards can be set to attract users to use this function. For example, if this function is used in the takeaway applet of an e-wallet client, every time the user completes 1000 squat actions (Here is just an example, and does not represent a limitation on fitness actions), reward users with certain points (points can be exchanged for red envelopes). If this function is used on a certain carbon saving statistics interface of an e-wallet client, the user can increase the amount of stolen energy in "stealing energy" through fitness actions.
  • the rankings that each user can see include other users who have a friend relationship with the user, and each of the user and these other users.
  • Daily exercise time/number of times of exercise for example, the ranking that can be seen by the user may include the daily number of squats of users who have a friend relationship with the user.
  • the client can be allowed to obtain the data required for the ranking at regular intervals, or only when the user opens the ranking Get the user's data.
  • the methods for obtaining the corresponding relationship between the operation instruction and the fitness action may include the following methods.
  • Determining the correspondence between preset operation instructions and fitness actions may be: determining the correspondence between user-defined operating instructions and fitness actions.
  • the correspondence between operation instructions and fitness actions is preset by the user, and determining the correspondence between the preset operation instructions and fitness actions is the acquisition of the correspondence between the user's preset operation instructions and fitness actions.
  • the user-defined setting may be that the user selects several fitness movements from all the operation instructions and all fitness movements that can be selected, and selects several operation instructions, and sets the corresponding relationship between the operation instructions and the fitness movements.
  • User-defined settings can also be selected according to content recommended by the client.
  • the client recommendation can be based on the user's operation instruction usage requirements, or based on the user's exercise needs, or based on the user's friend relationship.
  • the client needs to first recommend to the user the operation instructions, fitness actions, or the corresponding relationship between the operation instructions and the fitness actions that meet the user's usage needs, so that The user prefers what the client recommends.
  • the method for customizing settings includes: determining the user's usage requirements; the usage requirements include at least one of the following: the execution requirements of operation instructions, and the user's exercise requirements; according to the user's usage requirements, preferentially recommending The operation instruction, exercise action corresponding to the use requirement, or the correspondence between the operation instruction and the exercise action; determining the correspondence between the operation instruction selected by the user and the exercise exercise.
  • the priority recommendation is performed, that is, the operation instructions, fitness actions or the corresponding relationship between the two that meet the user's use needs are preferentially displayed, so that the user can preferentially choose the recommended ones.
  • the correspondence between the operation instructions and the exercise actions is user-defined, the correspondence between the operation instructions and the exercise actions customized by different users may also be different.
  • determining the user's usage requirements includes: determining the common operating instructions that the user executes more frequently than the preset frequency threshold, that is, determining the user's most frequently used operating instructions. According to the user's needs, recommend to the user the operation instructions, fitness actions corresponding to the use needs, or the correspondence between the operation instructions and the fitness actions, including: displaying common operation instructions to the user on the corresponding relationship selection page, that is, in On the corresponding relationship selection page, common operation instructions are in front of other operation instructions.
  • determining the user's use needs includes: determining the user's exercise needs.
  • the user's exercise needs can be determined according to the questionnaire filled out by the user, or determined according to the user's exercise type selection.
  • the exercise type can include fat loss, plasticity, muscle gain, etc., and can also be selected Areas for fat loss, plasticity and muscle gain.
  • preferentially recommend to the user the corresponding operation instructions, fitness actions, or the correspondence between the operation instructions and the fitness actions including: determining a preset number of recommended actions according to the user's exercise needs ; The preset number is greater than the number of configurable operation instructions; The recommended actions are preferentially displayed to the user on the corresponding relationship selection page.
  • the recommended actions are in line with the user's exercise needs. For example, if the user wants to increase abdominal muscles, the recommended actions may include planks, sit-ups, and so on.
  • the user's exercise needs may also include exercise levels, which may be selected by the user according to his or her own circumstances, or may be determined based on a questionnaire filled out by the user. .
  • the standard of fitness action can be comprehensively considered according to the user's exercise level, age and gender.
  • the client makes comprehensive recommendations based on the user's usage needs, then according to the user's usage needs, it will preferentially recommend to the user the operation instructions, fitness actions corresponding to the usage needs, or the corresponding relationship between the operation instructions and the fitness actions, including: According to the usage requirements of the user, determine a user group that has similar usage requirements to the user; determine the common correspondence of the user group; and preferentially display the common correspondence to the user on the correspondence selection page.
  • the user's usage requirements can include the user's exercise requirements and operation instruction usage requirements.
  • the exercise requirements here can be exercise types, such as fat loss, plasticity, or muscle gain.
  • the operation instruction usage requirements can be a certain page or Small program needs. Similar use requirements may refer to the same type of exercise and similar use requirements for operation instructions.
  • the commonly used corresponding relationship of the user group may refer to a corresponding relationship in which the number of users is greater than a preset threshold among the corresponding relationships between operating instructions and fitness actions that have been customized in the user group.
  • it is also possible to mark the recommended corresponding relationship for example, adding the label of "popularity TPO1", so that the user can preferentially select these corresponding relationships and improve the user experience.
  • the commonly used corresponding relationship can be obtained from the corresponding relationship recommended by the user uploaded by the user, or the corresponding relationship can be uploaded to the server after the client determines the corresponding relationship, and the commonly used corresponding relationship can be obtained from the corresponding relationship stored in the server obtained from .
  • the client can recommend to the user, among the users who have a friend relationship with the user, the corresponding relationship that uses the most. Specifically, the client needs to first confirm the user's friend relationship, determine the corresponding relationship used by users who have a friend relationship with the user, count the used corresponding relationships, and calculate the number of users who have a friend relationship with the user The most used sets of correspondences are used, and these correspondences are recommended to users first.
  • the user can customize the corresponding relationship between operation instructions and fitness actions, and combine fitness with daily use of the client, so that the user can complete fitness while using the client.
  • the client will consider the user's use needs or the user's friend relationship and make recommendations, the user can give priority to the operation instructions and fitness actions recommended by the client, or the corresponding relationship between the operation instructions and the fitness actions, so that the operation selected by the user
  • the corresponding relationship between instructions and fitness actions is in line with the user's needs for operating instructions or fitness needs, which improves the user experience.
  • Determining the correspondence between preset operation instructions and fitness actions can be determined not only according to user-defined settings, but also by determining the correspondence between system preset operation instructions and fitness actions.
  • the client is preset with the corresponding relationship between operation instructions and fitness actions. After the user triggers the operation instructions through fitness actions, the client will obtain the corresponding relationship between the operation instructions and fitness actions written in the client.
  • the preset correspondence between the operation instruction and the fitness action may be a correspondence that can enhance the interest.
  • the realization effect of the fitness action in the correspondence corresponds to the effect after the operation instruction is triggered.
  • its commonly used functions include watering others, stealing energy, and switching to the next friend user. Then, because watering requires a lot of strength, you can trigger watering through strength-based actions (such as squats, push-ups); because energy grows like fruit at a higher position on the tree, you need to jump up to pick it, you can use Jumping actions (such as raising your legs high) trigger energy stealing.
  • operation instructions can be triggered through fitness actions.
  • the client obtains the correspondence between the preset operation command and the fitness action.
  • the specific correspondence may include:
  • each action can set fitness action standards.
  • the specific fitness action standards are set according to the user's gender and age. For example, the user is 30 years old and male, and the set fitness action standards are: high leg raise, ten seconds 15 times, sit-ups, 10 in 10 seconds, 10 push-ups in 10 seconds, side bending at an angle greater than 60°, and 15 consecutive abdominal lifts with one leg.
  • the set fitness action standards are: high leg raise, ten seconds 15 times, sit-ups, 10 in 10 seconds, 10 push-ups in 10 seconds, side bending at an angle greater than 60°, and 15 consecutive abdominal lifts with one leg.
  • the proportion of the user's energy is also possible to give the user a certain reward when the fitness action performed by the user exceeds the fitness action standard to a certain extent. The proportion of the user's energy.
  • the client is a smart phone, which prompts the user to put the smart phone upright on the table and the ground through voice, and prompts the user to go far away.
  • the client obtains the captured image through the camera module of the smart phone, and detects the The location of the user in the module, when the location of the user is appropriate, the user will be prompted by voice that the user has arrived at the appropriate location.
  • the client After the user arrives at a suitable location, the client obtains the video captured by the camera module, inputs the captured video into the pre-trained feature extraction model, extracts the action features of the video, and inputs the action features into the action recognition model to identify whether there is a preset Fitness actions (the aforementioned high leg lifts, squats, push-ups, sit-ups, side bends, single-leg abdominal lifts, etc.).
  • a preset Fitness actions the aforementioned high leg lifts, squats, push-ups, sit-ups, side bends, single-leg abdominal lifts, etc.
  • determining that there is a preset fitness action such as determining that there is a high leg raising action
  • further determine the frequency of the user performing the high leg raising action if the user frequency exceeds the set high leg raising standard (15 in ten seconds), Then perform the energy steal corresponding to the leg raising action.
  • the user can be prompted by voice that the energy stealing action has been completed. For example, the user can be prompted "Congratulations, you have obtained the user's A10g energy”. Whether there are still energy balls that have not been stolen, etc. For example, the user can be prompted, "There are still 5 energy balls that have not been picked on the current page”.
  • the user can be prompted by voice, "Switched to the page of user B" . If the action performed by the user is single-leg abdominal lifting, and it meets the preset standard of single-leg abdominal lifting, the user can be prompted by voice, "You have entered your hybrid xx tree”. After entering the combined species, if the user performs sit-ups and satisfies the standard of sit-ups, the user can be prompted by voice "You have watered xx tree 10g".
  • this specification also provides an embodiment of a device for triggering a client operation command by using a fitness action and a terminal to which it is applied.
  • FIG. 3 is a block diagram of a device for triggering client operation instructions by using fitness actions according to an exemplary embodiment of this specification.
  • the device includes: a correspondence determination module 310, configured to determine a preset The corresponding relationship between the operation instruction and the fitness action; wherein, each operation instruction corresponds to at least one fitness action, and the fitness actions corresponding to any two operation instructions do not overlap.
  • the exercise posture data acquisition module 320 is configured to acquire the current exercise posture data of the user.
  • the exercise action recognition module 330 is configured to identify the exercise action corresponding to the acquired exercise posture data.
  • the operation instruction execution module 340 is configured to determine and execute the operation instruction corresponding to the exercise action according to the preset correspondence between the operation instruction and the exercise exercise.
  • the correspondence determining module 310 is specifically configured to: determine the correspondence between user-defined operating instructions and fitness actions.
  • the device may also include a custom setting module 300, configured to: determine the user's use requirements; the use requirements include at least one of the following: the execution requirements of operation instructions, the user's exercise Requirements; according to the user's use needs, on the corresponding relationship selection page, recommend to the user the operation instructions, fitness actions corresponding to the use needs, or the corresponding relationship between the operation instructions and the fitness actions; determine the relationship between the operation instructions selected by the user and the fitness actions Correspondence between.
  • the custom setting module 300 configured to: determine the user's use requirements; the use requirements include at least one of the following: the execution requirements of operation instructions, the user's exercise Requirements; according to the user's use needs, on the corresponding relationship selection page, recommend to the user the operation instructions, fitness actions corresponding to the use needs, or the corresponding relationship between the operation instructions and the fitness actions; determine the relationship between the operation instructions selected by the user and the fitness actions Correspondence between.
  • the determination of the user's usage requirements may include: determining that the user executes common operation instructions whose frequency is greater than a preset frequency threshold;
  • the operation instruction, exercise action, or the correspondence between the operation instruction and the exercise action may include: preferentially displaying common operation instructions to the user on the correspondence selection page.
  • the determination of the user's use needs may also include: determining the user's exercise needs; according to the user's use needs, preferentially recommending to the user the operation instructions and fitness actions corresponding to the use needs
  • the correspondence between operation instructions and fitness actions may also include: determining a preset number of recommended actions according to the user's exercise needs; the preset number is greater than the number of configurable operation instructions; Users are given priority to display recommended actions.
  • the operation instructions and fitness actions corresponding to the use needs are preferentially recommended to the user, or the corresponding relationship between the operation instructions and the fitness actions can also include: Determine the user group that has similar usage requirements as the user; determine the common correspondence of the user group; and preferentially display the common correspondence to the user on the correspondence selection page.
  • the motion posture data acquisition module 320 is specifically configured to obtain motion posture data captured by the camera module of the user terminal.
  • the fitness action recognition module 330 is specifically used to obtain a pre-trained feature extraction model and a pre-trained action recognition model; input the user's motion posture data into the feature extraction model, and extract the motion features of the motion posture data;
  • the action features are input into the action recognition model to determine the fitness action corresponding to the user's exercise posture data.
  • the action feature may be the position feature of the key points of the human body in each frame of image; the key points of the human body are used to identify the action of the human body.
  • the motion posture data acquisition module 320 can also be specifically used to obtain the measurement sequence of the motion sensor of the mobile terminal within a preset time, and use the measurement sequence as motion posture data; the motion sensor is used to detect the motion of the mobile terminal.
  • the fitness action recognition module 330 can also be specifically configured to obtain a pre-trained action recognition model; input the measurement sequence into the action recognition model, and determine the action identifier corresponding to the measurement sequence.
  • the device embodiment since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment.
  • the device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. It can be understood and implemented by those skilled in the art without creative effort.
  • the device may include: a processor 1010, a memory 1020, an input/output interface 1030 , a communication interface 1040 and a bus 1050 .
  • the processor 1010 , the memory 1020 , the input/output interface 1030 and the communication interface 1040 are connected to each other within the device through the bus 1050 .
  • the processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute related programs to realize the technical solutions provided by the embodiments of this specification.
  • a general-purpose CPU Central Processing Unit, central processing unit
  • a microprocessor an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits
  • ASIC Application Specific Integrated Circuit
  • the memory 1020 can be implemented in the form of ROM (Read Only Memory, read-only memory), RAM (Random Access Memory, random access memory), static storage device, dynamic storage device, and the like.
  • the memory 1020 can store operating systems and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020 and invoked by the processor 1010 for execution.
  • the input/output interface 1030 is used to connect the input/output module to realize information input and output.
  • the input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions.
  • the input device may include a keyboard, mouse, touch screen, microphone, various sensors, etc.
  • the output device may include a display, a speaker, a vibrator, an indicator light, and the like.
  • the communication interface 1040 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices.
  • the communication module can realize communication through wired means (such as USB, network cable, etc.), and can also realize communication through wireless means (such as mobile network, WIFI, Bluetooth, etc.).
  • Bus 1050 includes a path that carries information between the various components of the device (eg, processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).
  • the above device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components.
  • the above-mentioned device may only include components necessary to implement the solutions of the embodiments of this specification, and does not necessarily include all the components shown in the figure.
  • the embodiment of this specification also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned method for triggering an operation instruction of a client by using a fitness action is implemented.
  • Computer-readable media including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information.
  • Information may be computer readable instructions, data structures, modules of a program, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
  • computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Un ou plusieurs modes de réalisation de la présente invention concernent un procédé et un appareil permettant de déclencher une instruction d'opération de client à l'aide d'une action de fitness. Le procédé consiste : à acquérir des données de posture de mouvement d'un utilisateur ; à reconnaître une action de fitness correspondant aux données de posture de mouvement ; et à déterminer, selon une corrélation prédéfinie entre des instructions d'opération et des actions de fitness, une instruction d'opération correspondant à l'action de fitness, et à exécuter l'instruction d'opération correspondante. Dans les modes de réalisation de la présente invention, une instruction d'opération est déclenchée à l'aide d'une action de fitness, ce qui permet d'augmenter les modes d'interaction entre un utilisateur et un client, et d'améliorer l'expérience utilisateur.
PCT/CN2022/106220 2021-09-17 2022-07-18 Déclenchement d'une instruction d'opération de client à l'aide d'une action de fitness WO2023040449A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111093227.1 2021-09-17
CN202111093227.1A CN113849065A (zh) 2021-09-17 2021-09-17 一种利用健身动作触发客户端操作指令的方法及装置

Publications (1)

Publication Number Publication Date
WO2023040449A1 true WO2023040449A1 (fr) 2023-03-23

Family

ID=78974597

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/106220 WO2023040449A1 (fr) 2021-09-17 2022-07-18 Déclenchement d'une instruction d'opération de client à l'aide d'une action de fitness

Country Status (2)

Country Link
CN (1) CN113849065A (fr)
WO (1) WO2023040449A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116225238A (zh) * 2023-05-10 2023-06-06 荣耀终端有限公司 人机交互方法及系统
CN116225238B (zh) * 2023-05-10 2024-06-21 荣耀终端有限公司 人机交互方法及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113849065A (zh) * 2021-09-17 2021-12-28 支付宝(杭州)信息技术有限公司 一种利用健身动作触发客户端操作指令的方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109621332A (zh) * 2018-12-29 2019-04-16 北京卡路里信息技术有限公司 一种健身动作的属性确定方法、装置、设备和存储介质
CN110991482A (zh) * 2019-10-31 2020-04-10 深圳市木瓜有子健康科技有限公司 健身动作识别的方法、终端及计算机存储介质
CN112306220A (zh) * 2019-07-31 2021-02-02 北京字节跳动网络技术有限公司 基于肢体识别的控制方法、装置、电子设备及存储介质
CN113076002A (zh) * 2021-04-07 2021-07-06 江苏智恒文化科技有限公司 基于多部位动作识别的互联健身竞技系统及方法
CN113849065A (zh) * 2021-09-17 2021-12-28 支付宝(杭州)信息技术有限公司 一种利用健身动作触发客户端操作指令的方法及装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102622085A (zh) * 2012-04-11 2012-08-01 北京航空航天大学 多维感官人机交互系统及交互方法
CN103809733B (zh) * 2012-11-07 2018-07-20 北京三星通信技术研究有限公司 人机交互系统和方法
CN104881122B (zh) * 2015-05-29 2018-10-09 深圳奥比中光科技有限公司 一种体感交互系统激活方法、体感交互方法及系统
CN108803874A (zh) * 2018-05-30 2018-11-13 广东省智能制造研究所 一种基于机器视觉的人机行为交互方法
CN112543248A (zh) * 2020-11-24 2021-03-23 深圳传音控股股份有限公司 隔空操作方法、终端及存储介质
CN112637655A (zh) * 2021-01-08 2021-04-09 深圳市掌视互娱网络有限公司 智能电视的控制方法、系统及移动终端
CN112965592A (zh) * 2021-02-24 2021-06-15 中国工商银行股份有限公司 设备交互方法、装置及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109621332A (zh) * 2018-12-29 2019-04-16 北京卡路里信息技术有限公司 一种健身动作的属性确定方法、装置、设备和存储介质
CN112306220A (zh) * 2019-07-31 2021-02-02 北京字节跳动网络技术有限公司 基于肢体识别的控制方法、装置、电子设备及存储介质
CN110991482A (zh) * 2019-10-31 2020-04-10 深圳市木瓜有子健康科技有限公司 健身动作识别的方法、终端及计算机存储介质
CN113076002A (zh) * 2021-04-07 2021-07-06 江苏智恒文化科技有限公司 基于多部位动作识别的互联健身竞技系统及方法
CN113849065A (zh) * 2021-09-17 2021-12-28 支付宝(杭州)信息技术有限公司 一种利用健身动作触发客户端操作指令的方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116225238A (zh) * 2023-05-10 2023-06-06 荣耀终端有限公司 人机交互方法及系统
CN116225238B (zh) * 2023-05-10 2024-06-21 荣耀终端有限公司 人机交互方法及系统

Also Published As

Publication number Publication date
CN113849065A (zh) 2021-12-28

Similar Documents

Publication Publication Date Title
EP3131027B1 (fr) Procédé de détection d'informations d'activité d'utilisateur et son dispositif électronique
KR102091848B1 (ko) 전자 장치에서 사용자의 감정 정보를 제공하기 위한 장치 및 방법
KR102341985B1 (ko) 운동 보조 장치 및 운동 보조 방법
US10088901B2 (en) Display device and operating method thereof
KR102089002B1 (ko) 행동에 대한 피드백을 제공하는 웨어러블 디바이스 및 방법
CN112447273A (zh) 辅助健身的方法和电子装置
CN111698564B (zh) 信息推荐方法、装置、设备及存储介质
KR102466438B1 (ko) 인지 기능 평가 시스템 및 인지 기능 평가 방법
WO2021218940A1 (fr) Procédé et appareil de recommandation de classe d'entraînement
CN108132983A (zh) 服饰搭配的推荐方法及装置、可读存储介质、电子设备
WO2023040449A1 (fr) Déclenchement d'une instruction d'opération de client à l'aide d'une action de fitness
CN112349380A (zh) 基于云计算的健身指导方法、装置、设备及存储介质
WO2022174544A1 (fr) Procédé de comparaison d'actions, appareil, dispositif électronique, support de stockage, produit-programme informatique et programme informatique
JP2019016254A (ja) ユーザの姿勢を評価するための方法およびシステム
US20220284738A1 (en) Target user locking method and electronic device
CN106303939A (zh) 健康提醒的方法及装置
KR20220039440A (ko) 디스플레이 장치 및 그의 제어 방법
CN114432683A (zh) 一种智能语音播放方法及设备
CN108983956B (zh) 体感交互方法及装置
WO2023221233A1 (fr) Appareil, système et procédé de projection en miroir interactive
KR20230087352A (ko) 운동 상태 감지 장치 및 방법
CN113842622B (zh) 一种运动教学方法、装置、系统、电子设备及存储介质
US20220277845A1 (en) Prompt method and electronic device for fitness training
CN113139506A (zh) 一种计步方法、设备及存储介质
JP2022187952A (ja) プログラム、方法、情報処理装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22868826

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE