CN115177938A - Interactive intelligent body-building mirror device - Google Patents
Interactive intelligent body-building mirror device Download PDFInfo
- Publication number
- CN115177938A CN115177938A CN202210931241.2A CN202210931241A CN115177938A CN 115177938 A CN115177938 A CN 115177938A CN 202210931241 A CN202210931241 A CN 202210931241A CN 115177938 A CN115177938 A CN 115177938A
- Authority
- CN
- China
- Prior art keywords
- user
- page
- mirror
- user interface
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 114
- 230000009471 action Effects 0.000 claims abstract description 116
- 230000033001 locomotion Effects 0.000 claims abstract description 81
- 230000003993 interaction Effects 0.000 claims description 70
- 238000007667 floating Methods 0.000 claims description 30
- 230000000694 effects Effects 0.000 claims description 21
- 230000003068 static effect Effects 0.000 claims description 17
- 230000000007 visual effect Effects 0.000 claims description 5
- 230000009191 jumping Effects 0.000 claims description 2
- 238000013507 mapping Methods 0.000 claims description 2
- 230000000284 resting effect Effects 0.000 claims description 2
- 238000000034 method Methods 0.000 abstract description 33
- 239000000725 suspension Substances 0.000 abstract description 8
- 230000008569 process Effects 0.000 abstract description 7
- 230000006870 function Effects 0.000 description 61
- 210000000988 bone and bone Anatomy 0.000 description 27
- 238000003860 storage Methods 0.000 description 22
- 210000004197 pelvis Anatomy 0.000 description 10
- 238000012545 processing Methods 0.000 description 9
- 230000002829 reductive effect Effects 0.000 description 9
- 230000008451 emotion Effects 0.000 description 8
- 230000001815 facial effect Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 230000036772 blood pressure Effects 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 5
- 239000000463 material Substances 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 210000000577 adipose tissue Anatomy 0.000 description 4
- 230000027455 binding Effects 0.000 description 4
- 238000009739 binding Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 230000014509 gene expression Effects 0.000 description 4
- 238000005304 joining Methods 0.000 description 4
- 238000001931 thermography Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 230000009849 deactivation Effects 0.000 description 3
- 210000001624 hip Anatomy 0.000 description 3
- 230000000670 limiting effect Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 230000002618 waking effect Effects 0.000 description 3
- 208000027418 Wounds and injury Diseases 0.000 description 2
- 210000001015 abdomen Anatomy 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 210000000617 arm Anatomy 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 208000014674 injury Diseases 0.000 description 2
- 210000002414 leg Anatomy 0.000 description 2
- 230000036651 mood Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 210000003049 pelvic bone Anatomy 0.000 description 2
- 238000007493 shaping process Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 241000699670 Mus sp. Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000994 depressogenic effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000037213 diet Effects 0.000 description 1
- 235000005911 diet Nutrition 0.000 description 1
- 230000002996 emotional effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 238000005562 fading Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000005802 health problem Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 229910052760 oxygen Inorganic materials 0.000 description 1
- 239000001301 oxygen Substances 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 230000002040 relaxant effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 210000004243 sweat Anatomy 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
- A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
- A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
- A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
- A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
- A63B71/0622—Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physical Education & Sports Medicine (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses an interactive intelligent fitness mirror device, wherein a processor is also used for acquiring a motion track of a user action according to user action data acquired by a sensor and controlling a suspension icon to be displayed on a user interface according to the motion track, and the suspension icon is used for indicating the position indicated by the user action in the user interface. According to the invention, the motion trail of the user action is obtained through the user action data, and the suspension icon is controlled to be displayed on the user interface according to the motion trail, so that the accuracy rate of the user action in the motion process is ensured, and the use efficiency, the smoothness and the comfort are improved.
Description
Technical Field
The invention relates to the field of intelligent home and intelligent fitness, in particular to an interactive intelligent fitness mirror device.
Background
With the rapid development of economy, the pace of life of people is accelerated, people in urban life gradually have various health problems due to high-speed life and high-load work, and accordingly body building becomes a hot spot of people's attention. An intelligent body-building mirror is a novel body-building product integrating artificial intelligence and hardware content service.
At present, most of body-building personnel at home watch videos through the intelligent body-building mirror to learn and train actions, and the intelligent body-building mirror can be used as a mirror and can also see coach images with high image quality. The existing intelligent fitness mirror is troublesome in interaction mode, generally needs to be controlled through external intelligent equipment (such as a smart phone) or a touch screen, is relatively complex to operate when being controlled through the external intelligent equipment, needs to be distributed with the intelligent equipment and the intelligent fitness mirror, and is easy to leave fingerprints and sweat on the mirror surface by adopting touch control, so that the display effect of the screen is influenced. Meanwhile, the existing fitness mirror is large in size and occupies much family space, but the existing intelligent fitness mirror can only display fitness content and fitness related data, and the use frequency in daily life is low. Based on this, the inventor optimizes the interaction mode of the intelligent fitness mirror and the user by adding a personalized interface and a new interaction mode on the intelligent fitness mirror, and improves the use frequency of the intelligent fitness mirror in daily life.
With the development of the invention, the inventor finds that, when a user action controls a user interface, for example, when entering or selecting or switching in a control instruction menu, especially when selecting the control instruction menu, the user cannot intuitively acquire the position reached by the user action, and then a corresponding control instruction may appear without being triggered when the user action is made, so that the selection accuracy is reduced, and the use comfort and the fluency of the user are affected.
Disclosure of Invention
The invention aims to provide an interactive intelligent fitness mirror device, which displays a suspension icon through a motion track of user action when a user makes the action, so that the selection accuracy and the use comfort of the user are improved.
The purpose is realized by adopting the following technical scheme:
one example of a smart fitness mirror includes a communication interface for receiving video imagery of a fitness trainer, a display operatively coupled to the communication interface to display the video imagery of the fitness trainer, and a mirror disposed in front of the display to reflect an image of a person opposite the display. The mirror is for displaying a mirror image of a user on the mirror, and the display displays the user interface to the user through the mirror.
One example of an interactive fitness method comprises the following steps: (1) Streaming fitness content to an interactive video system, the interactive video system including a mirror having a partially reflective portion and a display disposed on one side of the partially reflective portion; (2) Displaying fitness content to a user via the display and the partially reflective portion of the mirror; (3) And reflecting the image of the user with the mirror such that the image of the user is at least partially superimposed on the fitness content displayed via the display and the partially reflective portion of the mirror.
In order to add a personalized interface and a new interaction mode on the intelligent body-building mirror, optimize the interaction mode of the intelligent body-building mirror and a user and improve the use frequency of the intelligent body-building mirror in daily life, the invention designs a set of operating system Launcher for the intelligent body-building mirror, and the method specifically comprises the following steps: (1) a GUI interface; (2) a system control method of motion control; (3) a system control method of voice control; (4) a control method of multimodal fusion; (5) executable control instructions; (6) a method for data acquisition management; and (7) some specific function management methods. A set of system and an operation method are customized for the intelligent fitness mirror, so that the intelligent fitness mirror has the functions of not only watching fitness and exercise videos, but also being used as an important component of intelligent home, a user can conveniently check more information, and the use efficiency is improved. Meanwhile, the customized operation method overcomes the defect that the traditional intelligent fitness mirror is easy to leave fingerprints on a partial reflecting mirror surface by touch control, and as the core function of the intelligent fitness mirror is to enable a user to watch the action of contrasting own mirror image when watching a fitness exercise video, the influence on the functions of the intelligent fitness mirror after leaving the fingerprints on the partial reflecting mirror surface is large, most users are reluctant to use touch interaction, and the intelligent terminal is used to control the intelligent fitness mirror to be unintuitive and troublesome. The method for controlling the movement is free from the problems, gesture control is also applied to other electronic products, but the gesture control is also applied to the intelligent fitness mirror, and breakthrough progress is achieved when the gesture control is applied to the intelligent fitness mirror.
However, the invention judges whether a control instruction is generated or not through the user action data, the action amplitude and the sense of each person are different, when the user performs the action on the body-building mirror, the action cannot be ensured to reach the interaction area, and the required interaction interface is selected. And then audio-visually make the user know whether the user action reaches the required position, ensure the accuracy rate of the user action, improve the smoothness and comfort level of use.
Preferably, when the processor judges that the action indicated by the user action data acquired by the sensor matches a preset suspended icon awakening action corresponding to suspended icon awakening interaction, the suspended icon is started to move along with the action of the user. The user may enable or disable the hover icon as desired.
Preferably, when the processor determines that the user has completed the selection operation on the current user interface, the processor turns off the floating icon function so that the floating icon does not move along with the user's action any more. Wherein the selection operation comprises any one of gesture stopping, gesture grabbing, forward palm pushing, backward palm pushing, hand swinging and fist making.
Preferably, when the processor determines that the floating icon moves to the position of the user interface object in the user interface according to the user motion data and determines that the user performs the selection operation at the position of the user interface object, the processor executes an operation corresponding to the user interface object.
All combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) may be contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein. Terms explicitly employed herein and which may also appear in any disclosure incorporated by reference should be given the most consistent meaning to the particular concepts disclosed herein.
One or more technical schemes provided by the invention at least have the following technical effects or advantages: according to the method and the device, the motion trail of the user motion is obtained through the user motion data, the floating icon is controlled to be displayed on the user interface according to the motion trail, and the floating icon is used for indicating the position indicated by the user motion in the user interface. The accuracy of the user action in the movement process is guaranteed, and the use efficiency, the smoothness and the comfort degree are improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:
FIG. 1 is a block diagram of an exemplary intelligent fitness mirror;
FIG. 2 is an exemplary GUI page on the smart fitness mirror;
FIG. 3 is an exemplary GUI page on the smart fitness mirror;
FIG. 4 is an exemplary GUI page on the intelligent fitness mirror;
FIG. 5-a is an exemplary GUI page on an intelligent fitness mirror;
5-b are exemplary GUI pages on an intelligent fitness mirror;
FIG. 6 is an exemplary GUI page on the smart fitness mirror;
FIG. 7 is an exemplary GUI page on the intelligent fitness mirror;
FIG. 8 is an exemplary GUI page on the smart fitness mirror;
FIG. 9 is an exemplary GUI page on the smart exercise mirror;
fig. 10 is a schematic view of motion recognition of the intelligent fitness mirror.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present invention may be combined with each other without conflicting with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described and thus the scope of the present invention is not limited by the specific embodiments disclosed below.
It will be understood by those skilled in the art that in the present disclosure, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are used in an orientation or positional relationship indicated in the drawings for ease of description and simplicity of description, and do not indicate or imply that the referenced devices or components must be constructed and operated in a particular orientation and thus are not to be considered limiting.
Accordingly, this patent relates to an interactive intelligent fitness mirror device (also referred to as "intelligent fitness mirror" and "interactive training system") and methods of using an interactive training apparatus. The smart fitness scope includes a display configured to display exercise content (pre-recorded video or live stream) and an interface that enables a user to personalize the exercise. In addition, smart gyms may allow a user and/or a trainer to interact with each other during a workout in a manner similar to a regular workout in a gym or mini-gym where the user and trainer are in the same room (e.g., providing feedback to the trainer regarding the pace of the workout, correcting for the user's pattern during a particular fitness program).
An exemplary intelligent fitness mirror.
An image recognition sensor (e.g., a camera) in the intelligent fitness scope may be used to acquire video and/or still images of the user while the user is performing an activity (e.g., an exercise). The video of the user may then be shared with the trainer to allow the trainer to observe and provide coaching to the user during the workout. The video may also be shared with other users of other intelligent fitness mirrors for comparison or competition. The user's video may also be displayed in real-time on the display 120 or stored for later playback. For example, by providing a visual comparison of the user with a trainer, the user's video can be used for self-assessment during or after exercise. The stored video may also allow the user to assess their progress or improvement over time while performing similar exercises.
The sensor may be configured to acquire a dynamic and/or static image of the user while the user performs an activity (e.g., exercise), and determine whether the user's motion matches a standard motion based on the dynamic and/or static image and output a determination result, and if the user's motion does not match the standard motion, output motion correction information.
Meanwhile, the intelligent body-building mirror displays a page through the display 120, when a user uses the intelligent body-building mirror, a page object selected by the user is determined according to the position of the right hand of the user, when interaction is performed, the right hand of the user makes an action instruction, skeleton point coordinates of the user are generated according to user action data, coordinates corresponding to a suspended icon on a user interface are obtained according to the skeleton point coordinates, a motion track corresponding to the coordinates is called, the suspended icon is controlled to be displayed on the user interface according to the motion track, and the suspended icon is used for indicating the position indicated by the action of the user in the user interface. Therefore, when the user makes an action instruction, the floating icon is displayed on the user interface along with the track of the action of the user, so that the user can intuitively know the position of the corresponding user interface after the action is made, and the selection accuracy is improved.
In one or more embodiments, the floating icon can be any shape including, but not limited to, spherical, triangular, star-shaped. The floating icon may be any color including, but not limited to, black, gray, white.
Specifically, the on or off of the floating icon function may be determined by a specific Graphical User Interface (GUI) or exercise page, such as default on the main page of the GUI and default off on other pages; the on or off of the floating icon function can also be determined according to data detected by other sensors.
The specific intelligent fitness mirror can control functions of the suspended icon according to user actions acquired by the image recognition sensor, and the suspended icon awakening page is used for starting the suspended icon when user action data acquired by the image recognition sensor is matched with preset suspended icon awakening actions. The preset suspension icon awakening action can be a preset action in the intelligent fitness mirror or a user recorded action. The hover icon wake action includes, but is not limited to, bimanual held-head-to-heart and various static gestures, with a duration typically in the range of 0-5 seconds or longer.
A specific intelligent fitness mirror can control the suspended icon function according to the voice collected by the voice recognition sensor, and the suspended icon function is turned on or off through a wakeup word.
A specific intelligent fitness mirror can control the function of the suspended icon according to the touch sensor, and the intelligent fitness mirror can control the function of the suspended icon according to a user touch instruction acquired by the touch screen.
Upon waking, the appearance position of the floating icon includes, but is not limited to, appearance from above the page, appearance below the page, appearance on the left of the page, appearance on the right of the page, and appearance in the middle of the page.
The floating icon may appear in a manner including, but not limited to, sliding out, popping up, fading out.
The specific example that the intelligent fitness mirror can control the function of the floating icon according to the touch sensor can be that after the processor judges that the user completes selection operation on the current user interface, the processor closes the function of the floating icon to enable the floating icon not to move along with the action of the user any more. And when the suspended icon needs to be used again, awakening the suspended icon through the preset suspended icon awakening action corresponding to the suspended icon awakening interaction. In one or more embodiments, the selection operation includes any one of gesture stop, gesture grab, push palm forward, push palm backward, hand swing, and fist making.
Coordinates of bone points
In one or more embodiments, generating coordinates of a bone point of a user and acquiring coordinates corresponding to a floating icon on a user interface includes multiple ways, and in this embodiment, a vertex bone point, a pelvic bone point, a left-hand bone point, and a right-hand bone point may be acquired according to user action data; acquiring a linear distance between a head top skeleton point and a pelvis skeleton point, and acquiring the unit length of a coordinate system according to the linear distance; acquiring a linear distance between a vertex bone point and a pelvis bone point, and taking one sixth of the linear distance as a unit length of a coordinate system, namely the unit length 1 of the coordinate system is one sixth of the linear distance; taking a straight line between the vertex bone point and the pelvis bone point as a longitudinal coordinate, taking the pelvis bone point as an origin point, and constructing a rectangular coordinate system according to the acquired unit length; and establishing a plurality of interaction areas according to the coordinate system, wherein the interaction areas corresponding to the pages are the interaction areas established by the coordinate system.
The actions may be provided with personal preferences (left-right handed, disabled, etc.) while having different modes with one hand and two hands, including left/right hand mode, mode with both hands simultaneously (e.g. left hand selection, right hand screen cut).
In addition, the control action corresponds to the control instruction, and if one control action is matched, the processor executes the control instruction matched with the control action.
Page action effect
The Graphical User Interface (GUI) also includes page animation effects including, but not limited to, page switch animation effects and page interaction animation effects. The page switching effect mainly aims at the page operation and change effect, and aims to enable the page operation and change to have better user experience when being displayed. The page interaction effect mainly aims at providing guidance for the user when the user interacts, such as prompting the current interaction and interaction progress of the user and prompting the user to perform required operation when the user interacts. Further details include, but are not limited to, highlighting or flashing the page information involved when the user interacts, displaying a progress bar on the page information involved, and the like. The method also comprises the steps of identifying the action which is performed by the user and displaying a prompt action or a suggested action and the like when the action control is performed. And when voice control is performed, completion prompting is performed according to a voice instruction of a user.
Further, a specific example of controlling the suspension motion of the intelligent fitness mirror is that the intelligent fitness mirror displays a third main page as shown in fig. 5-a through the display 120, the third main page comprises page objects which are arranged and displayed in parallel in four lattices, the head of a user is held for 5 seconds by holding the two hands of the user over the head, the suspension ball slides out from the upper side of the third main page, when the user uses the intelligent fitness mirror, the page object selected by the user is determined according to the position of the right hand of the user, that is, when the right hand of the user is positioned above the right shoulder of the human body, the "weather" of the page object is selected, and at this time, the suspension icon appears in the "weather" area along with the right hand of the user; when the right hand of the user is positioned below the shoulder of the right side of the human body, selecting a page object 'reminding', wherein the floating icon appears in a 'reminding' area along with the right hand of the user; when the right hand of the user is positioned above the shoulder of the left side of the human body, selecting a page object 'music', wherein the floating icon appears in a 'music' area along with the right hand of the user; when the right hand of the user is below the shoulder of the right side of the human body, the page object 'body fat scale' is selected, and at the moment, the floating icon appears in the 'body fat scale' area along with the right hand of the user.
Sensor with a sensor element
In one or more embodiments, the sensor may also be an image recognition sensor (camera) which acquires a dynamic and/or static image of the user, processes the acquired dynamic and/or static image of the user into a skeleton point image, determines whether the user action matches the action displayed by a standard action or a video image of an exercise interface according to the skeleton point image, and outputs the determination result, and may also process the dynamic and/or static image in real time while the user uses the intelligent fitness mirror or after the exercise is completed, so as to control the intelligent fitness mirror or derive biometric data of the user based on the movement and the action of the user.
The sensor can also be an infrared recognition sensor in the intelligent fitness mirror, the infrared recognition sensor can be used for acquiring a thermal imaging graph of a user when the user performs activities (such as exercises), judging the exercise part, the force exerting part and the consumed heat of the user during movement according to the thermal imaging graph, judging whether the user action is matched with the standard action or not according to the exercise part and the force exerting part of the user during movement, and outputting a judgment result, if the user action is not matched with the standard action, the action correction information is output
The sensor may also be a voice recognition sensor (microphone) in the intelligent fitness mirror, and the voice recognition sensor (microphone) may be used to acquire real-time processing voice information of the user when the user uses the intelligent fitness mirror, so as to control the intelligent fitness mirror or have a conversation with the user based on the voice information of the user.
An exemplary intelligent fitness mirror.
The user may directly interface with the intelligent fitness scope to control the intelligent fitness scope (e.g., motion commands, voice commands, touch commands, etc.). A Graphical User Interface (GUI) displayed on the display 120 may be provided to facilitate user interaction with the intelligent exercise mirror.
Action command control (including 'style control' and 'gesture control')
The intelligent fitness mirror can control a Graphical User Interface (GUI) according to user actions acquired by the image recognition sensor.
In particular, actions controlled by action commands include, but are not limited to, static actions and dynamic actions. Static motion refers to a user making a specific body part into a specific posture and maintaining for a specified time; a dynamic motion refers to a user making a particular motion with a particular body part.
Specifically, the static motion includes, but is not limited to, lifting left/right hands, lifting left/right legs, lifting the head of the user with both hands, various static gestures, and the like, and the range of the predetermined time to be maintained is generally 0 to 5 seconds or longer, and any motion determined by the positional relationship between a specific part of the human body with respect to the space, the whole human body, or other specific parts and having the predetermined time to be maintained should fall within the protection range of the static motion. The method is further concretely as follows: the left hand is straightened to form an angle of 15 degrees/30 degrees with the horizontal plane and is maintained for 1.5 seconds, the left hand is kept at a right angle with the right hand for 1 second, the left hand is extended forwards for 0.1 second and the like.
Specifically, the dynamic motion includes, but is not limited to, horizontal sliding left/right hand, up-down sliding left/right hand, double-hand clapping, jumping, squatting, various dynamic gestures, and the like, and any motion determined by a movement trajectory of a specific part of a human body with respect to a space, the whole human body, or other specific parts should fall within the protection range of the dynamic motion. Further concrete examples are: sliding the left hand horizontally moves the left hand from left to right, sliding the left hand horizontally moves the left hand from right to left, sliding the right hand horizontally moves the right hand from right to left, sliding the left hand up and down moves the left hand from above the shoulders to below the waist, sliding the left hand up and down moves the left hand from above 20% to below 40% of the entire human body, having both hands at the top of the head, having both hands at the left shoulders, and so on.
In particular, the control instructions executed by the action command control include, but are not limited to, control instructions for non-workout pages (wake page, main page, other pages) and control instructions for workout pages. The control instructions for the non-workout page primarily relate to user interaction instructions with the non-workout page. The control instruction for the exercise page mainly relates to the control instruction for playing the exercise course by the user.
Specifically, the control instruction for the non-exercise page includes, but is not limited to, switching of the non-exercise page, previous page, next page, determining, returning, selecting a course, collecting a course, joining a plan, entering a page switching interface, waking up a voice assistant, controlling multimedia volume, and the like.
Specifically, the control instruction for the exercise page includes, but is not limited to, pause of the course, play of the course, previous link, next link, background music volume control of the course, volume control of a coach in the course, evaluation of the course, and the like.
Specifically, the user may also implement different functions through a combination of a plurality of actions. Specific examples include, but are not limited to, using a first action to enter a first level control command menu, and then using a second action to make a command selection in the first level control command menu or to enter a second level control command menu, which may be presented to the user on a display. Further specifically, but not limited to, the first level control instruction menu may be a course selection control instruction menu, including instructions for playing a course, collecting a course, displaying details of the course, making a course appointment, and entering a second level control instruction menu, and the second level control instruction menu may be a course evaluation menu, including good-rated, medium-rated, and bad-rated instructions.
In particular, the method further comprises prompting the user action command control on a Graphical User Interface (GUI). Specific examples include, but are not limited to, displaying an icon of an actionable action on a Graphical User Interface (GUI), marking or highlighting a control command pointed by a user static action on the Graphical User Interface (GUI), displaying a progress bar for maintaining the user static action on the Graphical User Interface (GUI), guiding a user dynamic action on the Graphical User Interface (GUI) when the user dynamic action is completed by more than a specified ratio, and the like.
Specifically, the intelligent fitness mirror can provide auxiliary judgment by depending on other sensors when controlling a Graphical User Interface (GUI) according to user actions acquired by the image recognition sensor. Specific examples include, but are not limited to, turning on/interrupting/confirming motion command control by a voice recognition sensor, judging the front or back of a user by a facial recognition sensor (turning on motion command control only when the user is facing the front of the intelligent fitness scope), distinguishing different users by a facial recognition sensor to load custom motion command control instructions, optimizing motion/trajectory recognition accuracy by an IMU, etc.
Specifically, the motion controlled by the motion command may be a preset motion in the intelligent exercise mirror, or may be a motion recorded by the user. The control instruction executed by the action command control can be an instruction preset in the intelligent fitness mirror or an automatic instruction set by the user. The action controlled by the action command and the executed control command may be a preset corresponding relation, or a corresponding relation set by the user himself, or a one-to-one relation, or a many-to-one relation.
Specifically, the turning on or off of the motion command control function may be determined by a specific Graphical User Interface (GUI) or exercise page, such as turning on the motion command control function by default on a main page of the GUI and turning off the motion command control function by default on other pages; the opening or closing of the action command control function can be determined according to data detected by other sensors; the activation or deactivation of the motion command control function may also be the activation/deactivation of the entire function or the activation/deactivation of a part of the function.
A specific example of the intelligent fitness mirror capable of controlling a Graphical User Interface (GUI) according to a user action acquired by an image recognition sensor is that the intelligent fitness mirror displays a first page of the GUI through a display 120, a first prompt icon for lifting a left hand to enter a page switching mode is displayed on the first page, the left hand is lifted by a user, the first prompt icon enters a highlight state and opens a progress bar, 1.5 seconds later, the progress bar is completed, the intelligent fitness mirror displays a page switching page for switching page display through the display 120, the switching page includes a first page highlighted, a second page of the first page and a third page of a next page of the first page, still show last page on the switching page, next page, the suggestion icon of withdraw from and affirming, wherein the suggestion icon of last page represents the action that the left hand turned right horizontal wave from a left side, the suggestion icon of next page represents the action that the right hand turned left horizontal wave from the right side, the suggestion icon of withdraw from represents and lifts the left hand, the right hand is lifted to definite suggestion icon representation, the user makes this moment the left hand and turns right horizontal wave from a left side, the switching page becomes the highlight second page, the user lifts the right hand this moment, the icon that represents the affirmation gets into high bright state and opens the progress bar, 1.5 seconds later, the progress bar is accomplished, intelligent body-building mirror passes through display 120 and shows the second page. The first page, the second page, the third page, the fourth page, and the fifth page may be a wake-up page, a main page, a workout page, or other pages, respectively.
A specific example of the intelligent fitness mirror controlling a Graphical User Interface (GUI) according to a user action acquired by an image recognition sensor is that the intelligent fitness mirror displays a fourth page of the GUI through a display 120, the fourth page displays a second prompt icon representing that a left hand swings horizontally from left to right to enter a previous page and a third prompt icon representing that a right hand swings horizontally from right to left to enter a next page, the user makes a right-hand swing horizontally from right to left at the moment, and the intelligent fitness mirror displays a transition from the fourth page to a fifth page through the display 120.
A specific example of the intelligent body-building mirror capable of controlling a Graphical User Interface (GUI) according to a user action collected by an image recognition sensor is that the intelligent body-building mirror displays a third main page as shown in fig. 5-a through a display 120, the third main page includes a plurality of page objects displayed in parallel, the third main page further displays a selected page object and an interactive object entering the selected page object, and an interactive object switching the page object is further displayed below the third main page, and a user can switch the displayed page object through interaction of the interactive object switching the page object and enter a page corresponding to the selected page object by performing interaction of the interactive object entering the selected page object. The user can select the selected page object by lifting the right hand/left hand obliquely upwards to the left, obliquely downwards to the left, obliquely upwards to the right and obliquely downwards to the right, maintain the posture of the right hand/left hand after the selection is finished, and simultaneously make the hand make a specified static motion, such as opening the five fingers for 1 second, so that the page corresponding to the selected page object can be entered. Meanwhile, the user can make a horizontal waving motion from left to right of the right hand to switch the displayed page object and enter a fourth main page.
Voice command control
The intelligent fitness mirror can control a Graphical User Interface (GUI) according to user speech acquired by the speech recognition sensor.
Specifically, the user can replace the action controlled by the action command through the voice recognition result controlled by the voice command and realize the same control effect.
In particular, the control instructions executed by the action command control include, but are not limited to, control instructions for non-workout pages (wake page, main page, other pages) and control instructions for workout pages. The control instructions for the non-workout page primarily relate to user interaction instructions with the non-workout page. The control instruction for the exercise page mainly relates to the control instruction for the user to play the exercise course
Specifically, the control instruction for the non-exercise page includes, but is not limited to, switching of the non-exercise page, previous page, next page, determining, returning, selecting a course, collecting a course, joining a plan, entering a page switching interface, waking up a voice assistant, controlling multimedia volume, and the like.
Specifically, the control instructions for the exercise page include, but are not limited to, pause of a course, play of a course, previous link, next link, background music volume control of a course, volume control of a course coach, evaluation of a course, and the like.
Specifically, the user may also implement different functions through a combination of a plurality of actions. Specific examples include, but are not limited to, using a first voice to enter a first level control command menu, and then using a second voice to make a command selection in the first level control command menu or to enter a second level control command menu, which may be presented to the user on a display. Further, the specific examples include, but are not limited to, the first level control command menu may be a course selection control command menu, including a command for playing a course, collecting a course, displaying details of the course, reserving a course, and entering the second level control command menu, which may be a course evaluation menu, including a good comment command, a medium comment command, and a bad comment command.
In particular, prompting for user voice command control in a Graphical User Interface (GUI) is also included. Specific examples include, but are not limited to, displaying keywords of usable voices on a Graphical User Interface (GUI), completing a user voice command on the Graphical User Interface (GUI) (when the user's dynamic motion completion degree exceeds a predetermined ratio), and the like.
Specifically, the intelligent fitness mirror can provide auxiliary judgment by depending on other sensors when controlling a Graphical User Interface (GUI) according to user voice collected by the voice recognition sensor. Specific examples include, but are not limited to, turning on/interrupting/confirming motion command control by a motion recognition sensor, judging the front or back of a user by a facial recognition sensor (turning on motion command control only when the user is facing the front of the intelligent fitness scope), distinguishing different users by a facial recognition sensor to load customized voice command control instructions, etc.
Specifically, the keywords controlled by the voice command may be preset keywords in the intelligent fitness mirror, or may be keywords recorded by the user. The control instruction controlled and executed by the voice command can be an instruction preset in the intelligent fitness mirror or an automatic instruction set by the user. The keywords controlled by the voice command and the executed control instruction may be preset corresponding relations, or may be corresponding relations set by the user, or may be one-to-one relations, or may be many-to-one relations.
Specifically, the voice command control function may be turned on or off according to a specific Graphical User Interface (GUI) or exercise page, such as turning on the voice command control function by default on a main page of the GUI and turning off the action command control function by default on other pages; the voice command control function can be started by identifying a wake-up word; the on or off of the voice command control function can be determined according to data detected by other sensors; the turning on or off of the voice command control function may also be turning on/off of the entire function or turning on/off of part of the function.
A specific example of the intelligent fitness scope that can control a Graphical User Interface (GUI) according to a user action acquired by an image recognition sensor is that the intelligent fitness scope first displays a first page of the GUI through a display 120, the user turns on a voice command control function through a wakeup word, a fourth page displays a turn-on icon representing the voice command control function, the user speaks an instruction to "enter a switch page" or "page switch" at this time, the intelligent fitness scope displays a page switch page for switching page display through the display 120, the switch page includes, in addition to the highlighted first page, a first "second page" of the first page and a second "third page" of the first page, the switch page also displays a turn-on icon representing the voice command control function, the user speaks an instruction to "go to" or "turn forward" at this time, the switch page becomes a highlighted second page, the user speaks an instruction to "enter" or "determine" at this time, and the second page of the intelligent fitness scope is displayed through the display 120.
A specific example of the intelligent fitness scope that can control a Graphical User Interface (GUI) according to user actions acquired by an image recognition sensor is that the intelligent fitness scope first displays a fourth page of the GUI through a display 120, a user turns on a voice command control function through a wakeup word, a turn-on icon representing the voice command control function is displayed on the fourth page, the user speaks a "next page" or "turn back one page" instruction at the time, and the intelligent fitness scope turns over from the fourth page to a fifth page through the display 120.
A specific example of the intelligent fitness mirror capable of controlling a Graphical User Interface (GUI) according to a user action acquired by an image recognition sensor is that the intelligent fitness mirror displays a sixth page of the GUI through a display 120, the user controls to start a voice command control function through an action command, a start icon representing the voice command control function is displayed on a fourth page, the user speaks a command of "start training" or "confirm" at the moment, and the intelligent fitness mirror is displayed through the display 120 to turn from the sixth page to a seventh page.
The first page, the second page, the third page, the fourth page and the fifth page can be a wakeup page, a main page, an exercise page or other pages respectively, the sixth page is a page for displaying exercise cards, and the seventh page is an exercise page.
Touch command control
The intelligent fitness mirror can control a Graphical User Interface (GUI) according to a user touch instruction collected by the touch screen.
Multi-modal command control
The multi-modal command control includes combination control, enhancement control, and conflict control.
The combination control includes but is not limited to action combination control of multiple actions and voice combination control of multiple keywords, and more complex control instructions (such as selecting and collecting, selecting and recommending to friends, screenshot and sharing, and the like) can be realized through multiple actions or keywords which are continuous (the interval time does not exceed a threshold).
The combined control comprises but is not limited to enhancing and optimizing action instruction control or voice instruction control of a user through facial recognition or scene recognition, and further comprises but is not limited to recognizing that facial expressions of the user are not happy when the action instruction control or voice instruction control of the user is music playing, presenting a recommendation result of relieving mood for the user and caring for the user; when the action instruction control or voice instruction control of the user is the recommended course, it is recognized that the arrival field Jing Wendu is low, and the recommendation of the meditation course is reduced.
The conflict control includes, but is not limited to, when the user uses a plurality of different control methods to control the intelligent fitness mirror at the same time, a plurality of conflicting control instructions occur, and the control instructions which the user wants to execute are determined and executed by the intelligent fitness mirror by performing priority judgment on the different control methods.
Control instruction
The control instructions include, but are not limited to, system wake-up instructions, page switching instructions, page interaction instructions, and course control instructions, and the user can control the intelligent fitness mirror through the control instructions.
The system awakening instruction comprises an instruction for awakening the intelligent fitness mirror from a standby state to an awakening page, a main page, an exercise page and other pages and an instruction for switching the intelligent fitness mirror from the awakening page to the main page, the exercise page and other pages.
The page switching instruction is used for controlling the intelligent fitness mirror to switch among a plurality of different pages, and the different pages can be pages of different types or pages with different contents. Further page switching instructions include, but are not limited to, page turn instructions (e.g., next/previous page) and instructions to control the intelligent gym mirror to enter a page switch page.
The page interaction instruction refers to an instruction for controlling the intelligent fitness mirror to perform interaction or input, and the page interaction instruction comprises but is not limited to a control interaction instruction and an input interaction instruction. Further, the control interactive instruction refers to an instruction for controlling a Graphical User Interface (GUI) or a page object or an interactive object in the GUI to implement an interactive function thereof, such as confirmation/return, favorite joining, plan joining, voice assistant wake-up, volume control, account switching, mode switching, context menu (contextual menu) opening, and the like. Further, the input interactive instruction refers to an instruction for inputting information in a Graphical User Interface (GUI) or a page object or an interactive object in the GUI, for example, when course evaluation is performed, one core is added every time an instruction is input; or when chatting with other users through the intelligent fitness mirror, the preset expression/reply is switched every time an instruction is input
The course control instruction is used for controlling the intelligent fitness mirror to play the exercise video on the exercise page, and is mainly used for controlling the playing of the exercise video. Further course control instructions include, but are not limited to, pause/play, previous/next, background volume control, coach volume control, course rating, etc.
Data management
Data sources for the intelligent fitness mirror include, but are not limited to, sensor data and background data for input of user control instructions and presentation of page information.
The sensor data includes, but is not limited to, on-mirror sensor data and external sensor data, and the on-mirror sensor data includes, but is not limited to, data detected by sensors mounted on the intelligent fitness mirror, such as an image recognition sensor, an infrared recognition sensor, and a voice recognition sensor. More specifically, the data detected by the image recognition sensor includes, but is not limited to, an image detected by the image recognition sensor and data obtained by processing and recognizing the image, such as user motion data, user posture data, facial recognition data, user expression data, environment image data, and the like. More specifically, the data detected by the infrared recognition sensor includes, but is not limited to, a thermal image detected by the infrared recognition sensor and data obtained by processing and recognizing the thermal image, such as user thermal imaging data, user exercise location data, environmental thermal imaging data, and the like. More specifically, the data detected by the speech recognition sensor includes, but is not limited to, speech data detected by the speech recognition sensor and data obtained by processing and recognizing the speech data, such as a speech control command, a keyword command, a wakeup word command, and the like. The on-mirror sensor data includes, but is not limited to, inertial sensor data, physiological data sensor data (such as heart rate belt data, body fat scale data, etc.), and the like, which are detected by the sensor wirelessly connected with the intelligent fitness mirror. More specifically, the data detected by the inertial sensor includes, but is not limited to, speed/acceleration data detected by the inertial sensor and data obtained by processing and identifying the speed/acceleration data, such as user motion data. More specifically, the data detected by the physiological data sensor includes, but is not limited to, physiological data detected by the physiological data sensor and data obtained by processing and identifying the physiological data, such as heart rate data, blood pressure data, blood oxygen saturation data, and the like.
The background data comprises but is not limited to background user data and other background data, the background user data is data which is stored by the intelligent fitness mirror and/or a server connected with the intelligent fitness mirror and is directly related to a user, and the background user data comprises but is not limited to user physiological data, user social data, user calendar data, use history data and the like. The usage history data includes, but is not limited to, completed course data, collected course data, historical best performance data, physiological data and performance data during the course of the course, frequency data of class attendance, etc. The user physiological data includes, but is not limited to, the user's height, weight, gender, age, heart rate, blood pressure, medical history of injury, and the like. The user social data includes, but is not limited to, data related to people/friends concerned by the user and data related to groups/interest groups joined by the user, and these information may specifically be class invitation data for inviting the user to complete a specified course, challenge invitation data for inviting the user to perform one-to-one or more people, ranking among users, and dynamic data uploaded by the user himself or by other users. The dynamics may be synchronized to other social platforms (e.g., weChat, microblog, facebook, twitter, etc.). The user calendar data includes, but is not limited to, schedule information data, to-do data, lesson planning data, etc. of the user. The other data of the background is data which is stored by the intelligent body-building mirror and/or a server connected with the intelligent body-building mirror and has low relevance with the user, and the other data of the background comprises but is not limited to IP data, environment data, course data, third-party data and the like. Wherein the IP data includes, but is not limited to, IP address data, mac address data, etc. of the intelligent fitness scope. The environment data comprises but is not limited to time data, positioning data, weather data, temperature data, dressing index data and the like of the area where the IP of the intelligent fitness mirror is located. The course data includes, but is not limited to, exercise video data, trainer name, course type, course length, course difficulty, course tags, and/or expected consumption, among others. The types of lessons include, but are not limited to, presenting HIIT to the user through the smart fitness mirror, aerobics, yoga, shaping of strength, fighting exercises, barre, pilates, stretching, dancing, meditation, challenges, and the like. The course labels include, but are not limited to, whether the courses are identified for AI or not, whether the courses are parent-child courses or not, body parts mainly related to the courses (such as whole body, arms, waist and abdomen, legs and the like), and properties (such as no instrument, yoga mat, elastic ring, elastic band, dumbbell and the like) required to be used in the courses. The third party data includes, but is not limited to, data for third party functions installed on the intelligent exercise mirror.
Data storage
The intelligent fitness scope may also store user information locally on the intelligent fitness scope and/or a remote storage device (e.g., a cloud service) depending on the amount of storage space used. For example, user information that uses little storage space, including but not limited to the user's name, age, height, weight, and gender, may be stored locally on the intelligent fitness mirror. In addition, lesson data may also be stored in the intelligent fitness mirror to reduce the impact of network latency that may affect the quality of the video stream. The amount of stored video content may be limited by the storage capacity of the intelligent fitness mirror. In some configurations, the video content may be temporarily stored only daily or weekly, or depending on the percentage of the intelligent fitness scope capacity used. Background user data using a large amount of storage space may be stored on the remote storage device, including but not limited to physiological data such as the user's heart rate and calories burned and video or skeletal point data of the user taken during exercise. The intelligent fitness scope may retrieve this information for subsequent analysis and display.
Data transmission between the intelligent exercise mirror and the remote storage device may be protected (e.g., encrypted) in various ways to prevent unwanted loss or theft of user information. For example, the bluetooth low energy protocol includes built-in security features that may be used by devices utilizing the protocol. However, these security features can only be used if the bluetooth binding step is completed before the connection is established with the encryption. In some cases, various security mechanisms may not be implemented or may fail, at which point application level security may be implemented in conjunction with the above-described chunking specification of data. For example, advanced Encryption Standard (AES) encryption of the message may be applied before the preamble of the preamble message. In some aspects, the bluetooth low energy protocol performs similar processes via built-in security features at the firmware level, and may provide similar protection to prevent people from reading communications between the client and the server.
The GATT service added for the client to read/notify messages may be deleted from the service recording on the server device when the client is disconnected from the server. This ensures that no connections are opened and the system does not accidentally leak information to nefarious snoopers. Such connection termination may be triggered by the server or client and rely on the bluetooth low energy stack to provide notification to both parties that the connection has been closed. If bluetooth binding is used in the initial connection setup to provide firmware level encryption security, binding information may be stored on each device so that repeated bindings are not required after subsequent connections between the client and the server.
Function management
The intelligent fitness mirror can also realize the following functions according to the received data: user identification, mode switching, attention judgment, body/emotion assessment, dressing recommendation.
The user identification function allows the intelligent body-building mirror to identify a user using the intelligent body-building mirror through input data, and the function is mainly realized through the following method: the user inputs the characteristic data of the user through the data input device, the intelligent fitness mirror compares the characteristic data input by the user with the characteristic data in the database, the user using the intelligent fitness mirror is determined according to the comparison result, and the intelligent fitness mirror is allowed to adjust a Graphical User Interface (GUI) according to the identification result, such as adjusting the font size, page information preference and the like. The data input device comprises on-mirror sensor data and an external sensor, the characteristic data of the user comprises on-mirror sensor data such as bone points, voiceprints, weight, resting heart rate and the like of the user and data obtained by processing after the data are collected or collected by the external sensor, and the mapping relation between the user and the characteristic data is recorded in the database. Further, the user identification function may be used for identification and switching of a plurality of sub-account numbers under one primary account number, and may also be used for identification and switching between a plurality of primary account numbers. In another example, the user identification function allows the intelligent body-building mirror to identify characteristics of a user who is using the intelligent body-building mirror, such as a male user, a female user, an old person, and a child, through input data, and allows the intelligent body-building mirror to adjust a Graphical User Interface (GUI), such as font size, page information, and the like, according to the identification result.
The mode switching function allows the intelligent fitness mirror to be switched among a plurality of working modes to adapt to different working scenes. For example, the intelligent fitness mirror is allowed to be switched to a teenager mode, in the teenager mode, a teenager account is bound with a parent account or a device primary account, in the teenager mode, page information displayed by the intelligent fitness mirror is adjusted adaptively, for example, course recommendation is adjusted to preferentially present contents suitable for teenagers, such as rope skipping, motion sensing games and the like, and meanwhile, real-time conditions and motion data in the mode can be synchronized to an intelligent terminal of the parent account or the device primary account. In another example, the intelligent fitness mirror is allowed to be switched to a visitor mode, the intelligent fitness mirror can work without being bound with an account in the visitor mode, page information displayed by the intelligent fitness mirror can be adjusted adaptively, if course recommendation is performed, the content suitable for novices can be adjusted to be presented preferentially, if the difficulty label is low, the content is the same, and meanwhile, even if the intelligent fitness mirror is bound with the account, motion data in the mode cannot be counted into the bound account, and a training plan or a class completion condition of the bound account cannot be influenced.
Wherein the attention determination allows the intelligent fitness mirror to determine whether the user's attention is on the intelligent fitness mirror while the device is operating, such that the intelligent fitness mirror determines whether the user's intent to interact with the mirror is simultaneously operable to activate or deactivate the sensor, other number input device, and function. The attention judgment is mainly to judge whether the user wants to use the intelligent fitness mirror according to the performance of the user, and can be judged by the following characteristics: distance judgment (the distance between the user and the intelligent fitness mirror does not exceed a threshold value), posture judgment (whether the user is facing the intelligent fitness mirror or not), action judgment (whether the user performs a specified action or an action commonly used for exercise), semantic judgment (whether a recognition result of the voice of the user is related to a specific keyword or not), and the like.
Wherein the body/emotion assessment allows the intelligent fitness scope to determine a body/emotion condition of the user when using the device, such that the intelligent fitness scope is operable to adjust page information or other feedback according to the body/emotion condition of the user. The physical/emotional condition of the user can be recognized by a physiological data sensor and an image recognition sensor (face recognition). When the emotion of the user is recognized to be inattentive, a page information recommendation result of the relaxing mood is presented for the user, and meanwhile the user is concerned during voice interaction; in a further example, when the user wakes up the voice function of the intelligent fitness mirror through the wake-up word, the intelligent fitness mirror recognizes that the expression of the user is depressed through the image recognition sensor (face recognition), and the intelligent fitness mirror initiates a first turn of dialog "hey to make what you look not too happy and to not listen to a song to relax. The body/emotion condition recognition can be used for distinguishing the states of body/emotion such as emotion, stress, fatigue and the like of the user by recognizing information such as age, body posture, expression and the like of the user; the identification can also be carried out by blood pressure and heart rate data detected by a physiological data sensor.
Wherein the dressing recommendation allows the intelligent fitness scope to initiate active interaction to the user based on specific conditions. The intelligent body-building mirror judges whether the user needs the intelligent body-building mirror to initiate active interaction or not according to the acquired time data and positioning data and by combining the user information and the action of the user in front of the intelligent body-building mirror. For example, when the user appears in front of the intelligent fitness mirror for the first time in the morning and turns a circle, the intelligent fitness mirror starts the dressing recommendation function, and clothes which are positioned at present and are suitable for being worn today are recommended to the user. Further, when the user appears in front of the intelligent fitness mirror and turns a circle at the end of the week, the intelligent fitness mirror starts the clothes-wearing recommendation function and outputs the clothes-wearing suggestion of the current clothes to the user. The functions can also be used for other functions of the intelligent body-building mirror for active interaction based on the user and external information.
FIG. 1 illustrates an exemplary representation of an intelligent fitness scope. The intelligent fitness mirror may include a processor 110 for controlling, in part, the operation of various subcomponents in the intelligent fitness mirror and managing the flow of data (e.g., video content, audio from a coach or user, biometric feedback analysis) into/out of the intelligent fitness mirror. The intelligent fitness mirror may include a display 120 for displaying video content, a Graphical User Interface (GUI) with which a user may interact and control the intelligent fitness mirror, biometric feedback data, and/or other visual content. The sensor 130 may be coupled to the processor 110 to collect user-related data. An antenna 140 may be coupled to the processor 110 to provide data transmission between the intelligent exercise mirror and another device (e.g., a remote control device, biometric sensor, wireless router). The antenna 140 may include multiple transmitters and receivers, each customized for a particular frequency and/or wireless standard (e.g., bluetooth, 802.11a, 802.11b, 802.11G, 802.11n, 802.11ac, 2G, 3G, 4G LTE, 5G). An amplifier 150 may be coupled to the processor 110 to receive audio signals from the processor 110 for outputting subsequent sounds through a left speaker 152 and/or a right speaker 154.
The intelligent exercise scope may also include additional components not shown in fig. 1. For example, the smart fitness mirror may include a Switched Mode Power Supply (SMPS), switches, and onboard memory and storage devices (non-volatile and/or volatile memory) including, but not limited to, a Hard Disk Drive (HDD), a solid state drive (SDD), flash memory, random Access Memory (RAM), and a Secure Digital (SD) card. The on-board memory and/or storage device may be used to store firmware and/or software for the operation of the intelligent exercise mirror. As described above, the onboard memory and/or storage may also be used to store (temporarily and/or permanently) other data including, but not limited to, video content, audio, user video, biometric feedback data, and user settings. In another example, the intelligent exercise scope may include various components to mount and support the intelligent exercise scope.
The antenna 140 may include multiple antennas, each of which acts as a receiver and/or transmitter to communicate with various external devices, such as a user's smart terminal (e.g., computer, smartphone, tablet), an external sensor (e.g., heart rate belt, inertial sensor, body fat scale), and/or a remote server or cloud server for streaming or playing video content. Again, the antenna 140 may conform to various wireless standards including, but not limited to, bluetooth, 802.11a, 802.11b, 802.11G, 802.11n, 802.11ac, 2G, 3G, 4G LTE, and 5G standards.
The sensor 130 may include a plurality of sensors, which may be one or more of an image recognition sensor (camera), an infrared recognition sensor, or a voice recognition sensor (microphone), respectively.
An image recognition sensor (camera) in the intelligent gym mirror can be used to acquire video and/or still images of the user while the user is performing an activity (e.g., exercising). The video of the user may then be shared with the trainer to allow the trainer to observe and provide coaching to the user during the workout. The video may also be shared with other users of other intelligent fitness mirrors for comparison or competition. The user's video may also be displayed in real-time on the display 120 or stored for later playback. For example, by providing a visual comparison of the user with a trainer, the user's video may be used for self-assessment during or after exercise. The stored video may also allow the user to assess their progress or improvement over time while performing similar exercises.
Graphic User Interface (GUI)
The intelligent exercise scope may display a Graphical User Interface (GUI) via the display 120 to facilitate user interaction with the intelligent exercise scope.
Graphical User Interfaces (GUIs) include, but are not limited to, a wake page, a home page, a workout page, and other pages, among others.
The awakening page is an initial page when the intelligent fitness mirror is awakened and is used for displaying some basic page information, and the intelligent fitness mirror is completely awakened to enter a main page, a training page or other pages only when the user is detected to perform specified operation.
Wherein the main page presents at least one item of page information for presenting data, actionable interactive objects or interactive guidance to a user. The main page is used for displaying preset page information or page information set by the user to the user, and meanwhile, the user can control the intelligent fitness mirror by performing specified operation on the main page, wherein the control includes but is not limited to page switching, function switching or entering an application program accessed by a third party.
The exercise page displays a video playing window for playing an exercise video to guide a user to exercise, and can also display at least one of course playing information (such as the total duration of the course, the played duration and the like), accessory connection information (such as heart rate belt connection information and the like) and user physiological information (such as heart rate, blood pressure, calories consumed and the like). The exercise page can enter from the main page or wake up the page and return to the main page or wake up the page, and can also directly enter when the intelligent fitness mirror is started.
Wherein, other pages show pre-installed function pages (such as multimedia player pages, weather display pages, diet recommendation pages, etc.) or application program pages accessed by third parties (such as WeChat, buffalo, himalayan, etc.) entered through the main page, and other pages generally enter from the main page or wake up pages and return to the main page or wake up pages.
Page information
The Graphical User Interface (GUI) may present page information through page objects, each page object being used for presenting at least one or a group of page information, the presented page information may be information stored in a storage device local to the intelligent exercise mirror, or may be information received by the intelligent exercise mirror from other devices through the communication interface, including but not limited to course preview information, user information, environment information, multimedia information, external device information, etc. which may be presented switchably.
The course preview information includes, but is not limited to, a trainer name, a course type, a course length, a course difficulty level, a course label, and/or a projected consumption, among others. The types of lessons include, but are not limited to, presenting HIIT to the user through the smart fitness mirror, aerobics, yoga, shaping of strength, fighting exercises, barre, pilates, stretching, dancing, meditation, challenges, and the like. The course labels include, but are not limited to, whether the courses are identified for AI or not, whether the courses are parent-child courses or not, body parts mainly related to the courses (such as whole body, arms, waist and abdomen, legs and the like), and properties (such as no instrument, yoga mat, elastic ring, elastic band, dumbbell and the like) required to be used in the courses. The page object displaying the course preview information is also called a course card.
The user information includes, but is not limited to, usage history information, physiological information, social information, calendar information, and the like. The usage history information includes, but is not limited to, completed courses, collected courses, historical best results, frequency of class attendance, etc. which are presented to the user through the intelligent fitness scope. The physiological information includes, but is not limited to, height, weight, sex, age, heart rate, blood pressure, injury history, etc. displayed to the user through the intelligent fitness scope. The social information includes, but is not limited to, information related to people/friends concerned by the user and displayed to the user through the intelligent fitness mirror, and may also be information related to a group/interest group to which the user joins, and the information may specifically be an invitation to invite the user to complete a lesson for a specified course, an invitation to invite the user to challenge one or more people, a ranking between users, and a dynamic uploading state of the user himself or another user, and the like. The dynamics may be synchronized to other social platforms (e.g., weChat, microblog, facebook, twitter, etc.). The calendar information includes, but is not limited to, schedule information, backlogs, lesson plans, etc. of the user displayed to the user through the intelligent fitness mirror. The page object showing the user information is also called a user card.
The environmental information includes, but is not limited to, weather information. The weather information includes, but is not limited to, weather forecast, temperature, humidity, dressing recommendation and the like displayed to the user through the intelligent fitness mirror. The page object for displaying the weather information is also called a weather card.
The multimedia information includes but is not limited to music and video information played through the intelligent fitness mirror. The page object for displaying multimedia information is also called a multimedia card.
The external device information includes, but is not limited to, information of other fitness devices connected with the intelligent fitness mirror through intelligent fitness mirror display, and information of other intelligent furniture (such as a sweeping robot, an intelligent sound box, an intelligent gateway and the like). The page object for displaying the external device information is also called a device card.
The Graphical User Interface (GUI) may also present page information via interactive objects, which may be icons, text, highlighted selections, other forms that may prompt the user for interaction, or a combination thereof. The interactive objects are used to present to the user controls or operations (e.g., page switches, selections, pausing/playing of courses, etc.) that the user may perform. Further, the interactive object may also be combined with a page or page object of a Graphical User Interface (GUI) to present different interactive functions.
The page object and the interactive object can be fixed on each page, and can also be displayed in a manner of popup or scrolling playing on each page.
Some exemplary pages
The first scheme is shown in fig. 2, 3, 4, 5-a and 5-b, wherein fig. 2 is a first main page, fig. 3 is a page switch page, and fig. 4, 5-a and 5-b are a second main page, a third main page and a fourth main page, respectively. The first main page displays a plurality of page information through a plurality of page objects, wherein the page information comprises a weather card, a multimedia card, a recommended course card and the like, meanwhile, the first main page also comprises an interactive object which is displayed at the upper right corner and used for entering a page switching page and an interactive object which is combined with the course card, a user can enter a second main page through executing the interaction of the interactive object which is combined with the course card and also can enter the page switching page through executing the interaction of the interactive object which is used for entering the page switching page, the page switching page comprises a preview of the first main page, the second main page and a preview of a third main page and highlights one of the pages, meanwhile, the interactive object which returns and confirms to enter the highlighted page is displayed above the page switching page and the interactive object which is switched to the highlighted page is displayed below the page. The user may switch the highlighted page by performing an interaction that switches the interactive object of the highlighted page and enter the selected page by confirming the interaction of the interactive object that entered the highlighted page. When the user selects and enters the second main page, the second main page comprises a plurality of curriculum cards and highlights one of the curriculum cards, meanwhile, an interactive object for returning and confirming the curriculum entering the highlighted curriculum cards is displayed above the second main page, and an interactive object for switching the highlighted curriculum cards is displayed below the page switching page. When a user selects and enters a third main page, the third main page comprises a plurality of page objects displayed in parallel, the third main page also shows the selected page objects and interaction objects entering the selected page objects, and interaction objects switching the page objects are also shown below the third main page.
The second scheme is shown in fig. 6, 7, 8 and 9, wherein fig. 6 is a fifth main page, fig. 7 is a sixth main page, and fig. 8 and 9 are a seventh main page and an eighth main page, respectively. The fifth main page displays a plurality of page information including weather cards, multimedia cards, recommended course cards and the like through a plurality of page objects, and meanwhile, the fifth main page further comprises an interactive object displayed below and used for switching the main page and an interactive object combined with the course cards, and a user can enter the eighth main page by executing interaction of the interactive object combined with the course cards and can also switch the displayed main page by executing interaction of the interactive object used for switching the main page. When the user selects the main page displayed in the switching way to be the sixth main page, the sixth main page comprises a plurality of page objects displayed in parallel, meanwhile, the sixth main page also shows the selected page object and the interaction object entering the selected page object, and the interaction object used for switching the main page is also shown below the sixth main page. When the user selects the main page to be switched and displayed as a seventh main page, the seventh main page comprises a plurality of page objects which are displayed in parallel, the seventh main page also shows the selected page object and an interactive object entering the selected page object, and an interactive object used for switching the main page is also shown below the seventh main page. When the user selects the page corresponding to the selected page object as the eighth home page, the eighth home page comprises a plurality of curriculum cards displayed in parallel, the eighth home page also shows the selected curriculum cards and the interactive objects entering the selected curriculum cards, and the interactive objects used for switching the curriculum cards are also shown below the eighth home page.
The page switching page comprises preview images of a first main page, a second main page and a third main page, one page is highlighted, meanwhile, an interactive object for returning and confirming to enter the highlighted page is displayed above the page switching page, and an interactive object for switching the highlighted page is displayed below the page switching page. The user may switch the highlighted page by performing an interaction that switches the interactive object of the highlighted page and enter the selected page by confirming the interaction of the interactive object that entered the highlighted page. When the user selects and enters the second main page, the second main page comprises a plurality of curriculum cards and highlights one of the curriculum cards, meanwhile, an interactive object for returning and confirming the curriculum entering the highlighted curriculum cards is displayed above the second main page, and an interactive object for switching the highlighted curriculum cards is displayed below the page switching page. When a user selects and enters a third main page, the third main page comprises a plurality of page objects displayed in parallel, the third main page also shows the selected page objects and interaction objects entering the selected page objects, and interaction objects switching the page objects are also shown below the third main page.
An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the multi-modal interactive exercise scope when executing the computer program.
The processor may be a central processing unit, or may be other general-purpose processor, a digital signal processor, an application specific integrated circuit, an off-the-shelf programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory can be used to store the computer programs and/or modules, and the processor can implement the various functions of the device of the multi-modal interactive exercise scope of the invention by running or executing the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, at least one magnetic disk storage device, a flash memory device, or other volatile solid state storage device.
Video streaming method
The intelligent fitness mirror is configured to display video content from the video production plant on the display surface 120. The video content may be streamed as live content or prerecorded recorded content. Live content may also be recorded and stored in the cloud server so that the user can later request and play video content, thus becoming recorded and played content. When video streaming is carried out, firstly, the intelligent fitness mirror sends a request of a specified course to the API server, data returned by the API server are divided into two parts, and the first part is overview of the specified course (including course pictures and course brief introduction); the second part is the address (URL) of the course video, and then the intelligent fitness mirror requests the corresponding course video on the OSS server according to the received URL. When the course is recorded and played, the intelligent body-building mirror plays videos by using an HLS protocol, namely, the address (URL) of the course video is an M3U8 file corresponding to the course, and then the player of the intelligent body-building mirror downloads corresponding course video slices from the OSS server according to the record of the M3U8 file. When the course is live broadcast content, the intelligent body-building mirror plays videos by using an RTMP protocol, namely, the address (URL) of the course video is the RTMP live broadcast address of the corresponding course, and then the player of the intelligent body-building mirror downloads the corresponding course video live broadcast data from the OSS server according to the RTMP live broadcast address.
A computer readable storage medium storing a computer program which when executed by a processor implements the steps of the multi-modal interactive exercise mirror.
Computer storage media for embodiments of the present invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A method for recognizing an action of a user,
as shown in fig. 10, on the basis of the above embodiment, the apparatus specifically includes the following steps in the interaction process:
s121, acquiring a user image in an area by the body-building mirror to obtain a motion image of the user;
s122, generating skeleton point information of the user according to the action image, and establishing a coordinate system according to the skeleton point information;
s122.1, the skeleton point information comprises a vertex skeleton point, a pelvis skeleton point, a left-hand skeleton point and a right-hand skeleton point;
s122.2, acquiring a linear distance between a vertex bone point and a pelvis bone point, and acquiring the unit length of a coordinate system according to the linear distance;
s122.21 acquiring a linear distance between a head bone point and a pelvis bone point, and taking one sixth of the linear distance as a unit length of a coordinate system, namely the unit length 1 of the coordinate system is one sixth of the linear distance;
s122.3, taking a straight line between the vertex bone point and the pelvis bone point as a longitudinal coordinate, taking the pelvis bone point as an origin point, and constructing a rectangular coordinate system according to the obtained unit length;
s123, establishing a plurality of interaction areas according to a coordinate system;
in this embodiment, there are five interactive areas, which are respectively a first interactive area, a second interactive area, a third interactive area, a fourth interactive area and a fifth interactive area, as shown in fig. 4, where the first interactive area is an area a in fig. 3, the second interactive area is an area B in fig. 4, the third interactive area is an area C in fig. 4, the fourth interactive area is an area D in fig. 4, and the fifth interactive area is an area E in fig. 4.
The first interactive region is characterized in that the vertex bone point is used as the center, the longitudinal length is 5 unit lengths, the transverse length is 6 unit lengths, the first interactive region is 12 unit lengths, and the height is 10 unit lengths. The first interactive area takes a head top skeleton point as a center, the longitudinal upward 5 unit lengths are the upper sides, and the longitudinal downward 5 unit lengths are the lower sides; the vertex bone point is taken as the center, the horizontal left 6 unit lengths are taken as the left side, and the horizontal right 6 unit lengths are taken as the right side. The control instruction preset in the first interaction area is the determination A, namely when the interaction gesture is located in the first interaction area, the control instruction corresponding to the first interaction area is output, namely the control instruction for determining the A is output.
The control instruction of the second interactive area is leftward, the control instruction of the third interactive area is rightward, the second interactive area is an area which takes a pelvic bone point as a starting point, the length of 2 units upwards in the longitudinal direction is the upper side, the length of 10 units downwards in the longitudinal direction is the lower side, the bone point at the top of the head is taken as the starting point, the length of 3 units leftwards in the transverse direction is the right side, the length of 16 units leftwards in the transverse direction is the left side and is 13 units, and the height of the area is 12 units.
The third interactive region is a region which takes the pelvis bone point as an initial point, the longitudinal upward 2 unit lengths are used as the upper side, the longitudinal downward 10 unit lengths are used as the lower side, the vertex bone point is used as the initial point, the transverse rightward 3 unit lengths are used as the left side, the transverse rightward 16 unit lengths are used as the right side, the length formed by the right side is 13 unit lengths, and the height is 12 unit lengths.
The control instruction of the fourth interactive area is determination D, the control instruction of the fifth interactive area is determination E, the fourth interactive area is an area which takes the bone point at the top of the head as a starting point, the length of 4 units upwards longitudinally is the upper side, the length of 5 units downwards longitudinally is the lower side, the bone point at the top of the head is the starting point, the length of 7 units leftwards transversely is the right side, the length of 16 units leftwards transversely is the left side is 11 units, and the height is 9 units.
The fifth interactive region is a region which takes the bone point at the top of the head as an initial point, takes 4 unit lengths in the vertical upward direction as an upper side, takes 5 unit lengths in the vertical downward direction as a lower side, takes the bone point at the top of the head as an initial point, takes 7 unit lengths in the horizontal rightward direction as a left side, takes 16 unit lengths in the horizontal rightward direction as a right side, has a length of 11 unit lengths, and has a height of 9 unit lengths.
S124, identifying an interaction gesture according to the skeleton point information;
s124.1, acquiring the distance between the left-hand skeleton point and the right-hand skeleton point;
s124.2, comparing the distance between the left-hand skeleton point and the right-hand skeleton point with a threshold T, if the distance between the left-hand skeleton point and the right-hand skeleton point is smaller than or equal to the threshold T, recognizing an interactive gesture, wherein the threshold T is 15cm in the embodiment, and if the distance between the left-hand skeleton point and the right-hand skeleton point is smaller than or equal to the threshold T, judging that the user makes the interactive gesture, and recognizing the interactive gesture, for example, the interactive gesture is a clapping, namely, the user makes a clapping action in the embodiment; if the distance between the left-hand skeleton point and the right-hand skeleton point is greater than the threshold value T, the user does not perform a clapping action; the specific interaction gesture in this embodiment is only for illustration and is not limited.
S124.3, when the user makes an interaction gesture, acquiring a middle point coordinate between the left-hand skeleton point and the right-hand skeleton point;
s125, judging whether the middle point coordinate is positioned in the interaction area;
s125.1, judging whether the middle point coordinate is positioned in the interaction area;
s125.11, if the intermediate point coordinate is located in one of the interactive regions, the interactive region is in an activated state, and a control instruction corresponding to the interactive region is output; the other interactive areas are in an inactivated state;
s125.12 outputs a control instruction corresponding to the interactive region, the interactive region is in an activated state, and the rest interactive regions are in an inactivated state; the area of the interaction region in the activated state is increased, and the area of the interaction region in the inactivated state is reduced.
In this embodiment, the length of the interactive region in the activated state is increased by 1.5 unit lengths, and the height is increased by 0.5 unit lengths, specifically, in this embodiment, the length of the interactive region in the activated state is sequentially increased toward the left and right sides, that is, extended by 0.75 unit lengths, and the height is sequentially increased toward the upper and lower ends, that is, extended by 0.25 unit lengths. The length of the interactive area in the inactivated state is reduced by 2 units, and the height is reduced by 1 unit. Specifically, the length of the interaction region in the inactive state is sequentially reduced towards the left and right sides, i.e. is reduced inwards by 1 unit length, and the height is reduced towards the upper end and the lower end, i.e. is reduced inwards by 0.5 unit length.
And when the control instruction corresponding to the interactive region is not output, the area of the interactive region is not changed, the area of the interactive region is adjusted after the control instruction is output, and the area of the interactive region is restored to the original state after the control instruction is executed.
After the user makes an interactive gesture in the first interactive region, the fitness mirror recognizes the interactive gesture, outputs a control instruction corresponding to the first interactive region, and enlarges the area of the first interactive region, namely, the upper side is longitudinally and upwardly increased by 0.25 unit length, the lower side is longitudinally and downwardly increased by 0.25 unit length, the left side is transversely and leftwards increased by 0.75 unit length, and the right side is transversely and rightwards increased by 0.75 unit length.
All parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the teachings of the present invention is/are used. It is to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, embodiments of the invention may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein.
In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, if such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the scope of the present disclosure. Other substitutions, modifications, changes and omissions may be made in the design, operating conditions and arrangement of the corresponding elements of the exemplary embodiments without departing from the spirit of the present disclosure. The use of numerical ranges does not exclude equivalents which fall outside the ranges that perform the same function in the same way to produce the same result.
The above-described embodiments can be implemented in various ways. For example, embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code can be executed on a suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.
Further, a computer may be embodied in any of a number of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer. Additionally, a computer may be embedded in a device not normally considered a computer but with suitable processing capabilities, including a Personal Digital Assistant (PDA), a smart phone, or any other suitable portable or stationary electronic device.
In addition, a computer may have one or more input and output devices. These devices may be used to present a user interface, among other things. Examples of output devices that may be used to provide a user interface include printers or display screens for visual presentation of output and speakers or other sound-generating devices for audible presentation of output. Examples of input devices that may be used for the user interface include keyboards, and pointing devices, such as mice, touch pads, and digitizing tablets. As another example, a computer may receive input information through speech recognition or other audible format.
Such computers may be interconnected by one or more networks IN a suitable form, including as a local area network or a wide area network, such as an enterprise network, an Intelligent Network (IN), or the Internet. Such networks may be based on suitable technologies, may operate according to suitable protocols, and may include wireless networks, wired networks, or fiber optic networks.
The various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and/or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine. Some embodiments may specifically employ one or more of a particular operating system or platform and a particular programming language and/or scripting tool to facilitate execution.
Furthermore, various inventive concepts may be embodied as one or more methods, at least one example of which has been provided. The actions performed as part of the method may in some cases be ordered differently. Accordingly, in some inventive embodiments, the respective acts of a given method may be performed in an order different than that specifically illustrated, which may include performing some acts concurrently (even though such acts are illustrated as sequential acts in exemplary embodiments).
All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety.
All definitions, as defined and used herein, should be understood to control dictionary definitions, definitions in documents incorporated by reference, and/or ordinary meanings of the defined terms.
The indefinite articles "a" and "an" as used in this specification and in the claims are to be understood as meaning "at least one" unless clearly indicated to the contrary.
The phrase "and/or" as used in this specification and claims should be understood to mean "one or two" elements so combined, i.e., elements that are present in combination in some cases and are present in isolation in other cases. Multiple elements listed with "and/or" should be construed in the same manner, i.e., "one or more" elements so combined. Other elements may optionally be present in addition to the elements specifically identified by the "and/or" clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, when used in conjunction with open language such as "including", references to "a and/or B" may refer in one embodiment to only a (optionally including elements other than B); in another embodiment, reference is made to B only (optionally including elements other than a); in yet another embodiment, refers to both a and B (optionally including other elements); and so on.
As used in this specification and claims, "or" should be understood to have the same meaning as "and/or" as defined above. For example, when separating items in a list, "or" and/or "should be interpreted as being inclusive, i.e., including at least one, but also including more than one, of the plurality or list of elements, and (optionally) additional unlisted items. To the extent that contradictory statements are made, such as "only one" or "exactly one" or "consisting of" when used in a claim, shall mean that there is exactly one element in a number or list of elements. In general, as used herein, the term "or" followed by an exclusive term such as "one of the two", "one of the", "only one of", or "exactly one of" should only be construed to indicate an exclusive alternative (i.e., "one or the other but not both"). "consisting essentially of" as used in the claims has its ordinary meaning as used in the patent law.
As used in this specification and the claims, the phrase "at least one," in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each element specifically listed in the list of elements, and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements referred to by the phrase "at least one," whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, "at least one of a and B" (or, equivalently, "at least one of a or B," or, equivalently "at least one of a and/or B") can refer in one embodiment to at least one a, optionally including more than one a, without B (and optionally including elements other than B); in another embodiment, to at least one B, optionally including more than one B, with no a present (and optionally including elements other than a); in yet another embodiment, reference to at least one a, optionally includes more than one a and at least one B, optionally includes more than one B (and optionally includes other elements); and so on.
In the claims, as well as in the specification above, all transitional phrases such as "comprising," including, "" carrying, "" having, "" containing, "" involving, "" holding, "" consisting of,. Or the like are to be construed as open-ended, i.e., including but not limited to.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (13)
1. An interactive intelligent body-building mirror device is characterized by comprising
A display for displaying a user interface having at least one interactive object;
a mirror to show a mirror image of a user on the mirror, the display to display the user interface to the user through the mirror;
the sensor is used for collecting user action data of a user;
a processor, coupled to the display and the sensor, for generating a control instruction based on user action data collected by the sensor, the control instruction controlling the user interface to complete a specified operation;
the processor is further used for acquiring a motion track of a user action according to the user action data acquired by the sensor, and controlling a floating icon to be displayed on a user interface according to the motion track, wherein the floating icon is used for indicating the position indicated by the user action in the user interface.
2. The interactive intelligent fitness mirror device of claim 1, wherein the interactive objects of the user interface comprise a floating icon wake-up interaction, and when the processor determines that the motion indicated by the user motion data collected by the sensor matches a floating icon wake-up motion corresponding to a preset floating icon wake-up interaction, the processor enables the floating icon to move with the motion of the user.
3. The interactive intelligent fitness mirror device of claim 1 or 2, wherein the processor turns off the floating icon function so that the floating icon no longer follows the user's motion after the processor determines that the user has completed a selection operation at the current user interface.
4. The interactive intelligent fitness mirror device of claim 1 or 2, wherein the processor executes the operation corresponding to the user interface object when the processor determines that the floating icon is moved to the position of the user interface object in the user interface according to the user motion data and determines that the user has performed the selection operation at the position of the user interface object.
5. The interactive intelligent fitness mirror device of claim 4, wherein the selection operation comprises any one of gesturing, pushing palm forward, pushing palm backward, swinging hand, and making a fist.
6. The interactive intelligent fitness mirror device of claim 1 or 2, wherein the user interface further comprises a page movement effect, the page movement effect comprises a page switching movement effect and a page interaction movement effect, wherein the page switching movement effect is used for generating a dynamic effect when a page is operated and changed, and the page interaction movement effect is used for providing guidance for a user when interaction is performed.
7. The interactive intelligent fitness mirror device of claim 1, wherein the processor generates skeletal point coordinates of the user based on the user motion data collected by the sensor, obtains coordinates corresponding to the floating icon on the user interface according to the skeletal point coordinates, invokes a motion trajectory corresponding to the coordinates, and controls the floating icon to be displayed on the user interface according to the motion trajectory.
8. The interactive intelligent fitness mirror device of claim 1, wherein the user interface further comprises a wake-up page configured to switch to another user interface when the sensor collects user motion data matching a preset wake-up motion.
9. The interactive smart workout mirror apparatus of claim 1, wherein the user interface further comprises at least one page object for presenting at least one or at least one set of page information, the page information comprising workout videos, visual data, actionable interaction objects, interaction guides.
10. The interactive intelligent fitness mirror device of claim 1, wherein the user actions collected by the sensor comprise static actions and dynamic actions, the static actions being user gestures performed on predetermined body parts for a specified time, and the dynamic actions being user actions performed on predetermined body parts.
11. The interactive intelligent fitness mirror device of claim 10, wherein static actions include, but are not limited to, lifting the left/right hand for a preset time, lifting the left/right leg for a preset time, and lifting the hands over the top of the head for a preset time.
12. The interactive intelligent fitness mirror device of claim 10, wherein the dynamic actions include, but are not limited to, horizontal sliding left/right hand, up and down sliding left/right hand, double-handed clapping, jumping up, and squatting.
13. The interactive intelligent fitness mirror device of claim 1, wherein the sensor is further configured to collect characteristic data of a user, the processor compares the characteristic data input by the user with characteristic data in a database, determines a user using the mirror device based on the comparison, and allows the mirror device to adjust the graphical user interface based on the recognition, the characteristic data of the user including but not limited to a skeletal point, a voiceprint, a weight, and a resting heart rate of the user, the database having a mapping relationship between the user and the characteristic data recorded therein.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2022105302110 | 2022-05-16 | ||
| CN202210530211 | 2022-05-16 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115177938A true CN115177938A (en) | 2022-10-14 |
Family
ID=83520535
Family Applications (4)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210931230.4A Pending CN115569368A (en) | 2022-05-16 | 2022-08-04 | Interactive body-building mirror device and method |
| CN202210931098.7A Pending CN115212543A (en) | 2022-05-16 | 2022-08-04 | Interface switchable fitness system |
| CN202210931241.2A Pending CN115177938A (en) | 2022-05-16 | 2022-08-04 | Interactive intelligent body-building mirror device |
| CN202210931229.1A Pending CN115177937A (en) | 2022-05-16 | 2022-08-04 | Interactive intelligent body-building mirror device |
Family Applications Before (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210931230.4A Pending CN115569368A (en) | 2022-05-16 | 2022-08-04 | Interactive body-building mirror device and method |
| CN202210931098.7A Pending CN115212543A (en) | 2022-05-16 | 2022-08-04 | Interface switchable fitness system |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210931229.1A Pending CN115177937A (en) | 2022-05-16 | 2022-08-04 | Interactive intelligent body-building mirror device |
Country Status (2)
| Country | Link |
|---|---|
| CN (4) | CN115569368A (en) |
| WO (1) | WO2023221233A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117931357B (en) * | 2024-03-22 | 2024-06-18 | 东莞莱姆森科技建材有限公司 | Intelligent mirror based on interactive data processing, mirror cabinet and control method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664179A (en) * | 2017-04-01 | 2018-10-16 | 上海卓易科技股份有限公司 | A kind of interface deployment method and device |
| CN110191365A (en) * | 2019-05-30 | 2019-08-30 | 深圳创维－Rgb电子有限公司 | It is a kind of for the method for action imitation, storage medium and system |
| CN112007348A (en) * | 2018-05-29 | 2020-12-01 | 库里欧瑟产品公司 | Reflective video display device for interactive training and demonstration and method of use thereof |
| US20210379447A1 (en) * | 2020-06-09 | 2021-12-09 | Johnson HealthTech. Co., Ltd | Interactive exercise apparatus |
| CN114399833A (en) * | 2021-12-01 | 2022-04-26 | 鉴丰电子科技有限公司 | Interaction method and system suitable for intelligent fitness mirror |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7946961B2 (en) * | 2008-05-23 | 2011-05-24 | Yoga Today Llc | Exercise apparatus and methods |
| KR20160016263A (en) * | 2014-08-04 | 2016-02-15 | 엘지전자 주식회사 | Mirror display device and operating method thereof |
| CN105117112A (en) * | 2015-09-25 | 2015-12-02 | 王占奎 | Aerial interactive intelligent holographic display system |
| KR20180052224A (en) * | 2016-11-10 | 2018-05-18 | 인천대학교 산학협력단 | Home training mirror |
| US11331538B2 (en) * | 2018-08-07 | 2022-05-17 | Interactive Strength, Inc. | Interactive exercise machine data architecture |
| CN113457105B (en) * | 2020-03-30 | 2022-09-13 | 乔山健身器材（上海）有限公司 | Intelligent mirror with body-building menu |
| CN114073850A (en) * | 2020-08-14 | 2022-02-22 | 乔山健身器材（上海）有限公司 | System and method for synchronizing courses on line |
| CN215691547U (en) * | 2021-07-28 | 2022-02-01 | 乔山健身器材（上海）有限公司 | Motion guide apparatus |
| CN114028794A (en) * | 2021-11-12 | 2022-02-11 | 成都拟合未来科技有限公司 | Auxiliary fitness method and system with interaction function |
-
2022
- 2022-06-23 WO PCT/CN2022/100664 patent/WO2023221233A1/en unknown
- 2022-08-04 CN CN202210931230.4A patent/CN115569368A/en active Pending
- 2022-08-04 CN CN202210931098.7A patent/CN115212543A/en active Pending
- 2022-08-04 CN CN202210931241.2A patent/CN115177938A/en active Pending
- 2022-08-04 CN CN202210931229.1A patent/CN115177937A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664179A (en) * | 2017-04-01 | 2018-10-16 | 上海卓易科技股份有限公司 | A kind of interface deployment method and device |
| CN112007348A (en) * | 2018-05-29 | 2020-12-01 | 库里欧瑟产品公司 | Reflective video display device for interactive training and demonstration and method of use thereof |
| CN110191365A (en) * | 2019-05-30 | 2019-08-30 | 深圳创维－Rgb电子有限公司 | It is a kind of for the method for action imitation, storage medium and system |
| US20210379447A1 (en) * | 2020-06-09 | 2021-12-09 | Johnson HealthTech. Co., Ltd | Interactive exercise apparatus |
| CN114399833A (en) * | 2021-12-01 | 2022-04-26 | 鉴丰电子科技有限公司 | Interaction method and system suitable for intelligent fitness mirror |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115212543A (en) | 2022-10-21 |
| CN115569368A (en) | 2023-01-06 |
| WO2023221233A1 (en) | 2023-11-23 |
| CN115177937A (en) | 2022-10-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11497980B2 (en) | Reflective video display apparatus for interactive training and demonstration and methods of using same | |
| JP6816925B2 (en) | Data processing method and equipment for childcare robots | |
| EP3381175B1 (en) | Apparatus and method for operating personal agent | |
| CN113760142B (en) | Interaction method and device based on virtual roles, storage medium and computer equipment | |
| CN107491929A (en) | The natural language event detection of data-driven and classification | |
| KR20190053278A (en) | Controls and interfaces for user interaction in virtual space | |
| JP2023540856A (en) | Video streaming via multiplex communication and display via smart mirror | |
| CN107111430A (en) | Voice notes and the mixing of calling | |
| US20220093000A1 (en) | Systems and methods for multimodal book reading | |
| JP2011039860A (en) | Conversation system, conversation method, and computer program using virtual space | |
| CN112439180B (en) | Intelligent voice playing method and equipment | |
| JP7278307B2 (en) | Computer program, server device, terminal device and display method | |
| JP6040745B2 (en) | Information processing apparatus, information processing method, information processing program, and content providing system | |
| US20230116624A1 (en) | Methods and systems for assisted fitness | |
| WO2016136104A1 (en) | Information processing device, information processing method, and program | |
| KR20200059112A (en) | System for Providing User-Robot Interaction and Computer Program Therefore | |
| CN115177938A (en) | Interactive intelligent body-building mirror device | |
| KR20210007223A (en) | System and method for providing user customized feedback couching information based on video clip | |
| WO2019026396A1 (en) | Information processing device, information processing method, and program | |
| CN117055776B (en) | User interface for fitness content | |
| Almeida et al. | Gym at Home-A Proof-of-Concept | |
| US20230230293A1 (en) | Method and system for virtual intelligence user interaction | |
| WO2024214792A1 (en) | Behavior control system | |
| WO2019054009A1 (en) | Information processing device, information processing method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |