CN115177937A

CN115177937A - Interactive intelligent body-building mirror device

Info

Publication number: CN115177937A
Application number: CN202210931229.1A
Authority: CN
Inventors: 徐毅斐; 蒋婷; 顾才鹏; 张芷萍; 温伯华; 唐天广; 付强
Original assignee: Chengdu Fit Future Technology Co Ltd
Current assignee: Chengdu Fit Future Technology Co Ltd
Priority date: 2022-05-16
Filing date: 2022-08-04
Publication date: 2022-10-14
Also published as: CN115212543A; CN115177938A; CN115569368A; WO2023221233A1

Abstract

The invention discloses an interactive intelligent fitness mirror device, which relates to intelligent home and intelligent fitness, and comprises: a display operable to display a user interface having at least one interactive object for presenting control actions provided for execution by a user; the image recognition sensor is used for acquiring action data of a user; the processor is coupled to the display and the image recognition sensor and used for comparing whether the action data acquired by the image recognition sensor is matched with the control action provided by any interactive object display in at least one interactive object on the user interface or not, and generating a control instruction corresponding to the matched interactive object when the action data is matched based on a matching result, wherein the control instruction is used for controlling the user interface to be switched or finishing a specified operation on the user interface; a mirror for displaying a mirror image of a user on the mirror, comprising a partially reflective portion, the user interface displayed by the display to the user through the partially reflective portion of the mirror, and at least a portion of the image of the user in the mirror superimposed on the partially reflective portion.

Description

Interactive intelligent body-building mirror device

Technical Field

The invention relates to the field of intelligent home and intelligent fitness, in particular to an interactive intelligent fitness mirror device.

Background

With the rapid development of economy, the pace of life of people is accelerated, people in urban life gradually have various health problems due to high-speed life and high-load work, and accordingly body building becomes a hot spot of people's attention. An intelligent body-building mirror is a novel body-building product integrating artificial intelligence and hardware content service.

A patent with application number CN202110946212.9 entitled reflective video display device for interactive training and demonstration and method of use thereof discloses an exercise mirror and method of training using the same.

At present, most of body-building personnel at home watch videos through the intelligent body-building mirror to carry out learning and training actions, and the intelligent body-building mirror can be used as a mirror and can also see coach images with high image quality. However, the existing intelligent fitness mirror is troublesome in interaction mode, generally needs to be controlled through an external intelligent device (such as a smart phone) or a touch screen, is relatively complex to operate when being controlled through the external intelligent device, needs to be distributed with the intelligent device and the intelligent fitness mirror, and is easy to leave fingerprints and sweat on the mirror surface by adopting touch control, so that the display effect of the screen is influenced.

Disclosure of Invention

In order to solve the defects caused by the fact that the intelligent fitness mirror is controlled by using terminal equipment and the intelligent fitness mirror is controlled by touching in the prior art, the invention provides an interactive intelligent fitness mirror device, which comprises:

a display for displaying a user interface having at least one interactive object providing a control action for a user to perform;

the image recognition sensor is used for acquiring action data of a user;

the processor is coupled to the display and the image recognition sensor and used for comparing whether the action data acquired by the image recognition sensor is matched with the control action provided by any interactive object in at least one interactive object on the user interface or not, and when the action data is matched with the control action, a control instruction corresponding to the matched interactive object is generated, wherein the control instruction is used for controlling the user interface to be switched or finishing a specified operation on the user interface;

a mirror to show a mirror image of a user on the mirror, the display to display the user interface to the user through the mirror.

According to the intelligent fitness mirror, the image recognition sensor is used for collecting the action data of the user, and then the corresponding control instruction is generated based on the action data of the user to control the intelligent fitness mirror, so that the intelligent terminal is prevented from being used or the control is carried out in a touch mode.

Preferably, the completing the specified operation on the user interface further includes controlling switching of a page object corresponding to the interactive object or completing the specified operation on the corresponding page object.

Preferably, the user actions collected by the sensor include static actions and dynamic actions, the static actions are that the user makes a predetermined gesture with a predetermined body part and maintains for a specified time, and the dynamic actions are that the user makes a predetermined action with a predetermined body part.

Preferably, the static motion includes raising the left/right hands for a preset time, raising the left/right legs for a preset time, and raising the hands over the top of the head for a preset time.

Preferably, the dynamic actions include sliding left/right hand horizontally, sliding left/right hand up and down, clapping, jumping, and squatting.

Preferably, the user interface comprises an exercise page and a non-exercise page, and the control instruction comprises a control instruction for the exercise page and a control instruction for the user interface of the non-exercise page.

Preferably, the first and second liquid crystal materials are,

the non-exercise page comprises a wakeup page, a main page or other pages; the control instructions for the user interface of the non-workout page include user interaction instructions with the non-workout page, where the interaction instructions include at least one of entering a previous page, entering a next page, determining, returning, selecting a course, collecting a course, joining a plan, entering a page switch interface, waking a voice assistant, and controlling multimedia volume.

Preferably, the control instruction for the exercise page includes a control instruction for playing the exercise course by the user, including at least one of pause of the course, playing of the course, previous link, next link, background music volume control of the course, volume control of a coach of the course, and evaluation of the course.

Preferably, the control actions are pre-configured in the interactive mirror device or customized for the user.

The present invention also provides a reflective video display device (also referred to herein as "smart gym glasses" and "interactive fitness systems") configured to display video content, such as pre-recorded or live exercises, carried by a trainer, to a user and provide an interface that allows the user to interact and personalize the video content. The smart fitness mirror may be a networked device communicatively coupled to a content provider (e.g., a server, a cloud service) and/or a smart device (e.g., a smartphone, a tablet, a computer). The intelligent fitness mirror may include a display panel and a speaker to output video content and audio to a user. The smart exercise scope may also include a camera and microphone to capture video and audio of the user during exercise. Thus, the intelligent fitness mirror may enable bi-directional communication between the user and the trainer during the workout. In this manner, the intelligent gym mirror can provide a user with a convenient choice of receiving instructional exercises, while achieving a greater degree of personalization and personal guidance similar to exercises provided by personal trainers or coaches in conventional gyms.

One example of a smart fitness mirror includes a communication interface for receiving video imagery of a fitness trainer, a display operatively coupled to the communication interface to display the video imagery of the fitness trainer, and a mirror disposed in front of the display to reflect an image of a person opposite the display. The mirror has a partially reflective portion to transmit the video imagery of the fitness trainer to the person opposite the display such that the video imagery of the fitness trainer appears superimposed on a portion of the image of the person.

One example of an interactive fitness method comprises the following steps: (1) Streaming fitness content to an interactive video system, the interactive video system including a mirror having a partially reflective portion and a display disposed on one side of the partially reflective portion; (2) Displaying fitness content to a user via the display and the partially reflective portion of the mirror; (3) And reflecting the image of the user with the mirror such that the image of the user is at least partially superimposed on the fitness content displayed via the display and the partially reflective portion of the mirror.

One example of a method of using intelligent fitness scope includes the steps of: while displaying fitness content to the user on the video display behind the partially transmissive mirror: (1) reflecting an image of a user with a partially transmissive mirror; (2) Measuring a heart rate of a user with a heart rate monitor attached to the user; (3) Transmitting the heart rate from the heart rate monitor to an antenna operably coupled to a video display; (4) displaying the heart rate of the user on the video display; and (5) displaying the target heart rate of the user on the video display.

In order to add a personalized interface and a new interaction mode on the intelligent body-building mirror, optimize the interaction mode of the intelligent body-building mirror and a user and improve the use frequency of the intelligent body-building mirror in daily life, the invention designs a set of operating system Launcher for the intelligent body-building mirror, and the method specifically comprises the following steps: (1) a GUI interface; (2) a system control method of motion control; (3) a system control method of voice control; (4) a control method of multimodal fusion; (5) executable control instructions; (6) a method for data acquisition management; and (7) some specific function management methods. The core of the invention is that a set of system and an operation method are customized for the intelligent fitness mirror, so that the intelligent fitness mirror has the functions of not only watching fitness and exercise videos, but also being used as an important component of intelligent home, facilitating a user to check more information and improving the use efficiency. Meanwhile, the customized operation method overcomes the defect that the traditional intelligent fitness mirror is easy to leave fingerprints on a partial reflecting mirror surface by touching for control, and as the core function of the intelligent fitness mirror is to enable a user to compare the actions of mirror images when watching a fitness exercise video, the influence on the functions of the intelligent fitness mirror after leaving the fingerprints on the partial reflecting mirror surface is large, most users are reluctant to use touch interaction, and the intelligent terminal is used for controlling the intelligent fitness mirror, so that the intelligent fitness mirror is not intuitive and troublesome. The method for controlling the movement is free from the problems, gesture control is also applied to other electronic products, but the gesture control is also applied to the intelligent fitness mirror, and breakthrough progress is achieved when the gesture control is applied to the intelligent fitness mirror.

All combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein. Terms explicitly employed herein and which may also appear in any disclosure incorporated by reference should be given the most consistent meaning to the particular concepts disclosed herein.

One or more technical schemes provided by the invention at least have the following technical effects or advantages:

the intelligent fitness mirror is controlled by utilizing the action on the interface of the intelligent fitness mirror.

According to the invention, through switching the user interfaces with one or more user interface objects on the fitness mirror, the functions of the fitness mirror are enriched, the user experience of a plurality of user interface objects is promoted, and the enthusiasm and interest of the user in fitness are increased.

Drawings

The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:

FIG. 1 is a block diagram of an exemplary intelligent fitness mirror;

FIG. 2 is an exemplary GUI page on the smart fitness mirror;

FIG. 3 is an exemplary GUI page on the intelligent fitness mirror;

FIG. 4 is an exemplary GUI page on the intelligent fitness mirror;

FIG. 5-a is an exemplary GUI page on an intelligent fitness mirror;

5-b are exemplary GUI pages on an intelligent fitness mirror;

FIG. 6 is an exemplary GUI page on the smart fitness mirror;

FIG. 7 is an exemplary GUI page on the intelligent fitness mirror;

FIG. 8 is an exemplary GUI page on the smart fitness mirror;

FIG. 9 is an exemplary GUI page on the intelligent fitness mirror;

fig. 10 is a schematic view of motion recognition of the intelligent fitness mirror.

Detailed Description

In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments of the present invention and features of the embodiments may be combined with each other without conflicting with each other.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described and thus the scope of the present invention is not limited by the specific embodiments disclosed below.

It will be understood by those skilled in the art that in the present disclosure, the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate orientations or positional relationships that are based on those shown in the drawings, which are merely for convenience in describing the present disclosure and to simplify the description, and do not indicate or imply that the device or element so referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus the terms above should not be construed as limiting the present disclosure.

Accordingly, this patent relates to an interactive mirror apparatus (also referred to as "smart fitness mirror" and "interactive training system") and methods of using an interactive training device. The smart exercise glasses include a display configured to display exercise content (pre-recorded video or live stream) and an interface that enables a user to personalize the exercise. In addition, smart fitness mirrors may allow a user and/or a trainer to interact with each other during a workout in a manner similar to a regular workout in a gym or mini-gym where the user and trainer are in the same room (e.g., provide feedback to the trainer regarding the pace of the workout, correct the form of the user during a particular fitness program).

An exemplary intelligent fitness mirror.

FIG. 1 illustrates an exemplary representation of intelligent exercise goggles. The intelligent fitness mirror may include a processor 110 for controlling, in part, the operation of various subcomponents in the intelligent fitness mirror and managing the flow of data (e.g., video content, audio from a coach or user, biometric feedback analysis) into/out of the intelligent fitness mirror. The intelligent fitness mirror may include a display 120 for displaying video content, a Graphical User Interface (GUI) with which a user may interact and control the intelligent fitness mirror, biometric feedback data, and/or other visual content. The sensor 130 may be coupled to the processor 110 to collect user-related data. An antenna 140 may be coupled to the processor 110 to provide data transmission between the intelligent exercise mirror and another device (e.g., a remote control device, biometric sensor, wireless router). The antenna 140 may include multiple transmitters and receivers, each customized for a particular frequency and/or wireless standard (e.g., bluetooth, 802.11a, 802.11b, 802.11G, 802.11n, 802.11ac, 2G, 3G, 4G LTE, 5G). An amplifier 150 may be coupled to the processor 110 to receive audio signals from the processor 110 for outputting subsequent sounds through a left speaker 152 and/or a right speaker 154.

The intelligent exercise scope may also include additional components not shown in fig. 1. For example, the smart fitness mirror may include a Switched Mode Power Supply (SMPS), switches, and onboard memory and storage devices (non-volatile and/or volatile memory) including, but not limited to, a Hard Disk Drive (HDD), a solid state drive (SDD), flash memory, random Access Memory (RAM), and a Secure Digital (SD) card. The on-board memory and/or storage device may be used to store firmware and/or software for the operation of the intelligent exercise mirror. As described above, the onboard memory and/or storage may also be used to store (temporarily and/or permanently) other data including, but not limited to, video content, audio, user video, biometric feedback data, and user settings. In another example, the intelligent exercise scope may include various components to mount and support the intelligent exercise scope.

The antenna 140 may include multiple antennas, each acting as a receiver and/or transmitter to communicate with various external devices, such as a user's smart terminal (e.g., computer, smartphone, tablet), an external sensor (e.g., heart rate band, inertial sensor, body fat scale), and/or a remote server or cloud server for streaming or playing video content. Again, the antenna 140 may conform to various wireless standards including, but not limited to, bluetooth, 802.11a, 802.11b, 802.11G, 802.11n, 802.11ac, 2G, 3G, 4G LTE, and 5G standards.

The sensor 130 may include a plurality of sensors, which may be one or more of an image recognition sensor (camera), an infrared recognition sensor, or a voice recognition sensor (microphone), respectively.

An image recognition sensor (e.g., a camera) in the intelligent fitness scope may be used to acquire video and/or still images of the user while the user is performing an activity (e.g., an exercise). The video of the user may then be shared with the trainer to allow the trainer to observe and provide coaching to the user during the workout. The video may also be shared with other users of other intelligent fitness mirrors for comparison or competition. The user's video may also be displayed in real-time on the display 120 or stored for later playback. For example, by providing a visual comparison of the user with a trainer, the user's video can be used for self-assessment during or after exercise. The stored video may also allow the user to assess their progress or improvement over time while performing similar exercises. The image recognition sensor (camera) may also be configured to acquire and process dynamic and/or static images of the user into a skeletal point image while the user performs an activity (e.g., exercise), and determine whether the user's motion matches a standard motion based on the skeletal point image and output a determination result, and if the user's motion does not match the standard motion, output motion correction information. The dynamic and/or static images may also be processed in real-time as the user uses the intelligent fitness scope or after the workout is completed to control the intelligent fitness scope or derive biometric data of the user based on the user's motion and actions.

The infrared recognition sensor in the intelligent fitness mirror can be used for acquiring a thermal imaging image of a user when the user performs activities (such as exercise), judging the exercise part, the exertion part and the consumed heat quantity of the user during movement according to the thermal imaging image, judging whether the user action is matched with the standard action or not and outputting a judgment result according to the exercise part and the exertion part of the user during movement, and outputting action correction information if the user action is not matched with the standard action

A voice recognition sensor (microphone) in the intelligent exercise scope may be used to acquire real-time processing of the user's voice information as the user uses the intelligent exercise scope and to control or converse with the user based on the user's voice information.

Graphic User Interface (GUI)

The intelligent fitness mirror may display a Graphical User Interface (GUI) through the display 120 to facilitate user interaction with the intelligent fitness mirror.

Graphical User Interfaces (GUIs) include, but are not limited to, a wake page, a home page, a workout page, and other pages, among others.

The awakening page is an initial page when the intelligent fitness mirror is awakened and is used for displaying some basic page information, and the intelligent fitness mirror is completely awakened to enter a main page, a training page or other pages only when the user is detected to perform specified operation.

Wherein the main page presents at least one item of page information for presenting data, actionable interactive objects or interactive guidance to a user. The main page is used for displaying preset page information or page information set by the user to the user, and meanwhile, the user can control the intelligent fitness mirror by performing specified operation on the main page, wherein the control includes but is not limited to page switching, function switching or entering into an application program accessed by a third party.

The exercise page displays a video playing window for playing an exercise video to guide a user to exercise, and can also display at least one of course playing information (such as the total duration of the course, the played duration and the like), accessory connection information (such as heart rate belt connection information and the like) and user physiological information (such as heart rate, blood pressure, calories consumed and the like). The exercise page can enter from the main page or enter from the awakening page and return to the main page or the awakening page, and can also enter directly when the intelligent fitness mirror is started.

Wherein, other pages show pre-installed function pages (such as multimedia player pages, weather display pages, diet recommendation pages, etc.) or application program pages accessed by third parties (such as WeChat, buffalo, himalayan, etc.) entered through the main page, and other pages generally enter from the main page or wake up pages and return to the main page or wake up pages.

Page information

The Graphical User Interface (GUI) may present page information through page objects, each page object being used for presenting at least one or a group of page information, where the presented page information may be information stored in a storage device local to the intelligent exercise mirror, or may be information received by the intelligent exercise mirror from other devices through the communication interface, including but not limited to course preview information, user information, environment information, multimedia information, external device information, and the like, which may be switchably presented.

The course preview information includes, but is not limited to, a coach name, a course type, a course length, a course difficulty level, a course label, and/or a projected consumption, among others. The types of lessons include, but are not limited to, presenting HIIT to the user through the smart fitness mirror, aerobics, yoga, shaping of strength, fighting exercises, barre, pilates, stretches, dances, meditation, challenges, and the like. The course labels include, but are not limited to, whether the courses are identified for AI or not, whether the courses are parent-child courses or not, body parts mainly related to the courses (such as whole body, arms, waist and abdomen, legs and the like), and properties (such as no instrument, yoga mat, elastic ring, elastic band, dumbbell and the like) required to be used in the courses. The page object displaying the course preview information is also called a course card.

The user information includes, but is not limited to, usage history information, physiological information, social information, calendar information, and the like. The use history information includes, but is not limited to, completed courses, collected courses, historical best results, frequency of lessons presented to the user through the intelligent fitness mirror, and the like. The physiological information includes, but is not limited to, height, weight, sex, age, heart rate, blood pressure, injury history, etc. displayed to the user through the intelligent fitness scope. The social information includes, but is not limited to, information related to people/friends concerned by the user and displayed to the user through the intelligent fitness mirror, and may also be information related to a group/interest group to which the user joins, and the information may specifically be an invitation to invite the user to complete a lesson for a specified course, an invitation to invite the user to challenge one or more people, a ranking between users, and a dynamic uploading state of the user himself or another user, and the like. The dynamics may be synchronized to other social platforms (e.g., weChat, microblog, facebook, twitter, etc.). The calendar information includes, but is not limited to, schedule information, backlogs, lesson plans, etc. of the user displayed to the user through the intelligent fitness mirror. The page object showing the user information is also called a user card.

The environmental information includes, but is not limited to, weather information. The weather information includes, but is not limited to, weather forecast, temperature, humidity, dressing recommendation and the like displayed to the user through the intelligent fitness mirror. The page object for displaying the weather information is also called a weather card.

The multimedia information includes but is not limited to music and video information played through the intelligent fitness mirror. The page object for displaying multimedia information is also called a multimedia card.

The external device information includes, but is not limited to, information of other fitness devices connected with the intelligent fitness mirror through intelligent fitness mirror display, and information of other intelligent furniture (such as a sweeping robot, an intelligent sound box, an intelligent gateway and the like). The page object for displaying the external device information is also called a device card.

The Graphical User Interface (GUI) may also present page information via interactive objects, which may be icons, text, highlighted selections, other forms that may prompt the user for interaction, or a combination thereof. The interactive objects are used to present to the user controls or operations (e.g., page switches, selections, pausing/playing of courses, etc.) that the user may perform. Further, the interactive object may also be combined with a page or page object of a Graphical User Interface (GUI) to present different interactive functions.

The page object and the interactive object can be fixed on each page, and can also be displayed in a manner of popup or scrolling playing on each page.

Some exemplary pages

The first scheme is shown in fig. 2, 3, 4, 5-a and 5-b, wherein fig. 2 is a first main page, fig. 3 is a page switching page, and fig. 4, 5-a and 5-b are a second main page, a third main page and a fourth main page, respectively. The first main page displays a plurality of page information through a plurality of page objects, wherein the page information comprises a weather card, a multimedia card, a recommended course card and the like, meanwhile, the first main page also comprises an interactive object which is displayed at the upper right corner and used for entering a page switching page and an interactive object which is combined with the course card, a user can enter a second main page through executing the interaction of the interactive object which is combined with the course card and also can enter the page switching page through executing the interaction of the interactive object which is used for entering the page switching page, the page switching page comprises a preview image of the first main page, the second main page and a preview image of a third main page and highlights one of the pages, and meanwhile, the interactive object which returns and confirms to enter the highlighted page and the interactive object which is displayed below the page switching highlighted are also displayed above the page. The user may switch the highlighted page by performing an interaction that switches the interactive object of the highlighted page and enter the selected page by confirming the interaction of the interactive object that entered the highlighted page. When the user selects and enters the second main page, the second main page comprises a plurality of curriculum cards and highlights one of the curriculum cards, meanwhile, an interactive object for returning and confirming the curriculum entering the highlighted curriculum cards is displayed above the second main page, and an interactive object for switching the highlighted curriculum cards is displayed below the page switching page. When a user selects and enters a third main page, the third main page comprises a plurality of page objects displayed in parallel, the third main page also shows the selected page objects and interaction objects entering the selected page objects, and interaction objects switching the page objects are also shown below the third main page.

The second scheme is shown in fig. 6, 7, 8 and 9, wherein fig. 6 is a fifth main page, fig. 7 is a sixth main page, and fig. 8 and 9 are a seventh main page and an eighth main page, respectively. The fifth main page displays a plurality of page information including weather cards, multimedia cards, recommended course cards and the like through a plurality of page objects, and meanwhile, the fifth main page further comprises an interactive object displayed below and used for switching the main page and an interactive object combined with the course cards, and a user can enter the eighth main page by executing interaction of the interactive object combined with the course cards and can also switch the displayed main page by executing interaction of the interactive object used for switching the main page. When the user selects the main page displayed in the switching way to be the sixth main page, the sixth main page comprises a plurality of page objects displayed in parallel, meanwhile, the sixth main page also shows the selected page object and the interaction object entering the selected page object, and the interaction object used for switching the main page is also shown below the sixth main page. When the user selects the main page to be switched to be displayed as a seventh main page, the seventh main page comprises a plurality of page objects which are displayed in parallel, meanwhile, the seventh main page also shows the selected page object and an interaction object entering the selected page object, and an interaction object used for switching the main page is also shown below the seventh main page. When the user selects the page corresponding to the selected page object as the eighth home page, the eighth home page comprises a plurality of curriculum cards displayed in parallel, the eighth home page also shows the selected curriculum cards and the interactive objects entering the selected curriculum cards, and the interactive objects used for switching the curriculum cards are also shown below the eighth home page.

The page switching page comprises preview pictures of a first main page, a second main page and a third main page, one page is highlighted, meanwhile, an interactive object for returning and confirming to enter the highlighted page is displayed above the page switching page, and an interactive object for switching the highlighted page is displayed below the page switching page. The user may switch the highlighted page by performing an interaction that switches the interactive object of the highlighted page and enter the selected page by confirming the interaction of the interactive object entering the highlighted page. When the user selects and enters the second main page, the second main page comprises a plurality of curriculum cards and highlights one of the curriculum cards, meanwhile, an interactive object for returning and confirming the curriculum entering the highlighted curriculum cards is displayed above the second main page, and an interactive object for switching the highlighted curriculum cards is displayed below the page switching page. When a user selects and enters a third main page, the third main page comprises a plurality of page objects displayed in parallel, the third main page also shows the selected page objects and interaction objects entering the selected page objects, and interaction objects switching the page objects are also shown below the third main page.

Page action effect

The Graphical User Interface (GUI) also includes page animation effects including, but not limited to, page switch animation effects and page interaction animation effects.

The page switching effect mainly aims at the page operation and change effect, and aims to enable the page operation and change to have better user experience when being displayed.

The page interaction effect mainly aims at providing guidance for the user when the user performs interaction, such as prompting the current interaction and the interaction progress of the user and prompting the user to perform required operation when performing interaction. Further details include, but are not limited to, highlighting or flashing the page information involved when the user is interacting, displaying a progress bar on the page information involved, and the like. The method also comprises the steps of identifying the action which is performed by the user and displaying a prompt action or a suggested action and the like when the action control is performed. And when voice control is performed, completion prompting is performed according to a voice instruction of a user.

Intelligent fitness mirror control

The user may directly interface with the intelligent fitness scope to control the intelligent fitness scope (e.g., motion commands, voice commands, touch commands, etc.). A Graphical User Interface (GUI) displayed on the display 120 may be provided to facilitate user interaction with the intelligent exercise mirror.

Action command control (including 'body control' and 'gesture control')

The intelligent fitness mirror can control a Graphical User Interface (GUI) according to user actions acquired by the image recognition sensor.

In particular, actions controlled by action commands include, but are not limited to, static actions and dynamic actions. Static motion refers to a user making a specific body part into a specific posture and maintaining for a specified time; a dynamic motion refers to a user making a specific motion with a specific body part.

Specifically, the static motion includes, but is not limited to, raising left/right hands, raising left/right legs, raising the head of the user with both hands, various static gestures, and the like, and the range of the time for which the predetermined time is maintained is generally 0 to 5 seconds or longer, and any motion determined by the positional relationship between a specific part of the human body with respect to the space, the whole human body, or other specific parts, and having the predetermined time maintained, should fall within the protection range of the static motion. The method is further concretely as follows: the left hand is straightened to form an angle of 15 degrees/30 degrees with the horizontal plane and is maintained for 1.5 seconds, the left hand is kept at a right angle with the right hand for 1 second, the left hand is extended forwards for 0.1 second and the like.

Specifically, the dynamic motion includes, but is not limited to, horizontal sliding left/right hand, up-down sliding left/right hand, double-hand clapping, jumping, squatting, various dynamic gestures, and the like, and any motion determined by the movement trajectory of a specific part of the human body with respect to the space, the whole human body, or other specific parts should fall within the protection range of the dynamic motion. The method is further concretely as follows: sliding the left hand horizontally moves the left hand from left to right, sliding the left hand horizontally moves the left hand from right to left, sliding the right hand horizontally moves the right hand from right to left, sliding the left hand up and down moves the left hand from above the shoulders to below the waist, sliding the left hand up and down moves the left hand from above 20% to below 40% of the entire human body, having both hands at the top of the head, having both hands at the left shoulders, and so on.

In particular, the control instructions executed by the action command control include, but are not limited to, control instructions for non-workout pages (wake page, main page, other pages) and control instructions for workout pages. The control instructions for the non-workout pages relate primarily to the user's instructions of interaction with the non-workout pages. The control instruction for the exercise page mainly relates to the control instruction for the user to play the exercise course

Specifically, the control instructions for the non-exercise page include, but are not limited to, switching of the non-exercise page, previous page, next page, determining, returning, selecting a course, collecting a course, joining a plan, entering a page switching interface, waking up a voice assistant, controlling multimedia volume, and the like.

Specifically, the control instructions for the exercise page include, but are not limited to, pause of a course, play of a course, previous link, next link, background music volume control of a course, volume control of a course coach, evaluation of a course, and the like.

Specifically, the user may also implement different functions through a combination of a plurality of actions. Specific examples include, but are not limited to, using a first action to enter a first level control command menu, and then using a second action to make a command selection in the first level control command menu or to enter a second level control command menu, which may be presented to the user on a display. Further, the specific examples include, but are not limited to, the first level control command menu may be a course selection control command menu, including a command for playing a course, collecting a course, displaying details of the course, reserving a course, and entering the second level control command menu, which may be a course evaluation menu, including a good comment command, a medium comment command, and a bad comment command.

In particular, the method further comprises prompting the user action command control on a Graphical User Interface (GUI). Specific examples include, but are not limited to, displaying an icon of an actionable action on a Graphical User Interface (GUI), marking or highlighting a control command pointed by a user static action on the Graphical User Interface (GUI), displaying a progress bar for maintaining the user static action on the Graphical User Interface (GUI), guiding a user dynamic action on the Graphical User Interface (GUI) when the user dynamic action is completed by more than a specified ratio, and the like.

Specifically, the intelligent fitness mirror can provide auxiliary judgment by depending on other sensors when controlling a Graphical User Interface (GUI) according to user actions acquired by the image recognition sensor. Specific examples include, but are not limited to, turning on/interrupting/confirming motion command control by a voice recognition sensor, judging the front or back of a user by a facial recognition sensor (turning on motion command control only when the user is facing the front of the intelligent fitness scope), distinguishing different users by a facial recognition sensor to load custom motion command control instructions, optimizing motion/trajectory recognition accuracy by an IMU, etc.

Specifically, the motion controlled by the motion command may be a preset motion in the intelligent exercise mirror, or may be a motion recorded by the user. The control instruction executed by the action command control can be an instruction preset in the intelligent fitness mirror or an automatic instruction set by the user. The action controlled by the action command and the executed control command may be a preset corresponding relation, or a corresponding relation set by the user himself, or a one-to-one relation, or a many-to-one relation.

Specifically, the turning on or off of the motion command control function may be determined by a specific Graphical User Interface (GUI) or exercise page, such as turning on the motion command control function by default on a main page of the GUI and turning off the motion command control function by default on other pages; the opening or closing of the action command control function can be determined according to data detected by other sensors; the activation or deactivation of the motion command control function may also be the activation/deactivation of the entire function or the activation/deactivation of a part of the function.

A specific example of the intelligent fitness mirror capable of controlling a Graphical User Interface (GUI) according to a user action acquired by an image recognition sensor is that the intelligent fitness mirror displays a first page of the GUI through a display 120, a first prompt icon for lifting a left hand to enter a page switching mode is displayed on the first page, the left hand is lifted by a user, the first prompt icon enters a highlight state and opens a progress bar, 1.5 seconds later, the progress bar is completed, the intelligent fitness mirror displays a page switching page for switching page display through the display 120, the switching page includes a first page highlighted, a second page of the first page and a third page of a next page of the first page, still show last page on the switching page, next page, the suggestion icon of withdraw from and affirming, wherein the suggestion icon of last page represents the action that the left hand turned right horizontal wave from a left side, the suggestion icon of next page represents the action that the right hand turned left horizontal wave from the right side, the suggestion icon of withdraw from represents and lifts the left hand, the right hand is lifted to definite suggestion icon representation, the user makes this moment the left hand and turns right horizontal wave from a left side, the switching page becomes the highlight second page, the user lifts the right hand this moment, the icon that represents the affirmation gets into high bright state and opens the progress bar, 1.5 seconds later, the progress bar is accomplished, intelligent body-building mirror passes through display 120 and shows the second page.

A specific example of the intelligent fitness scope that can control a Graphical User Interface (GUI) according to a user action acquired by an image recognition sensor is that the intelligent fitness scope first displays a fourth page of the GUI through a display 120, the fourth page displays a second prompt icon representing that a left hand horizontally swings from left to right to enter a previous page and a third prompt icon representing that a right hand horizontally swings from right to left to enter a next page, the user makes a right-to-left horizontal swing motion of the right hand at this time, and the intelligent fitness scope displays a transition from the fourth page to a fifth page through the display 120.

A specific example of the intelligent fitness scope capable of controlling a Graphical User Interface (GUI) according to a user action acquired by an image recognition sensor is that the intelligent fitness scope first displays a third main page as shown in fig. 5-a through a display 120, where the third main page includes a plurality of page objects displayed in parallel, the third main page further displays a selected page object and an interactive object entering the selected page object, and an interactive object switching the page object is further displayed below the third main page, and a user can switch a displayed page object by switching an interaction of the interactive object switching the page object and enter a page corresponding to the selected page object by performing an interaction of the interactive object entering the selected page object. The user can select the selected page object by lifting the right hand/left hand obliquely upwards to the left, obliquely downwards to the left, obliquely upwards to the right and obliquely downwards to the right, and the posture of the right hand/left hand is maintained after the selection is finished, and meanwhile, the hand can enter the page corresponding to the selected page object by making a specified static action, such as opening the five fingers for 1 second. Meanwhile, the user can make a horizontal waving motion from left to right of the right hand to switch the displayed page object and enter a fourth main page.

Further, a specific example of the suspension motion control of the intelligent fitness mirror is that the intelligent fitness mirror first displays a third main page as shown in fig. 5-a through the display 120, the third main page includes page objects arranged and displayed in parallel in four lattices, when the user uses the intelligent fitness mirror, the page object selected by the user is determined according to the position of the right hand of the user, that is, when the right hand of the user is above the right shoulder of the human body, the page object "weather" is selected, when the right hand of the user is below the right shoulder of the human body, the page object "reminder" is selected, when the right hand of the user is above the left shoulder of the human body, the page object "music" is selected, and when the right hand of the user is below the right shoulder of the human body, the page object "body fat scale" is selected. When the right hand of the user acts, the intelligent fitness mirror judges the page object selected by the user according to the identified position of the right hand, and executes preset interaction aiming at the selected page object after the right hand stops moving and a certain time is exceeded, wherein the preset interaction aiming at the selected page object can be a function of opening the page object or a context menu (contextual menu) of the page object approximately. The page objects arranged and displayed in parallel in the four lattices can also be arranged in a left-right arrangement mode, a six-lattice arrangement mode, a nine-lattice arrangement mode and the like.

The actions may be provided with personal preferences (left-right handed, disabled, etc.) while having different modes with one hand and two hands, including left/right hand mode, mode with both hands simultaneously (e.g. left hand selection, right hand screen cut).

The first page, the second page, the third page, the fourth page, and the fifth page may be a wake-up page, a main page, a workout page, or other pages, respectively.

In addition, the control action corresponds to the control instruction, and if one control action is matched, the control instruction matched with the control action is executed on the processor.

Voice command control

The intelligent fitness mirror can control a Graphical User Interface (GUI) according to user speech acquired by the speech recognition sensor.

Specifically, the user can replace the action controlled by the action command and realize the same control effect through the voice recognition result controlled by the voice command.

In particular, the control instructions executed by the action command control include, but are not limited to, control instructions for non-workout pages (wake page, main page, other pages) and control instructions for workout pages. The control instructions for the non-workout page primarily relate to user interaction instructions with the non-workout page. The control instruction for the exercise page mainly relates to the control instruction for the user to play the exercise course

Specifically, the user may also implement different functions through a combination of a plurality of actions. Specific examples include, but are not limited to, using a first voice to enter a first level control command menu, and then using a second voice to make a command selection in the first level control command menu or to enter a second level control command menu, which may be presented to the user on a display. Further specifically, but not limited to, the first level control instruction menu may be a course selection control instruction menu, including instructions for playing a course, collecting a course, displaying details of the course, making a course appointment, and entering a second level control instruction menu, and the second level control instruction menu may be a course evaluation menu, including good-rated, medium-rated, and bad-rated instructions.

In particular, prompting for user voice command control in a Graphical User Interface (GUI) is also included. Specific examples include, but are not limited to, displaying keywords of usable voices on a Graphical User Interface (GUI), completing a user voice command on the Graphical User Interface (GUI) (when the user dynamic motion completion degree exceeds a predetermined ratio), and the like.

Specifically, the intelligent fitness mirror can provide auxiliary judgment by depending on other sensors when controlling a Graphical User Interface (GUI) according to user voice collected by the voice recognition sensor. Specific examples include, but are not limited to, turning on/interrupting/confirming motion command control by a motion recognition sensor, judging the front or back of a user by a facial recognition sensor (turning on motion command control only when the user is facing the front of the intelligent fitness scope), distinguishing different users by a facial recognition sensor to load customized voice command control instructions, etc.

Specifically, the keywords controlled by the voice command may be preset keywords in the intelligent fitness mirror, or may be keywords recorded by the user. The control instruction controlled and executed by the voice command can be an instruction preset in the intelligent fitness mirror or an automatic instruction set by the user. The keywords controlled by the voice command and the executed control instruction may be preset corresponding relations, or may be corresponding relations set by the user, or may be one-to-one relations, or may be many-to-one relations.

Specifically, the voice command control function may be turned on or off according to a specific Graphical User Interface (GUI) or exercise page, such as turning on the voice command control function by default on a main page of the GUI and turning off the action command control function by default on other pages; the voice command control function can be started by identifying a wake-up word; the on or off of the voice command control function can be determined according to data detected by other sensors; the turning on or off of the voice command control function may also be turning on/off of the entire function or turning on/off of part of the function.

A specific example of the intelligent fitness mirror that can control a Graphical User Interface (GUI) according to a user action acquired by an image recognition sensor is that the intelligent fitness mirror displays a first page of the GUI through a display 120, the user turns on a voice command control function through a wakeup word, a fourth page displays a turn-on icon representing the voice command control function, the user speaks an instruction to "enter a switch page" or "page switch" at this time, the intelligent fitness mirror displays a page switch page for switching page display through the display 120, the switch page includes, in addition to the highlighted first page, a second page of the first page and a third page of the first page, the switch page also displays the turn-on icon representing the voice command control function, the user speaks an instruction to "previous page" or "turn forward" at this time, the switch page becomes to highlight the second page, the user speaks an instruction to "enter" or "determine" at this time, and the intelligent fitness mirror displays the second page through the display 120.

A specific example of the intelligent fitness mirror capable of controlling a Graphical User Interface (GUI) according to a user action acquired by an image recognition sensor is that the intelligent fitness mirror displays a fourth page of the GUI through a display 120, the user starts a voice command control function through a wakeup word, a start icon representing the voice command control function is displayed on the fourth page, the user speaks a "next page" or "page turn backward" instruction at the moment, and the intelligent fitness mirror displays a transition from the fourth page to a fifth page through the display 120.

A specific example of the intelligent fitness scope that can control a Graphical User Interface (GUI) according to user actions acquired by an image recognition sensor is that the intelligent fitness scope first displays a sixth page of the GUI on a display 120, a user controls to start a voice command control function through an action command, a fourth page displays a start icon representing the voice command control function, the user speaks an instruction of "start training" or "confirm" at the moment, and the intelligent fitness scope switches from the sixth page to a seventh page through the display 120.

The first page, the second page, the third page, the fourth page and the fifth page can be a wakeup page, a main page, an exercise page or other pages respectively, the sixth page is a page for displaying exercise cards, and the seventh page is an exercise page.

Touch command control

The intelligent fitness mirror can control a Graphical User Interface (GUI) according to a user touch instruction collected by the touch screen.

Multi-modal command control

Multi-modal command control includes combination control, enhanced control, and conflict control.

The combination control includes but is not limited to action combination control of multiple actions and voice combination control of multiple keywords, and more complex control instructions (such as selecting and collecting, selecting and recommending to friends, screenshot and sharing, and the like) can be realized through multiple actions or keywords which are continuous (the interval time does not exceed a threshold).

The combined control comprises but is not limited to enhancing and optimizing action instruction control or voice instruction control of a user through facial recognition or scene recognition, and further comprises but is not limited to recognizing that facial expressions of the user are not happy when the action instruction control or voice instruction control of the user is music playing, presenting a recommendation result of relieving mood for the user and caring for the user; when the action instruction control or the voice instruction control of the user is the recommended course, the scene recognition temperature is low, and the recommendation of meditation courses is reduced.

The conflict control includes, but is not limited to, when the user uses a plurality of different control methods to control the intelligent fitness mirror at the same time, a plurality of conflicting control instructions occur, and the control instructions which the user wants to execute are determined and executed by the intelligent fitness mirror by performing priority judgment on the different control methods.

Control instruction

The control instructions include, but are not limited to, system wake-up instructions, page switching instructions, page interaction instructions, and course control instructions, and the user can control the intelligent fitness mirror through the control instructions.

The system awakening instruction comprises an instruction for awakening the intelligent fitness mirror from a standby state to an awakening page, a main page, an exercise page and other pages and an instruction for switching the intelligent fitness mirror from the awakening page to the main page, the exercise page and other pages.

The page switching instruction is used for controlling the intelligent fitness mirror to switch among a plurality of different pages, and the different pages can be pages of different types or pages with different contents. Further page switching instructions include, but are not limited to, page turning instructions (next/previous) and instructions to control the smart exercise mirror to enter a page switch.

The page interaction instruction refers to an instruction for controlling the intelligent fitness mirror to perform interaction or input, and the page interaction instruction comprises but is not limited to a control interaction instruction and an input interaction instruction. Further, the control interactive instruction refers to an instruction for controlling a Graphical User Interface (GUI) or a page object or an interactive object in the GUI to implement an interactive function thereof, such as confirmation/return, favorite joining, plan joining, voice assistant wake-up, volume control, account switching, mode switching, context menu (contextual menu) opening, and the like. Further, the input interactive instruction refers to an instruction for inputting information in a Graphical User Interface (GUI) or a page object or an interactive object in the GUI, for example, when course evaluation is performed, one core is added every time an instruction is input; or when chatting with other users through the intelligent fitness mirror, switching a preset expression/reply and the like every time an instruction is input

The course control instruction is used for controlling the intelligent fitness mirror to play the exercise video on the exercise page, and is mainly used for controlling the playing of the exercise video. Further course control instructions include, but are not limited to, pause/play, previous/next, background volume control, coach volume control, course rating, etc.

Data management

Data sources of the intelligent fitness mirror include, but are not limited to, sensor data and background data for input of user control instructions and presentation of page information.

The sensor data includes, but is not limited to, on-mirror sensor data and external sensor data, and the on-mirror sensor data includes, but is not limited to, data detected by sensors mounted on the intelligent fitness mirror, such as an image recognition sensor, an infrared recognition sensor, a voice recognition sensor, and the like. More specifically, the data detected by the image recognition sensor includes, but is not limited to, an image detected by the image recognition sensor and data obtained by processing and recognizing the image, such as user motion data, user posture data, facial recognition data, user expression data, environment image data, and the like. More specifically, the data detected by the infrared recognition sensor includes, but is not limited to, a thermal image detected by the infrared recognition sensor and data obtained by processing and recognizing the thermal image, such as user thermal imaging data, user exercise location data, environmental thermal imaging data, and the like. More specifically, the data detected by the voice recognition sensor includes, but is not limited to, the voice data detected by the voice recognition sensor and data obtained by processing and recognizing the voice data, such as a voice control command, a keyword command, a wakeup word command, and the like. The on-mirror sensor data includes, but is not limited to, inertial sensor data, physiological data sensor data (such as heart rate belt data, body fat scale data, etc.), and the like, which are detected by the sensor wirelessly connected with the intelligent fitness mirror. More specifically, the data detected by the inertial sensor includes, but is not limited to, speed/acceleration data detected by the inertial sensor and data obtained by processing and identifying the speed/acceleration data, such as user motion data. More specifically, the data detected by the physiological data sensor includes, but is not limited to, physiological data detected by the physiological data sensor and data obtained by processing and identifying the physiological data, such as heart rate data, blood pressure data, blood oxygen saturation data, and the like.

The background data comprises but is not limited to background user data and other background data, the background user data is data which is stored by the intelligent fitness mirror and/or a server connected with the intelligent fitness mirror and is directly related to a user, and the background user data comprises but is not limited to user physiological data, user social data, user calendar data, use history data and the like. The usage history data includes, but is not limited to, completed course data, collected course data, historical best performance data, physiological data and performance data during the course of the course, frequency data of class attendance, etc. The user physiological data includes, but is not limited to, the user's height, weight, gender, age, heart rate, blood pressure, medical history of injury, and the like. The user social data includes, but is not limited to, data related to people/friends concerned by the user and data related to groups/interest groups joined by the user, and these information may specifically be class invitation data for inviting the user to complete a specified course, challenge invitation data for inviting the user to perform one-to-one or multiple people, ranking among users, and dynamic data uploaded by the user himself or by other users. The dynamics may be synchronized to other social platforms (e.g., weChat, microblog, facebook, twitter, etc.). The user calendar data includes, but is not limited to, schedule information data, to-do data, lesson planning data, etc. of the user. The other data of the background is data which is stored by the intelligent body-building mirror and/or a server connected with the intelligent body-building mirror and has low relevance with the user, and the other data of the background comprises but is not limited to IP data, environment data, course data, third-party data and the like. Wherein the IP data includes, but is not limited to, IP address data, mac address data, etc. of the intelligent fitness mirror. The environment data comprises but is not limited to time data, positioning data, weather data, temperature data, dressing index data and the like of the area where the IP of the intelligent fitness mirror is located. The course data includes, but is not limited to, exercise video data, trainer name, course type, course length, course difficulty, course tags, and/or expected consumption, among others. The types of lessons include, but are not limited to, presenting HIIT to the user through the smart fitness mirror, aerobics, yoga, shaping of strength, fighting exercises, barre, pilates, stretching, dancing, meditation, challenges, and the like. The course labels include but are not limited to whether AI identification courses are presented to the user through the intelligent fitness mirror, whether the AI identification courses are parent-child courses, body parts mainly related to the courses (such as whole body, arms, waist, abdomen and legs) and props (such as no instrument, yoga mats, elastic rings, elastic bands and dumbbells) needed to be used in the courses. The third party data includes, but is not limited to, data for third party functions installed on the intelligent fitness mirror.

Data storage

The smart exercise scope may also store user information locally on the smart exercise scope and/or on a remote storage device (e.g., a cloud service) depending on the amount of storage space used. For example, user information that uses little storage space, including but not limited to the user's name, age, height, weight, and gender, may be stored locally on the intelligent fitness mirror. In addition, lesson data may also be stored in the intelligent fitness mirror to reduce the impact of network latency that may affect the quality of the video stream. The amount of stored video content may be limited by the storage capacity of the intelligent fitness mirror. In some configurations, the video content may be temporarily stored only daily or weekly, or depending on the percentage of the intelligent fitness scope capacity used. Background user data using a large amount of storage space may be stored on the remote storage device, including but not limited to physiological data such as the user's heart rate and calories burned and video or skeletal point data of the user taken during exercise. The intelligent fitness scope may retrieve this information for subsequent analysis and display.

Data transmission between the intelligent exercise mirror and the remote storage device may be protected (e.g., encrypted) in various ways to prevent unwanted loss or theft of user information. For example, the bluetooth low energy protocol includes built-in security features that may be used by devices utilizing the protocol. However, these security features can only be used if the bluetooth binding step is completed before the connection is established with the encryption. In some cases, various security mechanisms may not be implemented or may fail, at which point application level security may be implemented in conjunction with the above-described chunking specification of data. For example, advanced Encryption Standard (AES) encryption of the message may be applied before the preamble of the preamble message. In some aspects, the bluetooth low energy protocol performs similar processes via built-in security features at the firmware level, and may provide similar protection to prevent people from reading communications between the client and the server.

The GATT service added for the client to read/notify messages may be deleted from the service recording on the server device when the client is disconnected from the server. This ensures that no connections are opened and the system does not accidentally leak information to nefarious snoopers. Such connection termination may be triggered by the server or client and rely on the bluetooth low energy stack to provide notification to both parties that the connection has been closed. If bluetooth binding is used in the initial connection setup to provide firmware level encryption security, binding information may be stored on each device so that repeated bindings are not required after subsequent connections between the client and the server.

Function management

The intelligent fitness mirror can also realize the following functions according to the received data: user identification, mode switching, attention judgment, body/emotion assessment and dressing recommendation.

The user identification function allows the intelligent body-building mirror to identify a user using the intelligent body-building mirror through input data, and the function is mainly realized through the following method: the user inputs the characteristic data of the user through the data input device, the intelligent fitness mirror compares the characteristic data input by the user with the characteristic data in the database, the user using the intelligent fitness mirror is determined according to the comparison result, and the intelligent fitness mirror is allowed to adjust a Graphical User Interface (GUI) according to the identification result, such as adjusting the font size, page information preference and the like. The data input device comprises on-mirror sensor data and an external sensor, the characteristic data of the user comprises on-mirror sensor data such as bone points, voiceprints, weight, resting heart rate and the like of the user and data obtained by processing after the data are collected or collected by the external sensor, and the mapping relation between the user and the characteristic data is recorded in the database. Further, the user identification function may be used for identification and switching of a plurality of sub-account numbers under one primary account number, and may also be used for identification and switching between a plurality of primary account numbers. In another example, the user identification function allows the intelligent exercise scope to identify characteristics of a user who is using the intelligent exercise scope, such as a male user, a female user, an old person, and a child, through the input data, and allows the intelligent exercise scope to adjust a Graphical User Interface (GUI), such as adjusting font size, page information, and the like, according to the identification result.

The mode switching function allows the intelligent fitness mirror to be switched among a plurality of working modes to adapt to different working scenes. For example, the intelligent fitness mirror is allowed to be switched into a teenager mode, in the teenager mode, a teenager account is bound with a parent account or a device main account, in the teenager mode, page information displayed by the intelligent fitness mirror can be adjusted adaptively, for example, course recommendation can be adjusted to preferentially present contents suitable for teenagers, such as rope skipping and motion sensing games, and meanwhile, real-time conditions and motion data in the mode can be synchronized to an intelligent terminal of the parent account or the device main account. In another example, the intelligent fitness mirror is allowed to be switched to a visitor mode, the intelligent fitness mirror can work without being bound with an account in the visitor mode, page information displayed by the intelligent fitness mirror can be adjusted adaptively, if course recommendation is performed, the content suitable for novices can be adjusted to be presented preferentially, if the difficulty label is low, the content is the same, and meanwhile, even if the intelligent fitness mirror is bound with the account, motion data in the mode cannot be counted into the bound account, and a training plan or a class completion condition of the bound account cannot be influenced.

Wherein the attention determination allows the intelligent exercise mirror to determine whether the user's attention is on the intelligent exercise mirror while the device is in operation, such that the intelligent exercise mirror determines the user's intent to interact with the mirror while the sensor, other number input devices, and functions are activated or deactivated. The attention judgment is mainly to judge whether the user wants to use the intelligent fitness mirror according to the performance of the user, and can be judged by the following characteristics: distance judgment (the distance between the user and the intelligent fitness mirror does not exceed a threshold value), posture judgment (whether the user faces the intelligent fitness mirror in front), action judgment (whether the user performs a specified action or an action commonly used for exercise), semantic judgment (whether a recognition result of the voice of the user is related to a specific keyword) and the like.

Wherein the body/emotion assessment allows the intelligent fitness scope to determine a body/emotion condition of the user when using the device, such that the intelligent fitness scope is operable to adjust page information or other feedback according to the body/emotion condition of the user. The physical/emotional condition of the user can be recognized by a physiological data sensor and an image recognition sensor (face recognition). When the emotion of the user is recognized to be out of mind, a page information recommendation result of relieving the emotion is presented to the user, and the user is concerned during voice interaction; in a further example, when the user awakens the voice function of the intelligent fitness mirror through the awakening word, the intelligent fitness mirror recognizes that the expression of the user is depressed through the image recognition sensor (face recognition), and the intelligent fitness mirror initiates a first turn of conversation "hey" to make you look not too happy and not to listen to a song to relax. The body/emotion condition identification can be used for identifying the body/emotion states of the user, such as emotion, stress, fatigue and the like through identification of information of the user, such as age, body posture, expression and the like; the identification can also be carried out through blood pressure and heart rate data detected by a physiological data sensor.

Wherein the dressing recommendation allows the intelligent fitness scope to initiate active interaction to the user based on certain conditions. The intelligent body-building mirror judges whether the user needs the intelligent body-building mirror to initiate active interaction or not according to the acquired time data and positioning data and by combining the user information and the action of the user in front of the intelligent body-building mirror. For example, when the user appears in front of the intelligent fitness mirror for the first time in the morning and turns a circle, the intelligent fitness mirror starts the dressing recommendation function, and clothes which are positioned at present and are suitable for being worn today are recommended to the user. Further, when the user appears in front of the intelligent fitness mirror and turns a circle at the end of the week, the intelligent fitness mirror starts the clothes-wearing recommendation function and outputs the clothes-wearing suggestion of the current clothes to the user. The functions can also be used for other functions of the intelligent body-building mirror for active interaction based on the user and external information.

An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the multi-modal interactive exercise scope when executing the computer program.

The processor may be a central processing unit, or may be other general-purpose processor, a digital signal processor, an application specific integrated circuit, an off-the-shelf programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory may be used to store the computer programs and/or modules, and the processor may be used to implement the various functions of the apparatus of the multimodal interactive exercise apparatus of the invention by operating or executing the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, at least one magnetic disk storage device, a flash memory device, or other volatile solid state storage device.

Video streaming method

The smart exercise mirror is configured to display video content from the video production facility on the display surface 120. The video content may be streamed as live content or prerecorded recorded content. Live content may also be recorded and stored in the cloud server so that the user can later request and play video content, thus becoming recorded and played content. When video streaming is carried out, firstly, the intelligent fitness mirror sends a request of a specified course to the API server, data returned by the API server are divided into two parts, and the first part is overview of the specified course (including course pictures and course brief introduction); the second part is the address (URL) of the course video, and then the intelligent fitness mirror requests the corresponding course video on the OSS server according to the received URL. When the course is recorded and broadcast content, the intelligent fitness mirror plays videos by using an HLS protocol, namely, the address (URL) of the course video is an M3U8 file of the corresponding course, and then the player of the intelligent fitness mirror downloads the corresponding course video slice from the OSS server according to the record of the M3U8 file. When the course is live broadcast content, the intelligent body-building mirror plays videos by using an RTMP protocol, namely, the address (URL) of the course video is the RTMP live broadcast address of the corresponding course, and then the player of the intelligent body-building mirror downloads the corresponding course video live broadcast data from the OSS server according to the RTMP live broadcast address.

In some applications, the intelligent gym mirror may be connected to an online streaming service that provides the user with third-party video content that is streamed from a server (e.g., directly through a network router or indirectly through the user's intelligent device). Third party content may be provided to the user based on the subscription. A third party may provide content to a centralized distribution platform that communicates with the intelligent fitness mirror over a network. One benefit of a centralized distribution platform is that the distribution of content to the intelligent fitness scope is simpler. Alternatively, a third party may develop a separate distribution platform that may use a separate software application on the smart device for the user to access the content.

A computer readable storage medium storing a computer program which when executed by a processor implements the steps of the multi-modal interactive exercise mirror.

Computer storage media for embodiments of the present invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A method for recognizing an action of a user,

as shown in fig. 10, based on the above embodiment, the apparatus specifically includes the following steps in the interaction process:

s121, acquiring a user image in an area by the body-building mirror to obtain a motion image of the user;

s122, generating skeleton point information of the user according to the action image, and establishing a coordinate system according to the skeleton point information;

s122.1, the skeleton point information comprises a vertex skeleton point, a pelvis skeleton point, a left-hand skeleton point and a right-hand skeleton point;

s122.2, acquiring a linear distance between a vertex bone point and a pelvis bone point, and acquiring the unit length of a coordinate system according to the linear distance;

s122.21, acquiring a linear distance between the vertex bone point and the pelvis bone point, and taking one sixth of the linear distance as the unit length of the coordinate system, namely the unit length 1 of the coordinate system is one sixth of the linear distance;

s122.3, taking a straight line between the vertex bone point and the pelvis bone point as a longitudinal coordinate, taking the pelvis bone point as an origin, and constructing a rectangular coordinate system by using the obtained unit length;

s123, establishing a plurality of interaction areas according to the coordinate system;

in this embodiment, there are five interactive areas, which are respectively a first interactive area, a second interactive area, a third interactive area, a fourth interactive area and a fifth interactive area, as shown in fig. 4, where the first interactive area is an area a in fig. 3, the second interactive area is an area B in fig. 4, the third interactive area is an area C in fig. 4, the fourth interactive area is an area D in fig. 4, and the fifth interactive area is an area E in fig. 4.

The first interactive area takes the vertex bone point as the center, the longitudinal length is 5 units, the transverse length is 6 units as the first interactive area, namely the length of the first interactive area is 12 units, and the height is 10 units. The first interactive area takes a head top bone point as a center, the longitudinal upward 5 unit lengths are taken as the upper side, and the longitudinal downward 5 unit lengths are taken as the lower side; the vertex bone point is taken as the center, the transverse leftward 6 unit lengths are taken as the left side, and the transverse rightward 6 unit lengths are taken as the right side. The control instruction preset in the first interaction area is the determination A, namely when the interaction gesture is located in the first interaction area, the control instruction corresponding to the first interaction area is output, namely the control instruction for determining the A is output.

The control instruction of the second interactive area is leftward, the control instruction of the third interactive area is rightward, the second interactive area is an area which takes a pelvic bone point as a starting point, the length of 2 units upwards in the longitudinal direction is the upper side, the length of 10 units downwards in the longitudinal direction is the lower side, the bone point at the top of the head is taken as the starting point, the length of 3 units leftwards in the transverse direction is the right side, the length of 16 units leftwards in the transverse direction is the left side and is 13 units, and the height of the area is 12 units.

The third interactive region is a region which takes a pelvic bone point as an initial point, takes 2 unit lengths upwards in the longitudinal direction as an upper side, takes 10 unit lengths downwards in the longitudinal direction as a lower side, takes a vertex bone point as an initial point, takes 3 unit lengths to the right in the transverse direction as a left side, takes 16 unit lengths to the right in the transverse direction as a right side, has a length of 13 unit lengths, and has a height of 12 unit lengths.

The control instruction of the fourth interactive area is determination D, the control instruction of the fifth interactive area is determination E, the fourth interactive area is an area which takes the bone point at the top of the head as a starting point, the length of 4 units upwards longitudinally is the upper side, the length of 5 units downwards longitudinally is the lower side, the bone point at the top of the head is the starting point, the length of 7 units leftwards transversely is the right side, the length of 16 units leftwards transversely is the left side is 11 units, and the height is 9 units.

The fifth interactive region is a region which takes the bone point at the top of the head as an initial point, takes 4 unit lengths in the vertical upward direction as an upper side, takes 5 unit lengths in the vertical downward direction as a lower side, takes the bone point at the top of the head as an initial point, takes 7 unit lengths in the horizontal rightward direction as a left side, takes 16 unit lengths in the horizontal rightward direction as a right side, has a length of 11 unit lengths, and has a height of 9 unit lengths.

S124, identifying an interaction gesture according to the skeleton point information;

s124.1, acquiring the distance between the left-hand bone point and the right-hand bone point;

s124.2, comparing the distance between the left-hand bone point and the right-hand bone point with a threshold T, if the distance between the left-hand bone point and the right-hand bone point is smaller than or equal to the threshold T, identifying an interactive gesture, wherein the threshold T is 15cm in the embodiment, and if the distance between the left-hand bone point and the right-hand bone point is smaller than or equal to the threshold T, judging that the user makes the interactive gesture, and identifying the interactive gesture, for example, the interactive gesture is a clapping, namely, the user makes a clapping action in the embodiment; if the distance between the left-hand skeleton point and the right-hand skeleton point is greater than the threshold value T, the user does not perform a clapping action; the specific interaction gesture in this embodiment is only for illustration and is not limited.

S124.3, when the user makes an interaction gesture, acquiring the coordinates of the middle point between the left-hand skeleton point and the right-hand skeleton point;

s125, judging whether the middle point coordinate is positioned in the interaction area;

s125.1, judging whether the middle point coordinate is positioned in the interaction area;

s125.11, if the middle point coordinate is located in one of the interactive areas, the interactive area is in an activated state, and a control instruction corresponding to the interactive area is output; other interactive areas are in an inactivated state;

s125.12, outputting a control instruction corresponding to the interactive area, wherein the interactive area is in an activated state, and the rest interactive areas are in an inactivated state; the area of the interaction region in the activated state is increased, and the area of the interaction region in the inactivated state is reduced.

In this embodiment, the length of the active interaction region is increased by 1.5 unit lengths, and the height is increased by 0.5 unit lengths, specifically, in this embodiment, the length of the active interaction region is sequentially increased to the left and right sides, i.e., is extended by 0.75 unit lengths, and the height is sequentially increased to the upper and lower ends, i.e., is extended by 0.25 unit lengths. The length of the interactive region in the inactive state is reduced by 2 units, and the height is reduced by 1 unit. Specifically, the length of the interaction region in the inactive state is sequentially reduced towards the left and right sides, i.e. is reduced inwards by 1 unit length, and the height is reduced towards the upper end and the lower end, i.e. is reduced inwards by 0.5 unit length.

And when the control instruction corresponding to the interactive region is not output, the area of the interactive region is not changed, the area of the interactive region is adjusted after the control instruction is output, and the area of the interactive region is restored to the original state after the control instruction is executed.

After the user makes an interactive gesture in the first interactive region, the fitness mirror recognizes the interactive gesture, outputs a control instruction corresponding to the first interactive region, and enlarges the area of the first interactive region, namely, the upper side is longitudinally and upwardly increased by 0.25 unit length, the lower side is longitudinally and downwardly increased by 0.25 unit length, the left side is transversely and leftwards increased by 0.75 unit length, and the right side is transversely and rightwards increased by 0.75 unit length.

All parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the teachings of the present invention is/are used. It is to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, embodiments of the invention may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein.

In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, if such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the scope of the present disclosure. Other substitutions, modifications, changes and omissions may be made in the design, operating conditions and arrangement of the corresponding elements of the exemplary embodiments without departing from the spirit of the present disclosure. The use of numerical ranges does not exclude equivalents which fall outside the ranges that perform the same function in the same way to produce the same result.

The above-described embodiments can be implemented in various ways. For example, embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code can be executed on a suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.

Further, a computer may be embodied in any of a number of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer. Additionally, a computer may be embedded in a device not normally considered a computer but with suitable processing capabilities, including a Personal Digital Assistant (PDA), a smart phone, or any other suitable portable or stationary electronic device.

In addition, a computer may have one or more input and output devices. These devices may be used to present, among other things, a user interface. Examples of output devices that may be used to provide a user interface include printers or display screens for visual presentation of output and speakers or other sound-generating devices for audible presentation of output. Examples of input devices that may be used for the user interface include keyboards, and pointing devices, such as mice, touch pads, and digitizing tablets. As another example, a computer may receive input information through speech recognition or other audible format.

Such computers may be interconnected IN a suitable form by one or more networks, including a local area network or a wide area network, such as an enterprise network, an Intelligent Network (IN), or the Internet. Such networks may be based on suitable technologies, may operate according to suitable protocols, and may include wireless networks, wired networks, or fiber optic networks.

The various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and/or programming or scripting tools, and may also be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine. Some embodiments may specifically employ one or more of a particular operating system or platform and a particular programming language and/or scripting tool to facilitate execution.

Furthermore, various inventive concepts may be embodied as one or more methods, at least one example of which has been provided. The actions performed as part of the method may in some cases be ordered differently. Accordingly, in some inventive embodiments, the respective actions of a given method may be performed in an order different than that specifically illustrated, which may include performing some actions concurrently (even though such actions are illustrated as sequential actions in the exemplary embodiments).

All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety.

All definitions, as defined and used herein, should be understood to control dictionary definitions, definitions in documents incorporated by reference, and/or ordinary meanings of the defined terms.

The indefinite articles "a" and "an" as used in this specification and in the claims are to be understood as meaning "at least one" unless clearly indicated to the contrary.

The phrase "and/or" as used in this specification and claims should be understood to mean "one or two" elements so combined, that is, elements that are present in combination in some cases and are present in isolation in other cases. Multiple elements listed with "and/or" should be construed in the same manner, i.e., "one or more" elements so combined. Other elements may optionally be present in addition to the elements specifically identified by the "and/or" clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, when used in conjunction with open-ended language such as "including," references to "a and/or B" may refer in one embodiment to a alone (optionally including elements other than B); in another embodiment, only B (optionally including elements other than a); in yet another embodiment, refers to both a and B (optionally including other elements); and so on.

As used in this specification and claims, "or" should be understood to have the same meaning as "and/or" as defined above. For example, when separating items in a list, "or" and/or "should be interpreted as being inclusive, i.e., including at least one, but also including more than one, of the plurality or list of elements, and (optionally) additional unlisted items. To the extent that contradictory statements are made, such as "only one" or "exactly one" or "consisting of" when used in a claim, shall mean that there is exactly one element in a number or list of elements. In general, as used herein, the term "or" followed by an exclusive term such as "one of the two", "one of the", "only one of", or "exactly one of" should only be construed to indicate an exclusive alternative (i.e., "one or the other but not both"). "consisting essentially of" is used in the claims in its ordinary sense as used in the patent law field.

As used in this specification and claims, the phrase "at least one," in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each of the elements specifically listed in the list of elements, and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements referred to by the phrase "at least one," whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, "at least one of a and B" (or, equivalently, "at least one of a or B," or, equivalently "at least one of a and/or B") can refer in one embodiment to at least one a, optionally including more than one a, with no B present (and optionally including elements other than B); in another embodiment, to at least one B, optionally including more than one B, with no a present (and optionally including elements other than a); in yet another embodiment, reference to at least one a, optionally includes more than one a and at least one B, optionally includes more than one B (and optionally includes other elements); and so on.

In the claims, as well as in the specification above, all transitional phrases such as "comprising," including, "" carrying, "" having, "" containing, "" involving, "" holding, "" consisting of,. Or the like are to be construed as open-ended, i.e., including but not limited to.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the invention.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims

1. An interactive intelligent fitness mirror device, comprising:

the image recognition sensor is used for acquiring action data of a user;

2. The interactive intelligent fitness mirror device of claim 1, wherein performing the specified operation on the user interface further comprises controlling a page object corresponding to the interactive object to switch or performing the specified operation on the corresponding page object.

3. The interactive intelligent fitness mirror device of claim 1, wherein the user actions collected by the sensors comprise static actions and dynamic actions, the static actions are defined by the user making a predetermined gesture on a predetermined body part for a specified time, and the dynamic actions are defined by the user making a predetermined gesture on a predetermined body part.

4. The interactive smart fitness mirror device of claim 3, wherein the static actions comprise lifting the left/right hand for a predetermined time, lifting the left/right leg for a predetermined time, and lifting the hands over the top of the head for a predetermined time.

5. The interactive intelligent fitness mirror device of claim 3, wherein the dynamic actions include sliding left/right hand horizontally, sliding left/right hand up and down, clapping both hands, jumping up and squatting down.

6. The interactive intelligent fitness mirror device of claim 1, wherein the user interface comprises an exercise page and a non-exercise page, and the control instructions comprise control instructions for the exercise page and control instructions for the user interface of the non-exercise page.

7. The interactive smart fitness mirror device of claim 6, wherein the non-workout pages comprise a wake-up page, a home page, or other pages; the control instructions for the user interface of the non-workout page include user interaction instructions with the non-workout page, where the interaction instructions include at least one of entering a previous page, entering a next page, determining, returning, selecting a course, collecting a course, joining a plan, entering a page switch interface, waking a voice assistant, and controlling multimedia volume.

8. The interactive intelligent fitness mirror device of claim 6, wherein the control instructions for the exercise page comprise user control instructions for playing of an exercise session, including at least one of session pause, session play, previous session, next session, session background volume control, session coach volume control, and session assessment.

9. An interactive intelligent fitness mirror device according to claim 3, wherein the controls are pre-configured in the interactive mirror device or customized for the user.