CN114333055A

CN114333055A - Body-building mirror, image recognition method, equipment and medium

Info

Publication number: CN114333055A
Application number: CN202111619165.3A
Authority: CN
Inventors: 李广琴; 胡雁; 刘晓潇; 马琳杰; 孙浩
Original assignee: Hisense Group Holding Co Ltd
Current assignee: Hisense Group Holding Co Ltd
Priority date: 2021-12-27
Filing date: 2021-12-27
Publication date: 2022-04-12

Abstract

The embodiment of the application provides a fitness mirror, an image identification method, equipment and a medium, which are used for assisting a user in fitness. In the embodiment of the application, the controller of the fitness mirror can acquire the position information of each preset skeleton key point in the image according to the received image acquired by the target image acquisition unit and the bone key point recognition model trained in advance, determine the target motion parameter according to the motion parameter determination mode corresponding to the video frame of the video displayed by the display when the image is received and the position information of each preset skeleton key point, and output the preset reminding information when the target motion parameter is not within the standard motion parameter range corresponding to the video frame of the video displayed by the display when the image is received, so that the fitness mirror can assist the user in fitness.

Description

Body-building mirror, image recognition method, equipment and medium

Technical Field

The application relates to the technical field of intelligent household equipment, in particular to a fitness mirror, an image identification method, equipment and a medium.

Background

With the improvement of living standard of people, the requirement of people on stature is higher and higher, and the fitness mirror refers to a mirror which can be used by a user to observe self action in a fitness room or in a fitness process at home, is generally a plane mirror, has a smooth surface and the capability of reflecting light, and is often embedded with a metal, plastic or wood frame.

The user observes the body-building action of oneself according to body-building mirror. However, the ordinary body-building mirror cannot actively prompt and lack interaction when the body-building action of the user is not standard; in addition, with the continuous development of the interconnection of everything, the common body-building mirror can not meet the higher and higher body-building requirements of people, and has single function and small auxiliary effect on body building.

Disclosure of Invention

The embodiment of the application provides a fitness mirror, an image identification method, a device, equipment and a medium, which are used for assisting a user in fitness.

In a first aspect, embodiments of the present application provide an exercise mirror, which includes:

the target image acquisition unit is used for acquiring images;

a display for displaying video;

a controller to perform:

acquiring position information of each preset skeleton key point in the image according to the received image acquired by the target image acquisition unit and a bone key point identification model which is trained in advance; acquiring a standard motion parameter range and a motion parameter determining mode corresponding to a video frame of the video when the image is received; and determining a target motion parameter according to the position information of each preset skeleton key point and the motion parameter determination mode, judging whether the target motion parameter is within the standard motion parameter range, and if not, outputting preset reminding information.

In a second aspect, an embodiment of the present application further provides an image recognition method, where the method includes:

acquiring position information of each preset skeleton key point in the image according to the received image acquired by the target image acquisition unit and a bone key point identification model which is trained in advance;

acquiring a standard motion parameter range and a motion parameter determining mode corresponding to a video frame of the video when the image is received; and determining a target motion parameter according to the position information of each preset skeleton key point and the motion parameter determination mode, judging whether the target motion parameter is within the standard motion parameter range, and if not, outputting preset reminding information.

In a third aspect, an embodiment of the present application further provides an image recognition apparatus, where the apparatus includes:

the acquisition module is used for acquiring the position information of each preset skeleton key point in the image according to the received image acquired by the target image acquisition unit and the bone key point recognition model which is trained in advance;

the processing module is used for acquiring a standard motion parameter range and a motion parameter determining mode corresponding to a video frame of the video when the image is received; and determining a target motion parameter according to the position information of each preset skeleton key point and the motion parameter determination mode, judging whether the target motion parameter is within the standard motion parameter range, and if not, outputting preset reminding information.

In a fourth aspect, the present application further provides an electronic device, which at least includes a processor and a memory, and the processor is configured to implement the steps of any of the image recognition methods when executing the computer program stored in the memory.

In a fifth aspect, the present application further provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps of any of the image recognition methods described above.

In the embodiment of the application, the controller of the fitness mirror can acquire the position information of each preset skeleton key point in the image according to the received image acquired by the target image acquisition unit and the bone key point recognition model trained in advance, determine the target motion parameter according to the motion parameter determination mode corresponding to the video frame of the video displayed by the display when the image is received and the position information of each preset skeleton key point, and output the preset reminding information when the target motion parameter is not within the standard motion parameter range corresponding to the video frame of the video displayed by the display when the image is received, so that the fitness mirror can assist the user in fitness.

Drawings

In order to more clearly illustrate the embodiments of the present application or the implementation manner in the related art, a brief description will be given below of the drawings required for the description of the embodiments or the related art, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art according to the drawings.

FIG. 1 is a schematic view of an exercise mirror according to an embodiment of the present disclosure;

fig. 2a is a schematic image diagram provided in an embodiment of the present application;

fig. 2b is a schematic diagram illustrating location information of preset bone key points of an image according to an embodiment of the present disclosure;

FIG. 2c is a schematic view of another image provided by an embodiment of the present application;

fig. 2d is a schematic diagram illustrating location information of preset bone key points of an image according to an embodiment of the present disclosure;

FIG. 3a is a schematic view of a game mode exercise mirror according to an embodiment of the present disclosure;

fig. 3b is a schematic diagram of an image acquired by a target image acquisition unit according to an embodiment of the present disclosure;

FIG. 4a is a schematic diagram of an image capturing unit 11 disposed on the top of an exercise mirror according to an embodiment of the present disclosure;

FIG. 4b is a schematic diagram of two image capturing units 11 disposed on the exercise mirror according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of skeletal key points provided in an embodiment of the present application;

FIG. 6 is a schematic diagram of a body-building mirror for video-based body-building with friends according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a display 12 showing an average completion score, action achievement times and calories burned according to an embodiment of the present application;

FIG. 8 is a schematic view of an auxiliary exercise process of the exercise mirror according to an embodiment of the present disclosure;

fig. 9 is a schematic diagram of an image recognition process according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of an image recognition apparatus according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to assist a user in exercising, the embodiment of the application provides an exercise mirror, an image recognition method, an image recognition device, equipment and a medium.

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and/or software code that is capable of performing the functionality associated with that element.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

In the actual use process, the controller of the body-building mirror can acquire the position information of each preset skeleton key point in the image according to the received image acquired by the target image acquisition unit and the bone key point recognition model trained in advance, and can acquire the standard motion parameter range and the motion parameter determination mode corresponding to the video frame of the video displayed by the display when the image is received; and determining target motion parameters according to the position information of each preset skeleton key point and the motion parameter determination mode, and outputting preset reminding information if the target motion parameters are not within the standard motion parameter range, so as to realize the assistance of body building of the user.

Fig. 1 is a schematic structural view of an exercise mirror provided in an embodiment of the present application, where the exercise mirror includes:

a target image acquisition unit 11, wherein the target image acquisition unit 11 is used for acquiring images;

a display 12, the display 12 for displaying video;

a controller 13, the controller 13 being configured to perform:

acquiring position information of each preset skeleton key point in the image according to the received image acquired by the target image acquisition unit 11 and a bone key point identification model which is trained in advance; acquiring a standard motion parameter range and a motion parameter determining mode corresponding to a video frame of the video when the image is received; and determining a target motion parameter according to the position information of each preset skeleton key point and the motion parameter determination mode, judging whether the target motion parameter is within the standard motion parameter range, and if not, outputting preset reminding information.

In order to realize the control of the body-building mirror, a display 12 and a controller 13 are arranged in the body-building mirror, wherein the display 12 is connected with the controller 13. In one possible embodiment, the display 12 of the exercise mirror may be a mirror-integrated display 12, i.e., a mirror display 12, the mirror-integrated display 12 is embodied by providing mirror material on the outer surface of the display 12, and when the display 12 is closed, the exercise mirror is similar to a normal mirror and the user can view his posture; when display 12 is illuminated, display 12 may display content that intelligently interacts with the user.

Wherein, the user can also select the display mode of the display 12, such as a full-screen display mode or a half-screen display mode, if the user selects the half-screen display mode, the upper half of the display 12 of the fitness mirror can display the content intelligently interacted with the user, the lower half of the display 12 is similar to a common mirror and can be used for tidying appearance and the like, the lower half of the display 12 of the intelligent cosmetic mirror can display the content intelligently interacted with the user, the upper half of the display 12 is similar to the common mirror, and the user can be used for viewing the posture and the like of the user; if the user selects the full-screen display mode, the content intelligently interacted with the user can be displayed in the full screen mode.

In order to realize the assistance of the user in body building, a target image acquisition unit 11 for acquiring images is further arranged in the body building mirror, wherein the target image acquisition unit 11 is connected with a controller 13, and when the target image acquisition unit 11 acquires the images, the images are sent to the controller 13.

In this embodiment of the application, when the user has a fitness requirement, the display 12 of the fitness mirror may be used to play a video, where the video is a fitness video, and the user follows the fitness action in the video played in the display 11 to learn. The controller 13 determines whether the user's exercise movement is accurate according to the image acquired by the target image acquisition unit 11 and the content currently played by the display 12, and if not, outputs preset reminding information to assist the user in exercising.

In order to identify whether the exercise of the user is accurate, a trained bone key point identification model is stored in the controller 13 in advance, and the controller 13 identifies the received image acquired by the target image acquisition unit 11 based on the bone key point identification model. Specifically, the controller 13 may input the received image into a bone key point recognition model trained in advance, and obtain an output of the bone key point recognition model, where the output is position information of each preset bone key point in the image. In order to conveniently know which skeleton key points are output, the model outputs the corresponding identifiers of the preset skeleton key points according to the position information of each preset skeleton key point, wherein the identifiers can be information for uniquely identifying the skeleton key points, such as 1, 2, 3 and the like.

After the position information of each preset bone key point in the image is acquired, the controller 13 determines whether the action corresponding to the image is accurate according to the action followed by the user when the image is received. Since the action that the user learns in the image corresponds to the video frame of the video played by the display 12 when receiving the image, the electronic device acquires the video frame of the video played by the display 12 when receiving the image, and determines whether the current action of the user is standard.

In addition, whether an action is a criterion is typically evaluated during exercise according to two criteria, respectively: the distance between one skeleton key point and another skeleton key point is a preset distance; the angle formed between the connecting line of two skeleton key points and the connecting line of the other two skeleton key points reaches a preset angle. In the embodiment of the application, each set of motion is pre-specified with a standard parameter range to be reached, and whether the motion is standard or not is detected by calculating the position or the angle of the corresponding bone key point.

In order to determine whether the exercise of the user is standard, the motion parameter range and the motion parameter determination mode corresponding to each video frame are pre-stored in the video, so that after the controller 13 receives the image, the currently played video frame can be determined, and the standard motion parameter range and the motion parameter determination mode corresponding to the video frame are obtained. The motion parameter determining method may be determining a distance between a certain bone key point and another bone key point, or determining an angle between a connecting line corresponding to the certain two bone key points and a connecting line corresponding to the other two bone key points. After obtaining the motion parameter determination mode, the controller 13 determines the target motion parameter according to the position information of each preset bone key point of the image and the motion parameter determination mode.

After determining the target motion parameter corresponding to the image, the controller 13 determines whether the target motion parameter is within a standard motion parameter range corresponding to the video frame, if the target motion parameter is within the standard motion parameter range corresponding to the video frame, it indicates that the current exercise action of the user is standard, and if the target motion parameter is not within the standard motion parameter range corresponding to the video frame, it indicates that the current exercise action of the user is not standard enough, and outputs preset reminding information.

The preset reminding information can be 'current action is not enough standard', 'please keep fitness action standard' and the like, the preset reminding information can be voice information or text information, if the preset reminding information is voice information, a playing unit used for playing the voice information can be arranged in the fitness mirror, the playing unit is connected with the controller 13, the controller 13 controls the playing unit to play the preset reminding information, and if the preset reminding information is text information, the controller 13 can control the display 12 to display the preset reminding information. Therefore, in the embodiment of the application, the user can learn according to the movement of the fitness video in the display, and the controller 13 can judge whether the movement of the user is standard or not according to the position information of the preset skeleton key points in the collected image, so that the user can timely correct the fitness movement through real-time feedback.

In addition, in order to enable the user to accurately correct the exercise, the preset reminding information may also be reminding information that is correspondingly determined by the controller 13 according to the video frame, the target motion parameter and the standard motion parameter range. For example, a first reminding message that the target motion parameter corresponding to the video frame is greater than the maximum value of the standard motion parameter range and a second reminding message that the target motion parameter corresponding to the video frame is less than the minimum value of the standard motion parameter range may be stored in advance, and the controller 13 may determine what reminding message to output according to the corresponding relationship between the target motion parameter and the standard motion parameter range.

In the embodiment of the application, more than one standard motion parameter range and motion parameter determination mode corresponding to a video frame may be provided, if the standard motion parameter range and motion parameter determination mode corresponding to the video frame are multiple, whether the corresponding target motion parameter is located in the standard motion parameter range is sequentially determined, and if a certain target motion parameter is not located in the corresponding standard motion parameter range, corresponding reminding information is output to prompt a user of a part which does not meet the standard, so that the user can achieve a better body building effect.

After the follow-up learning mode can be selected for the user, the controller correspondingly executes the process to determine whether the action of the user is standard or not.

Fig. 2a is a schematic image diagram according to an embodiment of the present disclosure. Fig. 2a is an image input into the bone key point recognition model, wherein each dot is each preset bone key point.

Fig. 2b is a schematic diagram of position information of preset bone key points of an image according to an embodiment of the present application. Fig. 2b is the position information of each preset bone key point and the corresponding identification of the bone key point output by the bone key point recognition model provided in the embodiment of the present application, where fig. 2b is the output of the bone key point recognition model when the input is fig. 2 a.

The motion parameter determining method corresponding to the video frame of the video displayed by the display when receiving the video displayed in fig. 2a includes: the angle between the straight lines of the bone

key points

3 and 4 and the straight lines of the bone

key points

4 and 7; the distance between the bone

key points

4, 8. The motion parameter determining mode corresponding to the angle may be:

k₁＝(y4-y3)/(x4-x3)

k₂＝(y7-y4)/(x7-x4)

a＝(k₂-k₁)/(1+k₁k₂)

wherein y4 is the ordinate of the bone key point 4, y3 is the ordinate of the bone key point 3, x4 is the abscissa of the bone key point 4, x3 is the abscissa of the bone key point 3, k₁Is the slope of the line in which the bone

key points

3, 4 lie, y7 is the ordinate of the bone key point 7, x7 is the abscissa of the bone key point 7, k₂Is the slope of the line in which the bone

key points

4, 7 lie, and a is the angle between the line in which the bone

key points

3, 4 lie and the line in which the bone

key points

4, 7 lie.

The motion parameter determination method corresponding to the distance may be: h is₁＝|y4-y8|

Wherein y4 is the ordinate of the bone key point 4, y8 is the ordinate of the bone key point 8, h₁Is the distance between the bone

key points

4, 8.

In addition, the motion parameter determination method corresponding to the video frame of the video displayed by the display when receiving fig. 2a further includes: the angle between the straight lines of the bone

key points

4 and 7 and the straight lines of the bone

key points

7 and 8; the angle between the straight lines of the bone

key points

7 and 8 and the straight lines of the bone

key points

8 and 9; the angle between the straight line on which the bone

key points

4, 5 lie and the straight line on which the bone

key points

5, 6 lie.

Fig. 2c is another schematic image diagram provided in the embodiment of the present application. Fig. 2c is an image input into the bone key point recognition model, wherein each dot is each preset bone key point.

Fig. 2d is a schematic diagram of position information of preset bone key points of an image according to an embodiment of the present disclosure. Fig. 2d shows the position information of each preset bone key point and the corresponding identification of the bone key point output by the bone key point recognition model, wherein fig. 2d shows the output of the bone key point recognition model when the input is fig. 2 c.

The motion parameter determining method corresponding to the video frame of the video displayed by the display when receiving the video displayed in fig. 2c includes: the angle between the straight line of the bone

key points

1 and 4 and the straight line of the bone

key points

4 and 7; the angle between the straight line on which the bone

key points

4, 5 lie and the straight line on which the bone

key points

5, 6 lie; the angle between the straight lines of the bone

key points

4 and 7 and the straight lines of the bone

key points

7 and 8; the angle between the straight lines of the bone

key points

7 and 8 and the straight lines of the bone

key points

8 and 9; the distance between the bone

key points

4, 8.

The bone key point identification model provided by the embodiment of the application can adopt a High-resolution network (HRNet), and the HRNet can maintain High-resolution representation in the whole process. And performing multi-scale repeated fusion by repeatedly exchanging information on parallel multi-resolution sub-networks in the whole process, and estimating key points through high-resolution representation output by the network. Compared with other networks for attitude estimation, the HRNet network has two distinct advantages: high resolution to low resolution subnets, respectively, are connected in parallel, rather than in series as in most existing solutions, thus enabling high resolution to be maintained, rather than restoring resolution through a low to high process; most schemes group low-level and high-level representations, and instead, HRNet uses iterative multi-scale fusion to enhance the high-resolution representation with low-resolution representations of the same depth and similar levels, thereby making the high-resolution representation sufficient for pose estimation. Therefore, the HRNet adopted in the embodiment of the application can accurately identify the position information of the preset skeleton key points in the image.

Example 2:

in order to further realize the assistance to the body building of the user, on the basis of the above embodiments, in the embodiments of the present application:

the controller 13 is further configured to, after receiving the image acquired by the target image acquisition unit 11, acquire a set number of other images acquired before the image, input the image and each of the other images into a bone key point recognition model trained in advance, and acquire position information of each preset bone key point in the image and each of the other images; determining the target movement trend of the user according to the position information of each preset skeleton key point in the image and each other image and the time for acquiring each image; and controlling the display 12 to display the target interactive image corresponding to the target motion trend according to the corresponding relation between the video frame of the video and the interactive images with different motion trends when the image is received and the target motion trend.

In the embodiment of the application, the body building mirror can also provide body building in a game mode, such as badminton playing, racing and the like, so that the body building effect can be realized while entertainment is realized. In this embodiment, the controller 13 may determine the target interactive image in the subsequently displayed video according to the motion trend of the corresponding bone key point. For example, the driving direction of the racing car can be determined according to the moving direction of the hand key point, and an image corresponding to the driving direction is displayed subsequently, or the hitting direction of a badminton is simulated according to the swinging direction of the hand key point, and a subsequent corresponding image is determined according to the hitting direction.

In order to determine the corresponding target interaction image, in the embodiment of the present application, a target movement trend of the user may be determined first. Specifically, after receiving the image acquired by the target image acquisition unit 11, a set number of other images acquired before the image are acquired, and the target movement tendency of the user may be determined according to the image and each of the other images.

Specifically, the controller 13 may input the image and each of the other images into a bone key point recognition model respectively, and obtain an output of the bone key point recognition model, where the bone key point recognition model outputs, for each image, position information of each preset bone key point in the corresponding image and an identifier of each preset bone key point. After the position information of each preset skeleton key point and the identifier of each preset skeleton key point in the image and each other image are obtained, the controller 13 may determine the target movement trend of the user according to the position information of the preset skeleton key point and the identifier of each preset skeleton key point in each image and the time when each image is acquired. The corresponding relationship between the game and the skeletal key points may be pre-stored, and the controller 13 may determine the skeletal key points corresponding to the current game according to the currently selected game and the corresponding relationship between the game and the skeletal key points, and determine the movement trend of the skeletal key points, which is the target movement trend of the user. Taking the currently selected game as badminton playing, the skeleton key point is the hand skeleton key point.

In order to improve the experience of the user, in this embodiment of the application, the controller 13 may obtain a correspondence relationship between video frames of different videos and interactive images with different motion trends, that is, an image corresponding to a next video frame of the video frame, after determining a target motion trend, the controller may obtain the interactive image with the target motion trend corresponding to the video frame displayed by the display when receiving the image, determine that the interactive image is the target interactive image, and control the display to display the target interactive image.

Fig. 3a is a schematic view of a game mode exercise mirror during exercise according to an embodiment of the present application. Fig. 3a is a schematic diagram of the body-building mirror providing a game mode of body building, and as can be seen from fig. 3a, the display 12 can play interactive images, and the body building in the game mode can be badminton playing or racing.

Fig. 3b is a schematic diagram of an image acquired by a target image acquisition unit according to an embodiment of the present application. Fig. 3b shows the image captured by the target image capturing unit 11 when the fitness scope is used for providing fitness in a game mode.

As can be seen from the above description, in the embodiment of the present application, the next image displayed on the display 12 is the target interaction image determined according to the target motion trend of the user, so that the experience of the user can be improved.

Example 3:

in order to assist the user in exercising more accurately, on the basis of the above embodiments, in the embodiment of the present application, the exercise mirror includes at least two image capturing units 11;

the controller 13 is further configured to, if the start operation is identified, obtain position information of a preset skeleton key point in each image according to each image acquired by each image acquisition unit 11 at a uniform time and a skeleton key point identification model trained in advance; and for each image, if the number of preset skeleton key points contained in the image is a preset number and the position information of the preset skeleton key points meets the preset requirement, determining the image acquisition unit 11 corresponding to the image as a target image acquisition unit 11.

Due to different heights of different users, the habits of using the fitness mirror are different, and the distance from the fitness mirror is different correspondingly when the fitness mirror is used. If only one image acquisition unit is arranged at the higher position of the fitness mirror, when the whole body of the user is detected, the image acquisition unit can acquire the whole body image of the user only when the user is far away from the fitness mirror. If only one image acquisition unit is arranged at other positions, there is also a risk that the whole-body image of the user may not be acquired.

Fig. 4a is a schematic diagram of an image capturing unit 11 disposed on the top of an exercise mirror according to an embodiment of the present disclosure.

As can be seen from fig. 4a, if the image capturing unit 11 is only disposed at the top of the exercise mirror, the user needs to be far away from the exercise mirror, and the image capturing unit 11 can completely capture the whole body image of the user.

Fig. 4b is a schematic diagram of two image capturing units 11 disposed on the exercise mirror according to an embodiment of the present disclosure.

As can be seen from fig. 4b, if two image capturing units 11 are disposed on the left side of the display of the exercise mirror, the user does not need to be far away from the exercise mirror, and the image capturing unit 11 can completely capture the whole body image of the user.

As can be seen from fig. 4a and 4b, at least two image capturing units 11 are adopted, so that the distance from the user to the mirror during exercise can be effectively shortened and the household space can be effectively utilized, compared with the case that only one image capturing unit 11 is arranged at the top.

In the present embodiment, therefore, the exercise scope is provided with at least two image capturing units 11. For example, two upper and lower image capturing units 11 may be provided on the exercise mirror for image capturing.

Since at least two image capturing units 11 are provided, in the embodiment of the present application, the controller may determine a target image capturing unit among the at least two image capturing units 11, and subsequently perform recognition based on an image captured by the target image capturing unit 11.

Specifically, in this embodiment, if the controller 13 recognizes the start operation of the exercise mirror, the images acquired by the at least two image acquisition units 11 may be acquired, each image is respectively input into a bone key point identification model which is trained in advance, and the output of the bone key point identification model is acquired, where the output is the position information of the preset bone key point in each corresponding image and the identifier corresponding to the preset bone key point.

Since the number of the skeletal key points in the whole-body image of the user is the preset number, for each image, if the number of the skeletal key points in the image is determined to be the preset number, the image is determined to be the whole-body image of the user, and the image acquisition unit 11 acquiring the image is determined to be the target image acquisition unit 11. In addition, the controller 13 may further determine whether the position information of each preset bone key point in the image meets a preset requirement, and if the position information meets the preset requirement, it indicates that the user is in a preset posture in the acquired image, and then determines that the image acquisition unit 11 acquiring the image is the target image acquisition unit 11. If the preset posture is a standing posture, the preset requirement can be that a straight line corresponding to a knee key point and a foot key point is a vertical straight line. The preset number may be a number related to human bones, and may be 17, for example.

Fig. 5 is a schematic diagram of skeletal key points provided in the embodiment of the present application.

The recognition of the gesture can be efficiently performed by 17 skeletal key points in fig. 5. In the embodiment of the present application, when the bone key point recognition model is trained, the bone key point recognition model may be trained to recognize the 17 bone key points. Therefore, if the number of the preset skeleton key points in a certain image is determined to be the preset number, the image can be determined to be the whole body image of the user.

In addition, if each image is a whole-body image of the user and is an image in which the user assumes a preset posture, any one image capturing unit may be determined as the target image capturing unit 11, or a preset certain image capturing unit 11 may be determined as the target image capturing unit 11. The identification of the starting operation may be that a user clicks a starting button, or that preset voice starting information is identified.

In the embodiment of the application, in order to avoid that the user only opens the fitness mirror and does not do fitness, if it is recognized that the user does not present the preset posture in each image, the reminding information for reminding the user of presenting the preset posture can be output to remind the user to prepare for starting to exercise.

Example 4:

in order to improve the user experience of body building, on the basis of the above embodiments, in the embodiment of the present application, the body building mirror further includes:

the audio unit 14 is configured to collect a first call audio and send the first call audio to the controller 13;

the target image acquisition unit 12 is further configured to acquire a first call video and send the first call video to the controller 13;

the controller 13 is further configured to execute, if the connection request is obtained, obtaining identification information of other devices carried in the connection request, and sending a connection instruction to the other devices of the identification information; and if the second call audio and the second call video sent by the other equipment are received, controlling the audio unit 14 to play the second call audio, controlling the display 12 to display the second call video, and sending the first call audio and the first call video to the other equipment.

In the embodiment of the application, the fitness mirror can be connected with other fitness mirrors, so that a user can perform video fitness together with friends or a fitness coach can perform guiding fitness. Therefore, the body-building mirror further comprises an audio unit 14, wherein the audio unit 14 is connected with the controller 13, the audio unit 14 can collect the first call audio when the user and the friend perform the video body-building together or let the body-building coach guide, and the target image collecting unit 11 can also collect the first call video when the user and the friend perform the video body-building together or let the body-building coach guide.

Specifically, the user can select to connect with a friend or a fitness coach through the fitness mirror, if the user selects to connect through the fitness mirror, the controller 13 can obtain a connection request, and the user can correspondingly select the friend to be connected when connecting through the fitness mirror, the controller 13 can obtain identification information of other equipment correspondingly used by the friend, the connection request carries identification information of the other equipment, and the controller 13 sends a connection instruction to the other equipment of the identification information, wherein the connection instruction carries the identification information of the fitness mirror where the controller is located. After the other device receives the connection instruction and the other device receives the operation of accepting the connection instruction, the other device sends the second call audio and the second call video to the controller 13, and after the controller 13 receives the second call audio and the second call video, the controller 13 controls the audio unit 14 to play the received second call audio and controls the display 12 to display the received second call video. In addition, the controller 13 also sends the received first call audio collected by the audio unit 14 and the first call video collected by the target image collection unit 11 to the other device.

In the embodiment of the application, the user can communicate with a plurality of friends in a multi-channel video communication mode, specifically, the user can adjust windows of other friends and the fitness video at will through the fitness mirror, the user can encourage fitness with the friends mutually, fitness experience and feeling can be communicated at will, and the motivation and fun of fitness are improved.

Fig. 6 is a schematic view of a body-building mirror for video body-building with friends according to an embodiment of the present disclosure.

In fig. 6, the lower side (the lower side shown in the figure) of the display 12 in the fitness scope is a video frame of a fitness video, and the upper sides (the upper sides shown in the figure) are video frames of friends of the current video fitness respectively.

Example 5:

in order to further assist the user in exercising, on the basis of the above embodiments, in the embodiments of the present application:

the controller 13 is further configured to determine an action completion score corresponding to the video frame according to the target motion parameter and the standard motion parameter range, determine whether the action corresponding to the video frame reaches the standard according to the action completion score and the standard score, determine an average completion score according to the action completion score corresponding to each video frame in the played video, and control the display 12 to display the average completion score and the number of times that the action reaches the standard.

In this embodiment of the application, the controller 13 may further determine an average completion score and an action reaching-standard frequency of the user during exercise, and control the display 12 to display the average completion score and the action reaching-standard frequency, so that the user may intuitively feel the current exercise effect through the average completion score and the action reaching-standard frequency displayed by the display 12, thereby further assisting the user in exercise.

Specifically, the controller 13 may determine an action completion score corresponding to the video frame when the image is received, according to the target motion parameter and the standard motion parameter range. If the target motion parameter is determined to be within the standard motion parameter range, the preset score is determined to be an action completion score, and the preset score is a larger score, such as 100. If the target motion parameter is determined not to be within the standard motion parameter range, the average value of the standard motion parameter range can be determined as the standard motion parameter, the target difference value of the target motion parameter and the standard motion parameter is determined, and the score corresponding to the target difference value can be determined as the action completion score according to the corresponding relation between the pre-stored score and the difference value. If the target motion parameters include a plurality of target motion parameters, the action completion score corresponding to the video frame can be determined according to the average value of scores determined by the target motion parameters and the corresponding standard motion parameter ranges. Specifically, the average value may be determined as a completion score of an action corresponding to the video frame.

In addition, the controller 13 may determine whether the user's action in the image is up to standard according to the action completion score and the standard score. For example, if the action completion score is greater than the standard score, the action of the user in the image may be determined to be up to standard, otherwise, the action of the user in the image may be determined not to be up to standard.

In this embodiment, the controller 13 may determine an average completion score according to the action completion score corresponding to each video frame in the played video, may count the number of times that the user has reached the standard in each received image, and after determining the average completion score and the number of times that the user has reached the standard, controls the display to display the average completion score and the number of times that the user has reached the standard.

Example 6:

the controller 13 is further configured to determine calories consumed according to a duration of the video that has been played, and control the display 12 to display the calories.

In the embodiment of the present application, the controller 13 may also determine calories consumed and control the display 12 to display the calories consumed, thereby allowing the user to increase the motivation for exercise according to the calories displayed by the display 12.

The controller 13 may determine calories consumed based on the length of time that the video has been played. Specifically, the total amount of calories consumed per hour may be previously stored in the controller 13.

In addition, some ways of determining consumed calories, which are common in the prior art, may also be adopted in the embodiments of the present application to determine the corresponding consumed calories.

In addition, if the user is currently exercising with the video of the friend through the exercise mirror, the controller 13 may obtain the average completion score, the action reaching time and the consumed calorie, which are determined by other devices connected to the current line, and control the display 12 to display the average completion score, the action reaching time and the consumed calorie, which are determined by the controller 13, to the other devices.

FIG. 7 is a diagram of a display 12 showing an average completion score, action achievement times, and calories burned according to an embodiment of the present application.

As can be seen from fig. 7, the display 12 can display the average completion score, the action achievement number and the consumed calories of the user, and can also display the average completion score, the action achievement number and the consumed calories of the current video fitness friend.

the controller 13 is further configured to perform sending the determined calories and the duration of video playing to the server when the exercise is finished; and receives the exercise bar sent by the server, and controls the display 12 to display the exercise bar, wherein the exercise bar comprises a ranking bar determined according to at least one of duration or calories.

In the embodiment of the application, all users using the fitness mirror can form a fitness circle, when the users exercise, the controller 13 can send the determined calories and the video playing duration to the server, and the server generates the corresponding fitness list according to the calories of different users and the video playing duration, so that the users are further assisted to exercise, the users are encouraged to use the fitness mirror, and the user stickiness is improved.

Here, the controller 13 may send the calorie and the duration of the video playing to the server only when the user selects to participate in the exercise chart function in advance. After receiving the calories and the duration sent by the controller, the server determines a corresponding ranking list according to at least one of the received calories and the received duration sent by the controller of each fitness mirror, wherein the ranking list is a fitness list, and sends the fitness list to the controller, and after receiving the fitness list sent by the server, the controller 13 controls the display 12 to display the received fitness list.

In addition, the server may update the exercise charts at regular time intervals, and send the latest chart to the controller after receiving a chart viewing request sent by the controller. The time interval may be any time length, for example, one week.

In addition, in the embodiment of the present application, the controller 13 may also synchronously record the time length, the consumed calories, the average completion score, and the like of the user playing the fitness video each day, so as to form a fitness calendar. The user can also set the time of body building every day, and the controller 13 outputs reminding information at the set body building time to remind the user to insist on the movement.

In addition, in the embodiment of the present application, in order to ensure the privacy of the user, the controller 13 may also delete the image captured by the image capturing unit for a fixed period of time.

Fig. 8 is a schematic view illustrating an auxiliary exercise process of the exercise mirror according to an embodiment of the present disclosure.

S801: the controller receives the images sent by the at least two image acquisition units.

S802: the controller determines a target image acquisition unit according to each received image.

S803: the controller determines position information of preset bone key points in the image acquired by the target image acquisition unit.

S804: and the controller correspondingly processes according to the mode selected by the user.

If the fitness mode selected by the user is follow-up fitness, whether the action of the user in the image is standard or not is judged, and if the user selects the game model, the display can be controlled to display the corresponding target interaction image.

S805: the controller synchronously records the time length, the consumed calories and the average completion score of the user playing the fitness video every day to form a fitness calendar.

Example 6:

on the basis of the foregoing embodiments, the present application further provides an image recognition method, and fig. 9 is a schematic diagram of an image recognition process provided in the embodiments of the present application, as shown in fig. 9, the method includes:

s901: acquiring position information of each preset skeleton key point in the image according to the received image acquired by the target image acquisition unit and a bone key point identification model which is trained in advance;

s902: acquiring a standard motion parameter range and a motion parameter determining mode corresponding to a video frame of the video when the image is received; and determining a target motion parameter according to the position information of each preset skeleton key point and the motion parameter determination mode, judging whether the target motion parameter is within the standard motion parameter range, and if not, outputting preset reminding information.

In a possible implementation, after receiving the image acquired by the target image acquisition unit, the method further includes: acquiring a set number of other images acquired before the image, inputting the image and each other image into a bone key point recognition model trained in advance, and acquiring position information of each preset bone key point in the image and each other image; determining the target movement trend of the user according to the position information of each preset skeleton key point in the image and each other image and the time for acquiring each image; and displaying the target interactive image corresponding to the target motion trend according to the corresponding relation between the video frame of the video and the interactive images with different motion trends when the image is received and the target motion trend.

In one possible embodiment, the method further comprises: if the starting operation is identified, acquiring position information of preset skeleton key points in each image according to each image acquired by each image acquisition unit at a unified moment and a skeleton key point identification model trained in advance; and for each image, if the number of preset skeleton key points contained in the image is a preset number and the position information of the preset skeleton key points meets the preset requirement, determining the image acquisition unit corresponding to the image as a target image acquisition unit.

In one possible embodiment, the method further comprises: if a connection request is acquired, acquiring identification information of other equipment carried in the connection request, and sending a connection instruction to the other equipment of the identification information; and if receiving a second call audio and a second call video sent by the other equipment, playing the second call audio, displaying the second call video, and sending the first call audio and the first call video to the other equipment.

In one possible embodiment, the method further comprises: and determining the action completion score corresponding to the video frame according to the target motion parameter and the standard motion parameter range, determining whether the action corresponding to the video frame reaches the standard according to the action completion score and the standard score, determining the average completion score according to the action completion score corresponding to each video frame in the played video, and displaying the average completion score and the action reaching times.

In one possible embodiment, the method further comprises: the calories consumed are determined according to the length of time the video has been played and displayed.

In one possible embodiment, the method further comprises: when the exercise is finished, sending the determined calorie and the video playing time length to a server; and receiving the fitness list sent by the server and displaying the fitness list, wherein the fitness list comprises a ranking list determined according to at least one of duration or calorie.

Example 7:

on the basis of the foregoing embodiments, the present application provides an image recognition apparatus, and fig. 10 is a schematic structural diagram of the image recognition apparatus provided in the embodiments of the present application, and as shown in fig. 10, the apparatus includes:

an obtaining module 1001, configured to obtain, according to a received image acquired by a target image acquisition unit and a bone key point recognition model that is trained in advance, position information of each preset bone key point in the image;

the processing module 1002 is configured to obtain a standard motion parameter range and a motion parameter determination mode corresponding to a video frame of the video when the image is received; and determining a target motion parameter according to the position information of each preset skeleton key point and the motion parameter determination mode, judging whether the target motion parameter is within the standard motion parameter range, and if not, outputting preset reminding information.

In a possible implementation manner, the processing module 1002 is further configured to obtain a set number of other images acquired before the image, input the image and each of the other images into a bone key point recognition model trained in advance, and obtain position information of each preset bone key point in the image and each of the other images; determining the target movement trend of the user according to the position information of each preset skeleton key point in the image and each other image and the time for acquiring each image; and displaying the target interactive image corresponding to the target motion trend according to the corresponding relation between the video frame of the video and the interactive images with different motion trends when the image is received and the target motion trend.

In a possible implementation manner, the processing module 1002 is further configured to, if the start operation is identified, obtain position information of a preset bone key point in each image according to each image acquired by each image acquisition unit at a uniform time and a bone key point identification model trained in advance; and for each image, if the number of preset skeleton key points contained in the image is a preset number and the position information of the preset skeleton key points meets the preset requirement, determining the image acquisition unit corresponding to the image as a target image acquisition unit.

In a possible implementation manner, the processing module 1002 is further configured to, if a connection request is obtained, obtain identification information of other devices carried in the connection request, and send a connection instruction to the other devices of the identification information; and if receiving a second call audio and a second call video sent by the other equipment, playing the second call audio, displaying the second call video, and sending the first call audio and the first call video to the other equipment.

In a possible implementation manner, the processing module 1002 is further configured to determine an action completion score corresponding to the video frame according to the target motion parameter and the standard motion parameter range, determine whether an action corresponding to the video frame reaches the standard according to the action completion score and the standard score, determine an average completion score according to an action completion score corresponding to each video frame in the played video, and display the average completion score and the number of times that the action reaches the standard.

In a possible implementation, the processing module 1002 is further configured to determine calories consumed according to a duration of time that the video has been played, and display the calories.

In a possible embodiment, the processing module 1002 is further configured to send the determined calories and the duration of the video playing to the server when the exercise is finished; and receiving the fitness list sent by the server and displaying the fitness list, wherein the fitness list comprises a ranking list determined according to at least one of duration or calorie.

Example 8:

on the basis of the foregoing embodiments, the present application further provides an electronic device, and fig. 11 is a schematic structural diagram of the electronic device provided in the embodiments of the present application, as shown in fig. 11, including: the system comprises a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 complete mutual communication through the communication bus 114;

the memory 113 stores therein a computer program which, when executed by the processor 111, causes the processor 111 to carry out the steps of the above-mentioned recommendation method.

The communication bus mentioned in the above server may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital instruction processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.

Example 10:

on the basis of the foregoing embodiments, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program executable by an electronic device is stored, and when the program is run on the electronic device, the electronic device is caused to perform the following steps:

the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of:

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. An exercise scope, characterized in that the exercise scope comprises:

the target image acquisition unit is used for acquiring images;

a display for displaying video;

a controller to perform:

2. The exercise mirror of claim 1, wherein the controller is further configured to perform acquiring a set number of other images acquired before the image after receiving the image acquired by the target image acquisition unit, inputting the image and each of the other images into a bone key point recognition model trained in advance, and acquiring position information of each preset bone key point in the image and each of the other images; determining the target movement trend of the user according to the position information of each preset skeleton key point in the image and each other image and the time for acquiring each image; and controlling the display to display the target interactive image corresponding to the target motion trend according to the corresponding relation between the video frame of the video and the interactive images with different motion trends when the image is received and the target motion trend.

3. The exercise mirror of claim 1 or 2, wherein the exercise mirror comprises at least two image acquisition units;

the controller is further used for executing the operation of acquiring the position information of the preset skeleton key points in each image according to each image acquired by each image acquisition unit at a unified moment and a skeleton key point identification model which is trained in advance if the starting operation is identified; and for each image, if the number of preset skeleton key points contained in the image is a preset number and the position information of the preset skeleton key points meets the preset requirement, determining the image acquisition unit corresponding to the image as a target image acquisition unit.

4. The exercise mirror of claim 1, further comprising:

the audio unit is used for collecting first call audio and sending the first call audio to the controller;

the target image acquisition unit is also used for acquiring a first call video and sending the first call video to the controller;

the controller is further configured to execute, if a connection request is obtained, obtaining identification information of other devices carried in the connection request, and sending a connection instruction to the other devices of the identification information; and if a second call audio and a second call video sent by the other equipment are received, controlling the audio unit to play the second call audio, controlling the display to display the second call video, and sending the first call audio and the first call video to the other equipment.

5. The exercise mirror of claim 1, wherein the controller is further configured to determine an action completion score corresponding to the video frame according to the target motion parameter and the standard motion parameter range, determine whether an action corresponding to the video frame is up to standard according to the action completion score and the standard score, determine an average completion score according to the action completion score corresponding to each video frame in the played video, and control the display to display the average completion score and the number of times that the action is up to standard.

6. The exercise mirror of claim 1, wherein the controller is further configured to determine calories burned based on a length of time that the video has been played, and to control the display to display the calories.

7. The exercise mirror of claim 6, wherein the controller is further configured to perform sending the determined calories and the length of time the video is played to the server when the exercise is over; and receiving a health chart sent by the server, and controlling the display to display the health chart, wherein the health chart comprises a ranking chart determined according to at least one of duration or calories.

8. An image recognition method, characterized in that the method comprises:

9. An electronic device, characterized in that the electronic device comprises at least a processor and a memory, the processor being configured to implement the steps of the image recognition method according to claim 8 when executing the computer program stored in the memory.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the image recognition method of claim 8.