JP2020174910A - Exercise support system - Google Patents

Abstract

To efficiently support body exercise.SOLUTION: A system for supporting user's exercise includes: a camera 21 for imaging a user in exercise; a display 22 installed in a site where a user exercises; an instruction output part for outputting an instruction about exercise on the display; a position specifying part for specifying a position of a user's body part by analyzing an image captured by the camera; and an output part for outputting the position of the part to the display while superposing the position on the image.SELECTED DRAWING: Figure 1

Description

The present invention relates to an exercise support system.

Conventionally, as an analysis technique in the field of physical exercise, motion patterns that repeatedly appear in moving images are extracted, motion patterns at multiple positions in the extracted moving images at different time positions are compared from a predetermined viewpoint, and the comparison results are compared with the moving images. A technique for outputting as a still image (Patent Document 1), and in sports, for example, in golf, a technique for attaching a device to a user's body and analyzing a golf wing with a motion sensor (Patent Document 2). A technique (Patent Document 3) for calculating the transition time by photographing each stage of throwing (installation of a shaft foot, horizontal rotation of the trunk) with two cameras is disclosed.

Japanese Unexamined Patent Publication No. 2017-229081 JP-A-2010-068947 Japanese Unexamined Patent Publication No. 2018-114138

However, all the techniques of Patent Documents 1 to 3 require equipment and resources such as a plurality of moving images, a specific device, and a plurality of cameras, and one user can incorporate these devices and analyze physical exercise. It is difficult to understand the analysis results.

The present invention has been made in view of such a background, and an object of the present invention is to provide a technique capable of efficiently supporting physical exercise.

The main invention of the present invention for solving the above problems is a system that supports a user's movement, and includes a camera that captures the user during the movement, a display installed at a place where the movement is performed, and the above. An explanatory output unit that outputs an explanation about exercise to the display, a part identification part that analyzes an image taken by the camera to specify the position of a part of the user's body, and the position of the part are superimposed on the image. An output unit for outputting to the display is provided.

Other problems disclosed in the present application and solutions thereof will be clarified in the columns and drawings of the embodiments of the invention.

According to the present invention, physical exercise can be efficiently supported.

It is a figure which shows the whole structure example of the exercise support system which concerns on this embodiment. It is a figure which shows the hardware configuration example of the server apparatus 20. It is a figure which shows the software configuration example of the server apparatus 20. It is a figure which shows the structural example of the training information storage part 231. It is a figure which shows the configuration example of the user information storage unit 232. It is a figure which shows an example of the input screen 41 for inputting user information. .. It is a figure which shows the structural example of the image data storage part 233. It is a figure which shows the structural example of the reference information storage part 234. It is a figure which shows the structural example of the evaluation condition information storage part 235. It is a figure which shows the structural example of the training plan storage part 236. It is a figure which shows the structural example of the training history storage part 237. It is a figure which shows the structural example of the improvement condition storage part 238. It is a figure which shows an example of the output screen 42 of the training information. It is a figure which shows the structural example of the evaluation information created by the evaluation unit 215. It is a figure which shows an example of the screen 42 which the evaluation information output unit 216 outputs. It is a figure which shows an example of the screen 42 which the evaluation information output unit 216 outputs. It is a figure which shows an example of the screen 42 which the evaluation information output unit 216 outputs. It is a figure which shows the display example of the training history by the user terminal 10. It is a figure which shows an example of the output screen 44 of the training volume information. It is a figure which shows the flow of the process executed in the exercise support system of this embodiment.

The contents of the embodiments of the present invention will be described in a list. The exercise support system according to the embodiment of the present invention has the following configurations.

[Item 1]
A system that supports the movement of users
A camera that captures the user during the exercise and
A display installed at the place where the exercise is performed and
An explanatory output unit that outputs an explanation about the exercise to the display,
A part identification part that analyzes an image taken by the camera to identify the position of a part of the user's body,
An output unit that superimposes the position of the portion on the image and outputs it to the display.
An exercise support system characterized by being equipped with.
[Item 2]
The exercise support system according to item 1.
A reference value storage unit that stores a reference value related to the position of at least one of the parts,
An evaluation unit that determines the evaluation value of the exercise by comparing the position of the portion in the image and the reference value.
With more
The output unit further outputs the evaluation value to the display.
An exercise support system featuring.
[Item 3]
The exercise support system described in item 2,
The evaluation unit determines the evaluation value according to the distance from the reference value to the position.
An exercise support system featuring.
[Item 4]
The exercise support system described in item 2,
The reference value storage unit stores the reference value for the angle between the straight line connecting each of the first and second portions and the third portion.
The site specifying portion identifies each of the first to third sites from the image, and then
The evaluation unit calculates the angle with respect to the specified first to third parts, compares the calculated angle with the reference value, and determines the evaluation value.
An exercise support system featuring.
[Item 5]
The exercise support system according to item 1.
Further, a number counting unit for counting the number of times of the exercise based on the position of the portion is provided.
The output unit further outputs the number of times of the exercise to the display.
An exercise support system featuring.
[Item 6]
The exercise support system described in Item 2
The output unit further outputs the evaluation value by voice.
An exercise support system featuring.
[Item 7]
The exercise support system according to item 1.
The output unit further outputs the improvement measure information by voice based on the position of the body of the specified user.
An exercise support system featuring.

The exercise support system according to the embodiment of the present invention is intended to support the training of a user in a training gym. In the exercise support system of the present embodiment, a state in which the user is exercising physically using a training device is photographed by a camera, and the photographed image (still image or moving image may be used, but this embodiment). In the form, it is assumed that it is a moving image.) The number of trainings is counted, and the posture and movement of the user's body during training are evaluated.

FIG. 1 is a diagram showing an overall configuration example of the exercise support system according to the present embodiment. As shown in the figure, in the exercise support system of the present embodiment, the server device 20, the camera 21, and the display 22 are connected to each other so as to be able to communicate with each other via the communication network 23. The communication network 23 is, for example, the Internet or a LAN (Local Area Network), and is constructed by a public telephone line network, a dedicated telephone line network, a mobile phone line network, Ethernet (registered trademark), a wireless communication path, or the like.

The camera 21 captures the user 1 during training. As the camera 21, for example, a surveillance camera may be used, or the camera 21 may be installed at a training place, such as by attaching the camera 21 to a training device. The camera 21 can transmit the captured image to the server device 20 by a protocol such as TCP / IP or Bluetooth (registered trademark). The camera 21 may always shoot, may detect that the user 1 has come near and start shooting, or may start shooting in response to an instruction from the user 1. You may.

The display 22 is visibly installed by the user 1 before and during training. The display 22 outputs information under the control of the server device 20 via the communication network 23. By connecting a computer to the display 22 and communicating between the computer and the server device 20, information can be output to the display 22 in response to an instruction from the server 20. Further, the information can be output as audio by attaching an audio output device such as headphones, earphones, or a neck speaker to the user 1 together with or in place of the display 22.

The user terminal 10 is a computer operated by the user 1 or its supporter. The user terminal 10 is, for example, a smartphone, a tablet computer, a personal computer, or the like. The user terminal 10 may be provided with an image pickup device such as a camera, whereby the body of the user 1 during exercise may be imaged. The information displayed on the display 22 is also provided from the server device 20 to the user terminal 10. The user 1 can browse the information related to the training by checking the user terminal 10 later.

The server device 20 is a computer that supports training. The server device 20 is, for example, a workstation, a personal computer, a virtual computer logically realized by cloud computing, or the like. The server device 20 receives the moving image taken by the user terminal 10 and analyzes the received moving image to evaluate the physical exercise. The server device 20 also makes a proposal regarding measures for improving physical exercise. Details of the evaluation of physical exercise and the proposal of improvement measures will be described later.

The administrator terminal 30 is a computer operated by the administrator of the training gym. The administrator terminal 30 can be, for example, a smartphone, a tablet computer, a personal computer, or the like.

FIG. 2 is a diagram showing a hardware configuration example of the server device 20. The server device 20 includes an arithmetic unit 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The arithmetic unit 201 is, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) that performs various arithmetic processes. The storage device 203 stores various data and programs, such as a hard disk drive, a solid state drive, and a flash memory. The communication interface 204 is an interface for connecting to the communication network 30, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, and a wireless communication device for performing wireless communication. , USB (Universal Serial Bus) connector for serial communication, RS232C connector, etc. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, or the like for inputting data. The output device 206 is, for example, a display, a printer, a speaker, or the like that outputs data.

FIG. 3 is a diagram showing a software configuration example of the server device 20. As shown in the figure, the server device 20 includes a user authentication unit 210, a training information output unit 211, an image acquisition unit 212, an image analysis unit 213, a number of times counting unit 214, an evaluation unit 215, an evaluation information output unit 216, and a training history. Each function unit of the transmission unit 217 and the improvement measure information output unit 218, the training information storage unit 231 and the user information storage unit 232, the image data storage unit 233, the reference information storage unit 234, the evaluation condition information storage unit 235, and the training plan storage unit. Each storage unit includes a unit 236, a training history storage unit 237, and an improvement condition storage unit 238.

Each of the above functional units is realized by the arithmetic unit 201 included in the server device 20 reading a program stored in the storage device 203 into the memory 202 and executing the program, and each of the above storage units is provided in the server device 20. It is realized as part of the storage area provided by the memory 202 and the storage device 203.

The training information storage unit 231 stores information related to training. Information about training is, for example, a description of training. The training description includes how to use the training equipment and how to effectively train using the training equipment. FIG. 4 is a diagram showing a configuration example of the training information storage unit 231. As shown in the figure, the training information storage unit 231 stores training information and display information. Training information is information about training. The training information includes information for identifying the training menu (menu ID) and video data showing the training content, the number of times, and the training method in association with the purpose of the training. The purpose of the training is to show the results required by the training, such as relieving lack of exercise and increasing muscle mass. Arbitrary information such as how to use the training equipment and precautions regarding training can be set in the content of the training. The video data is, for example, a video of a person demonstrating how to train, a tutorial video showing how to use a training machine, and the like. The content of the training and the video data are linked to the purpose of the training and are prepared to suggest what kind of training should be performed and how many times to achieve the purpose using the training equipment. is there. The display information is information for setting a training menu to be performed while looking at the display 22. The display information includes a menu ID in association with a display ID that identifies the display 22.

The user information storage unit 232 stores information about the user 1 (hereinafter, referred to as user information). FIG. 5 is a diagram showing a configuration example of the user information storage unit 232. As shown in the figure, the user information stored in the user information storage unit 232 includes authentication information, height, weight, muscle mass, flexibility, and the like in association with the user ID that identifies the user 1. The authentication information is information that can be used to authenticate the user 1. The authentication information may be, for example, a feature amount of the face portion of the user 1 for use in face recognition, a device ID of the user terminal 10, or a combination of a user name and a password. It may be the public key of the user 1, but it is not limited to these as long as the information can authenticate the user 1. In addition to the above items, the user information can include various information about the user's body that can be measured or evaluated.

FIG. 6 is a diagram showing an example of an input screen 41 for inputting user information. For example, an input screen 41 as shown in the figure can be displayed on the user terminal 10 so that the user 1 can input each item 411 of the user information. In this case, each item of user information can be transmitted from the user terminal 10 to the server device 20 and registered in the user information storage unit 232.

The image data storage unit 233 stores the image taken by the camera 21. FIG. 7 is a diagram showing a configuration example of the image data storage unit 233. As shown in the figure, the image data storage unit 233 stores the image information, and the image information includes the image data in association with the camera ID that identifies the camera 21 that has taken the picture. The image data is data for displaying an image taken by the camera 21. The image data can be, for example, data in any format such as MP4, MPEG2, AVI, and MOV.

The reference information storage unit 234 stores information (hereinafter, referred to as reference information) including a reference value of a posture (position and angle of a portion) related to the physical exercise of the user 1. FIG. 8 is a diagram showing a configuration example of the reference information storage unit 234. As shown in the figure, the reference information storage unit 234 contains three reference information (hereinafter referred to as position reference information) regarding the absolute position of the part or a position relative to another part or another reference object, and three joint parts. For the site, the angle reference information (hereinafter referred to as angle reference information) formed by the straight line connecting each of the two sites and the joint site is stored.

The position reference information includes a part and a reference position of the part in association with the mode and the checkpoint ID. The checkpoint ID is information indicating a point (hereinafter referred to as a checkpoint) for checking a posture or the like in a series of movements of the user's body. There may be multiple parts. Regarding the position, the position in the vertical direction may be, for example, the height from the ground or the distance from any of the toes. The horizontal position may be a distance from a predetermined reference object (for example, a mark on the floor or an installed instrument), or a distance from a reference portion such as a shoulder, chest, or foot. It is assumed that the position reference information is registered in advance.

The angle reference information includes two parts (part 1 and part 2), one joint part, a straight line connecting the part 1 and the joint part, and the part 2 and the joint in association with the mode and the checkpoint ID. Includes a reference value for the angle between the straight line connecting the site.

The evaluation condition information storage unit 235 stores information for performing evaluation (hereinafter, referred to as evaluation condition information). FIG. 9 is a diagram showing a configuration example of the evaluation condition information storage unit 235. The evaluation condition information includes categories, conditions, evaluation ranks, and comments. The category is the evaluation category. The category can be, for example, "arm curl", "clean", "press" and the like. The condition is a condition for the position or movement (change in position in time series) of each part of the body in the image. For example, when analyzing the movement of weightlifting, it is possible to set conditions for the elbow angle, the waist angle, and the like for the checkpoint for cleaning the weight in the evaluation condition information. The evaluation rank is an evaluation value when the above conditions are satisfied. The comment is an explanation of the posture and movement of the body when the above conditions are satisfied.

The training plan storage unit 236 stores the training plan set by the user 1 or its supporter. FIG. 10 is a diagram showing a configuration example of the training plan storage unit 236. The training plan storage unit 236 stores the training plan, and in each training plan, the setting contents (for example, the menu ID indicating the training menu, the training device, etc.) are associated with the user ID indicating the user 1 and the date. Includes the number of times of training (for example, repeated actions such as raising weights and squatting) (such as weight of weights). The training plan shall be set in advance by User 1 or its supporters.

The training history storage unit 237 stores the training history of the user 1. FIG. 11 is a diagram showing a configuration example of the training history storage unit 237. As shown in the figure, the training history storage unit 237 stores the training history, and each training history is performed by the user 1 in association with the user ID indicating the user 1 and the date and time when the user 1 trained. It includes a menu ID indicating the training menu, setting information indicating the setting contents of the training device, the number of times of training, the time of training, and the like.

The improvement condition storage unit 238 stores information related to training improvement measures (hereinafter, referred to as improvement condition information). FIG. 12 is a diagram showing a configuration example of the improvement condition storage unit 238. As shown in the figure, the improvement condition information stored in the improvement condition storage unit 238 includes advice in association with the menu ID, purpose, category, and condition. The condition may be a condition for a physical condition (flexibility, etc.) or a condition for the position or movement of a part of the body. In the present embodiment, the advice is assumed to be a character string representing the improvement measure, but may be content that presents the improvement measure by an image, a moving image, or the like.

The user authentication unit 210 authenticates the user 1 who is looking at the display 22. The user authentication unit 210 can authenticate the user 1 by using the authentication information of the user information. In the present embodiment, the authentication information of the user information is the feature amount of the face portion of the user 1, and the user authentication unit 210 authenticates the user 1 by being able to extract the feature amount from the image taken by the camera 21. Is assumed. The user authentication unit 210 may, for example, communicate with the user terminal 10 to authenticate the user 1, or the display 22 communicates with the user terminal 10 to authenticate the user 1. Is authenticated, and the result may be transmitted from the display 22 to the user authentication unit 210.

The training information output unit 211 outputs training information to the display 22. The training information output unit 211 identifies the menu ID corresponding to the display ID indicating the display 22 from the display information, and outputs the content of the training information and the moving image data corresponding to the specified menu ID to the display 22. In the training information output unit 211, the user 1 has come near the display 22 (for example, the display 22 is provided with a motion sensor, the motion sensor detects the user 1, or the display 22 communicates with the user terminal 10. It is provided with a communication function to perform, and by detecting that the display 22 can communicate with the user terminal 10, it is possible to detect that the user 1 is in the vicinity.) Training the display 22 as an opportunity. Information can be output. The user 1 can refer to the training by viewing the training content and the video output on the display 22 before or during the training.

FIG. 13 is a diagram showing an example of the training information output screen 42. On the output screen 42, the purpose of training 421, the training content 422, and the moving image 423 are displayed. A plurality of moving images may be included in the training information, and a plurality of moving images may be displayed on the output screen 42.

The training information output unit 211 may correct the content of the training information based on the physical information. For example, the training information output unit 211 can change the height of the chair of the training device according to the height, and can set the weight of the recommended weight according to the height and the weight.

Further, the training information output unit 211 may correct the content of the training information according to the past training history of the user 1. For example, the training information output unit 211 reads the training history corresponding to the user ID indicating the user 1 from the training history storage unit 237, and the setting information (weight weight, chair height, etc.) included in the read training history. ) And the number of times, for example, the content of the training information to propose the same information as the content of the previous setting information, the weight heavier by a predetermined value than the previous weight, or the number of times more than the previous number by a predetermined value. Can be included in.

The image acquisition unit 212 acquires an image taken by the camera 21. In the present embodiment, the camera 21 transmits the captured image data to the server device 20 by communication, and the image acquisition unit 212 associates the image data transmitted from the camera 21 with the camera ID indicating the camera 21 and image data. Register in the storage unit 233. The image acquisition unit 212 may transmit an instruction to the camera 21 when the user authentication unit 210 succeeds in authenticating the user 1, and the camera 21 may start shooting.

The image analysis unit 213 (site identification unit) analyzes the image data. The image analysis unit 213 analyzes the image data, extracts the feature amount of each part of the body, and specifies the position of each part in the image. As the image analysis method by the image analysis unit 213, a general method is adopted, and detailed description thereof will be omitted here. The image analysis unit 213 may analyze the image data frame by frame or key frame by key frame, or may analyze the image data at random timings.

The image analysis unit 213 also compares the position of each part extracted from the image data with the position reference information stored in the reference information storage unit 232 for each checkpoint ID, and sets the closest time point as the checkpoint time point. Identify as.

The image analysis unit 213 can also recognize the movement of the user 1's body. In this case, the image analysis unit 213 creates, for example, a model capable of recognizing body movements (for example, squats, lift-ups, etc.) from moving image data by machine learning, and recognizes the movements using the model. Can be done. In this case, the image analysis unit 213 may be realized by performing parallel processing using, for example, a plurality of GPUs.

The number counting unit 214 counts the number of trainings. The number-of-times counting unit 214 can count the number of times the user 1 has changed to a predetermined posture (from another posture) by using the position of each part analyzed by the image analysis unit 213. The number-of-times counting unit 214 counts up, for example, for each of the parts analyzed by the image analysis unit 213, when the distance between the position of the part and the position included in the position reference information is within a predetermined value. can do. Further, when the angles of the two parts having the joint part as the apex included in the angle reference information are within a predetermined value, the count may be increased. Further, the number-of-times counting unit 214 prepares in advance a model created by machine learning, for example, using the posture of the body as teacher data and the position (relative positional relationship) of each part as input data, and the model. May be used to specify the posture of the user 1 and count up when the posture becomes a predetermined posture.

The number of times counting unit 214 can also calculate the workload of the user 1 by counting the number of times of training and the number of sets of training, multiplying these numerical values by the weight of the weight, and calculating the training volume. .. This training volume can be used as an index for measuring the degree of fatigue of user 1. By storing this training volume in the training history storage unit 237, the user 1 can display and confirm the training volume and its transition on the display 22 or the like at a desired timing.

The evaluation unit 215 evaluates the body movement of the user 1 based on the image data. In the present embodiment, the evaluation unit 215 searches the evaluation condition storage unit 235 for evaluation condition information including the condition that the position of each part and the movement of the part specified from the image data are satisfied, and the evaluation condition information that the condition is satisfied. If there is, get the evaluation rank and comments included in it.

FIG. 14 is a diagram showing a configuration example of evaluation information created by the evaluation unit 215. The evaluation information includes menu ID, user ID, position information, posture information, movement information, and checkpoint information.

The position information indicates the position in the image of each part of the body (for example, head, shoulders, elbows, hips, knees, ankles, etc.). The position information includes a part and a position of the part in association with a time point on the time axis of the moving image. Based on the position information, the state (bone) of the skeleton of the body can be displayed. That is, for example, a figure (for example, a circle) indicating a part can be superimposed and displayed on the image at the position indicated by the position information. It should be noted that the positions of a plurality of parts may be included at one time point. The position information may not be included in the part connecting the two parts (for example, the forearm connecting the wrist and the elbow or the thigh connecting the waist and the knee). In this case, by connecting a pair of marks (circles, etc.) indicating the two predetermined parts with a line, the part connecting the two parts can be expressed. The position information may be included for each frame constituting the video, for each key frame, for each frame of any number, or at a random time point. May be included. If My Frame does not contain location information, bones can be displayed based on the location information at the nearest past point in time.

Posture information is information related to the posture of the user's body. The posture information includes a part to be evaluated, a posture value, an evaluation rank, and an evaluation comment in association with a time point on the time axis of the moving image. The posture value is a value representing the posture. The posture values are, for example, the distance from the ground to the part, the distance between the two parts, the angle of the indirect part (the straight line from the part of the first end to the indirect part, and the part of the second end to the indirect part. The angle formed by the straight line to). The evaluation rank is a value representing the evaluation value by rank. The evaluation rank is expressed by, for example, 1 to 5 in 5 stages, ABC, or the like. The evaluation comment is a comment related to the evaluation regarding the posture. For example, if the mode is "lifting" and the flexion is not sufficient, it may include an evaluation comment such as "not sitting down".

The movement information is information related to the movement of the user's body. The motion information includes a part to be evaluated, a list of posture values, an evaluation rank, and an evaluation comment in association with a period on the time axis of the moving image. The list of posture values is a time-series posture value within a period. The evaluation comment is a comment related to the evaluation of movement. For example, if the mode is "lifting" and the knee extension is not smooth, it may include a rating comment such as "knee movement is not smooth".

Checkpoint information is information about checkpoints. Checkpoints include, for example, in the case of a weightlifting training menu, where you hold the weight bar, stand upright with the weight, or raise the weight to chest height. The checkpoint information stores the checkpoint in association with the time point on the time axis of the moving image. That is, it is possible to identify the frame (still image) in the moving image in which the checkpoint indicated by the checkpoint ID is displayed.

The evaluation unit 215 can generate position information including a time point on the time axis of the moving image specified by the image analysis unit 212 and the position of each part. When the position of the part satisfies the condition for the acquired evaluation rank and comment, the evaluation unit 215 generates posture information including the time point, the part and the posture value, and the evaluation rank and the comment, and moves the part (time). If the condition (change in position in the series) is satisfied, motion information including a list of time points, parts and posture values, and an evaluation rank and a comment is generated. Further, the evaluation unit 215 generates checkpoint information including a time point corresponding to each checkpoint analyzed by the image analysis unit 212 and a checkpoint ID indicating the checkpoint.

The evaluation information output unit 216 outputs the evaluation information to the display 22. The evaluation information output unit 216, for example, the evaluation display unit 114 superimposes on a moving image based on the position information included in the evaluation information to represent a figure (for example, an end or an indirect circle). And the line connecting them), the movement of the bones can be superimposed on the video. Further, the evaluation information output unit 216 can display, for example, a graph of changes in the position of the portion over time.

Further, the evaluation display unit 114 can display the evaluation rank and the evaluation comment together with the display of the moving image based on the posture information and the movement information included in the evaluation information. For example, the evaluation display unit 114 includes the video reproduction time in the posture information when it comes to the vicinity of the front and back (for example, about 5 seconds or the like) at the time when the video is included in the posture information. It is possible to display the evaluation rank and evaluation comment. Further, the evaluation display unit 114 can display the evaluation rank and the evaluation comment included in the motion information when the playback time of the moving image comes within the period included in the motion information. In addition, the evaluation display unit 114 can display the posture value included in the posture information. Further, the evaluation display unit 114 can display a graph of the time-series conversion of the posture values based on the list of posture values included in the movement information.

15, 16 and 17 are diagrams showing an example of the screen 42 output by the evaluation information output unit 216. In FIG. 15, the bone 422 is displayed on the screen 42 by superimposing the user 1 on the captured image data 421. Further, in the example of FIG. 15, the evaluation comment 423 of the posture information and the evaluation comment 424 of the movement information are displayed on the screen 42. Further, in the example of the screen 42, the number of times 425 of the training information and the number of times 426 counted by the number of times counting unit 214 are displayed. In the example of FIG. 16, an example of Graph 427 showing the speed of the shoulder, wrist, ankle, etc. is shown. In the example of FIG. 17, the moving image data 423 of the training information is displayed. By displaying such a screen 42 on the display 22, the user 1 can correct the posture and movement of the body during training, and can perform efficient and effective training.

The evaluation information output unit 216 can also transmit the evaluation information to the user terminal 10. The user 1 can use the user terminal 10 to look back on the training during and after the training.

The evaluation information output unit 216 registers the training history by the user 1 in the training history storage unit 237. The evaluation information output unit 216 includes a user ID indicating the user 1, a date and time when the user authentication unit 210 authenticates the user 1, a menu ID corresponding to the display 22, setting information, and the number of times counted by the number count unit 214. , The training history including the time from the time when the user is authenticated to the end of training can be registered in the training history storage unit 237. Regarding the setting information, the display 22 accepts the input of the setting information from the user 1 (for example, the display 22 can be a touch panel display, and the touch panel display can display options and the like to accept the input), and the setting information is set from the display 22. The information may be received, the user terminal 10 may accept the input of the setting information and receive the setting information from the user terminal 10, or the training device may be provided with a communication function. The setting information may be acquired from the training device.

The training history transmission unit 217 transmits the training history to the user terminal 10. The training history transmission unit 217 may transmit the training information to the user terminal 10 each time the training is completed, or may respond the training history in response to a request from the user terminal 10. The training history can be displayed on the user terminal 10. FIG. 18 is a diagram showing a display example of the training history by the user terminal 10. On the screen 43 shown in FIG. 18, the content 431 of the training performed by the user 1 and the moving image 432 of the user 1 during the training are displayed. In this way, the user 1 can look back on the state during training later, so that effective training becomes possible.

The training history transmission unit 217 transmits the training volume to the user terminal 10. The screen 44 shown in FIG. 19 shows an example of the training volume information output screen 44. As shown in FIG. 19, the training volume can show daily, weekly, and monthly transitions along with physical information such as height and weight of the user on a daily basis. Further, as shown in FIG. 19, by simultaneously showing the training volume information together with the fat percentage of the user, it is possible to compare the transition of the fat percentage with respect to the workload to the user.

In addition, the training history transmission unit 217 can also transmit the training history to the administrator terminal 30. The training history transmission unit 217 may transmit the training information to the administrator terminal 10 each time the training is completed, or may respond the training history in response to a request from the administrator terminal 30. ..

The improvement measure information output unit 218 outputs the training improvement measure to the display 22. In the improvement measure information output unit 218, among the improvement condition information corresponding to the menu ID corresponding to the display 22, the physical information of the user 1 and the position and movement of each part specified by the image analysis unit 213 satisfy the conditions. To search for. The improvement measure information output unit 218 acquires the advice of the searched improvement measure information, and outputs the acquired advice to the display 22. The remedy information output unit 218 can also output the position and angle of each part included in the reference information to the display 22. For example, the improvement measure information output unit 218 can display the state (bone) of the skeleton of the body based on the position and angle of each part included in the reference position. That is, for example, a figure (for example, a circle) indicating a part can be superimposed and displayed on the image at the position indicated by the position information.

FIG. 20 is a diagram showing a flow of processing executed in the exercise support system of the present embodiment.

The user authentication unit 210 of the server device 20 detects that the user 1 has come near the training device (S501). For the detection of the user 1, for example, a motion sensor may be installed near the display 22, or an image from the camera 21 may be monitored to detect a human from the image.

Next, the user authentication unit 210 authenticates the user 1 (S502). The authentication of the user 1 can be performed, for example, by extracting the feature amount of the face portion from the image and comparing it with the feature amount included in the authentication information of the user information. Further, the user authentication unit 210 authenticates the user 1 by, for example, communicating with the user terminal 10 (for example, accepting input of authentication information such as a user ID and password or a public key from the user terminal 10). You may do it. Further, an IC card reader may be installed near the display 22 to perform authentication using the membership card.

When the user 1 is authenticated, the image acquisition unit 212 starts to acquire the image from the camera 21 and register it in the image data storage unit 233 (S503). The training information output unit 211 identifies the menu ID corresponding to the display 22 from the training information storage unit 231, reads the training information corresponding to the specified menu ID from the training information storage unit 231 (S504), and reads the read training information. , The user information corresponding to the user 1 is used for correction (S505). The training information output unit 211 can modify the weight and the number of times of the weight according to, for example, height, weight, muscle mass, and the like. Further, the training information output unit 211 may modify the training information according to the training history of the user 1. The training information output unit 211 outputs the corrected training information to the display 22 (S506). The training information output unit 211 may receive the target input from the user 1 here, and may read the training information corresponding to the target and the menu ID. An example of output of training information is screen 42 shown in FIG. 13, for example.

The image analysis unit 213 analyzes the image data acquired from the camera 21 and identifies the position of each part of the body of the user 1 (S507). The number-of-times counting unit 214 counts the number of times of training performed by the user 1 based on the position of the part (S508). The number-of-times counting unit 214 counts that the leg has been bent and stretched once when the knee angle is within a predetermined angle, or squats 1 when the buttock position is within a predetermined distance on the floor. It can be counted as a round.

The evaluation information output unit 216 displays a so-called bone by superimposing the position of each part and its connection relationship on the image captured by the user 1 (S509). Further, the evaluation unit 215 evaluates the training based on the position and movement (time-series change of the position or angle) of the part, and the evaluation information output unit 216 outputs the evaluation result to the display 22 (S510). Here, when the user 1 is equipped with an audio output device such as earphones, headphones, or a neck speaker, the evaluation result can be output as audio.

The improvement measure information output unit 218 searches the improvement condition storage unit 238 for the improvement condition information that satisfies the physical information of the user 1 and the position and movement of each part specified by the image analysis unit 213, and the searched improvement condition information. The advice is output to the display 22 (S511). The improvement measure information output unit 218 may output other items included in the improvement condition information to the display 22. Further, when the user 1 is equipped with an audio output device such as earphones, headphones, or a neck speaker, information such as advice included in the improvement condition information may be output as audio.

Finally, the number counting unit 214 can register the training history in the training history storage unit 237 (S512). The number-of-times counting unit 214 may detect the end of training each time the number of times of training is counted, or when a predetermined time has elapsed since the training cannot be counted, and may register the history.

As described above, according to the exercise support system of the present embodiment, information such as a training explanation is displayed on the display 22 installed for each training device, and the body of the user 1 photographed by the camera 21 is displayed. It is analyzed and evaluation and improvement measures for posture and movement are presented. Therefore, since the system can be used to explain the equipment and evaluate / advise the movement of the body, which was conventionally performed manually by a personal trainer or the like, it is possible to improve the efficiency of the guidance of the user 1.

Further, according to the exercise support system of the present embodiment, it is possible to easily evaluate the physical exercise. Since it is possible to evaluate the positional relationship and movement of each part of the body regarding the physical exercise related to training, it is easy to lead to concrete improvement efforts, and it is expected that the training effect will be improved. In addition, since the exercise support system of the present embodiment also provides comments and advice, the user can easily grasp the current situation and improvement measures.

Although the present embodiment has been described above, the above embodiment is for facilitating the understanding of the present invention, and is not for limiting and interpreting the present invention. The present invention can be modified and improved without departing from the spirit thereof, and the present invention also includes an equivalent thereof.

For example, in the present embodiment, the server device 20 analyzes the image, but the present invention is not limited to this, and the camera 21 or the user terminal 10 analyzes the image to specify the positional relationship of each part. May be good.

Further, in the present embodiment, the position of the portion is assumed to be a position on a two-dimensional image, but the position is not limited to this and may be a three-dimensional position. For example, in addition to the camera 21, a depth camera can be placed at the installation location of the training equipment, and the three-dimensional position of the part can be specified based on the image from the camera 21 and the depth map from the depth camera. .. It is also possible to provide a depth camera instead of the camera 21 and specify the three-dimensional position only from the depth map from the depth camera.

Further, in the present embodiment, it is assumed that the part of the user's body to be photographed by the camera is the part viewed from the front direction or the lateral direction of the user, but the present invention is not limited to this, and the camera installed above the user Taking advantage of its characteristics, it may be a part viewed from above the user's head. In this case, the center of the user's head and the two-dimensional positions of both shoulders as seen from above can be specified and analyzed.

Further, in the present embodiment, the image of the body of the user during exercise is transmitted from the camera 21 to the server device 20, but the present invention is not limited to this, and the computer is placed at the installation location of the training equipment. A computer may extract a feature amount from an image and send the feature amount to the server device 20, or the computer estimates a body part based on the feature amount and the absolute position of the part (of the image). It may be a position on XY coordinates, it may be an actual distance from a reference position (for example, the ground, toes, head, center of gravity of the body, etc.), or it may be a position in any other coordinate system. It is also possible to acquire the relative positional relationship between a plurality of parts, and transmit these absolute positions and relative positional relationships to the server device 20.

Further, in the present embodiment, the content prepared on the server device 20 side is provided as the improvement measure information, but the present invention is not limited to this, and for example, the reference value is included so as to be correct based on the reference value. Bones that are postures (positions and angles of each part) may be superimposed and displayed on a moving image or a still image extracted from the moving image. As a result, it is possible to easily grasp what kind of posture should be taken.

Further, in the present embodiment, the position or movement (position over time) of the body part is evaluated, but the present invention is not limited to this, and the position of the tool gripped or worn by the user is specified and evaluated. You may do so. In this case, the reference information and the evaluation condition information about the position of the tool may be registered, the feature amount of the tool may be extracted, the position may be specified, and the evaluation may be performed in the same manner as the position of the body part.

Further, in the present embodiment, the content such as advice is provided for the improvement measure, but for example, the recommendation of the tool may be performed. In this case, the server device 20 stores the reference value of the tool and the size (length, etc.) of the tool in association with the physical information (height, weight, etc.) of the user, and the user uses the image data. The feature amount of the tool is extracted, the shape of the tool is specified, the size of the tool is estimated based on the shape and the size of the user (for example, height) included in the physical information, and the estimated size of the tool is used. If the difference from the reference value is equal to or greater than a predetermined threshold value, the tool having the size of the reference value can be recommended.

Further, by estimating the position of the tool, the server device 20 can estimate the speed of the tool. Further, the server device 20 can estimate the number of patterns as the number of operations using the tool by extracting the pattern of the change in the position of the tool in the time series.

Further, when a certain posture or movement is detected, a problem for the movement may be proposed. In this case, the server device 20 may store the task in place of the evaluation comment and output the task in association with one or a series of postures or movements.

In addition, when a certain posture or movement is detected, the training to be performed may be presented. In this case, the server device 20 may store the training content in place of the evaluation comment in association with one or a series of postures or movements, and output the content.

It is also possible to automatically detect the action performed by the player. In this case, the server device 20 stores the posture (position of each part of the body) for performing a predetermined operation such as shooting or passing as reference information, and the position and reference information of the body part analyzed from the image. Can be compared to identify the actions performed by the player in the image.

Further, in the present embodiment, the execution of a predetermined function and the storage of information are performed by the user terminal or the server device, but the present invention is not limited to this, and the execution of the function and the storage of information are performed by either device. You may do it. Alternatively, the functional unit and the storage unit may be distributed and provided in a form different from the present embodiment.

Further, in the present embodiment, the reference information (position reference information and angle reference information) is set in advance, but the reference information is not limited to this, and the reference information is created from the image data obtained by capturing a plurality of users 1. It may be. In this case, for example, it is possible to accept the input of whether or not the posture and movement of the user 1 are correct, and create the reference information based only on the ones evaluated as correct.

Further, in the present embodiment, information related to training, evaluation results, and the like are displayed on the display 22, or are output as voice from the voice output device worn by the user 1, but a message supporting the training is displayed. A voice output device worn by the user 1 may output a voice to be displayed on the 22 or to support the training. This makes it possible to improve the training motivation of the user 1.

10 User terminal 20 Server device 21 Camera 22 Display 23 Communication network spi 30 Administrator terminal 210 User authentication unit 211 Training information output unit 212 Image acquisition unit 213 Image analysis unit 214 Number of times counting unit 215 Evaluation unit 216 Evaluation information transmission unit 231 Training Information storage unit 232 User information storage unit 233 Image data storage unit 234 Reference information storage unit 235 Evaluation condition information storage unit 236 Training plan storage unit 237 Training history storage unit 238 Improvement condition storage unit

Claims (7)

A system that supports the movement of users
A camera that captures the user during the exercise and
A display installed at the place where the exercise is performed and
An explanatory output unit that outputs an explanation about the exercise to the display,
A part identification part that analyzes an image taken by the camera to identify the position of a part of the user's body,
An output unit that superimposes the position of the portion on the image and outputs it to the display.
An exercise support system characterized by being equipped with. The exercise support system according to claim 1.
A reference value storage unit that stores a reference value related to the position of at least one of the parts,
An evaluation unit that determines the evaluation value of the exercise by comparing the position of the portion in the image and the reference value.
With more
The output unit further outputs the evaluation value to the display.
An exercise support system featuring. The exercise support system according to claim 2.
The evaluation unit determines the evaluation value according to the distance from the reference value to the position.
An exercise support system featuring. The exercise support system according to claim 2.
The reference value storage unit stores the reference value for the angle between the straight line connecting each of the first and second portions and the third portion.
The site specifying portion identifies each of the first to third sites from the image, and then
The evaluation unit calculates the angle with respect to the specified first to third parts, compares the calculated angle with the reference value, and determines the evaluation value.
An exercise support system featuring. The exercise support system according to claim 1.
Further, a number counting unit for counting the number of times of the exercise based on the position of the portion is provided.
The output unit further outputs the number of times of the exercise to the display.
An exercise support system featuring. The exercise support system according to claim 2.
The output unit further outputs the evaluation value by voice.
An exercise support system featuring. The exercise support system according to claim 1.
The output unit further outputs the improvement measure information by voice based on the position of the body of the specified user.
An exercise support system featuring.