JP7008342B2 - Exercise evaluation system - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Published on the website "https://youtu.be/lsrAgAKVxMA" on November 1, 2018.

The present invention relates to a motion evaluation system.

Conventionally, as an analysis technique in the field of physical exercise, motion patterns that repeatedly appear in moving images are extracted, motion patterns at multiple positions at different time positions in the extracted moving images are compared from a predetermined viewpoint, and the comparison results are compared with the moving images. A technique for outputting as a still image (Patent Document 1), and in sports, for example, in golf, a technique of attaching a device to a user's body and analyzing a golf wing with a motion sensor (Patent Document 2). A technique (Patent Document 3) for calculating the transition time by photographing each stage of throwing (installation of a shaft foot, horizontal rotation of the trunk) with two cameras is disclosed.

Japanese Unexamined Patent Publication No. 2017-229081 Japanese Unexamined Patent Publication No. 2010-068947 Japanese Unexamined Patent Publication No. 2018-114138

However, any of the techniques of Patent Documents 1 to 3 requires equipment and resources such as a plurality of moving images, a specific device, and a plurality of cameras, and one user can incorporate these devices and the like to analyze physical exercise. It is difficult to understand the analysis results.

The present invention has been made in view of such a background, and an object of the present invention is to provide a technique capable of easily evaluating physical exercise.

The main invention of the present invention for solving the above-mentioned problems is a system for evaluating the movement of the body, which is a reference value storage unit for storing a reference value related to the position of at least one of the parts, and an image of the body. It is provided with a part specifying part that analyzes the image and identifies the part of the body, and an evaluation part that compares the position of the part and the reference value in the image to determine the evaluation value of the exercise.

Other problems disclosed in the present application and solutions thereof will be clarified by the columns and drawings of the embodiments of the invention.

According to the present invention, physical exercise can be easily evaluated.

It is a figure which shows the whole structure example of the exercise evaluation system which concerns on this embodiment. It is a figure which shows the hardware configuration example of the user terminal 10. It is a figure which shows the software configuration example of a user terminal 10. It is a figure which shows the structural example of the physical information which a physical information storage part 130 stores. It is a figure which shows the configuration example of the evaluation request which the evaluation request transmission unit 112 transmits to a server apparatus 20. It is a figure which shows the structural example of the evaluation information which the evaluation information receiving part 113 receives from a server apparatus 20. It is a figure which shows the configuration example of the improvement measure request which the improvement measure request transmission part 116 transmits to the server apparatus 20. It is a figure which shows the structure example of improvement measure information. It is a figure which shows the hardware configuration example of the server apparatus 20. It is a figure which shows the software configuration example of a server apparatus 20. It is a figure which shows the structural example of the image information stored in the image data storage unit 231. It is a figure which shows the structural example of the reference information which a reference information storage part 232 stores. It is a figure which shows the structural example of the evaluation condition information stored in the evaluation condition information storage unit 233. It is a figure which shows the structural example of the improvement condition information stored in the improvement condition storage part 234. It is a figure which shows the flow of the process executed in the exercise evaluation system of this embodiment. It is a figure explaining the case which imaged the moving image in the pitching mode. It is a figure which shows the example which displays the image of a checkpoint. It is a figure which shows an example of the evaluation screen of a moving image in a mode of weightlifting. It is a figure which shows an example of the evaluation screen of a moving image in a mode of weightlifting. It is a figure which shows an example of the evaluation screen of a moving image in a mode of weightlifting.

The contents of the embodiments of the present invention will be described in a list. The motion evaluation system according to the embodiment of the present invention has the following configurations.

[Item 1]
It is a system that evaluates body movements.
A reference value storage unit that stores a reference value related to the position of at least one of the parts, and a reference value storage unit.
A part specifying part that analyzes the image of the body to identify the part of the body, and
An evaluation unit that determines the evaluation value of the exercise by comparing the position of the portion in the image and the reference value.
An exercise evaluation system characterized by being equipped with.
[Item 2]
The exercise evaluation system according to item 1.
The evaluation unit determines the evaluation value according to the distance from the reference value to the position.
An exercise evaluation system featuring.
[Item 3]
The exercise evaluation system according to item 1.
The reference value storage unit stores the reference value for the angle between the straight line connecting each of the first and second portions and the third portion.
The site specifying portion identifies each of the first to third sites from the image.
The evaluation unit calculates the angle for the specified first to third parts, compares the calculated angle with the reference value, and determines the evaluation value.
An exercise evaluation system featuring.
[Item 4]
The exercise evaluation system according to item 1.
A comment storage unit that stores comments about the site in association with the conditions for the site and the position.
A comment output unit that outputs the comment corresponding to the condition that the position satisfies, and a comment output unit.
An exercise evaluation system characterized by further preparation.
[Item 5]
The exercise evaluation system according to item 1.
An improvement measure information storage unit that stores advice on improvement measures related to the exercise in association with the improvement target related to the exercise and the condition for the position.
An advice output unit that outputs the advice corresponding to the condition that the position satisfies, and
An exercise evaluation system characterized by further preparation.

The exercise evaluation system according to the embodiment of the present invention is intended to evaluate the physical exercise of the user. The user's physical exercise includes, for example, baseball pitching and weightlifting. The motion evaluation system of the present embodiment is an image obtained by capturing a state in which a user is exercising a body (a still image or a moving image may be used, but in the present embodiment, it is a moving image. ), The body part is identified, and the posture and movement of the body are evaluated based on the absolute position of the part and the relative positional relationship of multiple parts.

FIG. 1 is a diagram showing an overall configuration example of the exercise evaluation system according to the present embodiment. As shown in the figure, the motion evaluation system of the present embodiment includes a user terminal 10 and a server device 20. The user terminal 10 and the server device 20 are connected to each other so as to be able to communicate with each other via the communication network 30. The communication network 30 is, for example, the Internet or a LAN (Local Area Network), and is constructed of a public telephone line network, a dedicated telephone line network, a mobile telephone line network, an Ethernet (registered trademark), a wireless communication path, and the like.

The user terminal 10 is a computer operated by a user who performs physical exercise or a supporter thereof. The user terminal 10 is, for example, a smartphone, a tablet computer, a personal computer, or the like. The user terminal 10 includes an image pickup device such as a camera, which can image the user's body during exercise. In the present embodiment, it is assumed that the moving image of the body of the user during exercise is transmitted from the user terminal 10 to the server device 20.

The server device 20 is a computer that evaluates physical exercise. The server device 20 is, for example, a workstation, a personal computer, a virtual computer logically realized by cloud computing, or the like. The server device 20 receives the moving image taken by the user terminal 10, analyzes the received moving image, and evaluates the physical exercise. The server device 20 also makes a proposal for improving physical exercise. Details of the evaluation of physical exercise and the proposal of improvement measures will be described later.

FIG. 2 is a diagram showing a hardware configuration example of the user terminal 10. The user terminal 10 includes a CPU 101, a memory 102, a storage device 103, a communication interface 104, a touch panel display 105, and a camera 106. The storage device 103 stores various data and programs, such as a hard disk drive, a solid state drive, and a flash memory. The communication interface 104 is an interface for connecting to the communication network 30, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, and a wireless communication device for performing wireless communication. , USB (Universal Serial Bus) connector for serial communication, RS232C connector, etc. The touch panel display 105 is a device that inputs / outputs data. The user terminal 10 may further include an input device such as a keyboard, a mouse, a button, and a microphone, and an output device such as a speaker and a printer.

FIG. 3 is a diagram showing a software configuration example of the user terminal 10. The user terminal 10 includes an imaging unit 111, an evaluation request transmission unit 112, an evaluation information receiving unit 113, an evaluation display unit 114, a checkpoint display unit 115, an improvement measure request transmission unit 116, an improvement measure information receiving unit 117, and an improvement measure display unit. Each functional unit of 118 and each storage unit of a physical information storage unit 130, an image storage unit 131, an evaluation information storage unit 132, and an improvement measure storage unit 133 are provided.

Each of the above functional units is realized by the CPU 101 included in the user terminal 10 reading a program stored in the storage device 103 into the memory 102 and executing the program, and each of the above storage units is the memory 102 included in the user terminal 10. And implemented as part of the storage area provided by the storage device 103.

The image pickup unit 111 images the body of the user during exercise. By controlling the camera 106, the image pickup unit 111 can acquire a moving image of the body. The user or the user's supporter installs the user terminal 10 on a flat place, directs the optical axis of the camera 106 to the place where the user moves, and instructs the start of video shooting, and the imaging unit responds accordingly. The 111 may operate the camera 106 to acquire a moving image. The image pickup unit 111 stores the acquired moving image in the image storage unit 131.

The image storage unit 131 stores the image captured by the image pickup unit 111. In this embodiment, the image is a moving image. The image storage unit 131 can store a moving image as a file, for example.

The physical information storage unit 130 stores information about the user's body (hereinafter referred to as physical information). FIG. 4 is a diagram showing a configuration example of physical information stored by the physical information storage unit 130. As shown in the figure, physical information includes height, weight, flexibility, and the like.

The evaluation request transmission unit 112 transmits a request (hereinafter referred to as an evaluation request) for evaluating the physical exercise based on the image captured by the image pickup unit 111 to the server device 20.

FIG. 5 is a diagram showing a configuration example of an evaluation request transmitted by the evaluation request transmission unit 112 to the server device 20. As shown in the figure, the evaluation request includes a user ID, a mode, physical information and image data. The user ID is information that identifies the user. The mode is information indicating the exercise performed by the user. The mode can be, for example, "pitching" or "lifting". The mode shall be selected from predetermined options. The physical information is physical information stored in the physical information storage unit 130. The image data is moving image data acquired by the imaging unit 111.

The evaluation information receiving unit 113 receives information (hereinafter, referred to as evaluation information) related to the evaluation of physical exercise, which is responded to by the server device 20 in response to the evaluation request. The evaluation information receiving unit 113 registers the received evaluation information in the evaluation information storage unit 132.

FIG. 6 is a diagram showing a configuration example of evaluation information received from the server device 20 by the evaluation information receiving unit 113. As shown in the figure, the evaluation information includes a mode, a user ID, a position information, a posture information, a movement information, and a checkpoint information.

The user ID and mode are the user ID and mode included in the evaluation request. The captured image indicates that the body is photographed by the user performing the exercise indicated by the mode.

The position information indicates the position in the image of each part of the body (for example, head, shoulders, elbows, hips, knees, ankles, etc.). The position information includes a part and a position of the part in association with a time point on the time axis of the moving image. Based on the position information, the state (bone) of the skeleton of the body can be displayed. That is, for example, a figure (for example, a circle) indicating a part can be superimposed and displayed on an image at a position indicated by position information. It should be noted that the positions of a plurality of parts may be included at one time point. The position information may not be included in the part connecting the two parts (for example, the forearm connecting the wrist and the elbow or the thigh connecting the waist and the knee). In this case, by connecting a pair of marks (such as a circle) indicating two predetermined parts with a line, the part connecting these two parts can be expressed. The position information may be included for each frame constituting the video, for each key frame, for each frame of any number, or at a random time point. May be included. If My Frame does not contain location information, you can display bones based on the location information at the nearest past point in time.

Posture information is information related to the posture of the user's body. The posture information includes a part to be evaluated, a posture value, an evaluation rank, and an evaluation comment in association with a time point on the time axis of the moving image. The posture value is a value representing the posture. Posture values are, for example, the distance from the ground to the part, the distance between the two parts, the angle of the indirect part (the straight line from the first end part to the indirect part, and the second end part to the indirect part). The angle created by the straight line to). The evaluation rank is a value representing the evaluation value by rank. The evaluation rank is expressed by, for example, 1 to 5 in 5 stages, ABC, or the like. The evaluation comment is a comment related to the evaluation regarding the posture. For example, if the mode is "lifting" and the flexion is not sufficient, an evaluation comment such as "not sitting down" may be included.

The movement information is information related to the movement of the user's body. The motion information includes a part to be evaluated, a list of posture values, an evaluation rank, and an evaluation comment in association with a period on the time axis of the moving image. The list of posture values is a time-series posture value within a period. The evaluation comment is a comment related to the evaluation of movement. For example, if the mode is "lifting" and the knee extension is not smooth, it may include a rating comment such as "knee movement is not smooth".

The checkpoint information is information indicating a point (hereinafter referred to as a checkpoint) for checking a posture or the like in a series of movements of the user's body. Checkpoints include, for example, when the mode is "pitching", where the foot is raised, where the raised foot is lowered to shift weight, and where the ball is released. The checkpoint information stores information indicating a checkpoint (hereinafter referred to as a checkpoint ID) in association with a time point on the time axis of the moving image. That is, it is possible to specify a frame (still image) in the moving image in which the checkpoint indicated by the checkpoint ID is displayed.

The evaluation display unit 114 displays evaluation information. For example, the evaluation display unit 114 superimposes a figure representing a part of the body (for example, a circle representing an end or an indirect line and a line connecting them) on a moving image based on the position information included in the evaluation information. By displaying it, the movement of the bone can be superimposed on the moving image. Further, the evaluation display unit 114 can display, for example, a graph of changes in the position of the portion over time.

Further, the evaluation display unit 114 can display the evaluation rank and the evaluation comment together with the display of the moving image based on the posture information and the movement information included in the evaluation information. For example, the evaluation display unit 114 is included in the posture information when the playback time of the moving image is near the front and back of the time when it is included in the posture information (for example, it can be an arbitrary length such as around 5 seconds). It is possible to display the evaluation rank and evaluation comment that have been given. Further, the evaluation display unit 114 can display the evaluation rank and the evaluation comment included in the motion information when the reproduction time of the moving image comes within the period included in the motion information. Further, the evaluation display unit 114 can display the posture value included in the posture information. Further, the evaluation display unit 114 can display a graph of the time-series conversion of the posture values based on the list of posture values included in the motion information.

The checkpoint display unit 115 can extract and display a checkpoint image from a moving image. The checkpoint display unit 115 can read a frame corresponding to a time point included in the checkpoint information from the image data of the moving image stored in the image storage unit 131 and display it as a still image. Further, the checkpoint display unit 115 may, for example, extract and display only the body part from the read frame.

The improvement measure request transmission unit 116 transmits a request for acquiring the improvement measure regarding physical exercise (hereinafter, referred to as an improvement measure request) to the server device 20. FIG. 7 is a diagram showing a configuration example of an improvement measure request. As shown in the figure, the remedy request includes a user ID, a mode, and a purpose. The purpose is for the user to make improvements. The purpose can be, for example, "increasing the speed of the ball", "increasing muscle strength", "stabilizing the lower body", and the like. The purpose shall also be selected from the predetermined options.

The improvement measure information receiving unit 117 receives information on the improvement measure (hereinafter referred to as improvement measure information) transmitted from the server device 20 in response to the improvement measure request. The improvement measure information receiving unit 117 stores the received improvement measure information in the improvement condition storage unit 133. FIG. 8 shows a configuration example of the improvement measure information. As shown in the figure, the improvement measure information includes the purpose, advice, and standard information. In the present embodiment, the advice is assumed to be a character string representing the improvement measure, but may be content that presents the improvement measure by an image, a moving image, or the like. The reference information is a suitable body posture (position and angle of each part).

The improvement measure display unit 118 displays the improvement measure. The improvement measure information display unit 118 displays the advice included in the improvement measure information. In addition, if the improvement measure information includes a suitable position or angle of the part,

FIG. 9 is a diagram showing a hardware configuration example of the server device 20. The server device 20 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 stores various data and programs, such as a hard disk drive, a solid state drive, and a flash memory. The communication interface 204 is an interface for connecting to the communication network 30, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, and a wireless communication device for performing wireless communication. , USB (Universal Serial Bus) connector for serial communication, RS232C connector, etc. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, or the like for inputting data. The output device 206 is, for example, a display, a printer, a speaker, or the like that outputs data.

FIG. 10 is a diagram showing a software configuration example of the server device 20. As shown in the figure, the server device 20 is a functional unit of the evaluation request receiving unit 211, the image analysis unit 212, the evaluation unit 213, the evaluation information transmitting unit 214, the improvement measure request receiving unit 215, and the improving measure information transmitting unit 216. And each storage unit of the image data storage unit 231, the reference information storage unit 232, the evaluation condition information storage unit 233, and the improvement condition storage unit 234.

Each of the above functional units is realized by the CPU 201 included in the server device 20 reading a program stored in the storage device 203 into the memory 202 and executing the program, and each of the above storage units is the memory 202 included in the server device 20. And implemented as part of the storage area provided by the storage device 203.

The evaluation request receiving unit 211 receives the evaluation request transmitted from the user terminal 10. The evaluation request receiving unit 211 registers information including the image data included in the received evaluation request (hereinafter, referred to as image information) in the image data storage unit 231. FIG. 11 is a diagram showing a configuration example of image information stored in the image data storage unit 231. As shown in the figure, the image information includes image data in association with a user ID indicating a photographed user. The image data was included in the evaluation request.

The reference information storage unit 232 stores information (hereinafter referred to as reference information) including a reference value of a posture (position and angle of a portion) related to physical exercise. FIG. 12 is a diagram showing a configuration example of reference information stored in the reference information storage unit 232. As shown in the figure, the reference information includes reference information regarding the absolute position of the part or the relative position with respect to another part or another reference object (hereinafter referred to as position reference information), and three parts including the joint part. Includes angle reference information (hereinafter referred to as angle reference information) formed by a straight line connecting each of the two sites and the joint site.

The position reference information includes a part and a reference position of the part in association with the mode and the checkpoint ID. There may be multiple parts. Regarding the position, the vertical position may be, for example, the height from the ground or the distance from any of the toes. The horizontal position may be a distance from a predetermined reference object (for example, a mound plate, a mark on the floor, etc.) or a reference site such as a shoulder, chest, or foot. It is assumed that the position reference information is registered in advance.

The angle reference information includes two parts (part 1 and part 2), one joint part, a straight line connecting the part 1 and the joint part, and the part 2 and the joint in association with the mode and the checkpoint ID. Includes a reference value for the angle between the straight line connecting the site.

The evaluation condition information storage unit 233 stores information for performing evaluation (hereinafter, referred to as evaluation condition information). FIG. 13 is a diagram showing a configuration example of evaluation condition information stored in the evaluation condition information storage unit 233. The evaluation condition information includes categories, conditions, evaluation ranks, and comments. A category is a category of evaluation. The category can be, for example, "ball speed" or "control". The condition is a condition for the position or movement (change in position in time series) of each part of the body in the image. For example, when analyzing a pitching form, it is possible to set conditions for the angle of the elbow, the line speed of the arm, and the like for the checkpoint for releasing the ball in the evaluation condition information. The evaluation rank is an evaluation value when the above conditions are satisfied. The comment is an explanation of the posture and movement of the body when the above conditions are satisfied.

The image analysis unit 212 (site identification unit) analyzes the image data. The image analysis unit 212 analyzes the image data, extracts the feature amount of each part of the body, and specifies the position of each part in the image. As the image analysis method by the image analysis unit 212, a general method is adopted, and detailed description thereof will be omitted here. The image analysis unit 212 may analyze the image data frame by frame or key frame, or may analyze the image data at random timings.

The image analysis unit 212 also compares the position of each part extracted from the image data with the position reference information stored in the reference information storage unit 232 for each checkpoint ID, and sets the closest time point as the checkpoint time point. Specify as.

The evaluation unit 213 evaluates the movement of the user's body based on the image data. In the present embodiment, the evaluation unit 213 searches the evaluation condition storage unit 233 for evaluation condition information including the condition that the position of each part and the movement of the part specified from the image data are satisfied, and the evaluation condition information that the condition is satisfied. If there is, get the rating rank and comments included in it.

The evaluation information transmission unit 214 transmits the evaluation information to the user terminal 10. The evaluation information transmission unit 214 generates position information including a time point on the time axis of the moving image specified by the image analysis unit 212 and the position of each part. For the evaluation rank and comment acquired by the evaluation unit 213, if the position of the part satisfies the condition, the posture information including the time point, the part and the posture value, and the evaluation rank and the comment is generated, and the movement of the part (time series). If the condition (change in position in) is satisfied, motion information including a list of time points, parts and posture values, and evaluation ranks and comments is generated. Further, the evaluation condition transmission unit 214 generates checkpoint information including a time point corresponding to each checkpoint analyzed by the image analysis unit 212 and a checkpoint ID indicating the checkpoint. The evaluation information transmission unit 214 creates evaluation information including the generated position information, posture information, motion information, and checkpoint information, and transmits the evaluation information to the user terminal 10. The evaluation unit 213 and the evaluation information transmission unit 214 may correspond to the comment output unit of the present invention.

The improvement condition storage unit 234 stores information related to the improvement measure (hereinafter, referred to as improvement condition information). FIG. 14 is a diagram showing a configuration example of improvement condition information stored in the improvement condition storage unit 234. As shown in the figure, the improvement condition information includes advice in association with the purpose, category, and condition. The condition may be a condition for a physical condition (flexibility, etc.) or a condition for the position or movement of a part of the body.

The improvement measure request receiving unit 215 receives the improvement measure request transmitted from the user terminal 10.

The improvement measure information transmission unit 216 includes the physical information of the user included in the evaluation request and each part specified by the image analysis unit 212 among the improvement condition information corresponding to the mode and the purpose included in the improvement measure request. Search for a position or movement that meets the conditions. The improvement measure information transmission unit 216 acquires the advice of the searched improvement measure information, creates the improvement measure information for which the purpose and the advice are set, and responds the created improvement measure information to the user terminal 10. The improvement measure information transmission unit 216 also includes the position and angle of each part included in the reference information in the improvement measure information and transmits the information.

FIG. 15 is a diagram showing a flow of processing executed in the motion evaluation system of the present embodiment.

In the user terminal 10, the imaging unit 111 receives the input of the mode, images the body during the exercise of the user, and acquires the moving image data (S321). The evaluation request transmission unit 112 transmits an evaluation request including a user ID indicating a user, a received mode, physical information, and moving image data to the server device 20 (S322).

When the evaluation request receiving unit 211 receives the evaluation request in the server device 20, the image analysis unit 212 analyzes the moving image data to extract the feature amount (S323) and specifies the position of each part (S324). Here, the image analysis unit 212 may specify the position on the image, or may use the body information to determine the actual size position (height from the ground, distance from the reference point such as the center of gravity of the body, etc.). You may try to specify. The evaluation unit 213 acquires an evaluation rank and a comment from the evaluation condition information in which the position of each part and the movement of the part (change in time series of the position) satisfy the condition (S325). The evaluation information transmission unit 214 creates evaluation information and transmits it to the user terminal 10 (S326).

Based on the received evaluation information, the evaluation display unit 114 in the user terminal 10 displays the position (bone) of each unit indicating the posture of the body on the moving image data, and displays the evaluation rank and the comment (S327). Here, the evaluation display unit 114 may display a graph of changes in the position and movement of the portion over time. Further, the checkpoint display unit 115 may extract the checkpoint image from the moving image and display it. The improvement measure request transmission unit 116 transmits the improvement measure request to the server device 20 in response to an instruction from the user (S328).

In the server device 20, when the improvement measure request receiving unit 215 receives the improvement measure request transmitted from the user terminal 10, the improvement measure information transmitting unit 216 searches for the improvement condition information that satisfies the condition, and the searched improvement condition. The advice included in the information is acquired (S329), improvement measure information including the acquired advice is created, and the information is transmitted to the user terminal 10 (S330).

When the improvement measure information receiving unit 117 receives the improvement measure information in the user terminal 10, the improvement measure display unit 118 displays the advice included in the received improvement measure information and makes a moving image of a suitable body posture in the form of a bone. It can be superimposed on the data and displayed (S331).

As described above, according to the exercise evaluation system of the present embodiment, it is possible to easily evaluate the physical exercise. In particular, with regard to physical exercise related to sports, since it is possible to evaluate the positional relationship and movement of each part of the body, it is easy to lead to concrete improvement efforts, and improvement in performance is expected. Further, in the exercise evaluation system of the present embodiment, comments and advice are also provided, so that the user can easily grasp the current situation and improvement measures.

16 to 20 are diagrams showing an example of a screen displaying an evaluation or improvement measure of physical exercise.

FIG. 16 describes a case where a moving image in a pitching mode is captured. As shown in the figure, on the screen 41, the body is formed by a bone 411 indicating the position of each part of the body (for example, it may be defined as a line connecting 17 feature points such as eyes, ears, and joints). The posture of is displayed. Further, a comment corresponding to the deviation from the reference value is displayed in the comment column 412. The user can consider the posture to be corrected by referring to this.

FIG. 17 shows an example of displaying an image of a checkpoint. A checkpoint selection field 421 is displayed on the screen 42, from which a checkpoint can be selected. In the example of FIG. 17, the checkpoint 422 of the release point is selected, and the image 423 in the moving image corresponding to the checkpoint is displayed. This makes it possible to easily grasp the posture and movement of one's body at each checkpoint. As a checkpoint estimation method, for example, when the pitcher's upper body faces the front (throwing direction), when the coordinates of the pitcher's right hand (dominant hand) are the highest, and when the distance between the pitcher's shoulders is the widest. It can be determined that the release point has been reached when the conditions such as (when the distance is long) are satisfied.

18 to 20 are diagrams showing an example of a moving image evaluation screen in the weightlifting mode. In the example of the screen 43, in the bone 431, in addition to the predetermined position of the part specified from the image (the position where the user's buttocks reach the lowest point), the preferable posture set as the reference information is also displayed. Has been done. Further, on the screen 43, information 432 related to the movement of the shaft is displayed. In this way, it is possible to specify the position and movement of not only the body part but also the tool used by the user. This makes it possible to count, for example, how many times the weight is lifted from the image. Further, as shown in FIG. 19, a comment or advice 433 relating to the evaluation or improvement measure can be displayed on the screen 43 together with the moving image. Further, in the example of FIG. 20, a time-series graph 434 of the bending angle of the knee is displayed. The outlier 435 can also be detected by further analyzing the changes in graph 434. Also, although not shown, it is possible to display a time-series graph of the knee bending speed together with / or the graph 434. These graphs allow the user to be presented with the appropriate weight and number of weights.

Although the present embodiment has been described above, the above embodiment is for facilitating the understanding of the present invention, and is not for limiting the interpretation of the present invention. The present invention can be modified and improved without departing from the spirit thereof, and the present invention also includes an equivalent thereof.

For example, in the present embodiment, the image is analyzed by the server device 20, but the present invention is not limited to this, and the image may be analyzed by the user terminal 10 to specify the positional relationship of each part.

Further, in the present embodiment, the position of the portion is assumed to be a position on a two-dimensional image, but the position is not limited to this and may be a three-dimensional position. For example, when the user terminal 10 includes a depth camera in addition to the camera 106, the three-dimensional position of the portion can be specified based on the image from the camera 106 and the depth map from the depth camera. can. It is also possible to provide a depth camera instead of the camera 106 and specify the three-dimensional position only from the depth map from the depth camera. In this case, the depth map may be transmitted from the user terminal 10 together with the image data or instead of the image data to the server device 20, and the image analysis unit 212 of the server device 20 may analyze the three-dimensional position. can.

Further, in the present embodiment, the image of the body of the user during exercise is transmitted from the user terminal 10 to the server device 20, but the present invention is not limited to this, and the feature amount is extracted from the image in the user terminal 10. , The feature amount may be transmitted to the server device 20, or the user terminal 10 may estimate the body part based on the feature amount and set the absolute position of the part (the position on the XY coordinates of the image). However, it may be the actual distance from the reference position (for example, the ground, toes, head, center of gravity of the body, etc.), or it may be the position in any other coordinate system) or between multiple parts. The relative positional relationships of the above may be acquired, and these absolute positions and relative positional relationships may be transmitted to the server device 20.

Further, in the present embodiment, the content prepared on the server device 20 side is provided as the improvement measure information, but the present invention is not limited to this, and for example, the reference value is included to be correct based on the reference value. Bones that are postures (positions and angles of each part) may be superimposed and displayed on a moving image or a still image extracted from the moving image. This makes it possible to easily grasp what kind of posture should be taken.

Further, in the present embodiment, the position or movement (position over time) of the body part is evaluated, but the present invention is not limited to this, and the position of the tool gripped or worn by the user is specified and evaluated. You may do so. In this case, the reference information and the evaluation condition information about the position of the tool may be registered, the feature amount of the tool may be extracted, the position may be specified, and the evaluation may be performed in the same manner as the position of the body part.

Further, in the present embodiment, the content such as advice is provided for the improvement measure, but for example, the recommendation of the tool may be performed. In this case, the server device 20 stores the reference value of the tool and the size (length, etc.) of the tool in association with the physical information (height, weight, etc.) of the user, and the user uses the image data. The feature amount of the tool is extracted, the shape of the tool is specified, the size of the tool is estimated based on the shape and the size of the user (for example, height etc.) included in the physical information, and the estimated size of the tool is used. If the difference from the reference value is equal to or greater than a predetermined threshold value, the tool having the size of the reference value can be recommended.

Further, by estimating the position of the tool, the server device 20 can estimate the speed of the tool. Further, the server device 20 can estimate the number of patterns as the number of operations using the tool by extracting the pattern of the change in the position of the tool in time series.

Further, in the present embodiment, the movement is evaluated, but the present invention is not limited to this, and when a certain posture or movement is detected, a problem for the movement may be proposed. In this case, the server device 20 may store the task in place of the evaluation comment and output the task in association with one or a series of postures or movements.

Further, in the present embodiment, the movement is evaluated, but the present invention is not limited to this, and the training to be performed may be presented when a certain posture or movement is detected. In this case, the server device 20 may store the training content in place of the evaluation comment in association with one or a series of postures or movements, and output the content.

Further, in the present embodiment, the motion is evaluated, but the present invention is not limited to this, and the motion performed by the player can be automatically detected. In this case, the server device 20 stores the posture (position of each part of the body) for performing a predetermined motion such as a shoot or a pass as reference information, and the position and reference information of the body part analyzed from the image. Can be compared to identify the actions performed by the player in the image.

Further, in the present embodiment, the image captured in the past is analyzed to evaluate the motion, but the present invention is not limited to this, and when the analysis process is performed in real time and a predetermined motion is detected, the next action is taken. You may try to recommend the tactics that should be done. In this case, the tactics may be stored in place of the evaluation comment in association with the posture or movement, and the tactics may be output in real time.

Further, in the present embodiment, the execution of a predetermined function and the storage of information are performed by the user terminal or the server device, but the present invention is not limited to this, and the execution of the function and the storage of information are performed by either device. You may do it. Alternatively, the functional unit and the storage unit may be distributed and provided in a form different from the present embodiment.

10 User terminal 20 Server device 30 Communication network 111 Imaging unit 112 Evaluation request transmission unit 113 Evaluation information reception unit 114 Evaluation display unit 115 Checkpoint display unit 116 Improvement measure request transmission unit 117 Improvement measure information reception unit 118 Improvement measure display unit 130 Body Information storage unit 131 Image storage unit 132 Evaluation information storage unit 133 Improvement measure storage unit 211 Evaluation request storage unit 212 Image analysis unit 213 Evaluation unit 214 Evaluation information transmission unit 215 Improvement measure request reception unit 216 Improvement measure information transmission unit 231 Image data storage unit Unit 232 Reference information storage unit 233 Evaluation condition information storage unit 234 Improvement condition information storage unit

Claims (4)

It is a system that evaluates body movements.
A reference value storage unit that stores a reference value related to the position of at least one body part, and a reference value storage unit.
A part specifying part that identifies the part by analyzing the image of the body, and
An evaluation unit that determines the evaluation value of the exercise by comparing the position of the portion in the image and the reference value.
A checkpoint display unit that displays a plurality of checkpoints that are postures to be checked, which are included in the series of movements of the body, and the checkpoint display unit that simultaneously displays a plurality of figures representing the postures.
An improvement measure information storage unit that stores advice on improvement measures related to the exercise in association with the purpose of the improvement related to the exercise and the condition for the position.
Among the conditions corresponding to the purpose requested by the user, an advice output unit for outputting the advice corresponding to the condition satisfied by the position, and an advice output unit.
Equipped with
The evaluation unit evaluates based on the image associated with each of the plurality of checkpoints.
The checkpoint display unit accepts the selection of one of the plurality of checkpoints corresponding to the received figure by accepting the selection of one of the plurality of figures, and corresponds to the selected checkpoint. Displaying the attached image,
An exercise evaluation system featuring. The exercise evaluation system according to claim 1.
The evaluation unit determines the evaluation value according to the distance from the reference value to the position.
An exercise evaluation system featuring. The exercise evaluation system according to claim 1.
The reference value storage unit stores the reference value for the angle between the straight line connecting each of the first and second portions and the third portion.
The site specifying portion identifies each of the first to third sites from the image.
An exercise evaluation system featuring. The exercise evaluation system according to claim 1.
A comment storage unit that stores comments about the site in association with the conditions for the site and the position, and a comment storage unit.
A comment output unit that outputs the comment corresponding to the condition that the position satisfies, and a comment output unit.
An exercise evaluation system characterized by further preparation.

Images