JP2022158701A - Program, method, and information processor - Google Patents

Abstract

To provide a technique capable of more effectively evaluating the movement of a user and allowing the user to easily confirm the movement of a body by himself/herself.SOLUTION: In a method for a first user to perform training, a terminal of the first user causes a storage section to store a learned model on the basis of information based on point group data of one or more users. In the method, a processor of the terminal of the first user executes the steps of: identifying the content of coaching the first user on the basis of the information based on point group data of the first user and the learned model; and presenting the identified content of coaching to the first user.SELECTED DRAWING: Figure 29

Description

The present disclosure relates to programs, methods, and information processing apparatuses.

2. Description of the Related Art A technique is known in which a user's movement is captured by a camera or the like, and the user's movement is evaluated based on the captured moving image.

Japanese Patent Application Laid-Open No. 2016-81504 discloses a technology related to a device that recognizes the type of competition based on the skeleton of the human body using a depth camera, automatically evaluates the movement of the user using this, and determines the presence or absence of a step increase. Have been described.

JP 2016-81504 A

There is a need for a technique that can more effectively evaluate the user's movement and allow the user to easily check the movement of the user's body.

According to one embodiment, a method for training a first user, wherein a terminal of the first user stores in a memory a pre-trained training program based on information based on point cloud data of one or more users. The model is stored, and the method is such that the processor of the terminal of the first user, based on the information based on the point cloud data of the first user and the learned model, instructs the first user. A method is provided for performing the steps of identifying and presenting the identified instructional content to a first user.

According to the present disclosure, the user himself/herself can easily check the movement of the body.

1 is a diagram showing the overall configuration of a measurement system 1; FIG. 2 is a block diagram of a terminal device 10 configuring the measurement system 1 of Embodiment 1. FIG. 3 is a diagram showing a functional configuration of a server 20; FIG. 3 is a diagram showing data structures of a user information database 281, measurement item information 282, a measurement history database 283, and a friend list 284 stored by the server 20. FIG. The measurement system 1 shows the phase in which the user measures body movements. 4 is a diagram showing details of a mat 32; FIG. 6 is a flow chart showing a process in which the terminal device 10 performs a ROM check and updates the measurement history. 4 is a flow chart showing a process in which the terminal device 10 evaluates the range of motion of the joint by measuring the user's stepping motion. 4 is a flow chart showing a process in which the terminal device 10 evaluates the joint range of motion by measuring the user's upper body twisting motion. FIG. 10 is a diagram showing an example of the notification content of the terminal device 10 in the phase of measuring the movement of the user's body; A situation in which a user's walking motion is measured by the measurement system 1 according to the second embodiment is shown. Each part of the user's body specified by the terminal device 10 is shown based on the measurement result of the measurement device 31 . FIG. 10 is a diagram showing data structures of a measurement result database 287 and an evaluation result database 288 stored in the server 20 according to the second embodiment; 9 is a flow chart showing a process of measuring and evaluating a user's walking motion by the terminal device 10 in the second embodiment. FIG. 10 is a diagram showing an output example when the terminal device 10 outputs a measurement result of measuring the user's walking motion and an evaluation result of evaluating the user's walking motion based on the measurement result in the second embodiment. FIG. 10 is a diagram showing an output example when the terminal device 10 outputs a measurement result of measuring the user's walking motion and an evaluation result of evaluating the user's walking motion based on the measurement result in the second embodiment. FIG. 4 is a diagram showing an example in which the terminal device 10 outputs an evaluation result of the load on the knees in walking motion of the user. This corresponds to step S1409 in FIG. 10 is a flow chart showing a process of measuring and evaluating the balance of the user's trunk by the terminal device 10 in the third embodiment. FIG. 10 is a diagram showing an example in which the terminal device 10 outputs a measurement result of measuring the balance of the user's trunk and an evaluation result of the balance of the trunk in the third embodiment. FIG. 10 is a diagram showing data structures of various data stored in a server 20 according to the third embodiment; FIG. It is a screen example of a situation in which the terminal device 10 presents possible troubles to the user's body based on the measurement results and evaluation results of the user's bodily movements, and proposes an exercise menu corresponding to the troubles. 4 shows the processing flow of each device in the fourth embodiment. It is a figure which shows the data structure of the various data which the server 20 memorize|stores in the modification of 4th Embodiment. It is a figure which shows the flow of the operation|movement in the modification of 4th Embodiment. In a phase where the terminal device 10 presents possible physical concerns of the user based on the measurement results and evaluation results of the user's body movements and proposes an exercise menu corresponding to the concerns, the user answers a questionnaire. It is a screen example of a situation. FIG. 14 is a diagram showing data structures of a trainee information database 2601, a trainer information database 2602, and a training history database 2603 stored by the server 20 in the fifth embodiment. It is a flowchart which shows a series of processes which sense a 1st user's body motion as point cloud data, and transmit the information based on the 1st user's point cloud data obtained by sensing to a 2nd user's terminal. 4 is a flow chart showing a flow of processing for receiving an input of a trainer's voice from a trainer's terminal and transmitting it to a trainee's terminal. Sensing the movement of the first user's body as point cloud data, and specifying the instruction content for the first user based on the information based on the first user's point cloud data obtained by sensing and the trained model. 4 is a flow chart showing a series of processes to be performed. When the first user's terminal device 10A senses the movement of the first user's body as point cloud data and transmits information based on the first user's point cloud data obtained by sensing to the second user's terminal is a screen example of The first user including the overall configuration when sensing the first user's body movement as point cloud data and transmitting information based on the first user's point cloud data obtained by sensing to the second user's terminal and a screen example of a second user. It is an example of a screen when sensing the body movements of a plurality of users as point cloud data and transmitting information based on the point cloud data of each user obtained by sensing to a defined virtual space. 10A is a diagram showing a concept of processing for drawing an auxiliary line based on the estimation result of the user's posture by the terminal device 10A; FIG. 4 shows an example of a screen presented to the user.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<First embodiment>
<Overview>
In the following embodiments, by measuring the balance and movement of the user's body, the suppleness of the user's body is evaluated, and an exercise menu is provided so that the user can acquire a beautiful posture (for example, a beautiful walking posture). A measurement system 1 for providing will be described.

If the walking posture of the user is not in order, the user will perform a motion that is not necessarily required for walking, so the user's strength will be unnecessarily consumed, and fatigue will easily accumulate. By improving the user's walking posture, the user can move the body functionally, and as a result, the user can walk with less fatigue. Therefore, the measurement system 1 (1) measures the motion of the user during walking to evaluate the walking posture (evaluates whether the user can walk beautifully and with good balance); (3) Evaluating the user's muscle strength and providing a training menu. .

When the user is able to walk in a well-balanced manner (the way the body moves left and right is not biased, the movement of the muscles during walking is ideal, etc.), the way the user moves the body becomes functional. Here, the functional way of moving means that the user can move the muscles for walking without consuming unnecessary power. In other words, the user moves the muscles to obtain the driving force for walking, and the movement of the muscles leads to the driving force with high efficiency. As a result, it is possible to give the impression that the user is walking beautifully. It is effective to evaluate the joint range of motion in order to allow the user to learn such a well-balanced walking style. To provide a user with muscle training necessary for making a walking posture look beautiful on the premise.

As described above, the measurement system 1 provides measurement (checking) of the user's body movement and exercises for moving the user's body.

The measurement system 1 can be installed, for example, in an office of a business company, in a facility of a commercial facility, in a store space, in a studio, in an individual's home, or the like. For example, a business company may introduce the measurement system 1 in order to further encourage employees to manage their health. Also, by deploying the measurement system 1 in stores of commercial facilities, health management services may be provided to consumers.

<1 Configuration diagram of the entire system>
FIG. 1 is a diagram showing the overall configuration of a measurement system 1. As shown in FIG.

As shown in FIG. 1, the measurement system 1 includes a plurality of measurement units (measurement units 30A, 30B, and 30C in the illustrated example) configured by various sensors that measure the movement of the user's body. Each measurement unit is sometimes referred to as a “measurement unit 30”), and a terminal device 10 that controls the operation of the measurement unit (in the illustrated example, the terminal device 10 is shown). When describing a device, it may be described as "terminal device 10A", "terminal device 10B", etc. In addition, each element (such as a processor) that constitutes a terminal device may also be described in the same way.) and a server 20 that accumulates the measurement results of the user's body movements.

The measurement unit 30 is connected to the terminal device 10 by wire or wirelessly. As shown in the figure, a plurality of measurement units 30 may be connected to one terminal device 10, or the terminal device 10 may be connected to each measurement unit 30 (that is, one measurement unit 30 is connected to one measurement unit 30). It is also possible to prepare two terminal devices 10 and measure users.

The measurement unit 30 includes a measurement device (in the illustrated example, measurement devices 31A, 31B, and 31C are shown. These may be collectively referred to as “measurement device 31”), and a mat (in the illustrated example, , mats 32A, 32B, and 32C, which may be collectively referred to as “mats 32”) and three-dimensional poles (poles 33A, 33B, and 33C are shown in the illustrated example). These are sometimes collectively referred to as “pole 33”).

The measuring device 31 is for measuring the movement of the user's body, and detects the distance and direction to the measuring object by, for example, an RGB-D camera (depth sensor). The measurement device 31 may also include a microphone array or the like to detect the user's voice, the direction from which the voice arrives, and the like. For example, the measurement device 31 may be a device such as Kinect (registered trademark) that can accept user gestures and voice recognition operations. By detecting the user's body, the measuring device 31 detects the position of the user and the positions of the joints of the user's body. Thereby, the measuring device 31 sets a virtual bone (bone) that virtually connects each joint that constitutes the user, and measures the movement of the user's body. In addition, the measuring device 31 may be a terminal such as a smart phone in addition to the Kinect (registered trademark). For example, when a smartphone is provided with a depth camera, the smartphone can acquire the user's body as point cloud data (point cloud data).

In addition, even if the measuring device 31 does not have a function to measure the distance, the measurement device 31 can identify the subject based on the captured image and use a learned model for estimating the distance between the subject and the camera. The distance to the subject (for example, the distance to each joint of the subject) may be obtained based on the above. Also, even if the measuring device 31 cannot measure the distance (such as with an RGB camera), each part of the subject may be specified based on the captured image to estimate the skeleton (bone). Alternatively, a plurality of cameras may be provided as the measuring device 31 to obtain captured images with parallax, thereby estimating the distance to the subject.

The mat 32 is for displaying an indication of the user's standing position in order to measure the movement of the user's body. The mat 32 is arranged at a position having a predetermined positional relationship with respect to the measuring device 31 . That is, the mat 32 is placed within a range where the measurement device 31 can measure the movement of the user's body. The mat 32 includes a plurality of patterns for guiding the user's standing position, as will be described later. While viewing the pattern, the user adjusts the position of the foot or makes a stepping motion.

The poles 33 are used with the mat 32 to measure the motion of the user's body, eg, the user's twisting motion. The poles 33 are arranged on the mat 32 at positions associated with patterns indicating the standing positions on which the user's feet are placed. While keeping both feet on the mat 32 (that is, without moving the standing position of both feet), the user twists the body and touches the pair of poles placed on both sides of the mat 32 with the hands. Therefore, the mat 32 is used as a target point for moving the arm in order for the user to make a twisting motion. The mat 32 may be set to a height corresponding to the height of the user so that the user can hold it by hand when twisting the body.

As described above, the combination of the measuring device 31, the mat 32, and the poles 33 that constitute the measuring unit 30 has been described. In the following description, it is assumed that one user uses one measurement unit 30 . However, one measuring device 31 may measure the body movements of a plurality of users. In other words, a plurality of users may move their bodies within the measurement range of one measurement device 31 .

The terminal device 10 and the server 20 are connected for communication via the network 80 . Further, in the present embodiment, each device (terminal device, server, etc.) can also be understood as an information processing device. In other words, a set of devices can be understood as one "information processing device", and the measurement system 1 may be formed as a set of a plurality of devices. The method of distributing a plurality of functions required to implement the measurement system 1 according to this embodiment to one or a plurality of pieces of hardware depends on the processing capability of each piece of hardware and/or the specifications required for the measurement system 1. can be determined as appropriate in view of the above.

The terminal device 10 is a device operated by each user. The terminal device 10 may be, for example, a stationary PC (Personal Computer) or a laptop PC. In addition, the terminal device 10 is implemented by a mobile terminal such as a smart phone or a tablet compatible with a mobile communication system. As shown as terminal device 10B in FIG. The server 20 includes a communication IF 22 , an input/output IF 23 , a memory 25 , a storage 26 and a processor 29 .

The terminal device 10 is communicably connected to the server 20 via the network 80 . The terminal device 10 is compatible with communication standards such as 5G and LTE (Long Term Evolution), and wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11. is connected to the network 80 by communicating with the

The communication IF 12 is an interface for inputting and outputting signals so that the terminal device 10 communicates with an external device. The input device 13 is an input device (for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc.) for receiving an input operation from a user. The output device 14 is an output device (display, speaker, etc.) for presenting information to the user. The memory 15 temporarily stores programs and data processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage unit 16 is a storage device for storing data, and is, for example, a flash memory or a HDD (Hard Disc Drive). The processor 19 is hardware for executing an instruction set described in a program, and is composed of arithmetic units, registers, peripheral circuits, and the like.

As illustrated, the terminal device 10 is connected to the measurement unit 30 by wire or wirelessly. By measuring the movement of the user's body with the measurement unit 30, it is possible to evaluate the flexibility of the user's body and the extent of the joint range of motion.

The server 20 manages information of each user. The server 20 manages, as user information, parameters (such as height) that are referred to when measuring the body movements of each user.

Specifically, the server 20 performs processing for promoting interaction between users, such as user matching and the like, in order to assist a plurality of users in moving their bodies for health management and in measuring their body movements. I do. The server 20 also performs transmission and reception of messages between users.

The server 20 may manage virtual points owned by each user. For example, suppose there is an in-house point system that each employee can use like virtual currency within a business company. The points can be used for in-house purchase processing (can be used for payment at in-house vending machines, etc.) or given to other employees, like virtual currency. For example, the server 20 may give the points to the user when the user measures with the measurement system 1, or the user measures with the measurement system 1 together with other users (for example, the same The point may be given to the user when the measurement is performed at the same place, the measurement is performed at the same place at the same time, etc.), or the point is given when the evaluation result of the joint range of motion exceeds a predetermined value. good too.

The communication IF 22 is an interface for inputting and outputting signals for the server 20 to communicate with an external device. The input/output IF 23 functions as an interface with an input device for receiving input operations from the user and an output device for presenting information to the user. The memory 25 temporarily stores programs and data processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage 26 is a storage device for storing data, such as a flash memory or HDD (Hard Disc Drive). The processor 29 is hardware for executing an instruction set described in a program, and is composed of arithmetic units, registers, peripheral circuits, and the like.

<1.1 Configuration of terminal device 10>
FIG. 2 is a block diagram of the terminal device 10 configuring the measurement system 1 of the first embodiment. As shown in FIG. 2, the terminal device 10 includes a plurality of antennas (antenna 111, antenna 112), wireless communication units (first wireless communication unit 121, second wireless communication unit 122) corresponding to each antenna, and an operation unit. a reception unit 130 (including a touch-sensitive device 131; in addition, it may include various pointing devices such as a mouse and a touchpad; it may also include a keyboard and a gesture detection device); , an audio processing unit 140 , a microphone 141 , a speaker 142 , a position information sensor 150 , a camera 160 , a storage unit 180 and a control unit 190 . The terminal device 10 also has functions and configurations not particularly shown in FIG. 2 (for example, a battery for retaining power, a power supply circuit for controlling power supply from the battery to each circuit, etc.). As shown in FIG. 2, each block included in the terminal device 10 is electrically connected by a bus or the like.

The antenna 111 radiates a signal emitted by the terminal device 10 as radio waves. Antenna 111 also receives radio waves from space and provides a received signal to first wireless communication section 121 .

The antenna 112 radiates a signal emitted by the terminal device 10 as radio waves. Antenna 112 also receives radio waves from space and provides a received signal to second wireless communication section 122 .

The first wireless communication unit 121 performs modulation and demodulation processing for transmitting and receiving signals via the antenna 111 so that the terminal device 10 communicates with other wireless devices. The second wireless communication unit 122 performs modulation and demodulation processing for transmitting and receiving signals via the antenna 112 so that the terminal device 10 communicates with other wireless devices. The first radio communication unit 121 and the second radio communication unit 122 are communication modules including a tuner, an RSSI (Received Signal Strength Indicator) calculation circuit, a CRC (Cyclic Redundancy Check) calculation circuit, a high frequency circuit, and the like. The first radio communication unit 121 and the second radio communication unit 122 perform modulation/demodulation and frequency conversion of radio signals transmitted/received by the terminal device 10 and provide received signals to the control unit 190 .

The operation accepting unit 130 has a mechanism for accepting user input operations. Specifically, operation accepting unit 130 includes a touch-sensitive device 131 or the like as a pointing device. The touch-sensitive device 131 receives input operations of the user of the terminal device 10 . The touch-sensitive device 131 detects the user's touch position on the touch panel by using, for example, a capacitive touch panel. The touch-sensitive device 131 outputs a signal indicating the user's touch position detected by the touch panel to the control unit 190 as an input operation. The terminal device 10 may communicate with the measuring device 31 and receive the measurement result of the measuring device 31 to receive the user's input operation. For example, when the measurement device 31 acquires point cloud data, the user's body and each joint that constitutes the user's body are detected based on the point cloud data, and based on the detection result, the user's gesture is used as an input. accept. Gestures that can be input by the user include body movements such as user's hand movements and head movements. For example, the operation reception unit 130 may detect that the user takes a predetermined pose (raise both hands horizontally to take a “T”-shaped pose), or may detect that the user has made the hand into a predetermined shape. It is good also as detecting that it does.

The display 132 displays data such as images, moving images, and text under the control of the control unit 190 . The display 132 is implemented by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The terminal device 10 and the output device (display 132, speaker 142) may be implemented as separate devices.

The audio processing unit 140 modulates and demodulates an audio signal. Audio processing section 140 modulates the signal supplied from microphone 141 and supplies the modulated signal to control section 190 . Also, the audio processing unit 140 provides an audio signal to the speaker 142 . The audio processing unit 140 is implemented by, for example, a processor for audio processing. Microphone 141 receives a voice input and provides voice signal corresponding to the voice input to voice processing section 140 . The speaker 142 converts the audio signal provided from the audio processing unit 140 into audio and outputs the audio to the outside of the terminal device 10 .

The position information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. A GPS module is a receiving device used in a satellite positioning system. The satellite positioning system receives signals from at least three or four satellites, and detects the current position of the terminal device 10 equipped with a GPS module based on the received signals. For example, in the measurement system 1, when recording the place where the user performed the measurement, the detection result of the position information sensor 150 (latitude/longitude information, facility information specified by latitude/longitude and map information, associated with the facility) (such as information on the office where the measurement is performed) may be stored in the server 20 in association with the measurement result. In other words, the terminal device 10 may determine the location where the user performed the measurement based on the detection result of the position information sensor 150 without the user's operation of inputting the location.

The camera 160 is a device for receiving light with a light receiving element and outputting it as a photographed image. The camera 160 is, for example, a depth camera that can detect the distance from the camera 160 to the subject. In this embodiment, the measuring device 31 may be realized by the camera 160 of the terminal device 10. FIG. In other words, in addition to realizing the terminal device 10 and the measuring device 31 as separate devices, the terminal device 10 may be equipped with the functions of the measuring device 31 .

The storage unit 180 is configured by, for example, a flash memory or the like, and stores data and programs used by the terminal device 10 . In one aspect, storage unit 180 stores user information 181 , measurement item information 182 , measurement history information 183 , friend list 184 , and training item information 185 . The details of these will be described later, but the outline is as follows. The terminal device 10 may acquire these pieces of information from various databases of the server 20, which will be described later.

The user information 181 is user information in the measurement system 1 . The user information includes information for identifying the user, user's name, user's affiliation, user's physical information (height, weight, age, etc.) and the like.

The measurement item information 182 is information of each measurement item when ROM check is performed. The measurement item information 182 includes voice guidance information to be output to the user for each measurement item.

The measurement history information 183 indicates the history of the measurement result of measuring the user's body movement.

The friend list 184 shows information of each user registered as a friend between the users. Messages may be sent and received between users registered as friends.

The training item information 185 includes information on each training item when a muscle training menu is provided to the user. For example, it includes the name of the training, and video and audio information that guides the user to exercise. The terminal device 10 uses the measuring device 31 to measure the movement of the user's body based on the video and audio that serve as a guide, and sequentially displays the measurement results on the display 132 . On the display 132, the terminal device 10 compares the ideal way of moving the body with the measurement results (movements of each joint) when the user is moving the body, thereby obtaining the ideal way of moving the body. The user may be prompted to do so.

The control unit 190 controls the operation of the terminal device 10 by reading programs stored in the storage unit 180 and executing instructions included in the programs. The control unit 190 is, for example, an application processor. The control unit 190 functions as an input operation reception unit 191, a transmission/reception unit 192, a data processing unit 193, and a notification control unit 194 by operating according to a program.

The input operation reception unit 191 performs processing for receiving a user's input operation to the input device such as the touch sensitive device 131 . The input operation reception unit 191 determines whether the user's operation is a flick operation, a tap operation, or a drag (swipe) operation based on information on coordinates at which the user touches the touch-sensitive device 131 with a finger or the like. The type of operation, such as whether it is an operation or not, is determined.

The transmitting/receiving unit 192 performs processing for the terminal device 10 to transmit/receive data to/from external devices such as the server 20 and the measurement unit 30 according to a communication protocol.

The data processing unit 193 performs processing according to a program to perform computation on data input by the terminal device 10, and outputs the computation result to a memory or the like.

The notification control unit 194 performs processing for displaying a display image on the display 132, processing for outputting sound from the speaker 142, and the like. For example, the notification control unit 194 outputs audio guidance during a ROM check, and provides audio guidance and a video showing model body movements to the user when providing a training menu for exercise. Present.

<1.2 Functional Configuration of Server 20>
FIG. 3 is a diagram showing a functional configuration of the server 20. As shown in FIG. As shown in FIG. 3 , the server 20 functions as a communication section 201 , a storage section 202 and a control section 203 .

The communication unit 201 performs processing for the server 20 to communicate with an external device such as the terminal device 10 .

The storage unit 202 stores data and programs used by the server 20 . The storage unit 202 stores a user information database 281, measurement item information 282, a measurement history database 283, a friend list 284, and the like.

The user information database 281 shows information of each user who performs measurement using the measurement system 1 . Details will be described later.

The measurement item information 282 indicates information on each of a plurality of measurement items. Details will be described later.

The measurement history database 283 shows the history of measurements made by each user. Details will be described later.

The friend list 284 is a database for holding information on users registered as friends by each user.

The training item information 285 includes information on each training item when providing a training menu for exercise to the user. The training menu includes information on a plurality of training items according to the load on the user's body, the part of the body to be trained, and the like.

The control unit 203 exhibits functions shown as various modules when the processor of the server 20 performs processing according to a program.

The operation content acquisition module 2041 acquires the user's operation content. The operation content acquisition module 2041 performs, for example, an operation for the user to acquire past measurement results by the measurement unit 30 and an evaluation result of the user's body movement, an operation for transmitting and receiving messages with other users, and a measurement by the measurement unit 30. Accepts operations, etc. for applying to participate in measurement when there is an opportunity for

The reception control module 2042 controls processing for the server 20 to receive a signal from an external device according to a communication protocol.

The transmission control module 2043 controls the processing by which the server 20 transmits signals to external devices according to the communication protocol.

Matching module 2044 matches users to each other. The matching module 2044 may, for example, detect that multiple users have taken measurements using the measurement system 1 at the same measurement venue, match the multiple users, and prompt them to register as friends. In addition, the matching module 2044 may extract and match users with similar histories of measurements performed by the measurement system 1 , or extract and match users with similar ROM check results by the measurement unit 30 . It is also possible to let By matching a plurality of users and urging them to register as friends in this manner, it is possible to encourage a plurality of users to train together with the aim of achieving a beautiful posture. can support

The measurement progress module 2045 progresses the process of measuring the movement of the user's body by the measurement unit 30 . The measurement progress module 2045 reads out each measurement item shown in the measurement item information 282, and sequentially outputs to the user an audio guide or the like for measurement according to the measurement item. For example, the server 20 may transmit each measurement item to the terminal device 10 and cause the terminal device 10 to sequentially measure the movement of the user's body according to each measurement item.

The training progress module 2046 progresses the process of providing the user with a training menu by video and audio. The training progress module 2046 reads out each training item indicated in the training item information 285 and sequentially outputs audio guidance, video guidance, etc. for training according to the training item to the user.

<2 Data structure>
FIG. 4 shows the data structures of the user information database 281, measurement item information 282, measurement history database 283, and friend list 284 stored by the server 20. As shown in FIG.

As shown in FIG. 4, each record of the user information database 281 includes an item "user identification information (user ID)", an item "user name", an item "gender", an item "height", and an item " weight", item "affiliation", item "evaluation value", item "company points", and the like.

The item “user identification information (user ID)” is information for identifying each user.

The item "user name" is the name set by the user.

The item "gender" indicates the gender of the user. For example, the user may set the gender when registering user information to start using the measurement system 1 .

The item "height" indicates the user's height as information to be referred to when measuring the user's body movement.

The item "weight" indicates information on the weight of the user. In cases such as when weight information is not referred to when measuring the user's body movement, data for the item "weight" may not be retained.

The item "affiliation" indicates information on the user's affiliation, such as when the user belongs to an organization. For example, the item "Affiliation" includes information such as company name, organization name, and office name. The item “affiliation” may include information on the location of the office where the user works.

The item "evaluation value" includes information on an evaluation value evaluated based on the measurement results of the ROM check performed by the user in the measurement system 1, and an evaluation value based on the measurement results of the user's walking posture. The evaluation value includes, for example, classification result information when the terminal device 10 classifies the user's ROM check result based on the measurement result by the measurement unit 30 .

The item "in-house points" indicates the amount of points held by the user when there are virtual points used in the operating company to which the user belongs.

Each record of the measurement item information 282 includes the item “measurement item number (measurement item No.)”, the item “measurement site”, the item “guidance voice”, the item “foot position on the mat”, and the item “ part of the body to be measured".

The item "measurement item number (measurement item No.)" is information for identifying each measurement item.

The item "measurement site" indicates the site of the body to be measured. For example, there are the following parts as targets for evaluating joint range of motion.

(i) Measurement of Upper Limb Scapula: Directions of movement include flexion, extension, fist raising, and pulling down.
・Shoulder: Movement directions include flexion, extension, abduction, adduction, external rotation, and internal rotation.
・ Elbow: There are flexion and extension as motion directions.
・ Forearm: There are pronation and supination as directions of movement.
・ Hands: Movement directions include flexion, extension, flexion, and ulnar flexion.

(ii) Measurement of fingers - Thumb: Movement directions include flexor side abduction, ulnar side adduction, volar side abduction, volar side adduction, flexion, and extension.
・ Fingers: Movement directions include flexion, extension, abduction, and adduction.

(iii) Measurement of Lower Limbs · Crotch: Movement directions include flexion, extension, abduction, adduction, external rotation, and internal rotation.
・ Knee: Movement directions include flexion and extension.
・ Legs: Movement directions include flexion and extension.
・Foot: Movement directions include rolling, rolling, abduction, and adduction.
・ Thumb: Movement directions include flexion and extension.
・ Toes: Movement directions include flexion and extension.

(iv) Measurement of Trunk Neck: Movement directions include flexion, extension, rotation (left rotation, right rotation), and lateral bending (left bending, right bending).
・ Thoracic and lumbar region: Movement directions include flexion, extension, rotation (left rotation, right rotation), and lateral bending (left bending, right bending).

The item "guidance voice" indicates the content of the voice to be notified to the user when performing the measurement indicated in the measurement item. As a voice guide, there is a voice when the user does not put his/her feet on the prescribed positions on the mat 32 (for example, "Please place both feet on the prescribed positions on the mat." , the terminal device 10 detects that the user has placed his/her feet on the specified position, and prompts the user to move the body (for example, "Twist your body while placing both feet on the specified position on the mat"). , and a voice when the measurement data of the user's body movement is obtained based on the measurement result of the measurement device 31 (for example, "It was measured normally").

The item “position of feet on mat” includes information on where the user's feet are placed on the mat 32 (positions where the user places their feet on the mat 32) when performing the measurement. For example, the terminal device 10 detects whether the user's feet are placed on the specified positions on the mat, based on the information in the item "feet positions on the mat". By this detection, the terminal device 10 outputs to the user a sound indicating the content of the body motion for measurement.

The item “body part to be measured” indicates the part of the user's body that is to be measured in the measurement item.

Each record in the measurement history database 283 includes the item “measurement number (measurement No.)”, the item “date and time”, the item “user identification information (ID)”, the item “measurement location”, and the item “measurement data”. and the item "evaluation result". The server 20 receives from the terminal device 10 the result of measuring the user's body movement. The server 20 updates the record of the measurement history database 283 based on the information received from the terminal device 10 .

The item “measurement number (measurement No.)” indicates each measurement result when the user measures the movement of the body with the measurement unit 30 .

The item “date and time” indicates the timing at which the measurement was performed by the measurement unit 30 by the user. .

The item “user identification information (ID)” indicates information identifying the user whose body movement was measured by the measurement unit 30 .

The item “measurement place” indicates the place where the user performed the measurement with the measurement unit 30 . When the server 20 receives the measurement result from the terminal device 10, the server 20 refers to the user information associated with the measurement result, and the user performs the measurement based on the information associated with the user (user information database 281). You may specify the location where For example, based on the information on the user's affiliation shown in the user information database 281, the information on the place where the relevant affiliation is located may be used as the place where the measurement was performed by the measurement unit 30. FIG. Alternatively, the position of the terminal device 10 may be set as the location where the measurement is performed by the measurement unit 30 by the terminal device 10 acquiring the position information by the position information sensor 150 or the like.

The item “measurement data” indicates the measurement result of the user's body movement measured by the measurement unit 30 . For example, when the user's body movements are measured as time-series point cloud data, the time-series point cloud data may be the measurement results in the item "measurement data." In addition, with respect to the user's body measured by the measuring device 31, the terminal device 10 identifies the positions of the user's joints based on the measurement results, and the movement of each joint is recorded as the measurement result of the user's body movement in the server 20. It may be held. For example, the terminal device 10 can measure time-series movements of the user's joints by detecting specific parts of the user's body, such as shoulders and elbows, based on captured images of the user's body.

The item "evaluation result" indicates the evaluation result of evaluating the level of the user's body movement based on the measurement result indicated in the item "measurement data". For example, when the terminal device 10 holds the time-series data of the user's joint movements in the item "measurement data", the terminal device 10 evaluates the user's body movements by comparing the data with the reference body movement data. do. For example, the terminal device 10 may hold, as reference data, multiple stages of data according to the extent to which the user's body movement is favorable, thereby determining the most similar movement to the measurement result of the user's body movement. Data to be referenced can be specified.

Each of the records in the friend list 284 includes items such as the item “user 1”, the item “user 2”, the item “friend registration date”, the item “measurement history at the same timing”, and the item “message transmission/reception history”. include.

The item "user 1" indicates one of the users who are mutually registered as friends.

The item "user 2" indicates the other user among the users who are mutually registered as friends.

The item "friend registration date" indicates the date and time when the users were registered as friends.

The item “history of measurements at the same timing” indicates histories of measurements performed by each user registered as a friend while satisfying at least one of the same location or the same timing. For example, when the measurement unit 30 is arranged in an office space of a business establishment, a plurality of users may perform measurements during a certain period of time, and the plurality of users may perform measurements at the same timing. . The item “history of measurements at the same timing” may include information on locations where measurements were taken by a plurality of users. For example, by referring to the friend list 284, the server 20 can specify the frequency with which a plurality of users both measure their body movements with the measuring unit 30 and train with a training menu.

The item "message transmission/reception history" indicates the history of messages transmitted and received between users registered as friends.

<3 Configuration of measurement unit 30>
FIG. 5 shows a phase in which a user measures body movements with the measurement system 1 . FIG. 5 shows an example in which a plurality of measurement units 30 are installed and a plurality of users perform measurements. Also, an example in which one terminal device 10 is connected to a plurality of measurement units 30 is shown.

The measuring device 31A measures the body movement of the first user User_A in a sensing range 38A. In the detection range 38A, the mat 32A is installed at a position having a predetermined positional relationship with the measuring device 31A. Here, the position of the mat 32A may be, for example, at a certain distance (for example, several meters) from the front of the measuring device 31A. The mat 32A may be installed with respect to the measuring device 31A so that the measuring device 31A can measure the movement of the user's whole body when the first user User_A gets on the mat 32A. Poles 33A are installed on the right and left sides of the mat 32A, respectively. For example, if the mat 32A has a pattern indicating a predetermined position (which may be referred to as a home position) for the user to place both feet, a set of poles 33A are placed outside the mat 32A based on the pattern. It may be arranged. The pole 33A is installed as a reference for the user to grasp when the user makes a motion of twisting the upper limb on the mat 32A with both feet on the prescribed position. As a result, when the user twists the upper limbs, the user is urged to move the hand toward the pole 33A, so that the movement form of the upper limbs can be stabilized to some extent even when the measurement opportunities are increased. In other words, it is possible to suppress variations in time-series measurement results when measuring the user's body movement.

The measuring device 31B measures the body movement of the second user User_B in a sensing range 38B. In the detection range 38B, the mat 32B is installed at a position having a predetermined positional relationship with the measuring device 31B. Here, the position of the mat 32B may be, for example, a certain distance (about several meters) from the front of the measuring device 31B. The mat 32B may be installed with respect to the measuring device 31B so that the measuring device 31B can measure the motion of the user's whole body when the second user User_B gets on the mat 32B. Poles 33B are installed on the right and left sides of the mat 32B, respectively.

The first user User_A checks the position of the foot based on the pattern of the mat 32A and moves the body while viewing the video and audio displayed on the display 132A. Similarly, the second user User_B moves his or her body according to the video and audio guidance displayed on the display 132B. Thereby, the body movements of the first user User_A and the second user User_B can be measured by the measurement devices 31A and 31B, respectively.

In the illustrated example, based on the output from the display 132A, the first user User_A is about to start bending and stretching after placing both feet on the home position on the mat 32A. The second user User_B is making an action of stepping his right foot according to the output from the display 132B.

FIG. 6 is a diagram showing the details of the mat 32. As shown in FIG. As shown in the figure, patterns are arranged at a plurality of locations as guidelines for placing the user's right and left feet.

For example, for the right foot, there are a first home position 41R, a second home position 42R, and a third home position 43R where the right foot is placed at a position that serves as a guide for placing the user's feet over a plurality of measurement items. are shown multiple times.

For the left foot, a plurality of positions for placing the left foot are shown, such as a first home position 41L, a second home position 42L, and a third home position 43L.

In the following description, the home positions 41R and 41L may be collectively referred to as the home position 41 or the like. Similarly, the home positions 42R and 42L may be collectively referred to as the home position 42, and the home positions 43R and 43L may be collectively referred to as the home position 43.

For each measurement item (measurement item information 282), the user defines the distance between the right and left legs, and then the terminal device 10 guides the user to perform a predetermined body movement. For example, information on the position of the home position to be used may be provided in the audio guidance that accompanies the measurement of the measurement item.

When measuring the measurement items, the terminal device 10 determines whether or not the user is using a predetermined home position based on the measurement result of the measurement device 31 . For example, when the terminal device 10 identifies each pattern of the home position shown on the mat 32 based on the sensing result of the measuring device 31, the user's ankles and the like are detected based on the user's feet. It may be possible to identify the pattern that is present. If the user puts his/her feet in a position different from the assumed home position when measuring the measurement item, the user is prompted to place their feet in an appropriate position by voice guidance, video guidance, etc. Notification may be made.

In addition, the terminal device 10 includes, as a measurement item, the terminal device 10 measuring the movement of each joint when the user makes a stepping motion with the measurement unit 30 . The terminal device 10 can evaluate the goodness of balance, etc., when the user moves his/her body by taking a stepping motion, using the measurement items.

The mat 32 includes a pattern group 45R that serves as a target for stepping the right foot when there is a stepping action as a measurement item. Similarly, a pattern group 45L serving as a target for stepping the left foot is included.

These pattern groups 45R and 45L may be collectively referred to as "pattern group 45".

The pattern group 45 includes a plurality of patterns that serve as guides for the stepping position according to the height of the user. In the illustrated example, patterns are arranged in increments of 10 cm in the range of user's height from "140 cm" to "200 cm". By using a shape that imitates the toe portion of the foot as the pattern, it is possible to make the user easily recognize that it is a target position to step on. For example, the user places both feet at home positions 41R and 41L or patterns 44R and 44L (these may be collectively referred to as "pattern 44") as initial positions, and moves one foot at a time according to height. Then, step forward to the target pattern position. The terminal device 10 uses the measurement unit 30 to measure the movement of the user's body (motion of joints) when performing the stepping motion. When measuring the stepping action of the user, the terminal device 10 designates the home positions 41 to 43 or the pattern 44 as the initial position of the foot according to the distance that the user wants the user to step. It may be used as a guide.

The illustrated example also shows the position of the pole 33 . For example, the poles 33 (right side and left side) may be arranged so as to line up with the home positions 41 , 42 , 43 in a straight line. For example, the user tries to grasp the pole 33 by twisting the upper limb while placing both feet on one of the home positions 41 , 42 and 43 .

As shown in the figure, patterns of arrows may be arranged in the pattern groups 45R and 45L. In the illustrated example, the arrow pattern is superimposed on the pattern groups 45R and 45L corresponding to each height. This makes it possible for the user to be conscious of stepping forward, and reduces the possibility that the user will hesitate about how to move the body during measurement when the user looks at the mat 32.例文帳に追加

Also, as shown in the figure, regarding the size of the pattern, since the pattern serves as a guideline for the user to place the foot, the length in the vertical direction (the direction in which the foot is stepped) in the example shown in the figure is the size of a typical foot. It may be about the same as the size (about 22 cm to 28 cm in length). The shape of the pattern may be a footprint or a shape similar to a footprint so that the user can easily understand that a foot is placed on the pattern.

<4 Operation>
A measurement process in which the terminal device 10 measures the movement of the user's body will be described below.

FIG. 7 is a flow chart showing a process in which the terminal device 10 performs a ROM check and updates the measurement history.

In step S700, the control unit 190 of the terminal device 10 displays on the display 132 a screen for logging in by the user who performs the measurement, and receives an operation for logging in from the user. The terminal device 10 performs user authentication by, for example, reading information from a recording medium such as an IC card possessed by each user, reading a two-dimensional code displayed on a communication terminal possessed by the user, and the like. Get information. For example, by communicating with the server 20, the terminal device 10, as information on the user, includes information such as the user's height (user information database 281), past measurement history (measurement history database 283), registration of the user as a friend. Acquire information (friend list 284) of other users who have Further, the terminal device 10 acquires measurement item information (measurement item information 282 ) from the server 20 and stores it as measurement item information 182 .

In step S<b>701 , the terminal device 10 detects that the mat 32 and the poles 33 are installed within the measurement range of the measuring device 31 based on the measurement result of the measuring device 31 . For example, the terminal device 10 detects the mat 32 and the poles 33 by recognizing the measurement result of the measuring device 31 as a two-dimensional image. The terminal device 10 detects that the mat 32 and the poles 33 are at predetermined positions with respect to the measuring device 31 . In other words, the terminal device 10 detects that the mat 32 and the pole 33 are in positions suitable for measurement and in positions where the user can easily check the audio and video of the guide.

In step S703, the terminal device 10 refers to the measurement item information 182 and performs processing for sequentially measuring each measurement item. The terminal device 10 reads out the first measurement item and outputs information (such as the position where the user places his or her foot on the mat 32) that serves as a guide for measurement by audio and video. The terminal device 10 detects the joints of the user based on the measurement result of the measuring device 31, and detects that the user's feet are placed on predetermined positions (positions defined by the measurement items) on the mat.

In step S705, the terminal device 10 outputs a voice prompting the user to perform a predetermined physical action. The terminal device 10 measures the movement of the user's body by detecting the movement of the user's body with the measurement device 31 .

In step S707, the terminal device 10 sequentially reads out the measurement items from the measurement item information 182, outputs audio guidance, detects that the user's feet are at a specified position on the mat 32, and detects that the user's body is moving. to measure.

In step S709, when the measurement for each measurement item is completed, the terminal device 10 evaluates the range of motion of the user's joint based on the measurement result of the user's body movement. Specifically, the terminal device 10 identifies the position of each joint of the user based on the measurement results of the measurement device 31, and uses time-series data of the joint position (motion of the joint position that changes in time series) to measure Assess the range of motion of the target body part. The terminal device 10 outputs the evaluation result of the range of motion of the joint on the display 132 or the like, and transmits it to the server 20 .

In step S753, the server 20 receives the result of the user's ROM check from the terminal device 10, and updates the databases (measurement history database 283, user information database 281) based on the user's measurement result.

Next, processing of each device when measuring a user's stepping action as a measurement item will be described.

FIG. 8 is a flow chart showing a process in which the terminal device 10 evaluates the joint range of motion by measuring the user's stepping motion.

In step S801, the terminal device 10 confirms that both the user's feet are placed on the prescribed positions of the mat 32 (item "Position of feet on the mat" of the measurement item information 282) based on the measurement result of the measurement device 31. to detect The terminal device 10 may guide the user to put the feet on the prescribed positions when the user's feet are not on the prescribed positions.

In step S803, the terminal device 10 outputs audio and video guidance to the user to step the user's right foot from a prescribed position to a stepping position according to the height of the user.

In step S805, the terminal device 10 detects that the user's right foot has stepped from a specified position to another position on the mat based on the measurement result obtained by the measurement device 31, and measures the movement of the user's right foot stepping. Record the results. When the user's right foot is stepped to a position different from the target position to be stepped, the terminal device 10 guides the user to step the right foot again to the target position in order to re-measure. You can do it.

In step S807, the terminal device 10 outputs a voice guiding the user to step the left foot of the user from a prescribed position to a step position corresponding to the height (step length) of the user.

In step S809, the terminal device 10 detects that the user's left foot has stepped from the specified position to another position on the mat based on the measurement result of the measurement device 31, and measures the action of the user stepping the left foot. Record the results.

In step S<b>811 , the terminal device 10 evaluates the range of motion of the joints of the lower limbs based on the measurement result of the user's stepping motion on the mat 32 , outputs the evaluation result to the display 132 or the like, and transmits it to the server 20 .

Next, the processing of each device when measuring the user's upper body twisting motion will be described.

FIG. 9 is a flow chart showing a process in which the terminal device 10 evaluates the joint range of motion by measuring the user's upper body twisting motion.

In step S<b>901 , the terminal device 10 detects that the user's feet are placed on prescribed positions on the mat based on the measurement result of the measurement device 31 .

In step S903, the terminal device 10 determines whether the height of the pole 33 is appropriate based on the height information of the user. For example, the terminal device 10 associates and stores information on the height of the pole 33 that is appropriate according to the height of the user, so that the height of the pole 33 detected by the measurement result of the measuring device 31 is appropriate. Determine whether or not there is For example, the appropriate height of the pole 33 may be defined according to the position of the user's hands when the user stands upright. Here, the position of the hands when the user stands upright may vary depending on the height of the user. In other words, it is assumed that the higher the height, the higher the position of the hands when standing upright. In addition, the terminal device 10 detects the position of the user's hand (the height from the ground) in the upright posture based on the measurement result of the measuring device 31, thereby determining the appropriate height of the pole 33. It may be determined based on the position of the hand. That is, the height of the pole 33 may be adjusted according to the measurement result of the user's body by the measuring device 31 .

The height of the pole 33 is measured by specifying a first position where the pole 33 is in contact with the ground and a second position which is the top of the pole 33 based on the measurement result of the measuring device 31, for example. The determination may be made based on information on the distance from the position of the device 31 to the first and second positions of the poles 33 . In addition, when the pole 33 has a structure that can be extended, the pattern of the pole 33 includes a pattern indicating the height of the pole 33, and the terminal device 10 indicates the height of the pole 33. A pattern may be detected.

The terminal device 10 may determine whether or not the position where the pole 33 is installed is appropriate. For example, the terminal device 10 detects a pattern indicating a home position or the like on the pole 33, and determines whether the position of the pole 33 is appropriate based on the positional relationship between the position of the pattern and the position where the pole 33 is in contact with the ground. It may be determined whether or not there is.

In step S905, the terminal device 10 twists the user's body to the right while keeping both feet on the mat at prescribed positions, and the left hand is placed on the right pole (the pole 33 placed on the right) and the right hand is placed on the left pole (the left pole). It outputs voice and video guidance to grasp the pole 33).

In step S<b>907 , the terminal device 10 records the measurement result of the user's body twisting motion to the right based on the measurement result of the measurement device 31 .

In step S909, the terminal device 10 outputs a voice guiding the user to twist the user's body to the left while keeping both feet on the mat, and grasp the left pole with the right hand and the right pole with the left hand.

In step S911, the terminal device 10 records the measurement result of the user's body twisting motion to the left based on the measurement result.

In step S913, the terminal device 10 evaluates the range of motion of each joint of the body based on the measurement result of the user's twisting motion, and outputs the evaluation result. As the evaluation of the range of motion of the joint, whether or not the user was able to grasp the pole 33 by twisting the upper body may also be used as an evaluation item.

As described above, by setting the height of the pole 33 to be approximately the same as the height of the user's hand when the user stands upright, the user can visually recognize the pole while looking back when the user twists the upper body. With 33 as the target, try to move the hands and arms. As a result, it is possible to reduce the fluctuation width of the user's motion of twisting the upper body, and when the terminal device 10 accumulates the measurement results, it becomes easy to compare time-series data of the measurement results.

<5 Screen example>
FIG. 10 is a diagram showing an example of notification contents of the terminal device 10 in the phase of measuring the user's body movement.

The screen example of FIG. 10A shows a situation in which the terminal device 10 accepts an operation of logging in for measurement from the user. This corresponds to the processing of step S700 in FIG. For example, the measurement unit 30 is installed in a business office, store space, or the like of a business company, and when a plurality of users use it, user authentication is first performed.

As shown in FIG. 10A, the terminal device 10 displays a notification section 132L on the display 132. FIG. The terminal device 10 displays the operation content for the user to log in on the notification unit 132L. In the illustrated example, assuming that the measurement unit 30 is installed in the office, the notification section 132L indicates that login is possible by reading the user's employee ID card. The terminal device 10 displays in the notification section 132C that another user is about to perform measurement at the same timing. As in the example of FIG. 5, when a plurality of measurement units 30 are installed and a plurality of users perform measurement together, the terminal device 10 displays the progress of another user performing measurement on the notification unit 132C. . In the example of FIG. 10A, the terminal device 10 displays in the notification section 132C that another user who performs the measurement at the same timing has already completed the login process.

The screen example of FIG. 10B shows a situation in which the terminal device 10 measures the user's stepping motion. This corresponds to the processing of steps S801 to S811 in FIG.

As shown in FIG. 10B, the display 132 of the terminal device 10 displays a notification section 132D, a notification section 132E, a guide output section 132F, and a detection result display section 132G.

The terminal device 10 displays the information of the logged-in user (that is, the user who is performing the measurement) on the notification section 132D. Thereby, the user can easily check whether or not the measurement is performed as another user by mistake.

The terminal device 10 displays the history of the user's measurements on the notification unit 132E. In the illustrated example, the number of times the user has performed measurements using the measurement unit 30 is displayed, but in addition to this, the evaluation results of past measurements and the like may also be displayed. Thereby, the user can easily confirm that the measurement is being continued, and can be motivated to continue the measurement.

In the guide output unit 132F, the terminal device 10 notifies the user of the target pattern information according to the height of the user by video and audio. Thereby, the user can easily draw an image of stepping toward the target pattern, and can easily perform the operation for measurement.

The terminal device 10 displays the detection result by the measuring device 31 on the detection result display section 132G. For example, the terminal device 10 displays information such as that the user's feet are in the prescribed positions for measurement (both feet are placed at the home position) on the detection result display section 132G. As a result, the user can easily recognize that the measurement is being performed correctly according to the procedure, and can perform the measurement with a sense of security that measurement errors will not occur.

The screen example of FIG. 10C shows a situation in which the terminal device 10 measures the user's twisting motion.

In the screen example of FIG. 10C, the terminal device 10 detects that the height of the pole 33 is not appropriate, and instructs the user to extend the pole 33 to a predetermined height using the guide output unit 132F. It is displaying a prompt. This corresponds to the processing of step S903 in FIG.

The screen example of FIG. 10D shows a situation in which the terminal device 10 notifies the user of the completion of the measurement, the evaluation result based on the measurement result, and prompting the user to register as a friend with another user.

In the screen example of FIG. 10(D), the terminal device 10 displays in the guide output unit 132F that the measurement of all of the plurality of measurement items has been completed, and the results of evaluating the user's body movements for the measurement items. The terminal device 10 refers to the user's past measurement history, and displays the result of comparison with the past measurement result (such as a change in joint range of motion from the previous time) on the guide output unit 132F.

The terminal device 10 displays, in the notification unit 132M, a display prompting the other user to register as a friend, and a background of proposing the other user as a friend candidate (measurement was taken at the same place, etc.).

<Summary>
(1) As described above, according to the present embodiment, it is not necessary to apply a physical measuring instrument such as a goniometer to the body of the user when performing the ROM check. You can easily measure yourself. Further, by progressing the measurement according to the measurement items, it is possible to evaluate the range of motion of each joint of the user's lower body and upper body.

(2) When measuring the movement of the user's body, the mat 32 and the poles 33 can be used to make the user recognize the target point of the user's body movement. This stabilizes the form in which the user moves the body, so that the user can easily confirm changes in body movement by accumulating and comparing measurement results. For example, by using the mat 32, it is possible to specify (i) how to stride, such as the stride length and the range of motion when the user performs a stepping motion, and instruct the user to perform the motion specified in the measurement. can be encouraged. In the case of continuously measuring the user's bodily motions, it is possible to easily compare the measurement results over time by specifying the motions that the user performs during the measurement. In other words, by defining the motion of the user's body, it is possible to further facilitate the evaluation of how the motion of the user's body has changed. Further, by using the mat 32 , (ii) the standing position of the user with respect to the measuring device 31 can be defined. The terminal device 10 can determine whether the mat 32 is arranged at a predetermined position with respect to the measuring device 31 based on the measurement result of the measuring device 31 . Therefore, when the motion of the user's body is measured using the mat 32 , the user's standing position is defined by each pattern of the mat 32 . This makes it easier for the user's body motion to fall within the measurement range of the measurement device 31 . For example, even if the user sticks out his or her arm straight up or right to the side, by placing the mat 32 in a predetermined position with respect to the measuring device 31 in advance, the movement of the user's body will not affect the movement of the measuring device 31 . Deviating from the measurement range can be suppressed. Further, by using the mat 32, (iii) when making the user perform a specific action, the user can take an appropriate posture before taking measurements. For example, the user's standing position can be defined by the home position, and when the user performs a predetermined action (squatting, raising one leg, etc.), a posture suitable for measurement (predetermined foot width, etc.) Measurement can be started with

These make it possible to easily evaluate the suppleness of each joint of the user's whole body.

(3) Furthermore, in the first embodiment, the terminal device 10 may suggest an exercise suitable for the user according to the measurement result of the user's body motion (ROM check result, etc.). . For example, assume that the terminal device 10 holds an exercise menu (training item information 185, 285) in association with the classification result of the ROM check evaluation results (measurement history information 183, measurement history database 283). That is, it is assumed that the information of the training menu to be proposed is stored in advance in association with the measurement results and evaluation results of the user's body movements. The terminal device 10 refers to the measurement history database 283, specifies an exercise menu to be proposed to the user based on the measurement results of the user's body movement such as ROM check, and the evaluation results, and displays the specified exercise menu on the display 132 or the like. may be presented to the user by displaying the For example, as a result of the ROM check, the range of motion of each joint of the user can be evaluated. In the training item information 285, the server 20 may predetermine the range of motion of the joints required for the user to perform the exercise. For example, as a result of the ROM check, if the range of motion of the user's hip joint is smaller than a certain level (if the hip joint is stiff), the user is not provided with a predetermined training item as an exercise (the terminal device 10 may not provide the training item based on the training item). processing such as displaying an image cannot be executed). Also, based on the result of the ROM check, the user may be provided with an exercise menu according to the evaluation result of each joint of the user's body.

In addition, the terminal device 10 measures the movement of the user's body while performing the training indicated by the training item, and determines whether the user can move the body as specified in the training. judge. As a result of the user continuing to exercise, the muscle strength of each part of the body for improving the walking posture is gradually improved, and it is expected that the user will be able to perform the movements required for exercise. Accordingly, the terminal device 10 may determine that the user has completed the exercise menu and present another exercise menu to the user.

Alternatively, an exercise menu may be proposed to the user based on the history of exercise menus executed by other users that are similar to the evaluation results of the user's physical motion measurement results.

(4) In the description of the first embodiment above, an example of measuring and evaluating the movement of the user's body by installing the measuring device 31, the mat 32, and the pole 33 in the real world has been described.

Alternatively, an object corresponding to the mat 32 and an object corresponding to the pole 33 may be arranged in the virtual space to measure the movement of the user's body. For example, the user views the display 132 or wears a head mounted display (HMD). The terminal device 10 displays, on the display 132 or HMD, a field-of-view image corresponding to the setting of the virtual camera in the virtual space. By attaching a motion sensor to each part of the user's body (head, shoulder, elbow, hand, waist, knee, foot, etc.), the terminal device 10 tracks the movement of the user's body. The user moves the user's body while visually recognizing each object installed in the virtual space. As a result, the movement of the user's body can be measured and evaluated without installing the mat 32 and the poles 33 in the real world.

In addition to setting the mat 32 and the poles 33 in the real world, an image corresponding to the mat 32 may be projected onto the floor surface or the like, and an image (such as a stereoscopic image) corresponding to the pole 33 may be displayed in the real world. It may be displayed in For example, in the real world, light may be emitted from a laser, a projector, or the like to display the virtual mat 32 and the poles 33 so that the user can visually recognize them.

<Second Embodiment>
A measurement system 1 according to the second embodiment will be described. In the second embodiment, the user's walking motion is measured to evaluate the walking posture, thereby assisting the user in acquiring a walking style that is beautiful and facilitates the exercise of bodily functions. will be explained.

FIG. 11 shows a situation in which the user's walking motion is measured by the measurement system 1 according to the second embodiment.

<1 Overview>
The measuring device 31A measures the movement of the first user User_A walking from a predetermined position to the measuring device 31A in the detection range 38A. On the display 132A, the terminal device 10A notifies the first user User_A of measuring the state of walking and how to move the body during walking. As illustrated, the terminal device 10A notifies the user to walk so as to put the weight on the front leg when walking.

In the example of FIG. 11, coordinate axes are defined as follows. That is, with the position of the first user User_A as a reference, the walking direction is defined as "forward", the direction opposite to the walking direction is defined as "backward", and the axis determined by the walking direction is defined as the "front-rear axis". On the other hand, with the position of the first user User_A as a reference, the direction perpendicular to the walking direction on the surface where the walking foot touches the ground is defined as the "left-right axis", the direction of the user's right half of the body is defined as the "right direction", and the user's The direction of the left half of the body is defined as "left direction". Let the axis in the body direction from the feet to the top of the head of the first user User_A be the “vertical axis”.

On the display 132A, the terminal device 10A notifies the first user User_A to walk toward the measuring device 31A starting from a position a predetermined distance away from the position of the measuring device 31A. Here, the starting point where the first user User_A starts the walking motion is, for example, a position several meters away from the measuring device 31A, and the walking cycle of the first user User_A (the motion of stepping on the right foot and the motion of stepping on the left foot). ) can be repeated multiple times. The measurement device 31A can measure the walking cycle of the first user User_A multiple times. When the user's walking motion is measured by the measuring device 31A, the terminal device 10A notifies the first user User_A to move to the starting point if the first user User_A is not located at the starting point. may be performed by the display 132A or the like. In response to the presence of the first user User_A at the starting point, the terminal device 10A notifies the first user to start measuring the walking motion (or notifies the first user to start the walking motion). may

FIG. 12 shows each part of the user's body identified by the terminal device 10 based on the measurement result of the measuring device 31. As shown in FIG. The terminal device 10 identifies the joints of the extremities, the head, the shoulders, the trunk, the pelvis, the feet, the heels, the toes of the feet, the ribs, the anterior superior iliac spine, etc. do. The terminal device 10 identifies the thoracic spine (the center of the trunk), the center of the pelvis, and the like. For example, the terminal device 10 prompts the user to take a predetermined pose (such as a pose in which both arms are extended sideways to form a T shape, a pose in which both arms are attached to the body, etc.). Based on the measurement result of 31, the position of each joint of the user in the measurement result of the measuring device 31 may be identified.

<2 Data structure>
FIG. 13 is a diagram showing data structures of the measurement result database 287 and the evaluation result database 288 stored in the server 20 according to the second embodiment.

The measurement result database 287 is a database that holds the measurement results of each user's body motion (for example, walking motion, trunk balancing motion, etc.) measured by the measuring device 31 .

Each record of the measurement result database 287 includes the item “measurement data ID”, the item “date and time”, the item “user ID”, the item “elapsed time from measurement start”, and the item “measurement data”.

The item “measurement data ID” indicates information for identifying each piece of measurement data measured by the measuring device 31 . In the second embodiment, the terminal device 10 starts and ends the measurement of the user's bodily motion (for example, the measurement of the walking cycle) to associate information identifying each measurement data with the measurement data. and store it in the server 20.

The item “date and time” indicates the date and time when the measurement device 31 measured the motion of the user's body.

The item “user ID” is information for identifying each user whose body motion is measured by the measuring device 31 .

The item “elapsed time from start of measurement” indicates the elapsed time from the start of measurement of the user's body motion by the measuring device 31 .

The item "measurement data" indicates the position at the timing shown in the item "elapsed time from start of measurement" for each part of the user's body (head, shoulders, . . . ) as described with reference to FIG. For example, in the item "measurement data", in a coordinate system based on the position of the measuring device 31, the measurement result of measuring the distance from the position of the measuring device 31 to each part of the user's body may be stored.

The evaluation result database 288 is a database that holds evaluation results obtained by evaluating the user's bodily motion (for example, walking motion) based on the measurement results of the measuring device 31 (measurement result database 287).

Each record of the evaluation result database 288 includes the item “evaluation result ID”, the item “measurement data ID”, the item “measuring object”, and the item “evaluation result”.

The item “evaluation result ID” is information for identifying each evaluation result obtained by evaluating the measurement result of the user's body motion by the measuring device 31 .

The item "measurement data ID" is information for identifying each measurement result (measurement result database 287) of the measurement device 31 to be evaluated.

The item "measuring object" indicates the content to be measured as the user's body motion. Objects to be measured include, for example, the posture of the user during walking, the balance of the user's trunk, and the like.

The item “evaluation result” indicates the evaluation result of evaluating the user's body movement based on the measurement result of the measuring device 31 . For example, when evaluating the user's posture during walking, the objects to be evaluated are "the user's stepping forward", "whether the user's weight can be transferred to the stepped front leg", and "whether the user's weight can be transferred to the front leg". Are you ready?” Details will be described later. Also, when evaluating the balance of the user's trunk, the subject of evaluation includes "whether the positions of the shoulders, hips, and feet are out of alignment". Details will be described later.

<3 Operation>
Hereinafter, a process in which the terminal device 10 measures the movement of the user's body and evaluates based on the measurement results in the second embodiment will be described.

FIG. 14 is a flow chart showing processing for measuring and evaluating the user's walking motion by the terminal device 10 in the second embodiment.

In step S<b>1403 , the terminal device 10 detects that the user is at a predetermined position within the measurement range of the measurement device 31 based on the measurement result of the measurement device 31 .

In step S1405, the terminal device 10 outputs guidance (audio, video) to the user to perform a walking motion. The terminal device 10 detects each joint of the user based on the measurement result of the measuring device 31, and records the position of each joint during the user's walking motion.

In step S<b>1407 , the terminal device 10 classifies the user's walking cycle into a plurality of phases based on the measurement result of the measurement device 31 . The terminal device 10 determines an evaluation value obtained by evaluating the walking motion according to the evaluation items for each aspect of the user's walking motion.

In step S1409, the terminal device 10 outputs to the display 132 or the like the measurement result of the measuring device 31 and the evaluation result of evaluating the walking motion together with the ideal walking posture model. The terminal device 10 transmits measurement results and evaluation results to the server 20 .

In step S1453, the server 20 updates the measurement result database 287 and the evaluation result database 288 based on the measurement result and evaluation result of the user's walking motion.

<4 Output example of measurement results and evaluation results>
FIG. 15 is a diagram showing an output example when the terminal device 10 outputs the measurement result of measuring the user's walking motion and the evaluation result of evaluating the user's walking motion based on the measurement result in the second embodiment. is. In the illustrated example, the terminal device 10 displays these measurement results and evaluation results on the display 132. However, the terminal device 10 prints the measurement results and evaluation results on a paper medium or the like using a printer or the like. It is also possible to output the result.

FIG. 15A shows a situation in which the terminal device 10 prompts the measurement device 31 to walk a certain number of steps (for example, about several meters) in order to measure the user's walking motion. This corresponds to the processing of steps S1403 and S1405 in FIG.

In the display 132, the display unit 132P1 of the terminal device 10 displays to the user a "notification prompting the user to walk so as to put his or her weight on the front leg to be stepped" when performing a walking motion. The terminal device 10 displays, on the display units 132P2 and 132P3, an image showing the specified position of each part of the user's body (each part of the body illustrated in FIG. 12) as the measurement result of the measuring device 31. FIG. The terminal device 10 connects each part of the user's body in a linear fashion, thereby displaying the measurement result of the user's body so that the user can easily visually recognize it. The example of FIG. 15 shows an example in which the parts of the right half of the user's body are connected by dotted lines, and the parts of the left half of the body are connected by solid lines. Thereby, the user can easily recognize whether the measurement result of the measuring device 31 is the movement of the right half of the user's body or the movement of the left half of the user's body.

The terminal device 10 uses the display unit 132P2 to display, at any time, the result of specifying the position of each part of the user when viewed from the front of the user (the front-rear axis shown in FIG. 11) based on the measurement result of the measurement device 31. .

The terminal device 10 uses the display unit 132P3 to display the result of specifying the position of each part of the user when viewed from the side of the user (left-right axis shown in FIG. 11) at any time based on the measurement result of the measurement device 31. . That is, the terminal device 10 displays the measurement results of the measuring device 31 on the display units 132P2 and 132P3 in real time. This allows the user to obtain real-time feedback of the measurement result of the user's own body motion.

The terminal device 10 notifies the user to perform a walking motion to approach the measuring device 31 . Here, the terminal device 10 may display the measurement results on the display units 132P2 and 132P3 as if the movement of the user's body was measured at a certain distance from the user's body. For example, the terminal device 10 generates a three-dimensional virtual section, and based on the measurement result of the measuring device 31 (that is, the distance from the position of the measuring device 31 to the position of each part of the first user User_A), the three-dimensional An object corresponding to the position of each part of the user's body is placed in the virtual space. For example, the terminal device 10 identifies the coordinates corresponding to the measuring device 31 in the three-dimensional virtual space, and determines the position of each part based on the measurement result of the measuring device 31 with respect to the coordinates corresponding to the measuring device 31. An object corresponding to each part is placed at the corresponding coordinates. The terminal device 10 holds information on the position of the object corresponding to each part in the three-dimensional virtual space in association with the elapsed time that accompanies the measurement by the measuring device 31 . That is, by reproducing the trajectory of the movement of the object corresponding to the position of each part of the user in the three-dimensional virtual space, the walking motion of the user can be reproduced as the trajectory of the position of each part. The terminal device 10 coordinates the object group corresponding to the position of each part of the user with a certain positional relationship (for example, a position at a certain distance on the front-rear axis from the "shoulder center", or a chest position). A virtual camera may be set at a position at a certain distance on the left-right axis from the waist, and the virtual camera may be moved in conjunction with the movement of the object corresponding to the position of each part.

The terminal device 10 notifies the information for appropriately performing the measurement by the measuring device 31 on the display section 132P4. This corresponds to step S1403 in FIG. For example, when the user's walking motion is measured by the measuring device 31, the terminal device 10 determines whether or not the user starts walking at a predetermined position, and prompts the user to start walking from the predetermined position. give notice. As illustrated, when the position where the user starts walking is too close to the measuring device 31, the terminal device 10 notifies the user to start walking further from behind.

FIG. 15(B) shows a phase in which the terminal device 10 outputs measurement results and evaluation results for a phase among multiple phases of the user's walking motion. This corresponds to the processing of step S1409 in FIG.

In FIGS. 15B and 15C, the measurement results of the user's body are displayed with the vertical axis as the reference. Accordingly, it is possible to present to the user how the user's body is rotated with respect to the vertical axis.

The terminal device 10 outputs, on the display 132, evaluation results of shoulder rotation balance and waist rotation balance during walking as evaluation results of the beauty of the user's walking posture. The terminal device 10 divides the walking motion into a plurality of phases, and outputs evaluation results such as shoulder rotation balance and hip rotation balance in each phase. In this way, by presenting the user with the measurement results and evaluation results for a plurality of aspects of the walking motion, suggestions for making the user's walking motion beautiful can be provided.

The terminal device 10 displays the rotation inclination of the shoulder in each of the left rotation and the right rotation in the walking cycle (one cycle is the stepping of the right foot and then the left foot) on the display unit 132P5.

The terminal device 10 becomes the center of the "line connecting both shoulders" determined by the positions of the right and left shoulders, the "line connecting both hip joints" determined by the positions of both hip joints, and the positions of the right and left shoulders on the display unit 132P6. The “center position of the shoulders” and the “center position of the waist” determined by the positions of the hip joints (for example, the positions of the right and left pelvises) are displayed. The terminal device 10 calculates the degree of left shoulder rotation ( How far the shoulders rotate) is evaluated. For example, the terminal device 10 draws a “line connecting both shoulders” based on the amount of rotation of the shoulders by left rotation in one cycle with the position of the waist as a reference. The terminal device 10 displays the score (score “4.8 points”) as the evaluation result on the display unit 132P6.

The terminal device 10 displays the “line connecting both shoulders”, “line connecting both hip joints”, “center position of shoulder”, and “center position of waist” on the display unit 132P7. Based on the position of the right shoulder (or both shoulders) when the user steps out the right foot and the positions of the hip joints, the terminal device 10 calculates the degree of left rotation of the shoulders with respect to the vertical axis in one cycle of the walking motion. Evaluate. The terminal device 10 draws a “line connecting both shoulders” based on the amount of rotation of the shoulders by right rotation in one cycle with the position of the waist as a reference. The terminal device 10 displays the score (score “4.5 points”) as the evaluation result on the display unit 132P7, which is lower than the score for the left rotation.

The terminal device 10 displays, on the display units 132P6 and 132P7, the "center position of the shoulder" and the "center position of the waist" in the motion of stepping the left foot and the motion of stepping the right foot in one cycle of the walking motion. This allows the user to recognize how the upper body is tilted with respect to the position of the waist during walking. For example, when the user walks, the user can be made aware of the extent to which the upper body moves along the left-right axis and the extent to which the upper body moves along the front-back axis. In the illustrated example, in the display section 132P6, the degree of overlap between the "center position of the shoulder" and the "center position of the waist" during left rotation is large, and the upper body is not shaken, whereas in the display section 132P7, During clockwise rotation, the overlap between the "shoulder center position" and the "waist center position" is relatively small, and the upper body shakes more laterally than during left rotation. Therefore, the score, which is the evaluation result, is lower for right rotation than for left rotation.

FIG. 15(C) shows, as a phase of the user's walking motion, the terminal device 10 outputting the measurement result and the evaluation result when the forefoot is on the ground during the walking motion. This corresponds to step S1409 in FIG.

The terminal device 10 displays, on the display unit 132P8, the measurement result and the evaluation result of the inclination of rotation of the waist when the front foot of the user touches the ground during walking.

The terminal device 10 displays, on the display unit 132P9, "a line connecting both hip joints" or the like when the left foot is placed on the ground as the front foot. As shown, the "line connecting both hip joints" is slanted, indicating that the user is able to rotate the hips when the left foot is on the ground.

The terminal device 10 displays, on the display unit 132P10, "a line connecting both hip joints" when the right foot is on the ground as the front foot. As shown, the "line connecting both hip joints" is slanted, indicating that the user can rotate the hips when the right foot is on the ground.

FIG. 16 is a diagram showing an output example when the terminal device 10 outputs the measurement result of measuring the user's walking motion and the evaluation result of evaluating the user's walking motion based on the measurement result in the second embodiment. is. This corresponds to the processing of steps S1407 and S1409 shown in FIG.

In the example of FIG. 16 , the terminal device 10 displays the measurement results and evaluation results of one cycle of the user's walking motion in a manner that can be compared with a reference example (model case, model) for making the walking posture look beautiful. The output example shown in FIG. 16 may also be output by the terminal device 10 by printing on a paper medium using a printer or the like.

On the display 132, the terminal device 10 divides one cycle of the walking motion into a plurality of steps, and for each step, displays a model for making the walking posture look beautiful, the measurement result of the user's walking motion, and the evaluation result for the measurement result. to display.

In the example of FIG. 16, one cycle of the walking motion consists of the first step "stepping forward", the second step "trying to put the weight on the stepped front foot (trying to get on the front foot)", and the second step It is divided into 3 steps, "Put your weight on the stepped front leg (get over the front leg)".

Here, in the first step, the second step, and the third step, the items evaluated by the terminal device 10 include items common to each step and items not common to each step.
(1) Evaluation items for the first step • Inclination of the upper body during walking: Whether the parts of the upper body (head, shoulders, etc.) are inclined forward or backward with respect to the position of the waist. For example, is the angle at which the line determined by the position of the pelvis, the position of the chest and the waist, and the position of the center of the shoulder tilt with respect to the vertical axis within a certain range?
・ Fore-and-aft spread of the feet when trying to step forward: The degree of the fore-and-aft spread of both feet. For example, is the distance between the ankle of the front leg and the ankle of the back leg that has stepped forward a certain distance or more?
・ Hip position when trying to step forward: Is the hip left behind when trying to step forward? For example, is the position of the pelvis forward or backward with respect to the shoulders, chest and waist?
(2) Evaluation items for the 2nd step ・Inclination of the upper body at the timing when the front foot is on the ground: Whether the parts of the upper body (head, shoulders, etc.) are tilted forward or backward with respect to the position of the waist ・Bending of the knee when the front foot touches the ground: Whether each part of the front foot (ankle, knee, pelvis) is bent or bent greatly. For example, is the angle formed by the line connecting the ankle and knee and the line connecting the knee and pelvis within a certain angle? Inclination of the upper body at the timing of putting the weight on: Whether the parts of the upper body (head, shoulders, etc.) are tilted forward or backward with respect to the position of the waist. With the front foot that landed on the ground as the pivot foot, each part of the pivot foot (ankle, knee, pelvis) is stretched or bent when putting weight on the pivot foot. For example, does the angle formed by the line connecting the ankle and the knee and the line connecting the knee and the pelvis fall within a certain angle? Display the exemplar in the first step. The terminal device 10 displays the measurement result of the user's body in the first step on the display unit 132P21. The terminal device 10 displays the evaluation result of the user's body movement in the first step on the display unit 132P31.

The display units 132P11, 132P12, and 132P13 of the terminal device 10 display the position of each part (joint) of the body when each of these evaluation items is good for a beautiful walking posture. For example, in the display unit 132P11, when the upper body is not tilted, the front legs and the back legs are spread apart to a certain extent (the stride is not too short), and the hip position is appropriate (not leaning back). The position of each part of the body is indicated.

In the illustrated example, the display unit 132P31 displays, as the evaluation results, the evaluation result RF_ic for the right foot when the right foot is stepped and the evaluation result LF_ic for the left foot when the left foot is stepped for the evaluation item "inclination of upper body". It is displayed superimposed on a predetermined bar. The predetermined bar is drawn according to the range that can be taken as the evaluation result. In the predetermined bar, the terminal device 10 distinguishes between areas with good evaluation results and areas with poor evaluation results, and in the illustrated example, the areas with good evaluation results are highlighted. is doing. The terminal device 10 sets the evaluation result RF_ic for the right foot and the evaluation result LF_ic for the left foot in the range in which the evaluation result is neither excessively tilted forward nor excessively backward in the evaluation item “upper body tilt”. are displayed in association with each other.

In the display unit 132P31, the terminal device 10 associates the evaluation result of the right leg and the evaluation result of the left leg with a predetermined bar for the evaluation item "front and back spread of the legs" and the evaluation item "waist position" as the evaluation results. are displayed.

The terminal device 10 displays the model in the second step on the display unit 132P12. The terminal device 10 displays the measurement result of the user's body in the second step on the display unit 132P22. The terminal device 10 displays the evaluation result of the user's body movement in the second step on the display unit 132P32.

The terminal device 10 displays the model in the third step on the display unit 132P13. The terminal device 10 displays the measurement result of the user's body in the third step on the display unit 132P23. The terminal device 10 displays the evaluation result of the user's body movement in the third step on the display unit 132P33.

FIG. 17 is a diagram showing an example in which the terminal device 10 outputs an evaluation result of the load on the knees in the user's walking motion. This corresponds to step S1409 in FIG.

FIG. 17A shows an example of outputting measurement results and evaluation results of hip joint adduction and abduction in a user's walking motion. The terminal device 10 displays the measurement results of the user's walking motion on the display 132 based on the longitudinal axis (assuming that the user is viewed from the front), so that the user can easily check the bending of the knees during the walking motion. make it

The terminal device 10 displays the measurement results during the walking motion on the display unit 132P41. The terminal device 10 displays the measurement result when walking with the right foot as the pivot on the display unit 132P42. The terminal device 10 displays the measurement result when walking with the left foot as the pivot on the display unit 132P43.

On the display unit 132P44, the terminal device 10 displays a favorable measurement result pattern (“both the pivot leg and the free leg are straight”) and a measurement result pattern that may hurt the knee (“knee "Possibility of hurting the outside" and "Possibility of hurting the inside of the knee") are displayed. In other words, even if the hip joint is excessively internally rotated or externally rotated during walking, the knee may be injured during the walking.

As described above, by referring to the output result of the terminal device 10, the user can be encouraged to walk without hurting the knee.

FIG. 17B is a diagram showing an example of outputting an evaluation result as to whether the knee is too far ahead of the ankle in the user's walking motion. The terminal device 10 displays the measurement result of the user's walking motion on the display 132 based on the left-right axis (assuming that the user is viewed from the side), and checks whether the knee moves too far ahead during the walking motion. can be easily checked by the user.

The terminal device 10 displays the measurement results during the walking motion on the display unit 132P51. The terminal device 10 displays the measurement result when walking with the right foot as the pivot on the display unit 132P52. The terminal device 10 displays the measurement result when walking with the right foot as the pivot on the display unit 132P53. The terminal device 10 displays on the display unit 132P51 the result of the evaluation of the user's walking posture, indicating that "the load on the front of the knee is small".

On the display unit 132P54, the terminal device 10 displays a favorable measurement result pattern (“good”) and a measurement result pattern that may hurt the knee (“the knee is too far forward”) for the user’s reference. ) and about. That is, when the user attempts to step forward with the front foot, if the knee is too far ahead of the ankle, the knee may be injured. The terminal device 10 determines whether or not the angle at which the line connecting the ankle position and the knee position and the line connecting the knee position and the pelvis position intersect is equal to or greater than a predetermined angle (knee to ankle forward). out) to evaluate whether the knee is too far ahead when the user attempts to step forward.

As described above, the walking motion is evaluated for a plurality of phases of the walking cycle of the user, such as trying to step forward with the pivot leg, landing the pivot leg, and trying to put the weight on the pivot leg. Thereby, the user is prompted to make the walking motion more beautiful by referring to the evaluation result of each aspect. In addition, by allowing the user to refer to the measurement result of the user's body movement by comparing the model body movement with the user's body movement, the user is urged to approach a beautiful walking posture.

<Third Embodiment>
A measurement system 1 according to the third embodiment will be described. In the third embodiment, processing for displaying the measurement result of the user's trunk balance and enabling the user to obtain a state in which the trunk is in balance will be described.

<Action>
Hereinafter, in the third embodiment, a process in which the terminal device 10 measures the movement of the user's body and evaluates based on the measurement results will be described.

FIG. 18 is a flow chart showing processing for measuring and evaluating the user's trunk balance by the terminal device 10 in the third embodiment.

In step S<b>1803 , the terminal device 10 outputs guidance (audio, video) to measure the user's trunk balance (shoulders, hips, legs) based on the measurement result of the measurement device 31 . The terminal device 10 displays the measurement results by displaying the positions of the user's shoulders, hips, and feet as circles based on the vertical axis. The terminal device 10 outputs a guide prompting the user to balance the trunk according to how the shoulders, hips, and feet of the user overlap.

In step S1805, the terminal device 10 evaluates the trunk balance of the user based on the measurement result of the measurement device 31 and outputs an evaluation value. The terminal device 10 transmits the measurement result of the measurement device 31 and the evaluation result to the server 20 .

In step S1853, the server 20 updates the measurement result database 287 and the evaluation result database 288 based on the user's trunk balance measurement result and evaluation result.

<Example of output>
FIG. 19 is a diagram showing an example in which the terminal device 10 outputs a measurement result of measuring the trunk balance of the user and an evaluation result of the trunk balance in the third embodiment.

FIG. 19A shows a screen example of a phase in which the terminal device 10 outputs measurement results in real time in order to measure the user's trunk balance using the measurement device 31 . This corresponds to the processing of step S1803 in FIG. In the example of FIG. 19A, the display unit 132P61 displays circular graphics (perfect circle, ellipse, etc.) corresponding to the shoulders, waist, and feet of the user's body.

The terminal device 10 displays the measurement results of each part of the user's body on the display unit 132P62 based on the front-rear axis.

The terminal device 10 displays the positions of the shoulders, waist, and feet of the user's body on the display unit 132P63 based on the vertical axis based on the measurement result of the measurement device 31 . The terminal device 10 displays the measurement result of the measuring device 31 on the display section 132P63 at any time. Therefore, when the user's body shakes, the circular figures corresponding to the shoulders, hips, and legs do not overlap, causing deviation. The screen example of FIG. 19A shows that each part of the user's body is not arranged along the vertical axis and is shifted. The user is prompted to move the body so as to balance the trunk while watching the display on the display section 132P63 of the display 132 .

The terminal device 10 notifies the user on the display unit 132P64 to prompt the user to balance the trunk. Based on the measurement result of the measuring device 31, the terminal device 10 identifies the movement of the user's body required to balance each part of the user's body. In the illustrated example, the position of the head and feet is on the back side with respect to the position of the waist of the user. In other words, the waist of the user is in a state of protruding forward. The terminal device 10 displays on the display unit 132P64 a display prompting the user to adjust the inclination of the body in the front-rear direction. This allows the user to easily recognize the movement of the body required to bring the trunk into a balanced state.

FIG. 19B shows a display example of measurement results when the trunk of the user is well-balanced. In the display section 132P63, the user's shoulders, waist, and feet are aligned along the vertical axis, and the trunk is well balanced. The terminal device 10 notifies the user that the trunk is in a balanced state on the display unit 132P64.

FIG. 19(C) shows a display example when the trunk balance measurement is finished and the evaluation result is output. This corresponds to the processing of step S1805 in FIG.

The terminal device 10 displays on the display unit 132P65 that the trunk balance measurement has been completed, and outputs an evaluation result based on the measurement result.

The terminal device 10 determines a training menu for balancing the trunk as a training menu to be proposed to the user based on the evaluation result of the trunk balance. In the illustrated example, the terminal device 10 displays, on the display unit 132P65, an evaluation result of the user's trunk balance and training menu candidates to be proposed to the user according to the evaluation result. The terminal device 10 receives an operation for starting the training menu proposed to the user on the display unit 132P66. The terminal device 10 reads out the corresponding training menu in response to the user's input operation to the display unit 132P66.

The terminal device 10 receives designation of a training menu from the user on the display unit 132P67. In other words, the display unit 132P67 accepts an input operation for starting other training regardless of the training menu that the terminal device 10 proposes to the user based on the trunk balance evaluation result.

The items described in each of the above embodiments may be combined.

<Fourth Embodiment>
In the description of the fourth embodiment, the measurement results of the user's body movements are ROM check (first embodiment), walking posture check (second embodiment), trunk balance measurement. A technique for proposing a training menu suitable for the user according to (third embodiment) and the like will be described. By measuring and evaluating the movement of the user's body as described above, the terminal device 10 can identify which part of the user's body should be trained. , suggest a suitable training menu.

With this configuration, when new training menu content is added, the content of the training menu can be customized according to which part of the body the training menu is suitable for, or whether the content is suitable for the user's body. By associating whether or not, it is possible to provide the user with content corresponding to the measurement result of the user's body movement, making it easier to add content.

(1) As described above, the terminal device 10 may propose an exercise menu to the user based on the measurement results and evaluation results in each step of the user's walking cycle. For example, the terminal device 10 compares a model body movement with the user's body movement (example in FIG. 16), and identifies muscle parts that the user should improve. For example, the server 20 selects an exercise menu that is expected to allow the user to take a large step forward when the distance between the front and back of the foot is small in the evaluation result of the first step (movement of stepping forward) as the walking motion of the user. It is specified by referring to the training item information 285 . In other words, the server 20 may associate and hold a proposed exercise menu in advance according to the evaluation results of the evaluation items of each step of the walking motion and the evaluation items of the balance of the trunk.

(2) The terminal device 10 may propose a training menu to the user so that the left-right difference in the user's body motion can be eliminated. As described above, the terminal device 10 evaluates the left-right balance of the user's body motion by performing the ROM check and the walking posture check. The terminal device 10, for example, evaluates the left-right balance of the rotational motion with respect to the vertical axis based on the evaluation result of the shoulder rotation and waist rotation during walking motion of the user. The terminal device 10 identifies the muscle part that causes the user's body motion to be left-right symmetrical. For example, the terminal device 10 holds in advance information on muscle parts that are factors that cause left-right differences in the amount of shoulder rotation. As described above, the terminal device 10 preliminarily associates and retains the evaluation target during walking and the information on the muscle part that causes the left-right difference, thereby eliminating the left-right balance difference. You can identify the parts of the muscles that you should train for.

(3) The terminal device 10 predicts possible future states of the user's body based on the evaluation results of the user's body movements, and proposes a training menu to the user based on the prediction results. good too.

For example, the terminal device 10 identifies (i) possible physical concerns of the user, such as “neck/shoulder stiffness”, “sagging upper arms”, “rolled shoulders, forward head posture”, “lower back pain”, “ Each item such as "leg swelling" and "arm stiffness" is associated with (ii) information on the body part of the user and stored as predetermined information. In addition, the terminal device 10 stores information on a training menu to be proposed as training in association with each item of the user's possible physical concerns.

The terminal device 10 performs a ROM check and a walking motion check to specify a muscle part that is desired to be improved based on these measurement results and evaluation results. For example, in the ROM check, the terminal device 10 stores in advance which muscle part is associated when the twist of the body is not good. For example, in the ROM check, the terminal device 10 holds muscle information in association with evaluation items for the range of motion of shoulders, hips, and the like.

The terminal device 10 identifies possible future troubles by referring to the predetermined information, such as by identifying muscle parts that move hard. The terminal device 10 determines a training menu to be proposed to the user according to the result of the identification so as to cope with possible troubles that may occur in the user's body in the future.

As described above, the information of each part of the muscle is associated with the evaluation items of the balance during walking motion and the torsion of the body, and the information of each part of the muscle and each item of trouble that may occur in the user's body are associated. associated with the information of Thereby, the terminal device 10 can propose a training menu customized for each user to the user based on the evaluation result of the user's body motion.

<Data structure in the fourth embodiment>
FIG. 20 is a diagram showing the data structure of various data stored by the server 20 according to the fourth embodiment.

In the measurement item information 282, the server 20 holds the item "probable future trouble" and the item "related muscle part" for each measurement item.

The item "probable future troubles" is information that identifies possible troubles that may occur in the user's body in the future, depending on the results of evaluating the measurement results for each measurement item (for example, tags such as "stiff shoulders" and "stiff neck"). ).

The item "related muscle part" is information on a body part (muscle part) that is desired to be trained for each measurement item, depending on the evaluation results of the measurement results. In the item "relevant muscle part", the information of the body part may be stored step by step according to the degree of the evaluation result.

In the training item information 285, the server 20 holds the item "content", the item "corresponding trouble", and the item "related muscle part" for each training item.

The item "content" indicates content (video, motion data that serves as a reference for the user's body movement, audio data, etc.) presented to the user when training is performed.

The item "corresponding trouble" is information (for example, tags such as "stiff shoulder" and "stiff neck") specifying a trouble that may occur in the user's body in the future depending on the evaluation result for each training item.

The item "related muscle part" is information on a body part (muscle part) that is desired to be trained for each training item depending on the evaluation result. In the item "relevant muscle part", the information of the body part may be stored step by step according to the degree of the evaluation result.

The server 20 retains the item “possible future troubles” in the evaluation result database 288 . The item "probable future troubles" is information (for example, tags such as "stiff shoulder" and "stiff neck") that specify troubles that may occur to the user's body in the future depending on the evaluation results for each evaluation result.

As described above, the server 20 updates the evaluation result database 288 according to the evaluation result of the user's body movement. For example, according to the measurement results of the ROM check (measurement item information 282), it is possible to specify information about possible future troubles in the user's body. The server 20 refers to the evaluation result database 288 to identify information about possible future worries of the user, and refers to the training item information 285 to extract training items to be proposed to the user. When extracting a plurality of training items to be proposed to the user, the server 20 sets priorities based on a predetermined rule. For example, the server 20 may give priority to a training item capable of resolving the concerns associated with the unsatisfactory evaluation result of the user's body movement, depending on the extent to which the evaluation result is unsatisfactory.

<Screen example in the fourth embodiment>
FIG. 21 is a screen example of a phase in which the terminal device 10 presents possible troubles to the user's body based on the measurement results and evaluation results of the user's body movements, and proposes an exercise menu corresponding to the troubles. be.

FIG. 21(A) displays the result of the terminal device 10 identifying possible troubles of the user's body based on the measurement result and the evaluation result of the user's body motion.

The terminal device 10 prompts the user to take daily measurements via the terminal device 10 on the display unit 132P71, and displays possible future conditions of the user's body on the display unit 132P72. In the illustrated example, the operation reception unit 132P73 displays that the user may suffer from "stiff neck/shoulder", and responds to the user's input operation to the operation reception unit 132P73. Display the training menu corresponding to In addition, the terminal device 10 displays in the operation reception unit 132P74 that the user may be troubled by “sagging of the upper arm”, and responds to the user's input operation to the operation reception unit 132P74, and displays “sagging of the upper arm”. display a training menu corresponding to the troubles of In addition, the terminal device 10 responds to the user's input operation to the operation reception unit 132P75 while displaying that the trouble of "rolling shoulders inward, posture with the head forward" may occur in the operation reception unit 132P75. Then, a training menu corresponding to the problem of "inwardly rolled shoulders, forward head posture" is displayed.

On the display unit 132P76 of the terminal device 10, for users who are short on time, among the training menu candidates to be proposed to the user, the number of training menu candidates for starting a training menu that is smaller than the number displayed on the display unit 132P76. Display the operation reception section 132P77. Based on a predetermined rule, the terminal device 10 sets priorities, etc., to training menu candidates to be proposed to the user, and displays them on the display unit 132P76.

The terminal device 10 receives an operation for editing a training menu to be executed, such as "favorite", from the user on the display unit 132P78.

The terminal device 10 presents a questionnaire to the user by accepting an input operation to the operation accepting unit 132P91, and accepts an answer to the questionnaire. The terminal device 10 may identify the user's worries based on the input result of the questionnaire, and propose a training menu corresponding to the identified worries.

FIG. 21B shows a phase of proposing a recommended training menu to the user. The terminal device 10 displays to the user, on the display unit 132P81, information about body parts (“Do you suffer from stiff neck or stiff shoulders?”, etc.) regarding the troubles identified based on the measurement results and evaluation results. . The terminal device 10 displays, on the display unit 132P81, the cause of the user's concern about the body (such as "the position of the shoulder tends to protrude forward") based on the measurement result and the evaluation result. On the display unit 132P81, the terminal device 10 displays a guideline for resolving the user's physical troubles ("It is expected to improve the blood flow in the upper body," "Stretch the back of the thigh, ” etc.) and suggesting an exercise corresponding to the guideline.

For example, in response to an input operation to the operation reception unit 132P73, the terminal device 10 displays the operation reception units 132P83 to 132P87 for starting training using the recommended training menu, as shown in the display unit 132P82. The terminal device 10 may reproduce a moving image for starting training in response to the input operation to these operation reception units 132P83 to 132P87.

In response to the input operation to the operation reception unit 132P77, the terminal device 10, as shown in the display unit 132P88, narrows down the number of training menus to be proposed to the user who is short on time, and selects the operation reception unit. 132P83 to 132P85 are displayed.

Thereby, it is possible to encourage the user to continue training according to the user's condition at the time.

<Operation example>
The operation of each device in the above fourth embodiment will be described.

FIG. 22 shows the processing flow of each device in the fourth embodiment.

In step S2203, the terminal device 10 evaluates at least one of the user's joint range of motion, walking posture, and trunk balance based on the measurement result of the user's body movement, and outputs the evaluation result. The terminal device 10 transmits measurement results and evaluation results to the server 20 .

In step S2253, the server 20 updates the database (evaluation result database 288, etc.) based on various user measurement results such as measurement results and evaluation results received from the terminal device 10. FIG.

In step S2255, the server 20 identifies possible future physical troubles of the user based on the evaluation result database 288 and the like. Based on the training item information 285, the server 20 selects a training menu related to the body part corresponding to the identified trouble from among a plurality of training menus, and responds to the terminal device 10 of the user with the selected training menu.

In step S2205, the terminal device 10 displays to the user a training menu corresponding to possible future physical troubles.

<Modification>
In the fourth embodiment, a question may be presented to the user, and the priority of the training menu to be proposed to the user may be set according to the answer (questionnaire result) received from the user.

This not only proposes a training menu based on the measurement results of the user's body, but also presents questions about the user's body and proposes a training menu based on the results of receiving answers to the questions from the user. be able to. That is, the server 20 updates the priority of the training menu proposed to the user based on the results of the answers to the questions. The server 20 presents the user with a training menu having a high priority for the user, based not only on the physical evaluation results but also on the results of the user's answers to the questions. Thereby, the server 20 can propose a training menu more suitable for the user's physical condition.

For example, the terminal device 10 accepts a questionnaire about the user's body, thereby determining whether the muscles of the user's body that have not yet been measured by the measurement unit 30 in the measurement system 1 (evaluation based on the measurement results has not been performed). can receive information on the part of the from the user. The terminal device 10 can also identify possible future concerns regarding muscle parts that have not yet been evaluated by the measurement system 1, based on the results of the questionnaire input by the user. The terminal device 10 proposes to the user how to appropriately deal with future concerns that may arise in the user's body, based on the concerns identified based on the measurement results of body movements and the results of questionnaire responses from the user. Decide on a training menu.

<Data structure in modified example>
FIG. 23 is a diagram showing the data structure of various data stored by the server 20 in the modification of the fourth embodiment.

In the user questionnaire information 289, the server 20 divides each questionnaire result into an item "answer option", an item "current trouble", an item "possible future trouble", and an item "related muscle part". Hold.

The item “answer options” includes information on options for the user to answer questions (questionnaire) presented to the user of the terminal device 10 . For example, if the content of the question is that the user is feeling unwell, the user is presented with options for possible body parts. In this case, it is possible to identify the user's concern about the body part according to the result of the user's designation for each option.

The item "current troubles" is information specifying the physical troubles that the user is likely to have (for example, "back pain", "knee pain”).

The item "probable future troubles" is information that identifies possible troubles that may occur to the user's body in the future (for example, "stiff shoulder", "stiff neck", etc.) depending on the evaluation results of evaluating the contents of the questionnaire responses for each questionnaire result. tag).

The item "related muscle part" is information on a body part (muscle part) that is desirable to be trained for each questionnaire result, depending on the evaluation result of evaluating the contents of the questionnaire answers. In the item "related muscle part", the information of the body part may be stored step by step according to the degree of the content of the answer.

The server 20 holds the item “current worries” in the evaluation result database 288 . The item "current trouble" is information specifying the physical trouble that the user is likely to have (for example, "back pain", "knee pain", etc.) depending on the evaluation result of evaluating the contents of the questionnaire responses for each of the evaluation results. pain”).

<Operation in modified example>
As described above, the server 20 updates the evaluation result database 288 according to the user's body movement evaluation results and the user questionnaire information 289 . For example, according to the measurement results of the ROM check (measurement item information 282), it is possible to specify information about possible future troubles in the user's body. For example, according to the result of a questionnaire about the body from the user (user questionnaire information 289), it is possible to identify the physical troubles that the user currently has or will have in the near future. The server 20 refers to the evaluation result database 290 to identify information about the user's possible future worries, and refers to the training item information 285 to extract training items to be proposed to the user.

When extracting a plurality of training items to be proposed to the user, the server 20 sets a priority based on a predetermined rule according to the extent to which the evaluation result of the user's body movement is not good. For example, the server 20 may give priority to a training item capable of resolving the concerns associated with the unsatisfactory evaluation result of the user's body movement, depending on the extent to which the evaluation result is unsatisfactory. The server 20 corrects and sets the priority of training items to be proposed to the user based on the results of the questionnaire. For example, the server 20 selects training items that can solve the user's physical troubles identified from the questionnaire results, depending on the extent to which the physical evaluation results related to the contents of the questionnaire answers are less favorable than the evaluation results of the ROM check. may be given priority.

FIG. 24 is a diagram showing the flow of operations in the modification of the fourth embodiment.

As shown in FIG. 24, the terminal device 10 presents a question (questionnaire) to the user of the terminal device 10 at any timing such as step S2202 or step S2204.

The processing of step S2202 will be described. The terminal device 10 displays the question on the display 132, for example, in response to the user completing the process of logging into the measurement system 1 in step S700. The terminal device 10 receives an input of an answer to the question from the user. The terminal device 10 transmits to the server 20 the response result for which the input was accepted.

In step S<b>2252 , the server 20 updates the user questionnaire information 289 based on the information received from the terminal device 10 .

In step S2253, the server 20 stores various databases such as the measurement result database 287 in response to receiving from the terminal device 10 the measurement result by the measurement unit 30 and the like in step S2203 and the evaluation result based on the measurement result. Update.

The processing of step S2204 will be described. For example, the terminal device 10 presents a question to the user (displays the question on the display 132, etc.) in response to completing the measurement of the user's body movement in step S2203. The terminal device 10 transmits to the server 20 the response result for which the input was accepted.

In step S<b>2254 , the server 20 updates the user questionnaire information 289 based on the information received from the terminal device 10 .

In step S2256, the server 20 identifies the user's current troubles and possible future physical troubles of the user based on the user questionnaire information 289, the evaluation result database 288, and the like. Based on the training item information 285, the server 20 selects a training menu corresponding to the identified trouble from among a plurality of training menus, and responds to the user's terminal device with the selected training menu. For example, the user's worries are specified based on the results of the user's answers to the question sentences (user questionnaire information 289), and the training menu to be proposed to the user is specified according to the specified worries (training item information 285). . Moreover, the server 20 identifies a training menu to be proposed to the user based on the evaluation result database 288 as described above. The server 20 responds to the terminal device 10 regarding one or more training menus identified in this manner. At this time, the server 20 may set priorities for the specified training menus based on a predetermined rule, and respond to the terminal device 10 with the training menus according to the priorities.

<Screen example 1 in the fourth embodiment>
FIG. 25 shows a situation in which the terminal device 10 presents possible troubles of the user's body based on the measurement results and evaluation results of the user's body movements, and proposes an exercise menu corresponding to the troubles. It is an example of a screen of a stage of answering a questionnaire. For example, the terminal device 10 accesses the server 20 or the like to display the screen (display unit 132P71) shown in FIG.

The terminal device 10 displays the screen at the timing (timing at which information for displaying the screen is received from the server 20 and displayed by a browser or the like), or when a predetermined time has elapsed since the screen was displayed. In response to the satisfaction of predetermined conditions such as timing and that the user has performed a predetermined operation on the screen, the display unit 132PA2 shown in FIG. 25A is displayed to prompt the user to answer the questionnaire.

FIG. 25(A) shows a screen (FIG. 21(A)) on which the terminal device 10 displays the results of identifying possible troubles of the user's body based on the measurement results and evaluation results of the user's body movements. The display unit 132PA2 for the user to answer the questionnaire is displayed so as to be overlaid on the display unit 132P71) described in .

The terminal device 10 displays, on the display section 132PA2, a question sentence for the user ("What kind of trouble do you have with your body?") on the display section 132PA3.
Server 20 determines the content of the question to be presented to the user on display unit 132PA3. For example, the server 20 holds a plurality of question sentences in advance, and holds information about body parts of the user and information of candidate answers to the question sentences in association with each question sentence. The server 20 refers to the measurement result database 287 and the evaluation result database 288, determines a question to be presented to the user based on the history of measurements performed by the user, and causes the terminal device 10 to display the determined question. You can do it. For example, based on the user's history of measurements with the measurement system 1 (such as the time intervals between measurements), the server 20 determines if a certain period of time has passed since the user last measured the physical condition of the body. It is also possible to present the user with a question (such as "What kind of trouble do you have in your body?") as to whether or not the user is aware that each part is ill.

Further, for example, the server 20 may determine body parts to be included in a question to be presented to the user based on the measurement results of the user's body parts. For example, if there is a part of the user's body that has not been measured for at least a certain period of time (for example, there is no history of measuring the waist, or a certain period of time has passed since the waist was measured) (i.e., "How is your waist?"), and the relevant body part may be included in the question.

In the example of FIG. 25(A), the terminal device 10 displays candidate answers to the question on the display unit 132PA4, and accepts selection from the user. In the illustrated example, on the display unit 132PA4, as candidates for the answer to the question sentence, "waist trouble" for designating a waist-related disorder, "knee trouble" for designating a knee-related disorder, and other physical conditions are displayed. "***" is displayed for displaying answer candidates for the body part. The terminal device 10 accepts the designation of "low back pain" from the user, and by highlighting the candidate, the user can easily recognize that the user has designated it.

The terminal device 10 displays a question about the degree of anxieties (the degree of disorder of a part of the body) on the display unit 132PA5.

The terminal device 10 displays, on the display unit 132PA6, answer candidates for the question displayed on the display unit 132PA5, and accepts selection from the user. In the illustrated example, the terminal device 10 displays, on the display unit 132PA6, “mild” indicating a light level of concern, “medium” indicating a medium level of concern, and “heavy” indicating a high level of concern, as options. ing. The terminal device 10 accepts the specification of the option “moderate” from the user and highlights the specified option.

The terminal device 10 receives an operation from the user on the display unit 132PA1 to indicate that the answer to the question has been completed. Thereby, the terminal device 10 transmits the user's answer result to the server 20 . The server 20 updates the user questionnaire information 289 by receiving the user's answers. For example, when the user specifies information about a body part such as "waist trouble" as an answer result, the server 20 updates the user questionnaire information 289 based on the user's trouble "waist trouble".

FIG. 25B is a screen example of a situation in which a training menu is proposed to the user based on the result of receiving the user's answer to the question, the measurement result of each part of the user's body, and the evaluation result. .

Based on the results of the user's response to the question (user questionnaire information 289), the server 20 collects information about the user's troubles about the body part ("You seem to have a moderate burden on your knees," etc.). information including the body part and the degree of worry, etc.) is displayed on the display unit 132P81. The terminal device 10 displays the cause of the user's concern about the body ("Are you in an unreasonable posture for a long time?", etc.) on the display unit 132P81. The terminal device 10 displays, on the display unit 132P81, a guideline for relieving the user's physical troubles (eg, "relax the muscles around the waist") and a suggestion of an exercise corresponding to the guideline.

<Other Modifications>
(1) Asking User Questions about Exercise Experiences In the above description of the embodiments, an example in which the server 20 presents a question sentence to the user and proposes a training menu based on the result of the answer to the question. explained. The server 20 may ask the user about their experience of exercising as the content of the question presented to the user.

For example, the server 20 may present the user with a question about his/her experience of playing sports in the past and habits of exercising. The server 20 receives an answer to the question from the user (such as the sports the user has participated in in the past, whether or not the user has a habit of exercising, etc.). Based on the user's answer, the server 20 asks the user if the user has relatively little experience of exercising (for example, the length of time the user has played sports is less than a certain period of time, the habit of exercising is less than a predetermined frequency, etc.). On the other hand, among the training items included in the training item information 285, a training item with a relatively small load on the body or a training with a relatively low degree of difficulty may be proposed. For example, for each training item in the training item information 285, the degree of difficulty (training that can be performed by a person who is not accustomed to exercise without problems is lower, etc.), the load on the body (for example, a specific part (upper arm, etc.) ), the load on the body is different for each training method). Thereby, a training menu can be presented based on the user's exercise experience.

According to the above example, by proposing a training menu that is relatively easy to achieve, a user who is not accustomed to exercise can easily feel a sense of accomplishment by completing the training menu. This can motivate the user to continue training.

(2) Evaluate whether the user is accustomed to exercise based on the measurement results of the user's body movement. may be evaluated, and a training menu to be presented to the user may be determined based on the evaluation results.

For example, the server 20 associates the measurement results of the user's body movements, such as the range of motion of the joints, the walking posture, and the balance of the trunk, with the patterns of the measurement results when the user is not accustomed to exercise, and the patterns of the measurement results when the user is accustomed to the exercise. and the pattern of the measurement result of . The server 20 evaluates the degree to which the user is accustomed to exercise based on the measurement result of the user's body movement. For example, the server 20 outputs an evaluation value indicating whether or not the user is accustomed to exercising by comparing the above pattern with the measurement result of the user's body movement and evaluating the degree of agreement. For example, the server 20 presents a model body movement to the user and evaluates that the user is accustomed to exercising when the user can move the body to the same degree as the body movement. At this time, if there is a discrepancy between the movement of the model's body and the measurement result of the user's body movement, a voice may be output to the user so as to correct the discrepancy. For example, if the position of the hand is different from the modeled movement, the voice output prompts the hand to follow the modeled movement (such as "move the hand position in a predetermined direction"). good too. Also, when the user's body movement and the model movement match (that is, when the user can perform the same movement as the model movement), the user's body movement is appropriate. output by voice, etc.

For example, when measuring the movement of the user's body, the server 20 determines whether the user is accustomed to exercise based on whether the user can move the body according to the video or audio guidance presented to the user. It may be output. For example, the server 20 may evaluate that the user is used to exercising when the user is able to move his/her body in accordance with the voice output.

The server 20 determines a training menu to be proposed to the user based on the evaluation value.

As described above, the server 20 holds the history of the measurement results by the measurement unit 30 in the measurement history database 283, and provides the user with the time and evaluation result of evaluating whether the user is accustomed to exercise, A history of training menus proposed to the user may be displayed. Thereby, the user can confirm that the user is gradually accustomed to the exercise and that the training menu is changing, and can realize a sense of self-efficacy.

(3) Asking User a Question About Continuing Training In addition, the server 20 may present a question about continuing training to the user. For example, questions may be presented regarding the user's personality, whether or not he or she was able to continue training in the past, and the length of time until training is stopped.

The server 20 determines the training menu to be presented to the user based on the results of the answers to the questions. For example, as described in the above fourth embodiment and the like, it is assumed that a problem that may occur in the user's body is identified and a training menu related to body parts associated with the problem is presented to the user. If the user gives a negative answer as to whether or not the training can be continued (there is a tendency that the training cannot be continued, etc.), the server 20 changes the training menu proposed to the user according to the task. You can do it. For example, when there are a plurality of training menus related to body parts associated with the task, another training menu is proposed to the user. This is expected to make it easier for the user to work on the training without getting bored, so that the user can continue the training.

Based on the measurement history database 283 and the like, the server 20 identifies intervals between timings at which the user has performed training. That is, the frequency with which the user exercises is specified. Server 20 may present questions to the user regarding continuing training based on the length of the interval. For example, when the frequency of training is decreasing, a different training menu can be proposed so that the user does not get bored by accepting questions about continuing training from the user. Also, the server 20 may change the training menu proposed to the user based on the length of the interval. As a result, the user can continue training without getting bored, and the user can continue training.

<Fifth Embodiment>
<Overview>
In the fifth embodiment, a technique will be described in which the user can more easily check whether or not the user is making ideal movements by moving his/her body.

(1) By visually reducing the amount of information about the body movement when the user moves the body, the body movement can be easily confirmed.
Specifically, when the user performs a specific exercise (for example, exercise for evaluating trunk muscle movement and joint range of motion such as side elevation, or exercise for strength training), the user's The motion of the body is sensed as point cloud data, and based on the point cloud data, an avatar corresponding to the user and auxiliary lines corresponding to the motion of the user's body are drawn and presented to the user. Thereby, the user can more easily check whether each part of the body can be moved as intended.

(2) Make it easier for trainers to guide trainees online using video conferencing tools and the like.
In the fifth embodiment, based on the above technology, a trainee (first user) who receives instruction on how to move the body and a trainer (second user) who instructs the trainee on how to move the body, Describe the aspects of teaching online.

Specifically, by enabling the trainer to check the avatar that expresses the trainee's body movement as point cloud data on the trainer's terminal, the trainer can view not only the image of the trainee's body, but also the image of the trainee's body. To make it possible to check the movement of an avatar with less visual information than images. This makes it easier for trainers to give trainees online advice on how to move their bodies.

That is, in the fifth embodiment, (1) the first processor of the terminal of the trainee (first user) senses the movement of the trainee (first user) as point cloud data using the measurement unit 30 or the like. (2) The first processor transmits information based on the point cloud data of the trainee (first user) (for example, an image of the trainee's avatar drawn based on the point cloud data, or the point cloud data itself, etc.) to the trainer (first user) A series of techniques for performing various processes such as transmitting to terminals of two users) will be described.

Here, the body movement of the trainee (first user) is sensed as point cloud data, and the first processor transmits information based on the point cloud data of the trainee (first user) to the terminal of the trainer (second user). However, devices that execute various processes include terminal devices such as computers, smartphones, and tablet devices possessed by users.

<Data structure in the fifth embodiment>
FIG. 26 is a diagram showing data structures of a trainee information database 2601, a trainer information database 2602, and a training history database 2603 stored by the server 20 in the fifth embodiment.
Although not shown, the server 20 also detects the types of body movements performed by the user (movements for evaluating the range of motion of joints such as side elevations, movements for strength training such as push-ups, Predetermined body movements such as yoga and other types of movement), information on the type of auxiliary lines when displaying the user's body movements as an avatar of a collection of point cloud data, and the auxiliary lines (position associated with the part of the user's body represented by the avatar) at which the .

The trainee information database 2601 is a database for managing information on users who become trainees. The trainer information database 2602 is a database for managing information on users who become trainers. The training history database 2603 is a database for managing the history of the trainee's training under the guidance of the trainer.

As shown in FIG. 26, the trainee information database 2601 includes an item "trainee ID", an item "name", an item "age", an item "sex", an item "weight", and an item "body fat percentage". , the item "purpose", the item "ideal figure", the item "screen display", the item "physical fitness level", the item "remarks", and the like.

The item "trainee ID" is information for identifying each user who receives training.

The item "name" is information indicating the name of each user who receives training.

The item "age" is information indicating the age of each user who receives training.

The item “gender” is information indicating the gender of each user who receives training.

The item “weight” is information indicating the weight of each user who undergoes training.

The item "percentage of body fat" is information indicating the percentage of body fat of each user who undergoes training.

The item "purpose" is information indicating the purpose of each user who receives training. Specifically, the user identified by the name "Mori" with the user ID "U001" has the purpose of training to "shape up", and the user identified by the name "Sato" with the user ID "U002" "Diet" is the purpose of training. Here, in a certain aspect, a user (trainer) who is in a position to instruct a user (trainee) undergoing training may present a training menu to the user (trainee) according to the purpose. In a certain aspect, terminal device 10 or server 20 may specify a training menu to be presented to the user based on a learned model or the like, which will be described later.

The item "ideal figure" is information indicating the ideal figure of each user who undergoes training. Specifically, the user identified by the name "Mori" of the user ID "U001" has an ideal figure of "not specified", and the user identified by the name "Sato" of the user ID "U002" "〇〇" is specified as the ideal figure. Here, this ideal figure may be specified by a real entertainer or the like, or a fictional figure (for example, if the body shape of a cartoon character is ideal, the user can specify the character). may be In a certain aspect, server 20 may specify and present a training menu based on the contents of the "ideal figure" and "purpose".

The item "screen display" is information indicating a display mode when the user is photographed by a camera or the like. As a display mode, when the user is photographed with an RGB camera or the like, the photographed image generated by the photographing is displayed as it is, and a sensor capable of measuring distance (depth camera, LiDAR (light detection and ranging), etc.) ) measures the movement of the user's body as point cloud data and displays it as an avatar based on the point cloud data. Specifically, the user identified by the name "Mori" of the user ID "U001" is displayed as "avatar" on the screen, and when the moving image of the user captured by a camera or the like is displayed on the screen, It indicates that the user himself/herself is displayed as an avatar (for example, an avatar using a character) based on the captured moving image instead of the captured moving image as it is.

As a result, it is possible to respond to the requests of the user undergoing training, for example, because it is the first training, the user does not want others to see his/her face and body, and wants to protect privacy, so that the user can receive the training with peace of mind. can be done.

The item “physical fitness level” is information indicating the physical fitness level obtained by evaluating the physical exercise ability of each user who undergoes training in stages. Specifically, when sensing the movement of the user's body undergoing training, at the same time, based on the information on the movement of the user's body, the physical fitness level is used as a criterion for judging how much exercise the user is likely to be able to do. are identified. The user identified by the user ID "U001" and the name "Mori" has a physical strength level of "5", and the user identified by the user ID "U002" and the name "Sato" has a physical strength level of "2". be. At this time, as a method of specifying the physical fitness level, the physical fitness level of the user may be specified based on a learned model or the like when the terminal device 10 or the server 20 senses the movement of the user's body. A user (trainer) who is in a position to instruct the user (trainee) who receives the training may specify the physical fitness level of the trainee based on the sensing result.

As a result, the user can receive a training menu suitable for his or her physical fitness level, so that the user can continue training without difficulty. In addition, in the case of group training, the server 20 can identify users with the same physical strength level and match users (trainers, trainees) so that the identified users can train together. For example, when users (trainees) who have different physical strength levels perform training, there may be differences in the trainer's instruction content and the trainee's training content, which may lead to a decrease in motivation. You can prevent the physical strength level so that.

The item "remarks" is information indicating circumstances that should be specially considered for each user who undergoes training. Circumstances that should be taken into special consideration include items that should be considered based on safety during training (safety based on a medical point of view, safety of the training content itself, etc.). For example, if the user has high blood pressure and the doctor has stopped excessive training, the relevant matter is described in the remarks. The server 20 may specify the optimal training menu for the user who receives the training based on the contents described in the item "Remarks". In addition, by acquiring the information of the item "remarks" from the server 20 and presenting it to the trainee in the terminal device 10B of the trainer, the trainer can provide information that can be used as a reference when determining the training menu for the trainee. good too.

As a result, even if the user has a chronic disease, the server 20 or the like identifies the training menu to be presented to the user based on the item "remarks", and the trainer selects a training menu based on the item "remarks". As a result, the user can continue training without difficulty.

The trainer information database 2602 includes an item “trainer ID”, an item “name”, an item “gender”, an item “years of experience”, an item “strength area”, an item “recommended physical fitness level”, and an item “guidance possible types” and items such as “remarks”.

The item “trainer ID” is information for identifying each user who is in a position to instruct the user who is training.

The item "name" is information indicating the name of each user who is in a position to guide the user who is undergoing training.

The item “gender” is information indicating the gender of each user who is in a position to guide the user who is undergoing training.

The item "years of experience" is information indicating the number of years of experience in training guidance of each user who is in a position to guide the user who is to undergo training. Specifically, the user identified by the name “Tanaka” with the trainer ID “T001” has “5 years” of experience, and the user with the trainer ID “T003” with the name “Kato” is shown. indicates that the user identified by has "10 years" of experience.

As a result, for example, in a certain situation, when a user (trainee) who receives training designates a user (trainer) who is in a position to give guidance, by designating a user who is in a position to give guidance with reference to the number of years of experience, You can train with confidence. As a user (trainer) who is in a position to give guidance, by being designated as a user who is in a position to give guidance based on the information, it is possible to improve his/her name recognition.

The item "area of specialty" is information indicating the area of specialty in training guidance of each user (trainer) who is in a position to instruct a user (trainee) who is to undergo training. Specifically, the user identified by the name “Tanaka” of the trainer ID “T001” is good at “yoga” and “exercise”, and the user identified by the name “Saito” of the trainer ID “T002”. indicates that "muscle training" and "exercise" are areas of expertise.

As a result, the server 20 or the like can be a user (trainer) who is in a position to instruct the user according to information such as the item "purpose" and the item "ideal figure" of the above-described trainee information database 2601 for the user undergoing training. may be specified. As a result, it is possible to match a trainer that matches the purpose of the trainee's training. For example, while a trainee aims at exercise, the server 20 or the like can assign a trainer who is good at training that matches the purpose as a leader.

The item "recommended physical fitness level" indicates the physical fitness level of the trainee, which is defined for each user (trainer) who is in a position to instruct the user (trainee) who receives training and is recommended when receiving training from the user (trainee). . Specifically, for guidance from a user (trainer) identified by the name "Tanaka" of the trainer ID "T001", the recommended physical strength level is set to "3 or higher", and the user ID "U001" Since the physical strength level of the user (trainee) identified by the name "Mori" is "5", it can be seen that the physical strength level for receiving guidance from the user has been reached. Since the user (trainee) identified by the name "Sato" in the above has a physical fitness level of "2", it can be seen that the physical fitness level has not reached the standard for receiving guidance from the user.

As a result, the user who undergoes training can receive training suitable for his/her physical fitness level. In addition, since users who are in a position to instruct users who receive training can also instruct training to users who are above a predetermined level, it is possible to suppress the occurrence of variations in training menus and proficiency levels among users. It can also be expected to maintain the motivation of users who are undergoing training. The server 20 or the like may match the trainer and the trainee based on the item "recommended physical fitness level" of the trainer and the item "physical fitness level" of the trainee.

The item "guidable type" indicates the type of training that can be instructed by a user (trainer) who is in a position to instruct a user undergoing training. Specifically, the user identified by the name “Tanaka” with the trainer ID “T001” has the coachable types “individual” and “group”, and is identified by the name “Saito” with the trainer ID “T002”. The user indicates that the coachable type is "individual" only.

As a result, it can be seen whether the user (trainer) in the position of instructing the user undergoing training is suitable for individual instruction or for group instruction. In matching with , it is possible to distribute training by one-to-one individual guidance and training by one-to-many group guidance. Also, the terminal device 10 of the trainee may present the trainer's "trainable type" to the trainee.

The item "remarks" is information such as other special notes of each user (trainer) who is in a position to instruct the user who is to undergo training. Remarks may include information that objectively evaluates the performance of training by a trainer. Specifically, it can be seen that the user identified by the name "Tanaka" of the trainer ID "T001" is "** grand prix winner".

As a result, in a certain situation, when a user (trainee) undergoing training designates a user (trainer) in a position to instruct the user undergoing training, information in the item "remarks" is provided to the trainee on the terminal device 10 of the trainee. By presenting, the trainee can designate a trainer with this information also taken into consideration. A user (trainer) who is in a position to instruct a user undergoing training can also expect to increase the chances of being designated as a trainer by recording the information.

The training history database 2603 includes an item “training ID”, an item “date and time”, an item “meeting ID”, an item “trainee ID”, an item “trainer ID”, an item “sensing result”, an item “guidance content”, item “trainer evaluation”, item “matching ID”, and the like.

The item “training ID” is information for identifying training by trainers and trainees.

The item "date and time" indicates the date and time when the training was performed.

The item "meeting ID" is information for identifying each communication session in the case of online training guidance through communication between the terminal device 10B of the trainer and the terminal device 10A of the trainee. For example, information identifying a communication session includes information such as a URL for an online meeting.

The item "trainee ID" is information for identifying the trainer who has performed the training.

The item “trainer ID” is information for identifying a trainee who received training guidance from the trainer.

The item "sensing result" is the result (point cloud data) of the terminal device 10A measuring the movement of the trainer's body with the measurement unit 30 during training, or the point cloud data is placed in the virtual space and set to the virtual camera. (Avatar expressing the trainee's body movement as a set of point cloud data in the video and how the trainee moves (the type of body movement presented to the user, such as side lifting)) (including auxiliary lines drawn according to ).

The item "guidance content" indicates the content of guidance given by the trainer to the trainee regarding how to move the trainee's body. For example, when the trainer instructs the trainee how to move the body by voice (including advice to guide the user to the desired posture in how to move the body. (Trainer's advice to take a desired posture when arms, legs, etc. are bent), the voice including the guidance content, or the analysis result of the voice is stored in the item "guidance content".

The item "trainer evaluation" indicates the evaluation received by the trainer from the trainee. For example, after conducting online training, the trainer's evaluation is accepted from the trainee by presenting the trainee with a screen for inputting the trainer's evaluation on the trainee's terminal device 10A.

The item “matching ID” is information that identifies each matching when the server 20 matches the trainer and the trainee. If the matching is established, the trainer and the trainee may start training, or even if the matching is established, the training may not be carried out.

<Example of operation in the fifth embodiment>
Hereinafter, in the fifth embodiment, the first processor of the terminal of the first user (trainee) in the system 1 senses the body movement of the first user as point cloud data, and the first processor senses the first A series of processes when information based on the user's point cloud data is transmitted to the terminal of the second user (trainer) and various processes are executed will be described.

(Sensing the first user's movement and sending it to the second user's terminal)
FIG. 27 is a flowchart showing a series of processes for sensing the movement of the first user's body as point cloud data and transmitting information based on the first user's point cloud data obtained by sensing to the second user's terminal. is.

In the following description, regarding the terminal device 10 shown in FIG. 1, the terminal device 10 of the first user is referred to as "terminal device 10A", and the configuration of terminal device 10A is referred to as "processor 19A" and the like. Similarly, the terminal device 10 of the second user is described as "terminal device 10B", and the configuration of terminal device 10B is described as "processor 19B" and the like.

Further, in the example of FIG. 27 , by establishing communication between the terminal device 10A of the first user and the terminal device 10B of the second user, the video captured by the camera 160 of each terminal and the sound collected by the microphone 141 are collected. It is assumed that the two terminals transmit and receive voice to and from each other.

For example, the terminal device 10A and the terminal device 10B are running an application for conducting an online video meeting, and identification information for participating in the meeting (for example, meeting ID issued by the application, URL etc.).

The terminal device 10A acquires the movement of the first user's body as point cloud data using the measurement unit 30 or the like. The terminal device 10A arranges the point cloud data in the virtual space, and sets the virtual camera in the virtual space to the position of the front of the first user's body (front-rear axis in the description of FIG. 11) and the side of the first user's body. (the left and right axis in the description of FIG. 11), rendering is performed, and the rendering result is displayed. The rendering result includes an avatar as a set of point cloud data when the first user is viewed from the front and an avatar as a set of point cloud data when the first user is viewed from the side. The terminal device 10A draws an auxiliary line corresponding to how the user moves the body, in association with the display area of the rendering result (the avatar viewed from the front and the avatar viewed from the side).

The terminal device 10A displays an image drawn with the avatar and auxiliary lines on the screen, and screen-shares the image with the terminal device 10B by an application for online video meeting. As a result, the image of the first user captured by camera 160A, the sound picked up by microphone 141, and the image in which the avatar and auxiliary lines are drawn are transmitted from terminal device 10A to terminal device 10B. The second user of the terminal device 10B confirms the movement of the body of the first user by visually recognizing the video captured by the camera 160A and the image shared on the screen (the image in which the avatar and auxiliary lines are drawn). Then, the advice to the first user is notified to the first user by voice or the like.

In step S2701, the processor 19A of the terminal device 10A of the first user senses the movement of the first user's body as point cloud data using the measurement unit 30 or the like.

Specifically, the processor 19A displays to the first user a menu of body movements performed by the first user so that the first user can perform specific movements (e.g., side movements) for body movement sensing. Encourage them to do movements such as side lifts, movements that can be used as training items (squats, etc.). The processor 19A of the terminal device 10A of the first user captures the specific movement using the measurement unit 30 or the like, thereby sensing the body movement of the first user as point cloud data.

In step S2703, the processor 19A generates an avatar representing the first user (trainee) and body movements performed by the first user as information based on the first user's point cloud data obtained by sensing by the measurement unit 30 or the like. (type of exercise menu presented to the first user).

In step S2705, the processor 19A displays the image generated in step S2703 on the display 132A of the terminal device 10A. The processor 19A processes the displayed image (image including the avatar and the auxiliary lines) and the separately captured image generated by the camera 160A that captures the first user (not the set of point cloud data, but the image captured by the RGB camera). captured image) to the terminal device 10B of the second user.

In step S2751, the processor 19B of the terminal device 10B of the second user receives from the terminal device 10A of the first user the image (avatar and auxiliary line ). The processor 19B receives the captured image of the first user (the captured image that is not the set of point cloud data) generated in step S2705.

In step S2753, the processor 19B generates the first avatar corresponding to the movement of the first user's body as a set of point cloud data based on the video (including the avatar and auxiliary lines) received in step S2751 and the captured image. is displayed on the display 132B or the like. The processor 19B displays the captured image of the first user (not the set of point cloud data) on the display 132B or the like. As a result, the first avatar and the photographed image of the first user are presented to the second user as the information on the body movement of the first user.

Here, in step S2703, the processor 19A identifies various types of information regarding the first user's exercise (which part of the body is easy to move, hard to move, etc.), physical fitness level, etc. based on the trainee information database 2601 of the server 20. It is also possible to At this time, the processor 19A may store the specified information in the storage unit 180 of the terminal device 10A of the first user, or may transmit the information to the server 20 and store it in the trainee information database of the storage unit 202 of the server 20. 2601 may be stored.

Further, the processor 19A refers to the trainer information database 2602 based on the result of sensing the body movement of the first user by the measurement unit 30 or the like, and identifies the second user (trainer) to whom the information is to be transmitted. You may

As a result, a user who is a trainee can receive training from an optimal instructor based on the information about the body movement at the start of training.

Here, in step S2751, data sensed by the measurement unit 30 or the like is received from the terminal device 10A of the first user, and rendering is performed in the terminal device 10B based on the point cloud data based on the received information. may be That is, the terminal device 10B, not the terminal device 10A, may render the first avatar representing the body movement of the first user based on the point cloud data measured by the terminal device 10A. The terminal device 10B generates an image including the first avatar, and displays the generated image on the display 132B or the like. At this time, the motion of the body of the first user may be drawn from a plurality of viewpoints, and an image in which the first avatars drawn for each viewpoint are arranged may be presented to the second user.

In one aspect, the terminal device 10B draws an auxiliary line at a position based on the body part for evaluating whether or not the body movement performed by the first user is ideal, and displays the video with the drawn auxiliary line as a second user. may be presented to the user. For example, when the trainer evaluates whether the user's body movement is ideal based on whether the user's back is straight and whether the knee is at a predetermined angle to the ground in the exercise that the user should perform , the terminal device 10B displays an auxiliary line corresponding to the shape of the stretched back muscle near the part of the back muscle of the avatar. When the terminal device 10B evaluates whether the knees are at a predetermined angle with respect to the ground, the terminal device 10B places the knees near the avatar's knees (or in the region where the knees are displayed) at the predetermined angle. display the auxiliary lines near the avatar's knees.

In addition, the display mode of the body surface position other than the joint positions is changed based on the part of the body that is evaluated as to whether or not the body movement performed by the first user is ideal, and the display mode of the body surface is changed. The video may be presented to the second user.

The surface of the body that is not the joint position is, for example, a surface that can be recognized as the external appearance of the body, such as the surface of the abdomen. For example, in the terminal device 10B, the displayed color may be changed depending on whether the movement of the first user's body is too large or too small compared to the ideal movement. For example, when the abdomen protrudes too far forward than the ideal movement, the abdomen is displayed in a warm color, and when the movement of the abdomen is smaller than the ideal movement, the abdomen is displayed in a cold color. , to change the colors displayed. Accordingly, the first user or the second user can easily confirm how to move the user's body in an ideal manner. In addition, this allows the user to train while grasping how much the movement of his/her body deviates from the ideal movement. In addition, by changing the display mode of the body surface, it is possible to grasp the movements of the muscles on the body surface instead of the joints.

Here, in the above example, regarding the video including the first user's first avatar displayed on the display 132B or the like, the first user's captured image captured when generating the first avatar and the first avatar's described as presenting both to the second user. In addition, the terminal device 10B on the trainer side may refer to the trainee information database 2601 and display a screen according to the designation of the item "screen display" on the display 132 or the like.

For example, in the terminal device 10B, the movement of the body of the first user is displayed as the first avatar (that is, the collection of point cloud data), while the captured image of the first user (not the collection of point cloud data) may be hidden. Further, when a plurality of trainees receive training from a trainer, the photographed images of the trainees (those that are not aggregates of point cloud data) can be displayed on the terminal device 10 of the trainer, and simultaneously with the trainer. It may be hidden on the terminal devices 10 of other trainees who are taking the training. As a result, both the point cloud data and the photographed image of the trainee's sensing results can be disclosed to the trainer, while the avatar (collection of point cloud data) is shared among the trainees, but the photographed image is not shared. can also

This allows users to train while protecting their privacy.

Also, in one aspect, at least one of processor 19A, processor 19B, or processor 29 of server 20 defines a virtual space and displays an image including the first avatar associated with the first user in the virtual space. and a second object for displaying a video including a second avatar associated with a third user who trains with the first user, and processor 19A or processor 19B performs virtual space An image including the first object and the second object may be displayed on display 132A, display 132B, or the like.

As a result, the user can experience training in the same virtual space, such as training performed by a plurality of people, and can obtain a sense of unity and a sense of accomplishment in the training, which can be expected to maintain and improve motivation.

Next, with reference to FIG. 28, the flow of processing when the trainer instructs the trainees online will be described.

In the following description, while the trainer instructs the trainee by voice, by controlling the application running on the trainee's terminal device 10 with the trainer's voice, the trainer can easily obtain information suitable for the instruction. An example of how to Specifically, on the trainee's terminal, the trainer can control the drawing of the avatar based on the point cloud data, the drawing of the auxiliary lines, and the determination of the type of body movement to be imposed on the trainee by voice input. make it

FIG. 28 is a flow chart showing the flow of processing for receiving an input of the trainer's voice from the trainer's terminal and transmitting it to the trainee's terminal.

In step S2851, the processor 19B of the terminal device 10B of the second user accepts the voice input of the second user through the microphone 141B or the like, and transmits the voice of the second user from the terminal device 10B of the second user to the voice of the first user. It transmits to the terminal device 10A.

In step S2801, the processor 19A of the terminal device 10A of the first user outputs the voice of the second user received from the terminal device 10B to the first user through the speaker 142A or the like.

In step S2803, the processor 19A analyzes the second user's voice received from the terminal device 10B by voice recognition. The processor 19A identifies the content of the second user's voice based on the result of the voice recognition analysis. The processor 19A identifies the content of the instruction to the application for drawing the avatar based on the point cloud data of the first user, based on the identified content of the voice of the second user. The processor 19A controls the application based on the specified instruction to the application.

Specifically, the content of the voice of the second user specified by the processor 19A is as follows:
(i) There is content for making the first user's body movement ideal. The processor 19A, for example, through voice recognition, recognizes words related to the user's body parts (knees, elbows, arms, shoulders, etc.) in the voice of the second user. In addition, the processor 19A selects, from the voice of the second user, a word indicating how to move the body part of the first user in an ideal motion (for example, "bring your knees close to your body", "legs level”, “stretch your back”, etc.).

(ii) The content of the voice specified by the processor 19A includes content indicating the type of exercise (training menu, etc.) that the first user should undertake for training of the first user.

If the second user's voice includes the type of exercise, the processor 19A sets the type of exercise to be imposed on the first user to be the exercise specified by the voice of the second user, and The auxiliary line drawn in association with the corresponding avatar is made to correspond to the designated exercise type. This makes it easier for the second user to evaluate the body movements of the first user.

(iii) The content of the voice specified by the processor 19A includes content that causes an application that draws the first user's avatar and auxiliary lines, which operates on the terminal device 10A, to perform a specific process. For example, in the application, commands indicated by the voice of the second user include the following.
・ Avatar drawing mode (including the position where the virtual camera is arranged with respect to the point cloud data representing the body movement of the first user. In other words, the avatar corresponding to the first user is drawn from the front or from the side) or draw from other directions, etc.)
・ How to draw the auxiliary line (as the auxiliary line, draw the auxiliary line parallel to the user's body movement, draw the auxiliary line along the shape of the user's body, whether to draw the auxiliary line etc.)
・ How to draw the avatar when there is a difference compared to the ideal movement (including drawing the parts of
・ Instructions to suspend the application and other contents

When the processor 19A recognizes that the speaker is a trainer by speech recognition (for example, if the speaker is a trainer registered in the trainer information database 2602), the processor 19A recognizes the first user based on the speech of the speaker. may control an application that draws the avatar of the This can prevent a user other than the trainer from unintentionally controlling the application. In addition, when communicating with another terminal device 10 such as the terminal device 10B by using a web conference tool or the like, the processor 19A, based on the information of the speaker specified by the web conference tool or the like, It is good also as controlling the application which draws an avatar. For example, in a web conferencing tool, etc., information (such as an icon) that identifies each user participating in the meeting is displayed, and the information that identifies each user is displayed separately for the user who is speaking. There is The processor 19A may perform the above processing assuming that the speaker is the trainer when the icon or the like corresponding to the trainer is displayed in a web conference tool or the like to be distinguished from the icons or the like of other users. good.

For example, when a plurality of trainees receive guidance from a trainer, depending on the trainee's voice, the application may perform processing related to training (processing to change the type of exercise to be performed by the trainee, processing to draw auxiliary lines with the avatar along with the change). change processing, etc.), and the trainer can be made to perform the processing. As a result, even if each user speaks during training, the training can be performed as intended by the trainer.

As described above, as a result of the second user giving voice guidance to the first user, the display mode of the video including the first avatar is changed, and based on the video including the changed avatar, the second user can One user can be instructed.

(Sensing the movement of the first user and identifying the instruction content based on the learned model)
FIG. 29 is a diagram showing the motion of the body of the first user sensed as point cloud data, and based on the information based on the point cloud data of the first user obtained by sensing and the learned model, the It is a flowchart which shows a series of processes which identify the content of instruction. As a result, even without a trainer, the trainee can check how the trainee moves his/her own body using the learned model.

In step S2901, the processor 19A of the terminal device 10A of the first user senses the movement of the first user's body as point cloud data using the measurement unit 30 or the like.

In step S2902, the processor 19A identifies guidance content for the first user based on the information based on the first user's point cloud data obtained by sensing and the learned model.

Here, a method for creating a learned model will be illustrated.
(i) For example, after acquiring information on point cloud data of one or more users, various information (eg, age, weight, purpose, etc.) and information of the user (trainer) who is in a position to instruct the user undergoing training Then, based on the content of the training actually received by the trainee and information such as the history of changes in the information of various trainees due to training, the information of the point cloud data of one or more users and the training content are associated. Create a trained model.

In other words, the trained model can be said to output how much the trainee changes as the trainee is trained by learning the information of the trainee and the content of instruction by the trainer as time-series data.

In addition, the trained model is associated with the information of the user who acquired the information based on the point cloud data, which is the learning data of the trained model (that is, the information of the trainee), or the information of the trainer who instructed the trainee. may have been Accordingly, when a trainee or a trainer designates a trained model to be used by the trainee, the trainee's information or the trainer's information associated with the trained model can be referred to. Alternatively, the processor 19A or the like may specify a trained model to be proposed to the user who is a trainee based on user information associated with the trained model.

(ii) Alternatively, in the trained model, the trainer attaches tags (body parts Generate the time-series data as learning data by commenting on how to move ideally, that you should breathe, that your heart rate is too high and you should take a break, etc.) You can do it.

For example, in one aspect, based on the result of measuring the body movement of the trainee and the learned model, a training menu is presented that matches the physical constitution of the user who is the trainee, the timing at which to breathe during training, and the heart rate. The timing to get up, the timing to drink water, the timing to take a break, etc. can be identified and presented to the user.

For example, in a certain aspect, a trainee wears a wearable device (wristwatch type, eyeglass type, ring type, etc.) and uses the above learned model while measuring the heart rate, breathing timing, etc. of the user who is a trainee. , the learned model may output the content to be instructed to the trainee.

In another aspect, the user's exercise intensity (exercise intensity) including calorie consumption is specified from information on point cloud data acquired from the user, and advice such as rest and hydration is presented to the user. be able to.
(iii) As a method for generating a trained model, as described in the first embodiment, the measured data (measurement history database 283) accumulated as a result of the ROM check is used as teacher data to generate a trained model. may be For example, a learned model may be generated based on the measurement data, the training menu performed by the user, and the results of responses to a questionnaire or the like input by the user.
As a result, the type of exercise proposed to the user, the content of advice to the user, and the like may be output to the learned model generated based on the measurement data of the ROM check.

As a method for generating a trained model, as described in the second embodiment, measured data accumulated as a result of evaluating the user's walking posture (measurement result database 287) is used as teacher data to generate a trained model. may be generated. For example, a learned model may be generated based on the measurement data, the training menu performed by the user, and the results of responses to a questionnaire or the like input by the user.
As a result, the type of exercise proposed to the user, the content of the advice to the user, and the like may be output by the trained model generated based on the measurement data for walking posture evaluation.

For example, as a method of generating a trained model, a trained model may be generated based on measurement data accumulated as a result of estimating a user's posture (joint positions, etc.). Specifically, as a method of generating a learned model in the server 20 or the like, a learned model is generated based on the result of estimating the user's joint positions, the training menu performed by the user, and the content of advice from the trainer. It is also possible to generate and store the generated learned model.

As described above, the server 20 may specify the instruction content for the user based on the information based on the estimation result of the user's posture and the learned model, and present the specified instruction content to the user.

Further, the server 20 or the like may store the learned model in association with the information of the user who acquired the learning data (result of estimating the user's posture) of the learned model.

Also, the server 20 and the like may identify the user's exercise intensity in the same manner as described above, based on the information based on the estimation result of the user's posture and the learned model. The server 20 may identify the guidance content for the user based on the result of identifying the exercise intensity of the user, and present the identified guidance content to the user.

In step S2903, the processor 19A presents the guidance content specified by the learned model to the first user by displaying it on the display 132A or the like.

<Screen example in the fifth embodiment>
30 to 32 are diagrams showing examples of screens displayed on the terminal device 10. FIG.

FIG. 30 shows that the terminal device 10A of the first user senses the movement of the body of the first user as point cloud data, and transmits information obtained by sensing based on the point cloud data of the first user to the terminal device 10A of the second user. It is an example of the screen when sending to.

In FIG. 30, the camera 2901 of the measurement unit 30 shows a device for sensing the body movements of the first user.

Specifically, the camera 2901 captures an image of the first user through a capturing unit provided in the camera 2901 . At this time, the camera 2901 is configured to be able to measure distance and direction, and is configured by, for example, an RGB-D camera and LIDAR (Laser Imaging Detection and Ranging) technology.

The previous screen 2902 is a screen that displays an image including the first user's avatar (that is, an image drawn based on point cloud data) when the first user performed training of the same kind before. be. The terminal device 10A acquires point cloud data representing the body movements of the first user when the first user received training in the past, for example, by referring to the training history database 2603 of the server 20 .

In the illustrated example, on the previous screen 2902, the first user's avatar is shown as an aggregate of figures such as circles, but it may be displayed as an aggregate of point cloud data measured by the measurement unit 30. .

To allow a first user and a second user to easily confirm how much the body movement of the first user has changed due to training by displaying an avatar based on point cloud data of the first user's past training. can be done. For example, when the first user holds the knees and performs a motion of curling the body, the terminal device 10A displays the previous screen 2902, which is the previous training screen. The first user or the second user can compare the previous screen 2902 with the display contents of the current screen 2903, which will be described later. Thereby, the first user can confirm the difference in the movement of his/her own body.

Further, the terminal device 10A may always display the previous screen 2902, or may display or turn off the display according to an instruction from the first user or the second user. At this time, the terminal device 10A may display, as an avatar representing the first user, images of the first user drawn from a plurality of viewpoints such as the front, the side, and the top. In addition, an auxiliary line may be drawn at a position based on a body part for evaluating whether or not the body movement performed by the first user is ideal. In the illustrated example, on the past screen 2902, an auxiliary line is drawn in association with the area where the first user's avatar is displayed.

The current time screen 2903 is a screen that displays an image of the first user's avatar based on the sensing result of the measurement unit 30 while the first user is training.

In the illustrated example, a first user 2094 is shown, showing a state in which the first user is actually training on a mat or the like. The terminal device 10A captures the movement of the first user with the camera 2901, and acquires the point cloud data information of the first user.

A first user image 2905 indicates a captured image of the first user captured by the camera 160 or the like. In the first user image 2905, an image captured by a 2D camera or the like is displayed instead of rendering the first user as an aggregate of point cloud data. That is, in the first user image 2905, a photographed image equivalent to that which can be visually recognized by the human eye is displayed.

Thereby, the terminal device 10A allows the first user to visually compare the actual image of the first user himself and the image based on the information of the point cloud data. In addition, the terminal device 10 does not display the photographed image of the first user himself/herself in the first user image 2905 according to the designation of the first user, and the second user or the like can identify the first user. icons, character strings, etc. may be displayed.

A second user image 2906 shows an image of a second user who is in a position to guide a user undergoing training. The terminal device 10A displays the second user's captured image in the second user image 2906 by receiving the second user's captured image from the second user's terminal device 10B. Thereby, the first user and the second user can perform training while looking at each other's images and communicating with each other through a terminal-to-terminal video call or the like through the network 80 or the like.

FIG. 31 shows the overall configuration when sensing the movement of the first user's body as point cloud data and transmitting information based on the first user's point cloud data obtained by sensing to the terminal of the second user. It is an example of the screen of the 1st user and the 2nd user including.

In FIG. 31, a camera 3001 is a device for sensing the motion of the first user's body as point cloud data, similar to the camera 2901 in FIG.

This time images 3002 and 3006 are screens that display images of the first user as an aggregate of point cloud data when the first user is training, similarly to the current time screen 2903 in FIG. At this time, the current image 3002 is displayed on the display 132A or the like of the first user's terminal device 10A, and the current time image 3006 is displayed on the display 132B or the like of the second user's terminal device 10B.

A first user 3003 is a diagram showing a state in which the first user is actually training on a mat or the like, similar to the first user 2904 in FIG.

First user images 3004 and 3007 show actual captured images, not videos containing avatars, when the first user is captured by camera 2901, similarly to first user image 2905 in FIG. At this time, the first user image 3004 is displayed on the display 132A or the like of the first user's terminal device 10A, and the first user image 3007 is displayed on the display 132B or the like of the second user's terminal device 10B. .

Second user images 3005 and 3008 show images of a second user who is in a position to guide a user undergoing training, similar to second user image 2906 in FIG. At this time, the second user image 3005 is displayed on the display 132A or the like of the first user's terminal device 10A, and the second user image 3008 is displayed on the display 132B or the like of the second user's terminal device 10B. . Further, in a certain aspect, as in the screen example, an input operation by voice from the second user causes the second user's voice (such as "Let's sit down more!") through the speaker 142A or the like of the terminal device 10A. ) may be output.

The second user 3009 visually recognizes the screen displayed on the display 132B or the like by the terminal device 10B, and gives guidance based on the image of the first user. At this time, the second user can give guidance to the first user by voice. The second user confirms both or one of an image based on the point cloud data and an image equivalent to that seen with the naked eye as an image of the first user, and responds to the movement of the body of the first user. can give advice. As a collection of point cloud data, the amount of information for the second user is reduced from the image of the first user shown by the actual photographed image, so that the second user can more easily identify the movement of the first user's body. and facilitate evaluation and advice.

FIG. 32 is an example of a screen when the body movements of a plurality of users are sensed as point cloud data, and information based on the point cloud data of each user obtained by sensing is transmitted to a defined virtual space. .

In FIG. 32, virtual space 3101 represents a space for one or more trained users to place objects for displaying videos, including avatars associated with them.

A trainer image 3102 shows an image of a second user who is in a position to guide a user undergoing training. The terminal device 10B of the second user takes a picture of the second user and transmits the taken image to the computer managing the virtual space as needed.

A user image 3103 is a screen that displays an image of each user when one or more users are training. At this time, the image to be displayed may be only a video containing an avatar as a collection of point cloud data associated with the user, or an actual photographed image of each user, which is not a video containing an avatar, may be displayed together. good too.

As a result, each user can place an image including an avatar corresponding to the movement of the user's own body in the virtual space, and each user can visually recognize the image of the avatar of each user placed in the virtual space. By doing so, it is possible to get the feeling that a plurality of users are training at the same place. As a result, a sense of unity and a sense of accomplishment in training can be obtained, and motivation to continue training can be provided to each user. In response to matching each trainee who receives training from the trainer, the server 20 enables communication between each trainee's terminal and the trainer's terminal, and builds a virtual space in either the server 20 or each user's terminal. It is also possible to That is, the server 20 may designate a computer that serves as a host when each user communicates.

<6 Modifications>
A modification of this embodiment will be described. That is, the following aspects may be adopted.

(1) In the examples of FIGS. 31 and 32 above, an avatar based on the user's sensing results when the user underwent training in the past and an avatar based on the current sensing results of training from a trainer. An example of displaying them side by side was explained.

Alternatively, avatars corresponding to trainees may be displayed side by side based on the time-series data of the trainee's point cloud data sensed along the time series during training with the trainer.

In other words, the avatar based on the point cloud data sensed at the timing of starting training or within a certain period of time after the start, and the result of real-time sensing based on the trainer's guidance or a certain amount of time after the trainer's guidance started The terminal device 10A may display the avatars based on the sensing results at the timings set side by side.

Also, in the course of one training session, the part of the trainee's body that has moved closer to the ideal one is emphasized compared to other parts and displayed on the avatar. It is also possible to For example, the processor 19A may evaluate how closely the body movements of the trainee are to the auxiliary lines drawn together with the avatar, and draw the screen according to the evaluation result. For example, if the ideal motion is to stretch the knee in a specific direction, the processor 19A draws an auxiliary line along the specific direction. The processor 19A changes the knee part of the avatar or the drawing mode of the auxiliary line associated with the knee part as the movement of the knee approaches the direction along the auxiliary line in the trainee's body movement. good too.

As a result, the trainee can easily check how much the trainee has changed before and after a single training session or during the course of training, so the trainee himself can check how much he has changed (such as whether he is getting closer to his goal). easier to do. This can further improve the trainee's motivation to continue training.

When training online, the visual information is reduced compared to training face-to-face instead of online (visible to the naked eye in face-to-face vs. two-dimensional display image for online). (visually recognized as such) may reduce the quality of the communication experience. Moreover, when training is conducted online, it is very difficult for the trainer to touch the body of the trainee. Therefore, by providing training result feedback to the trainee as described above, it is possible to provide the trainee with motivation to continue training online.

As described above, a learned model may be generated based on the sensing result when the user's body movement is changed to an ideal one by one training session.

(2) In the above embodiments, the avatar is displayed based on the result of sensing the movement of the user's body, and the avatar corresponding to the trainer is displayed to the trainee in online training between the trainer and the trainee.

In addition, the processor 19A may perform training while confirming how the user himself or herself moves as the avatar described above, regardless of the communication between the trainer and the trainee. For example, in the terminal device 10A, the user sets how to move the body, draws an auxiliary line or the like according to the set way to move the body, and notifies the user of the information on how to move the body ( The user's body movement may also be sensed by the measurement unit 30 or the like.

(2-1) Method for Sensing Movement of User's Body The terminal device 10A presents the result of sensing the movement of the user's body to the user as an avatar or the like. Further, as described in the first and second embodiments, the terminal device 10A identifies the positions of the joints of the user's body based on the sensing results of the user's body, and estimates the skeleton (bone). (Fig. 12).

The terminal device 10A may estimate the posture based on the position of each joint of the user and the result of estimating the skeleton in the following manner.
- The terminal device 10A estimates the posture of the user shown in the moving image based on the photographed result (two-dimensional moving image) of the user, who is the subject, without using the point cloud data.
That is, the terminal device 10A shoots the user with a camera that does not have a configuration for optically measuring the distance to the subject, such as LiDAR, and based on the shooting result, the user's posture in the shot image is: It is also possible to estimate joint positions, skeletons, and the like.
For example, by using a captured image as an input and preparing a trained model that outputs the positions of joints and postures such as the skeleton of a person who is the subject in the captured image, the terminal device 10A can obtain the user's captured result, The user's pose can be estimated based on the trained model.
Such a trained model is generated by preparing learning data in which information such as joint positions and bone structure in a photographed image is associated with a person who is a subject in the photographed image, and causing a computer to perform the learning. can.
- As described in other embodiments, the terminal device 10A senses the movement of the user's body as point cloud data based on the camera or the measurement unit 30 mounted on the terminal device 10A or the like, and the sensing result is Based on this, the positions of the joints, the skeleton, etc. are estimated.

(2-2) Drawing of Auxiliary Lines Based on Joint Position and Skeleton Estimation Result The terminal device 10A draws an auxiliary line for the user as a guideline for moving the body, based on the result of estimating the user's posture. It is also possible to The terminal device 10A may draw the auxiliary line based on the estimation result of the user's posture as follows.
The type of body movement presented to the user by the terminal device 10A or the like (for example, exercise for evaluating movement of trunk muscles and range of motion of joints such as side elevation, or exercise for strength training) motion, etc.), indirect positions, and skeleton estimation results, the positions for drawing auxiliary lines are determined.
For example, some types of body movements include those in which the user moves the body by a certain amount (for example, an action of stepping a front foot to evaluate the joint range of motion). While displaying the avatar to the user, the terminal device 10A draws an auxiliary line as a guideline for the amount of body movement based on the results of estimating the positions of specific joints of the user.
For example, when the user steps forward, the terminal device 10A first prompts the user to take a predetermined posture (for example, to stand upright before stepping forward) by voice or video. The terminal device 10A estimates the amount of stride of the user based on the positions of the joints in the posture (for example, shoulder joint, hip joint, knee joint) (for example, the height of the user, the length of the lower limbs of the user, The amount of stride to be taken can be specified), and based on the estimated amount of stride, determine the position where the auxiliary line should be drawn based on the joint position estimation result.
The terminal device 10A associates in advance the position of the joint referred to for drawing the auxiliary line and the positional relationship for drawing the auxiliary line with respect to the position of the joint, according to the type of body movement performed by the user. It is also possible to store the

FIG. 33 is a diagram showing a concept of processing in which the terminal device 10A draws auxiliary lines based on the result of estimating the posture of the user.

In the example (A) of FIG. 33, the terminal device 10A senses the movement of the user's body as point cloud data. The terminal device 10A stores the three-dimensional object data of the avatar AVT_A corresponding to the user based on the sensing result. Based on the 3D object data of the avatar AVT_A, the terminal device 10A estimates the positions of the user's joints (for example, each joint shown in FIG. 12) of the 3D object data. In example (A), when the three-dimensional object data of avatar AVT_A is arranged in the virtual space, terminal device 10A identifies joint position JNT_01A, which is the coordinate resulting from estimating the position of the user's hip joint in the avatar.
In the example (A) of FIG. 33, the terminal device 10A moves the three-dimensional object data of the auxiliary line LN_01A to a position separated by a certain distance DIST_01A from the joint position JNT_01A according to the type of movement of the user's body. Decide to place. The terminal device 10A may set the fixed distance based on the physique of the user (obtained by measuring the user's height, sitting height, shoulder width, etc. based on the result of sensing the movement of the user's body). . Depending on the physique of the user, the amount that the user should move the body can be determined. In example (A), the terminal device 10A arranges the three-dimensional object data of the auxiliary line LN_01A so as to be parallel to the ground when the user moves his or her body.

In the example (B) of FIG. 33 , the terminal device 10A arranges the three-dimensional object data of the auxiliary line LN_02A based on the joint position JNT_01A so that it overlaps with the joint position JNT_01A and is aligned with the ground surface when the user moves the body. are arranged so as to be perpendicular to the

FIG. 34 shows an example screen presented to the user.
As shown in FIG. 34, the terminal device 10A, as a result of sensing the movement of the user's body on the screen 3403, displays the avatar corresponding to the user in a plurality of positions such as the left side 3403A, the front 3403B, and the right side 3403C. Place the result drawn from the viewpoint on the same screen.

For example, the terminal device 10A arranges three-dimensional object data (that can be recognized by the user as an avatar) corresponding to the sensing result of the motion of the user's body (for example, point cloud data) in the virtual space. In the virtual space, the terminal device 10A captures three-dimensional object data corresponding to the sensing result of the user by using a plurality of virtual cameras (for example, a virtual camera corresponding to the right side, a virtual camera corresponding to the front, and a virtual camera corresponding to the left side). corresponding virtual camera). Thereby, the terminal device 10A can present to the user the result of drawing the avatar corresponding to the user from a plurality of viewpoints.

As illustrated, the terminal device 10A may present the avatar to the user, but may not present the result of estimating the skeleton (bone). For example, compared to visually recognizing oneself in a mirror or the like, it is assumed that the user has fewer opportunities to visually perceive the estimation result of the skeleton (bone) itself with the naked eye on a daily basis. Therefore, the terminal device 10A presents the avatar to the user and does not display the skeleton (bone) estimation result, so that the user can see on the screen an avatar similar to his/her own body shape that can be seen on a daily basis with the naked eye. Awareness and avatars projected onto the screen can be immersive as if the user were themselves.

The terminal device 10A arranges the three-dimensional object data of auxiliary lines LN_01A and LN_02A in association with the three-dimensional object data corresponding to the user in the virtual space. Therefore, the terminal device 10A can display the auxiliary line together with the avatar corresponding to the user by performing rendering according to the setting of the virtual camera.

Specifically, the terminal device 10A draws an auxiliary line LN_01A and an auxiliary line LN_02A in association with the avatar AVT_A on the left side 3403A and displays them to the user. The terminal device 10A draws an auxiliary line LN_01B and an auxiliary line LN_02B in association with the avatar AVT_B on the front 3403B and displays them to the user. The terminal device 10A draws an auxiliary line LN_01C and an auxiliary line LN_02C in association with the avatar AVT_C on the right side 3403C and displays them to the user.

(2-3) Method of Evaluating User's Body Movement (i) As described above, the terminal device 10A displays the avatar to the user without displaying the skeleton (bone) estimation result. By making them realize that they are a projection of themselves, they are immersed in moving their bodies and training. On the other hand, the terminal device 10A evaluates the movement of the user's body in order to evaluate whether the user's posture is desirable and whether the movement of the user's body is ideal according to the type of body movement. , an estimation result of the user's skeleton (bone) may be used.

For example, in the example of FIG. 34, when it is desirable that the user's knees are vertical, the terminal device 10A determines (the knees are vertical ), it can be determined whether the user's posture is desirable.
(ii) The terminal device 10A evaluates the movement of the user's body, and corresponds to three-dimensional object data (for example, point cloud data) forming an avatar, which is the result of sensing the movement of the user's body, and auxiliary lines. It may be based on three-dimensional object data to be used.

For example, in the terminal device 10A, the three-dimensional object data forming the user's avatar (for example, the avatar AVT_A on the left side 3403A) collides with the three-dimensional object data corresponding to the auxiliary line (for example, the auxiliary line LN_01A on the left side 3403A). Whether or not the user's body movement is ideal may be evaluated by determining that the user's body movement is ideal, or determining that the user has approached within a certain distance.

When the terminal device 10A senses the movement of the user's body without relying on the point cloud data (for example, when capturing the user as a two-dimensional image), based on the sensing result (two-dimensional captured image data), Three-dimensional object data corresponding to the user's body may be generated. The terminal device 10A evaluates whether the movement of the user's body is ideal or not, based on whether the three-dimensional object data and the three-dimensional object data corresponding to the auxiliary line collide or come close to a certain distance. It is also possible to

When the terminal device 10A senses the movement of the user's body without relying on the point cloud data (when capturing the user as a two-dimensional image), the user's posture (joint joints) is captured in the two-dimensional image as described above. Position, skeleton) may be estimated (using a trained model for estimating posture as described above), and based on the estimated result, auxiliary lines may be drawn as object data in the two-dimensional image. For example, the terminal device 10A, from the position of the user's joints (hip joint, shoulder joint, etc.) estimated in the two-dimensional image, corresponding to the type of exercise performed by the user, in the two-dimensional image, as object data Determines where to draw extension lines. The terminal device 10A determines whether the user's body movement is ideal based on whether the two-dimensional image corresponding to the user and the auxiliary line drawn as the object data collided or approached a certain distance. may be evaluated.

(2-4) Summary (summary 1)
A method for a first user to train, comprising:
A processor of the terminal of the first user,
a step of sensing movement of the body of the first user as point cloud data;
presenting, to the first user, an image including a first avatar corresponding to the movement of the body of the first user as a set of point cloud data based on the sensing result. death,
In the presenting step,
Along with the first avatar, an auxiliary line is drawn at a position based on the part of the body for evaluating whether or not the body movement performed by the first user is ideal, and the image on which the auxiliary line is drawn is displayed. A method of presenting to the first user.

(Summary 2)
In the presenting step, the second processor:
Identifying the type of the auxiliary line to be drawn corresponding to the type of body movement performed by the first user, and presenting the image in which the identified auxiliary line is drawn to the first user; The method described in (Summary 1).

(Summary 3)
In the presenting step, the processor:
Based on the point cloud data of the first user, the movement of the body of the first user is drawn from a plurality of viewpoints, and the video in which the first avatars drawn for each viewpoint are arranged is displayed as the first image. The method described in (Summary 1), presenting to 1 users.

(Summary 4)
In the presenting step, the processor:
Along with the first avatar, changing the display mode of the surface position of the body that is not the joint position based on the part of the body for evaluating whether the body movement performed by the first user is ideal, The method according to (Summary 1), wherein the image in which the display mode of the surface position of the body is changed is presented to the first user.

(Summary 5)
A program for operating a computer for training by a first user,
The program causes the processor of the computer to:
a step of sensing movement of the body of the first user as point cloud data;
presenting, to the first user, an image including a first avatar corresponding to the movement of the body of the first user as a set of point cloud data based on the sensing result. death,
In the presenting step,
Along with the first avatar, an auxiliary line is drawn at a position based on the part of the body for evaluating whether or not the body movement performed by the first user is ideal, and the image on which the auxiliary line is drawn is displayed. A program presented to the first user.

(Summary 6)
An information processing device for a first user to train,
The control unit of the information processing device,
a step of sensing movement of the body of the first user as point cloud data;
presenting, to the first user, an image including a first avatar corresponding to the movement of the body of the first user as a set of point cloud data based on the sensing result. death,
In the presenting step,
Along with the first avatar, an auxiliary line is drawn at a position based on the part of the body for evaluating whether or not the body movement performed by the first user is ideal, and the image on which the auxiliary line is drawn is displayed. An information processing device presented to the first user.

(Summary 7)
A method for a first user to train, comprising:
A processor of the terminal of the first user,
sensing body movements of the first user;
presenting to the first user a video including a first avatar corresponding to the body movement of the first user based on the sensed result;
In the presenting step,
Along with the first avatar, an auxiliary line is drawn at a position based on the part of the body for evaluating whether or not the body movement performed by the first user is ideal, and the image on which the auxiliary line is drawn is displayed. presenting to the first user;
The method of drawing the auxiliary line based on a result of estimating the posture of the first user in the first avatar.

(Summary 8)
The method of (Summary 7), wherein, as a result of estimating the posture of the first user, the auxiliary line is drawn based on an estimation result of joint positions of the first user.

(Summary 9)
The method of (Summary 8), wherein the first avatar is presented to the first user without displaying the joint position estimation result.

(Summary 10)
The method of (Summary 8), wherein the body movement of the first user is evaluated based on the first avatar and the auxiliary lines.

(Summary 11)
The method of (Summary 10), wherein the movement of the body of the first user is evaluated according to whether the first avatar and the auxiliary line have collided or come within a certain distance.

(Summary 12)
A program for operating a computer for training by a first user, the program comprising:
to the processor of said computer;
sensing body movements of the first user;
presenting to the first user a video including a first avatar corresponding to the movement of the body of the first user based on the sensed result;
In the presenting step,
Along with the first avatar, an auxiliary line is drawn at a position based on the part of the body for evaluating whether or not the body movement performed by the first user is ideal, and the image on which the auxiliary line is drawn is displayed. presenting to the first user;
A program for drawing the auxiliary line based on a result of estimating the posture of the first user in the first avatar.

(Summary 13)
An information processing device for a first user to train,
The control unit of the information processing device,
sensing body movements of the first user;
presenting to the first user a video including a first avatar corresponding to the body movement of the first user based on the sensed result;
In the presenting step,
Along with the first avatar, an auxiliary line is drawn at a position based on the part of the body for evaluating whether or not the body movement performed by the first user is ideal, and the image on which the auxiliary line is drawn is displayed. presenting to the first user;
An information processing device that draws the auxiliary line based on a result of estimating the posture of the first user in the first avatar.

(Summary 14)
A method for a first user to train, comprising:
A processor of the terminal of the first user,
a step of sensing movement of the body of the first user as point cloud data;
Based on the sensed result, the first user or the first user creates an image including a first avatar corresponding to the movement of the body of the first user as a set of point cloud data. presenting to at least one of a second user's terminal communicating with a terminal of
In the presenting step,
When presenting the image to the second user, by transmitting the sensed result from the terminal of the first user to the terminal of the second user and receiving it at the terminal of the second user, presenting the video to the second user;
Along with the first avatar, an auxiliary line is drawn at a position based on the part of the body for evaluating whether or not the body movement performed by the first user is ideal, and the image on which the auxiliary line is drawn is displayed. presenting to at least one of said first user or said second user.

(Summary 15)
In the presenting step, the second processor:
The type of the auxiliary line to be drawn is specified in accordance with the type of body movement performed by the first user, and the image on which the specified auxiliary line is drawn is displayed by the first user or the second user. 14. The method of Clause 14, presenting to at least one of the users.

(Summary 16)
In the presenting step, the processor:
Based on the point cloud data of the first user, the movement of the body of the first user is drawn from a plurality of viewpoints, and the video in which the first avatars drawn for each viewpoint are arranged is displayed as the first image. 14. The method of Clause 14, wherein presenting to one user and/or to said second user.

(Summary 17)
In the presenting step, the processor:
Along with the first avatar, changing the display mode of the surface position of the body that is not the joint position based on the part of the body for evaluating whether the body movement performed by the first user is ideal, The method according to (Summary 14), wherein the image in which the display mode of the surface position of the body is changed is presented to at least one of the first user and the second user.

(Summary 18)
A program for operating a computer for training by a first user,
The program causes the processor of the computer to:
a step of sensing movement of the body of the first user as point cloud data;
Based on the sensed result, communicating an image including a first avatar corresponding to the body movement of the first user as a set of point cloud data with the first user or the computer. presenting to at least one of said second users of a second user's terminal;
In the presenting step,
When presenting the image to the second user, by transmitting the sensed result from the computer to the terminal of the second user and receiving it at the terminal of the second user, present the video to
Along with the first avatar, an auxiliary line is drawn at a position based on the part of the body for evaluating whether or not the body movement performed by the first user is ideal, and the image on which the auxiliary line is drawn is displayed. A program for presentation to at least one of said first user or said second user.

(Summary 19)
An information processing device for a first user to train,
The control unit of the information processing device,
a step of sensing movement of the body of the first user as point cloud data;
Based on the sensed result, communicating an image including a first avatar corresponding to the body movement of the first user as a set of point cloud data with the first user or the computer. presenting to at least one of said second users of a second user's terminal;
In the presenting step,
When presenting the image to the second user, by transmitting the sensed result from the computer to the terminal of the second user and receiving it at the terminal of the second user, present the video to
Along with the first avatar, an auxiliary line is drawn at a position based on the part of the body for evaluating whether or not the body movement performed by the first user is ideal, and the image on which the auxiliary line is drawn is displayed. An information processing device that is presented to at least one of the first user and the second user.

(3) An information processing device, in which a program that constitutes this system may be installed in advance or may be installed afterward, and such a program may be stored in an external non-temporary storage medium. It may be stored or operated by cloud computing.

(4) A method for training a user, in which a computer functions as an information processing device, and a program that constitutes the system may be pre-installed in the information processing device, or may be installed afterward. Alternatively, such a program may be stored in an external non-temporary storage medium, or may be operated by cloud computing.

(5) In the description of the above embodiments, the processing performed while communicating between each terminal device 10 and the server 20, or between a plurality of terminal devices 10 may be performed by one of the plurality of devices.

(6) In the description of the fifth embodiment above, an example has been described in which the learned model outputs the content to be instructed to the user based on the information based on the user's point cloud data and the learned model.
The contents explained in the fifth embodiment are described below as (Summary 6-1) to (Summary 6-5).

(Summary 6-1)
A method for a first user to train, comprising:
The terminal of the first user stores a learned model based on information based on the point cloud data of one or more users in a storage unit,
The method comprises: a processor of the terminal of the first user;
Identifying instruction content for the first user based on information based on the point cloud data of the first user and the learned model,
and performing the step of presenting the identified instructional content to the first user.

(Summary 6-2)
The trained model is associated with information of a user who obtained information based on the point cloud data, which is learning data for the trained model,
The method comprises: the processor;
In the step of presenting to the first user, based on the information based on the point cloud data of the first user, the information on the first user, and the trained model, to the first user The method according to (Summary 6-1), wherein the instruction content of is specified.

(Summary 6-3)
In the step of presenting to the first user, the processor:
identifying exercise intensity of the first user based on information based on the point cloud data of the first user and the trained model;
Identifying guidance content for the first user based on the identified exercise intensity,
and presenting the identified guidance content to the first user. The method according to (Summary 6-1).

(Summary 6-4)
A program for operating a computer for training by a first user,
The computer causes the storage unit to store a learned model based on information based on the point cloud data of one or more users,
The program causes the processor of the computer to:
Identifying instruction content for the first user based on information based on the point cloud data of the first user and the learned model,
A program causing execution of a step of presenting the identified guidance content to the first user.

(Summary 6-5)
An information processing device for a first user to train,
The information processing device causes a storage unit to store a learned model based on information based on the point cloud data of one or more users,
The control unit of the information processing device,
Identifying instruction content for the first user based on information based on the point cloud data of the first user and the learned model,
An information processing apparatus that executes a step of presenting the identified guidance content to the first user.

(Summary 6-6)
A method for a first user to train, comprising:
The terminal of the first user stores, in a storage unit, a learned model based on information based on results of estimating postures of one or more users, and
The method comprises: a processor of the terminal of the first user;
Identifying instruction content for the first user based on information based on the result of estimating the posture of the first user and the learned model;
and performing the step of presenting the identified instructional content to the first user.

<Appendix>
The items described in the above embodiments will be added below.

(Appendix 1)
A system 1 for user training, wherein a first processor (19A) of a first user's terminal (10A) senses the movement of the first user's body as point cloud data; (19A) transmitting information based on the first user's point cloud data to a second user's terminal (10B); presenting to a second user an image including a first avatar corresponding to the body movement of the first user as a set of point cloud data based on information received from the user's terminal (10A); and a second processor. (19B) accepts the second user's voice input at the second user's terminal (10B), and transmits the second user's voice from the second user's terminal (10B) to the first user's terminal (10A). and outputting to the first user the second user's voice received from the second user's terminal (10B) by the first processor (19A).

(Appendix 2)
In the step of presenting to the second user, the second processor (19B), together with the first avatar, draws auxiliary lines at positions based on body parts for evaluating whether the body movements performed by the first user are ideal. and presenting the second user with an image on which the auxiliary lines are drawn.

(Appendix 3)
In the step of presenting to the second user, the second processor (19B) identifies the type of auxiliary line to be drawn corresponding to the type of body movement performed by the first user, and draws the identified auxiliary line. 3. The system of Clause 2, wherein the video is presented to the second user.

(Appendix 4)
In the step of presenting to the second user, the second processor (19B) draws the body movement of the first user from a plurality of viewpoints based on the point cloud data of the first user, and draws the first user's body movement from each viewpoint. 4. The system according to any one of appendices 1 to 3, wherein a video in which one avatar is arranged is presented to the second user.

(Appendix 5)
In the system 1, the first processor (19A) further executes a step of acquiring a captured image of the first user captured by the camera (160A), and in the step of the first processor (19A) transmitting the point cloud data generates a video including the first avatar by performing rendering based on, transmits the generated video and the acquired photographed image of the first user to the second user's terminal (10B), and presents it to the second user 5. According to any one of appendices 1 to 4, wherein in the step, the second processor (19B) presents the video received from the first user's terminal (10A) and the first user's captured image to the second user. system.

(Appendix 6)
In the step of presenting to the second user, a second processor (19B), together with the first avatar, evaluates whether the body movements performed by the first user are ideal based on body parts, not joint positions. 6. The system according to any one of appendices 1 to 5, wherein the display mode of the body surface position is changed, and the image in which the body surface display mode is changed is presented to the second user.

(Appendix 7)
In the step of transmitting the second user's voice from the second user's terminal (10B) to the first user's terminal (10A), the second processor (19B) responds to the content of the received second user's voice 7. The system according to any one of appendices 1 to 6, wherein the display mode of the video including the first avatar is changed, and the video including the first avatar with the changed display mode is presented to the second user.

(Appendix 8)
In the sensing step, the first processor (19A) identifies a specific evaluation value related to the body movement of the first user based on the point cloud data of the first user, and based on the identified evaluation value, the second 8. A system according to any one of appendices 1 to 7, wherein in the step of transmitting to a user's terminal (10B), a candidate second user for transmission is identified.

(Appendix 9)
In the sensing step, the first processor (19A) identifies a specific evaluation value regarding the body movement of the first user based on the point cloud data of the first user, the first processor (19A), the second processor (19B), or at least one of the processor (29) of the computer (20) capable of communicating with the first user's terminal (10A) and the second user's terminal (10B), based on the evaluation value of the first user , in the step of identifying a third user to train with the first user and presenting it to the second user, a second processor (19B) senses the point cloud data of the first user and the movement of the body of the third user; 8. Any one of appendices 1 to 7, wherein an image including the first avatar and the second avatar corresponding to the movement of the body of the third user is presented to the second user by receiving information based on the point cloud data System as described.

(Appendix 10)
Further, in the system 1, the first processor (19A) receives from the first user input of information about the ideal body of the first user, the first processor (19A), the second processor (19B), or the At least one of the processor (29) of the computer (20) capable of communicating with the terminal (10A) of the first user and the terminal (10B) of the second user receives information on the ideal body of the first user, At least one of guidance content for the first user and candidates for the second user is specified based on the point cloud data of one user, and a first processor (19A) identifies guidance content for the first user. or 10. The system of any of the clauses 1-9, wherein the second user candidates are presented to the first user.

(Appendix 11)
The first processor (19A) performs a step of receiving an input operation specifying a display mode of the first user's information to be presented to the second user, and in the step of presenting to the second user, the second processor (19B) 11. The system of any one of Clauses 1-10, wherein presents the video in response to the first user's designation.

(Appendix 12)
At least one of the first processor (19A), the second processor (19B), or the processor (29) of the computer (20) capable of communicating with the first user's terminal (10A) and the second user's terminal (10B) , defining a virtual space for displaying a first object for displaying a video including a first avatar associated with a first user and a video including a second avatar associated with a third user in the virtual space; , and the first processor (19A) or the second processor (19B) causes the display unit (132) to display an image including the first object and the second object in the virtual space, System as described.

(Appendix 13)
The first processor (19A) stores a trained model based on information based on point cloud data of one or more users in the storage unit (180A). Based on the information based on the point cloud data and the trained model, the guidance content for the first user is provided without transmitting the information based on the point cloud data of the first user to the second user's terminal (10B) 13. The system of any of Clauses 1-12, performing the steps of identifying and presenting the identified instructional content to a first user.

(Appendix 14)
The learned model is associated with the information of the user who acquired the information based on the point cloud data which is the learning data of the learned model, and the first processor (19A) presents the first user with the 14. The system according to appendix 13, wherein the guidance content for the first user is specified based on the information based on the point cloud data of the one user, the information about the first user, and the trained model.

(Appendix 15)
In the step of presenting to the first user, the first processor (19A) identifies the exercise intensity of the first user based on the information based on the point cloud data of the first user and the learned model; 15. The system according to appendix 13 or 14, further comprising: identifying guidance content for the first user based on the identified exercise intensity, and presenting the identified guidance content to the first user.

10 terminal device, 20 server, 30 measurement unit, 31 measurement device, 32 mat, 33 pole, 132 display, 181 user information, 182 measurement item information, 183 measurement history information, 184 friend list, 185 training item information, 281 user information Database, 282 measurement item information, 283 measurement history database, 284 friend list, 285 training item information, 287 measurement result database, 288 evaluation result database, 289 user questionnaire information, 290 evaluation result database, 2601 trainee information database, 2602 trainer information database , 2603 Training History Database

Claims (5)

A method for a first user to train, comprising:
The terminal of the first user stores a learned model based on information based on the point cloud data of one or more users in a storage unit,
The method comprises: a processor of the terminal of the first user;
Identifying instruction content for the first user based on information based on the point cloud data of the first user and the learned model,
and performing the step of presenting the identified instructional content to the first user. The trained model is associated with information of a user who obtained information based on the point cloud data, which is learning data for the trained model,
The method comprises: the processor;
In the step of presenting to the first user, based on the information based on the point cloud data of the first user, the information on the first user, and the trained model, to the first user 2. The method of claim 1, wherein the instructional content of a. In the step of presenting to the first user, the processor:
identifying exercise intensity of the first user based on information based on the point cloud data of the first user and the trained model;
Identifying guidance content for the first user based on the identified exercise intensity,
and presenting the identified instructional content to the first user. A program for operating a computer for training by a first user,
The computer causes the storage unit to store a learned model based on information based on the point cloud data of one or more users,
The program causes the processor of the computer to:
Identifying instruction content for the first user based on information based on the point cloud data of the first user and the learned model,
A program causing execution of a step of presenting the identified guidance content to the first user. An information processing device for a first user to train,
The information processing device causes a storage unit to store a learned model based on information based on the point cloud data of one or more users,
The control unit of the information processing device,
Identifying instruction content for the first user based on information based on the point cloud data of the first user and the learned model,
An information processing apparatus that executes a step of presenting the identified guidance content to the first user.

Images