JP2021027917A - Information processing device, information processing system, and machine learning device - Google Patents

Abstract

To provide a technique which becomes an aid of improving a ratio of persons having exercise habits by supporting a user from standpoints other than a standpoint of improving an exercise technique.SOLUTION: A management device 10 includes a storage part (storage) for storing information for reference to selecting instruction content to be outputted to a user as a behavior related to a user's condition. The user's condition includes motivation information of exercise in the user. The management device 10 includes an acquisition part (a network controller) for acquiring the user's condition the exercising user, a selection part (a processor) for selecting instruction content to be outputted to the user on the basis of the user's condition and the information for reference, and an output part (a display and/or a network controller) for outputting the selected instruction content.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to the output of instruction content related to exercise using the value function learned by reinforcement learning.

Conventionally, various proposals have been made for exercise support to support the improvement of exercise skills of users. For example, Patent Document 1 (Patent No. 6332830) discloses an exercise support system that provides appropriate advice for each instruction point regarding a user's exercise posture and the like.

Further, in Patent Document 2 (Japanese Patent No. 6307457), in order to feed back the determination result of the current exercise state with respect to the reference exercise (correct exercise), the comparison result of the exercise states of a plurality of subjects was provided. A method of outputting a feedback signal is disclosed ([Claim 1], paragraph [0012], etc.). Patent Document 3 (Patent No. 6270115) discloses an exercise support system that sets a reduction rate of the remaining time for each position where sensory information is presented based on the bias of the exercise site or the like ([claim]. Item 1] etc.)). Patent Document 4 (Japanese Unexamined Patent Publication No. 2018-42771) discloses a device for appropriately supporting skill improvement according to the physical strength of the user ([summary], etc.). Patent Document 5 (Japanese Unexamined Patent Publication No. 2019-58285) discloses an activity support method for selecting an activity menu according to an evaluation of athletic ability (paragraph [0027], etc.). In Patent Document 6 (Japanese Unexamined Patent Publication No. 2019-63558), a motivational message is provided according to a target state calculated based on an activity point metric indicating the amount of physical activity performed by the user during the first time period. A device for transmitting to a user is disclosed ([Claim 1], etc.).

On the other hand, although health consciousness is increasing, the percentage of people who do not have exercise habits remains constant. This includes people who once had an exercise habit but could not continue. Therefore, it is required to support the continuation of exercise.

Non-Patent Document 1 (Hiroaki Uechi, Sport Psychology Research 2013, Vol. 40, No. 2, pp. 165-172, "Relationship between orientation toward ideal self and participation status in exercise / sports scenes", [online], Heisei September 30, 2013, Japan Sports Psychology Association, [Search June 19, 2019], Internet <https://www.jstage.jst.go.jp/article/ In jjspopsy / 40/2 / 40_2013-1218 / _pdf>), ideal self-oriented strength is associated with participating in exercise (described herein as a concept that includes "sports"). It is also suggested that a sense of unity with others (for example, an instructor) that is important to the self forms the ideal self. Therefore, in supporting the user to continue exercising, it is considered important to give the user a sense of unity with others, which is important for increasing the proportion of people who have exercise habits.

Japanese Patent No. 6332830 Japanese Patent No. 6307457 Japanese Patent No. 6270115 JP-A-2018-42771 JP-A-2019-58285 JP-A-2019-63558

Hiroaki Uechi, Sport Psychology Research 2013 Vol. 40, No. 2, pp. 165-172, "Relationship between Orientation to Ideal Self and Participation Status in Exercise / Sports Scenes", [online], 2013 (2013) ) September 30, Japan Society of Sport Psychology, [Search June 19, 2019], Internet <https://www.jstage.jst.go.jp/article/jjspopsy/40/2 / 40_2013-1218 / _pdf>

Considering the suggestion by Non-Patent Document 1, it is considered that the user cannot be promoted to continue the exercise only by the support from the viewpoint of improving the exercise technique. From this, it is considered that the exercise support proposed in the past is not sufficient in terms of increasing the proportion of people who have exercise habits.

This disclosure was conceived in view of such circumstances, and its purpose is to help increase the proportion of people with exercise habits by supporting users from a viewpoint other than the viewpoint of improving exercise skills. Is to provide the technology.

According to a certain aspect of the present disclosure, a storage unit for storing reference information for selecting an instruction content output to the user as an action related to the user's state is provided, and the user's state is the movement of the user. An acquisition unit that includes continuation motivation information and acquires the user's status for the exercising user, and a selection unit that selects instruction content to be output to the user based on the user's status and reference information. An information processing apparatus is provided that includes an output unit that outputs the selected instruction content.

Preferably, the reference information includes a value function learned by reinforcement learning to select the instruction content output to the user as an action related to the user's state, and the selection unit is about the user in exercise. Among the actions related to the user's state, the action with the maximum value function value is selected, and the action with the maximum value function value is the action in which the user's state as a reward becomes the highest at the end of exercise. is there.

Preferably, the continuation motivation information includes a sense of unity information indicating a sense of unity between the instructor and the user.

Preferably, the sense of unity information includes information based on the smiles of the instructor and the user.

Preferably, the sense of unity information represents the ratio at which the facial expressions of the instructor and the user match at a plurality of timings.

Preferably, the user's condition further includes the rate at which the exercise load matches between the instructor and the user.

Preferably, the exercise load is based on the respective pulse waves of the instructor and the user.

Preferably, the user's state further includes the rate at which the motor state matches between the instructor and the user.

Preferably, the state of motion is based on at least one of acceleration and angular velocity measured for each body of the instructor and the user.

Preferably, the output unit outputs an image of the instructor, and the motion state is at least one of the output acceleration and angular velocity related to the movement of the instructor's body, and the acceleration and angular velocity measured for the user's body. Based on at least one of them.

Preferably, the continuation motivation information includes information that represents a difference from the ideal value of the user's state.

Preferably, the reference information represents an ideal value for the user's state, and the selection unit selects the instruction content to be output to the user based on the difference between the user's state and the reference information. To do.

Preferably, the user's condition includes the degree of smile of the user.

Preferably, the user's state includes at least one of the user's biometric information and motor state.

Preferably, the plurality of instructions corresponding to each of the plurality of actions includes at least one of the advice content or the exercise intensity.

According to other aspects of the present disclosure, an information processing device that selects an instruction content output to the user as an action related to the user's state and a value function of the instruction content output to the user are learned by enhanced learning. It is an information processing system including a machine learning device that performs, and the information processing device stores a value function learned by enhanced learning in order to select an instruction content output to the user as an action related to the user's state. The user's state includes the user's continuation motivation information, and the information processing device includes the acquisition unit for acquiring the continuation motivation information about the exercising user and the continuation motivation information about the exercising user. Among such actions, the action having the maximum value function value includes a selection unit that selects the action having the maximum value function value and an output unit that outputs the instruction content corresponding to the selected action. , The action that the continuous motivation information, which is a reward, becomes the highest at the end of the exercise, and the machine learning device sets the state acquisition unit that acquires the user's state and the user's state before the output of the instruction content as the state, and gives the user An information processing system is provided that includes a value function update unit that updates the value function by using the instruction content output to the user as an action and the user's state after the instruction content is output as a reward.

According to still another aspect of the present disclosure, it is a machine learning device that learns the value function of the instruction content output to the user as an action related to the user's state by reinforcement learning, and is a user including the user's continuous motivation information. The state acquisition unit that acquires the state of, and the state of the user before the output of the instruction content is used as the state, the instruction content output to the user is used as the action, and the user state after the instruction content is output is used as the reward. , A machine learning device is provided that includes a value function update unit that updates the value function.

According to the present disclosure, the instruction content to be output is determined based on the information on the user's willingness to continue exercising. Thereby, the present disclosure realizes support for the continuation of exercise for the user, and as a result, contributes to an increase in the proportion of people who have exercise habits.

It is a figure for demonstrating the structure of an exercise support system. It is a figure which shows an example of the hardware composition of management apparatus 10. It is a block diagram which shows the hardware composition of an instructor terminal 900. It is a block diagram which shows the hardware composition of a user terminal 910. It is a figure which conceptually shows the model of reinforcement learning in 1st Embodiment. It is a figure which shows an example of the functional structure of management apparatus 10. It is a figure for demonstrating an example of extraction of a face region in an image acquired from a camera 920. It is a figure which shows an example of the functional structure of the state recognition part 610. It is a flowchart of the process for outputting the instruction content using the estimation model. It is a figure which shows another example of the functional structure of the state recognition part 610. It is a figure which shows still another example of the functional structure of the state recognition part 610. It is a figure which shows still another example of the functional structure of the state recognition part 610. It is a figure which shows an example of the hardware composition of management apparatus 10A. It is a figure for demonstrating a specific example of a smile reference information 16P. It is a figure for demonstrating a specific example of heart rate reference information 16Q. It is a figure for demonstrating the specific example of the motion coincidence degree reference information 16R. It is a figure for demonstrating the specific example of the instruction content (advice and / or exercise intensity) to be output in the 2nd Embodiment.

Hereinafter, an embodiment of the exercise support system will be described as an example of realization of the information processing system according to the present disclosure with reference to the drawings. In the following description, the same parts and components are designated by the same reference numerals. Their names and functions are the same. Therefore, the detailed description of these will not be repeated.

<< First Embodiment >>
[1. Outline of the structure of the exercise support system]
FIG. 1 is a diagram for schematically explaining the configuration of the exercise support system. In the example of FIG. 1, in the exercise support system, two users 810 are exercising under the guidance of one instructor 800. The management device 10 outputs the instruction content to the user 810. The management device 10 uses the information representing the sense of unity between the instructor 800 and each user 810 when outputting the instruction content as state information, the output instruction content as action information, and after the instruction content is output. The output instruction content is determined by using the value function of each instruction content that has been strengthened and learned by using the information expressing the above-mentioned sense of unity as a reward. "Information expressing a sense of unity" is an example of continuation motivation information in the present embodiment.

The instructor 800 is equipped with an instructor terminal 900. Each user 810 is equipped with a user terminal 910. In one implementation example, the management device 10 notifies the instructor terminal 900 of the instruction content. The instructor terminal 900 displays the notified instruction content. That is, in the output of the instruction content by the management device 10, not only the management device 10 itself directly displays the instruction content or the like, but also the management device 10 prompts other devices to display the instruction content or the like. It also means to output.

(Instruction content)
An example of the instruction content is advice to be output regarding exercise. An example of advice is a message of praise such as "It's perfect!", "OK!", "It's good!", "It's done!" Another example of advice is a message of encouragement, such as "Do your best!" Or "Move bigger!". When the instructor terminal 900 displays the advice to be output regarding the exercise, the instructor 800 utters a voice according to the displayed advice.

The instructor 800 may perform an action in accordance with the advice instead of or in addition to making a voice in accordance with the advice. For example, in connection with the advice "OK!", The instructor 800 may make a gesture indicating that the user 810's operation is correct (for example, a hand indicating an "OK" sign). In connection with the advice "Do your best!", The instructor 800 may make gestures (eg, swing his arms) to encourage the user 810.

Another example of the instruction is exercise intensity. When the instructor terminal 900 displays the exercise intensity, the instructor 800 issues a voice instructing the user 810 to change the exercise intensity when the current exercise intensity and the displayed exercise intensity are different. In one embodiment, one example of increased exercise intensity is that stepping is transformed into jumping. An example of reduced exercise intensity is that jumping is transformed into stepping.

In another embodiment, one example of increasing exercise intensity is increasing the speed of movement. In this case, the instructor 800 may make voices such as "faster!" And "bigger!". An example of reducing exercise intensity is to reduce the speed of movement. In this case, the instructor 800 may make a voice such as "It's okay to be slower!".

If the current exercise intensity is different from the displayed exercise intensity, instead of or in addition to speaking, the instructor 800 allows the user 810 to exercise according to the displayed exercise intensity. You may make a gesture. In one embodiment, the instructor 800 operates according to the operation according to the changed intensity. More specifically, the instructor 800 may show the user 810 a sample operation sequence in which the stepped portion is changed to a jump. The instructor 800 may show the user 810 a faster or larger or slower movement as a sample motion sequence.

(Output method of instruction contents)
The management device 10 may output the instruction content from the speaker 950 instead of outputting the instruction content to the instructor terminal 900, or in addition to outputting the instruction content to the instructor terminal 900.

When the instruction content is advice, the management device 10 outputs the voice corresponding to the advice from the speaker 950.

When the instruction content is exercise intensity, the management device 10 outputs a voice according to the changed exercise intensity from the speaker 950. An example of an increase in exercise intensity may be an increase in exercise speed, and an example of a decrease in exercise intensity may be a decrease in exercise speed. According to this example, the management device 10 may adjust the speed of the music output from the speaker 950 so that the exercise can be performed at a speed according to the changed exercise intensity. The adjustment of the music speed may be the adjustment of the reproduction speed of the music output from the speaker 950. The adjustment of the speed of the music may be a change of the song output from the speaker 950. When "high", "medium", and "low" are set as the exercise intensity, the management device 10 has a song corresponding to the exercise intensity "high", a song corresponding to the exercise intensity "medium", and an exercise intensity "medium". A song corresponding to "low" is stored, and the management device 10 may output a song corresponding to the changed exercise intensity from the speaker 950. The management device 10 may adjust the volume of the music output from the speaker 950 so that the exercise can be performed at an intensity according to the exercise intensity after the change. If the exercise intensity increases, the volume may be increased.

The output of the instruction content from the instructor terminal 900 may be voice or vibration instead of or in addition to the display. The instructor terminal 900 may include an oscillator for outputting the instruction content as a vibration pattern.

(Estimation of instruction content to be output)
The management device 10 estimates the instruction content to be output to the user 810 by using the estimation model. The estimation model includes a value function Q as described below. The management device 10 acquires the state of the user 810 and outputs the instruction content that maximizes the value of the value function for the acquired state. "The value of the value function is maximized" means that the user 810 feels a sense of unity with the instructor 800 at the highest level.

The state of the user 810 is information indicating how much the user 810 is motivated to continue exercising (that is, motivation to continue exercising after the end of this exercise) (hereinafter, "continuation motivation information"). Also referred to as). As an example of the continuation motivation information, there is information indicating how much the user 810 feels "sense of unity" with respect to the instructor 800 (hereinafter, also referred to as "sense of unity information").

The exercise support system may utilize images taken by the camera 920 to generate sense of unity information. The camera 920 captures an area where the user 810 is moving (eg, a workout room). The camera 920 may be a 360 degree camera. As a result, the camera 920 can capture not only the user 810 but also the instructor 800 facing the user 810. In one embodiment, the management device 10 uses the region of the instructor 800 and the region of the user 810 of the image captured by the camera 920 to generate unity information. The method of generating the sense of unity information will be described later.

[2. Management device configuration]
FIG. 2 is a diagram showing an example of the hardware configuration of the management device 10. With reference to FIG. 2, the management device 10 includes a processor 11, a memory 12, a display 13, an input unit 14, a network controller 15, and a storage 16 as main hardware elements.

The display 13 displays information necessary for processing by the management device 10. The display 13 is composed of, for example, a liquid crystal display and / or an organic EL (organic electro-luminescence) display.

The processor 11 is an arithmetic unit that executes processing necessary for realizing the management device 10 by executing various programs as described later. The processor 11 includes, for example, one or a plurality of CPUs (Central Processing Units). And / or composed of GPU (Graphics Processing Unit) and the like. A CPU or GPU having a plurality of cores may be used. It is preferable that the management device 10 employs a GPU or the like suitable for the learning process for generating the trained model.

The memory 12 provides a storage area for temporarily storing a program code, a work memory, and the like when the processor 11 executes a program. As the memory 12, for example, a volatile memory device such as DRAM (Dynamic Random Access Memory) and / or SRAM (Static Random Access Memory) may be used.

The network controller 15 realizes transmission / reception of data between an arbitrary communication device including the instructor terminal 900 and the user terminal 910 and the management device 10 via the local network. The network controller 15 may be compatible with any communication method such as Ethernet (registered trademark), wireless LAN (Local Area Network), and Bluetooth (registered trademark). The management device 10 may communicate with the camera 920, the instructor terminal 900, and / or the user terminal 910 via the network controller 15.

The management device 10 may communicate with the instructor terminal 900 and the user terminal 910 by wire or wirelessly.

The storage 16 is for generating a learning data set 16F from data such as an OS (Operating System) 16D executed by the processor 11, an application program 16C for realizing a functional configuration as described later, and image information 16H. The preprocessing program 16B, the learning program 16A for learning the value function 16E using the learning data set 16F, and the like are stored.

The image information 16H is, for example, an image taken by a camera 920. The acquisition and preprocessing of the image information 16H will be described later.

As the storage 16, for example, a hard disk and / or a non-volatile memory device such as an SSD (Solid State Drive) may be used.

As a part of the library or functional module required when executing the application program 16C, the preprocessing program 16B, and the learning program 16A on the processor 11, the library or functional module provided as standard by the OS 16D may be used. In this case, each of the application program 16C, the preprocessing program 16B, and the learning program 16A does not include all the program modules necessary to realize the corresponding functions, but under the execution environment of the OS 16D. By installing it in, it is possible to realize the functional configuration described later. Therefore, even a program that does not include some such libraries or functional modules may be included in the technical scope of the present invention.

The application program 16C, the preprocessing program 16B, and the learning program 16A include an optical recording medium such as an optical disk, a semiconductor recording medium such as a flash memory, a magnetic recording medium such as a hard disk or a storage tape, and an MO (Magneto-Optical disk). It may be stored in a non-transient recording medium such as a magneto-optical recording medium, distributed, and installed in the storage 16. Therefore, the subject of the present invention may be the program itself installed in the storage 16 or the like, or a recording medium in which a program for realizing a function or process according to the present embodiment is stored.

Alternatively, the program for realizing the management device 10 may be distributed not only by being stored in an arbitrary recording medium as described above and distributed, but also by downloading from a server device or the like via the Internet or an intranet. ..

The input unit 14 accepts various input operations. As the input unit 14, for example, a keyboard, a mouse, a touch panel, a pen, or the like may be used.

FIG. 2 shows a configuration example in which the general-purpose computer (processor 11) realizes the management device 10 by executing the application program 16C, the preprocessing program 16B, and the learning program 16A, but in order to realize the management device 10. All or part of the required functions may be realized using hard-wired circuits such as integrated circuits. For example, it may be realized by using an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

(Integrated configuration / Cloud configuration)
FIG. 2 exemplifies a configuration in which a processor is provided to realize a learning function and an estimation function using an estimation model as a typical example, but the present invention is not limited to this, and functions necessary for realizing an exercise support system are not limited to this. May be adopted in an integrated configuration that realizes the above with fewer arithmetic units.

As an example of such an integrated configuration, a system configuration in which a learning function and / or an estimation function is substantially realized in the management device 10 and a tablet terminal displaying a user interface is used as a so-called thin client. May be provided.

Further, the management device 10 may also be realized by a plurality of computers connected via a computer network in explicit or implicit cooperation. When a plurality of computers are linked, some of the computers may be unspecified computers on the network, so-called cloud computers.

A person skilled in the art will be able to realize an exercise support system according to the present embodiment by appropriately using a technique suitable for the times when the present invention is implemented.

[3. Instructor terminal configuration]
FIG. 3 is a block diagram showing a hardware configuration of the instructor terminal 900. The instructor terminal 900 is realized by, for example, a wristwatch-type device.

With reference to FIG. 3, the instructor terminal 900 has, as main components, a CPU 901 for executing various processes, a memory 902 for storing a program and / or data executed by the CPU 901, and a triaxial direction. Accelerometer 903 that can detect the acceleration of the device, angular velocity sensor 904 that can detect the angular velocity around each of the three axes, and a communication interface (I / F) 905 for communicating with an external device such as the management device 10. It includes a storage battery 906 that supplies power to various components in the instructor terminal 900, a pulse wave sensor 907, a display 908 composed of a liquid crystal display device and the like, and a speaker 909.

[4. User terminal configuration]
FIG. 4 is a block diagram showing a hardware configuration of the user terminal 910. The user terminal 910 is realized by, for example, a wristwatch-type device.

With reference to FIG. 4, the user terminal 910 has, as main components, a CPU 911 for executing various processes, a memory 912 for storing programs and / or data executed by the CPU 911, and a three-axis direction. Accelerometer 913 that can detect the acceleration of the device, angular velocity sensor 914 that can detect the angular velocity around each of the three axes, communication I / F 915 for communicating with an external device such as the management device 10, and the user terminal 910. Includes a storage battery 916 for supplying power to various components of the above, a pulse wave sensor 917, a display 918 composed of a liquid crystal display device and the like, and a speaker 919.

[5. Machine learning]
The management device 10 executes reinforcement learning using the sense of unity information during exercise and the sense of unity information at the end of exercise as learning of the value function. Any learning method can be adopted as reinforcement learning, but in the following description, the case of using Q-learning will be described as an example.

Q-learning is a learning method for learning the value obtained when an action a is given to a certain state s. The value obtained is called the Q value and can be expressed by Q (s, a).

(State, behavior, agent, reward)
FIG. 5 is a diagram conceptually showing a model of reinforcement learning in the present embodiment.

In the reinforcement learning of the present embodiment, the state s is the state of the user during exercise (for example, information on the sense of unity during exercise).

An example of unity information is the smile synchronization rate. The smile synchronization rate is the rate at which smiles occur simultaneously between the instructor and the user within a given time period. For this ratio, for example, N images taken by the instructor and the user acquired at regular intervals in a given period are acquired, and the instructor's face area and the user's face area are extracted in each image. , It can be determined as to whether or not each of the instructor and the user is smiling, and it can be calculated as a ratio to the total number of images (N images) in which smiles are commonly generated. The extraction of the facial region in the image and the determination of the facial expression (whether or not the person is smiling) can be realized by a known technique realized as, for example, OKAO (registered trademark) Vision (OMRON Corporation).

For the determination of whether or not a person has a smile, for example, a predetermined threshold value may be used for a number indicating the degree of smile. That is, when the degree of the smile measured on the face image of a person is equal to or higher than the threshold value, it may be determined that the person is a smile, and the smile measured on the face image of a person If the degree is below the threshold value, it may be determined that the person is not smiling.

The action a is an output of an instruction content to the user.

Here, the number of Q (s, a) is equal to the number of combinations of the state s and the action a. Therefore, when the number that the state s can take and the number that the action a can take become enormous, the number of Q (s, a) also becomes enormous, and it becomes difficult to obtain all Q (s, a) by Q learning. .. Therefore, a method called DQN (Deep Q Network), which uses deep learning to express Q (s, a), can be adopted. By training the neural network using DQN, a model in which the Q value is output by inputting the state s and the action a can be obtained.

The agent that selects the action a is an exercise program that outputs the instruction content. The exercise program is, for example, an application program 16C (FIG. 2) executed by the management device 10.

The reward is the state of the user (for example, sense of unity information) for which the instruction content is output.

(Update formula of value function)
At the time of first starting Q-learning, the correct value of the value function Q (s, a) is unknown for the combination of the state s and the action a. Therefore, the agent selects various actions a under a certain state s, and selects a better action based on the reward given to the action a at that time, so that the correct value function Q (s, Learn a).

Because we want to maximize the sum of the rewards future, finally Q (s, a) = E aims to ^{[Σ (γ t) r t} ] become so. Here E [] denotes the expected value, t is the time, parameter γ is called the discount rate to be described later, is r _t compensation at time t, sigma is the sum by the time t. The expected value in this equation is the expected value when the state changes according to the optimum behavior.

However, since it is unclear what the optimal behavior is in the process of Q-learning, reinforcement learning is carried out while searching by performing various behaviors. Such an update formula of the value function Q (s, a) can be expressed by, for example, the following formula (1).

Equation (1) as a result of the trial _{a t,} based on the reward _{r t + 1,} which has been returned, action in the state _{s t a t} of value function _{Q (s t, a} t) represents a way to update the.

This update equation, behavior in state _{s t a t} the value function _{Q (s} t, _{a t)} than, action _{a t} value of the best behavior in the next state _{s t + 1} by _{max a Q (s t + 1} , a) if is larger _{in, Q (s t, a t} ) is increased and smaller _{Conversely, Q (s t, a t} ) have been shown to reduce the. That is, it brings the value of one action in one state closer to the value of the best action in the next state. However, the difference depends on the discount rate γ and the reward rt _{+ 1} , but basically, the value of the best action in a certain state propagates to the value of the action in the previous state. It is a mechanism to go.

Here, in Q-learning, there is a method of creating a table of Q (s, a) for all state-behavior pairs (s, a) and performing learning. However, the number of states is too large to obtain the Q (s, a) values of all the state-behavior pairs, and it may take a long time for Q-learning to converge.

Therefore, a known technique called DQN (Deep Q-Network) may be used. Specifically, the value function Q is constructed by using an appropriate neural network, and the value function Q (s, a) is approximated by approximating the value function Q with an appropriate neural network by adjusting the parameters of the neural network. The value of may be calculated.

By using DQN, the time required for Q-learning to converge can be shortened. For DQN, known techniques can be referred to. As a known technique, a known document (for example, "Human-level control through deep reinforcement learning", by Volodymyr Mnih1, URL: http://files.davidqiu.com/research/nature14236.pdf) can be referred to.

The Q-learning described above is performed by the management device 10.

Specifically, the management device 10 learns a value function Q that sets the sense of unity information of the user during exercise as the state s and selects the instruction content output to the user as the action a related to the state s.

The management device 10 observes the information s and determines the action a. After executing the action a (outputting the instruction content), the management device 10 acquires the user's sense of unity information as a reward. The management device 10 searches for the optimum action a by trial and error so that the total reward is maximized at the end of the exercise for a given time period, for example. Larger rewards mean a higher sense of unity. That is, the higher the degree of empathy between the instructor and the user, the higher the value of the sense of unity information. By doing so, the management device 10 can select the optimum action a (that is, the optimum instruction content) for the state s acquired by executing the exercise program.

As described above, by selecting the action a that maximizes the value of the value function Q among the actions a related to a certain state s, based on the value function Q learned by the management device 10. , It becomes possible to select the action a (that is, the instruction content) that enhances the sense of unity between the instructor and the user at the end of the exercise.

[6. Management device function configuration]
FIG. 6 is a diagram showing an example of a functional configuration of the management device 10. The management device 10 includes a state recognition unit 610, a current state of unity estimation unit 620, a final unity estimation unit 630, a value function update unit 640, an action determination unit 650, and an instruction output unit 660. The current sense of unity estimation unit 620, the final sense of unity estimation unit 630, the value function update unit 640, and the action determination unit 650 are components of the estimation model (reinforcement learning model).

The state recognition unit 610 generates information for the estimation model to acquire the state s. In one implementation example, the state recognition unit 610 acquires an image from the camera 920 and extracts the face area of the instructor and the face area of the user from the image.

FIG. 7 is a diagram for explaining an example of extracting a face region in an image acquired from the camera 920. FIG. 7 shows image 9201 as an example of an image acquired from camera 920. Image 9201 includes a first portion 9201A that includes an instructor 800 and a second portion 9201B that includes two users 810. In the first portion 9201A, the region AR11 can be identified as the face region of the instructor 800. In the second portion 9201B, the area AR12 may be identified as the face area of the left user of the two users 810, and the area AR13 may be identified as the face area of the right user of the two users 810. .. For example, a known pattern recognition technique can be adopted for extracting the face region.

The state recognition unit 610 further determines whether or not the instructor is smiling based on the image of the instructor's face area, and determines whether or not the user is smiling based on the image of the user's face area. These determination results are output to the current state of unity estimation unit 620.

In one implementation example, the state recognition unit 610 can be realized by the processor 11 executing the preprocessing program 16B.

FIG. 8 is a diagram showing an example of the functional configuration of the state recognition unit 610. The state recognition unit 610 includes an area extraction module 611 that extracts each face area of the instructor and the user. In one embodiment, the region extraction module 611 may specify whether the recognized face region belongs to the instructor or the user according to a given condition. An example of a given condition is a face image of each instructor and each user stored in advance in the storage 16 or the like. The region extraction module 611 identifies the face region based on pattern recognition, and then features the facial region (for example, the ratio between the distance between the eyes (distance) and the distance from the center point of both eyes to the center of gravity of the nose). It may be specified whether the face area corresponds to the instructor or the user by comparing the face area with the feature amount acquired from the face image stored in advance.

The state recognition unit 610 includes a facial expression determination module 612. The facial expression determination module 612 determines whether or not the facial expression indicated by each face region is a smile.

According to the configuration shown in FIG. 8, the state recognition unit 610 displays data indicating whether or not the instructor is smiling and data indicating whether or not the user is smiling in the image acquired from the camera 920. Can be output.

Returning to FIG. 6, the current state of unity estimation unit 620 calculates the unity information of each user during exercise using the data output from the state recognition unit 610.

An example of information on the sense of unity during exercise is the “smile synchronization rate” during exercise. In one implementation example, the state recognition unit 610 asks whether the instructor is smiling for each of the images at regular intervals from the start of exercise (or from the time when a specific time elapses from the start of exercise) to the elapse of a given period. The data indicating whether or not the user is smiling and the data indicating whether or not the user is smiling are output to the current state unity estimation unit 620. The current state of unity estimation unit 620 uses these data to calculate the “smile synchronization rate” during exercise.

For example, the state recognition unit 610 acquires a total of six facial images of the instructor and the user every second for 5 seconds at regular intervals after the start of exercise, and the instructor smiles at each of these images. Data indicating whether or not there is, and data indicating whether or not the user is smiling are generated. The face image of both the instructor and the user may be included in one captured image, or the face image of the instructor and the face image of the user may be included in separate captured images. Then, these data are output to the current state unity estimation unit 620 as data at time t. The current state of unity estimation unit 620 uses these data to calculate the “smile synchronization rate” during exercise (the state “smile synchronization rate” at _{time t).}

The value function update unit 640 updates the value function in the estimation model by performing reinforcement learning. In one implementation example, the value function update unit 640 is realized by the processor 11 executing the learning program 16A.

More specifically, the value function update unit 640 has a "smile synchronization rate" at time t, an instruction content output at time t, and a "smile synchronization rate" (time t + 1) according to the output instruction content. Update the value function using "Smile Synchro Rate"). A data set for learning is associated with the "smile synchronization rate" at time t calculated by the current state of unity estimation unit 620 and the instruction content (behavior) output at time t even determined by the action determination unit 650 described later. May be stored in the storage 16.

The action determination unit 650 determines the instruction content to be output at time t. More specifically, the action determination unit 650 determines the instruction content that maximizes the value of the value function with respect to the "smile synchronization rate" (state) at time t by using the Markov determination process as the above instruction content. To do. Then, the action determination unit 650 outputs the information for specifying the determined instruction content to the instruction output unit 660. In one implementation example, the action decision unit 650 is realized by the processor 11 executing the application program 16C.

The instruction output unit 660 gives an instruction to output the instruction content from the action determination unit 650 to the instructor terminal 900. In one implementation example, the action determination unit 650 may encode the elements (for example, advice, exercise intensity, or a combination thereof) that constitute each of the plurality of instruction contents. In this case, the instruction output unit 660 converts the code output from the action determination unit 650 (the code that identifies the element constituting the instruction content) into a specific instruction content, and instructs the output of the converted instruction content. May be given to the instructor terminal 900.

[7. Guidance content estimation]
FIG. 9 is a flowchart of processing for outputting the instruction content using the estimation model. Hereinafter, the output of the instruction content will be described with reference to FIG.

In step S100, the state recognition unit 610 acquires the start condition of the exercise, and applies the start condition to the exercise program to acquire the condition at the start of the exercise.

An example of a condition at the start of exercise is a user's characteristics (attributes such as gender and / or age, personality, etc.). When a plurality of users exercise under one instructor, a characteristic indicating a majority of the characteristics of the plurality of users may be selected as the condition at the start of the exercise.

Another example of an exercise start condition is the user's vital data (eg, heart rate) at the start of exercise. When a plurality of users exercise under one instructor, the condition at the start of the exercise may be an average value of the characteristics of the plurality of users, or may be a maximum value or a minimum value. Yet another example of the conditions at the start of exercise is a combination of the characteristics and vital data described above.

The state recognition unit 610 selects the content of the exercise at the start of the exercise according to the conditions acquired as described above. The content of the exercise selected is, for example, exercise intensity. In the present embodiment, three stages of "high", "medium", and "low" may be set as the exercise intensity. The state recognition unit 610 selects one of the three stages of "high", "medium", and "low" as the content of the exercise at the start according to the conditions acquired as described above.

For example, if the age of the user is 35 years or younger, the state recognition unit 610 may select “high” exercise intensity as the content of the exercise at the start. When the age of the user is included in the range of 36 to 50 years old, the state recognition unit 610 may select the exercise intensity “medium” as the content of the exercise at the start. When the age of the user is 51 years or older, the state recognition unit 610 may select "low" exercise intensity as the content of the exercise at the start.

In step S102, the state recognition unit 610 performs the exercise with the content selected in step S100. When a certain time elapses from the start of the exercise, or when a certain time elapses from the time when the control in step S104 was executed last time, the state recognition unit 610 advances the control to step S104.

In step S104, the state recognition unit 610 determines whether or not a given period (for example, 15 minutes) has elapsed from the start of the exercise. When the state recognition unit 510 determines that the given period has not yet elapsed from the start of the exercise (NO in step S104), the state recognition unit 510 advances control to step S106, and the given period has elapsed from the start of the exercise. If it is determined that there is (YES in step S104), the control proceeds to step S112.

That is, in the process of FIG. 9, step S104 is performed at regular time intervals (for example, 1 minute) during exercise for a given period (for example, 15 minutes).

In step S106, the state recognition unit 610 generates data for calculating the sense of unity information during exercise. When the "smile synchronization rate" is adopted as the sense of unity information, the data generated in step S106 is the determination result of whether or not the instructor is smiling in the image acquired from the camera 920, and the user smiles. Includes the judgment result of whether or not.

In step S108, the current state of unity estimation unit 620 calculates the sense of unity information of the user during exercise using the data generated in step S106. The unity information of the users during exercise is the "smile synchronization rate" for each user during exercise. In step S108 this time, the "smile synchronization rate" at time t is generated. As a result, the time information of the "smile synchronization rate" generated in step S108 prior to this is changed. For example, the "smile synchronization rate" at time t generated in the previous step S108 is changed to the "smile synchronization rate" at time t-1.

In step S110, the action determination unit 650 determines and determines the instruction content that maximizes the value of the value function as the instruction content at time t with respect to the "smile synchronization rate" at time t calculated in step S108. The instruction content at the time t is output to the instruction output unit 660. The instruction output unit 660 instructs the instructor terminal 900 to output the instruction content. After that, control is returned to step S102.

On the other hand, in step S112, the value function update unit 640 stores a combination of the “smile synchronization rate” and the instruction content at each time in the storage 16 as a learning data set. After that, the process of FIG. 9 ends. If the management device 10 does not need to collect the data set used for reinforcement learning, step S112 may be omitted.

According to the process of FIG. 9 described above, the value function updated by reinforcement learning is used to output the instruction content at time t at which the value of the value function is maximum for the “smile synchronization rate” at time t. Will be done.

Further, according to the process of FIG. 9, the “smile synchronization rate” (state) and the instruction content (behavior) used for the reinforcement learning of the value function are collected at each time. The "smile synchronization rate" at time t can be used as a reward for the action at time t-1.

[Modification example]
(Number of users)
In the above-described embodiment, two users exercise under one instructor, but the number of instructors and users in the exercise is not limited thereto. For example, there may be a plurality of instructors, or there may be one user. When multiple users exercise under the same instructor, the "state", "behavior", and "reward" data used to update the value function are generated independently for each user. It may be generated by combining the data generated for each of a plurality of users.

(Virtual instructor)
In the above-described embodiment, as information on facial expressions of the instructor and the user, information on whether or not the smile is determined from the face area of each image is adopted.

The management device 10 may display the virtual instructor on the display 13 instead of the instructor 800 by executing the exercise program. In this case, the management device 10 may acquire information on the facial expression of the instructor from the data used for display control of the virtual instructor to be displayed on the display 13. Further, the management device 10 may output the instruction content by operating the virtual instructor according to the instruction content. That is, the management device 10 may have the virtual instructor say a message of advice content according to the instruction content, or perform an operation indicating exercise intensity according to the instruction content. Multiple virtual instructors may be displayed in one exercise class. More specifically, in the workout room, the first virtual instructor may be displayed on the front right side and the second virtual instructor may be displayed on the front left side. In this case, the instruction content may always be output from both of the two virtual instructors, or the instruction content may be output alternately from the two virtual instructors.

(Physical sense of unity)
In the present embodiment, the sense of unity information indicates how much the user feels a sense of unity with the instructor. In this sense, the management device 10 can select the instruction content so that the user feels a greater sense of mental unity with the instructor at the end of the exercise by using the value function. The value function may specify the instruction content so that the user can feel a greater physical sense of unity in addition to the mental sense of unity to the instructor at the end of the exercise. For this reason, the "user state" used as the "state" and "reward" in the update of the value function may include information representing a physical sense of unity in addition to the smile synchronization rate.

<Synchronization rate of exercise load>
An example of information that expresses a sense of physical unity is the synchronization rate of exercise load. The synchro rate of the exercise load represents how close the load in the user during exercise is to the load in the instructor during exercise. The state recognition unit 610 calculates, for example, a value representing each exercise load (for example, RR distance (R wave appearance interval)) based on the measurement results of pulse waves of the instructor and the user for a certain period of time. Then, it is output to the current state of unity estimation unit 620. The current state of unity estimation unit 620 derives the difference between the exercise load of the instructor and the exercise load of the user as the synchronization rate of the exercise load.

FIG. 10 is a diagram showing another example of the functional configuration of the state recognition unit 610. In the example of FIG. 10, the state recognition unit 610 further includes signal processing modules 621 and 622. The signal processing module 621 acquires the measurement result 1001 of the pulse wave of the instructor, derives the RR distance by processing the measurement result by a known method, and estimates the derived RR distance as a sense of unity at present. Output to unit 620. The signal processing module 622 acquires the measurement result 1002 of the user's pulse wave, derives the RR distance by processing the measurement result by a known method, and estimates the derived RR distance as a sense of unity at present. Output to unit 620.

The measurement result 1001 is transmitted from, for example, the pulse wave sensor 907 to the management device 10. The measurement result 1002 is transmitted from, for example, the pulse wave sensor 917 to the management device 10.

In the example according to FIG. 10, the "user state" is derived as a function of the smile synchronization rate SS and the exercise load synchronization rate LS. An example of a function that defines the user's state US (state-defining function) is US = mSS + nLS (m and n are constants), but is not limited thereto. The current state of unity estimation unit 620 derives the action value and the reward value used for updating the value function by applying the smile synchronization rate and the exercise load synchronization rate to the above-mentioned state-determining function.

<Synchronization rate of exercise state>
Another example of information that represents a sense of physical unity is the rate of synchronization of motor states. The synchronization rate of the exercise state indicates how close the movement of the user during exercise is to the movement of the instructor during exercise. The state recognition unit 610 generates motion vectors for the instructor and the user, for example, by using the measurement results of the acceleration and / or the angular velocity generated on the wrists of the instructor and the user for a certain period of time. Then, the state recognition unit 610 outputs the motion vector of the instructor and the motion vector of the user to the current state unity estimation unit 620. The current state of unity estimation unit 620 derives the degree of agreement between the direction of the instructor's motion vector and the user's motion vector (for example, it can be obtained by the inner product of the instructor's motion vector and the user's motion vector) as the synchronization rate of the motion state. To do.

In addition, in the calculation of the synchronization rate of the exercise state, it is not always necessary to consider the coincidence of directions. That is, the synchronization rate of the exercise state may consider only the magnitude of the exercise. The motion magnitude sync rate can be calculated as the difference between the root mean square of the instructor's motion vector and the root mean square of the user's motion vector. It can be judged that the smaller the difference, the higher the synchronization rate of the exercise state.

Further, it may be determined based on the characteristics of the user whether the calculation of the synchronization rate of the motion state requires the matching of the directions of the motion vectors or not.

An example of a user's characteristics is the user's exercise proficiency. When the user's skill level of exercise is high, even the matching of the directions of the motion vectors is taken into consideration in the calculation of the synchronization rate. When the user's skill level of exercise is low, the matching of the directions of the motion vectors is not considered in the calculation of the synchronization rate.

Another example of a user's characteristics is the age of the user. When the user is young (when the user is young), even the matching of the directions of the motion vectors is taken into consideration in the calculation of the synchronization rate. When the user is old (when the user is not young), the coincidence of the directions of the motion vectors is not considered in the calculation of the synchronization rate.

FIG. 11 is a diagram showing still another example of the functional configuration of the state recognition unit 610. In the example of FIG. 11, the state recognition unit 610 further includes motion vector generation modules 631, 632. The motion vector generation module 631 acquires the acceleration data 1011 and the angular velocity data 1021 of the instructor, derives the motion vector of the instructor using these, and outputs the derived motion vector to the current state unity estimation unit 620. The motion vector generation module 632 acquires the user's acceleration data 1012 and the angular velocity data 1022, derives the user's motion vector using these, and outputs the derived motion vector to the current state unity estimation unit 620.

The acceleration data 1011 is transmitted from the acceleration sensor 903, the angular velocity data 1021 is transmitted from the angular velocity sensor 904, the acceleration data 1012 is transmitted from the acceleration sensor 913, and the angular velocity data 1023 is transmitted from the angular velocity sensor 914 to the management device 10.

In the example according to FIG. 11, the "user state" is derived as a function of the smile synchronization rate SS and the motion state synchronization rate ES. An example of a function that defines the user's state US (state-defining function) is US = mSS + pES (m and p are constants), but the present invention is not limited to this. The current state unity estimation unit 620 derives the action value and the reward value used for updating the value function by applying the smile synchronization rate and the motion state synchronization rate to the above-mentioned state-determining function.

When the instructor is virtually displayed, the smile synchronization rate and the exercise state synchronization rate can be derived by using the display data of the displayed virtual instructor. That is, the smile synchronization rate can be derived by using the data used for the instructor display control instead of the information regarding the facial expression of the instructor. The motion information synchronization rate can be derived by using the data used for the display control of the virtual instructor instead of the acceleration data and the angular velocity data of the instructor.

FIG. 12 is a diagram showing still another example of the functional configuration of the state recognition unit 610. In the example of FIG. 12, the state recognition unit 610 further includes signal processing modules 621 and 622 and motion vector generation modules 631, 632.

In the example of FIG. 12, the “user state” is derived as a function of the smile synchro rate SS, the synchro rate LS of the exercise load, and the synchro rate ES of the exercise state. An example of a function that defines the user's state US (state-defining function) is US = mSS + nLS + pES (m, n, and p are constants), but the present invention is not limited to this. The current state of unity estimation unit 620 applies the smile synchronization rate, the exercise load synchronization rate, and the exercise state synchronization rate to the above-mentioned state-determining function, so that the action value and the reward value used for updating the value function are used. Is derived.

The above constants (m, n, p) can be set based on the user's attributes. The specific setting method may be described later.

<Reward at the end of exercise>
The management device 10 may further use the score given by the user for a given period as a reward at the end of the exercise. In one embodiment, when the exercise for a given period is completed, in step S112, the state recognition unit 610 displays a message prompting the input of a score on the display 13. The user inputs a score by using the input unit 14.

In this example, the state recognition unit 610 sets the smile synchronization rate (“smile synchronization rate” at time t representing the final fixed time) calculated for the last fixed time in the exercise for a given period as described above. After updating to add the input score, it is stored in the storage as a learning data set.

As a result, as a reward at the end of exercise, in addition to the information objectively detected as the facial expressions of the instructor and the user, the subjective information of the user can be used for reinforcement learning.

<Reward (smile related)>
In the estimation model, instead of the sense of unity information, information representing the heightened mood of the individual user (for example, "feeling of fun information") may be used. More specifically, in the estimation model, the management device 10 may use a value that does not use the information about the smile of the instructor instead of the “smile synchronization rate” used in the example of the sense of unity information. As an example of such a value, "smile-related value" will be described. The "smile-related value" is an example of fun information.

The smile-related value CVS is a comparison result of the smile degree calculated last time and the smile degree calculated this time during the exercise when the smile degree of the user is calculated at regular intervals (for example, 1 minute) during the exercise. As a result, it is derived as follows. The degree of smile of the user can be realized by a known technique realized as, for example, OKAO (registered trademark) Vision (OMRON Corporation).

・ If the smile level this time is higher than the previous smile level, "CVS = 1.0"
・ If the smile level this time is less than the smile level of the previous time, "CVS = 0.0"
For example, the current smile degree (the smile degree calculated immediately after the instruction content selected as the instruction content 3 minutes after the start of exercise is output) is "78", and the previous smile degree (2 from the start of exercise). It is assumed that the smile degree calculated immediately after the instruction content selected as the instruction content after minutes is output) is "64". In this case, the smile level this time is higher than the smile level of the previous time. Therefore, the smile-related value CVS 3 minutes after the start of exercise is "1.0".

<Reward (exercise load related)>
In the estimation model, instead of the "synchronization rate of exercise load" used in the example of the sense of unity information, a value that does not use the biological information of the instructor may be used as an example of the sense of enjoyment information. As an example of such a value, "exercise load related value" will be described.

A preset ideal value for a change in heart rate during exercise can be used to calculate the exercise load-related value CVL. The ideal value is given as an ideal value for each minute in a 15-minute exercise, as shown by a solid line in FIG. 15 described later. The exercise load-related value CVL is a case where the user's pulse is measured at regular intervals (for example, 1 minute) during exercise, and is derived as follows as a change in the relationship between the measured value and the ideal value. Will be done.

-If the difference between the current measurement value and the ideal value is less than or equal to the difference between the previous measurement value and the ideal value, "CVL = 1.0"
・ If the difference between the current measurement value and the ideal value is larger than the difference between the previous measurement value and the ideal value, "CVL = 0.0"
For example, the difference between the current measurement value (the pulse measured immediately after the instruction content selected as the instruction content 3 minutes after the start of exercise is output) and the ideal value 3 minutes after the start of exercise is the previous time. If it becomes smaller than the difference between the measured value (the pulse measured immediately after the instruction content selected as the instruction content 2 minutes after the start of exercise is output) and the ideal value 2 minutes after the start of exercise, exercise The exercise load-related value CVL 3 minutes after the start is "1.0". On the other hand, when the difference between the former and the latter is larger than the difference between the latter, the exercise load-related value CVL 3 minutes after the start of exercise is "0.0". That is, the exercise load related value CVL value is "1.0" if it approaches the ideal value at the time of this measurement from the previous measurement, and the exercise load if it deviates from the ideal value at the time of this measurement from the previous measurement. The value of the related value CVL is "0.0".

<Reward (related to exercise status)>
In the estimation model, instead of the "synchronization rate of the motion state" used in the example of the sense of unity information, a value that does not use the information related to the movement of the instructor may be used as an example of the sense of enjoyment information. As an example of such a value, "exercise state-related value" will be described.

An ideal motion vector at predetermined fixed time intervals (for example, 1 minute) may be used for calculating the motion state-related value CVE. The motion state-related value CVE is an ideal corresponding to the generated user motion vector and the generated time when the user motion vector is generated at regular intervals (for example, 1 minute) during exercise. A change in the degree of agreement (for example, the inner product) with the motion vector may be used. More specifically, the motion state-related value CVE is derived as follows as a result of comparison between the degree of coincidence regarding the motion vector generated this time and the degree of coincidence regarding the motion vector generated last time. In this specification, it is assumed that the closer the user's motion is to the motion defined by the ideal motion vector (the direction and / or speed of the motion is similar), the higher the value of the degree of coincidence.

・ If the degree of matching this time is equal to or higher than the degree of matching last time, "CVE = 1.0"
・ If the degree of matching this time is less than the degree of matching of the previous time, "CVE = 0.0"
For example, the degree of agreement between the motion vector generated immediately after the instruction content selected as the instruction content 3 minutes after the start of exercise is output and the ideal motion vector 3 minutes after the start of exercise is from the start of exercise. If the degree of agreement between the motion vector generated immediately after the instruction content selected as the instruction content after 3 minutes is output and the ideal motion vector 3 minutes after the start of exercise is higher than that of "CVE = 1. It is "0". When the degree of agreement of the former is lower than the degree of agreement of the latter, it is "CVE = 0.0".

<How to use the reward>
A method of using the above-mentioned three types of rewards (smile-related value CVS, exercise load-related value CVL, and exercise state-related value CVE) in an estimation model will be described.

In the estimation model, the smile-related value CVS may be used alone or in combination with the exercise load-related value CVL and / or the exercise state-related value CVE. When combined with the exercise load-related value CVL and / or the exercise state-related value CVE, the reward may be the sum of the smile-related value CVS, the exercise load-related value CVL, and the exercise state-related value CVE, or the following formula ( It may be derived according to 2).

Reward = (mCVS + nCVL + pCVE) / m + n + p ... (2)
All of m, n, and p in the mathematical formula (2) are constants, and the function of "user state" using the smile synchro rate SS, the synchro rate LS of the exercise load, and the synchro rate ES of the exercise state is described above. It can be the same as m, n, p.

The constants m, n, and p can be set arbitrarily, or can be set according to the characteristics of the user. An example of a user's characteristic is a characteristic relating to user's positivity. The m, n, and p can be set as follows depending on whether the user has the characteristic of "wanting to be the center" or the characteristic of "passive".

"I want to be the center" → m = 0.1, n = 0.3, p = 0.6
"Passive" → m = 0.3, n = 0.6, p = 0.1
In other words, for "want to be central" users (highly aggressive users), information on exercise load greatly contributes to the sense of unity with the instructor, and for "passive" users, information on exercise status is the instructor. Can greatly contribute to the sense of unity.

The user's characteristics can be entered, for example, as a start condition (FIG. 9).

The characteristics of the user may be set by the user himself / herself, or may be set based on the result of the questionnaire to the user. As for the method of setting the characteristics of the user based on the result of the questionnaire to the user, a known technique can be appropriately used. Known technologies include, for example, "Psychological segment related to leisure activities of the elderly" (announced at the 7th National Convention of the Service Society on March 3, 2019, URL: http://ja.serviceology.org/events /domestic2019/program190226.pdf).

<< Second Embodiment >>
[1. Management device configuration]
In the second embodiment of the exercise support system, the instruction content is output from the management device as in FIG. In the second embodiment, the management device can determine and output the instruction content without using the estimation model. In the second embodiment, the management device is referred to as "management device 10A" in order to distinguish it from the management device 10 of the first embodiment. FIG. 13 is a diagram showing an example of the hardware configuration of the management device 10A.

The management device 10A includes the same hardware elements (processor 11 and the like) as the management device 10. In the management device 10A, the smile reference information 16P, the heart rate reference information 16Q, and the exercise matching degree reference information 16R are stored in the storage 16.

The smile reference information 16P defines an ideal change in the degree of smile during exercise. FIG. 14 is a diagram for explaining a specific example of the smile reference information 16P. In the graph of FIG. 14, the solid line represents the ideal change in smile level. More specifically, the graph of FIG. 14 represents the ideal value of the degree of smile in each phase when one exercise is divided into 15 phases. An example of one exercise is a 15-minute exercise, and each phase is a minute-by-minute exercise in the 15-minute exercise. The broken line represents an example of an actual change in the smile level of the user.

Heart rate reference information 16Q defines an ideal change in heart rate during exercise. FIG. 15 is a diagram for explaining a specific example of the heartbeat reference information 16Q. In the graph of FIG. 15, the solid line represents the ideal change in heart rate during exercise. More specifically, the solid line in FIG. 15 represents the ideal value of the heartbeat in each of the 15 phases similar to those described for FIG. The dashed line represents an example of the actual change in the user's pulse.

The exercise concordance reference information 16R defines an ideal change in heart rate during exercise. FIG. 16 is a diagram for explaining a specific example of the motion coincidence reference information 16R. In the graph of FIG. 16, the solid line represents the ideal change in motion coincidence during exercise. The exercise concordance degree is, for example, the exercise concordance degree described in the above item "<Reward (exercise state-related)>".

[2. Selection of instruction content]
FIG. 17 is a diagram for explaining a specific example of the instruction content (advice and / or exercise intensity) to be output in the second embodiment. FIG. 17 shows a combination of two types of values, “high” and “low”, for each of the three types of items “smile”, “heartbeat”, and “exercise”.

"Smile" means the degree of smile. "Heart rate" means heart rate. "Exercise" means the degree of exercise agreement.

“High” means that the degree of smile, heart rate, or degree of exercise agreement is equal to or higher than the ideal value (value on the solid line in FIGS. 14 to 16). "Low" means that the smile, heart rate, or exercise match is lower than ideal.

In FIG. 17, the instruction content is determined according to the combination of values for each of the above three types of items.

In an example according to FIG. 17, the user's smile level is higher than the ideal value (“smile” is “high”), the user's heart rate is higher than the ideal value (“heart rate” is “high”), and the exercise agreement is high. If it is higher than the ideal value (“exercise” is “high”), the management device 10A outputs the instruction content at the top of FIG. 17, that is, praise advice (for example, the message “Perfect!”). Extract as the instruction content to be output, and output the extracted instruction content.

In another example according to FIG. 17, the user's smile level matches the ideal value (“smile” is “high”), the user's heart rate is lower than the ideal value (“heart rate” is “low”), and exercise. When the degree of agreement is lower than the ideal value (“exercise” is “low”), the management device 10A extracts advice for encouragement (for example, “do your best!”) As instruction content, and the extracted instruction content. Is output.

In an example of the policy for setting the selected instruction content, when the smile level of the user is higher than the ideal value, the selected instruction content may increase the exercise intensity and encourage the user. It may be advice to give. On the other hand, even if the user's smile level is higher than the ideal value, if the user's heart rate is higher than the ideal value, as an instruction content in order to avoid applying an excessively intense load to the user. , Praise advice, or advice that suppresses exercise may be selected.

It should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of the present invention is shown by the scope of claims rather than the above description, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

10 management device, 11 processor, 800 instructor, 810 user, 900 instructor terminal, 901,911 CPU, 903,913 acceleration sensor, 904,914 angular velocity sensor, 906,916 storage battery, 907,917 pulse wave sensor, 909,919, 950 speakers, 910 user terminals, 920 cameras.

Claims (17)

It is equipped with a storage unit that stores reference information for selecting the instruction content output to the user as an action related to the user's state.
The user's state includes information on the user's willingness to continue exercising.
An acquisition unit that acquires the state of the user who is exercising,
A selection unit that selects the instruction content to be output to the user based on the state of the user and the information for reference.
An information processing device including an output unit that outputs the selected instruction content. The reference information includes a value function learned by reinforcement learning to select the instruction content output to the user as an action related to the user's state.
The selection unit selects the action having the maximum value of the value function among the actions related to the state of the user with respect to the user during exercise.
The information processing device according to claim 1, wherein the action in which the value of the value function is maximized is the action in which the state of the user, which is a reward, becomes the highest at the end of the exercise. The information processing device according to claim 1 or 2, wherein the continuation motivation information includes unity information indicating a sense of unity between the instructor and the user. The information processing device according to claim 3, wherein the sense of unity information includes information based on the smiles of the instructor and the user. The information processing device according to claim 3 or 4, wherein the sense of unity information represents a ratio in which the facial expressions of the leader and the user match at a plurality of timings. The information processing device according to any one of claims 1 to 5, wherein the state of the user further includes a ratio in which the exercise load matches between the instructor and the user. The information processing device according to claim 6, wherein the exercise load is based on the pulse waves of the instructor and the user. The information processing device according to any one of claims 1 to 7, wherein the state of the user further includes a ratio in which the exercise state matches between the instructor and the user. The information processing apparatus according to claim 8, wherein the exercise state is based on at least one of acceleration and angular velocity, which is measured for each body of the instructor and the user. The output unit outputs an image of the instructor,
According to claim 8, the motion state is based on at least one of the output acceleration and angular velocity related to the movement of the body of the leader and at least one of the acceleration and angular velocity measured for the user's body. The information processing device described. The information processing device according to claim 1 or 2, wherein the continuation motivation information includes information representing a difference from an ideal value of the user's state. The reference information represents an ideal value for the user's condition.
The information processing apparatus according to claim 1, wherein the selection unit selects an instruction content to be output to the user based on a difference between the user's state and the reference information. The information processing device according to claim 12, wherein the state of the user includes a degree of smile of the user. The information processing apparatus according to claim 13, wherein the user's state includes at least one of the user's biological information and the exercise state. The information processing apparatus according to any one of claims 1 to 14, wherein the plurality of instruction contents corresponding to each of the plurality of actions include at least one of advice contents and exercise intensity. Information processing including an information processing device that selects the instruction content output to the user as an action related to the user's state, and a machine learning device that learns the value function of the instruction content output to the user by reinforcement learning. It's a system
The information processing device
It includes a storage unit that stores the value function learned by reinforcement learning in order to select the instruction content output to the user as an action related to the user's state.
The user's state includes the user's willingness to continue information.
The information processing device
The acquisition unit that acquires the continuation motivation information about the user who is exercising,
Among the actions related to the continuation motivation information about the user during exercise, a selection unit for selecting the action having the maximum value of the value function, and
Including an output unit that outputs an instruction content corresponding to the selected action,
The action in which the value of the value function is maximized is the action in which the reward, the continuation motivation information, is highest at the end of the exercise.
The machine learning device
A state acquisition unit that acquires the state of the user,
The value of updating the value function by using the state of the user before the output of the instruction content as a state, the instruction content output to the user as an action, and the state of the user after the output of the instruction content as a reward. An information processing system that includes a function update unit. It is a machine learning device that learns the value function of the instruction content output to the user as an action related to the user's state by reinforcement learning.
A status acquisition unit that acquires the user's status including the user's continuation motivation information,
The value of updating the value function by using the state of the user before the output of the instruction content as a state, the instruction content output to the user as an action, and the state of the user after the output of the instruction content as a reward. A machine learning device equipped with a function update unit.

Images