JP2021101263A - Information processing equipment, information processing method, and program - Google Patents

Abstract

To provide a technique capable of acquiring a more appropriate data set for performing machine learning regarding provision of support information for playing sports in response to a motion characteristic of a user.SOLUTION: An information processing device 30 according to the embodiment of the present disclosure is provided with a support information output unit 304 for outputting support information according to a movement of a target person based on movement data acquired by a movement data acquisition unit 301, by using a trained model LM, an usefulness selection unit 305 for determining whether or not the output support information is an appropriate content for the acquired operation data, and a support information editing unit 306 for editing at least one of the support information of the data set including the support information output from the support information output unit 304, and operation data relatively close to the acquired operation data in a data set of a database 21, according to an input received by the input device 37, if it is determined that the content is not the appropriate content by the usefulness selection unit 305.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to an information processing device and the like.

Conventionally, there is known a technique for performing machine learning based on a data set (so-called teacher data set) that is a combination of input data and correct answer data (see, for example, Patent Document 1).

JP-A-2019-75159

By the way, in sports such as golf and tennis, the clerk of the store that sells the equipment proposes the recommended equipment such as clubs and rackets, and the instructor proposes the improvement of the movement according to the exercise characteristics of the target person. There are times when things like that are done. In this case, data that is a combination of data representing a person's movements acquired by measurement, etc., and data on support information for a person to perform sports according to their movement characteristics, such as recommended equipment and suggestions for improving movements. Machine learning can be performed based on the set group. As a result, a trained model that can output support information for playing sports according to the motor characteristics of the target person is generated by inputting data representing the movement, and the trained model is used to recommend tools and movements. Support information such as improvement can be easily provided.

However, in a situation where a customer is visiting a store for a specific product, the clerk may recommend the product that the customer wants to purchase as it is. In addition, depending on the clerk, it is possible to propose equipment that does not match the customer's exercise characteristics, or to propose to the customer the equipment that he / she wants to sell for some reason regardless of the customer's exercise characteristics. There is sex. In addition, depending on the instructor's guidance policy, there is a possibility that proposals for improving unique movements will be made. Therefore, even if machine learning is performed based on a data set that combines data related to human movements and data related to support information such as recommended tools and suggestions for improving movements, it is not possible to output appropriate support information. there is a possibility.

Therefore, in view of the above problems, the purpose is to provide a technique capable of acquiring a more appropriate data set for performing machine learning regarding the provision of support information for performing sports according to the motor characteristics of the user. To do.

In order to achieve the above object, in one embodiment of the present disclosure,
An input unit that accepts predetermined input and
An acquisition unit that acquires data related to the behavior of the target person,
Acquired by the acquisition unit using a trained model that has been machine-learned based on a plurality of data sets that combine data related to human movements and data related to support information for humans to perform sports according to their motor characteristics. A support information output unit that outputs the support information according to the movement of the target person based on the data to be generated, and a support information output unit.
A determination unit that determines whether or not the support information output from the support information output unit has appropriate contents for the data acquired by the acquisition unit.
When the determination unit determines that the content is not appropriate, the support information output from the support information output unit and the acquisition of the plurality of data sets according to the input received by the input unit. It includes an editorial unit that edits at least one of the support information of the dataset including data on the movement of a person that is relatively close to the data acquired by the unit.
An information processing device is provided.

Also, in other embodiments of the present disclosure,
An information processing method executed by an information processing device provided with an input unit that accepts a predetermined input.
The acquisition step to acquire the data related to the behavior of the target person, and
Acquired in the acquisition step using a trained model that has been machine-learned based on multiple datasets that combine data on human movements and data on support information for humans to play sports according to their motor characteristics. A support information output step that outputs the support information according to the movement of the target person based on the data to be performed, and a support information output step.
A determination step for determining whether or not the support information output in the support information output step is appropriate for the data acquired in the acquisition step, and a determination step.
When it is determined that the content is not appropriate in the determination step, the support information output in the support information output step and the acquisition of the plurality of data sets are obtained according to the input received by the input unit. Includes an edit step that edits at least one of the support information in a dataset that contains data about human behavior that is relatively close to the data acquired in the step.
Information processing methods are provided.

Also, in still other embodiments of the present disclosure,
An information processing device equipped with an input unit that accepts a predetermined input
The acquisition step to acquire the data related to the behavior of the target person, and
Acquired in the acquisition step using a trained model that has been machine-learned based on multiple datasets that combine data on human movements and data on support information for humans to play sports according to their motor characteristics. A support information output step that outputs the support information according to the movement of the target person based on the data to be performed, and a support information output step.
A determination step for determining whether or not the support information output in the support information output step is appropriate for the data acquired in the acquisition step, and a determination step.
When it is determined that the content is not appropriate in the determination step, the support information output in the support information output step and the acquisition of the plurality of data sets are obtained according to the input received by the input unit. To execute, an editing step of editing at least one of the support information of the dataset containing data on human behavior that is relatively close to the data acquired in the step.
The program is provided.

According to the above-described embodiment, the information processing device edits the current support information that is inappropriate for the data related to the operation of the target person, for example, in response to the input of an editor such as a store clerk or an instructor. Can be done. Therefore, the information processing apparatus can incorporate the data set of the combination of the data related to the movement of the target person acquired this time and the data related to the edited support information into the data set group for machine learning. In addition, when the support information output from the trained model is inappropriate for the data related to the user's behavior, the information processing device acquires this time in the data set group for machine learning according to the input of the editor. It is possible to edit the support information of the dataset including the data close to the data related to the user's behavior. Therefore, the information processing device can acquire a more appropriate data set for machine learning by editing the support information.

According to the above-described embodiment, it is possible to provide a technique capable of acquiring a more appropriate data set for performing machine learning regarding the provision of support information for performing sports according to the motor characteristics of the user. ..

It is a schematic diagram of an information providing system. It is a figure which shows an example of the hardware configuration of an information processing apparatus. It is a functional block diagram which shows an example of the structure of an information provision system. It is a flowchart which shows typically an example of the control processing by a usefulness selection part. It is a flowchart which shows the other example of the control processing by a usefulness selection part schematicly. It is a flowchart which shows an example of the control process by a support information editorial department. It is a flowchart which shows the other example of the control processing by a support information editorial department schematicly.

Hereinafter, embodiments will be described with reference to the drawings.

[Overview of information provision system]
First, the outline of the information providing system 1 according to the present embodiment will be described with reference to FIG.

FIG. 1 is a schematic view of the information providing system 1.

As shown in FIG. 1, the information providing system 1 includes an operation data acquisition device 10, a storage device 20, and an information processing device 30. The operation data acquisition device 10, the storage device 20, and the information processing device 30 may all be installed in the same place (for example, a facility), or a part of the operation data acquisition device 10, the storage device 20, and the information processing device 30 may be installed in a remote place different from the other parts. You may be.

The operation data acquisition device 10 acquires data related to human operation (hereinafter, “operation data”). The motion data acquisition device 10 acquires motion data when a person plays a specific sport, for example. The operation data acquisition device 10 and the information processing device 30 are communicably connected, and the operation data of the operation data acquisition device 10 is taken into the information processing device 30.

The storage device 20 is communicably connected to the information processing device 30 and stores data used by the information processing device 30.

The function of the storage device 20 may be incorporated inside the information processing device 30.

The information processing device 30 uses the data of the storage device 20 to perform a specific sport according to the movement characteristics (motor characteristics) of the target person based on the movement data of the target person input from the movement data acquisition device 10. Providing support information to the user. The user includes, for example, a store clerk who proposes sports equipment to the target person. In addition, the user includes, for example, an instructor (for example, an instructor) who teaches sports to the target person. Further, the user includes, for example, a target person corresponding to the operation data. The support information includes, for example, information on tools recommended according to the movement characteristics (motor characteristics) of the subject (hereinafter, “recommended tool information”). In addition, the support information includes, for example, information on improvement of movement according to the movement characteristics (motor characteristics) of the subject (hereinafter, "motion improvement information").

In this way, the information providing system 1 provides the user with support information according to the operation data acquired by the operation data acquisition device 10 through the information processing device 30.

[Configuration of information provision system]
Next, a specific configuration of the information providing system 1 (information processing device 30) will be described with reference to FIGS. 2 and 3 in addition to FIG.

FIG. 2 is a diagram showing an example of the hardware configuration of the information processing device 30. FIG. 3 is a functional block diagram showing an example of the configuration of the information providing system 1.

The function of the information processing apparatus 30 may be realized by arbitrary hardware, an arbitrary combination of hardware and software, and the like. For example, as shown in FIG. 2, the information processing device 30 includes a drive device 31, an auxiliary storage device 32, a memory device 33, a CPU 34, an interface device 35, a display device 36, and an input device 37. , Each is connected by bus B.

A program that realizes various functions of the information processing device 30 can be, for example, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. Provided by a portable recording medium 31A. When the recording medium 31A on which the program is recorded is set in the drive device 31, the program is installed in the auxiliary storage device 32 from the recording medium 31A via the drive device 31. The program may also be downloaded from another computer via the communication network and installed in the auxiliary storage device 32.

The auxiliary storage device 32 stores various installed programs and also stores necessary files, data, and the like.

The memory device 33 reads and stores the program from the auxiliary storage device 32 when the program is instructed to start.

The CPU 34 executes various programs stored in the memory device 33, and realizes various functions related to the information processing device 30 according to the programs.

The interface device 35 is used as an interface for connecting to an external device through an external communication line (for example, a one-to-one communication line or a communication network). The interface device 35 may include a plurality of types of interface devices depending on the difference in communication form with an external device.

The display device 36 displays a GUI (Graphical User Interface) according to, for example, a program executed by the CPU 34. The display device 36 is, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like.

The input device 37 (an example of the input unit) is used to make the operator, the manager, or the like of the information processing device 30 input various instructions regarding the information processing device 30. The input device 37 includes, for example, an operation input device that accepts a user's operation input. The operation input device may include, for example, a keyboard, a mouse, a touch panel mounted on the display device 36, a touch pad, a button, a toggle, a lever, a rotary switch, and the like. Further, the input device 37 may include, for example, a voice input device or a gesture input device that accepts a user's voice input or gesture input. The voice input device includes, for example, a microphone. The gesture input device includes an imaging device that captures the content of the user's gesture (for example, a moving image).

The information processing device 30 realizes various functions by, for example, executing a program installed in the auxiliary storage device 32 on the CPU 34. Specifically, as shown in FIG. 3, the information processing apparatus 30 has, as functional elements, an operation data acquisition unit 301, a learning unit 302, a support information output unit 304, a usefulness selection unit 305, and support information. Includes editorial unit 306. Further, the information processing device 30 includes, for example, a storage unit 303 as a storage area defined in a non-volatile internal memory such as the auxiliary storage device 32.

The operation data acquisition unit 301 (an example of the acquisition unit) acquires operation data input (received) from the operation data acquisition device 10 from a reception buffer or the like.

The learning unit 302 performs machine learning based on a plurality of data sets (teacher data) stored in the storage device 20.

The storage device 20 is a database 21 containing a plurality of data sets (record data) obtained by combining previously acquired human motion data and data related to support information corresponding to the motion data (hereinafter, “support information data”). Has been built. The operation data of the data set in the database 21 may be the operation data acquired by the operation data acquisition device 10 of the information providing system 1, or may be the operation data acquired by another device. Further, the support information data included in the data set in the database 21 is, for example, a tool recommended to the customer in the actual customer service process based on the operation data by the store clerk or a guidance process based on the operation data by the instructor. It is the data of the information about the improvement of the behavior proposed to the student. Further, the database 21 is based on the operation data uploaded from the terminal of the store or the terminal of the sports practice facility to the storage device 20 or the like and the support information corresponding to the operation data (for example, recommended equipment information or operation improvement information). It may be updated as appropriate.

The learning unit 302 performs machine learning (so-called supervised learning) based on a plurality of data sets registered in the database 21 and generates a learned model LM.

Further, when the database 21 of the storage device 20 is updated, the learning unit 302 updates the learned model LM of the updated database 21. The learning unit 302 may update the trained model LM by additionally learning the existing trained model LM using only the updated data set among all the data sets of the database 21. Further, the learning unit 302 may update the trained model LM by generating a new trained model LM using all the data sets of the database 21.

The learned model LM generated by the learning unit 302 is stored in the storage unit 303 (an example of the first storage unit). The latest learned model LM is stored in the storage unit 303. Further, the storage unit 303 may store the trained model LM before the update. As a result, the user can use not only the latest trained model LM but also the previously generated trained model LM including the trained model LM before the update. Therefore, the user can select the trained model LM to be used, which improves the convenience of the user.

The support information output unit 304 uses the trained model LM, and based on the motion data of the target person input from the motion data acquisition section 301, the support information corresponding to the motion data, that is, the motion (motor characteristics) of the target person. ) Is generated (output). As a result, the information processing device 30 can provide the support information corresponding to the operation data of the target person to the user (for example, the target person himself / herself, the store clerk, the instructor, etc.) through the display device 36 or the like.

The usefulness selection unit 305 (an example of the determination unit) selects (determines) whether or not the support information is useful to the target person based on the support information input from the support information output unit 304. Specifically, the usefulness selection unit 305 determines whether or not the support information generated by the support information output unit 304 is appropriate for the operation data (that is, operation characteristics) of the target person. The specific determination method will be described later (see FIGS. 4 and 5).

The support information editing unit 306 (an example of the editorial unit) operates according to the information representing the selection result (hereinafter, “selection result information”) input from the usefulness selection unit 305. The selection result information represents the selection result of whether or not the support information generated by the support information output unit 304 is useful to the target person, and includes either "beneficial" or "non-beneficial" information. The support information editing unit 306 edits the support information in response to the input from the user through the input device 37 when "no usefulness" is specified in the selection result information. Information about the editing history of the support information by the support information editing unit 306 is an external storage device capable of communicating with the internal memory of the information processing device 30 (for example, the storage unit 303 (an example of the second storage unit)) or the information processing device 30. It may be recorded in (for example, the storage device 20). The specific editing method will be described later (see FIGS. 6 and 7).

The function of the information processing device 30 may be distributed and realized by a plurality of information processing devices and the like. For example, the operation data acquisition unit 301 may be transferred to another information processing device (for example, a mobile terminal of the target person). As a result, for example, a target person in a remote location away from the information processing device 30 can obtain an image captured by the operation data acquisition device 10 (for example, a camera) mounted on the mobile terminal. It can be transmitted from the acquisition unit 301 to the remote information processing device 30. Then, the information processing device 30 outputs support information based on the operation data transmitted from the mobile terminal, and the instructor who is in the same place as the information processing device 30 uses the support information and makes a video call or the like. Guidance on improving the movement of the subject can be provided from a remote location. Further, for example, the function of the learning unit 302, the function of the storage unit 303, and the function of generating the support information of the support information output unit 304 are transferred to another information processing device (for example, a cloud server for machine learning). You may. In this case, the information processing device 30 transmits the operation data to the cloud server for machine learning via a predetermined network, and also receives the support information corresponding to the operation data from the cloud server, and the support information output unit 304 receives the support information corresponding to the operation data. , The received support information may be output.

[Control processing of benefit selection section]
Next, the control process by the benefit selection unit 305 will be described with reference to FIGS. 4 and 5.

<Example of control processing of the benefit selection unit>
FIG. 4 is a flowchart schematically showing an example of control processing by the benefit selection unit 305. This flowchart is started, for example, when the support information is output from the support information output unit 304. Hereinafter, the same applies to the flowchart of FIG.

As shown in FIG. 4, in step S102, the usefulness selection unit 305 acquires the support information input from the support information output unit 304. When the process of step S102 is completed, the benefit selection unit 305 proceeds to step S104.

In step S104, the benefit selection unit 305 causes the display device 36 to display the acquired support information. When the process of step S104 is completed, the benefit selection unit 305 proceeds to step S106.

In step S106, the benefit selection unit 305 causes the display device 36 to display the content requesting the user to select the presence or absence of benefit. The usefulness selection unit 305, for example, pops up including a button icon corresponding to each of useful and non-beneficial and text information requesting to select one in a manner in which the content of the support information can be confirmed. The window may be displayed on the display device 36. As a result, the user can select and input whether or not the support information is useful to the target person through the input device 37 while checking the content of the support information displayed on the display device 36. When the process of step S106 is completed, the benefit selection unit 305 proceeds to step S108.

In step S108, the benefit selection unit 305 determines whether or not there is a selection input regarding the presence or absence of benefit. If there is a selection input, the benefit selection unit 305 proceeds to step S110. On the other hand, if there is no selection input even after the lapse of a predetermined time, the benefit selection unit 305 considers that the selection input with benefit has been made, and proceeds to step S114.

In step S110, the benefit selection unit 305 determines whether or not the selection input has benefit. The benefit selection unit 305 proceeds to step S112 if it is not a beneficial selection input, and proceeds to step S114 if it is a beneficial selection input.

In step S112, the benefit selection unit 305 outputs selection result information including the content of "no benefit", and ends the current process.

On the other hand, in step S114, the benefit selection unit 305 outputs selection result information including the content of "beneficial", and ends the current process.

As described above, in this example, the benefit selection unit 305 causes the user, such as a store clerk or an instructor of the practice facility, to judge the benefit and input the operation through the input device 37. Whether or not the support information generated by the output unit 304 is useful can be determined.

<Other examples of control processing of the benefit selection unit>
FIG. 5 is a flowchart schematically showing another example of the control process by the benefit selection unit 305.

As shown in FIG. 5, in step S202, the usefulness selection unit 305 acquires the support information input from the support information output unit 304. When the process of step S202 is completed, the benefit selection unit 305 proceeds to step S204.

In step S204, the usefulness selection unit 305 causes the display device 36 to display the content requesting the acquisition of the operation data to which the support information is applied. The operation data to which the support information is applied is, for example, operation data related to an operation performed by using the tool recommended in the recommended tool information. Further, the operation data to which the support information is applied is, for example, operation data when the target person actually performs the content of the operation improvement proposed in the operation improvement information. When the benefit selection unit 305 completes the process of step S204, the benefit selection unit 305 proceeds to step S206.

In step S206, the usefulness selection unit 305 determines whether or not new operation data has been acquired by the operation data acquisition unit 301. The usefulness selection unit 305 proceeds to step S208 when new operation data is acquired by the operation data acquisition unit 301. On the other hand, if the usefulness selection unit 305 does not acquire new operation data by the operation data acquisition unit 301 even after the lapse of a predetermined time, the usefulness selection unit 305 considers that the support information is useful and proceeds to step S214.

In step S208, the benefit selection unit 305 compares the result (performance) output by inputting the action of the target person before and after applying the support information. For example, when the motion data acquired by the motion data acquisition unit 301 is data related to the golf swing motion described later, the results output by inputting the motion of the target person include the flight distance of the ball, the bending width of the ball, and the like. The head speed of the club, the face angle of the club, the amount of backspin of the ball, the contribution of the lower body to the head speed (for example, energy) and the like may be included. When the process of step S208 is completed, the benefit selection unit 305 proceeds to step S210.

In step S210, the benefit selection unit 305 determines whether or not the result is improved (improved) before and after the application of the support information. The benefit selection unit 305 proceeds to step S212 if the result is not improved (improved) before and after the application of the support information, and proceeds to step S214 if the result is improved (improved).

Since the processing of steps S212 and S214 is the same as that of steps S112 and S114 of FIG. 4, the description thereof will be omitted.

As described above, in this example, the usefulness selection unit 305 determines the usefulness of the support information generated by the support information output unit 304 based on whether or not the result (performance) is improved (improved) before and after the application of the support information. The presence or absence can be determined.

[Control processing of support information editorial department]
Next, the control process of the support information editing unit 306 will be described with reference to FIGS. 6 and 7.

<Example of control processing of the support information editorial department>
FIG. 6 is a flowchart schematically showing an example of control processing by the support information editing unit 306. This flowchart is started, for example, when the selection result information without benefit is input from the benefit selection unit 305. Hereinafter, the same applies to the flowchart of FIG. 7.

As shown in FIG. 6, in step S302, the support information editing unit 306 acquires (reads) the current operation data and the support information from the buffer or the like of the memory device 23. When the process of step S302 is completed, the support information editing unit 306 proceeds to step S304.

In step S304, the support information editing unit 306 causes the display device 36 to display the content (hereinafter, “editing screen”) prompting the editing of the support information this time. On the edit screen, for example, the current operation data and the support information corresponding to the operation data are displayed. Further, on the edit screen, for example, a plurality of candidates that can be selected as support information are displayed. As a result, the user can edit the support information generated by the support information output unit 304 by selecting the support information suitable for the operation data of the target person acquired this time from the plurality of candidates. When the process of step S304 is completed, the support information editing unit 306 proceeds to step S306.

In step S306, the support information editing unit 306 determines whether or not the editing is completed. For example, the support information editing unit 306 determines that the editing is completed when one candidate is selected as the support information from the plurality of candidates through the input device 37 and the input indicating the end of editing is accepted. Good. If the editing is not completed, the support information editing unit 306 proceeds to step S308, and if the editing is completed, the support information editing unit 306 proceeds to step S310.

In step S308, the support information editing unit 306 determines whether or not any of the conditions of the acceptance of the release input by the input device 37 and the elapse of a certain time from the start of the display of the editing screen is satisfied. The cancellation input is an input requesting cancellation of the display of the edit screen. When any of the conditions is satisfied, the support information editing unit 306 cancels the display of the edit screen, ends the processing of the flowchart this time, and returns to step S306 when any of the conditions is not satisfied.

In step S310, the support information editing unit 306 registers a data set (hereinafter, “edited data set”) obtained by combining the current operation data and the data related to the edited support information in the database 21 of the storage device 20. (Record. When the processing of step S310 is completed, the support information editing unit 306 ends the processing of the current flowchart.

As described above, in this example, the support information editing unit 306 edits the support information generated (output) by the support information output unit 304 in response to the input from the user through the input device 37, and the edited support The database 21 can be updated based on the information. Therefore, the learning unit 302 can perform machine learning based on the data set including the data related to the edited support information and update the learned model LM. Therefore, the information processing device 30 can update the trained model LM in a manner capable of outputting more appropriate support information according to the accumulation of the data set including the data related to the edited support information.

Further, instead of updating the data set included in the database 21, the support information editing unit 306 creates a database with a new data set including a data set already registered in the database 21 and a newly registered data set. It may be constructed in the storage device 20. As a result, the data set of the existing database 21 can be retained.

For example, the guidance policy may differ for each instructor who guides the target person. In this case, it is possible that some instructors can be provided with appropriate support information in the trained model LM based on the data set of the database 21, but some instructors cannot be provided with appropriate support information.

On the other hand, a new database is constructed while maintaining the existing database 21. As a result, the information processing device 30 can prepare a plurality of learned model LMs according to the instruction policy for each instructor along with a plurality of databases (data sets) according to the instruction policy for each instructor. .. Therefore, the information processing device 30 can provide support information (operation improvement information) according to the guidance policy for each instructor.

<Other examples of control processing of the support information editorial department>
FIG. 7 is a flowchart schematically showing another example of the control process by the support information editing unit 306.

As shown in FIG. 7, the process of step S402 is the same as that of step S302 of FIG. 6, and therefore the description thereof will be omitted.

In step S404, the support information editing unit 306 extracts from the database 21 a data set including operation data that is relatively close to the current operation data acquired by the operation data acquisition unit 301. Whether or not a certain operation data is relatively close to the current operation data may be determined by, for example, whether or not the correlation coefficient between the certain operation data and the current operation data is larger than a predetermined threshold value. Further, whether or not a certain operation data is relatively close to the current operation data may be determined by, for example, whether or not the difference between the certain operation data and the current operation data is smaller than a predetermined threshold value. Further, whether or not a certain operation data is relatively close to the current operation data is determined by, for example, whether or not the cosine similarity between the certain operation data and the current operation data is larger than a predetermined threshold value. May be done. Whether or not a certain operation data is relatively close to the current operation data is determined by, for example, a predetermined clustering process for the current operation data and all the operation data of the database 21. It may be judged by whether or not it is classified into the same group as. Further, for example, the operation data may include a plurality of types of operation data. In this case, it may be determined whether or not the operation data is relatively close to each other according to any of the above criteria. Then, when it is determined that all types are relatively close, or when the ratio of the types determined to be relatively close among all types is relatively high, a certain operation data and the current operation data And may be determined to be relatively close. When the process of step S404 is completed, the support information editing unit 306 proceeds to step S406.

In step S406, the support information editing unit 306 displays an edit screen for prompting the editing of the support information this time and the support information corresponding to the operation data relatively close to the operation data registered in the database 21 this time. To display. On the edit screen, for example, the current operation data and the support information corresponding to the current operation data are displayed. Further, on the edit screen, for example, the operation data and the support information of the data set extracted from the database 21 in step S404 are displayed. Further, on the edit screen, for example, a plurality of candidates that can be selected as support information after editing are displayed for each combination of operation data and support information. As a result, the user can select the support information suitable for each operation data from the plurality of candidates. When the process of step S406 is completed, the support information editing unit 306 proceeds to step S408.

Since the processing of steps S408 and S410 is the same as that of steps S306 and S308 of FIG. 6, the description thereof will be omitted.

In step S412, the support information editing unit 306 registers (records) the edited data set obtained by combining the operation data and the edited support information in the database 21 of the storage device 20. Specifically, the support information editing unit 306 registers (records) the data set obtained by combining the current operation data and the edited support information in the database 21 of the storage device 20, as in the case of step S310 of FIG. ). Further, the support information editing unit 306 updates the support information of the data set extracted in step S404 among the data sets registered in the database 21 with the edited support information. When the process of step S412 is completed, the support information editing unit 306 ends the process of the current flowchart.

As described above, in this example, the support information editing unit 306 makes the data related to the support information of the data set registered in the database 21 more appropriate for the operation data in response to the input from the user through the input device 37. You can edit and update the contents. Therefore, the learning unit 302 can perform machine learning based on the data set including the data related to the edited support information and update the learned model LM. Therefore, the information processing device 30 can update the trained model LM in a manner capable of outputting more appropriate support information in response to the update of the support information data of the data set of the database 21.

Further, the support information editing unit 306 may have the storage device 20 construct a new database instead of updating the data set included in the database 21, as in the case of the above example (FIG. 6). As a result, the data set of the existing database 21 can be retained. Therefore, the information processing device 30 can prepare a plurality of learned model LMs according to the instruction policy for each instructor along with a plurality of databases (data sets) according to the instruction policy for each instructor. Therefore, the information processing device 30 can provide support information (operation improvement information) according to the guidance policy for each instructor.

The support information editing unit 306 may only edit the support information registered in the database 21 and may not edit the current support information generated by the support information output unit 304. That is, the database 21 may not record the current operation data and the edited support information corresponding to the current operation data, and only the support information of the database 21 may be updated by editing.

[Specific application example of information provision system]
Next, a specific application example of the information providing system 1 according to the present embodiment will be described.

<First application example of the information providing system>
In this example, the information providing system 1 provides the user with support information regarding the improvement of the swing motion of golf (hereinafter, “swing motion improvement information”). The user is, for example, a target person who performs a swing motion or an instructor who guides the target person.

The motion data acquisition device 10 acquires motion data (hereinafter, “swing data”) related to the golf swing motion of the target person. For example, the motion data acquisition device 10 acquires swing data representing the movement of the club during the swing motion of the target person. For example, swing data includes data about club passes. The club pass represents the trajectory (direction of the head) of the club (head) in the horizontal direction from immediately before to immediately after the impact with the golf ball in the trajectory of the club (head) at the time of swing. Further, the swing data includes, for example, data regarding an attack angle. The attack angle represents the angle of incidence of the club (head) on the golf ball with respect to the horizontal plane at the time of impact with the golf ball. Further, the swing data includes, for example, data relating to the face angle. The face angle represents the orientation of the face surface in the horizontal direction at the time of impact with the golf ball. Further, the swing data includes, for example, data related to face rotation. The face rotation represents the open / closed state of the face during the swing operation, for example, the rotational angular velocity of the face around the shaft axis. In addition, the swing data includes, for example, data related to the tame. Tame represents the time lag between the movement of the arm and the movement of the club head during the downswing. Further, the swing data includes, for example, data regarding release timing. The release timing represents the timing for returning (releasing) the cocked wrist to the original state during the downswing. Further, the swing data includes, for example, data relating to a swing plane at the time of backswing (hereinafter, “BS swing plane”). Further, the swing data includes data related to the swing plane at the time of downswing (hereinafter, “DS swing plane”). The BS swing lane and the DS swing plane are virtual planes corresponding to the orbits of the club (head) during the backswing and the downswing, respectively. Further, the swing data includes data related to the head speed. The head speed represents the speed of the club (head) at the time of impact with the golf ball. The motion data acquisition device 10 includes, for example, an inertial sensor attached to the club. Further, the motion data acquisition device 10 includes, for example, an image pickup device that captures a swing motion of a target person and acquires moving image information.

Further, for example, the motion data acquisition device 10 may acquire data related to the evaluation by the instructor for the swing motion of the target person. The evaluation by the instructor includes, for example, a round evaluation based on at least one of the subject's movement during a round on a simulator simulating a golf course, the subject's thoughts, the result for the swing movement, and the like. In the round evaluation, for example, each item of situation judgment, club selection, routine, ball direction, and sense of distance is evaluated. In addition, the evaluation by the instructor includes, for example, motion evaluation regarding the flexibility and range of motion of the user's body part. In the motion evaluation, it is evaluated whether or not a predetermined motion (for example, seven prescribed motions) can be performed correctly. The operation data acquisition device 10 includes, for example, a terminal (for example, a computer terminal of a practice facility) that receives an input regarding evaluation from an instructor.

In the database 21 of the storage device 20, a plurality of data sets are registered by combining the swing data acquired in the past and the data related to the swing motion improvement information corresponding to the swing data.

The operation data acquisition unit 301 acquires (reads) swing data acquired from the operation data acquisition device 10.

The learning unit 302 performs machine learning based on a plurality of data sets registered in the database 21 of the storage device 20 and generates (outputs) a learned model LM. The trained model LM can input swing data and output swing motion improvement information.

The support information output unit 304 uses the trained model LM to generate (output) swing motion improvement information according to the swing motion of the target person based on the swing data input from the motion data acquisition unit 301. As a result, the information processing device 30 can provide the user (for example, the target person himself / herself, the instructor, etc.) with swing motion improvement information corresponding to the swing data of the target person through the display device 36 or the like.

The swing motion improvement information includes, for example, information on the axis during the swing motion of the subject and information on whether the upper body is ahead or the lower body is ahead when comparing the upper body and the lower body. Specifically, the swing motion improvement information is "Your swing motion is on the right axis and the lower body precedes. There is no particular need to correct the swing motion." Or "Swing motion is on the left axis and has a large sway. It may include comments on improving swing movements, such as "upper body first. Learn how to move your weight correctly." Hereinafter, the same may apply to the third application example described later.

The usefulness selection unit 305 selects (determines) whether or not the swing motion improvement information input from the support information output unit 304 is useful to the target person. The usefulness selection unit 305 may select (determine) whether or not the swing motion improvement information is useful to the target person, for example, according to the flowchart of FIG. In this case, the usefulness selection unit 305 selects (determines) whether or not the swing motion improvement information is useful to the target person, for example, in response to the input from the instructor as the user through the input device 37.

When the benefit selection unit 305 selects (determines) that the swing motion improvement information is not useful for the target person, the support information editing unit 306 inputs the information through the input device 37 of the instructor as a user. The swing motion improvement information is edited accordingly.

The support information editing unit 306 may edit the swing motion improvement information according to the flowchart of FIG. 6 described above. As a result, the support information editing unit 306 edits the swing motion improvement information generated (output) by the support information output section 304 in response to the input from the instructor through the input device 37, and the edited swing motion improvement The database 21 can be updated based on the information. Therefore, the learning unit 302 can perform machine learning based on the data set including the edited data related to the swing motion improvement information, and update the learned model LM. Therefore, the information processing device 30 updates the trained model LM in a manner capable of outputting more appropriate swing motion improvement information in accordance with the accumulation of the data set including the data related to the edited swing motion improvement information. Can be done.

Further, as described above, the support information editing unit 306 may have the storage device 20 construct a new database instead of updating the data set included in the database 21. As a result, the data set of the existing database 21 can be retained. Therefore, the information processing device 30 can prepare a plurality of learned model LMs according to the instruction policy for each instructor along with a plurality of databases (data sets) according to the instruction policy for each instructor. Therefore, the information processing device 30 can provide swing motion improvement information according to the guidance policy for each instructor.

<Second application example of information provision system>
In this example, the information providing system 1 provides the user with support information (hereinafter, “recommended club information”) regarding a club recommended according to the swing motion of golf. The user is, for example, a target person who performs a swing operation or a store clerk who serves the target person.

The motion data acquisition device 10 acquires motion data (swing data) related to the golf swing motion of the target person. For example, the motion data acquisition device 10 acquires swing data representing the movement of the club during the swing motion of the target person. For example, swing data includes data about club passes. Further, the swing data includes, for example, data regarding an attack angle. Further, the swing data includes, for example, data relating to the face angle. Further, the swing data includes, for example, data related to face rotation. In addition, the swing data includes, for example, data related to the tame. Further, the swing data includes, for example, data regarding release timing. Further, the swing data includes, for example, data relating to the BS swing plane. Further, the swing data includes data related to the DS swing plane. Further, the swing data includes data related to the head speed. Further, for example, the motion data acquisition device 10 acquires swing data representing the trajectory (ballistic) of the golf ball as a result of the swing motion of the target person. The swing data includes, for example, data regarding the flight distance of a golf ball. In addition, the swing data includes data related to left and right blurring. The left-right blur represents a state in which the trajectory of the golf ball is displaced to the left or right with respect to a predetermined target representing the front. The motion data acquisition device 10 includes, for example, an inertial sensor attached to the club. Further, the motion data acquisition device 10 includes, for example, an image pickup device that captures at least one of the swing motion of the target person and the trajectory of the golf ball and acquires moving image information. Further, the operation data acquisition device 10 includes, for example, a GPS sensor attached to a golf ball. Further, the operation data acquisition device 10 includes, for example, a distance sensor that acquires data regarding the distance to the golf ball. The distance sensor may include, for example, an ultrasonic sensor, a millimeter wave radar, a LIDAR (Light Detecting and Ranging), a depth sensor, and the like. Further, the operation data acquisition device 10 includes, for example, a trajectory measurement device that acquires data regarding the trajectory of a golf ball.

Further, for example, the motion data acquisition device 10 may acquire data relating to the evaluation by the instructor for the swing motion of the target person, as in the case of the first application example described above. The evaluation by the instructor includes, for example, a round evaluation based on at least one of the subject's movement during a round on a simulator simulating a golf course, the subject's thoughts, the result for the swing movement, and the like. In addition, the evaluation by the instructor includes, for example, motion evaluation regarding the flexibility and range of motion of the user's body part. The operation data acquisition device 10 includes, for example, a terminal that receives an input regarding evaluation from an instructor.

In the database 21 of the storage device 20, a plurality of data sets are registered by combining the swing data acquired in the past and the data related to the recommended club information corresponding to the swing data.

The operation data acquisition unit 301 acquires (reads) swing data acquired from the operation data acquisition device 10.

The learning unit 302 performs machine learning based on a plurality of data sets registered in the database 21 of the storage device 20 and generates (outputs) a learned model LM. The trained model LM can output recommended club information by inputting swing data.

The support information output unit 304 of the information processing device 30 uses the trained model LM to generate (output) recommended club information according to the swing motion of the target person based on the swing data input from the motion data acquisition unit 301. To do. As a result, the information processing device 30 can provide the recommended club information corresponding to the swing data of the target person to the user (for example, the target person himself / herself, a clerk, etc.) through the display device 36 or the like. Recommended club information includes, for example, information on the distance of the center of gravity of the club (head) that matches the swing motion of the target person, information on the height of the center of gravity, information on the angle of the center of gravity, information on whether the player is in a good condition or in hand, and information on weight. Including. Specifically, the recommended club information is "Recommended club functions are as follows. The distance of the center of gravity is short. The height of the center of gravity is appropriate. The condition is good. The weight is heavy." And "Recommended." The club has a long center of gravity, a low center of gravity, and a light weight (trade name). "

The usefulness selection unit 305 selects (determines) whether or not the information input from the support information output unit 304 is useful to the target person. The benefit selection unit 305 may select (determine) whether or not the recommended club information is useful to the target person, for example, according to the flowchart of FIG. In this case, the benefit selection unit 305, for example, has a golf ball flight distance, left-right blurring, etc. between the state before the provision of the recommended club information and the state in which the golf club specified in the recommended club information is used. To compare. Then, the usefulness selection unit 305 determines whether or not the recommended club information this time is useful to the target person based on whether or not the flight distance has been improved (improved) and the left and right blurring has been improved (reduced). Select (judgment).

When the benefit selection unit 305 selects (determines) that the recommended club information is not useful for the target person, the support information editing unit 306 responds to the input through the input device 37 of the clerk as a user. , Edit recommended club information.

The support information editing unit 306 may edit the recommended club information according to the flowchart of FIG. 6 described above. As a result, the support information editing unit 306 edits the recommended club information generated (output) by the support information output unit 304 in response to the input from the clerk through the input device 37, and is based on the edited recommended club information. , The database 21 can be updated. Therefore, the learning unit 302 can perform machine learning based on the data set including the edited recommended club information data and update the learned model LM. Therefore, the information processing device 30 can update the trained model LM in a manner capable of outputting more appropriate recommended club information according to the accumulation of the data set including the data related to the edited recommended club information. ..

Further, the support information editing unit 306 may edit the recommended club information, for example, according to the flowchart of FIG. 7 described above. As a result, the support information editing unit 306 edits and updates the recommended club information of the data set registered in the database 21 to appropriate contents for the swing data in response to the input from the clerk through the input device 37. can do. Therefore, the learning unit 302 can perform machine learning based on the data set including the edited recommended club information data and update the learned model LM. Therefore, the information processing device 30 can update the trained model LM in a manner capable of outputting more appropriate recommended club information in accordance with the update of the data regarding the recommended club information in the data set of the database 21.

<Third application example of information provision system>
In this example, the information providing system 1 provides the user with support information (swing motion improvement information) regarding improvement of the golf swing motion. The user is, for example, a target person who performs a swing motion or an instructor who guides the target person.

The motion data acquisition device 10 acquires motion data (swing data) related to the golf swing motion of the target person. For example, the motion data acquisition device 10 acquires swing data representing the movement of each part of a person during a swing motion. The swing data includes, for example, data regarding the position of each part of the human body during the swing motion. Further, the swing data includes, for example, data relating to the speed of each part of the human body during the swing motion. Further, the swing data includes, for example, data relating to the acceleration of each part of the human body during the swing operation. Further, the swing data includes, for example, data relating to the angle of each part (for example, a joint portion) of the human body during the swing motion. Further, the swing data includes, for example, data relating to the angular velocity of each part (joint portion) of the human body during the swing motion. Further, the swing data includes, for example, data relating to the angular acceleration of each part (joint portion) of the human body during the swing motion. Further, the swing data includes, for example, data on the surface pressure of the sole of a person during a swing motion (hereinafter, “foot pressure”). Further, the swing data includes, for example, data on the floor reaction force acting on a person during the swing operation. Further, the swing data includes data representing the degree of contribution (energy) to the head speed of each part of the person during the swing operation. Further, for example, the motion data acquisition device 10 acquires swing data representing the trajectory (ballistic) of the golf ball as a result of the swing motion of the target person. The swing data includes, for example, data regarding the flight distance of a golf ball. In addition, the swing data includes data related to left and right blurring. The motion data acquisition device 10 includes, for example, an inertial sensor attached to each part of the body of the subject. Further, the motion data acquisition device 10 includes, for example, an image pickup device that captures at least one of the swing motion of the target person and the trajectory of the golf ball and acquires moving image information. Further, the operation data acquisition device 10 includes, for example, a GPS sensor attached to a golf ball. Further, the operation data acquisition device 10 includes, for example, a distance sensor that acquires data regarding the distance to the golf ball. Further, the operation data acquisition device 10 includes, for example, a trajectory measurement device that acquires data regarding the trajectory of a golf ball.

Further, for example, the motion data acquisition device 10 may acquire data related to the evaluation by the instructor for the swing motion of the target person, as in the case of the first application example described above. The evaluation by the instructor includes, for example, a round evaluation based on at least one of the subject's movement during a round on a simulator simulating a golf course, the subject's thoughts, the result for the swing movement, and the like. In addition, the evaluation by the instructor includes, for example, motion evaluation regarding the flexibility and range of motion of the user's body part. The operation data acquisition device 10 includes, for example, a terminal that receives an input regarding evaluation from an instructor.

In the database 21 of the storage device 20, a plurality of data sets are registered by combining the swing data acquired in the past and the data related to the swing motion improvement information corresponding to the swing data.

The operation data acquisition unit 301 acquires (reads) swing data acquired from the operation data acquisition device 10.

The learning unit 302 performs machine learning based on a plurality of data sets registered in the database 21 of the storage device 20 and generates (outputs) a learned model LM. The trained model LM can input swing data and output swing motion improvement information.

The support information output unit 304 uses the trained model LM to generate (output) swing motion improvement information according to the swing motion of the target person based on the swing data input from the motion data acquisition unit 301. As a result, the information processing device 30 can provide the user (for example, the target person himself / herself, the instructor, etc.) with swing motion improvement information corresponding to the swing data of the target person through the display device 36 or the like.

The usefulness selection unit 305 selects (determines) whether or not the swing motion improvement information input from the support information output unit 304 is useful to the target person. The usefulness selection unit 305 may select (determine) whether or not the swing motion improvement information is useful to the target person, for example, according to the flowchart of FIG. In this case, the usefulness selection unit 305 compares, for example, the flight distance of the golf ball, the left-right blur, and the like between the state before the provision of the swing motion improvement information and the state after the provision of the swing motion improvement information. Then, the usefulness selection unit 305 determines whether or not the current swing motion improvement information is useful to the subject based on whether or not the flight distance has been improved (improved) and the left and right blurring has been improved (reduced). Is selected (judgment).

When the benefit selection unit 305 selects (determines) that the swing motion improvement information is not useful for the target person, the support information editing unit 306 inputs the information through the input device 37 of the instructor as a user. The swing motion improvement information is edited accordingly.

Further, the support information editing unit 306 may edit the swing motion improvement information, for example, according to the flowchart of FIG. 7 described above. As a result, the support information editing unit 306 edits the swing motion improvement information of the data set registered in the database 21 into appropriate contents for the swing data in response to the input from the instructor through the input device 37. , Can be updated. Therefore, the learning unit 302 can perform machine learning based on the data set including the edited data related to the swing motion improvement information, and update the learned model LM. Therefore, the information processing device 30 may update the trained model LM in a manner capable of outputting more appropriate swing motion improvement information in accordance with the update of the data related to the swing motion improvement information of the data set of the database 21. it can.

Further, as described above, the support information editing unit 306 may have the storage device 20 construct a new database instead of updating the data set included in the database 21. As a result, the data set of the existing database 21 can be retained. Therefore, the information processing device 30 can prepare a plurality of learned model LMs according to the instruction policy for each instructor along with a plurality of databases (data sets) according to the instruction policy for each instructor. Therefore, the information processing device 30 can provide swing motion improvement information according to the guidance policy for each instructor.

<Other application examples of information provision system>
The information providing system 1 may provide the user with support information (movement improvement information) regarding improvement of movements of a specific sport other than golf and support information (recommended equipment information) regarding equipment recommended according to the movement.

For example, the information providing system 1 may provide the user with support information (motion improvement information) regarding improvement of the tennis motion and support information (equipment support information) regarding the tennis racket recommended according to the motion.

Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and improvements can be made within the scope of the gist described in the claims.

1 Information provision system 10 Operation data acquisition device 20 Storage device 21 Database 30 Information processing device 37 Input device (input unit)
301 Operation data acquisition unit (acquisition unit)
302 Learning unit 303 Storage unit 304 Support information output unit 305 Benefit selection unit (judgment unit)
306 Support Information Editorial Department (Editorial Department)
LM trained model

Claims (12)

An input unit that accepts predetermined input and
An acquisition unit that acquires data related to the behavior of the target person,
Acquired by the acquisition unit using a trained model that has been machine-learned based on a plurality of data sets that combine data related to human movements and data related to support information for humans to perform sports according to their motor characteristics. A support information output unit that outputs the support information according to the movement of the target person based on the data to be generated, and a support information output unit.
A determination unit that determines whether or not the support information output from the support information output unit has appropriate contents for the data acquired by the acquisition unit.
When the determination unit determines that the content is not appropriate, the support information output from the support information output unit and the acquisition of the plurality of data sets according to the input received by the input unit. It includes an editorial unit that edits at least one of the support information of the dataset including data on the movement of a person that is relatively close to the data acquired by the unit.
Information processing device. A learning unit that performs the machine learning and generates the trained model is provided.
The learning unit updates the trained model based on the plurality of data sets updated by the support information edited by the editorial unit.
The information processing device according to claim 1. A first storage unit for storing the trained model is provided.
The first storage unit stores the learned model before the update by the learning unit and the learned model after the update.
The information processing device according to claim 2. A second storage unit for storing the editing history of the support information by the editorial unit is provided.
The information processing device according to any one of claims 1 to 3. The determination unit determines whether or not the support information output from the support information output unit is appropriate for the data acquired by the acquisition unit in response to the input received by the input unit. To do
The information processing device according to any one of claims 1 to 4. The determination unit includes data acquired by the acquisition unit before the support information output unit outputs the support information, and data acquired by the acquisition unit after the support information output unit outputs the support information. Based on the comparison, it is determined whether or not the support information output from the support information output unit has appropriate contents for the data acquired by the acquisition unit.
The information processing device according to any one of claims 1 to 4. The movement is a golf swing movement.
The information processing device according to any one of claims 1 to 6. The data related to the movement includes an inertia sensor attached to at least one of the club and the person performing the swing movement, an imaging device that images at least one of the swing movement and the trajectory of the ball, a body part of the person performing the swing movement, and at least one of the balls. Acquired by at least one of a distance sensor that acquires data on the distance to and from, and a measuring device that measures the trajectory of the ball.
The information processing device according to claim 7. The data relating to the movement includes data relating to at least one of a club pass, an attack angle, a face angle, a face rotation, a tame, a release timing, a swing plane during a backswing, and a swing plane during a downswing.
The information processing device according to claim 7 or 8. The data related to the movement includes the position, velocity, acceleration, angular acceleration, surface pressure of the sole of the foot, the floor reaction force acting on the person performing the swing movement, the trajectory of the ball, and the swing movement. Includes data on at least one of at least one captured image of the performer and the ball,
The information processing device according to any one of claims 7 to 9. An information processing method executed by an information processing device provided with an input unit that accepts a predetermined input.
The acquisition step to acquire the data related to the behavior of the target person, and
Acquired in the acquisition step using a trained model that has been machine-learned based on multiple datasets that combine data on human movements and data on support information for humans to play sports according to their motor characteristics. A support information output step that outputs the support information according to the movement of the target person based on the data to be performed, and a support information output step.
A determination step for determining whether or not the support information output in the support information output step is appropriate for the data acquired in the acquisition step, and a determination step.
When it is determined that the content is not appropriate in the determination step, the support information output in the support information output step and the acquisition of the plurality of data sets are obtained according to the input received by the input unit. Includes an edit step that edits at least one of the support information in a dataset that contains data about human behavior that is relatively close to the data acquired in the step.
Information processing method. An information processing device equipped with an input unit that accepts a predetermined input
The acquisition step to acquire the data related to the behavior of the target person, and
Acquired in the acquisition step using a trained model that has been machine-learned based on multiple datasets that combine data on human movements and data on support information for humans to play sports according to their motor characteristics. A support information output step that outputs the support information according to the movement of the target person based on the data to be performed, and a support information output step.
A determination step for determining whether or not the support information output in the support information output step is appropriate for the data acquired in the acquisition step, and a determination step.
When it is determined that the content is not appropriate in the determination step, the support information output in the support information output step and the acquisition of the plurality of data sets are obtained according to the input received by the input unit. To execute, an editing step of editing at least one of the support information of the dataset containing data on human behavior that is relatively close to the data acquired in the step.
program.

Images