WO2021251222A1 - Learning device, presentation device, and technique acquisition method - Google Patents

Learning device, presentation device, and technique acquisition method Download PDF

Info

Publication number
WO2021251222A1
WO2021251222A1 PCT/JP2021/020914 JP2021020914W WO2021251222A1 WO 2021251222 A1 WO2021251222 A1 WO 2021251222A1 JP 2021020914 W JP2021020914 W JP 2021020914W WO 2021251222 A1 WO2021251222 A1 WO 2021251222A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
user behavior
behavior
correct answer
level
Prior art date
Application number
PCT/JP2021/020914
Other languages
French (fr)
Japanese (ja)
Inventor
圭祐 千田
Original Assignee
ソニーグループ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニーグループ株式会社 filed Critical ソニーグループ株式会社
Publication of WO2021251222A1 publication Critical patent/WO2021251222A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B69/00—Training appliances or apparatus for special sports
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00—Teaching not covered by other main groups of this subclass
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02—Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
    • G09B7/04—Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation

Definitions

  • This disclosure relates to a learning device, a presentation device, and a technique acquisition method.
  • Proposed a voice data conversion device that can experience a good vocal that is close to the model with the person's singing voice by reverse-correcting the voice data sung according to the accompaniment data of the song and the model voice data by the difference data compared with the model voice data.
  • the content of the advice changes according to the action content of the user, and a learning device, a presentation device, and a technology acquisition method capable of efficient technology acquisition / learning are proposed.
  • one form of the learning device includes a behavior level determination unit that determines a user's behavior level based on user behavior data, and an behavior level determination that determines the current user's behavior level.
  • the predictive transformation unit that predictively transforms the user behavior data into correct answer / model data superior to the behavior level of the behavior level determination result in order to reach the ideal behavior result, the correct answer / model data, and the user. It is provided with a presentation unit that presents the difference from the behavior data.
  • FIG. 1 is a diagram showing an example of data processing in the data processing system 1 according to the first embodiment of the present disclosure.
  • the data processing according to the embodiment of the present disclosure is realized by the data processing system 1 including the data processing device 100 and the data processing device 200.
  • FIG. 1 describes an outline of data processing realized by the data processing system 1.
  • the data processing device 100 functions as a learning device or a presentation device.
  • the data processing device 100 converts the current user's action level determination result from the correct answer / model data learned by machine learning into correct answer / model data superior to the action level of the action level determination result in order to reach the ideal action result. It is an information processing device that predicts and transforms and presents the difference between correct answer / model data and user behavior data in an easy-to-understand manner.
  • the data processing device 100 is used, for example, for the purpose of learning technology.
  • Targets of skill acquisition include painting, writing, playing musical instruments, singing, and sports. Further, the data processing device 100 is used for learning, for example. Targets of learning include languages such as foreign language pronunciation.
  • a data conversion device 200 that provides data related to user behavior to the data processing device 100
  • the user behavior is converted into image data.
  • the data conversion device 200 is not limited to the camera and may be various devices as long as it can provide the data related to the user behavior to the data processing device 100.
  • the data conversion device 200 may be a UAV (Unmanned Aerial Vehicle) such as a drone or various sensors.
  • UAV Unmanned Aerial Vehicle
  • the data conversion device 200 converts user behavior into sensing data (sensor reaction value, image analysis result, etc.).
  • the data conversion device 200 converts a product such as a painting or a calligraphy into image data by shooting, scanning, or the like. Further, the data conversion device 200 records performances such as musical instrument performances and songs, and converts them into audio data which is waveform information. In addition, the data conversion device 200 converts images into image data, such as by photographing a posture in sports. Further, the data conversion device 200 records foreign language pronunciation and the like and converts it into voice data which is waveform information.
  • the data processing device 100 includes an action level determination unit 30, a model prediction transformation unit 40 which is a prediction transformation unit, an advice presentation unit 50 which is a presentation unit, and a learned database 60. Be prepared.
  • the learned database 60 includes a database 60a for determining an action level and a database 60b for generating advice.
  • the database 60a for determining the action level and the database 60b for generating advice are paired for each action level Lv.
  • the database 60a for determining the behavior level stores user behavior data of the user for each behavior level Lv.
  • the database 60b for generating advice stores correct answer / model data that is advice for each action level Lv according to the content of user behavior.
  • the action level determination unit 30 and the model prediction transformation unit 40 learn the correct answer / model data prepared in advance by using machine learning, predictively generate a large number of correct answers / model data, and store them in the learned database 60.
  • the action level determination unit 30 estimates the user's action level based on the user action data by using the database 60a for action level determination for each action level Lv. In the case of clear determination, the action level determination unit 30 shifts to the action level Lv one step higher in the database 60a for action level determination. The action level determination unit 30 repeatedly determines the user's action level Lv until an NG determination is reached.
  • the model prediction transformation unit 40 selects a database 60b for advice generation paired with the behavior level Lv, which is the behavior level determination result by the behavior level determination unit 30, and uses the correct answer / model data according to the user's behavior level Lv. Based on this, user behavior data that is predicted and transformed from user behavior data is generated.
  • the advice presentation unit 50 for example, superimposes and visualizes the difference between the user behavior data generated by transforming the model prediction transformation unit 40 and the correct answer / model data on the user behavior data and presents it to the user.
  • FIG. 2 is a flowchart showing an example of the flow of data processing from the action level determination to the advice presentation in the data processing system 1 according to the first embodiment of the present disclosure.
  • the data processing device 200 first converts the user behavior into data (step S1).
  • the action level determination unit 30 starts determining the action level Lv of the user. Specifically, the behavior level determination unit 30 determines the user behavior data using the behavior level LvN (initial value is 0) of the database 60a for behavior level determination (step S2).
  • the action level determination unit 30 raises the action level Lv of the determination target by 1 (step S4).
  • the determination result is NG (NG in step S3), the action level determination unit 30 completes the determination.
  • the model prediction transformation unit 40 When the action level Lv determination by the action level determination unit 30 is completed, the model prediction transformation unit 40 generates correct answer / model data according to the determined action level Lv (step S5).
  • the advice presentation unit 50 presents the correct answer / model data to the user (step S6).
  • Patent Document 1 Japanese Unexamined Patent Publication No. 2001-125582
  • the voice data sung according to the accompaniment data of the song and the model voice data is reverse-corrected by the difference data compared with the model voice data, and the singing voice of the person himself / herself is used.
  • a voice data conversion device that allows you to experience good vocals that are close to a model has been proposed.
  • Patent Document 2 Japanese Unexamined Patent Publication No. 6-149145
  • Patent Document 2 Japanese Unexamined Patent Publication No. 6-149145
  • An educational method has been proposed that automatically discovers and evaluates the learning level.
  • the data processing system 1 can predict any user input without retaining the correct answer / model data itself.
  • the data processing system 1 can present elements that have a large difference in common to the data group, not individual differences from the correct answer / model data group, so it captures the technically essential ones and sets the technical stage. It is easy to show. Further, in the data processing system 1, if only the samples of the correct answer / model data group and the user input group are collected, the leveling work is automatically performed by machine learning, so that there is no need to manually level.
  • the data processing system 1 is constructed even if the system builder does not have knowledge of the technical field because the leveling rule is automatically and stepwise constructed from the part where there is a statistically large difference in the process of machine learning. It is possible. This point will be described below.
  • the data processing system 1 can automatically generate correct answer / model data (learning data) used for learning by CycleGAN, which is a kind of machine learning that utilizes multiple GANs (Generative Adversarial Network). can.
  • FIG. 3 is a diagram showing a configuration example of a data processing system according to the first embodiment of the present disclosure.
  • the data processing system 1 shown in FIG. 3 may include a plurality of data processing devices 100.
  • the data processing device 100 converts the current user's action level determination result from the correct answer / model data learned by machine learning into correct answer / model data superior to the action level of the action level determination result in order to reach the ideal action result. It is an information processing device (computer) that predicts and transforms and presents the difference between correct answer / model data and user behavior data in an easy-to-understand manner.
  • the data conversion device 200 is a computer that provides data related to user behavior to the data processing device 100.
  • the data digitizing device 200 is a camera having an imaging function for imaging a user.
  • the data conversion device 200 may be various sensors that convert user behavior into sensing data (sensor reaction value, image analysis result, etc.).
  • the data conversion device 200 may be a moving body such as a UAV such as a drone or a vehicle such as an automobile.
  • the data conversion device 200 has an image pickup function such as an image sensor (imager), moves to a position corresponding to a request from the data processing device 100, captures an image or a moving image at that position, and captures the captured image or moving image. It may be transmitted to the data processing device 100.
  • imager image sensor
  • the data conversion device 200 may be any device as long as the processing in the embodiment can be realized.
  • the data conversion device 200 may be, for example, a device such as a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, or a PDA (Personal Digital Assistant).
  • the data conversion device 200 may be a wearable terminal (Wearable Device) or the like that the user can wear.
  • the data digitizing device 200 may be a wristwatch type terminal, a glasses type terminal, or the like.
  • the data digitizing device 200 may be a so-called home electric appliance such as a television or a refrigerator.
  • the data conversion device 200 may be a robot that interacts with a human (user), such as a smart speaker, an entertainment robot, or a domestic robot. Further, the data conversion device 200 may be a device arranged at a predetermined position such as a digital signage.
  • FIG. 4 is a diagram showing a configuration example of the data processing device 100 according to the first embodiment of the present disclosure.
  • the data processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.
  • the data processing device 100 includes an input unit (for example, a keyboard, a mouse, etc.) that receives various operations from the administrator of the data processing device 100, and a display unit (for example, a liquid crystal display, etc.) for displaying various information. You may have.
  • the communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. Then, the communication unit 110 is connected to the network N (see FIG. 3) by wire or wirelessly, and transmits / receives information to / from another information processing device such as the data conversion device 200. Further, the communication unit 110 may send and receive information to and from the data conversion device 200.
  • a NIC Network Interface Card
  • the storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. As shown in FIG. 4, the storage unit 120 according to the embodiment includes a data information storage unit 121 and an action level information storage unit 122.
  • the data information storage unit 121 stores sample data used for learning correct answer / model data (learning data).
  • the data information storage unit 121 has a correct answer / model sample data group that is a sample of correct answer / model data, and a user behavior sample data group that is a sample of user behavior data.
  • the action level information storage unit 122 stores information on the user's action level for each action level Lv and the advice for each action level Lv according to the content of the user action.
  • the action level information storage unit 122 has a database 60a for determining the action level that stores the action level of the user for each action level Lv, and correct answer / model data that provides advice for each action level Lv according to the content of the user action.
  • the database 60b for generating advice is stored.
  • control unit 130 for example, a program stored inside the data processing device 100 (for example, an information processing program such as a data processing program according to the present disclosure) by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like is used. It is realized by executing RAM (Random Access Memory) etc. as a work area. Further, the control unit 130 is a controller, and is realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).
  • ASIC Application Specific Integrated Circuit
  • FPGA Field Programmable Gate Array
  • control unit 130 includes an action level determination unit 30, a model prediction transformation unit 40, and an advice presentation unit 50, and realizes or executes the functions and actions of information processing described below. do.
  • the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 4, and may be any other configuration as long as it is configured to perform information processing described later.
  • connection relationship of each processing unit included in the control unit 130 is not limited to the connection relationship shown in FIG. 4, and may be another connection relationship.
  • the action level determination unit 30 and the model prediction transformation unit 40 use machine learning to learn correct answer / model data, and predict and generate a large number of correct answer / model data.
  • the action level determination unit 30 and the model prediction transformation unit 40 perform learning processing based on the learning data (correct answer / model data) stored in the data information storage unit 121.
  • the behavior level determination unit 30 and the model prediction transformation unit 40 perform learning processing using the learning data stored in the data information storage unit 121, thereby performing a learning process on the user's behavior level and user behavior for each behavior level Lv. Learn (predictively generate) advice for each action level Lv according to the content.
  • the learning method by the action level determination unit 30 and the model prediction transformation unit 40 is not particularly limited to CycleGAN, but for example, learning data (correct answer / model data) is prepared, and the learning data is calculated based on the multi-layer neural network. You may input to the model and learn. Further, for example, a method based on DNN (Deep Neural Network) such as CNN (Convolutional Neural Network) or 3D-CNN may be used.
  • DNN Deep Neural Network
  • CNN Convolutional Neural Network
  • 3D-CNN 3D-CNN
  • a method based on (Long Short-Term Memory units) may be used.
  • the behavior level determination unit 30 acquires the user behavior data and determines the user behavior level based on the user behavior data.
  • the model prediction transformation unit 40 is based on the behavior level determination result of determining the current user's behavior level, and in order to reach the ideal behavior result, the user behavior data is a correct answer / model superior to the behavior level of the behavior level determination result. Predictively transforms into data (for example, correct answer / model data equivalent to the action level one step higher).
  • the advice presentation unit 50 presents the difference between the correct answer / model data and the user behavior data in an easy-to-understand manner in order to give specific advice to the user's behavior level determination result.
  • the data processing apparatus 100 may use a model (network) in the form of a neural network (NN) such as a deep neural network (DNN).
  • NN neural network
  • DNN deep neural network
  • the data processing device 100 is not limited to the neural network, and may use various types of models (functions) such as a regression model such as SVM (Support Vector Machine).
  • SVM Small Vector Machine
  • the data processing apparatus 100 may use a model (function) of any format.
  • the data processing device 100 may use various regression models such as a non-linear regression model and a linear regression model.
  • FIG. 5 is a diagram showing an example of database learning processing
  • FIG. 6 is a flowchart showing an example of the flow of database learning processing.
  • the data processing device 100 performs database learning processing by the model prediction transformation unit 40 and the action level determination unit 30.
  • the action level determination unit 30 includes a transformation / correct answer determination unit 301, a user behavior prediction transformation unit 302, and a prediction / user behavior determination unit 303.
  • the transformation / correct answer determination unit 301 distinguishes between the user behavior data obtained by predicting and transforming the user behavior by the model prediction transformation unit 40 based on the correct answer / model data and the original correct answer / model data.
  • the user behavior prediction transformation unit 302 predicts and transforms the correct answer / model data into the user behavior data. That is, the user behavior prediction transformation unit 302 predicts and transforms from good to bad.
  • the prediction / user behavior determination unit 303 distinguishes between the user behavior data actually predicted and transformed by the user behavior prediction transformation unit 302 and the sample user behavior data.
  • the data processing device 100 first prepares. First, the data processing device 100 learns the behavior level Lv poorly ⁇ well. It will be described in detail below.
  • the data processing device 100 selects user behavior data from the user behavior sample data group stored in advance and passes it to the model prediction transformation unit 40 (step S11).
  • the data processing device 100 predicts and transforms the user behavior data based on the correct answer / model data by using the model predictive transformation unit 40 (step S12). That is, the user behavior prediction transformation unit 302 predicts and transforms poorly ⁇ well.
  • the data processing device 100 uses the transformation / correct answer determination unit 301 to obtain the user behavior data actually predicted and transformed by the model prediction transformation unit 40 learned in the process (1) and the original correct answer / model data. The distinction is determined and learning is performed (step S13).
  • the data processing device 100 learns the behavior level Lv from good to bad. It will be described in detail below.
  • the data processing device 100 selects the correct answer / model data from the correct answer / model sample data group stored in advance and passes it to the user behavior prediction transformation unit 302 (step S14). Next, the data processing device 100 predicts and transforms the correct answer / model data into the user behavior data based on the user behavior data by using the user behavior prediction transformation unit 302 (step S15). That is, the user behavior prediction transformation unit 302 predicts and transforms from good to bad.
  • the data processing device 100 uses the prediction / user behavior determination unit 303 to obtain the user behavior data actually transformed by the user behavior prediction transformation unit 302 learned in the process (3), the original user behavior data, and the original user behavior data. (Step S16).
  • the data processing device 100 completes the preparation.
  • the data processing device 100 starts iterative learning.
  • the data processing device 100 transforms the user behavior data into correct answer / model data by the model prediction transformation unit 40 (step S21).
  • the data processing device 100 retransforms the result of transforming the user behavior data into the correct answer / model data by the model prediction transformation unit 40 into the user behavior data by the user behavior prediction transformation unit 302, and obtains the user behavior data ( Step S22).
  • the result of transforming the user behavior data into the correct answer / model data by the model prediction transformation unit 40 causes the transformation / correct answer determination unit 301 to make a mistake, and the user behavior data is converted into the correct answer / model data.
  • the model prediction transformation unit 40 transforms the user behavior data into correct answer / model data and trains the transformed result so that the result of retransformation by the user behavior prediction transformation unit 302 matches the original user behavior data (step). S23).
  • the data processing device 100 records the correct answer / model data, which is the result of learning in the model prediction / transforming unit 40, in the database 60b for generating advice of the learned database 60 (step S24).
  • the data processing device 100 predicts and transforms the correct answer / model data into the user behavior data by the user behavior prediction transformation unit 302 (step S25). Subsequently, the data processing device 100 transforms the result of transforming the correct answer / model data into the user behavior data by the user behavior prediction transformation unit 302, and retransforms the result by the model prediction transformation unit 40 (step S26).
  • the data processing device 100 further models the result of transforming the correct answer / model data into the user behavior data so that the transformation / correct answer determination unit 301 makes a mistake in the result of transforming the correct answer / model data into the user behavior data.
  • the user behavior prediction transformation unit 302 is trained so that the result of re-transformation by the prediction transformation unit 40 matches the original correct answer / model data (step S27).
  • the data processing device 100 distinguishes and learns the correct answer / model data actually predicted by the model prediction transformation unit 40 learned in step S23 and the original correct answer / model data in the transformation / correct answer determination unit 301 ( Step S28).
  • the data processing device 100 records the determination learning result in the database 60a for determining the action level of the learned database 60 (step S29).
  • the data processing device 100 distinguishes and determines between the user behavior data actually deformed by the user behavior prediction transformation unit 302 learned in step S27 and the original user behavior data by the prediction / user behavior determination unit 303, and learns. (Step S30).
  • the data processing device 100 repeats processing (5) to processing (6). That is, the data processing apparatus 100 alternately repeats the processes of steps S21 to S27 and the processes of steps S28 to S30 described above. The data processing device 100 sequentially holds data in the process of repeated learning.
  • the data processing device 100 first learns to reduce a large difference by machine learning, and then repeatedly learns to reduce a detailed difference in order.
  • the user behavior data is image data (example: painting, calligraphy, posture of sports)
  • the advice presentation unit 50 displays (a) both the correct answer and the model data, which are the ideal behavior results of the behavior level Lv one step higher than the user behavior data for determining the behavior level of the user and the user behavior data. Display on the display / Display side by side on the display.
  • FIG. 7 is a diagram showing an example of display.
  • the user behavior data a for determining the user's behavior level the correct answer / model data b of the behavior level Lv one step higher, and the correct answer of the behavior level Lv one step higher are shown.
  • -It shows how the model data c and the model data c are displayed side by side.
  • the advice presentation unit 50 is 1.
  • the correct answer / model data which is the ideal behavior result of the behavior level Lv n (n> 1) higher than the user behavior data, is displayed on the display / side by side. Display in the section.
  • FIG. 8 is a diagram showing an example of display.
  • the user behavior data a for determining the behavior level of the user the correct answer / model data b of the behavior level Lv one step higher, and the ideal behavior level Lv one step higher than n (n> 1). It shows how the correct answer / model data c, which is the result of the action of, is superimposed and displayed.
  • the advice presentation unit 50 is 1. Or 2. Of the display methods of, highlight the part where the difference between the user behavior data and (b) and (c) is large. Highlighting methods include changing the color, changing the brightness level, and (in the case of video) slowing down the playback speed.
  • FIG. 9 is a diagram showing an example of display.
  • the user behavior data a for determining the user behavior level and the correct answer / model data b which is the ideal behavior result of the behavior level Lv one step higher are displayed in an overlapping manner, and the user behavior is displayed.
  • the thick line d highlights the part where the difference between the data and the action level Lv one step higher is large.
  • the advice presentation unit 50 is a correct answer, which is (a) the user behavior data for determining the user's behavior level, and (d) the ideal behavior result of the behavior level Lv one step higher than the user behavior data and the user behavior data.
  • the difference from the model data is displayed on the display unit in an overlapping manner / side by side on the display unit.
  • FIG. 10 is a diagram showing an example of display.
  • the difference e between the user behavior data a for determining the user behavior level, the user behavior data a, and the correct answer / model data which is the ideal behavior result of the behavior level Lv one step higher, is shown. It shows the state of overlapping and displaying.
  • the advice presentation unit 50 is 4.
  • (e) the difference between the user behavior data that determines the user behavior level and the correct answer / model data that is the ideal behavior result of the behavior level Lv n (n> 1) higher than the user behavior data. Are displayed on the display unit in an overlapping manner / displayed side by side on the display unit.
  • the advice presentation unit 50 is 4. Or 5. Of the display methods of, highlight the part where the difference between the user behavior data and (b) and (c) is large.
  • the highlighting method includes changing the color, changing the brightness level, and (in the case of a movie) slowing down the playback speed.
  • the user behavior data is voice data (example: in the case of playing a musical instrument or learning a language) (example: in the case of playing a musical instrument or learning a language) will be described.
  • the advice presentation unit 50 can take the same advice format as when the user behavior data is image data.
  • the advice presenting unit 50 can take the following form of advice.
  • the advice presentation unit 50 sequentially reproduces both (a) the user behavior data for determining the user's behavior level and (b) the correct answer / model data of the behavior level Lv one step higher than the user behavior data. ..
  • the advice presentation unit 50 is 1. In addition to (c), the correct answer / model data of the behavior level Lv n (n> 1) higher than the user behavior data is reproduced in order.
  • the advice presentation unit 50 is 1. Or 2. Of the presentation methods of, the volume is increased / the reproduction speed is slowed down in the place where the difference between the user behavior data and (b) and (c) is large.
  • the correct answer / model data that provides step-by-step advice for the user's behavior to reach the ideal behavior result (professional / master's model) is predicted, and the prediction result is used to predict the correct answer / model data.
  • Correct answer / model data that is at least one step higher than the behavior level of the user behavior level judgment result is shown by adapting (superimposing) to the user behavior level judgment result. By doing so, correct answer / model data that can be used as advice according to the content and level of each user behavior can be obtained, and efficient skill acquisition / learning becomes possible.
  • the present embodiment it is possible to realize a system that concretely presents a method of improvement in stages. Further, according to the present embodiment, it is possible to realize a system in which the content of the advice is changed according to the content of the user's behavior instead of the standard advice, and efficient skill acquisition / learning is possible.
  • the learning progress is used for leveling in order to give stepwise advice, but in the second embodiment, the sampling interval of the user behavior data and the correct answer / model data is coarse (n). ) ⁇ Dense (1) is changed step by step, which is different from the first embodiment.
  • FIG. 11 is a diagram showing an example of database learning processing according to the data processing system 1 according to the second embodiment of the present disclosure
  • FIG. 12 is a diagram showing database learning according to the data processing system 1 according to the second embodiment of the present disclosure. It is a flowchart which shows an example of the processing flow.
  • the data processing apparatus 100 sets the sampling interval i (decreases by 1 from n to 1).
  • the processes (1) to (6) executed by the data processing device 100 are the same as those in the first embodiment of the present disclosure.
  • the data processing device 100 obtains the correct answer / model data, which is the result of learning in the model prediction transformation unit 40 of step S24 described in the first embodiment of the present disclosure, in the trained database 60.
  • the process of recording in the database 60b for producing advice is not performed.
  • the data processing device 100 records the determination learning result of step S29 described in the first embodiment of the present disclosure in the database 60a for determining the action level of the learned database 60. Do not do.
  • the data processing apparatus 100 records the learning result of the model prediction transformation unit 40 in the database 60b for advice generation.
  • the learning result of the prediction / user action determination unit 303 is recorded in the database 60a for action level determination (step S31).
  • the data processing apparatus 100 sets the sampling interval to n-1 and changes the data sampling interval in the dense direction, and repeatedly executes the processes (1) to (6) and the process (8).
  • the same result can be obtained by blocking the high-frequency information (fine) information of the data with a band limiting filter such as LPF (Low-Pass Filter).
  • LPF Low-Pass Filter
  • the behavior level determination unit 30 and the model prediction transformation unit 40 of the data processing device 100 present advice step by step by gradually setting the sampling interval of the user behavior data. Can be done.
  • the data processing device 100 may be a camera, a smartphone, a television, a car, a drone, a robot, or the like. As described above, the data processing device 100 may be a terminal device that autonomously collects learning data having a high degree of influence.
  • each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of them may be functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured.
  • the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.
  • the effects described herein may be at least one.
  • the learning device includes an action level determination unit, a prediction deformation unit, and a presentation unit.
  • the behavior level determination unit determines the user's behavior level based on the user behavior data.
  • the predictive transformation unit predictively transforms the user behavior data into correct answer / model data for reaching the ideal behavior result based on the behavior level determination result of determining the current user behavior level.
  • the presentation unit presents the difference between the correct answer / model data and the user behavior data.
  • the learning device predicts the correct answer / model data that provides step-by-step advice for the user's behavior to reach the ideal behavior result (professional / master's model), and predicts the correct answer / model data. From the result, the difference between the correct answer / model data and the user behavior data, which is at least one step higher than the behavior level of the user's behavior level judgment result, is presented. By doing so, correct answer / model data that can be used as advice according to the content and level of each user behavior can be obtained, and efficient skill acquisition / learning becomes possible.
  • the behavior level determination unit and the prediction transformation unit pair the behavior level determination result in the behavior level determination unit with the correct answer / model data in the prediction transformation unit by machine learning, and learn alternately. Therefore, the behavior level can be automatically leveled.
  • the predictive transformation unit combines multiple results during learning leading to learning convergence in machine learning as correct answer / model data. Therefore, advice can be presented step by step.
  • the behavior level judgment unit and the prediction transformation unit gradually set the sampling interval of user behavior data. Therefore, advice can be presented step by step.
  • the predictive transformation unit sets the ideal action result as the action result equivalent to one or more levels of the action level judgment result of the user. Therefore, it is possible to specifically present a method for improving step by step.
  • the presentation device includes a presentation unit.
  • the presentation unit includes the user behavior data and the correct answer / model data obtained by predicting and transforming the user behavior data to reach the ideal behavior result based on the behavior level judgment result in which the user behavior level is determined based on the user behavior data. Show the difference between.
  • the presentation device predicts the correct answer / model data that provides step-by-step advice for the user's behavior to reach the ideal behavior result (professional / master's model).
  • correct answer / model data that can be used as advice according to the content and level of each user behavior can be obtained, and efficient skill acquisition / learning becomes possible.
  • the presentation unit displays the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in an overlapping manner or side by side. .. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
  • the presentation unit further displays the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in an overlapping manner or side by side. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
  • the presentation section highlights the points where there is a large difference between the correct answer / model data and the user behavior data. Therefore, it is possible to confirm the part where there is a large difference between the correct answer / model data, which is the advice according to the content and level of each user behavior, and it is possible to efficiently acquire and learn the technique.
  • the presentation unit When the user behavior data is image data or voice data, the presentation unit superimposes the difference between the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data and the user behavior data. Display or display side by side. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
  • the presentation unit further displays the difference between the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data and the user behavior data, or displays them side by side. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
  • the presentation section highlights the points where there is a large difference between the correct answer / model data and the user behavior data. Therefore, it is possible to confirm the part where there is a large difference between the correct answer / model data, which is the advice according to the content and level of each user behavior, and it is possible to efficiently acquire and learn the technique.
  • the presentation unit reproduces the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in order. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
  • the presentation unit further reproduces the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in order. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
  • the presentation unit raises the volume or slows down the playback speed of the correct answer / model data where there is a large difference from the user behavior data. Therefore, it is possible to confirm the part where there is a large difference between the correct answer / model data, which is the advice according to the content and level of each user behavior, and it is possible to efficiently acquire and learn the technique.
  • the technique acquisition method includes an action level determination step, a predictive transformation step, and a presentation step.
  • the behavior level determination step determines the user's behavior level based on the user behavior data.
  • the prediction transformation step predicts the user behavior data to the correct answer / model data superior to the behavior level of the behavior level judgment result in order to reach the ideal behavior result based on the behavior level judgment result of judging the current user behavior level. transform.
  • the presentation step presents the difference between the correct answer / model data and the user behavior data.
  • the technology acquisition method related to this disclosure predicts correct answer / model data that will be step-by-step advice for the user's behavior to reach the ideal behavior result (professional / master's model). From the prediction result, the difference between the correct answer / model data and the user behavior data, which is at least one level of advice on the behavior level of the user behavior level judgment result, is presented. By doing so, correct answer / model data that can be used as advice according to the content and level of each user behavior can be obtained, and efficient skill acquisition / learning becomes possible.
  • FIG. 13 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of information processing devices such as a data processing device 100 and a data processing device 200.
  • the computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600.
  • Each part of the computer 1000 is connected by a bus 1050.
  • the CPU 1100 operates based on the program stored in the ROM 1300 or the HDD 1400, and controls each part. For example, the CPU 1100 expands the program stored in the ROM 1300 or the HDD 1400 into the RAM 1200, and executes processing corresponding to various programs.
  • the ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started, a program depending on the hardware of the computer 1000, and the like.
  • BIOS Basic Input Output System
  • the HDD 1400 is a computer-readable recording medium that non-temporarily records a program executed by the CPU 1100 and data used by the program.
  • the HDD 1400 is a recording medium for recording an information processing program according to the present disclosure, which is an example of program data 1450.
  • the communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (for example, the Internet).
  • the CPU 1100 receives data from another device or transmits data generated by the CPU 1100 to another device via the communication interface 1500.
  • the input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000.
  • the CPU 1100 receives data from an input device such as a keyboard or mouse via the input / output interface 1600. Further, the CPU 1100 transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. Further, the input / output interface 1600 may function as a media interface for reading a program or the like recorded on a predetermined recording medium (media).
  • the media is, for example, an optical recording medium such as DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk), a magneto-optical recording medium such as MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.
  • an optical recording medium such as DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk)
  • a magneto-optical recording medium such as MO (Magneto-Optical disk)
  • tape medium such as DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk)
  • MO Magneto-optical disk
  • the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 and the like by executing the information processing program loaded on the RAM 1200.
  • the information processing program according to the present disclosure and the data in the storage unit 120 are stored in the HDD 1400.
  • the CPU 1100 reads the program data 1450 from the HDD 1400 and executes the program, but as another example, these programs may be acquired from another device via the external network 1550.
  • the present technology can also have the following configurations.
  • (1) The behavior level judgment unit that judges the user's behavior level based on the user behavior data, Prediction that the user behavior data is predicted and transformed into correct answer / model data superior to the behavior level of the behavior level judgment result in order to reach the ideal behavior result based on the behavior level judgment result of judging the current user behavior level. Deformed part and A presentation unit that presents the difference between the correct answer / model data and the user behavior data, A learning device characterized by being equipped with.
  • the behavior level determination unit and the prediction transformation unit alternately learn by pairing the behavior level determination result in the behavior level determination unit and the correct answer / model data in the prediction transformation unit by machine learning. , The learning device according to (1).
  • the predictive transformation unit combines a plurality of results during learning leading to learning convergence in the machine learning as the correct answer / model data.
  • the learning device according to (2). (4)
  • the behavior level determination unit and the prediction transformation unit gradually set the sampling interval of the user behavior data.
  • the learning device according to (1) or (2). (5)
  • the predictive transformation unit sets the ideal action result as an action result corresponding to one or more steps of the action level determination result of the user.
  • the learning device according to any one of (1) to (4).
  • (6) The difference between the user behavior data and the correct answer / model data obtained by predicting and transforming the user behavior data to reach the ideal behavior result based on the behavior level judgment result in which the user behavior level is determined based on the user behavior data. Equipped with a presentation unit to present A presentation device characterized by that.
  • the presentation unit displays the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in an overlapping manner. Or display side by side, The presentation device according to (6).
  • the presentation unit further displays the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in an overlapping manner or side by side.
  • the presentation unit highlights the points where the difference between the correct answer / model data and the user behavior data is large.
  • the presentation unit When the user behavior data is image data or voice data, the presentation unit includes the user behavior data, the user behavior data, and the correct answer / model data of the behavior level one step higher than the user behavior data. Display the differences in layers or side by side, The presentation device according to (6). (11) The presentation unit further displays or side by side the difference between the user behavior data and the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data. The presentation device according to (10). (12) The presentation unit highlights the points where the difference between the correct answer / model data and the user behavior data is large. The presentation device according to (10) or (11).
  • the presentation unit When the user behavior data is voice data, the presentation unit reproduces the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in order. The presentation device according to (6). (14) The presentation unit further reproduces the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in order. The presentation device according to (13). (15) The presenting unit raises the volume or slows down the reproduction speed of the correct answer / model data at a place where the difference from the user behavior data is large. The presentation device according to (13) or (14).
  • the behavior level determination step that determines the user's behavior level based on the user behavior data, Prediction that the user behavior data is predicted and transformed into correct answer / model data superior to the behavior level of the behavior level judgment result in order to reach the ideal behavior result based on the behavior level judgment result of judging the current user behavior level. Transformation step and A presentation step that presents the difference between the correct answer / model data and the user behavior data, A technique acquisition method characterized by including.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Tourism & Hospitality (AREA)
  • Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Data Mining & Analysis (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Resources & Organizations (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Electrically Operated Instructional Devices (AREA)

Abstract

Proposed are a learning method, a presentation device, and a technique acquisition method which do not provide typical advice but provide advice in which content changes in response to action content of a user, and through which a technique can be efficiently acquired/learnt. The present invention comprises: an action level determination unit (30) which determines an action level of a user on the basis of user action data; a prediction and deformation unit (40) which predicts and deforms, on the basis of the action level determination result obtained by determining the current action level of the user, the user action data into correct answer/model data that is more favorable than an action level of the action level determination result in order to reach an ideal action result; and a presentation unit (50) which presents the difference between the correct answer/model data and the user action data.

Description

学習装置、提示装置及び技術習得方法Learning device, presentation device and skill acquisition method
　本開示は、学習装置、提示装置及び技術習得方法に関する。 This disclosure relates to a learning device, a presentation device, and a technique acquisition method.
　曲の伴奏データとお手本音声データに合わせて歌った音声データを、お手本音声データと比較した差分データによって逆補正し、本人の歌声でお手本に近い上手なボーカルを体験可能な音声データ変換装置が提案されている（例えば、特許文献１参照）。 Proposed a voice data conversion device that can experience a good vocal that is close to the model with the person's singing voice by reverse-correcting the voice data sung according to the accompaniment data of the song and the model voice data by the difference data compared with the model voice data. (See, for example, Patent Document 1).
　また、各種の国家試験または検定試験における受験者の学習において、生徒各人の学習レベルに応じた出題をし、各人のウィークポイントの発見、学習レベルの評価を自動的に行う教育方法が提案されている（例えば、特許文献２参照）。 In addition, in the learning of examinees in various national examinations or certification exams, we propose an educational method that automatically asks questions according to each student's learning level, discovers weak points for each student, and evaluates the learning level. (See, for example, Patent Document 2).
特開２００１－１２５５８２号公報Japanese Unexamined Patent Publication No. 2001-125582 特開平６－１４９１４５号公報Japanese Unexamined Patent Publication No. 6-149145
　しかしながら、上記の従来技術では、一段階上の助言を得るために、システムに理想的な正解・お手本データ（もしくは段階化された正解・お手本データ）を記憶させておく必要がある。また、上記の従来技術では、無数の正解・お手本データをあらかじめ記憶することは収集の手間、記憶容量から困難であるという問題がある。 However, in the above-mentioned conventional technology, it is necessary to store ideal correct answer / model data (or stepped correct answer / model data) in the system in order to obtain advice one step higher. Further, in the above-mentioned conventional technique, there is a problem that it is difficult to store innumerable correct answer / model data in advance due to the trouble of collecting and the storage capacity.
　そこで、本開示では、定型的な助言ではなく、ユーザーの行動内容に応じて助言内容が変化し、効率的な技術習得・学習が可能な学習装置、提示装置及び技術習得方法を提案する。 Therefore, in this disclosure, instead of the standard advice, the content of the advice changes according to the action content of the user, and a learning device, a presentation device, and a technology acquisition method capable of efficient technology acquisition / learning are proposed.
　上記の課題を解決するために、本開示に係る一形態の学習装置は、ユーザー行動データに基づくユーザーの行動レベルを判定する行動レベル判定部と、現在のユーザーの行動レベルを判定した行動レベル判定結果に基づき、前記ユーザー行動データを、理想の行動結果に至るために前記行動レベル判定結果の行動レベルより優れた正解・お手本データに予測変形する予測変形部と、前記正解・お手本データと前記ユーザー行動データとの差を提示する提示部と、を備える。 In order to solve the above-mentioned problems, one form of the learning device according to the present disclosure includes a behavior level determination unit that determines a user's behavior level based on user behavior data, and an behavior level determination that determines the current user's behavior level. Based on the result, the predictive transformation unit that predictively transforms the user behavior data into correct answer / model data superior to the behavior level of the behavior level determination result in order to reach the ideal behavior result, the correct answer / model data, and the user. It is provided with a presentation unit that presents the difference from the behavior data.
本開示の第１の実施形態に係るデータ処理システムにおけるデータ処理の一例を示す図である。It is a figure which shows an example of the data processing in the data processing system which concerns on 1st Embodiment of this disclosure. 本開示の第１の実施形態に係るデータ処理システムにおける行動レベル判定から助言提示に係るデータ処理の流れの一例を示すフローチャートである。It is a flowchart which shows an example of the flow of the data processing which concerns on the advice presentation from the action level determination in the data processing system which concerns on 1st Embodiment of this disclosure. 本開示の第１の実施形態に係るデータ処理システムの構成例を示す図である。It is a figure which shows the structural example of the data processing system which concerns on 1st Embodiment of this disclosure. 本開示の第１の実施形態に係るデータ処理装置の構成例を示す図である。It is a figure which shows the structural example of the data processing apparatus which concerns on 1st Embodiment of this disclosure. データベース学習処理の一例を示す図である。It is a figure which shows an example of a database learning process. データベース学習処理の流れの一例を示すフローチャートである。It is a flowchart which shows an example of the flow of a database learning process. 表示の一例を示す図である。It is a figure which shows an example of the display. 表示の一例を示す図である。It is a figure which shows an example of the display. 表示の一例を示す図である。It is a figure which shows an example of the display. 表示の一例を示す図である。It is a figure which shows an example of the display. 本開示の第２の実施形態に係るデータ処理システムの構成例を示す図である。It is a figure which shows the structural example of the data processing system which concerns on the 2nd Embodiment of this disclosure. 本開示の第２の実施形態に係るデータ処理装置の構成例を示す図である。It is a figure which shows the structural example of the data processing apparatus which concerns on the 2nd Embodiment of this disclosure. データ処理装置やデータ化装置等の情報処理装置の機能を実現するコンピュータの一例を示すハードウェア構成図である。It is a hardware block diagram which shows an example of the computer which realizes the function of the information processing apparatus such as a data processing apparatus and a data conversion apparatus.
　以下に、本開示の実施形態について図面に基づいて詳細に説明する。なお、以下の各実施形態において、同一の部位には同一の符号を付することにより重複する説明を省略する。 Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each of the following embodiments, the same parts are designated by the same reference numerals, so that overlapping description will be omitted.
　以下に示す項目順序に従って本開示を説明する。
１．第１の実施形態
　１－１．本開示の実施形態に係るデータ処理の概要
　　１－１－１．背景及び効果等
　　１－１－２．データ処理システムの概念
　１－２．実施形態に係るデータ処理システムの構成
　１－３．実施形態に係るデータ処理装置の構成
　　１－３－１．モデル（ネットワーク）例
　１－４．実施形態に係る情報処理の手順
　　１－４－１．データ処理装置に係るユーザー行動のデータ化処理の手順
　　１－４－２．データ処理装置に係る助言提示処理の手順
２．第２の実施形態
　２－１．実施形態に係る情報処理の手順
　　２－１－１．データ処理装置に係るユーザー行動のデータ化処理の手順
３．その他の実施形態
　３－１．その他の構成例
　３－２．その他
４．本開示に係る効果
５．ハードウェア構成 The present disclosure will be described according to the order of items shown below.
1. 1. First Embodiment 1-1. Outline of data processing according to the embodiment of the present disclosure 1-1-1. Background and effects 1-1-2. Concept of data processing system 1-2. Configuration of data processing system according to the embodiment 1-3. Configuration of Data Processing Device According to the Embodiment 1-3-1. Model (network) example 1-4. Information processing procedure according to the embodiment 1-4-1. Procedure for digitizing user behavior related to the data processing device 1-4-2. Procedure of advice presentation processing related to data processing equipment 2. Second Embodiment 2-1. Information processing procedure according to the embodiment 2-1-1. Procedure for digitizing user behavior related to the data processing device 3. Other Embodiments 3-1. Other configuration examples 3-2. Others 4. Effect of this disclosure 5. Hardware configuration
［１．第１の実施形態］
　本開示の第１の実施形態について説明する。 [1. First Embodiment]
The first embodiment of the present disclosure will be described.
［１－１．本開示の実施形態に係るデータ処理の概要］
　図１は本開示の第１の実施形態に係るデータ処理システム１におけるデータ処理の一例を示す図である。本開示の実施形態に係るデータ処理は、データ処理装置１００やデータ化装置２００を含むデータ処理システム１によって実現される。図１では、データ処理システム１によって実現されるデータ処理の概要を説明する。 [1-1. Outline of data processing according to the embodiment of the present disclosure]
FIG. 1 is a diagram showing an example of data processing in the data processing system 1 according to the first embodiment of the present disclosure. The data processing according to the embodiment of the present disclosure is realized by the data processing system 1 including the data processing device 100 and the data processing device 200. FIG. 1 describes an outline of data processing realized by the data processing system 1.
　データ処理装置１００は、学習装置または提示装置として機能する。データ処理装置１００は、機械学習によって学習された正解・お手本データから現在のユーザーの行動レベル判定結果を、理想の行動結果に至るために行動レベル判定結果の行動レベルより優れた正解・お手本データに予測変形し、正解・お手本データとユーザー行動データとの差を分かりやすく提示する情報処理装置である。 The data processing device 100 functions as a learning device or a presentation device. The data processing device 100 converts the current user's action level determination result from the correct answer / model data learned by machine learning into correct answer / model data superior to the action level of the action level determination result in order to reach the ideal action result. It is an information processing device that predicts and transforms and presents the difference between correct answer / model data and user behavior data in an easy-to-understand manner.
　データ処理装置１００は、例えば技術習得の用途に用いられる。技術習得の対象としては、絵画、習字、楽器演奏、歌唱、スポーツなどが挙げられる。また、データ処理装置１００は、例えば学習に用いられる。学習の対象としては、外国語発音などの語学が挙げられる。 The data processing device 100 is used, for example, for the purpose of learning technology. Targets of skill acquisition include painting, writing, playing musical instruments, singing, and sports. Further, the data processing device 100 is used for learning, for example. Targets of learning include languages such as foreign language pronunciation.
　また、図１では、データ処理装置１００に対してユーザー行動に係るデータを提供するデータ化装置２００の一例として、ユーザーを撮像するカメラである場合には、ユーザー行動を画像データ化する。なお、データ化装置２００は、データ処理装置１００に対してユーザー行動に係るデータを提供可能であれば、カメラに限らず、種々の装置であってもよい。例えば、データ化装置２００は、ドローン等のＵＡＶ（Unmanned　Aerial　Vehicle）や各種センサであってもよい。データ化装置２００は、各種センサである場合には、ユーザー行動をセンシングデータ化（センサ反応値、画像解析結果など）する。 Further, in FIG. 1, as an example of a data conversion device 200 that provides data related to user behavior to the data processing device 100, in the case of a camera that captures a user, the user behavior is converted into image data. The data conversion device 200 is not limited to the camera and may be various devices as long as it can provide the data related to the user behavior to the data processing device 100. For example, the data conversion device 200 may be a UAV (Unmanned Aerial Vehicle) such as a drone or various sensors. In the case of various sensors, the data conversion device 200 converts user behavior into sensing data (sensor reaction value, image analysis result, etc.).
　具体的には、データ化装置２００は、絵画や習字などの制作物を、撮影、スキャンなどにより画像データ化を行う。また、データ化装置２００は、楽器演奏や歌などの演奏を録音して波形情報である音声データ化を行う。また、データ化装置２００は、スポーツにおける姿勢を撮影するなどして画像データ化を行う。また、データ化装置２００は、外国語発音などを録音して波形情報である音声データ化を行う。 Specifically, the data conversion device 200 converts a product such as a painting or a calligraphy into image data by shooting, scanning, or the like. Further, the data conversion device 200 records performances such as musical instrument performances and songs, and converts them into audio data which is waveform information. In addition, the data conversion device 200 converts images into image data, such as by photographing a posture in sports. Further, the data conversion device 200 records foreign language pronunciation and the like and converts it into voice data which is waveform information.
　ここから、図１に示す処理の概要を説明する。まず、図１の例では、データ処理装置１００は、行動レベル判定部３０と、予測変形部であるお手本予測変形部４０と、提示部である助言提示部５０と、学習済データベース６０と、を備える。 From here, the outline of the process shown in FIG. 1 will be described. First, in the example of FIG. 1, the data processing device 100 includes an action level determination unit 30, a model prediction transformation unit 40 which is a prediction transformation unit, an advice presentation unit 50 which is a presentation unit, and a learned database 60. Be prepared.
　学習済データベース６０は、行動レベル判定用のデータベース６０ａと、助言生成用のデータベース６０ｂと、を備える。行動レベル判定用のデータベース６０ａと助言生成用のデータベース６０ｂとは、行動レベルＬｖ毎にペアリングされている。行動レベル判定用のデータベース６０ａは、行動レベルＬｖ毎のユーザーのユーザー行動データを記憶する。助言生成用のデータベース６０ｂは、ユーザー行動の内容に応じた行動レベルＬｖ毎の助言となる正解・お手本データを記憶する。 The learned database 60 includes a database 60a for determining an action level and a database 60b for generating advice. The database 60a for determining the action level and the database 60b for generating advice are paired for each action level Lv. The database 60a for determining the behavior level stores user behavior data of the user for each behavior level Lv. The database 60b for generating advice stores correct answer / model data that is advice for each action level Lv according to the content of user behavior.
　行動レベル判定部３０およびお手本予測変形部４０は、機械学習を用いて予め用意した正解・お手本データを学習させて多数の正解・お手本データを予測生成し、学習済データベース６０に記憶する。 The action level determination unit 30 and the model prediction transformation unit 40 learn the correct answer / model data prepared in advance by using machine learning, predictively generate a large number of correct answers / model data, and store them in the learned database 60.
　行動レベル判定部３０は、行動レベルＬｖ毎の行動レベル判定用のデータベース６０ａを利用して、ユーザー行動データに基づくユーザーの行動レベルを推測する。行動レベル判定部３０は、クリア判定の場合は行動レベル判定用のデータベース６０ａにおける１段上の行動レベルＬｖに移行する。行動レベル判定部３０は、ＮＧ判定になるまで繰り返してユーザーの行動レベルＬｖを判定する。 The action level determination unit 30 estimates the user's action level based on the user action data by using the database 60a for action level determination for each action level Lv. In the case of clear determination, the action level determination unit 30 shifts to the action level Lv one step higher in the database 60a for action level determination. The action level determination unit 30 repeatedly determines the user's action level Lv until an NG determination is reached.
　お手本予測変形部４０は、行動レベル判定部３０による行動レベル判定結果である行動レベルＬｖにペアリングされた助言生成用のデータベース６０ｂを選択し、ユーザーの行動レベルＬｖに応じた正解・お手本データに基づき、ユーザー行動データを予測変形したユーザー行動データを生成する。 The model prediction transformation unit 40 selects a database 60b for advice generation paired with the behavior level Lv, which is the behavior level determination result by the behavior level determination unit 30, and uses the correct answer / model data according to the user's behavior level Lv. Based on this, user behavior data that is predicted and transformed from user behavior data is generated.
　助言提示部５０は、例えば、お手本予測変形部４０で変形して生成したユーザー行動データと正解・お手本データとの差分を、ユーザー行動データに重畳して可視化してユーザーに提示する。 The advice presentation unit 50, for example, superimposes and visualizes the difference between the user behavior data generated by transforming the model prediction transformation unit 40 and the correct answer / model data on the user behavior data and presents it to the user.
　次に、データ処理システム１における行動レベル判定から助言提示に係るデータ処理の流れについて説明する。図２は本開示の第１の実施形態に係るデータ処理システム１における行動レベル判定から助言提示に係るデータ処理の流れの一例を示すフローチャートである。本開示の実施形態に係るデータ処理システム１における行動レベル判定から助言提示に係るデータ処理は、図２に示すように、まず、データ化装置２００が、ユーザー行動をデータ化する（ステップＳ１）。 Next, the flow of data processing from the action level determination to the advice presentation in the data processing system 1 will be described. FIG. 2 is a flowchart showing an example of the flow of data processing from the action level determination to the advice presentation in the data processing system 1 according to the first embodiment of the present disclosure. As shown in FIG. 2, in the data processing related to the action level determination to the advice presentation in the data processing system 1 according to the embodiment of the present disclosure, the data processing device 200 first converts the user behavior into data (step S1).
　次いで、行動レベル判定部３０は、ユーザーの行動レベルＬｖの判定を開始する。具体的には、行動レベル判定部３０は、ユーザー行動データを行動レベル判定用のデータベース６０ａの行動レベルＬｖＮ（初期値は０）を用いて判定する（ステップＳ２）。 Next, the action level determination unit 30 starts determining the action level Lv of the user. Specifically, the behavior level determination unit 30 determines the user behavior data using the behavior level LvN (initial value is 0) of the database 60a for behavior level determination (step S2).
　行動レベル判定部３０は、判定結果がＯＫの場合（ステップＳ３のＯＫ）、判定対象の行動レベルＬｖを１だけ上げる（ステップＳ４）。一方、行動レベル判定部３０は、判定結果がＮＧの場合（ステップＳ３のＮＧ）、判定を完了する。 When the determination result is OK (OK in step S3), the action level determination unit 30 raises the action level Lv of the determination target by 1 (step S4). On the other hand, when the determination result is NG (NG in step S3), the action level determination unit 30 completes the determination.
　行動レベル判定部３０による行動レベルＬｖの判定が完了すると、お手本予測変形部４０は、判定された行動レベルＬｖに応じた正解・お手本データを生成する（ステップＳ５）。 When the action level Lv determination by the action level determination unit 30 is completed, the model prediction transformation unit 40 generates correct answer / model data according to the determined action level Lv (step S5).
　そして、助言提示部５０は、正解・お手本データをユーザーに提示する（ステップＳ６）。 Then, the advice presentation unit 50 presents the correct answer / model data to the user (step S6).
［１－１－１．背景及び効果等］
　ここで、上述したデータ処理システム１の背景や効果等について説明する。 [1-1-1. Background and effects]
Here, the background and effects of the above-mentioned data processing system 1 will be described.
　例えば特許文献１（特開２００１－１２５５８２号公報）においては、曲の伴奏データとお手本音声データに合わせて歌った音声データを、お手本音声データと比較した差分データによって逆補正し、本人の歌声でお手本に近い上手なボーカルを体験可能な音声データ変換装置が提案されている。 For example, in Patent Document 1 (Japanese Unexamined Patent Publication No. 2001-125582), the voice data sung according to the accompaniment data of the song and the model voice data is reverse-corrected by the difference data compared with the model voice data, and the singing voice of the person himself / herself is used. A voice data conversion device that allows you to experience good vocals that are close to a model has been proposed.
　しかしながら、特許文献１に開示の技術によれば、正解データが必ず必要であり、記録済みの正解データの存在する楽曲にしか対応しない、という問題点がある。また、特許文献１に開示の技術によれば、単純差分しか示しておらず、大きな差分がある部分が技術的に重要な差であるとは限らない、という問題点がある。 However, according to the technique disclosed in Patent Document 1, there is a problem that correct answer data is always required and only music having recorded correct answer data is supported. Further, according to the technique disclosed in Patent Document 1, there is a problem that only a simple difference is shown, and a portion having a large difference is not necessarily a technically important difference.
　また、例えば特許文献２（特開平６－１４９１４５号公報）においては、各種の国家試験または検定試験における受験者の学習において、生徒各人の学習レベルに応じた出題をし、各人のウィークポイントの発見、学習レベルの評価を自動的に行う教育方法が提案されている。 Further, for example, in Patent Document 2 (Japanese Unexamined Patent Publication No. 6-149145), in the learning of examinees in various national examinations or certification examinations, questions are given according to the learning level of each student, and each student's weak point. An educational method has been proposed that automatically discovers and evaluates the learning level.
　しかしながら、特許文献２に開示の技術によれば、ユーザーの学習レベルを判定するために、このような問題で間違えた場合はこのようなレベルであるというユーザーモデル構築および、出題問題の事前レベリング作業が必要である。特に、お手本・正解に個性のようなものがあると、そのバリエーションが必要となってくる。そのため、段階的に表示するために、個別に用意もしくは手動でレベリングが必要となり、大変な工数が必要となる。 However, according to the technique disclosed in Patent Document 2, in order to determine the learning level of the user, the user model construction that if a mistake is made in such a problem, the level is such a level, and the pre-leveling work of the question. is required. In particular, if there is something like individuality in the model / correct answer, that variation is needed. Therefore, in order to display them step by step, it is necessary to prepare them individually or manually level them, which requires a lot of man-hours.
　また、特許文献２に開示の技術によれば、ユーザーモデルはより精度を高めようとする場合、多数のサンプル解析が必要である。さらに、特許文献２に開示の技術によれば、出題問題をユーザーのレベルに細かく合わせるためには、多数の問題のレベリングを手動で、膨大な数を実施する必要があり、コストが本質的に高い、という問題がある。さらにまた、特許文献２に開示の技術によれば、レベリングのルールはシステム設計者が手動で設計する必要があるため、その技術分野の知識がなければ適切なレベリングはできない、という問題がある。 Further, according to the technique disclosed in Patent Document 2, a large number of sample analyzes are required for the user model in order to improve the accuracy. Further, according to the technique disclosed in Patent Document 2, in order to finely adjust the question to the user's level, it is necessary to manually level a large number of questions and to carry out a huge number of questions, which is essentially costly. There is a problem that it is expensive. Furthermore, according to the technique disclosed in Patent Document 2, since the leveling rule needs to be manually designed by the system designer, there is a problem that appropriate leveling cannot be performed without knowledge in the technical field.
　これに対し、本開示の第１の実施形態に係るデータ処理システム１は、どのようなユーザー入力に対しても正解・お手本データ自体を保持することなく予測可能である。また、データ処理システム１は、正解・お手本データ群から個別の差分ではなく、データ群に共通して差が大きい要素を提示できるので、技術的に本質的なものをとらえており、技術段階を示しやすくなっている。さらに、データ処理システム１は、正解・お手本データ群とユーザー入力群のサンプルのみを集めれば、レベリングの作業は機械学習自動で行われるため、手動でレベリングする必要はない。さらにまた、データ処理システム１は、レベリングルールは機械学習の過程において、統計的に大きな差異のある部分から自動で段階的に構築されるため、システム構築者がその技術分野の知識がなくとも構築可能である。この点についての説明を、以下で行う。 On the other hand, the data processing system 1 according to the first embodiment of the present disclosure can predict any user input without retaining the correct answer / model data itself. In addition, the data processing system 1 can present elements that have a large difference in common to the data group, not individual differences from the correct answer / model data group, so it captures the technically essential ones and sets the technical stage. It is easy to show. Further, in the data processing system 1, if only the samples of the correct answer / model data group and the user input group are collected, the leveling work is automatically performed by machine learning, so that there is no need to manually level. Furthermore, the data processing system 1 is constructed even if the system builder does not have knowledge of the technical field because the leveling rule is automatically and stepwise constructed from the part where there is a statistically large difference in the process of machine learning. It is possible. This point will be described below.
［１－１－２．データ処理システムの概念］
　例えば、データ処理システム１は、ＧＡＮ（Generative　Adversarial　Network：敵対的生成ネットワーク）を複数活用した機械学習の一種であるＣｙｃｌｅＧＡＮによって、学習に用いる正解・お手本データ（学習データ）を自動で生成することができる。 [1-1-2. Data processing system concept]
For example, the data processing system 1 can automatically generate correct answer / model data (learning data) used for learning by CycleGAN, which is a kind of machine learning that utilizes multiple GANs (Generative Adversarial Network). can.
［１－２．実施形態に係るデータ処理システムの構成］
　図３に示すデータ処理システム１について説明する。図３に示すように、データ処理システム１は、データ処理装置１００と、データ化装置２００とが含まれる。なお、図３では、１個のデータ化装置２００を図示するが、データ処理システム１には、１個より多い数のデータ化装置２００が含まれてもよい。データ化装置２００と、データ処理装置１００とは所定の通信網（ネットワークＮ）を介して、有線または無線により通信可能に接続される。図３は、本開示の第１の実施形態に係るデータ処理システムの構成例を示す図である。なお、図３に示したデータ処理システム１には、複数台のデータ処理装置１００が含まれてもよい。 [1-2. Configuration of data processing system according to the embodiment]
The data processing system 1 shown in FIG. 3 will be described. As shown in FIG. 3, the data processing system 1 includes a data processing device 100 and a data processing device 200. Although one data conversion device 200 is illustrated in FIG. 3, the data processing system 1 may include a larger number of data conversion devices 200. The data processing device 200 and the data processing device 100 are connected to each other via a predetermined communication network (network N) so as to be communicable by wire or wirelessly. FIG. 3 is a diagram showing a configuration example of a data processing system according to the first embodiment of the present disclosure. The data processing system 1 shown in FIG. 3 may include a plurality of data processing devices 100.
　データ処理装置１００は、機械学習によって学習された正解・お手本データから現在のユーザーの行動レベル判定結果を、理想の行動結果に至るために行動レベル判定結果の行動レベルより優れた正解・お手本データに予測変形し、正解・お手本データとユーザー行動データとの差を分かりやすく提示する情報処理装置（コンピュータ）である。 The data processing device 100 converts the current user's action level determination result from the correct answer / model data learned by machine learning into correct answer / model data superior to the action level of the action level determination result in order to reach the ideal action result. It is an information processing device (computer) that predicts and transforms and presents the difference between correct answer / model data and user behavior data in an easy-to-understand manner.
　データ化装置２００は、データ処理装置１００に対してユーザー行動に係るデータを提供するコンピュータである。例えば、図３の例では、データ化装置２００は、ユーザーを撮像する撮像機能を有するカメラである。 The data conversion device 200 is a computer that provides data related to user behavior to the data processing device 100. For example, in the example of FIG. 3, the data digitizing device 200 is a camera having an imaging function for imaging a user.
　なお、データ化装置２００は、ユーザー行動をセンシングデータ化（センサ反応値、画像解析結果など）する各種センサであってもよい。 The data conversion device 200 may be various sensors that convert user behavior into sensing data (sensor reaction value, image analysis result, etc.).
　また、データ化装置２００は、ドローン等のＵＡＶや自動車等の車両等の移動体であってもよい。データ化装置２００は、イメージセンサ（イメージャ）等の撮像機能を有し、データ処理装置１００からの要求に応じた位置まで移動し、その位置で画像や動画を撮像し、撮像した画像や動画をデータ処理装置１００に送信するようにしてもよい。 Further, the data conversion device 200 may be a moving body such as a UAV such as a drone or a vehicle such as an automobile. The data conversion device 200 has an image pickup function such as an image sensor (imager), moves to a position corresponding to a request from the data processing device 100, captures an image or a moving image at that position, and captures the captured image or moving image. It may be transmitted to the data processing device 100.
　なお、データ化装置２００は、実施形態における処理を実現可能であれば、どのような装置であってもよい。データ化装置２００は、例えば、スマートフォンや、タブレット型端末や、ノート型ＰＣ（Personal　Computer）や、デスクトップＰＣや、携帯電話機や、ＰＤＡ（Personal　Digital　Assistant）等の装置であってもよい。データ化装置２００は、ユーザーが身に着けるウェアラブル端末（Wearable　Device）等であってもよい。例えば、データ化装置２００は、腕時計型端末やメガネ型端末等であってもよい。また、データ化装置２００は、テレビや冷蔵庫等のいわゆる家電製品であってもよい。例えば、データ化装置２００は、スマートスピーカやエンタテインメントロボットや家庭用ロボットと称されるような、人間（ユーザー）と対話するロボットであってもよい。また、データ化装置２００は、デジタルサイネージ等の所定の位置に配置される装置であってもよい。 The data conversion device 200 may be any device as long as the processing in the embodiment can be realized. The data conversion device 200 may be, for example, a device such as a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, or a PDA (Personal Digital Assistant). The data conversion device 200 may be a wearable terminal (Wearable Device) or the like that the user can wear. For example, the data digitizing device 200 may be a wristwatch type terminal, a glasses type terminal, or the like. Further, the data digitizing device 200 may be a so-called home electric appliance such as a television or a refrigerator. For example, the data conversion device 200 may be a robot that interacts with a human (user), such as a smart speaker, an entertainment robot, or a domestic robot. Further, the data conversion device 200 may be a device arranged at a predetermined position such as a digital signage.
［１－３．実施形態に係るデータ処理装置の構成］
　次に、実施形態に係るデータ処理を実行するデータ処理装置の一例であるデータ処理装置１００の構成について説明する。図４は、本開示の第１の実施形態に係るデータ処理装置１００の構成例を示す図である。 [1-3. Configuration of data processing device according to the embodiment]
Next, the configuration of the data processing device 100, which is an example of the data processing device that executes the data processing according to the embodiment, will be described. FIG. 4 is a diagram showing a configuration example of the data processing device 100 according to the first embodiment of the present disclosure.
　図４に示すように、データ処理装置１００は、通信部１１０と、記憶部１２０と、制御部１３０とを有する。なお、データ処理装置１００は、データ処理装置１００の管理者等から各種操作を受け付ける入力部（例えば、キーボードやマウス等）や、各種情報を表示するための表示部（例えば、液晶ディスプレイ等）を有してもよい。 As shown in FIG. 4, the data processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The data processing device 100 includes an input unit (for example, a keyboard, a mouse, etc.) that receives various operations from the administrator of the data processing device 100, and a display unit (for example, a liquid crystal display, etc.) for displaying various information. You may have.
　通信部１１０は、例えば、ＮＩＣ（Network　Interface　Card）等によって実現される。そして、通信部１１０は、ネットワークＮ（図３参照）と有線または無線で接続され、データ化装置２００等の他の情報処理装置との間で情報の送受信を行う。また、通信部１１０は、データ化装置２００との間で情報の送受信を行ってもよい。 The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. Then, the communication unit 110 is connected to the network N (see FIG. 3) by wire or wirelessly, and transmits / receives information to / from another information processing device such as the data conversion device 200. Further, the communication unit 110 may send and receive information to and from the data conversion device 200.
　記憶部１２０は、例えば、ＲＡＭ（Random　Access　Memory）、フラッシュメモリ（Flash　Memory）等の半導体メモリ素子、または、ハードディスク、光ディスク等の記憶装置によって実現される。実施形態に係る記憶部１２０は、図４に示すように、データ情報記憶部１２１と、行動レベル情報記憶部１２２と、を有する。 The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. As shown in FIG. 4, the storage unit 120 according to the embodiment includes a data information storage unit 121 and an action level information storage unit 122.
　データ情報記憶部１２１は、正解・お手本データ（学習データ）の学習に用いるサンプルデータを記憶する。例えば、データ情報記憶部１２１は、正解・お手本データのサンプルである正解・お手本サンプルデータ群、ユーザー行動データのサンプルであるユーザー行動サンプルデータ群を有する。 The data information storage unit 121 stores sample data used for learning correct answer / model data (learning data). For example, the data information storage unit 121 has a correct answer / model sample data group that is a sample of correct answer / model data, and a user behavior sample data group that is a sample of user behavior data.
　実施形態に係る行動レベル情報記憶部１２２は、行動レベルＬｖ毎のユーザーの行動レベル、およびユーザー行動の内容に応じた行動レベルＬｖ毎の助言に関する情報を記憶する。例えば、行動レベル情報記憶部１２２は、行動レベルＬｖ毎のユーザーの行動レベルを記憶する行動レベル判定用のデータベース６０ａと、ユーザー行動の内容に応じた行動レベルＬｖ毎の助言となる正解・お手本データを記憶する助言生成用のデータベース６０ｂとを記憶する。 The action level information storage unit 122 according to the embodiment stores information on the user's action level for each action level Lv and the advice for each action level Lv according to the content of the user action. For example, the action level information storage unit 122 has a database 60a for determining the action level that stores the action level of the user for each action level Lv, and correct answer / model data that provides advice for each action level Lv according to the content of the user action. The database 60b for generating advice is stored.
　図４に戻り、説明を続ける。制御部１３０は、例えば、ＣＰＵ（Central　Processing　Unit）やＭＰＵ（Micro　Processing　Unit）等によって、データ処理装置１００内部に記憶されたプログラム（例えば、本開示に係るデータ処理プログラム等の情報処理プログラム）がＲＡＭ（Random　Access　Memory）等を作業領域として実行されることにより実現される。また、制御部１３０は、コントローラ（controller）であり、例えば、ＡＳＩＣ（Application　Specific　Integrated　Circuit）やＦＰＧＡ（Field　Programmable　Gate　Array）等の集積回路により実現される。 Return to Fig. 4 and continue the explanation. In the control unit 130, for example, a program stored inside the data processing device 100 (for example, an information processing program such as a data processing program according to the present disclosure) by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like is used. It is realized by executing RAM (Random Access Memory) etc. as a work area. Further, the control unit 130 is a controller, and is realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).
　図４に示すように、制御部１３０は、行動レベル判定部３０と、お手本予測変形部４０と、助言提示部５０と、を有し、以下に説明する情報処理の機能や作用を実現または実行する。なお、制御部１３０の内部構成は、図４に示した構成に限られず、後述する情報処理を行う構成であれば他の構成であってもよい。また、制御部１３０が有する各処理部の接続関係は、図４に示した接続関係に限られず、他の接続関係であってもよい。 As shown in FIG. 4, the control unit 130 includes an action level determination unit 30, a model prediction transformation unit 40, and an advice presentation unit 50, and realizes or executes the functions and actions of information processing described below. do. The internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 4, and may be any other configuration as long as it is configured to perform information processing described later. Further, the connection relationship of each processing unit included in the control unit 130 is not limited to the connection relationship shown in FIG. 4, and may be another connection relationship.
　行動レベル判定部３０およびお手本予測変形部４０は、機械学習を用いて正解・お手本データを学習させ、多数の正解・お手本データを予測生成する。 The action level determination unit 30 and the model prediction transformation unit 40 use machine learning to learn correct answer / model data, and predict and generate a large number of correct answer / model data.
　行動レベル判定部３０およびお手本予測変形部４０は、データ情報記憶部１２１に記憶された学習用データ（正解・お手本データ）に基づいて、学習処理を行う。行動レベル判定部３０およびお手本予測変形部４０は、データ情報記憶部１２１に記憶された学習用データを用いて、学習処理を行うことにより、行動レベルＬｖ毎のユーザーの行動レベル、およびユーザー行動の内容に応じた行動レベルＬｖ毎の助言を学習（予測生成）する。 The action level determination unit 30 and the model prediction transformation unit 40 perform learning processing based on the learning data (correct answer / model data) stored in the data information storage unit 121. The behavior level determination unit 30 and the model prediction transformation unit 40 perform learning processing using the learning data stored in the data information storage unit 121, thereby performing a learning process on the user's behavior level and user behavior for each behavior level Lv. Learn (predictively generate) advice for each action level Lv according to the content.
　行動レベル判定部３０およびお手本予測変形部４０による学習の手法は特にＣｙｃｌｅＧＡＮに限定されないが、例えば、学習用データ（正解・お手本データ）を用意し、その学習用データを多層ニューラルネットワークに基づいた計算モデルに入力して学習してもよい。また、例えばＣＮＮ（Convolutional　Neural　Network）、３Ｄ－ＣＮＮ等のＤＮＮ（Deep　Neural　Network）に基づく手法が用いられてもよい。行動レベル判定部３０およびお手本予測変形部４０は、映像等の動画像（動画）のような時系列データを対象とする場合、再帰型ニューラルネットワーク（Recurrent　Neural　Network：ＲＮＮ）やＲＮＮを拡張したＬＳＴＭ（Long　Short-Term　Memory　units）に基づく手法を用いてもよい。 The learning method by the action level determination unit 30 and the model prediction transformation unit 40 is not particularly limited to CycleGAN, but for example, learning data (correct answer / model data) is prepared, and the learning data is calculated based on the multi-layer neural network. You may input to the model and learn. Further, for example, a method based on DNN (Deep Neural Network) such as CNN (Convolutional Neural Network) or 3D-CNN may be used. When the action level determination unit 30 and the model prediction transformation unit 40 target time-series data such as moving images (moving images) such as images, the recurrent neural network (RNN) and LSTM extended with RNN are used. A method based on (Long Short-Term Memory units) may be used.
　行動レベル判定部３０は、ユーザー行動データを取得し、ユーザー行動データに基づくユーザーの行動レベルを判定する。 The behavior level determination unit 30 acquires the user behavior data and determines the user behavior level based on the user behavior data.
　お手本予測変形部４０は、現在のユーザーの行動レベルを判定した行動レベル判定結果に基づき、ユーザー行動データを、理想の行動結果に至るために前記行動レベル判定結果の行動レベルより優れた正解・お手本データ（例えば１段階上の行動レベル相当の正解・お手本データ）に予測変形する。 The model prediction transformation unit 40 is based on the behavior level determination result of determining the current user's behavior level, and in order to reach the ideal behavior result, the user behavior data is a correct answer / model superior to the behavior level of the behavior level determination result. Predictively transforms into data (for example, correct answer / model data equivalent to the action level one step higher).
　助言提示部５０は、ユーザーの行動レベル判定結果に対して具体的な助言を示すために、正解・お手本データとユーザー行動データとの差を分かりやすく提示する。 The advice presentation unit 50 presents the difference between the correct answer / model data and the user behavior data in an easy-to-understand manner in order to give specific advice to the user's behavior level determination result.
［１－３－１．モデル（ネットワーク）例］
　上述したように、データ処理装置１００は、ディープニューラルネットワーク（ＤＮＮ）等のニューラルネットワーク（ＮＮ）の形式のモデル（ネットワーク）を用いてもよい。なお、データ処理装置１００は、ニューラルネットワークに限らず、ＳＶＭ（Support　Vector　Machine）等の回帰モデルや等の種々の形式のモデル（関数）を用いてもよい。このように、データ処理装置１００は、任意の形式のモデル（関数）を用いてもよい。データ処理装置１００は、非線形の回帰モデルや線形の回帰モデル等、種々の回帰モデルを用いてもよい。 [1-3-1. Model (network) example]
As described above, the data processing apparatus 100 may use a model (network) in the form of a neural network (NN) such as a deep neural network (DNN). The data processing device 100 is not limited to the neural network, and may use various types of models (functions) such as a regression model such as SVM (Support Vector Machine). As described above, the data processing apparatus 100 may use a model (function) of any format. The data processing device 100 may use various regression models such as a non-linear regression model and a linear regression model.
［１－４．実施形態に係る情報処理の手順］
　次に、実施形態に係る各種情報処理にかかるユーザー行動のデータ化処理および助言提示処理の手順について説明する。 [1-4. Information processing procedure according to the embodiment]
Next, the procedure of data conversion processing of user behavior and advice presentation processing related to various information processing according to the embodiment will be described.
［１－４－１．データ処理装置に係るユーザー行動のデータ化処理の手順］
　まず、学習済データベース６０に対するデータベース学習方法について説明する。 [1-4-1. Procedure for digitizing user behavior related to data processing equipment]
First, a database learning method for the trained database 60 will be described.
　図５はデータベース学習処理の一例を示す図、図６はデータベース学習処理の流れの一例を示すフローチャートである。図５に示すように、データ処理装置１００は、お手本予測変形部４０および行動レベル判定部３０によりデータベース学習処理を実施する。 FIG. 5 is a diagram showing an example of database learning processing, and FIG. 6 is a flowchart showing an example of the flow of database learning processing. As shown in FIG. 5, the data processing device 100 performs database learning processing by the model prediction transformation unit 40 and the action level determination unit 30.
　行動レベル判定部３０は、変形・正解判定部３０１と、ユーザー行動予測変形部３０２と、予測・ユーザー行動判定部３０３と、を備える。 The action level determination unit 30 includes a transformation / correct answer determination unit 301, a user behavior prediction transformation unit 302, and a prediction / user behavior determination unit 303.
　変形・正解判定部３０１は、正解・お手本データに基づきお手本予測変形部４０でユーザー行動を予測変形したユーザー行動データと、元の正解・お手本データと、を区別判定する。 The transformation / correct answer determination unit 301 distinguishes between the user behavior data obtained by predicting and transforming the user behavior by the model prediction transformation unit 40 based on the correct answer / model data and the original correct answer / model data.
　ユーザー行動予測変形部３０２は、正解・お手本データをユーザー行動データに予測変形する。すなわち、ユーザー行動予測変形部３０２は、上手→下手に予測変形する。 The user behavior prediction transformation unit 302 predicts and transforms the correct answer / model data into the user behavior data. That is, the user behavior prediction transformation unit 302 predicts and transforms from good to bad.
　予測・ユーザー行動判定部３０３は、ユーザー行動予測変形部３０２で実際に予測変形したユーザー行動データと、サンプルのユーザー行動データと、を区別判定する。 The prediction / user behavior determination unit 303 distinguishes between the user behavior data actually predicted and transformed by the user behavior prediction transformation unit 302 and the sample user behavior data.
　データ処理装置１００は、まず、下準備を行う。第１に、データ処理装置１００は、行動レベルＬｖを下手→上手に学習する。以下において詳述する。 The data processing device 100 first prepares. First, the data processing device 100 learns the behavior level Lv poorly → well. It will be described in detail below.
　処理（１）
　まず、データ処理装置１００は、予め記憶したユーザー行動サンプルデータ群からユーザー行動データを選択してお手本予測変形部４０に渡す（ステップＳ１１）。次に、データ処理装置１００は、お手本予測変形部４０を用いて正解・お手本データに基づきユーザー行動データを予測変形する（ステップＳ１２）。すなわち、ユーザー行動予測変形部３０２は、下手→上手に予測変形する。 Processing (1)
First, the data processing device 100 selects user behavior data from the user behavior sample data group stored in advance and passes it to the model prediction transformation unit 40 (step S11). Next, the data processing device 100 predicts and transforms the user behavior data based on the correct answer / model data by using the model predictive transformation unit 40 (step S12). That is, the user behavior prediction transformation unit 302 predicts and transforms poorly → well.
　処理（２）
　次に、データ処理装置１００は、変形・正解判定部３０１を用いて、処理（１）で学習したお手本予測変形部４０で実際に予測変形したユーザー行動データと、元の正解・お手本データとを区別判定して学習する（ステップＳ１３）。 Processing (2)
Next, the data processing device 100 uses the transformation / correct answer determination unit 301 to obtain the user behavior data actually predicted and transformed by the model prediction transformation unit 40 learned in the process (1) and the original correct answer / model data. The distinction is determined and learning is performed (step S13).
　第２に、データ処理装置１００は、行動レベルＬｖを上手→下手に学習する。以下において詳述する。 Second, the data processing device 100 learns the behavior level Lv from good to bad. It will be described in detail below.
　処理（３）
　データ処理装置１００は、予め記憶した正解・お手本サンプルデータ群から正解・お手本データを選択してユーザー行動予測変形部３０２に渡す（ステップＳ１４）。次に、データ処理装置１００は、ユーザー行動予測変形部３０２を用いて正解・お手本データをユーザー行動データに基づいてユーザー行動データに予測変形する（ステップＳ１５）。すなわち、ユーザー行動予測変形部３０２は、上手→下手に予測変形する。 Processing (3)
The data processing device 100 selects the correct answer / model data from the correct answer / model sample data group stored in advance and passes it to the user behavior prediction transformation unit 302 (step S14). Next, the data processing device 100 predicts and transforms the correct answer / model data into the user behavior data based on the user behavior data by using the user behavior prediction transformation unit 302 (step S15). That is, the user behavior prediction transformation unit 302 predicts and transforms from good to bad.
　処理（４）
　次に、データ処理装置１００は、予測・ユーザー行動判定部３０３を用いて、処理（３）で学習したユーザー行動予測変形部３０２で実際に変形したユーザー行動データと、元のユーザー行動データと、を区別判定して学習する（ステップＳ１６）。 Processing (4)
Next, the data processing device 100 uses the prediction / user behavior determination unit 303 to obtain the user behavior data actually transformed by the user behavior prediction transformation unit 302 learned in the process (3), the original user behavior data, and the original user behavior data. (Step S16).
　以上により、データ処理装置１００は、下準備を終了する。上述した初期の学習は、お手本とユーザーの行動の大きな差＝行動レベルの基本的な部分に着目したものである。 With the above, the data processing device 100 completes the preparation. The initial learning described above focuses on the large difference between the model and the user's behavior = the basic part of the behavior level.
　続いて、データ処理装置１００は、繰り返し学習を開始する。 Subsequently, the data processing device 100 starts iterative learning.
　処理（５）
　まず、データ処理装置１００は、お手本予測変形部４０でユーザー行動データを正解・お手本データに変形する（ステップＳ２１）。次に、データ処理装置１００は、お手本予測変形部４０でユーザー行動データを正解・お手本データに変形した結果を、ユーザー行動予測変形部３０２でユーザー行動データに再変形し、ユーザー行動データとする（ステップＳ２２）。 Processing (5)
First, the data processing device 100 transforms the user behavior data into correct answer / model data by the model prediction transformation unit 40 (step S21). Next, the data processing device 100 retransforms the result of transforming the user behavior data into the correct answer / model data by the model prediction transformation unit 40 into the user behavior data by the user behavior prediction transformation unit 302, and obtains the user behavior data ( Step S22).
　また、データ処理装置１００は、お手本予測変形部４０でユーザー行動データを正解・お手本データに変形した結果が変形・正解判定部３０１を間違えさせるように、かつ、ユーザー行動データを正解・お手本データに変形した結果をさらにユーザー行動予測変形部３０２で再変形した結果が元のユーザー行動データと一致するように、お手本予測変形部４０によりユーザー行動データを正解・お手本データに変形し、学習させる（ステップＳ２３）。 Further, in the data processing device 100, the result of transforming the user behavior data into the correct answer / model data by the model prediction transformation unit 40 causes the transformation / correct answer determination unit 301 to make a mistake, and the user behavior data is converted into the correct answer / model data. The model prediction transformation unit 40 transforms the user behavior data into correct answer / model data and trains the transformed result so that the result of retransformation by the user behavior prediction transformation unit 302 matches the original user behavior data (step). S23).
　そして、データ処理装置１００は、お手本予測変形部４０で学習した結果である正解・お手本データを学習済データベース６０の助言生成用のデータベース６０ｂに記録する（ステップＳ２４）。 Then, the data processing device 100 records the correct answer / model data, which is the result of learning in the model prediction / transforming unit 40, in the database 60b for generating advice of the learned database 60 (step S24).
　次に、データ処理装置１００は、ユーザー行動予測変形部３０２で正解・お手本データをユーザー行動データに予測変形する（ステップＳ２５）。続いて、データ処理装置１００は、ユーザー行動予測変形部３０２で正解・お手本データをユーザー行動データに変形した結果を、お手本予測変形部４０で再変形する（ステップＳ２６）。 Next, the data processing device 100 predicts and transforms the correct answer / model data into the user behavior data by the user behavior prediction transformation unit 302 (step S25). Subsequently, the data processing device 100 transforms the result of transforming the correct answer / model data into the user behavior data by the user behavior prediction transformation unit 302, and retransforms the result by the model prediction transformation unit 40 (step S26).
　また、データ処理装置１００は、正解・お手本データをユーザー行動データに変形した結果が変形・正解判定部３０１を間違えさせるように、かつ、正解・お手本データをユーザー行動データに変形した結果をさらにお手本予測変形部４０で再変形した結果が元の正解・お手本データと一致するように、ユーザー行動予測変形部３０２に学習させる（ステップＳ２７）。 Further, the data processing device 100 further models the result of transforming the correct answer / model data into the user behavior data so that the transformation / correct answer determination unit 301 makes a mistake in the result of transforming the correct answer / model data into the user behavior data. The user behavior prediction transformation unit 302 is trained so that the result of re-transformation by the prediction transformation unit 40 matches the original correct answer / model data (step S27).
　処理（６）
　データ処理装置１００は、変形・正解判定部３０１において、ステップＳ２３で学習したお手本予測変形部４０で実際に予測した正解・お手本データと、元の正解・お手本データとを区別判定し、学習させる（ステップＳ２８）。 Processing (6)
The data processing device 100 distinguishes and learns the correct answer / model data actually predicted by the model prediction transformation unit 40 learned in step S23 and the original correct answer / model data in the transformation / correct answer determination unit 301 ( Step S28).
　そして、データ処理装置１００は、判定学習結果を学習済データベース６０の行動レベル判定用のデータベース６０ａに記録する（ステップＳ２９）。このように機械学習の収束に至る学習途中の判定学習結果を、理想の結果として複数組み合わせて記憶するのは、レベリングに利用するためであり、後述するように段階的に助言を提示することができる。 Then, the data processing device 100 records the determination learning result in the database 60a for determining the action level of the learned database 60 (step S29). The reason why a plurality of judgment learning results in the middle of learning leading to the convergence of machine learning are combined and stored as an ideal result is to be used for leveling, and advice can be presented step by step as described later. can.
　また、データ処理装置１００は、予測・ユーザー行動判定部３０３によって、ステップＳ２７で学習したユーザー行動予測変形部３０２で実際に変形したユーザー行動データと、元のユーザー行動データとを区別判定し、学習させる（ステップＳ３０）。 Further, the data processing device 100 distinguishes and determines between the user behavior data actually deformed by the user behavior prediction transformation unit 302 learned in step S27 and the original user behavior data by the prediction / user behavior determination unit 303, and learns. (Step S30).
　処理（７）
　データ処理装置１００は、処理（５）～処理（６）を繰り返す。すなわち、データ処理装置１００は、以上説明したステップＳ２１～Ｓ２７の処理と、ステップＳ２８～Ｓ３０の処理とを、交互に繰り返す。データ処理装置１００は、繰り返しの学習の過程のデータを順番に保持する。 Processing (7)
The data processing device 100 repeats processing (5) to processing (6). That is, the data processing apparatus 100 alternately repeats the processes of steps S21 to S27 and the processes of steps S28 to S30 described above. The data processing device 100 sequentially holds data in the process of repeated learning.
　上述のように、データ処理装置１００は、機械学習により、まず大きな差分を小さくするように学習し、繰り返していくことで、順々に詳細な差を小さくするように学習する。初期の学習は、正解・お手本データとユーザーの行動の大きな差＝行動レベルの基本的な部分に着目する。終盤の学習は、小さな差＝行動レベルの高度な部分に着目する。 As described above, the data processing device 100 first learns to reduce a large difference by machine learning, and then repeatedly learns to reduce a detailed difference in order. The initial learning focuses on the large difference between the correct answer / model data and the user's behavior = the basic part of the behavior level. Late learning focuses on small differences = high levels of behavior.
　上述のように、行動レベル判定部３０における行動レベル判定と、お手本予測変形部４０における理想の行動結果に至るために行動レベル判定結果の行動レベルより優れた正解・お手本データへの予測変形とは、機械学習によって、両者をペアリングして交互に学習する。これにより、行動レベルのレベリングを自動で行うことができる。 As described above, what is the behavior level determination in the behavior level determination unit 30 and the prediction transformation to the correct answer / model data superior to the behavior level of the behavior level determination result in order to reach the ideal behavior result in the model prediction transformation unit 40? , By machine learning, both are paired and learned alternately. This makes it possible to automatically level the behavior level.
［１－４－２．データ処理装置に係る助言提示処理の手順］
　次に、助言提示部５０における助言提示方法について説明する。 [1-4-2. Procedure for advice presentation processing related to data processing equipment]
Next, the advice presentation method in the advice presentation unit 50 will be described.
　まず、ユーザー行動データが画像データの場合（例：絵画、習字、スポーツの姿勢）について説明する。 First, the case where the user behavior data is image data (example: painting, calligraphy, posture of sports) will be explained.
　１．助言提示部５０は、（ａ）ユーザーの行動レベルを判定したユーザー行動データとユーザー行動データに対して１段上の行動レベルＬｖの理想の行動結果である正解・お手本データの両方を重ねて表示部に表示する／横に並べて表示部に表示する。 1. The advice presentation unit 50 displays (a) both the correct answer and the model data, which are the ideal behavior results of the behavior level Lv one step higher than the user behavior data for determining the behavior level of the user and the user behavior data. Display on the display / Display side by side on the display.
　ここで、図７は表示の一例を示す図である。図７に示す表示例は、ユーザーの行動レベルを判定したユーザー行動データａと、１段上の行動レベルＬｖの正解・お手本データｂと、ｎ（ｎ＞１）段上の行動レベルＬｖの正解・お手本データｃと、を横に並べて表示した様子を示したものである。 Here, FIG. 7 is a diagram showing an example of display. In the display example shown in FIG. 7, the user behavior data a for determining the user's behavior level, the correct answer / model data b of the behavior level Lv one step higher, and the correct answer of the behavior level Lv one step higher are shown. -It shows how the model data c and the model data c are displayed side by side.
　２．助言提示部５０は、１．に加えてさらに、（ｃ）ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルＬｖの理想の行動結果である正解・お手本データを重ねて表示部に表示する／横に並べて表示部に表示する。 2. The advice presentation unit 50 is 1. In addition to (c), the correct answer / model data, which is the ideal behavior result of the behavior level Lv n (n> 1) higher than the user behavior data, is displayed on the display / side by side. Display in the section.
　ここで、図８は表示の一例を示す図である。図８に示す表示例は、ユーザーの行動レベルを判定したユーザー行動データａと、１段上の行動レベルＬｖの正解・お手本データｂと、ｎ（ｎ＞１）段上の行動レベルＬｖの理想の行動結果である正解・お手本データｃと、を重ねて表示した様子を示したものである。 Here, FIG. 8 is a diagram showing an example of display. In the display example shown in FIG. 8, the user behavior data a for determining the behavior level of the user, the correct answer / model data b of the behavior level Lv one step higher, and the ideal behavior level Lv one step higher than n (n> 1). It shows how the correct answer / model data c, which is the result of the action of, is superimposed and displayed.
　３．助言提示部５０は、１．または２．の表示方法のうち、ユーザー行動データと、（ｂ），（ｃ）との差が大きい箇所を強調表示する。強調表示の方法は、色を変える、明るさのレベルを変える、（動画の場合）再生速度を遅くする、などの方法がある。 3. The advice presentation unit 50 is 1. Or 2. Of the display methods of, highlight the part where the difference between the user behavior data and (b) and (c) is large. Highlighting methods include changing the color, changing the brightness level, and (in the case of video) slowing down the playback speed.
　ここで、図９は表示の一例を示す図である。図９に示す表示例は、ユーザーの行動レベルを判定したユーザー行動データａと、１段上の行動レベルＬｖの理想の行動結果である正解・お手本データｂと、を重ねて表示し、ユーザー行動データと１段上の行動レベルＬｖとの差が大きい箇所を太線ｄで強調表示した様子を示したものである。 Here, FIG. 9 is a diagram showing an example of display. In the display example shown in FIG. 9, the user behavior data a for determining the user behavior level and the correct answer / model data b which is the ideal behavior result of the behavior level Lv one step higher are displayed in an overlapping manner, and the user behavior is displayed. The thick line d highlights the part where the difference between the data and the action level Lv one step higher is large.
　４．助言提示部５０は、（ａ）ユーザーの行動レベルを判定したユーザー行動データと、（ｄ）ユーザー行動データとユーザー行動データに対して１段上の行動レベルＬｖの理想の行動結果である正解・お手本データとの差分を、重ねて表示部に表示する／横に並べて表示部に表示する。 4. The advice presentation unit 50 is a correct answer, which is (a) the user behavior data for determining the user's behavior level, and (d) the ideal behavior result of the behavior level Lv one step higher than the user behavior data and the user behavior data. The difference from the model data is displayed on the display unit in an overlapping manner / side by side on the display unit.
　ここで、図１０は表示の一例を示す図である。図１０に示す表示例は、ユーザーの行動レベルを判定したユーザー行動データａと、ユーザー行動データａと１段上の行動レベルＬｖの理想の行動結果である正解・お手本データとの差分ｅと、を重ねて表示した様子を示したものである。 Here, FIG. 10 is a diagram showing an example of display. In the display example shown in FIG. 10, the difference e between the user behavior data a for determining the user behavior level, the user behavior data a, and the correct answer / model data which is the ideal behavior result of the behavior level Lv one step higher, is shown. It shows the state of overlapping and displaying.
　５．助言提示部５０は、４．に加えて、（ｅ）ユーザーの行動レベルを判定したユーザー行動データとユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルＬｖの理想の行動結果である正解・お手本データとの差分を、重ねて表示部に表示する／横に並べて表示部に表示する。 5. The advice presentation unit 50 is 4. In addition, (e) the difference between the user behavior data that determines the user behavior level and the correct answer / model data that is the ideal behavior result of the behavior level Lv n (n> 1) higher than the user behavior data. Are displayed on the display unit in an overlapping manner / displayed side by side on the display unit.
　６．助言提示部５０は、４．または５．の表示方法のうち、ユーザー行動データと、（ｂ），（ｃ）との差が大きい箇所を強調表示する。強調表示の方法は、色を変える、明るさのレベルを変える、（動画の場合）再生速度を遅くする、の方法がある。 6. The advice presentation unit 50 is 4. Or 5. Of the display methods of, highlight the part where the difference between the user behavior data and (b) and (c) is large. The highlighting method includes changing the color, changing the brightness level, and (in the case of a movie) slowing down the playback speed.
　次に、ユーザー行動データが音声データの場合（例：楽器演奏、語学学習の場合）について説明する。 Next, the case where the user behavior data is voice data (example: in the case of playing a musical instrument or learning a language) will be described.
　助言提示部５０は、波形信号として画像表示する場合は、上記ユーザー行動データが画像データの場合と同様の助言の形式を採ることができる。一方、波形信号を音声信号に再変換してユーザーに提示する場合は、助言提示部５０は、以下の助言の形式を採ることができる。 When displaying an image as a waveform signal, the advice presentation unit 50 can take the same advice format as when the user behavior data is image data. On the other hand, when the waveform signal is reconverted into an audio signal and presented to the user, the advice presenting unit 50 can take the following form of advice.
　１．助言提示部５０は、（ａ）ユーザーの行動レベルを判定したユーザー行動データと、（ｂ）ユーザー行動データに対して１段上の行動レベルＬｖの正解・お手本データと、の両方を順に再生する。 1. The advice presentation unit 50 sequentially reproduces both (a) the user behavior data for determining the user's behavior level and (b) the correct answer / model data of the behavior level Lv one step higher than the user behavior data. ..
　２．助言提示部５０は、１．に加えて、さらに（ｃ）ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルＬｖの正解・お手本データを順に再生する。 2. The advice presentation unit 50 is 1. In addition to (c), the correct answer / model data of the behavior level Lv n (n> 1) higher than the user behavior data is reproduced in order.
　３．助言提示部５０は、１．または２．の提示方法のうち、ユーザー行動データと、（ｂ），（ｃ）との差が大きい箇所について、音量を上げる／再生速度を遅くする。 3. The advice presentation unit 50 is 1. Or 2. Of the presentation methods of, the volume is increased / the reproduction speed is slowed down in the place where the difference between the user behavior data and (b) and (c) is large.
　このように本実施形態によれば、ユーザーの行動に対して、理想の行動結果（プロ・達人のお手本）に至るための段階的な助言となる正解・お手本データを予測し、その予測結果から、ユーザーの行動レベル判定結果の行動レベルの少なくとも一段上の助言となる正解・お手本データを、ユーザー行動レベル判定結果に適応して（重畳して）示す。このようにすることで、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データを得られ、効率的な技術習得・学習が可能となる。 In this way, according to this embodiment, the correct answer / model data that provides step-by-step advice for the user's behavior to reach the ideal behavior result (professional / master's model) is predicted, and the prediction result is used to predict the correct answer / model data. , Correct answer / model data that is at least one step higher than the behavior level of the user behavior level judgment result is shown by adapting (superimposing) to the user behavior level judgment result. By doing so, correct answer / model data that can be used as advice according to the content and level of each user behavior can be obtained, and efficient skill acquisition / learning becomes possible.
　すなわち、本実施形態によれば、段階的に上達の方法を具体的に提示するシステムを実現することができる。また、本実施形態によれば、定型的な助言ではなく、ユーザーの行動内容に応じて助言内容が変化し、効率的な技術習得・学習が可能なシステムを実現することができる。 That is, according to the present embodiment, it is possible to realize a system that concretely presents a method of improvement in stages. Further, according to the present embodiment, it is possible to realize a system in which the content of the advice is changed according to the content of the user's behavior instead of the standard advice, and efficient skill acquisition / learning is possible.
［２．第２の実施形態］
　次に、第２の実施の形態について説明する。 [2. Second embodiment]
Next, the second embodiment will be described.
　第１の実施形態では段階的な助言を行うために学習途中経過をレベリングに利用するようにしたが、第２の実施の形態は、ユーザー行動データおよび正解・お手本データのサンプリング間隔を粗い（ｎ）→密（１）に段階的に変更する点が、第１の実施の形態と異なる。 In the first embodiment, the learning progress is used for leveling in order to give stepwise advice, but in the second embodiment, the sampling interval of the user behavior data and the correct answer / model data is coarse (n). ) → Dense (1) is changed step by step, which is different from the first embodiment.
［２－１．実施形態に係る情報処理の手順］
［２－１－１．データ処理装置に係るユーザー行動のデータ化処理の手順］
　まず、学習済データベース６０に対するデータベース学習方法について説明する。 [2-1. Information processing procedure according to the embodiment]
[2-1-1. Procedure for digitizing user behavior related to data processing equipment]
First, a database learning method for the trained database 60 will be described.
　図１１は、本開示の第２の実施形態に係るデータ処理システム１に係るデータベース学習処理の一例を示す図、図１２は本開示の第２の実施形態に係るデータ処理システム１に係るデータベース学習処理の流れの一例を示すフローチャートである。 FIG. 11 is a diagram showing an example of database learning processing according to the data processing system 1 according to the second embodiment of the present disclosure, and FIG. 12 is a diagram showing database learning according to the data processing system 1 according to the second embodiment of the present disclosure. It is a flowchart which shows an example of the processing flow.
　図１１に示すように、データ処理装置１００は、サンプリング間隔ｉを設定（ｎ～１まで１ずつ減らす）する。なお、データ処理装置１００が実行する処理（１）～処理（６）までは本開示の第１の実施形態と同様である。 As shown in FIG. 11, the data processing apparatus 100 sets the sampling interval i (decreases by 1 from n to 1). The processes (1) to (6) executed by the data processing device 100 are the same as those in the first embodiment of the present disclosure.
　ただし、データ処理装置１００は、図１２に示すように、本開示の第１の実施形態で説明したステップＳ２４のお手本予測変形部４０で学習した結果である正解・お手本データを学習済データベース６０の助言生成用のデータベース６０ｂに記録する処理は行わない。加えて、データ処理装置１００は、図１２に示すように、本開示の第１の実施形態で説明したステップＳ２９の判定学習結果を学習済データベース６０の行動レベル判定用のデータベース６０ａに記録する処理は行わない。 However, as shown in FIG. 12, the data processing device 100 obtains the correct answer / model data, which is the result of learning in the model prediction transformation unit 40 of step S24 described in the first embodiment of the present disclosure, in the trained database 60. The process of recording in the database 60b for producing advice is not performed. In addition, as shown in FIG. 12, the data processing device 100 records the determination learning result of step S29 described in the first embodiment of the present disclosure in the database 60a for determining the action level of the learned database 60. Do not do.
　処理（８）
　図１１に示すように、データ処理装置１００は、サンプリング間隔ｉ＝ｎにおけるステップＳ２８およびステップＳ３０についての学習が終了したら、お手本予測変形部４０の学習結果を助言生成用のデータベース６０ｂに記録するとともに、予測・ユーザー行動判定部３０３の学習結果を行動レベル判定用のデータベース６０ａに記録する（ステップＳ３１）。 Processing (8)
As shown in FIG. 11, when the learning of steps S28 and S30 at the sampling interval i = n is completed, the data processing apparatus 100 records the learning result of the model prediction transformation unit 40 in the database 60b for advice generation. , The learning result of the prediction / user action determination unit 303 is recorded in the database 60a for action level determination (step S31).
　データ処理装置１００は、サンプリング間隔をｎ－１としてデータサンプリング間隔を密方向に変更し、処理（１）～（６）、および処理（８）を繰り返し実行する。データ処理装置１００は、サンプリング間隔ｉ＝１となるまで、処理（１）～（６）、および処理（８）を繰り返し実行する。 The data processing apparatus 100 sets the sampling interval to n-1 and changes the data sampling interval in the dense direction, and repeatedly executes the processes (1) to (6) and the process (8). The data processing apparatus 100 repeatedly executes the processes (1) to (6) and the process (8) until the sampling interval i = 1.
　データのサンプリング間隔が粗い状態では、対象データの概略を対象に学習を行うので、行動レベルの基本的な部分を学習することができる。一方、データのサンプリング間隔が密となった状態では、対象データの細かい部分を対象に学習を行うので、行動レベルの上級の部分を学習することができる。 In a state where the data sampling interval is coarse, learning is performed on the outline of the target data, so that the basic part of the behavior level can be learned. On the other hand, in a state where the data sampling intervals are close, learning is performed on a small part of the target data, so that it is possible to learn an advanced part of the behavior level.
　なお、サンプリング間隔を変更する以外にも、ＬＰＦ（Low-Pass　Filter）などの帯域制限フィルタによってデータの高周波情報（細かい）情報を遮断することでも、同様の結果が得られる。 In addition to changing the sampling interval, the same result can be obtained by blocking the high-frequency information (fine) information of the data with a band limiting filter such as LPF (Low-Pass Filter).
　このように本実施形態によれば、データ処理装置１００の行動レベル判定部３０およびお手本予測変形部４０は、ユーザー行動データのサンプリング間隔を段階的にすることにより、段階的に助言を提示することができる。 As described above, according to the present embodiment, the behavior level determination unit 30 and the model prediction transformation unit 40 of the data processing device 100 present advice step by step by gradually setting the sampling interval of the user behavior data. Can be done.
［３．その他の実施形態］
　上述した各実施形態に係る処理は、上記各実施形態や変形例以外にも種々の異なる形態（変形例）にて実施されてよい。 [3. Other embodiments]
The processing according to each of the above-described embodiments may be carried out in various different forms (modifications) in addition to the above-mentioned embodiments and modifications.
［３－１．その他の構成例］
　なお、上記の例では、データ処理装置１００とデータ化装置２００とが別体である場合を示したが、これらの装置は一体であってもよい。データ処理装置１００は、カメラ、スマートフォン、テレビ、自動車、ドローン、ロボット等であってもよい。このように、データ処理装置１００は、自律的に影響度の高い学習データを収集する端末装置であってもよい。 [3-1. Other configuration examples]
In the above example, the case where the data processing device 100 and the data conversion device 200 are separate bodies is shown, but these devices may be integrated. The data processing device 100 may be a camera, a smartphone, a television, a car, a drone, a robot, or the like. As described above, the data processing device 100 may be a terminal device that autonomously collects learning data having a high degree of influence.
［３－２．その他］
　また、上記各実施形態において説明した各処理のうち、自動的に行われるものとして説明した処理の全部または一部を手動的に行うこともでき、あるいは、手動的に行われるものとして説明した処理の全部または一部を公知の方法で自動的に行うこともできる。この他、上記文書中や図面中で示した処理手順、具体的名称、各種のデータやパラメータを含む情報については、特記する場合を除いて任意に変更することができる。例えば、各図に示した各種情報は、図示した情報に限られない。 [3-2. others]
Further, among the processes described in each of the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed. It is also possible to automatically perform all or part of the above by a known method. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown in the figure.
　また、図示した各装置の各構成要素は機能概念的なものであり、必ずしも物理的に図示の如く構成されていることを要しない。すなわち、各装置の分散・統合の具体的形態は図示のものに限られず、その全部または一部を、各種の負荷や使用状況などに応じて、任意の単位で機能的または物理的に分散・統合して構成することができる。 Further, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of them may be functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured.
　また、上述してきた各実施形態は、処理内容を矛盾させない範囲で適宜組み合わせることが可能である。 Further, each of the above-described embodiments can be appropriately combined as long as the processing contents do not contradict each other.
　また、本明細書に記載された効果はあくまで例示であって限定されるものでは無く、他の効果があってもよい。本明細書に記載された効果は、少なくとも１つであっても良い。 Further, the effects described in the present specification are merely examples and are not limited, and other effects may be obtained. The effects described herein may be at least one.
［４．本開示に係る効果］
　上述のように、本開示に係る学習装置（実施形態ではデータ処理装置１００）は、行動レベル判定部と、予測変形部と、提示部と、を備える。行動レベル判定部は、ユーザー行動データに基づくユーザーの行動レベルを判定する。予測変形部は、現在のユーザーの行動レベルを判定した行動レベル判定結果に基づき、ユーザー行動データを、理想の行動結果に至るための正解・お手本データに予測変形する。提示部は、正解・お手本データとユーザー行動データとの差を提示する。 [4. Effect of this disclosure]
As described above, the learning device (data processing device 100 in the embodiment) according to the present disclosure includes an action level determination unit, a prediction deformation unit, and a presentation unit. The behavior level determination unit determines the user's behavior level based on the user behavior data. The predictive transformation unit predictively transforms the user behavior data into correct answer / model data for reaching the ideal behavior result based on the behavior level determination result of determining the current user behavior level. The presentation unit presents the difference between the correct answer / model data and the user behavior data.
　このように、本開示に係る学習装置は、ユーザーの行動に対して、理想の行動結果（プロ・達人のお手本）に至るための段階的な助言となる正解・お手本データを予測し、その予測結果から、ユーザーの行動レベル判定結果の行動レベルの少なくとも一段上の助言となる正解・お手本データとユーザー行動データとの差を提示する。このようにすることで、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データを得られ、効率的な技術習得・学習が可能となる。 In this way, the learning device according to the present disclosure predicts the correct answer / model data that provides step-by-step advice for the user's behavior to reach the ideal behavior result (professional / master's model), and predicts the correct answer / model data. From the result, the difference between the correct answer / model data and the user behavior data, which is at least one step higher than the behavior level of the user's behavior level judgment result, is presented. By doing so, correct answer / model data that can be used as advice according to the content and level of each user behavior can be obtained, and efficient skill acquisition / learning becomes possible.
　行動レベル判定部および予測変形部は、機械学習によって、行動レベル判定部における行動レベル判定結果と、予測変形部における正解・お手本データと、をペアリングして交互に学習する。したがって、行動レベルのレベリングを自動で行うことができる。 The behavior level determination unit and the prediction transformation unit pair the behavior level determination result in the behavior level determination unit with the correct answer / model data in the prediction transformation unit by machine learning, and learn alternately. Therefore, the behavior level can be automatically leveled.
　予測変形部は、機械学習における学習収束に至る学習途中の結果を正解・お手本データとして複数組み合わせる。したがって、段階的に助言を提示することができる。 The predictive transformation unit combines multiple results during learning leading to learning convergence in machine learning as correct answer / model data. Therefore, advice can be presented step by step.
　行動レベル判定部および予測変形部は、ユーザー行動データのサンプリング間隔を段階的にする。したがって、段階的に助言を提示することができる。 The behavior level judgment unit and the prediction transformation unit gradually set the sampling interval of user behavior data. Therefore, advice can be presented step by step.
　予測変形部は、理想の行動結果を、ユーザーの行動レベル判定結果に対して１段階以上の行動レベル相当の行動結果とする。したがって、段階的に上達の方法を具体的に提示することができる。 The predictive transformation unit sets the ideal action result as the action result equivalent to one or more levels of the action level judgment result of the user. Therefore, it is possible to specifically present a method for improving step by step.
　本開示に係る提示装置（実施形態ではデータ処理装置１００）は、提示部を備える。提示部は、ユーザー行動データと、前記ユーザー行動データに基づいてユーザーの行動レベルを判定した行動レベル判定結果に基づき当該ユーザー行動データを理想の行動結果に至るために予測変形した正解・お手本データと、の差を提示する。 The presentation device (data processing device 100 in the embodiment) according to the present disclosure includes a presentation unit. The presentation unit includes the user behavior data and the correct answer / model data obtained by predicting and transforming the user behavior data to reach the ideal behavior result based on the behavior level judgment result in which the user behavior level is determined based on the user behavior data. Show the difference between.
　このように、本開示に係る提示装置は、ユーザーの行動に対して、理想の行動結果（プロ・達人のお手本）に至るための段階的な助言となる正解・お手本データを予測した結果から、ユーザーの行動レベル判定結果の行動レベルの少なくとも一段上の助言となる正解・お手本データとユーザー行動データとの差を提示する。このようにすることで、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データを得られ、効率的な技術習得・学習が可能となる。 In this way, the presentation device according to the present disclosure predicts the correct answer / model data that provides step-by-step advice for the user's behavior to reach the ideal behavior result (professional / master's model). The difference between the correct answer / model data and the user behavior data, which is at least one step higher than the behavior level of the user's behavior level judgment result, is presented. By doing so, correct answer / model data that can be used as advice according to the content and level of each user behavior can be obtained, and efficient skill acquisition / learning becomes possible.
　提示部は、ユーザー行動データが画像データまたは音声データである場合、ユーザー行動データと、ユーザー行動データに対して１段上の行動レベルの正解・お手本データとを、重ねて表示するまたは並べて表示する。したがって、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データとを比較することができ、効率的な技術習得・学習が可能となる。 When the user behavior data is image data or voice data, the presentation unit displays the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in an overlapping manner or side by side. .. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
　提示部は、さらに、ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルの正解・お手本データを、重ねて表示するまたは並べて表示する。したがって、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データとを比較することができ、効率的な技術習得・学習が可能となる。 The presentation unit further displays the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in an overlapping manner or side by side. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
　提示部は、正解・お手本データについて、ユーザー行動データとの差が大きい箇所を強調表示する。したがって、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データとの差が大きい箇所を確認することができ、効率的な技術習得・学習が可能となる。 The presentation section highlights the points where there is a large difference between the correct answer / model data and the user behavior data. Therefore, it is possible to confirm the part where there is a large difference between the correct answer / model data, which is the advice according to the content and level of each user behavior, and it is possible to efficiently acquire and learn the technique.
　提示部は、ユーザー行動データが画像データまたは音声データである場合、ユーザー行動データと、ユーザー行動データとユーザー行動データに対して１段上の行動レベルの正解・お手本データとの差分を、重ねて表示するまたは並べて表示する。したがって、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データとを比較することができ、効率的な技術習得・学習が可能となる。 When the user behavior data is image data or voice data, the presentation unit superimposes the difference between the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data and the user behavior data. Display or display side by side. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
　提示部は、さらに、ユーザー行動データとユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルの正解・お手本データの差分を、重ねて表示するまたは並べて表示する。したがって、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データとを比較することができ、効率的な技術習得・学習が可能となる。 The presentation unit further displays the difference between the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data and the user behavior data, or displays them side by side. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
　提示部は、正解・お手本データについて、ユーザー行動データとの差が大きい箇所を強調表示する。したがって、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データとの差が大きい箇所を確認することができ、効率的な技術習得・学習が可能となる。 The presentation section highlights the points where there is a large difference between the correct answer / model data and the user behavior data. Therefore, it is possible to confirm the part where there is a large difference between the correct answer / model data, which is the advice according to the content and level of each user behavior, and it is possible to efficiently acquire and learn the technique.
　提示部は、ユーザー行動データが音声データである場合、ユーザー行動データと、ユーザー行動データに対して１段上の行動レベルの正解・お手本データとを、順に再生する。したがって、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データとを比較することができ、効率的な技術習得・学習が可能となる。 When the user behavior data is voice data, the presentation unit reproduces the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in order. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
　提示部は、さらに、ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルの前記正解・お手本データを順に再生する。したがって、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データとを比較することができ、効率的な技術習得・学習が可能となる。 The presentation unit further reproduces the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in order. Therefore, it is possible to compare the content of each user behavior with the correct answer / model data that provides advice according to the level, and it is possible to efficiently acquire and learn skills.
　提示部は、前記正解・お手本データについて、ユーザー行動データとの差が大きい箇所を、音量を上げるまたは再生速度を遅くする。したがって、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データとの差が大きい箇所を確認することができ、効率的な技術習得・学習が可能となる。 The presentation unit raises the volume or slows down the playback speed of the correct answer / model data where there is a large difference from the user behavior data. Therefore, it is possible to confirm the part where there is a large difference between the correct answer / model data, which is the advice according to the content and level of each user behavior, and it is possible to efficiently acquire and learn the technique.
　上述のように、本開示に係る技術習得方法は、行動レベル判定ステップと、予測変形ステップと、提示ステップと、を含む。行動レベル判定ステップは、ユーザー行動データに基づくユーザーの行動レベルを判定する。予測変形ステップは、現在のユーザーの行動レベルを判定した行動レベル判定結果に基づき、ユーザー行動データを、理想の行動結果に至るために行動レベル判定結果の行動レベルより優れた正解・お手本データに予測変形する。提示ステップは、正解・お手本データとユーザー行動データとの差を提示する。 As described above, the technique acquisition method according to the present disclosure includes an action level determination step, a predictive transformation step, and a presentation step. The behavior level determination step determines the user's behavior level based on the user behavior data. The prediction transformation step predicts the user behavior data to the correct answer / model data superior to the behavior level of the behavior level judgment result in order to reach the ideal behavior result based on the behavior level judgment result of judging the current user behavior level. transform. The presentation step presents the difference between the correct answer / model data and the user behavior data.
　このように、本開示に係る技術習得方法は、ユーザーの行動に対して、理想の行動結果（プロ・達人のお手本）に至るための段階的な助言となる正解・お手本データを予測し、その予測結果から、ユーザーの行動レベル判定結果の行動レベルの少なくとも一段上の助言となる正解・お手本データとユーザー行動データとの差を提示する。このようにすることで、各ユーザー行動の内容とレベルに応じた助言となる正解・お手本データを得られ、効率的な技術習得・学習が可能となる。 In this way, the technology acquisition method related to this disclosure predicts correct answer / model data that will be step-by-step advice for the user's behavior to reach the ideal behavior result (professional / master's model). From the prediction result, the difference between the correct answer / model data and the user behavior data, which is at least one level of advice on the behavior level of the user behavior level judgment result, is presented. By doing so, correct answer / model data that can be used as advice according to the content and level of each user behavior can be obtained, and efficient skill acquisition / learning becomes possible.
［５．ハードウェア構成］
　上述してきた各実施形態や変形例に係るデータ処理装置１００やデータ化装置２００等の情報機器は、例えば図１３に示すような構成のコンピュータ１０００によって実現される。図１３は、データ処理装置１００やデータ化装置２００等の情報処理装置の機能を実現するコンピュータ１０００の一例を示すハードウェア構成図である。以下、実施形態に係るデータ処理装置１００を例に挙げて説明する。コンピュータ１０００は、ＣＰＵ１１００、ＲＡＭ１２００、ＲＯＭ（Read　Only　Memory）１３００、ＨＤＤ（Hard　Disk　Drive）１４００、通信インターフェイス１５００、及び入出力インターフェイス１６００を有する。コンピュータ１０００の各部は、バス１０５０によって接続される。 [5. Hardware configuration]
Information devices such as the data processing device 100 and the data conversion device 200 according to each of the above-described embodiments and modifications are realized by a computer 1000 having a configuration as shown in FIG. 13, for example. FIG. 13 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of information processing devices such as a data processing device 100 and a data processing device 200. Hereinafter, the data processing apparatus 100 according to the embodiment will be described as an example. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each part of the computer 1000 is connected by a bus 1050.
　ＣＰＵ１１００は、ＲＯＭ１３００又はＨＤＤ１４００に格納されたプログラムに基づいて動作し、各部の制御を行う。例えば、ＣＰＵ１１００は、ＲＯＭ１３００又はＨＤＤ１４００に格納されたプログラムをＲＡＭ１２００に展開し、各種プログラムに対応した処理を実行する。 The CPU 1100 operates based on the program stored in the ROM 1300 or the HDD 1400, and controls each part. For example, the CPU 1100 expands the program stored in the ROM 1300 or the HDD 1400 into the RAM 1200, and executes processing corresponding to various programs.
　ＲＯＭ１３００は、コンピュータ１０００の起動時にＣＰＵ１１００によって実行されるＢＩＯＳ（Basic　Input　Output　System）等のブートプログラムや、コンピュータ１０００のハードウェアに依存するプログラム等を格納する。 The ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started, a program depending on the hardware of the computer 1000, and the like.
　ＨＤＤ１４００は、ＣＰＵ１１００によって実行されるプログラム、及び、かかるプログラムによって使用されるデータ等を非一時的に記録する、コンピュータが読み取り可能な記録媒体である。具体的には、ＨＤＤ１４００は、プログラムデータ１４５０の一例である本開示に係る情報処理プログラムを記録する記録媒体である。 The HDD 1400 is a computer-readable recording medium that non-temporarily records a program executed by the CPU 1100 and data used by the program. Specifically, the HDD 1400 is a recording medium for recording an information processing program according to the present disclosure, which is an example of program data 1450.
　通信インターフェイス１５００は、コンピュータ１０００が外部ネットワーク１５５０（例えばインターネット）と接続するためのインターフェイスである。例えば、ＣＰＵ１１００は、通信インターフェイス１５００を介して、他の機器からデータを受信したり、ＣＰＵ１１００が生成したデータを他の機器へ送信したりする。 The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (for example, the Internet). For example, the CPU 1100 receives data from another device or transmits data generated by the CPU 1100 to another device via the communication interface 1500.
　入出力インターフェイス１６００は、入出力デバイス１６５０とコンピュータ１０００とを接続するためのインターフェイスである。例えば、ＣＰＵ１１００は、入出力インターフェイス１６００を介して、キーボードやマウス等の入力デバイスからデータを受信する。また、ＣＰＵ１１００は、入出力インターフェイス１６００を介して、ディスプレイやスピーカーやプリンタ等の出力デバイスにデータを送信する。また、入出力インターフェイス１６００は、所定の記録媒体（メディア）に記録されたプログラム等を読み取るメディアインターフェイスとして機能してもよい。メディアとは、例えばＤＶＤ（Digital　Versatile　Disc）、ＰＤ（Phase　change　rewritable　Disk）等の光学記録媒体、ＭＯ（Magneto-Optical　disk）等の光磁気記録媒体、テープ媒体、磁気記録媒体、または半導体メモリ等である。 The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or mouse via the input / output interface 1600. Further, the CPU 1100 transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. Further, the input / output interface 1600 may function as a media interface for reading a program or the like recorded on a predetermined recording medium (media). The media is, for example, an optical recording medium such as DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk), a magneto-optical recording medium such as MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Is.
　例えば、コンピュータ１０００が実施形態に係るデータ処理装置１００として機能する場合、コンピュータ１０００のＣＰＵ１１００は、ＲＡＭ１２００上にロードされた情報処理プログラムを実行することにより、制御部１３０等の機能を実現する。また、ＨＤＤ１４００には、本開示に係る情報処理プログラムや、記憶部１２０内のデータが格納される。なお、ＣＰＵ１１００は、プログラムデータ１４５０をＨＤＤ１４００から読み取って実行するが、他の例として、外部ネットワーク１５５０を介して、他の装置からこれらのプログラムを取得してもよい。 For example, when the computer 1000 functions as the data processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 and the like by executing the information processing program loaded on the RAM 1200. Further, the information processing program according to the present disclosure and the data in the storage unit 120 are stored in the HDD 1400. The CPU 1100 reads the program data 1450 from the HDD 1400 and executes the program, but as another example, these programs may be acquired from another device via the external network 1550.
　なお、本技術は以下のような構成も取ることができる。
（１）
　ユーザー行動データに基づくユーザーの行動レベルを判定する行動レベル判定部と、
　現在のユーザーの行動レベルを判定した行動レベル判定結果に基づき、前記ユーザー行動データを、理想の行動結果に至るために前記行動レベル判定結果の行動レベルより優れた正解・お手本データに予測変形する予測変形部と、
　前記正解・お手本データと前記ユーザー行動データとの差を提示する提示部と、
を備えることを特徴とする学習装置。
（２）
　前記行動レベル判定部および前記予測変形部は、機械学習によって、前記行動レベル判定部における前記行動レベル判定結果と、前記予測変形部における前記正解・お手本データと、をペアリングして交互に学習する、
ことを特徴とする（１）に記載の学習装置。
（３）
　前記予測変形部は、前記機械学習における学習収束に至る学習途中の結果を前記正解・お手本データとして複数組み合わせる、
ことを特徴とする（２）に記載の学習装置。
（４）
　前記行動レベル判定部および前記予測変形部は、前記ユーザー行動データのサンプリング間隔を段階的にする、
ことを特徴とする（１）または（２）に記載の学習装置。
（５）
　前記予測変形部は、前記理想の行動結果を、ユーザーの前記行動レベル判定結果に対して１段階以上の行動レベル相当の行動結果とする、
ことを特徴とする（１）ないし（４）の何れか一つに記載の学習装置。
（６）
　ユーザー行動データと、前記ユーザー行動データに基づいてユーザーの行動レベルを判定した行動レベル判定結果に基づき当該ユーザー行動データを理想の行動結果に至るために予測変形した正解・お手本データと、の差を提示する提示部を備える、
ことを特徴とする提示装置。
（７）
　前記提示部は、前記ユーザー行動データが画像データまたは音声データである場合、前記ユーザー行動データと、前記ユーザー行動データに対して１段上の行動レベルの前記正解・お手本データとを、重ねて表示するまたは並べて表示する、
ことを特徴とする（６）に記載の提示装置。
（８）
　前記提示部は、さらに、前記ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルの前記正解・お手本データを、重ねて表示するまたは並べて表示する、
ことを特徴とする（７）に記載の提示装置。
（９）
　前記提示部は、前記正解・お手本データについて、前記ユーザー行動データとの差が大きい箇所を強調表示する、
ことを特徴とする（７）または（８）に記載の提示装置。
（１０）
　前記提示部は、前記ユーザー行動データが画像データまたは音声データである場合、前記ユーザー行動データと、前記ユーザー行動データと前記ユーザー行動データに対して１段上の行動レベルの前記正解・お手本データとの差分を、重ねて表示するまたは並べて表示する、
ことを特徴とする（６）に記載の提示装置。
（１１）
　前記提示部は、さらに、前記ユーザー行動データと前記ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルの前記正解・お手本データの差分を、重ねて表示するまたは並べて表示する、
ことを特徴とする（１０）に記載の提示装置。
（１２）
　前記提示部は、前記正解・お手本データについて、前記ユーザー行動データとの差が大きい箇所を強調表示する、
ことを特徴とする（１０）または（１１）に記載の提示装置。
（１３）
　前記提示部は、前記ユーザー行動データが音声データである場合、前記ユーザー行動データと、前記ユーザー行動データに対して１段上の行動レベルの前記正解・お手本データとを、順に再生する、
ことを特徴とする（６）に記載の提示装置。
（１４）
　前記提示部は、さらに、前記ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルの前記正解・お手本データを順に再生する、
ことを特徴とする（１３）に記載の提示装置。
（１５）
　前記提示部は、前記正解・お手本データについて、前記ユーザー行動データとの差が大きい箇所を、音量を上げるまたは再生速度を遅くする、
ことを特徴とする（１３）または（１４）に記載の提示装置。
（１６）
　ユーザー行動データに基づくユーザーの行動レベルを判定する行動レベル判定ステップと、
　現在のユーザーの行動レベルを判定した行動レベル判定結果に基づき、前記ユーザー行動データを、理想の行動結果に至るために前記行動レベル判定結果の行動レベルより優れた正解・お手本データに予測変形する予測変形ステップと、
　前記正解・お手本データと前記ユーザー行動データとの差を提示する提示ステップと、
を含むことを特徴とする技術習得方法。 The present technology can also have the following configurations.
(1)
The behavior level judgment unit that judges the user's behavior level based on the user behavior data,
Prediction that the user behavior data is predicted and transformed into correct answer / model data superior to the behavior level of the behavior level judgment result in order to reach the ideal behavior result based on the behavior level judgment result of judging the current user behavior level. Deformed part and
A presentation unit that presents the difference between the correct answer / model data and the user behavior data,
A learning device characterized by being equipped with.
(2)
The behavior level determination unit and the prediction transformation unit alternately learn by pairing the behavior level determination result in the behavior level determination unit and the correct answer / model data in the prediction transformation unit by machine learning. ,
The learning device according to (1).
(3)
The predictive transformation unit combines a plurality of results during learning leading to learning convergence in the machine learning as the correct answer / model data.
The learning device according to (2).
(4)
The behavior level determination unit and the prediction transformation unit gradually set the sampling interval of the user behavior data.
The learning device according to (1) or (2).
(5)
The predictive transformation unit sets the ideal action result as an action result corresponding to one or more steps of the action level determination result of the user.
The learning device according to any one of (1) to (4).
(6)
The difference between the user behavior data and the correct answer / model data obtained by predicting and transforming the user behavior data to reach the ideal behavior result based on the behavior level judgment result in which the user behavior level is determined based on the user behavior data. Equipped with a presentation unit to present
A presentation device characterized by that.
(7)
When the user behavior data is image data or voice data, the presentation unit displays the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in an overlapping manner. Or display side by side,
The presentation device according to (6).
(8)
The presentation unit further displays the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in an overlapping manner or side by side.
The presentation device according to (7).
(9)
The presentation unit highlights the points where the difference between the correct answer / model data and the user behavior data is large.
The presentation device according to (7) or (8).
(10)
When the user behavior data is image data or voice data, the presentation unit includes the user behavior data, the user behavior data, and the correct answer / model data of the behavior level one step higher than the user behavior data. Display the differences in layers or side by side,
The presentation device according to (6).
(11)
The presentation unit further displays or side by side the difference between the user behavior data and the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data.
The presentation device according to (10).
(12)
The presentation unit highlights the points where the difference between the correct answer / model data and the user behavior data is large.
The presentation device according to (10) or (11).
(13)
When the user behavior data is voice data, the presentation unit reproduces the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in order.
The presentation device according to (6).
(14)
The presentation unit further reproduces the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in order.
The presentation device according to (13).
(15)
The presenting unit raises the volume or slows down the reproduction speed of the correct answer / model data at a place where the difference from the user behavior data is large.
The presentation device according to (13) or (14).
(16)
The behavior level determination step that determines the user's behavior level based on the user behavior data,
Prediction that the user behavior data is predicted and transformed into correct answer / model data superior to the behavior level of the behavior level judgment result in order to reach the ideal behavior result based on the behavior level judgment result of judging the current user behavior level. Transformation step and
A presentation step that presents the difference between the correct answer / model data and the user behavior data,
A technique acquisition method characterized by including.
　３０　　　行動レベル判定部
　４０　　　お手本予測変形部
　５０　　　助言提示部
　１００　　データ処理装置
 
  30 Behavior level judgment unit 40 Model prediction transformation unit 50 Advice presentation unit 100 Data processing device

Claims (16)

  1. 　ユーザー行動データに基づくユーザーの行動レベルを判定する行動レベル判定部と、
    　現在のユーザーの行動レベルを判定した行動レベル判定結果に基づき、前記ユーザー行動データを、理想の行動結果に至るために前記行動レベル判定結果の行動レベルより優れた正解・お手本データに予測変形する予測変形部と、
    　前記正解・お手本データと前記ユーザー行動データとの差を提示する提示部と、
    を備える学習装置。     The behavior level judgment unit that judges the user's behavior level based on the user behavior data,
    Prediction that the user behavior data is predicted and transformed into correct answer / model data superior to the behavior level of the behavior level judgment result in order to reach the ideal behavior result based on the behavior level judgment result of judging the current user behavior level. Deformed part and
    A presentation unit that presents the difference between the correct answer / model data and the user behavior data,
    A learning device equipped with.
  2. 　前記行動レベル判定部および前記予測変形部は、機械学習によって、前記行動レベル判定部における前記行動レベル判定結果と、前記予測変形部における前記正解・お手本データと、をペアリングして交互に学習する、
    請求項１に記載の学習装置。     The behavior level determination unit and the prediction transformation unit alternately learn by pairing the behavior level determination result in the behavior level determination unit and the correct answer / model data in the prediction transformation unit by machine learning. ,
    The learning device according to claim 1.
  3. 　前記予測変形部は、前記機械学習における学習収束に至る学習途中の結果を前記正解・お手本データとして複数組み合わせる、
    請求項２に記載の学習装置。     The predictive transformation unit combines a plurality of results during learning leading to learning convergence in the machine learning as the correct answer / model data.
    The learning device according to claim 2.
  4. 　前記行動レベル判定部および前記予測変形部は、前記ユーザー行動データのサンプリング間隔を段階的にする、
    請求項１に記載の学習装置。     The behavior level determination unit and the prediction transformation unit gradually set the sampling interval of the user behavior data.
    The learning device according to claim 1.
  5. 　前記予測変形部は、前記理想の行動結果を、ユーザーの前記行動レベル判定結果に対して１段階以上の行動レベル相当の行動結果とする、
    請求項１に記載の学習装置。     The predictive transformation unit sets the ideal action result as an action result corresponding to one or more steps of the action level determination result of the user.
    The learning device according to claim 1.
  6. 　ユーザー行動データと、前記ユーザー行動データに基づいてユーザーの行動レベルを判定した行動レベル判定結果に基づき当該ユーザー行動データを理想の行動結果に至るために予測変形した正解・お手本データと、の差を提示する提示部を備える、
    提示装置。     The difference between the user behavior data and the correct answer / model data obtained by predicting and transforming the user behavior data to reach the ideal behavior result based on the behavior level judgment result in which the user behavior level is determined based on the user behavior data. Equipped with a presentation unit to present
    Presentation device.
  7. 　前記提示部は、前記ユーザー行動データが画像データまたは音声データである場合、前記ユーザー行動データと、前記ユーザー行動データに対して１段上の行動レベルの前記正解・お手本データとを、重ねて表示するまたは並べて表示する、
    請求項６に記載の提示装置。     When the user behavior data is image data or voice data, the presentation unit displays the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in an overlapping manner. Or display side by side,
    The presentation device according to claim 6.
  8. 　前記提示部は、さらに、前記ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルの前記正解・お手本データを、重ねて表示するまたは並べて表示する、
    請求項７に記載の提示装置。     The presentation unit further displays the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in an overlapping manner or side by side.
    The presentation device according to claim 7.
  9. 　前記提示部は、前記正解・お手本データについて、前記ユーザー行動データとの差が大きい箇所を強調表示する、
    請求項７に記載の提示装置。     The presentation unit highlights the points where the difference between the correct answer / model data and the user behavior data is large.
    The presentation device according to claim 7.
  10. 　前記提示部は、前記ユーザー行動データが画像データまたは音声データである場合、前記ユーザー行動データと、前記ユーザー行動データと前記ユーザー行動データに対して１段上の行動レベルの前記正解・お手本データとの差分を、重ねて表示するまたは並べて表示する、
    請求項６に記載の提示装置。     When the user behavior data is image data or voice data, the presentation unit includes the user behavior data, the user behavior data, and the correct answer / model data of the behavior level one step higher than the user behavior data. Display the differences in layers or side by side,
    The presentation device according to claim 6.
  11. 　前記提示部は、さらに、前記ユーザー行動データと前記ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルの前記正解・お手本データの差分を、重ねて表示するまたは並べて表示する、
    請求項１０に記載の提示装置。     The presentation unit further displays or side by side the difference between the user behavior data and the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data.
    The presentation device according to claim 10.
  12. 　前記提示部は、前記正解・お手本データについて、前記ユーザー行動データとの差が大きい箇所を強調表示する、
    請求項１０に記載の提示装置。     The presentation unit highlights the points where the difference between the correct answer / model data and the user behavior data is large.
    The presentation device according to claim 10.
  13. 　前記提示部は、前記ユーザー行動データが音声データである場合、前記ユーザー行動データと、前記ユーザー行動データに対して１段上の行動レベルの前記正解・お手本データとを、順に再生する、
    請求項６に記載の提示装置。     When the user behavior data is voice data, the presentation unit reproduces the user behavior data and the correct answer / model data of the behavior level one step higher than the user behavior data in order.
    The presentation device according to claim 6.
  14. 　前記提示部は、さらに、前記ユーザー行動データに対してｎ（ｎ＞１）段上の行動レベルの前記正解・お手本データを順に再生する、
    請求項１３に記載の提示装置。     The presentation unit further reproduces the correct answer / model data of the behavior level n (n> 1) higher than the user behavior data in order.
    The presentation device according to claim 13.
  15. 　前記提示部は、前記正解・お手本データについて、前記ユーザー行動データとの差が大きい箇所を、音量を上げるまたは再生速度を遅くする、
    請求項１３に記載の提示装置。     The presenting unit raises the volume or slows down the reproduction speed of the correct answer / model data at a place where the difference from the user behavior data is large.
    The presentation device according to claim 13.
  16. 　ユーザー行動データに基づくユーザーの行動レベルを判定する行動レベル判定ステップと、
    　現在のユーザーの行動レベルを判定した行動レベル判定結果に基づき、前記ユーザー行動データを、理想の行動結果に至るために前記行動レベル判定結果の行動レベルより優れた正解・お手本データに予測変形する予測変形ステップと、
    　前記正解・お手本データと前記ユーザー行動データとの差を提示する提示ステップと、
    を含む技術習得方法。
     
          The behavior level determination step that determines the user's behavior level based on the user behavior data,
    Prediction that the user behavior data is predicted and transformed into correct answer / model data superior to the behavior level of the behavior level judgment result in order to reach the ideal behavior result based on the behavior level judgment result of judging the current user behavior level. Transformation step and
    A presentation step that presents the difference between the correct answer / model data and the user behavior data,
    How to learn skills including.
PCT/JP2021/020914 2020-06-08 2021-06-01 Learning device, presentation device, and technique acquisition method WO2021251222A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020099073 2020-06-08
JP2020-099073 2020-06-08

Publications (1)

Publication Number Publication Date
WO2021251222A1 true WO2021251222A1 (en) 2021-12-16

Family

ID=78845614

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/020914 WO2021251222A1 (en) 2020-06-08 2021-06-01 Learning device, presentation device, and technique acquisition method

Country Status (1)

Country Link
WO (1) WO2021251222A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006139162A (en) * 2004-11-15 2006-06-01 Yamaha Corp Language learning system
JP2019024550A (en) * 2017-07-25 2019-02-21 株式会社クオンタム Detection device, detection system, processing device, detection method and detection program
JP2019040194A (en) * 2018-10-04 2019-03-14 カシオ計算機株式会社 Electronic apparatus, speech output recording method, and program
JP2020034849A (en) * 2018-08-31 2020-03-05 オムロン株式会社 Work support device, work support method, and work support program

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006139162A (en) * 2004-11-15 2006-06-01 Yamaha Corp Language learning system
JP2019024550A (en) * 2017-07-25 2019-02-21 株式会社クオンタム Detection device, detection system, processing device, detection method and detection program
JP2020034849A (en) * 2018-08-31 2020-03-05 オムロン株式会社 Work support device, work support method, and work support program
JP2019040194A (en) * 2018-10-04 2019-03-14 カシオ計算機株式会社 Electronic apparatus, speech output recording method, and program

Similar Documents

Publication Publication Date Title
CN109801194B (en) Follow-up teaching method with remote evaluation function
WO2021082823A1 (en) Audio processing method, apparatus, computer device, and storage medium
CN109697906B (en) Following teaching method based on Internet teaching platform
US20150088513A1 (en) Sound processing system and related method
JP2020197778A (en) Information processing device, control method, and program
JP5217250B2 (en) Learning device and learning method, information processing device and information processing method, and program
WO2021251222A1 (en) Learning device, presentation device, and technique acquisition method
US11141669B2 (en) Speech synthesizing dolls for mimicking voices of parents and guardians of children
CN108922275B (en) Distance education system and education method
JP2009260718A (en) Image reproduction system and image reproduction processing program
WO2022264461A1 (en) Information processing system and information processing method
KR20230085333A (en) Apparatus for ai based children education solution
US11010036B2 (en) Edit experience for transformation of digital content
WO2021090587A1 (en) Teacher data generation method, trained model generation method, device, recording medium, program, and information processing device
JP2009075937A (en) Equipment operation setting device
JP2023027697A (en) Terminal device, transmission method, transmission program and information processing system
JP6619072B2 (en) SOUND SYNTHESIS DEVICE, SOUND SYNTHESIS METHOD, AND PROGRAM THEREOF
CN113099305A (en) Play control method and device
WO2020100532A1 (en) Information processing device, information processing method, and information processing program
JP6637917B2 (en) Education support system and education support method
JP6199598B2 (en) Skill acquisition device, skill acquisition program, skill acquisition method
US20200388270A1 (en) Speech synthesizing devices and methods for mimicking voices of children for cartoons and other content
Chang et al. Intelligent Analysis and Classification of Piano Music Gestures with Multimodal Recordings
KR102154859B1 (en) Cognition improvement training system for prevention of dementia
JP2020014132A (en) Information processing method and information processing device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21820944

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21820944

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP