WO2022034685A1 - Explanation presentation device, explanation presentation method, and explanation presentation program - Google Patents

Abstract

An explanation presentation device (1) comprises: a feature acquisition unit (102) that acquires a feature amount of an operator's action from a database for storing sensor data acquired by detecting the action and format knowledge which is knowledge of a human's understanding the acquired data; a skill determination unit (103) that determines the skill level of the operator on the basis of the feature amount and registers the skill level in the database; a format knowledge linking unit (104) that links the format knowledge with the feature amount in the database; and an explanation extraction unit (105) that extracts presentation information including the feature amount and the format knowledge linked with the feature amount.

Description

Explanation presentation device, explanation presentation method, and explanation presentation program

This disclosure relates to an explanation presentation device, an explanation presentation method, and an explanation presentation program.

As the behavior of AI (Artificial Integrity) using the trained model acquired by machine learning, a counterfactual explanation such as "If ..., if ... was achieved." There is a presentation of. The anti-real hypothetical explanation explains what will be achieved if a situation occurs that is not actually observed as data but is hypothesized.

For example, in a mortgage loan review using AI, AI will explain to the loan applicant what will happen if a situation that is not actually realized occurs (ie, anti-). A real virtual explanation) may be presented to the applicant. For example, AI says, "If your annual income is ... 10,000 yen higher, you can borrow the loan you applied for." Or "If the balance of other loans is ... 10,000 yen less, you can borrow the loan you applied for." It presents an anti-real virtual explanation such as "."

Further, Patent Document 1 proposes a work support device for making the subject learn the work. This device is a device for presenting a template of instruction content to a target person (for example, a worker) so that the movement of the target person approaches the movement of a model person (for example, a skilled person). As in the case of the above-mentioned mortgage examination, the AI is applied to the device described in Patent Document 1, and the subject is based on the difference in the feature amount between the subject and the model (for example, the difference in skill level). On the other hand, it is possible to present an anti-real virtual explanation such as "If ..., if ... was achieved."

Japanese Unexamined Patent Publication No. 2020-034849 (for example, abstract, paragraphs 0043, 0054)

However, for example, when the feature quantity used in machine learning is tacit knowledge, which is knowledge used empirically but cannot be easily explained in words, the subject is presented. It may not be possible to understand the meaning of the anti-real virtual explanation (that is, the meaning of the behavior of AI). Here, tacit knowledge is subjective knowledge consisting of an individual's past experience, for example, knowledge based on experience or intuition.

This disclosure is made to solve the above problems, and aims to make it easier to understand the anti-real and virtual explanations presented based on the results of machine learning.

The explanation presenting device according to the present disclosure is a feature that acquires the feature amount of the behavior from a database that stores the sensor data obtained by detecting the behavior of the worker and the explicit knowledge that is the acquired knowledge that can be interpreted by a human. The acquisition unit, the skill determination unit that determines the skill level of the worker from the feature amount and registers the skill level in the database, and the explicit knowledge linking unit that links explicit knowledge to the feature amount in the database. And an explanatory extraction unit for extracting presentation information including the feature amount and the explicit knowledge associated with the feature amount.

The explanation presentation method according to the present disclosure is a method executed by the explanation presentation device, and stores the sensor data obtained by detecting the behavior of the worker and the explicit knowledge that is the acquired knowledge that can be interpreted by a human. A step of acquiring the feature amount of the action from the database, a step of determining the skill level of the worker from the feature amount and registering the skill level in the database, and explicit knowledge of the feature amount in the database. It has a step of associating the feature amount and a step of extracting the presentation information including the explicit knowledge associated with the feature amount.

According to this disclosure, it is possible to make it easier to understand the anti-real virtual explanation presented based on the result of machine learning.

It is a figure which shows the example of the hardware composition of the explanatory presentation device. (A) and (B) are diagrams showing an example of the configuration of a learning device that performs machine learning to generate a trained model and a reasoning device that performs inference using the trained model and outputs an inference result. It is a functional block diagram which shows the structure of the explanatory presentation apparatus which concerns on Embodiment 1. FIG. It is a flowchart which shows the generation operation of the trained model in the explanatory presentation apparatus which concerns on Embodiment 1. FIG. It is a figure which shows the example of the model using a fully connected neural network. It is a flowchart which shows the generation operation of the trained model which associates explicit knowledge with the implicit feature quantity in the explanatory presentation apparatus which concerns on Embodiment 1. FIG. It is a flowchart which shows the example of the operation of the explanatory presentation apparatus which concerns on Embodiment 1. FIG. It is a figure which shows the example of the data generated by the explanatory presentation apparatus which concerns on Embodiment 1 in a table format (Table 1). It is a figure which shows the example of the data generated by the explanatory presentation apparatus which concerns on Embodiment 1 in a table format (Table 2). It is a figure which shows the example of the search in the implicit feature space. It is a figure which shows the example of the visualized explanation. It is a functional block diagram which shows the structure of the explanatory presentation apparatus which concerns on Embodiment 2. FIG. It is a flowchart which shows the example of the operation of the explanatory presentation apparatus which concerns on Embodiment 2. FIG. It is a figure which shows the example of the data generated by the explanatory presentation apparatus which concerns on Embodiment 2 in the table form (Table 3). It is a figure which shows the example of the obtained correlation coefficient in tabular form (Table 4). It is a figure which shows the behavior recommended for obtaining the skill obtained by the sum of the values weighted by the correlation coefficient of the explicit knowledge associated with the behavior in the tabular form (Table 5). It is a functional block diagram which shows the structure of the explanatory presentation apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the configuration example of the network which executes multitask learning. It is a flowchart which shows the example of the operation of the explanatory presentation apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the example of the data generated by the explanatory presentation apparatus which concerns on Embodiment 3 in a tabular form (Table 6). It is a figure which visualized the sensor data shown in FIG. 20, and displayed particularly different parts as change # 1 and change # 2. It is a figure which shows the example of Attention Branch Network. It is a functional block diagram which shows the structure of the explanatory presentation apparatus which concerns on Embodiment 4. FIG. It is a functional block diagram which shows the structure of the explanatory presentation apparatus which concerns on Embodiment 5. It is a figure which shows the example of the time division sensor data in the explanatory presentation apparatus which concerns on Embodiment 5. It is a figure which shows the example of the presentation information visualized in the explanatory presentation apparatus which concerns on Embodiment 5.

Hereinafter, the explanation presentation device, the explanation presentation method, and the explanation presentation program according to the embodiment will be described with reference to the drawings. According to the explanation presenting device, the explanation presentation method, and the explanation presentation program according to the embodiment, when the explanation is presented using the result of machine learning, the explanation accompanied by explicit knowledge is presented. For example, even if the anti-real virtual explanation is tacit knowledge, the anti-real virtual explanation can be easily understood. Here, explicit knowledge is knowledge that humans can understand. Explicit knowledge is knowledge that can be explained and expressed by, for example, sentences, charts, mathematical formulas, and the like. The following embodiments are merely examples, and it is possible to appropriately combine the embodiments and change the embodiments as appropriate.

<< Embodiment 1 >>
FIG. 1 is a diagram showing an example of a hardware (H / W) configuration of the explanatory presentation device 1 according to the first embodiment. As shown in FIG. 1, the explanatory presentation device 1 includes a processor 11 as an information processing unit, a memory 12 for storing information, a storage device 13 such as a hard disk drive (HDD) and a solid state drive (SSD), and a storage device 13. It has an operation device 14 as a user interface that accepts user operations. The description presentation device 1 is, for example, a computer. The explanation presenting device 1 may have a display 15 which is an image display unit for displaying information to a human being. The display 15 may be a device having a function of audio output. Further, the explanation presenting device 1 may have a sensor 16 which is a detection unit for detecting the behavior (for example, movement) of a human (for example, a worker). The sensor 16 may be a camera that is an image pickup device that captures an image. The sensor 16 outputs sensor data. When the sensor 16 is a camera, the sensor data includes video data.

The processor 11 executes, for example, generation of a trained model by machine learning and inference using the trained model that is the result of machine learning. In FIG. 1, the processor that performs machine learning and the processor that performs inference are one common processor, but may be different processors. The processor 11 executes, for example, a program that is software stored in the memory 12. The program stored in the memory 12 can include an explanatory presentation program for causing the explanatory presentation device 1 according to the first embodiment to implement the explanatory presentation method. The explanatory presentation program is provided to the explanatory presentation device 1 as a program recorded on a recording medium readable by a computer, for example. Alternatively, the explanatory presentation program is provided to the explanatory presentation device 1 by downloading via a network. The explanatory presentation devices 2 to 5 according to the second to fifth embodiments described later also have the same configuration as the hardware configuration shown in FIG.

FIGS. 2A and 2B show an example of the configuration of a learning device 20 that performs machine learning to generate a trained model and an inference device 30 that performs inference using the trained model and outputs an inference result. It is a figure. As shown in FIG. 2A, the learning device 20 has a data acquisition unit 21 and a model generation unit 22. The learning device 20 may include a trained model storage unit 23. The data acquisition unit 21 and the model generation unit 22 can be realized by, for example, the memory 12 and the processor 11 shown in FIG. The trained model storage unit 23 can be realized by, for example, the storage device 13 shown in FIG.

The data acquisition unit 21 acquires learning data. The training data includes, for example, sensor data (that is, input signal) and correct answer data (that is, teacher signal). The model generation unit 22 generates a trained model used for inferring the optimum output based on the learning data output from the data acquisition unit 21.

As the learning algorithm used by the model generation unit 22, known algorithms such as supervised learning, unsupervised learning, and reinforcement learning can be used. The model generation unit 22 learns by supervised learning according to, for example, a model of a neural network. Supervised learning is a method of learning the features existing in the learning data by giving a set of data of input and result (label) to the learning device as learning data, and making it possible to infer the result from the input. .. The model generation unit 22 generates and outputs a trained model by executing the above learning. The trained model storage unit 23 stores the trained model output from the model generation unit 22.

As shown in FIG. 2B, the inference device 30 includes a data acquisition unit 31 and an inference unit 32. The inference device 30 may include a trained model storage unit 33. The data acquisition unit 31 and the inference unit 32 can be realized by, for example, the memory 12 and the processor 11 shown in FIG. The trained model storage unit 33 can be realized by, for example, the storage device 13 shown in FIG. Although the trained model storage unit 33 and the trained model storage unit 23 shown in FIG. 2A are provided in different storage devices, they may be provided in the same storage device.

The data acquisition unit 31 acquires the sensor data of the worker who is the target person who wants to acquire the skill. The sensor data of the worker is, for example, video data obtained by photographing the worker. The inference unit 32 outputs the inference result obtained by using the trained model generated by the learning device 20. That is, the reasoning unit 32 can output the reasoning result by inputting the sensor data acquired by the data acquisition unit 31 into the trained model stored in the trained model storage unit 33.

FIG. 3 is a functional block diagram showing the configuration of the explanatory presentation device 1 according to the first embodiment. As shown in FIG. 3, the explanation presenting device 1 includes a data storage unit 101, a feature extraction unit 102 as a feature acquisition unit, a skill determination unit 103, an explicit knowledge linking unit 104, and an anti-real virtual explanation. It has an explanatory extraction unit 105, which is an extraction unit, and an explicit knowledge selection unit 106.

The data storage unit 101 is, for example, a part of the storage device 13 shown in FIG. The data storage unit 101 stores the database. The data storage unit 101 may be an external storage device (for example, a storage device of a server on a network) capable of communicating with the explanation presentation device 1. The feature extraction unit 102, the skill determination unit 103, the explicit knowledge association unit 104, the explicit knowledge extraction unit 105, and the explicit knowledge selection unit 106 are realized by, for example, a processor 11 that executes an explicit knowledge presentation program. The feature extraction unit 102, the skill determination unit 103, and the explicit knowledge association unit 104 can form, for example, the learning device 20 shown in FIG. 2 (A). Further, the explanatory extraction unit 105 and the explicit knowledge selection unit 106 can form, for example, the inference device 30 shown in FIG. 2 (B).

At the time of machine learning, the data storage unit 101 uses explicit knowledge acquired from experts in various fields (that is, skilled workers who have already acquired advanced skills) and the same behavior as the expert's behavior (similar behavior). The result obtained by taking.) Is stored as a teacher signal. Here, explicit knowledge is knowledge that can be interpreted by the acquired human being. Explicit knowledge can include, for example, sensor data (for example, video data) obtained by photographing an expert, information obtained from an expert by hearing or the like, and the like. Results obtained by taking the same actions as those of a skilled person include skill levels (eg, skill level, machining accuracy, etc.). The skill level is a value indicating the skill level of the worker. Skill level is the degree of ability to perform a task. Proficiency may be subjective and qualitative. The skill level is, for example, a value obtained by hearing from a worker through hearings (including a questionnaire) and evaluating the skill as a numerical value. Further, the skill level may be corrected based on the result of comparing the value obtained from the worker with the quantitative data or the like. In the questionnaire, some people have the same skill level but highly value their own skill level (for example, those who answer "I have a very high skill level"), while others themselves. Some people humbly evaluate their proficiency (for example, those who say "my skill level is not enough"). Therefore, it is also possible to set the proficiency level as, for example, a value obtained by removing the deviation from the average of the scores of all the collected questionnaires. In this way, the skill level is a quantification of the skill of the worker obtained through hearings such as the worker or his / her manager. Further, at the time of learning, in addition to the above-mentioned qualitative data obtained through hearings and the like, it is also possible to learn quantitative data such as processing accuracy together as a teacher. The processing accuracy is an index for evaluating the accuracy of the dimensions and shape of the object processed by the worker, and is included in the skill level because it is a value corresponding to the skill level. Skill levels can also include the accuracy, speed, stability, etc. of the work performed by the worker. Accuracy is an index showing the degree of whether or not each work is performed in the correct procedure when performing multiple operations. Stability is an index showing the degree of whether or not each work is performed in a certain procedure when a plurality of works are performed. Speed is an index showing the length of time spent on each work when performing multiple works.

FIG. 4 is a flowchart showing the operation of the explanatory presentation device 1 according to the first embodiment during machine learning. In step S101, the feature extraction unit 102 and the skill determination unit 103 acquire learning data stored in the data storage unit 101. The learning data includes sensor data di ( _i = 1, 2, ...) And a teacher signal.

In step S102, for example, a combination of sensor data and teacher signal data stored in the data storage unit 101 is used as learning data by a learning device (for example, according to a model of a neural network (for example, shown in FIG. 5 described later)). , The output obtained by machine learning the features in the learning data 20) in FIG. 2A and inferring the result from the input (for example, skill level, processing accuracy, etc.). Machine learning. In the machine learning of the neural network, the trained model used by the skill determination unit 103 and the feature extraction unit 102 is used by using an error back propagation method or the like so that the error between the teacher signal and the output of the neural network becomes small. Automatically adjust parameters.

FIG. 5 is a diagram showing an example of a model using a fully connected neural network. In FIG. 5, X1 to X3 are input layers, Y1 and Y2 are intermediate layers, Z1 and Z2 are output layers, and w11 to w16 and w21 to w24 are weight coefficients. However, when the sensor data is time-series data, it is preferable to use LSTM (Long short-term memory). When the sensor data is image data, it is preferable to use CNN (Convolutional Neural Network). It is possible to use various known neural networks to generate the trained model.

In step S103 of FIG. 4, the data storage unit 101 stores the trained model M1, which is the first trained model generated by the feature extraction unit 102 and the skill determination unit 103.

The feature extraction unit 102 takes sensor data as an input, and extracts an implicit feature amount which is a feature amount for the skill determination unit 103 to determine a skill level such as skill level and processing accuracy. Tacit features are an example of tacit knowledge. The implicit feature quantity is extracted from the layer before the final output layer (for example, Z1 and Z2 in FIG. 5) of the neural network. The skill determination unit 103 obtains an implicit feature amount from the feature extraction unit 102, and determines a skill level such as skill level and processing accuracy based on the implicit feature amount.

Next, we will explain the explicit knowledge linking operation that is performed so that a person can easily understand the behavior of implicit features. FIG. 6 is a flowchart showing the generation operation of a trained model that associates explicit knowledge with implicit features. In step S201, the feature extraction unit 102 acquires the sensor data stored in the data storage unit 101.

In step S202, the feature extraction unit 102 inputs the sensor data to the trained model M1. In step S203, the feature extraction unit 102 extracts and outputs an implicit feature amount. Here, the feature extraction unit 102 extracts, for example, the output value of the layer in front of the final output layer (for example, Z1 and Z2 in FIG. 5) of the neural network as an implicit feature amount.

In step S204, the explicit knowledge linking unit 104 acquires the explicit knowledge stored in the data storage unit 101. Explicit knowledge includes, for example, video data obtained by photographing an expert, information obtained from an expert by hearing or the like, and the like.

In step S205, for example, according to the model of the neural network as shown in FIG. 5, a combination of data of implicit features and formal knowledge is given to the learning device (for example, the learning device 20 in FIG. 2A). The features in those training data are machine-learned, and the output is machine-learned by inferring the result from the input. In the machine learning of the neural network, the model parameters of the skill determination unit 103 and the feature extraction unit 102 are automatically adjusted by using an error back propagation method or the like so that the error between the teacher signal and the output of the neural network becomes small. In step S206, the data storage unit 101 stores the trained model M2, which is the second trained model generated by the explicit knowledge association unit 104.

The trained model that inputs implicit features and outputs explicit knowledge is not limited to neural networks, but estimates of missing values in the explicit knowledge part by SVR (Support Vector Regression), Naive Bayes, etc. The method of may be used.

Finally, a model in which the worker P (that is, the user) who wants to acquire the skill is associated by the explicit knowledge linking unit 104, and the skill determination unit 103 determines the skill level (for example, skill level and processing accuracy). The operation of requesting the presentation of an anti-real virtual explanation will be described in detail.

FIG. 7 is a flowchart showing an example of the operation of the explanatory presentation device 1 according to the first embodiment. In step S301, the worker P registers the sensor data related to the skill behavior to be improved in the data storage unit 101. That is, the worker P registers, for example, what the sensor data regarding the skill for which he / she desires to improve the skill level is, for example, in the operation device (operation device 14 in FIG. 1).

In step S302, the feature extraction unit 102 acquires sensor data related to the skill that the worker P registered in the data storage unit 101 wants to improve the skill level. In step S303, the feature extraction unit 102 inputs the acquired sensor data to the trained model M1 stored in the data storage unit 101.

In step S304, the feature extraction unit 102 acquires an implicit feature amount from the trained model M1. Here, the feature extraction unit 102 extracts, for example, the output value of the layer before the final output layer of the neural network as an implicit feature amount.

FIG. 8 is a diagram showing an example of data generated by the explanatory presentation device 1 according to the first embodiment in a table format (Table 1). Note that L is a positive integer, and i is an integer of 1 or more and L or less.

As described as Table 1 in FIG. 8, explicit knowledge S is labeled with techniques s ₁ , s ₂ , s ₃ , ..., S _m (m is a positive integer). As shown in FIG. 8, the data storage unit 101 is associated with the worker's behavior Ai ( _i = 1, ..., L), and has techniques s ₁ , technique s ₂ , technique s ₃ , ..., The skill sm, the sensor data _d , the implicit feature amount x, and the skill level y are stored. Then, the action _Aa that the worker P wants to acquire the skill is input to the data storage unit 101. Here, it is assumed that it is unknown what kind of explicit knowledge the worker P is performing.

FIG. 9 is a diagram showing an example of data generated by the explanatory presentation device 1 according to the first embodiment in a table format (Table 2). By repeating steps S305 to S310, the explanatory extraction unit 105 changes the value of each element of the feature vector obtained based on the implicit feature amount x _a acquired by the feature extraction unit 102, and changes the value of each element of the feature vector to the skill determination unit 103. Through this, we search how the skill level changes and obtain the data used for the anti-real virtual explanation as shown in Table 2 of FIG.

In step S305, the explanatory extraction unit 105 changes the value of each element of the obtained feature vector based on the implicit feature quantity x _a acquired by the feature extraction unit 102 to obtain the implicit feature quantity x _i . In step S306, the skill determination unit 103 inputs the implicit feature amount _xi acquired from the explanation extraction unit 105 into the trained model M1. In step S307, the skill determination unit 103 outputs the skill level y _i as the skill level corresponding to the input implicit feature amount x _i .

In step S308, the explicit knowledge association unit 104 inputs an implicit feature amount x _i into the trained model M2. In step S309, the explicit knowledge linking unit 104 outputs explicit knowledge corresponding to the input implicit feature amount x _i . In step S310, the data storage unit 101 stores the implicit feature amount, skill level, and explicit knowledge obtained in each step in the database.

In step S305, as a method of changing the value of the explanatory extraction unit 105, when the value of each element is changed by Δx from x _a , the element having the highest skill level among x _a + Δ x is stored. An algorithm may be used that preferentially searches as x _a gives a higher degree of skill (see, for example, FIG. 10).

FIG. 10 is a diagram showing an example of a search in an implicit feature space. Starting from x _a , Δx is added to each element, and the skill level determined by the skill determination unit 103 is stored in advance in the search queue. The search queue is sorted so that the higher the skill level, the earlier it is. Then select the first element of the search queue. FIG. 10 shows the case where the skill level was the highest when Δx was added to the first element. In this case, the implicit feature amount of _{xi =} xa + Δx1 is selected as the feature amount to be searched next, Δx is further added to _each element of the implicit feature amount _xi , and the skill determination unit 103 corresponds to each feature amount. Gain skill level, which is the skill level to do. By repeating the search in this way, an anti-real virtual implicit feature can be obtained.

In FIG. 10, for example, an evaluation function is used in which the evaluation becomes higher as the value Y of the specified explicit knowledge approaches 1. Alternatively, in FIG. 10, an evaluation function is used in which the closer the implicit features are, the higher the evaluation is. Then, the implicit feature quantity with high evaluation is preferentially searched.

In step S311, the explanatory extraction unit 105 is based on the explicit knowledge input by the worker P to the explicit knowledge selection unit 106, for example, an implicit feature amount having a high correlation with the selected explicit knowledge in Table 2 of FIG. The axis is selected, and the plot information on the feature space due to the change of explicit knowledge and the presentation information for visualizing and displaying the skill level information are extracted. The display 15 shown in FIG. 1 displays an image based on the presented information. A visualized example is shown in FIG. 11 below.

Further, x _a is input to the explicit knowledge linking unit 104, and the obtained explicit knowledge S techniques s _i , ..., _Sm are used as the initial label of the explicit knowledge selection unit 106, and the worker P formats. It may be configured so that explicit knowledge can be changed by the knowledge selection unit 106. In addition to this, the explanation extraction unit 105 may extract presentation information for presenting the probability that the skill level becomes equal to or higher than the target value depending on the presence or absence of the explicit knowledge S techniques s _i , ..., _Sm . .. Explanation The extraction unit 105 includes the techniques s _i , ..., S _m when the skill level exceeds the target under the assumption that the techniques s _i , ..., S _m are independent of each other. The probability is calculated, and the probability that the explicit knowledge S, s _i , ..., _Sm is included as a factor that makes the skill level higher than the target by the Bay's rule is calculated, and this is calculated by the explicit knowledge selection unit 106. The presentation information to be presented to the worker P may be extracted as the setting reference value of.

FIG. 11 is a diagram showing an example of a visualized explanation. In FIG. 11, 901a and 901b are explicit knowledge S selected by the explicit knowledge selection unit 106 based on the operation of the worker P, from the search of the explicit knowledge s _i , ..., _sm , or the explanatory extraction unit 105. The explicit knowledge S that contributes most to the improvement of skill level is s _i , ..., _sm . 902a and _902b indicate changes in the techniques s _i , ..., Sm, which are explicit knowledge S. Further, 903 is a plot of the position of the implicit feature amount corresponding to the sensor data input by the worker P. 904 is a visualization of the target skill level by a heat map. The heat map of FIG. 11 is a visualization graph expressing the values of two-dimensional data as shading. In FIG. 11, _905a and 905b are the axis of the implicit feature amount having a high correlation with the explicit knowledge S techniques s _i , ..., Sm.

However, visualization displays only the explicit knowledge space without displaying the feature space as a way to help the worker P determine which technique s _i , ..., S _m to master. It may be done as follows. Further, the visualization may be performed so that the thermal map plotting the implicit feature amount corresponding to the sensor data other than the sensor data related to the skill desired to be acquired by the worker P is displayed side by side. Alternatively, the visualization may be performed so as to display side by side a thermal map plotting other implicit features obtained in the search of the explanatory extraction unit 105.

In addition to this, the explanation extraction unit 105 may extract presentation information for presenting the probability that the skill level becomes equal to or higher than the target value depending on the presence or absence of the techniques s _i , ..., _Sm . For example, the explanation extraction unit 105 includes the techniques s _i , ..., S _m when the skill level exceeds the target under the assumption that the techniques s _i , ..., S _m are independent of each other. The probability that the skill s _i , ..., _Sm is included as a factor that the skill level exceeds the target is calculated by the Bayes rule, and this is used as the setting reference value of the explicit knowledge selection unit 106 for the worker P. The presentation information for presentation may be extracted. The display 15 shown in FIG. 1 displays an image based on the presented information.

By using the explanation presentation device 1 configured as described above, the worker P intuitively understands how to acquire the skill based on the skills s _i , ..., _Sm . Will be possible.

<< Embodiment 2 >>
In the first embodiment, explicit knowledge is associated with the implicit feature in order to easily understand the anti-real virtual explanation using the implicit feature, and the implicit feature and the explicit knowledge are combined. Is extracted to present to the worker P who wishes to improve his / her skill. However, in the first embodiment, the sensor data relating to the worker P who wishes to improve the skill is compared with the sensor data relating to another worker, and the process of referring to the comparison result is not performed. Therefore, for example, the following situations (1) to (3) may occur.
(1) Even if acquiring explicit knowledge is the shortest path to achieve the output of the target class, there are no examples and it is an unrealistic proposal.
(2) Situations where it is difficult to acquire skills because there is a long way to go.
(3) Since the search space for the anti-real virtual explanation is wide, the processing time becomes long.
Therefore, in the second embodiment, we propose an explanatory presentation device 2 that extracts presentation information for presenting a technique to be learned in consideration of sensor data of other workers.

FIG. 12 is a functional block diagram showing the configuration of the explanatory presentation device 2 according to the second embodiment. As shown in FIG. 12, the explanation presenting device 2 includes a data storage unit 201, a feature extraction unit 202 as a feature acquisition unit, a skill determination unit 203, an explicit knowledge linking unit 204, and an anti-real virtual explanation. It has an explanatory extraction unit 205, which is an extraction unit, an explicit knowledge selection unit 206, and a feature comparison unit 207. The explanatory presentation device 2 according to the second embodiment has an implicit feature amount obtained by inputting the sensor data of the worker P into the feature extraction unit 202 and an implicit feature obtained from the sensor data of another worker. It is different from the explanatory presentation device 1 according to the first embodiment in that the feature comparison unit 207 for comparing the amount is provided. The data storage unit 201, the feature extraction unit 202, the skill determination unit 203, the explicit knowledge linking unit 204, the explicit knowledge extraction unit 205, and the explicit knowledge selection unit 206 are the data storage unit 101, the feature extraction unit 102, in the first embodiment. This is the same as the skill determination unit 103, the explicit knowledge linking unit 104, the explicit knowledge extraction unit 105, and the explicit knowledge selection unit 106, respectively.

FIG. 13 is a flowchart showing an example of the operation of the explanatory presentation device 2 according to the second embodiment. The operation of the explanatory presentation device 2 is different from the operation of the explanatory presentation device 1 according to the first embodiment in that it has step S412 to be compared with the characteristics of other workers. Steps S401 to S404 and S411 are the same as steps S300 to S304 and S311 in FIG. 7.

In step S412, the feature comparison unit 207 is a distance between the implicit feature amount x _a obtained from the sensor data of the worker P and the implicit feature amount of another worker stored in the data storage unit 201. Is calculated and stored in advance in the data storage unit 201.

For example, the feature comparison unit 207 calculates the correlation-based similarity indicating the closeness of the implicit feature quantities by, for example, a variance-covariance matrix, and the data has _a higher skill level than the skill level ya of the action A _a . From among them, K pieces of data (K is a positive integer) are obtained in order of increasing distance. The feature extraction unit 202 extracts a feature amount based on the distance.

Here, the distance is based on correlation. However, the distance used may be any distance obtained by various known distance calculation methods such as cosine distance, Euclidean distance after dimensional compression by PCA (principal component analysis), and the like. ..

In steps S405 to S410, the explanatory extraction unit 205 performs a search by changing the implicit feature amount. Here, the explanatory extraction unit 205 preferentially changes the element of the implicit feature amount so as to increase the expected value of the skill level based on the K data extracted by the feature comparison unit 207, and K. The search is performed so as not to greatly deviate from the range of implicit features of individual data.

By extracting the anti-real virtual explanation based on the data whose implicit features are close to those of the worker P in this way, it is possible to promote the acquisition of explicit knowledge in consideration of actual examples, and efficient learning is possible. Yes, it can be expected to shorten the learning time.

In addition, by realizing comparisons between implicit features, the explicit knowledge linking unit 204 is not only linked by supervised learning such as a neural network, but also a linking method that does not require model generation such as collaborative filtering. Can be used.

An example of collaborative filtering is collaborative filtering based on correlation-based distance. FIG. 14 is a diagram showing an example of data generated by the explanatory presentation device 2 according to the second embodiment in a table format (Table 3). FIG. 15 is a diagram showing an example of the obtained correlation coefficient in a table format (Table 4). FIG. 16 is a diagram showing in a table format (Table 5) the behaviors recommended for obtaining the skill obtained by the sum of the explicit knowledge associated with the behaviors weighted by the correlation coefficient.

As shown in Table 3 of FIG. 14, the behaviors 1 _a , ..., La for which the skill level y ₁ , ..., Y _L > _ya are extracted, and the correlation coefficient cov ( _a ,) between these feature quantities is extracted. j) is calculated by the equation (1).

As a result, the correlation coefficient as shown in Table 4 of FIG. 15 is obtained. When an action _{A1a and an action having a correlation coefficient of 0.7 or more are selected, the action A3a and the action ALa are selected} as data for collaborative filtering. Based on this, as shown in Table 5 of FIG. 16, the explanatory extraction unit 205 weights the explicit knowledge associated with each action with the above correlation coefficient, and acts on the sum of the weighted explicit knowledge. Extract the presentation information for Aa to present as _a recommended action to acquire skill. The display 15 shown in FIG. 1 displays an image based on the presented information.

Explanation The extraction unit 205 takes into consideration the explicit knowledge of other workers by changing how many threshold values of the correlation coefficient are set and how many target achievement proficiency levels are set in step S405. , It is possible to associate implicit features with explicit knowledge.

The explicit knowledge linking unit 204 is not limited to the method such as the above-mentioned cooperative filtering, and when the skill level y> ya is obtained by using _a Bayesian network, the factor is explicit knowledge. By estimating the value based on the probability of occurrence of an event, such as by calculating the probability, the implicit feature amount and explicit knowledge may be linked.

The above explanation shows an example based on a comparison of only a worker and another worker. However, by changing the method of searching for the virtual and realistic implicit feature amount of the explanation extraction unit 205, the explanation extraction unit 205 gradually learns the method of a person whose behavior is close to each other, assuming that the worker P has acquired the skill. , It is possible to extract the presentation information for presenting the explanation so that the learning method approaches the target class.

For example, the evaluation function F has a format in which the evaluation is lower as the difference in the distance between the implicit feature amount of another worker who has stored the search queue in FIG. 10 and the feature amount during the search is larger, and the evaluation reaches the target skill level. It is designed so that the smaller the fluctuation ΣΔS of knowledge, the higher the evaluation, and the higher the skill level, the higher the evaluation. Specifically, the evaluation function F may be designed as shown in the following equation (2).

In step S412, the feature comparison unit 207 has an implicit feature amount x a obtained from the sensor data of the worker P and an implicit feature amount x _a obtained from the sensor data of another worker stored in the data storage unit 201. The sensor data is compared with the feature amount, and the sensor data is extracted so that the value is larger than the skill level _ya and K or more sensor data js that can achieve the target skill level are included. Then, the feature comparison unit 207 obtains a score by the evaluation function F for all the obtained sensor data, and adds j to the comparison target set J for distance calculation.

In steps S405 to S410, the explanatory extraction unit 205 extracts the implicit feature quantity x _i from the search queue in descending order of the value of the evaluation function F, starting from the action (a in Equation 2), and changes it to this value by Δx. Is added, the score is calculated by the evaluation function F based on the sum of the J data extracted by the feature comparison unit 207 and the amount of change in the behavior of formal knowledge (from a to i-1 in Equation 2), and the search queue is calculated. Add data to. By repeating the process using the evaluation function F as described above, it is possible to perform a search in which the feature amount does not deviate significantly from the acquired data while achieving the target skill level.

If the explanation presenting device 2 according to the second embodiment is used, a feature comparison unit 207 is provided to compare with the behavior of another worker, and the result is used to provide an anti-real virtual explanation that is far from reality. Can be prevented from being presented.

<< Embodiment 3 >>
The explanation presenting device 1 according to the first embodiment associates explicit knowledge with the implicit feature amount automatically obtained, and works on what kind of technique should be learned to improve the skill level. Present to the person. However, if it is possible to specifically present to the worker what kind of behavior change should be made, it is considered that the acquisition of the worker's skill will be further accelerated. Therefore, the explanation presentation device 3 according to the third embodiment includes a skill data generation unit 302 as a feature acquisition unit. In the third embodiment, the skill data generation unit 302 does not automatically extract the feature amount by the feature extraction unit so that the skill determination unit can determine the skill level such as skill level and processing accuracy, but the original sensor data. Is extracted so that it can be reproduced as generated data.

FIG. 17 is a functional block diagram showing the configuration of the explanatory presentation device 3 according to the third embodiment. As shown in FIG. 17, the explanatory presentation device 3 includes a data storage unit 301, a skill data generation unit 302 as a feature acquisition unit, a skill determination unit 303, an explicit knowledge linking unit 304, and an anti-real virtual. It has an explanatory extraction unit 305, which is an explanatory extraction unit, and an explicit knowledge selection unit 306. The explanation presenting device 3 according to the third embodiment is different from the explanation presenting device 1 according to the first embodiment in that the skill data generation unit 302 is provided as a feature acquisition unit. That is, the skill data generation unit 302 extracts the feature amount so that the original sensor data can be reproduced so that the skill determination unit can determine the skill level and the processing accuracy. The data storage unit 301, the explicit knowledge determination unit 303, the explicit knowledge association unit 304, the explicit knowledge extraction unit 305, and the explicit knowledge selection unit 306 are the data storage unit 101, the explicit knowledge determination unit 103, and the explicit knowledge association unit in the first embodiment. It is the same as 104, the explicit knowledge extraction unit 105, and the explicit knowledge selection unit 106, respectively.

The skill data generation unit 302 compresses the feature amount by the neural network. After that, the skill determination unit 303 determines the skill level as the skill level. Further, a decoder is provided in the latter half of the skill data generation unit 302, and the skill data generation unit 302 performs multitask learning such as restoring the original sensor data.

FIG. 18 is a diagram showing a configuration example of a network that executes multitask learning. In the network of FIG. 18, sensor data _{di is given to the neural network 142a from the input layer 141a, and the skill level yi is} output via the intermediate layer 145 and the neural network 142b. At the same time, the sensor data _di is restored from the intermediate layer 145 of the neural network via the decoder 143, and is output from the output layer 141b. The skill data generation unit 302 extracts the output value of this branch point as an implicit feature amount x _i .

The learning method of FIG. 18 is the same as the model of the basic configuration diagram, and uses the skill level y _i and the sensor data di _i as teacher signals, and the skill level y _i and the sensor data di _i output from the neural network 142b. The parameters are adjusted so that the loss function L composed of is small. After adjusting the parameters, the data storage unit 301 stores the learned model M1 generated by the skill data generation unit 302 and the skill determination unit 303.

The loss function L can be defined, for example, by the _weighted sum of the loss L _decode of the decoder 143 and the loss Ly of the estimated portion of the skill level.

The decoder 143 in FIG. 18 can be a known model such as a VAE (variational autoencoder) or a GAN (Generative Advanced Network) as long as it is a model for generating data.

Next, the operation of providing an anti-real virtual explanation will be described. FIG. 19 is a flowchart showing an operation for providing an anti-real virtual explanation.

In step S507, the skill data generation unit 302 and the skill determination unit 303 input the data acquired from the data storage unit 301 into the trained model M1 as an implicit feature amount x _i . Since the model learned by the skill data generation unit 302 is configured as shown in FIG. 18, two data, skill level y _i and sensor data _di , are output from the implicit feature amount x _i . .. In step S510, the data storage unit 301 stores the implicit feature amount x _i , the sensor data di _{i, the skill level y i ,} and the explicit knowledge S, and obtains Table 6.

FIG. 20 is a diagram showing an example of data generated by the explanatory presentation device 3 according to the third embodiment in a table format (Table 6). FIG. 21 is a diagram in which the sensor data shown in FIG. 20 is visualized, and particularly different parts are displayed as change # 1 and change # 2. Explanation in step _S511 The extraction unit 305 visualizes the sensor data _di of the anti _- real virtual actions C1 to CN in Table 6 of FIG. 20, and the part particularly different from the sensor data _da is the change # 1 in FIG. Extract the presentation information for highlighting like the part of change # 2. The display 15 shown in FIG. 1 displays an image based on the presented information.

As described above, in the third embodiment, the skill data generation unit 302 performs both the extraction of the implicit feature amount and the restoration of the sensor data, and associates the feature expression for determining the skill level with the sensor data. This generates presentation information that shows how the sensor data actually changes when explicit knowledge is changed. In the third embodiment, this correspondence (that is, the correspondence between the change of explicit knowledge and the change of sensor data) can be used to show hints of behavior change to be taken in order to master the technique.

In the third embodiment, a generative model for generating sensor data is used, but as a method of paying attention to which part of the sensor data the behavior should be transformed, an attention mechanism (Attention Mechanism) may be used. good.

FIG. 22 is a diagram showing an example of Attention Branch Network (ABN). In the third embodiment, the fire and explanation presenting device 3 is provided with an attention mechanism for extracting which of the intermediate feature quantities should be focused on, as shown in ABN shown in FIG. 22, for example. The presentation information for highlighting the sensor data corresponding to the place where the attention is high may be generated. For this highlighting, for example, the method described in Non-Patent Document 1 can be used.

<< Embodiment 4 >>
In the third embodiment, the skill data generation unit 302 is used to confirm the area to be behavior-changed (for example, change # 1 and change # 2). In the fourth embodiment, by giving a perturbation to the sensor data in the area related to the skill level, how the skill level such as the skill level and the processing accuracy is likely to change, and the credibility of the skill level are confirmed. It will be possible to do. In order to realize this, the explanation presenting device 4 according to the fourth embodiment includes a perturbation confirmation unit.

FIG. 23 is a functional block diagram showing the configuration of the explanatory presentation device 4 according to the fourth embodiment. As shown in FIG. 23, the explanatory presentation device 4 includes a data storage unit 401, a skill data generation unit 402 as a feature acquisition unit, a skill determination unit 403, an explicit knowledge linking unit 404, and an anti-real virtual. It has an explanatory extraction unit 405, which is an explanatory extraction unit, an explicit knowledge selection unit 406, and a perturbation confirmation unit 408. The explanatory presentation device 4 according to the fourth embodiment is different from the explanatory presentation device 3 according to the third embodiment in that the perturbation confirmation unit 408 is provided. The data storage unit 401, skill data generation unit 402, skill determination unit 403, explicit knowledge linking unit 404, explicit knowledge extraction unit 405, and explicit knowledge selection unit 406 are the data storage unit 301 and skill data generation unit in the third embodiment. It is the same as 302, skill determination unit 303, explicit knowledge association unit 304, explicit knowledge extraction unit 305, and explicit knowledge selection unit 306, respectively.

The flow of perturbation confirmation is shown below. First, the worker P inputs sensor data corresponding to the behavior for which he / she wants to acquire the skill, and explicit knowledge is selected by the explicit knowledge selection unit 406. In response to the result, the explanation extraction unit 405 changes the evaluation function F of the search so that only the selected explicit knowledge changes as much as possible, and changes the implicit feature quantity to implicitly as an anti-real virtual explanation. The target feature amount x _i is generated. The explicit knowledge linking unit 404 acquires explicit knowledge corresponding to the generated implicit feature amount x _i , the skill determination unit 403 determines the skill level y _i , and the skill data generation unit 402 determines the sensor data d. _i is reproduced as generated data.

The explanation extraction unit 405 presents a candidate for explicit and virtual explicit knowledge to the worker P in order to satisfy the target skill level y by the above search. Next, the worker P perturbates (that is, changes the input sensor data) by the perturbation confirmation unit 408 for the important part of the presented sensor data that particularly affects the skill level. As a result, the perturbation confirmation unit 408 registers the sensor data changed by the perturbation in the data storage unit 401, and the skill data generation unit 402 reads the sensor data to which the perturbation is applied and generates the original data. Register the sensor data in the data storage unit 401.

The perturbation confirmation unit 408 compares the sensor data input to the skill data generation unit 402 with the sensor data output by the skill data generation unit 402, and tells the worker P the difference between the generated data and the sensor data to which the perturbation is added. Perform the process for presenting. In addition, the explanatory extraction unit 405 extracts the presentation information for presenting how the skill skill has changed as a result of the perturbation being added to the sensor data. The display 15 shown in FIG. 1 displays an image based on the presented information.

As described above, the explanation presenting device 4 according to the fourth embodiment is appropriate when the data not included in the trained model is used by visualizing the behavior of the sensor data generated at the time of perturbation. Take advantage of the fact that explanations are difficult to generate and the accuracy of explanations drops significantly. The explanatory presentation device 4 according to the fourth embodiment can determine the allowable range of data that can be handled by the trained model generated by such a method, and how the perturbation affects the skill level. You can know.

Further, as a modification of the fourth embodiment, the worker P inputs the range of the sensor data visualization of the explanatory extraction unit 405 to the perturbation confirmation unit 408 (for example, the change # in FIG. 21). (1) Select either change # 2), the explanation extraction unit 405 searches for an implicit feature amount so that only the sensor data of the relevant part changes and the skill level is higher than the current level, and the sensor is used. The fluctuation range of the change in the data may be presented.

As described above, by using the explanation presenting device 4 according to the fourth embodiment, by presenting the perturbation range of the sensor data, the worker P can change the behavior that is allowed to acquire the skill. , It is possible to confirm before registering the action again.

<< Embodiment 5 >>
In the first embodiment, as shown in FIG. 3, the trained model formed by the learning device which is a set of the skill determination unit 103, the feature extraction unit 102, and the explicit knowledge association unit 104 is one learning. An example of the finished model M1 is described. On the other hand, the explanatory presentation device 5 according to the fifth embodiment generates a plurality of learned models M1 and learns about a skill (that is, a related skill) close to the skill that the worker wants to acquire from among them. Find a finished model.

FIG. 24 is a functional block diagram showing the configuration of the explanatory presentation device 5 according to the fifth embodiment. As shown in FIG. 24, the explanatory presentation device 5 includes a data storage unit 501, a feature extraction unit 502 as a feature acquisition unit, a skill determination unit 503, an explicit knowledge linking unit 504, and an anti-real virtual explanation. It has an explanatory extraction unit 505, which is an extraction unit, an explicit knowledge selection unit 506, and a model priority determination unit 509. The explanation presenting device 5 according to the fifth embodiment is close to the point that a plurality of trained models M1 are generated and the skill that the model priority determination unit 509 wants to acquire from among the plurality of trained models M1. It differs from the explanation presenting device 1 according to the first embodiment in that it finds a learned model related to the skill. In other words, the explanation presenting device 5 has a plurality of sets of the skill determination unit 503, the feature extraction unit 502, and the explicit knowledge association unit 504, and the model priority determination unit 509 determines the priority of the plurality of sets. .. That is, the explanation presenting device 5 has a plurality of learning sets each consisting of a skill determination unit 503, a feature extraction unit 502, and an explicit knowledge association unit 504, and a model priority determination unit that determines the priority of the plurality of learning sets. With 509, the plurality of learning sets acquire sensor data from the database in a time-divided manner. The data storage unit 501, the feature extraction unit 502, the skill determination unit 503, the explicit knowledge linking unit 504, the explicit knowledge extraction unit 505, and the explicit knowledge selection unit 506 are the data storage unit 101, the explicit knowledge extraction unit 102, in the first embodiment. This is the same as the skill determination unit 103, the explicit knowledge linking unit 104, the explicit knowledge extraction unit 105, and the explicit knowledge selection unit 106, respectively.

The model priority determination unit 509 reads out the sensor data of the worker P registered in the data storage unit 501, and the sensor data having a time width suitable for each of the plurality of trained models (that is, the time-divided sensor data). ).

FIG. 25 is a diagram showing an example of time-division sensor data in the explanatory presentation device 5 according to the fifth embodiment. Since the trained model # 1 uses the time data of the time R1 at the time of machine learning, the sensor data is divided by the time R1 and the time is advanced by t1 to perform time division. Since the trained model # 2 uses the time data of the time R2 at the time of machine learning, the sensor data is divided by the time R2, and the time is advanced by t2 to perform time division.

Next, the model priority determination unit 509 inputs the acquired sensor data of each time width into all the trained models, and the distribution of the data in the final layer of the neural network of the skill determination unit 503 and the feature extraction unit 502 is normal. Check if it is out of the distribution. For this check, for example, the method described in Non-Patent Document 2 can be used.

In the final layer, the model priority determination unit 509 preferentially selects a combination of the trained model and the sensor data whose data distribution does not deviate from the normal distribution, and searches for the feature space of the explanation extraction unit 505. .. When the worker P selects the explicit knowledge to be focused on in the explicit knowledge selection unit 506 in advance, the model priority determination unit 509 narrows the selection range to the trained model including the explicit knowledge as learning data.

The model priority determination unit 509 inputs the time-divisioned sensor data d _t1 , d _t2 , ..., D _tr to each trained model, and selects the trained model based on the easiness of fluctuation of the skill level. May be good. Further, the model priority determination unit 509 considers that the trained model in which the skill level does not change at all is irrelevant, and may preferentially use the trained model in which the change in skill level can be confirmed.

Further, as described in the third and fourth embodiments, the skill data generation units 302 and 402 are dimensionally compressed so that the distribution of the data output from the intermediate layer becomes a normal distribution by using a method such as VAE or GAN. , And the model priority determination unit 509 in the fifth embodiment is configured to restore the original sensor data, and the sensor data and the trained model are correlated depending on whether the distribution of the data deviates from the normal distribution. You may decide the set of.

FIG. 26 is a diagram showing an example of visualized presentation information. First, the explanatory extraction unit 505 includes explicit knowledge 1 to m as a factor that makes the skill level higher than the target according to the Bayes rule, as shown in Table 7, based on the information in the database accumulated so far. The probability may be calculated and displayed on a display (shown in FIG. 1) in order to present it to the worker P who wants to acquire the skill as a setting reference value for operating the explicit knowledge selection unit 506. .. Table 8 of FIG. 26 shows an example of visualization when the worker P who wants to acquire the skill selects “Skill 1” as explicit knowledge in the explicit knowledge selection unit 506. In Table 8, examples of anti-real virtual actions C ₁ to C ₁₀ in which the value of "Skill 1" is large and the skill level is higher than the current one are listed. For example, in the example of Table 8 in FIG. 26, the skill level is larger than that determined by the skill determination unit 503 based on the sensor data of the current worker, and "Skill 1" is given in descending order of 10 cases. Examples of anti-real virtual behaviors are rearranged and visualized. Further, in the third column from the left of Table 8, one related skill (S _a , S _b , etc.), which is a skill highly related to “Skill 1”, is presented. Further, in the sensor data column of Table 8, an example is shown in which a portion where a large change has occurred is highlighted with a thick line frame or a color frame based on the data generated by the skill data generation unit 503. .. If the worker P does not select explicit knowledge, 10 examples of anti-real virtual behaviors are sorted in order of skill level, correlation of implicit features, or a combination of these scores. May be done. The method of visualizing explicit knowledge shown in FIG. 26 is an example. The explicit knowledge is visualized based on the degree of superimposition of sensor data, the degree of correlation of feature amounts, and the like, and the connection between explicit knowledge is connected to the operator. It may be presented so that P can be understood. The example of the presented information shown in FIG. 26 may be applied to other embodiments.

As described above, by using the explanation presentation device 5 according to the fifth embodiment, the worker P selects a trained model suitable for the data to be input to the trained model from the plurality of trained models. The operation to be performed becomes unnecessary or easy, and the explanation presenting device 5 can automatically select the part where the skill is likely to be extracted, and select the corresponding trained model together with this.

Further, the explanatory presentation device 5 according to the fifth embodiment is associated with the sensor data in which the corresponding explicit knowledge is taken in and input when the worker P selects explicit knowledge in the explicit knowledge selection unit 506. It is possible to select a trained model that seems to have. Then, when dealing with a plurality of skills, an appropriate trained model is selected, and explicit knowledge suitable for acquiring the skill can be presented in addition to the implicit features.

1-5 Explanation presentation device, 11 processor, 12 memory, 13 storage device, 14 operation device, 15 display, 16 sensor, 20 learning device, 21 data acquisition unit, 22 model generation unit, 23 trained model storage unit, 30 Inference device, 31 data acquisition unit, 32 inference unit, 33 trained model storage unit, 101, 201, 301, 401, 501 data storage unit, 102, 202, 502 feature extraction unit (feature acquisition unit), 302, 402 skills Data generation unit (feature acquisition unit), 103, 203, 303, 403, 503 skill judgment unit, 104, 204, 304, 404, 504 format knowledge linking unit, 105, 205, 305, 405, 505 explanation extraction unit, 106, 206, 306, 406, 506 Format knowledge selection unit, 207 feature comparison unit, 408 perturbation confirmation unit, 509 model priority determination unit.

Claims (15)

1. A feature acquisition unit that acquires the feature amount of the behavior from a database that stores the sensor data obtained by detecting the behavior of the worker and the explicit knowledge that is the acquired knowledge that can be interpreted by humans.
   A skill determination unit that determines the skill level of the worker from the feature amount and registers the skill level in the database.
   In the database, the explicit knowledge linking unit that links explicit knowledge to the feature amount,
   An explanatory extraction unit that extracts presentation information including the feature amount and the explicit knowledge associated with the feature amount, and an explanatory extraction unit.
   Explanation presentation device having.
2. The feature acquisition unit and the skill determination unit use the first learning data including the sensor data and the correct answer signal of the sensor data, and the first learning for determining the skill level from the sensor data. The explanatory presentation device according to claim 1, which generates a completed model.
3. The explanatory presentation device according to claim 2, wherein the feature acquisition unit and the skill determination unit determine the skill level from the sensor data using the first learned model.
4. The feature acquisition unit, the skill determination unit, and the explicit knowledge linking unit output explicit knowledge associated with the feature amount using the second learning data including the feature amount and the skill level. The explanatory presentation device according to any one of claims 1 to 3, which generates a second trained model for the purpose of generating the second trained model.
5. The explanatory presentation device according to claim 4, wherein the feature acquisition unit, the skill determination unit, and the explicit knowledge linking unit output the explicit knowledge using the second learned model.
6. The explanatory presentation device according to any one of claims 1 to 5, wherein the skill level includes at least one of skill level and processing accuracy.
7. The explanatory presentation device according to any one of claims 1 to 6, wherein the sensor data includes video data.
8. The explanatory presentation device according to any one of claims 1 to 7, further comprising a display for displaying an image based on the presentation information.
9. It also has a feature comparison unit that calculates the distance between the implicit feature amount obtained from the sensor data of the worker and the other implicit feature amount obtained from the sensor data of another worker and stores it in the database in advance. death,
   The explanatory presentation device according to any one of claims 1 to 8, wherein the feature acquisition unit acquires the feature amount based on the distance.
10. The explanatory presentation device according to claim 9, wherein the feature acquisition unit acquires a limited number of features extracted based on the distance.
11. The feature acquisition unit is a skill data generation unit that acquires the feature amount and reproduces the original sensor data from the feature amount as generated data.
    The explanatory presentation device according to any one of claims 1 to 10, wherein the explanatory extraction unit displays the presented information so as to highlight a range in which a large difference between the sensor data and the generated data is large.
12. It further has a perturbation confirmation unit that changes the sensor data input to the database and stores the perturbated sensor data in the database.
    The feature acquisition unit is a skill data generation unit that acquires the feature amount and reproduces the original sensor data from the feature amount as generated data.
    The explanatory presentation device according to any one of claims 1 to 10, wherein the explanatory extraction unit presents a difference between the generated data and the sensor data to which the perturbation is applied.
13. A plurality of learning sets each consisting of the skill determination unit, the feature acquisition unit, and the explicit knowledge linking unit,
    It has a model priority determination unit that determines the priority of the plurality of learning sets.
    The explanatory presentation device according to any one of claims 1 to 12, wherein the plurality of learning sets acquire sensor data from the database in a time-division manner.
14. An explanation presentation method performed by the explanation presentation device.
    The step of acquiring the feature amount of the behavior from the database that stores the sensor data obtained by detecting the behavior of the worker and the explicit knowledge that is the acquired knowledge that can be interpreted by humans.
    A step of determining the skill level of the worker from the feature amount and registering the skill level in the database, and
    In the database, the step of associating explicit knowledge with the feature quantity,
    A step of extracting presentation information including the feature amount and the explicit knowledge associated with the feature amount, and
    Explanation presentation method having.
15. The step of acquiring the feature amount of the behavior from the database that stores the sensor data obtained by detecting the behavior of the worker and the explicit knowledge that is the acquired knowledge that can be interpreted by humans.
    A step of determining the skill level of the worker from the feature amount and registering the skill level in the database, and
    In the database, the step of associating explicit knowledge with the feature quantity,
    A step of extracting presentation information including the feature amount and the explicit knowledge associated with the feature amount, and
    An explanation presentation program that causes the computer to execute.

Images