JP2019174870A - Information processing apparatus, information processing method and program - Google Patents

Abstract

To improve a quality of supervised machine learning data.SOLUTION: In an information processing apparatus 1, a model acquisition unit 110 inputs failure information regarding a failure of a network, and acquires a learning model trained so that one or a plurality of candidate of a failure recovery procedure is output. A recovery procedure acquisition unit 112 acquires an output of the learning model that receives the failure information received from a user. A user evaluation acquisition unit 113 acquires the failure recover procedure selected by the user among the failure recovery procedures acquired by the recovery procedure acquisition unit 112 and an evaluation of a usefulness of the failure recovery procedure by the user. A user information acquisition unit 114 acquires skill information indicating the user's skills for failure recovery. A weight setting unit 115 sets a weight of an evaluation by the user regarding the usefulness of the failure recovery procedure selected by the user in the learning data used for relearning the learning model based on the skill information of the user.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  In recent years, supervised machine learning techniques such as neural networks have been rapidly developed, and it is becoming possible for computers to perform intelligent processing such as recognition and judgment. In these supervised machine learning, the performance of the generated learning model depends on the quality of learning data used for learning.

  For this reason, for example, Patent Document 1 proposes a technique for appropriately grasping the total error evaluation function and improving the calculation accuracy even for inconsistent teacher data.

JP 2011-107885 A

  In the above technique, the application of the technique may be difficult when the evaluation axis of the teacher data is qualitative and difficult to quantify, such as when based on human rule of thumb.

  The present invention has been made in view of the above problems, and an object thereof is to provide a technique for improving the quality of supervised machine learning learning data.

  A first aspect of the present invention is an information processing apparatus. This device inputs failure information related to a failure in the network, inputs a model acquisition unit that acquires a learning model learned so as to output one or a plurality of failure recovery procedure candidates, and inputs failure information received from a user A recovery procedure acquisition unit that acquires the output of the learning model, a failure recovery procedure selected by the user from the failure recovery procedures acquired by the recovery procedure acquisition unit, and the usefulness of the failure recovery procedure by the user A user evaluation acquisition unit that acquires the evaluation of the user; a user information acquisition unit that acquires skill information indicating the failure recovery skill of the user; and the re-learning of the learning model based on the skill information of the user A weight setting unit configured to set a weight of evaluation by the user regarding the usefulness of the failure recovery procedure selected by the user in the learning data.

  The weight setting unit may increase the evaluation weight by the user regarding the usefulness of the failure recovery procedure selected by the user, rather than a low evaluation, when the user has high skill in the failure recovery.

  The user evaluation acquisition unit may further acquire information indicating whether or not the failure has been successfully recovered by the failure recovery procedure selected by the user, and the weight setting unit has successfully recovered the failure. In this case, the weight of evaluation by the user regarding the usefulness of the failure recovery procedure selected by the user may be increased as the failure recovery skill of the user is lower.

  When the failure recovery fails, the weight setting unit may decrease the evaluation weight by the user regarding the usefulness of the failure recovery procedure selected by the user as the failure recovery skill of the user increases.

  The learning model may be learned so that the failure information related to the failure of the network is input, and further information related to the related equipment attached to the failure recovery procedure is further output, and the weight setting unit includes the skill information of the user The user may further set a weight of evaluation by the user regarding the usefulness of the related facility investigation procedure incidental to the failure recovery procedure selected by the user in the learning data used for the relearning of the learning model.

  The user evaluation acquisition unit may further acquire a user identifier for identifying the user, and the user information acquisition unit includes a user identifier and a skill related to failure recovery of the user corresponding to the user identifier. The skill information may be acquired by referring to a skill database stored in association with each other.

  The second aspect of the present invention is an information processing method. In this method, the processor obtains a learning model learned to input fault information related to a fault in the network and output one or more fault recovery procedure candidates, and fault information received from the user. Obtaining an output of the learning model as an input; obtaining a failure restoration procedure selected by the user from the obtained failure restoration procedures; and obtaining an evaluation of the usefulness of the failure restoration procedure by the user A step of acquiring skill information indicating skill of failure recovery of the user, and usefulness of the failure recovery procedure selected by the user in learning data used for re-learning of the learning model based on the skill information of the user Setting a weight of evaluation by the user regarding the degree.

  The third aspect of the present invention is a program. This program inputs fault information related to a fault in the network to a computer, acquires a learning model learned to output one or more fault recovery procedure candidates, and fault information received from a user. A function for acquiring the output of the learning model as input, a failure recovery procedure selected by the user from the acquired failure recovery procedures, and a function for acquiring an evaluation of the usefulness of the failure recovery procedures by the user And a function of acquiring skill information indicating the skill of failure recovery of the user, and a usefulness of the failure recovery procedure selected by the user in learning data used for re-learning of the learning model based on the skill information of the user And a function of setting a weight of evaluation by the user regarding the degree.

  According to the present invention, the quality of supervised machine learning learning data can be improved.

It is a figure for demonstrating the outline of the process of the information processing apparatus which concerns on embodiment. It is a figure which shows typically the function structure of the information processing apparatus which concerns on embodiment. It is a figure for demonstrating the weight calculation of the weight setting part which concerns on embodiment. It is a figure which shows typically the data structure of the skill database which concerns on embodiment. It is a flowchart for demonstrating the flow of the information processing which the information processing apparatus which concerns on embodiment performs. It is a flowchart for demonstrating the flow of the weight setting process which the weight setting part which concerns on embodiment performs.

<Outline of Embodiment>
An outline of the embodiment will be described. An information processing apparatus according to an embodiment uses a learning model that receives failure information related to a failure of the network as an input and outputs one or a plurality of failure recovery procedure candidates for dealing with the failure. Device. The learning model used by the information processing apparatus according to the embodiment is a learning model generated by supervised machine learning such as a neural network or SVM (Support Vector Machine).

  In supervised machine learning, once a model is generated by machine learning, re-learning may be performed by adding new teacher data to improve the accuracy of the model. Here, the teacher data for generating the learning model used by the information processing apparatus according to the embodiment includes failure information related to a network failure, a recovery procedure for dealing with the failure, and a user related to the usefulness of the recovery procedure. (For example, whether or not the failure has been recovered).

  The information processing apparatus according to the embodiment presents a failure recovery procedure candidate to the user by inputting unknown failure information related to the failure of the network into the learning model. Furthermore, the information processing apparatus according to the embodiment receives the recovery procedure selected by the user, the user's evaluation regarding the usefulness of the recovery procedure, and information for identifying the user from the user. Thus, whenever the user uses the information processing apparatus according to the embodiment to recover from a network failure, teacher data for generating a learning model can be collected.

  The “user's evaluation regarding the usefulness of the recovery procedure” by the user is reflected in the “weight” of the teacher data in the relearning of the learning model. Specifically, the restoration procedure with a high evaluation by the user is more strongly reflected in the learning model at the time of re-learning than the restoration procedure with a low evaluation. That is, “the user's evaluation regarding the usefulness of the recovery procedure” can be said to be information that greatly affects the performance of the learning model generated as a result of relearning.

  Here, the evaluator of “user evaluation regarding the usefulness of the recovery procedure” is, of course, a user. For this reason, since the user's evaluation regarding the usefulness of the restoration procedure can reflect the subjectivity of each individual user, guarantee of objectivity is not necessarily guaranteed. In view of the fact that “user evaluation regarding the usefulness of the recovery procedure” affects the performance of the learning model, it is preferable to correct the “user evaluation regarding the usefulness of the recovery procedure” by some method.

  Therefore, the information processing apparatus according to the embodiment sets the weight of evaluation by the user regarding the usefulness of the failure recovery procedure selected by the user based on the user's failure recovery skill. The information processing apparatus quantifies and stores the user's failure recovery skills in advance for each user. As a result, the information processing apparatus can correct the “user's evaluation regarding the usefulness of the recovery procedure”, which can be influenced by the user's subjectivity, based on the objectively quantified “user failure recovery skill”. it can. As a result, the quality of supervised machine learning learning data can be improved.

<Processing Process of Information Processing Device According to Embodiment>
FIG. 1 is a diagram for explaining an outline of a processing process of the information processing apparatus according to the embodiment. The information processing apparatus according to the embodiment can execute three processing steps: a learning process, a model application process, and a relearning data generation process.

  Hereinafter, with reference to FIG. 1, the process performed by the information processing apparatus according to the embodiment will be described with reference to (1) to (8). The description will be made with reference to (1) to (8) in FIG. 1. And corresponding.

[Learning process]
(1) The administrator of the information processing apparatus prepares learning data for executing machine learning in the information processing apparatus. Specifically, the user of the information processing apparatus prepares network failure information, failure recovery procedures and results, and data including weights as learning data.

(2) The information processing apparatus performs machine learning on learning data using a known machine learning technique such as a neural network.
(3) The information processing apparatus generates a learning model by machine learning of learning data. Specifically, the information processing apparatus uses a network failure information, a failure recovery procedure and its result, and data including weights as learning data, and uses machine learning to input failure information and output a failure recovery procedure. Is generated.

[Model application process]
(4) The information processing apparatus accepts input of unknown fault information in the network.
(5) The information processing apparatus outputs a candidate list of procedures for restoring a failure by applying the learning model with the received unknown failure information as an input.

[Re-learning data generation process]
(6) A user who handles a failure in the network selects one recovery procedure from the recovery procedure candidate list output by the information processing apparatus.
(7) The user performs the recovery work according to the selected recovery procedure.

  (8) The user inputs the selected recovery procedure and the user's evaluation for the recovery procedure. At this time, the information processing apparatus converts the user's evaluation with respect to the recovery procedure into the weight of the recovery procedure to obtain new learning data. As a result, new learning data is accumulated each time the user performs a recovery operation in accordance with the recovery procedure output by the information processing apparatus based on the learning model.

  The information processing apparatus relearns the learning model based on the learning data to which new learning data is added. By repeating the above, the information processing apparatus according to the embodiment can continuously improve the quality of the learning model.

<Functional Configuration of Information Processing Device According to Embodiment>
FIG. 2 is a diagram schematically illustrating a functional configuration of the information processing apparatus 1 according to the embodiment. The information processing apparatus 1 according to the embodiment includes a storage unit 10 and a control unit 11.

  The storage unit 10 includes a ROM (Read Only Memory) that stores a BIOS (Basic Input Output System) of a computer that implements the information processing apparatus 1, a RAM (Random Access Memory) that is a work area of the information processing apparatus 1, an OS ( Operating system), application programs, and mass storage devices such as HDDs (Hard Disk Drives) and SSDs (Solid State Drives) that store various types of information referred to when the application programs are executed.

  The control unit 11 is a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) of the information processing apparatus 1, and executes a program stored in the storage unit 10 to execute a model acquisition unit 110, unknown failure information. It functions as an input unit 111, a recovery procedure acquisition unit 112, a user evaluation acquisition unit 113, a user information acquisition unit 114, a weight setting unit 115, a learning data input unit 116, and a machine learning unit 117.

  The model acquisition unit 110 acquires the learning model learned so that the failure information regarding the failure of the network is input and one or a plurality of failure recovery procedure candidates are output. The learning model is stored in the storage unit 10, and the model acquisition unit 110 acquires the learning model with reference to the storage unit 10.

  The unknown failure information input unit 111 acquires unknown failure information related to network failures from the user. Specifically, the unknown fault information input unit 111 receives unknown fault information from the user via an input interface of the information processing apparatus 1 (not shown) such as a keyboard and pointing.

  The recovery procedure acquisition unit 112 inputs the failure information received from the user by the unknown failure information input unit 111 to the learning model, and acquires a failure recovery procedure candidate as an output. The user evaluation acquisition unit 113 includes a failure recovery procedure selected by the user from the failure recovery procedures acquired by the recovery procedure acquisition unit 112, an evaluation of the usefulness of the failure recovery procedure by the user, and information for identifying the user. To get. Specifically, the user evaluation acquisition unit 113 acquires a recovery procedure, its evaluation, and a user identifier assigned to each user from the user via the input interface of the information processing apparatus 1.

  The user information acquisition unit 114 acquires skill information indicating a user's failure recovery skill. Skill information indicating a user's failure recovery skill is stored in advance in the storage unit 10 as a skill database. The user evaluation acquisition unit 113 acquires skill information by referring to the skill database based on the user identifier.

  The weight setting unit 115 sets the evaluation weight by the user regarding the usefulness of the failure recovery procedure selected by the user in the learning data used for relearning the learning model based on the skill information of the user. Thereby, the quality of the learning data of supervised machine learning can be improved.

  The failure recovery procedure for which the weight setting unit 115 sets the weight is output to the learning data input unit 116 as new learning data together with the evaluation. The machine learning unit 117 acquires learning data from the learning data input unit 116 and re-learns the learning model. The machine learning unit 117 stores the learning model generated by relearning in the storage unit 10. Thereby, the machine learning unit 117 can update the learning model based on the learning data to which new learning data is added.

[Calculation of weight]
FIGS. 3A to 3B are diagrams for explaining the weight calculation of the weight setting unit 115 according to the embodiment. Specifically, FIG. 3A is a diagram showing a result data table that the weight setting unit 115 refers to in order to calculate a user's result regarding failure recovery. On the other hand, FIG. 3B is a diagram showing a conversion data table that is referred to in order to convert the results calculated by the weight setting unit 115 into weights. These data tables are stored in the storage unit 10.

  As shown in FIG. 3A, the grade data table stores the grade of the user corresponding to each user identifier. The result data table shown in FIG. 3A indicates that the user whose user identifier is UID00099 has 15 fault recovery experiences, and the most recent 10 success rate is 90%. When the failure recovery experience is 15 times, the result is “60”, and when the success rate of the latest 10 times is 90%, the result is “90”. The same applies hereinafter. The grade corresponding to each score of the grade data is normalized so that the value is between 0 and 100.

  The weight setting unit 115 refers to the result data table based on the user identifier acquired by the user evaluation acquisition unit 113, thereby calculating the average value of the user's results. The average value of the user's score is skill information indicating the failure recovery skill of the corresponding user. The weight setting unit 115 stores the average value of the calculated grades as skill information in the skill database.

  The weight setting unit 115 obtains the weight by referring to the conversion data table shown in FIG. As shown in FIG. 3B, the conversion data table is set so that the weight becomes larger as the average value of the results is larger. Thus, the weight setting unit 115 can increase the evaluation weight by the user regarding the usefulness of the failure recovery procedure selected by the user, rather than the low evaluation, when the user's skill of failure recovery is high.

[Weight correction]
FIG. 4 is a diagram schematically illustrating a data structure of the skill database according to the embodiment. The skill database is stored in the storage unit 10 and is managed by the weight setting unit 115.

  As shown in FIG. 4, the skill database stores user skill information (average results), a first weight correction coefficient, and a second weight correction coefficient in association with each user identifier for identifying each user. is doing. Here, the first weight correction coefficient is a correction coefficient used by the weight setting unit 115 when the user succeeds in recovery from the failure, and the second correction coefficient is set as a weight when the user fails in recovery from the failure. This is a correction coefficient used by the unit 115.

  In the example shown in FIG. 4, the user whose user identifier is UID00001 has a failure recovery skill of 91 in 100 stages. In addition, the user with the user identifier UID00002 has 24 skills in failure recovery out of 100 stages. Similarly, a user whose user identifier is UID09999 has a failure recovery skill of 100 out of 100.

  The user evaluation acquisition unit 113 acquires information indicating whether or not failure recovery has been successfully performed by the failure recovery procedure selected by the user as part of the evaluation by the user regarding the usefulness of the failure recovery procedure. When the failure recovery is successful, the weight setting unit 115 increases the evaluation weight by the user regarding the usefulness of the failure recovery procedure selected by the user as the user's failure recovery skill is lower. Specifically, the weight setting unit 115 corrects the weight by multiplying the acquired weight by a first weight correction coefficient. For this reason, the first weight correction coefficient is set so as to increase as the user's failure recovery skill decreases.

  Even if a user with low failure recovery skills succeeds in recovering from a failure, the failure recovery procedure is considered to be a useful procedure. The weight setting unit 115 further improves the quality of learning data for re-learning by correcting the weight with the first weight correction coefficient that is set so as to become larger as the user's skill of failure recovery is lower. Can be made.

  On the other hand, even if the user has a high skill in failure recovery, if failure recovery fails, the failure recovery procedure is likely to be a procedure that does not contribute to failure recovery. Therefore, when failure recovery fails, the weight setting unit 115 reduces the evaluation weight by the user regarding the usefulness of the failure recovery procedure selected by the user as the user's failure recovery skill increases. Specifically, the weight setting unit 115 corrects the weight by multiplying the acquired weight by a second weight correction coefficient. For this reason, the second weight correction coefficient is set so as to be smaller as the user's skill of failure recovery is higher. Thereby, the information processing apparatus 1 can further improve the accuracy of learning data for relearning.

  Here, the machine learning unit 117 may learn to receive failure information related to the failure of the network as an input and also output information related to the related equipment attached to the failure recovery procedure. Specifically, the machine learning unit 117 uses a learning model including learning information including network failure information, failure recovery procedures and results, weights, and related facility information that is equipment related to the network failure. Is generated.

  In this case, the weight setting unit 115 also sets the weight of evaluation by the user regarding the usefulness of the related facility investigation procedure in the learning data used for relearning of the learning model based on the skill information of the user. By causing the learning model to output information on the related equipment, the information processing apparatus 1 can provide more information to a user who attempts to recover from a network failure.

<Processing Flow of Information Processing Performed by Information Processing Device 1>
FIG. 5 is a flowchart for explaining the flow of information processing executed by the information processing apparatus 1 according to the embodiment. The processing in this flowchart starts when the information processing apparatus 1 is turned on, for example.

  The user information acquisition unit 114 refers to the storage unit 10 and acquires skill information of the user who has recovered the network failure (S2). The user evaluation acquisition unit 113 acquires a recovery procedure in which the user tried to recover from a network failure (S4).

  Furthermore, the user evaluation acquisition unit 113 acquires an evaluation by the user regarding the usefulness of the failure recovery procedure (S6). This evaluation includes information indicating whether or not failure recovery has been successfully performed by the failure recovery procedure selected by the user.

  The weight setting unit 115 sets the evaluation weight by the user regarding the usefulness of the failure recovery procedure selected by the user in the learning data used for relearning the learning model based on the skill information of the user (S8). When the weight setting unit 115 sets the weight, the processing in this flowchart ends.

  FIG. 6 is a flowchart for explaining the flow of the weight setting process executed by the weight setting unit 115 according to the embodiment, and is a diagram for explaining step S8 in FIG. 5 in detail.

  The weight setting unit 115 refers to the result data to calculate the average value of the results of the users who tried to recover from the network failure (S80). The weight setting unit 115 acquires the weight by referring to the conversion data based on the average value of the grades (S81).

  When the user's recovery work is successful (Yes in S82), the weight setting unit 115 calculates a value obtained by multiplying the acquired weight by the first weight correction coefficient set according to the user's skill (S83). ). When the user's recovery work has failed (No in S82), the weight setting unit 115 calculates a value obtained by multiplying the acquired weight by the second weight correction coefficient set according to the user's skill (S84). ).

  The weight setting unit 115 sets the weight corrected by multiplying the first weight correction coefficient or the second weight correction coefficient as the weight of the relearning data (S85). In this way, the quality of the learning data for supervised machine learning can be improved by setting the weight of the user's evaluation on the usefulness of the failure recovery procedure based on the user's skills related to network failure recovery and success or failure of the recovery. Can be improved.

<Effects of information processing apparatus 1 according to the embodiment>
As described above, according to the information processing apparatus 1 according to the embodiment, the quality of learning data of supervised machine learning can be improved.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment, A various deformation | transformation and change are possible within the range of the summary. is there. For example, the specific embodiments of device distribution / integration are not limited to the above-described embodiments, and all or a part of them may be configured to be functionally or physically distributed / integrated in arbitrary units. Can do. In addition, new embodiments generated by any combination of a plurality of embodiments are also included in the embodiments of the present invention. The effect of the new embodiment produced by the combination has the effect of the original embodiment.

  Although the case where the information processing apparatus 1 executes the machine learning process has been described above, the machine learning process may be executed by an apparatus different from the information processing apparatus 1. In general, learning data used for machine learning has a large data capacity, and executing the machine learning process requires more resources than the case of executing the model application process. By separating the machine learning process from the information processing apparatus 1, it is possible to balance the resources required for processing and the specifications of the apparatus.

DESCRIPTION OF SYMBOLS 1 ... Information processing apparatus 10 ... Memory | storage part 11 ... Control part 110 ... Model acquisition part 111 ... Unknown fault information input part 112 ... Recovery procedure acquisition part 113 ... User evaluation acquisition Unit 114 ... user information acquisition unit 115 ... weight setting unit 116 ... learning data input unit 117 ... machine learning unit

Claims (8)

A model acquisition unit that acquires failure information related to a failure in the network and acquires a learning model that has been learned to output one or more failure recovery procedure candidates;
A recovery procedure acquisition unit that acquires an output of the learning model that receives failure information received from a user;
A user evaluation acquisition unit that acquires the failure recovery procedure selected by the user from the failure recovery procedures acquired by the recovery procedure acquisition unit, and an evaluation of the usefulness of the failure recovery procedure by the user;
A user information acquisition unit that acquires skill information indicating the skill of failure recovery of the user;
Based on the skill information of the user, a weight setting unit that sets a weight of evaluation by the user regarding the usefulness of the failure recovery procedure selected by the user in learning data used for relearning of the learning model;
An information processing apparatus comprising: The weight setting unit increases the evaluation weight by the user regarding the usefulness of the failure recovery procedure selected by the user, rather than the low evaluation, when the user's failure recovery skill is high.
The information processing apparatus according to claim 1. The user evaluation acquisition unit further acquires information indicating whether or not the failure has been successfully recovered by the failure recovery procedure selected by the user,
The weight setting unit increases the evaluation weight by the user regarding the usefulness of the failure recovery procedure selected by the user as the failure recovery skill of the user is lower when the failure recovery is successful.
The information processing apparatus according to claim 1 or 2. The weight setting unit reduces the weight of evaluation by the user regarding the usefulness of the failure recovery procedure selected by the user as the failure recovery skill of the user is higher when the failure recovery fails.
The information processing apparatus according to claim 3. The learning model is learned to further output information about related equipment incidental to the fault recovery procedure, with fault information about faults in the network as input.
The weight setting unit is based on the skill information of the user by the user regarding the usefulness of the related facility investigation procedure incidental to the failure recovery procedure selected by the user in the learning data used for relearning of the learning model. Further set the weight of the evaluation,
The information processing apparatus according to any one of claims 1 to 4. The user evaluation acquisition unit further acquires a user identifier for specifying the user,
The user information acquisition unit acquires the skill information by referring to a skill database that stores a user identifier and skills related to failure recovery of the user corresponding to the user identifier in association with each other,
The information processing apparatus according to any one of claims 1 to 5. Processor
Obtaining a learning model learned to input failure information related to a failure of the network and to output one or more candidates for failure recovery procedure;
Obtaining the output of the learning model with the failure information received from the user as input;
Obtaining a failure recovery procedure selected by the user from the acquired failure recovery procedures, and evaluating the usefulness of the failure recovery procedure by the user;
Obtaining skill information indicating the failure recovery skill of the user;
Setting a weight of evaluation by the user regarding the usefulness of the failure recovery procedure selected by the user in learning data used for re-learning of the learning model based on the skill information of the user;
Information processing method to execute. On the computer,
A function for acquiring a learning model learned to input failure information related to a failure of the network and to output one or a plurality of failure recovery procedure candidates;
A function of acquiring the output of the learning model with the failure information received from the user as an input;
A function of acquiring the failure recovery procedure selected by the user from the acquired failure recovery procedure, and an evaluation of the usefulness of the failure recovery procedure by the user;
A function of acquiring skill information indicating the failure recovery skill of the user;
A function for setting a weight of evaluation by the user regarding the usefulness of the failure recovery procedure selected by the user in learning data used for re-learning of the learning model based on the skill information of the user;
A program that realizes

Images