JP2021189889A - Evaluation apparatus, evaluation method, and evaluation program - Google Patents

Abstract

To provide an evaluation apparatus which presents a tendency of influence of a specific feature quantity in a dataset on a prediction result of a machine learning model, an evaluation method, and an evaluation program.SOLUTION: An evaluation apparatus 1 includes: a dataset input unit 11 which receives multiple pieces of text data and datasets including correct categories of the text data; a prediction value input unit 12 which receives a prediction value of classification with a machine learning model using the text data; a description value input unit 13 which receives a description value which describes a degree of influence on the prediction value for each of words constituting the text data; and a distribution output unit 14 which outputs distribution data of the description values in each spot appearing in the dataset, for a designated word.SELECTED DRAWING: Figure 1

Description

The present invention relates to an evaluation device, an evaluation method, and an evaluation program for evaluating a machine learning model.

Conventionally, an automatic classification method using a machine learning model has been provided when classifying sentences such as movie reviews as positive or negative, detecting unsolicited emails, or clustering images. Has been done.

At this time, it is important whether the classification accuracy of the machine learning model to be used is reliable, and a method for evaluating the machine learning model has been proposed. For example, Non-Patent Documents 1 and 2 propose a method of visually presenting a prediction result by a machine learning model for a feature amount.
Further, for example, Non-Patent Documents 3 and 4 propose a method of calculating a value explaining the degree of influence of each feature amount on a prediction result of a machine learning model.

Google AI Blog, “The What-If Tool: Code-Free Probing of Machine Learning Models”, Sep. 11, 2018, <https://pair-code.github.io/what-if-tool>. Error terrain analysis for machine learning: Tool and visualizations. Rick Barraza, Russell Eames, Yan Esteve Balducci, Josh Hinds, Scott Hoogerwerf, Eric Horvitz, Ece Kamar, Jacquelyn Krones, Josh Lovejoy, Parham Mohadjer, Ben Noah and Besmira Nushi (Contributed talk) ), Presented at ICLR 2019 Debugging Machine Learning Models Workshop, May 6, 2019, <https://debug-ml-iclr2019.github.io/>. Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. 2016. “'Why Should I Trust You?': Explaining the Predictions of Any Classifier.” In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD' 16, New York, NY, USA: ACM, 1135-1144. Code and visualizations at <https://github.com/marcotcr/lime>. Lundberg, Scott M, and Su-In Lee. 2017. “A Unified Approach to Interpreting Model Predictions.” In Advances in Neural Information Processing Systems 30, eds. I. Guyon et al. Curran Associates, Inc., 4765-4774. Code and visualizations at <https://github.com/slundberg/shap>.

However, in the conventional evaluation method, for example, even if a person can determine which feature amount influences the result in a specific input data, various parts of various input data (text data) can be determined. It was difficult to judge whether or not the specific features (words) appearing in the machine learning model were properly handled.

The present invention is an evaluation device and evaluation capable of presenting a tendency of how a specific feature amount affects the prediction result of a machine learning model in a data set in order to evaluate the reliability of the machine learning model. The purpose is to provide methods and evaluation programs.

The evaluation device according to the present invention has a data set input unit that accepts a plurality of text data and a data set including the correct classification of each of the text data, and a predicted value of the classification by a machine learning model using the text data as an input. A predicted value input unit that accepts an explanatory value input unit that accepts an explanatory value that explains the degree of influence of each word constituting the text data on the predicted value, and a location that appears in the data set for a specified phrase. It is provided with a distribution output unit for outputting the distribution data of the above-mentioned explanatory values.

The distribution output unit may compare and output the distribution data of the explanatory values for the plurality of words and phrases according to the designation of the plurality of words and phrases.

The distribution output unit may output a graph in which the explanatory values are plotted.

The distribution output unit may output the statistical value of the distribution data.

The evaluation device may include a list output unit that outputs the explanatory value, the predicted value, and the correct classification together with the list of the text data including the designated phrase.

The list output unit may output a statistic regarding the correctness of the predicted value in the list.

The list output unit outputs the entire text of the text data selected from the list by emphasizing the specified phrase and the phrase having the upper predetermined number of explanatory values in the text data. May be good.

The evaluation device may include a word / phrase output unit that outputs a predetermined number of words / phrases whose explanatory values are higher in size among the words / phrases included in the data set.

The evaluation device may include a filter unit that selects text data to be processed from the data set based on the correctness classification of the predicted value.

The filter unit may further select the text data to be processed based on the length of the text data.

The evaluation method according to the present invention includes a data set input step that accepts a plurality of text data and a data set including the correct classification of each of the text data, and a predicted value of the classification by a machine learning model using the text data as an input. A predicted value input step to be accepted, an explanatory value input step to accept an explanatory value for explaining the degree of influence of each word constituting the text data on the predicted value, and a place appearing in the data set for a specified phrase. The computer executes the distribution output step of outputting the distribution data of the above-mentioned explanatory values.

The evaluation program according to the present invention is for operating a computer as the evaluation device.

According to the present invention, in order to evaluate the reliability of a machine learning model, it is possible to present a tendency of how a specific feature amount affects the prediction result of the machine learning model in the data set.

It is a figure which shows the functional structure of the evaluation apparatus in embodiment. It is a figure which shows the screen composition example of the visualization tool in an embodiment. It is a figure which shows the screen example of the search result by the keyword list in an embodiment. It is a figure which shows the screen example when each line is selected in the list display of the text data in an embodiment. It is a figure which shows the screen example which displayed the distribution of the explanatory value in a graph in an embodiment. It is a figure which shows the display example of the statistical information about the distribution of the explanatory value in an embodiment. It is a figure which shows the screen example which compared and displayed the distribution of the explanatory value in an embodiment between words.

Hereinafter, an example of the embodiment of the present invention will be described.
FIG. 1 is a diagram showing a functional configuration of the evaluation device 1 in the present embodiment.
The evaluation device 1 is an information processing device (computer) such as a server device or a personal computer, and includes a control unit 10 and a storage unit 20, as well as various data input / output devices and communication devices.

The control unit 10 is a part that controls the entire evaluation device 1, and realizes each function in the present embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU.

The storage unit 20 is a storage area for various programs and various data for making the hardware group function as the evaluation device 1, and may be a ROM, RAM, flash memory, hard disk drive (HDD), or the like. Specifically, the storage unit 20 includes a program (evaluation program) for causing the control unit 10 to execute each function of the present embodiment, a machine learning model to be evaluated, a data set to be input to the machine learning model, and the like. Remember.

Here, the machine learning model performs two-class classification by inputting text data. For example, the probability that a sentence (text data) such as a word of mouth of a movie has a positive content and the probability that the text (text data) has a negative content are output as predicted values.

The control unit 10 includes a data set input unit 11, a predicted value input unit 12, an explanatory value input unit 13, a distribution output unit 14, a list output unit 15, a word output unit 16 (word output unit), and a filter. A unit 17 is provided.

The data set input unit 11 accepts a plurality of text data (Datapoints) and a data set including a correct classification (for example, positive or negative) determined by the user for each of the text data.
The words that make up the text data are the elements that are input to the machine learning model as features.

The predicted value input unit 12 accepts predicted values of classification by a machine learning model using text data as an input.

The explanatory value input unit 13 receives explanatory values (Explanation values) for explaining the degree of influence on the predicted value by the machine learning model for each word constituting the text data.
In the present embodiment, the explanatory value contributes to a positive prediction result when it is a positive value, and contributes to a negative prediction result when it is a negative value. In addition, the larger the absolute value of the explanatory value, the higher the importance of the word, and the greater the degree of influence on the prediction result.
The method for calculating the explanatory value is not limited, but for example, the existing explanatory method proposed in Non-Patent Document 3 or 4 can be applied.

The distribution output unit 14 outputs the distribution data of the explanatory values for each part appearing in the data set for the word specified by the user. At this time, when a plurality of words are specified, the distribution output unit 14 compares and outputs the distribution data of the explanatory values for these the plurality of words.
Specifically, the distribution output unit 14 displays a graph plotting the explanatory values on the display.

Further, the distribution output unit 14 may output the statistical value of this distribution data. Statistics are, for example, total number of occurrences, average, median, minimum, maximum, number affecting positive predictions, number affecting negative predictions, neutral numbers not affecting predictions, etc. be.

The list output unit 15 outputs an explanatory value, a predicted value, and a correct classification together with a list of text data including the designated word.
At this time, the list output unit 15 may output a statistical value regarding the correctness of the predicted value in the list. The statistical values are, for example, the number and ratio of correct answers, the number and ratio of false errors (False Positive), the number and ratio of false detection (False Negative), and the like.

Further, the list output unit 15 uses the entire text of the text data selected from the list as a designated word and a predetermined number (for example, 10) of which the magnitude (absolute value) of the explanatory value is higher in the text data. The word is emphasized and output.
That is, regardless of the prediction result, words with a high degree of influence that are classified into either positive or negative are coordinated.

The word output unit 16 outputs, among the words included in the data set, the words having the upper predetermined number (for example, 10) of the magnitude (absolute value) of the explanatory value. At this time, a predetermined number of high-ranking words may be extracted separately for positive and negative (positive and negative), and may be listed in order of appearance frequency, for example.

The filter unit 17 selects the text data to be processed from the data set based on the input for designating the correct / incorrect classification of the predicted value, that is, the correct answer, the false detection, the detection omission, and the like.
Further, the filter unit 17 may further select the text data to be processed based on the length of the text data.

Next, a screen display example by an evaluation program (visualization tool) for operating the evaluation device 1 as each of the above-mentioned functional units will be described.

FIG. 2 is a diagram showing an example of screen configuration of the visualization tool in the present embodiment.
On this screen, a search function is provided, and it is possible to perform a search by a specified word (keyword) and a batch search by importing a keyword list.

In the area 21, the filter condition for the data set is input. In this example, in the "Result" column, it is required to select at least one of correct answer (Correct), false detection (False Positive), and detection omission (False Negative) as the classification of the correctness of the predicted value.
Further, in the "Word Language" column, it is required to select at least one of 700 words or less, 701 to 1000 words, 1001 to 1400 words, and 1401 words or more as the length of the text data. The method of specifying the length of the text data may be, for example, a mode in which a numerical range is directly input.

In the region 22, 10 words (Top Expansion Keywords) having a higher explanatory value output by the word output unit 16 are displayed in a histogram in order of appearance frequency for each of positive and negative. This display is an output result for the data set after selection by the filter unit 17.

The area 23 is an area in which the data or graph of the search result by the keyword or the keyword list is displayed.
Further, in the area 24, the classification accuracy by the machine learning model and the total number of text data (Datapoints) are displayed.

FIG. 3 is a diagram showing a screen example of a search result by a keyword list in this embodiment.
When the keyword list is taken in by a CSV file or the like, the text data in which each of the plurality of keywords appears is searched, and the list is displayed in the area 31 in the order of the number of the searched text data. If the keyword is not included in the filtered data set, the number of text data in which the keyword appears is 0.
The number of searches may be the number of occurrences of the keyword.

Further, when one is selected from the list of keywords, the list output unit 15 causes the search result of the text data including the selected keyword to be displayed in a list in the area 32.
In this example, in each line of the list, the search results are displayed as a concordance list so that at least a portion of the text data is placed in the center of the selected keyword and with the preceding and following word groups.
This allows the user to easily understand how the specified keyword is used in the dataset.

In each line of the list, the ID of the text data (data point) and the explanatory value of the keyword in this text data are displayed. Further, the predicted value by the machine learning model is displayed as the probability (%) for each division (positive and negative) together with the correct classification (Ground Truth) given by the user.
When a plurality of keywords appear in one text data, each time the keyword appears, it is output to another line.

Further, in the area 33, keywords and filter conditions as search conditions, the number of occurrences of keywords (Occurences) as statistical information of search results, the number of text data including keywords (Datapoints), the number and ratio of correct answers, and the like. The number and percentage of false positives and the number and percentage of missed detections are displayed.

FIG. 4 is a diagram showing a screen example when each line is selected in the list display of text data in the present embodiment.
The entire text of the selected text data is displayed in the area 41. At this time, among the words included in the text data, 10 important words having a high absolute value of the explanatory value and a high influence are highlighted together with the designated keyword.
Specifically, the color of the keyword "good" has been changed, and important words that have a positive effect and important words that have a negative effect are displayed in different background colors.

FIG. 5 is a diagram showing a screen example in which the distribution of explanatory values in the present embodiment is displayed as a graph.
When the graph option is selected, the distribution output unit 14 displays the distribution of the explanatory values of the searched keywords in a graph in the area 51.

In this example, the explanatory values that indicate how the specified keyword "good" is treated in the machine learning model, that is, how much it affects which classification (positive or negative), is the data set. It is plotted for each place where "good" appears.

With the horizontal axis as the explanatory value, the plotted points are color-coded by positive and negative values.
In this case, there are many evaluations that are neutral (explanatory value ≒ 0), but it is visually understood that there are more cases where evaluation is positive (explanatory value> 0) than when evaluation is negative (explanatory value <0). can.

Further, in the area 52, statistical information representing the characteristics of the distribution of the explanatory values plotted on the graph is displayed.
It is also possible to compare and output a plurality of keywords in the distribution graph of the explanatory values and the statistical information.

FIG. 6 is a diagram showing a display example of statistical information regarding the distribution of explanatory values in the present embodiment.
In this example, the two words "better" and "worst" are compared and their respective statistics are displayed side by side.

Specifically, for example, the number of occurrences of the keyword (Occurences), the mean value (Mean), the median value (Median), the minimum value (Min), the maximum value (Max), and the number affecting the positive prediction (explanation). A value> 0), a number affecting the negative prediction (explanatory value <0), and a neutral number not affecting the prediction (explanatory value ≈ 0) are shown.
The threshold value of the explanatory value for separating positive, neutral, and negative may be set in advance.

From this statistical information, for example, the following features can be grasped.
-"Better" appears more frequently than "worst".
-The average and median values of "better" are neutral near 0, but "worst" is closer to the negative side.
-Both "better" and "worst" have the same minimum and maximum values and are spread on both the positive and negative sides, but "worst" has a larger value, so it has a greater effect on the prediction results. The degree is high.
-"Better" has the same number of positive evaluations and negative evaluations, but "worst" has almost negative evaluations.

FIG. 7 is a diagram showing a screen example in which the distribution of explanatory values in the present embodiment is compared and displayed between words.
In this example, the distribution of explanatory values is compared between "good" and "bad".
It can be seen that the explanatory value of "good" is biased toward the positive side, and the explanatory value of "bad" is biased toward the negative side.

According to the present embodiment, the evaluation device 1 sets a data set including a plurality of text data, a predicted value of classification by a machine learning model using the text data as an input, and an explanatory value for each word constituting the text data. As an input, the distribution data of the explanatory values for each part appearing in the data set is output for the specified word.
As a result, the evaluation device 1 can visualize and present the tendency of how a specific word (feature amount) affects the prediction result of the machine learning model in the data set.

As a result, the user can confirm the certainty of the distribution of the explanatory values for each word. For example, in the data set owned by the user, the distribution of the explanatory values of the words that are considered to be neutral is either positive or negative. Unnatural distributions such as being biased or, conversely, the explanatory values of words that are considered important are concentrated near 0 can be easily found. As a result, the user can appropriately evaluate the reliability of the machine learning model to be used.

Since the evaluation device 1 compares and outputs the distribution data of the explanatory values for these words according to the designation of a plurality of words, the user can easily grasp the difference in the distribution between the words. You can check the certainty of the distribution.

Since the evaluation device 1 outputs a graph plotting the explanatory values, the user can easily visually grasp the distribution of the explanatory values and confirm the certainty.
Since the evaluation device 1 outputs the statistical value of the distribution data, the user can confirm the certainty of the distribution of the explanatory values based on the objective numerical data.

The evaluation device 1 outputs an explanatory value, a predicted value, and a correct classification together with a list of text data including the designated word.
This allows the user to see how the specified word is used in the dataset and to investigate in detail how much each occurrence affects the predicted value of the machine learning model. ..

The evaluation device 1 outputs a statistical value regarding the correctness of the predicted value in the list of text data. As a result, the user can confirm the prediction accuracy of the machine learning model for the text data including the specified word and understand the influence of this word.

The evaluation device 1 outputs the entire text of the text data selected from the list after emphasizing the designated words and the words having the upper predetermined number of explanatory values in the text data.
As a result, the user can check the entire text data including the specified word, easily grasp other word groups having a high degree of influence, and comprehensively evaluate the reliability of the machine learning model.
Further, the evaluation device 1 can appropriately present the feature amount to the user by outputting the words having the magnitude of the upper predetermined number of the explanatory values among the words included in the data set.

The evaluation device 1 selects the text data to be processed from the data set based on the classification of the correctness of the predicted value or the length of the text data.
As a result, the user can narrow down the types of text data and evaluate the degree of influence of a specific word in this range in detail.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Moreover, the effects described in the above-described embodiments are merely a list of the most suitable effects resulting from the present invention, and the effects according to the present invention are not limited to those described in the embodiments.

In the above-described embodiment, the case where the text data is in English has been exemplified, but the present invention is not limited to this, and the present invention can be applied to text data in any language.

Further, in the above-described embodiment, when comparing the distribution data of the explanatory values between a plurality of words, an example in which two keywords are specified is shown, but the number of words to be compared is not limited to two, and three or more words are compared. Words may be compared and shown by graphs and statistical information.

Further, in the above-described embodiment, as the feature amount of the text data, a word constituting the text data is exemplified, and a mode of outputting the distribution of the explanatory values has been described. In the present invention, the feature amount of the text data is not limited to a word, and may be a phrase composed of one or a plurality of words.
In this case, one or a plurality of consecutive words are input as the keyword for the search, and the presentation of important words by the list output unit 15, the word output unit 16, and the like results in the presentation of a predetermined number of words and phrases. In addition, in order to discriminate words and phrases in text data, a dictionary in which words and phrases are defined in advance may be provided, or word / phrase delimiters may be automatically discriminated by existing parsing techniques.

The evaluation method by the evaluation device 1 is realized by software. When realized by software, the programs that make up this software are installed in the information processing device (computer). Further, these programs may be recorded on a removable medium such as a CD-ROM and distributed to the user, or may be distributed by being downloaded to the user's computer via a network. Further, these programs may be provided to the user's computer as a Web service via a network without being downloaded.

1 Evaluation device 10 Control unit 11 Data set input unit 12 Predicted value input unit 13 Explanation value input unit 14 Distribution output unit 15 List output unit 16 Word output unit (word output unit)
17 Filter section 20 Storage section

Claims (12)

A dataset input unit that accepts multiple text data and a dataset containing the correct classification of each text data,
A predicted value input unit that accepts predicted values of the classification by a machine learning model that inputs the text data, and
An explanatory value input unit that accepts explanatory values that explain the degree of influence on the predicted value for each word that constitutes the text data, and an explanatory value input unit.
An evaluation device including a distribution output unit that outputs distribution data of the explanatory values for each location appearing in the data set for a specified phrase. The evaluation device according to claim 1, wherein the distribution output unit compares and outputs distribution data of the explanatory values for the plurality of words and phrases in response to the designation of the plurality of words and phrases. The evaluation device according to claim 1 or 2, wherein the distribution output unit outputs a graph in which the explanatory values are plotted. The evaluation device according to any one of claims 1 to 3, wherein the distribution output unit outputs statistical values of the distribution data. The evaluation device according to any one of claims 1 to 4, further comprising a list output unit that outputs the explanatory value, the predicted value, and the correct classification together with the list of the text data including the designated phrase. The evaluation device according to claim 5, wherein the list output unit outputs statistical values relating to correctness of the predicted values in the list. The list output unit outputs the entire text of the text data selected from the list by emphasizing the specified phrase and the phrase having the upper predetermined number of explanatory values in the text data. Item 5. The evaluation device according to claim 6. The evaluation device according to any one of claims 1 to 7, further comprising a word / phrase output unit that outputs a word / phrase having a predetermined number of words / phrases having a magnitude higher than that of the explanatory value among the words / phrases included in the data set. The evaluation device according to any one of claims 1 to 8, further comprising a filter unit for selecting text data to be processed from the data set based on the classification of correctness of the predicted value. The evaluation device according to claim 9, wherein the filter unit further selects the text data to be processed based on the length of the text data. A dataset input step that accepts multiple text data and a dataset containing the correct classification of each of the text data.
A predicted value input step that accepts predicted values of the classification by a machine learning model using the text data as input, and
An explanatory value input step that accepts an explanatory value that explains the degree of influence on the predicted value for each word that constitutes the text data, and an explanatory value input step.
An evaluation method in which a computer executes a distribution output step for outputting distribution data of the explanatory values for each location appearing in the data set for a specified phrase. An evaluation program for operating a computer as the evaluation device according to any one of claims 1 to 10.

Images