KR102630394B1 - Method for providing table data analysis information based on explainable artificial intelligence and learning server using the same - Google Patents

Abstract

According to the present invention, in a method for providing table data analysis information based on explainable artificial intelligence (XAI), (a) first table data for verifying each of the first machine learning model to the nth machine learning model Each set to the nth table data set is input to each of the first machine learning model to the nth machine learning model, so that the first to nth learning states corresponding to each of the first table data set to the nth table data set are provided. In a state in which a learning state is created, the learning server causes the user terminal to display the first machine learning model selection UI to the nth machine learning model selection UI corresponding to each of the first machine learning model to the nth machine learning model. Supporting display in a first area on the screen of the user terminal; (b) Among the first machine learning model selection UI and the nth machine learning model selection UI, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected, the learning server is sent to the user terminal. In the learning result verification tab corresponding to the kth machine learning model selection UI, each of the k_1th partial table data set to the k_mth table data set included in the kth table data set input to the kth machine learning model is displayed. Supporting display of a k_1th partial table data set selection UI to a k_mth partial table data set selection UI in a second area on the screen of the user terminal; and (c) among the k_1th partial table data set selection UI to the k_mth partial table data set selection UI, when the k_ith partial table data set selection UI corresponding to the k_ith partial table data set is selected, the learning server , Based on the explainable artificial intelligence, first table row data to p-th table row data included in the k_i th partial table data set in the XAI tab corresponding to the k_i th partial table data set selection UI, the first First classification label data to p-th classification label data corresponding to each of the table row data to the p-th table row data, and first classification prediction data to the first classification label data corresponding to each of the first table row data to the p-th table row data. A method including: supporting display of p classification prediction data as the table data analysis information in a third area on the screen of the user terminal and a server using the same are disclosed.

Description

Method for providing table data analysis information based on explainable artificial intelligence (XAI) and learning server using the same {METHOD FOR PROVIDING TABLE DATA ANALYSIS INFORMATION BASED ON EXPLAINABLE ARTIFICIAL INTELLIGENCE AND LEARNING SERVER USING THE SAME}

The present invention relates to a method for interpretably visualizing prediction results derived from table data by a machine learning model based on explainable artificial intelligence (XAI; eXplainable Artificial Intelligence), and a learning server using the same. More specifically, the present invention relates to , a method of training table data to a machine learning model based on explainable artificial intelligence and providing table data analysis information so that prediction results for the table data derived by the machine learning model can be interpreted, and a learning server using the same. will be.

With the development of machine learning and deep learning algorithms, artificial intelligence (AI) technology shows excellent performance in quickly processing large amounts of data and solving and predicting complex tasks. The fields of application are becoming increasingly diverse.

However, as the data that needs to be analyzed becomes more diverse and the tasks performed by artificial intelligence become more complex, the transparency of the decision-making process and the transparency of the decision-making process regarding what criteria and basis the machine learning model uses to make decisions on a given task become more and more important. Securing reliability is becoming increasingly important.

However, since the internal operation of machine learning models such as deep learning to generate output according to input is hidden like a “black box”, how do structures such as hidden layers that make up the machine learning model produce results? In many cases, it is difficult or impossible to explain how decisions are made and based on what criteria decisions are made.

This opacity causes anxiety about the reliability and stability of artificial intelligence technology and makes it difficult to understand the basis for decisions made by machine learning models. Recently, to solve this problem, the decision-making process of machine learning models has been introduced. Research and development on “interpretable artificial intelligence” (XAI; eXplainable Artificial Intelligence) technology that can analyze is actively underway.

However, explainable artificial intelligence technology usually focuses only on acquiring visual data about the decision-making process of a machine learning model, lacking intuitive guidelines on how to interpret such visual data, and predicting the machine learning model's predicted results. In many cases, it does not provide the detailed information needed to interpret effectively.

Therefore, improvement measures to solve the above problems are required.

The purpose of the present invention is to solve all of the above-mentioned problems.

In addition, the purpose of the present invention is to enable an intuitive understanding of the process of deriving the prediction results of a machine learning model by visualizing the values on the table data that contribute to the generation of prediction results of a machine learning model that analyzes table data in a table or graph. .

In addition, the present invention visualizes information about the degree to which numerical changes in values in table data contribute to the generation of prediction results of a machine learning model and the impact of each value, individually or in combination, on the generation of prediction results of a machine learning model. provided for a different purpose.

In addition, the present invention provides a user-friendly interface that can flexibly visualize and explore table data analysis information used to interpret the prediction results of a machine learning model for table data and the process of deriving the prediction results of the machine learning model. It is for another purpose.

In addition, another purpose of the present invention is to facilitate the interpretation of the prediction result derivation process by providing both visual data and numerical data regarding the prediction result derivation process of the machine learning model as needed.

In order to achieve the object of the present invention as described above and realize the characteristic effects of the present invention described later, the characteristic configuration of the present invention is as follows.

According to one aspect of the present invention, in a method for providing table data analysis information based on explainable artificial intelligence (XAI), (a) a method for verifying each of the first to nth machine learning models; A first learning state corresponding to each of the first table data set to the nth table data set by inputting each of the first table data set to the nth table data set into each of the first machine learning model to the nth machine learning model. In a state where the to nth learning state is created, the learning server causes the user terminal to select a first machine learning model corresponding to each of the first to nth machine learning models. UI to select a nth machine learning model. Supporting display of a UI in a first area on the screen of the user terminal; (b) Among the first machine learning model selection UI and the nth machine learning model selection UI, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected, the learning server is sent to the user terminal. In the learning result verification tab corresponding to the kth machine learning model selection UI, each of the k_1th partial table data set to the k_mth table data set included in the kth table data set input to the kth machine learning model is displayed. Supporting display of a k_1th partial table data set selection UI to a k_mth partial table data set selection UI in a second area on the screen of the user terminal; and (c) among the k_1th partial table data set selection UI to the k_mth partial table data set selection UI, when the k_ith partial table data set selection UI corresponding to the k_ith partial table data set is selected, the learning server , Based on the explainable artificial intelligence, first table row data to p-th table row data included in the k_i th partial table data set in the XAI tab corresponding to the k_i th partial table data set selection UI, the first First classification label data to p-th classification label data corresponding to each of the table row data to the p-th table row data, and first classification prediction data to the first classification label data corresponding to each of the first table row data to the p-th table row data. A method including supporting display of p classification prediction data as the table data analysis information in a third area on the screen of the user terminal is disclosed.

As an example, in step (c), when the k_i-th partial table data set selection UI is selected, the learning server causes the user terminal to select the first table row data to the p-th table row data, respectively. Supports displaying 1st table row data to jth table row data in the third area on the screen of the user terminal, and when specific table row data is selected among the 1st table row data to the pth table row data, The learning server, based on the explainable artificial intelligence, provides a plurality of classification classes to be predicted by the kth machine learning model and a plurality of prediction probabilities predicted by the kth machine learning model for each classification class to the user. A method is disclosed, characterized in that it supports display in a fourth area on the screen of a terminal.

As an example, when the specific table row data is selected from the first table row data to the p-th table row data, the learning server, based on the explainable artificial intelligence, (i) the first table row data to Among the j-th table row data, table row data with a high contribution to at least one of the plurality of classification classes are arranged in descending order to determine first to x-th contributing table row data, where x is the is an integer less than or equal to j, and (ii) the first contribution to the xth contribution corresponding to the first contribution table row data to the A method is disclosed, characterized in that it supports additional display in an area.

As an example, when the specific table row data is selected from the first table row data to the p-th table row data, the learning server, based on the explainable artificial intelligence, (i) the first table row data to Perform preprocessing on each of the jth table row data to generate first table row preprocessing data to jth table row preprocessing data to be additionally displayed in the fourth area on the screen of the user terminal, (ii) ) A method is disclosed, wherein the first contribution to the xth contribution are calculated with reference to the first table row preprocessing data to the jth table row preprocessing data.

As an example, the learning server causes the user terminal to display the first table row data to the j-th table row data, respectively, corresponding to each of the first table row data to the j-th table row data. Supports displaying a table row table of contents selection UI to the jth table row table of contents selection UI in the fourth area on the screen of the user terminal, and the learning server allows the user terminal to (i) the first table row table of contents When the first specific table row table of contents selection UI is selected among the selection UI or the jth table row table of contents selection UI, a first graph creation option UI is generated in the first specific table row table of contents selection UI, and (ii) the first graph creation option UI is selected. When the graph creation option UI is selected, based on the explainable artificial intelligence, the first specific table row data corresponding to the first specific table row table of contents selection UI is selected according to an increase or decrease in the first specific table row data value. A method is disclosed, characterized in that it supports displaying a specific single item relationship graph showing the relationship of contribution for each classification class in a fifth area on the screen of the user terminal.

As an example, the learning server causes the user terminal to display the first table row data to the j-th table row data, respectively, corresponding to each of the first table row data to the j-th table row data. Supports displaying a table row table of contents selection UI to a jth table row table of contents selection UI in the fourth area on the screen of the user terminal, and the learning server causes the user terminal to display the first table row table of contents selection UI to jth table row table of contents selection UI. When the second specific table row table of contents selection UI is selected together with the first specific table row table of contents selection UI among the j-th table row table of contents selection UI, the first specific table row table of contents selection UI and the second specific table row table of contents selection UI Supports creating a second graph creation option UI that allows the user to select the creation of either a single-item relationship graph or a multiple-item relationship graph for at least one of the following, and the learning server, based on the explainable artificial intelligence, allows the user to (i) When the single item relationship graph is selected through the second graph creation option UI, the terminal causes the first specific table row data or the second specific table row corresponding to the first specific table row table of contents selection UI. For the second specific table row data corresponding to the table of contents selection UI, a first single item relationship graph or a second specific table row showing the relationship of contribution to each classification class according to an increase or decrease in the first specific table row data value. Supporting displaying a second single item relationship graph showing the relationship of contribution to each classification class according to an increase or decrease in data value as the single item relationship graph in the fifth area on the screen of the user terminal, (ii) When the multi-item relationship graph is selected through the second graph creation option UI, for the first specific table row data and the second specific table row data, the first specific table row data value and the second specific table row data A method is disclosed, characterized in that it supports displaying a relationship graph showing the relationship of contribution to each classification class according to an increase or decrease in a column data value as the multi-item relationship graph in the fifth area on the screen of the user terminal. do.

As an example, in step (c), when the k_i th partial table data set selection UI is selected, the learning server selects the first classification label data to the p th classification label data to the first classification prediction data to the first classification label data. In comparison with the p classification prediction data, for specific table row data, which are each of the first table row data to the p th table row data, the user terminal is provided with (i) specific classification label data corresponding to the specific table row data. If it matches specific classification prediction data, the specific classification label data and the specific classification prediction data are displayed with a first visual effect, and (ii) the specific classification label data corresponding to the specific table row data is displayed in the specific classification. If they do not match the prediction data, a method is disclosed, wherein the specific classification label data and the specific classification prediction data are displayed as a second visual effect different from the first visual effect.

As an example, in step (b), the learning server causes the user terminal to use the Displays the k_1th partial learning state to the k_mth partial learning state included in the kth learning state generated for the kth machine learning model on the k_1 partial table data set selection UI or the k_mth partial table data set selection UI, respectively. A method characterized in that it supports to do so is disclosed.

As an example, the learning server allows the user terminal to evaluate the learning model code, partial data set name, data count, running start time, running end time, and running accuracy as the k_1th partial running state to the k_mth partial running state, respectively. A method is disclosed, characterized in that it supports displaying at least one of a numerical value and whether running is completed on the k_1th partial table data set selection UI or the k_mth partial table data set selection UI.

According to another aspect of the present invention, a learning server for providing table data analysis information based on explainable artificial intelligence (XAI), comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions, wherein the processor includes: (1) a first table data set to an n-th table data set for verifying each of the first to n-th machine learning models; In a state where the first learning state to the nth learning state corresponding to each of the first table data set to the nth table data set is generated by inputting each of the first machine learning model to the nth machine learning model. , causing the user terminal to display the first machine learning model selection UI to the nth machine learning model selection UI corresponding to each of the first machine learning model to the nth machine learning model in the first area on the screen of the user terminal. Supporting process, (2) among the first machine learning model selection UI to the nth machine learning model selection UI, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected, the user terminal In the learning result verification tab corresponding to the kth machine learning model selection UI, the first table data set corresponding to each of the k_1th partial table data set to the k_mth table data set included in the kth table data set input to the kth machine learning model A process for supporting display of a k_1th partial table data set selection UI to the k_mth partial table data set selection UI in a second area on a screen of the user terminal, and (3) the k_1th partial table data set selection UI to the k_mth partial table data set selection UI. Among the partial table data set selection UIs, when the k_ith partial table data set selection UI corresponding to the k_ith partial table data set is selected, based on the explainable artificial intelligence, the k_ith partial table data set selection UI corresponding to the k_ith partial table data set selection UI is selected. In the Classification label data and first classification prediction data to p-th classification prediction data corresponding to each of the first table row data to p-th table row data, as the table data analysis information, is displayed on the third screen of the user terminal. A learning server that performs a process to support display in an area is disclosed.

As an example, in the process (3), when the k_i-th partial table data set selection UI is selected, the processor causes the user terminal to select a first table row data for each of the first table row data to the p-th table row data. Supports displaying table row data to j-th table row data in the third area on the screen of the user terminal, and when specific table row data is selected among the first table row data to the p-th table row data, Based on the explainable artificial intelligence, the processor generates a plurality of classification classes to be predicted by the kth machine learning model and a plurality of prediction probabilities predicted by the kth machine learning model for each classification class of the user terminal. A learning server is disclosed, characterized in that it supports display in a fourth area on the screen.

As an example, when the specific table row data is selected among the first table row data to the p-th table row data, the processor, based on the explainable artificial intelligence, (i) the first table row data to the p-th table row data. Among the j-th table row data, table row data with a high contribution to at least one of the plurality of classification classes are arranged in descending order to determine the first to x-th contributing table row data, where x is the j is an integer less than or equal to - and (ii) the fourth area on the screen of the user terminal graphs the first contribution to the x contribution corresponding to the first contribution table row data to the x contribution table row data A learning server is disclosed, characterized in that it supports additional display.

As an example, when the specific table row data is selected among the first table row data to the p-th table row data, the processor, based on the explainable artificial intelligence, (i) the first table row data to the p-th table row data. Perform preprocessing on each of the jth table row data to generate first table row preprocessing data to jth table row preprocessing data to be additionally displayed in the fourth area on the screen of the user terminal, (ii) A learning server is disclosed, wherein the first contribution to the xth contribution are calculated with reference to the first table row preprocessing data to the jth table row preprocessing data.

As an example, the processor causes the user terminal to display a first table corresponding to each of the first table row data to the j-th table row data displayed for each of the first table row data to the p-th table row data. Supports displaying a column table of contents selection UI to a jth table row table of contents selection UI in the fourth area on the screen of the user terminal, wherein the processor causes the user terminal to (i) the first table row table of contents selection UI When the first specific table row table of contents selection UI is selected among the jth table row table of contents selection UI, a first graph creation option UI is generated in the first specific table row table of contents selection UI, and (ii) the first graph is generated. When the option UI is selected, based on the explainable artificial intelligence, each classification according to the increase or decrease of the first specific table row data value for the first specific table row data corresponding to the first specific table row table of contents selection UI. A learning server is disclosed, characterized in that it supports displaying a specific single item relationship graph showing the relationship of contribution to a class in a fifth area on the screen of the user terminal.

As an example, the processor causes the user terminal to display a first table corresponding to each of the first table row data to the j-th table row data displayed for each of the first table row data to the p-th table row data. Supports displaying a column table of contents selection UI to a jth table row table of contents selection UI in the fourth area on the screen of the user terminal, wherein the processor causes the user terminal to display the first table row table of contents selection UI to the jth table row table of contents selection UI to the jth table row. j When the second specific table row table of contents selection UI is selected together with the first specific table row table of contents selection UI among the table row table of contents selection UI, at least one of the first specific table row table of contents selection UI and the second specific table row table of contents selection UI Supports generating a second graph creation option UI that can select the creation of either a single item relationship graph or a multiple item relationship graph, and the processor, based on the explainable artificial intelligence, causes the user terminal to (i) When the single item relationship graph is selected through the second graph creation option UI, the first specific table row data corresponding to the first specific table row table of contents selection UI or the second specific table row table of contents selection UI For the second specific table row data corresponding to, a first single item relationship graph showing a relationship of contribution to each classification class according to an increase or decrease in the first specific table row data value or a second specific table row data value. Supporting displaying a second single item relationship graph showing the relationship of contribution to each classification class according to increase or decrease as the single item relationship graph in the fifth area on the screen of the user terminal, (ii) the second single item relationship graph; When the multi-item relationship graph is selected through the graph creation option UI, for the first specific table row data and the second specific table row data, the first specific table row data value and the second specific table row data value A learning server is disclosed, characterized in that it supports displaying a relationship graph showing the relationship of contribution to each classification class according to an increase or decrease of as the multi-item relationship graph in the fifth area on the screen of the user terminal.

As an example, in the process (3), when the k_i-th partial table data set selection UI is selected, the processor selects the first classification label data to the p-th classification label data to the first classification prediction data to the p-th classification label data. Compared with the classification prediction data, for the specific table row data, which is each of the first table row data to the pth table row data, the user terminal is prompted to (i) have specific classification label data corresponding to the specific table row data. If it matches specific classification prediction data, the specific classification label data and the specific classification prediction data are displayed as a first visual effect, and (ii) the specific classification label data corresponding to the specific table row data predicts the specific classification. If they do not match the data, a learning server is disclosed, wherein the specific classification label data and the specific classification prediction data are displayed as a second visual effect different from the first visual effect.

As an example, in the process (2), the processor causes the user terminal to use the k_1th partial table data set corresponding to each of the k_1th partial table data set to the k_mth table data set used for learning the kth machine learning model. To display the k_1th partial learning state to the k_mth partial learning state included in the kth learning state generated for the kth machine learning model on the partial table data set selection UI or the k_mth partial table data set selection UI, respectively. A learning server characterized by support is disclosed.

As an example, the processor causes the user terminal to use the k_1th partial running state to the k_mth partial running state, respectively, with a learning model code, partial data set name, data count, running start time, running end time, and running accuracy evaluation value. A learning server is disclosed, characterized in that it supports displaying at least one of whether running is complete or not on the k_1th partial table data set selection UI to the k_mth partial table data set selection UI.

In addition, a computer-readable recording medium for recording a computer program for executing the method of the present invention is further provided.

The present invention has the effect of enabling an intuitive understanding of the process of deriving the prediction results of a machine learning model by visualizing the values on the table data that contribute to the generation of prediction results of a machine learning model that analyzes table data in a table or graph.

In addition, the present invention visualizes information about the degree to which numerical changes in values in table data contribute to the generation of prediction results of a machine learning model and the impact of each value, individually or in combination, on the generation of prediction results of a machine learning model. It has an effect.

In addition, the present invention has the effect of providing a user-friendly interface that can flexibly visualize and explore the table data analysis information used to interpret the prediction results of the machine learning model for table data and the process of deriving the prediction results of the machine learning model. There is.

In addition, the present invention has the effect of facilitating the interpretation of the prediction result derivation process by providing both visual data and numerical data regarding the prediction result derivation process of the machine learning model as needed.

The following drawings attached for use in explaining embodiments of the present invention are only some of the embodiments of the present invention, and to those skilled in the art (hereinafter “those skilled in the art”), the invention Other drawings can be obtained based on these drawings without further work being done.
Figure 1 schematically illustrates a learning server that provides table data analysis information based on explainable artificial intelligence according to an embodiment of the present invention.
Figure 2 is a flowchart schematically showing a process for providing table data analysis information based on explainable artificial intelligence according to an embodiment of the present invention;
Figure 3 schematically shows how the first machine learning model selection UI to the nth machine learning model selection UI are displayed in the first area on the screen of the user terminal according to an embodiment of the present invention;
Figure 4 shows the k_1th partial table data set corresponding to each of the k_1th partial table data set to the k_mth partial table data set included in the kth table data set input to the kth machine learning model according to an embodiment of the present invention. It schematically shows how the selection UI to the k_mth partial table data set selection UI is displayed in the second area on the screen of the user terminal,
5 and 6 show information on the first to pth table row data included in the k_i partial table data set in the XAI tab corresponding to the k_i partial table data set selection UI according to an embodiment of the present invention. It schematically shows how table data analysis information is displayed in the third area on the screen of the user terminal,
7 is a diagram showing that when specific table row data is selected among the first to p-th table row data in the third area according to an embodiment of the present invention, the first to p-th table row data included in the specific table row data are shown in FIG. j Schematically shows how analysis information about table row data is displayed in the fourth area on the screen of the user terminal,
FIG. 8 illustrates selecting one of the first table row table of contents selection UI to the j table row table of contents selection UI corresponding to each of the first to j table row data in the fourth area according to an embodiment of the present invention. It schematically shows how a single item relationship graph is displayed in the fifth area on the screen of the user terminal,
9 shows selection of two of the first table row table of contents selection UI to the j table row table of contents selection UI corresponding to each of the first to j table row data in the fourth area according to an embodiment of the present invention. This schematically shows how a multi-item relationship graph is displayed in the fifth area on the screen of the user terminal.

The detailed description of the present invention described below refers to the accompanying drawings, which show by way of example specific embodiments in which the present invention may be practiced to make clear the objectives, technical solutions and advantages of the present invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention.

Additionally, throughout the description and claims, the word “comprise” and variations thereof are not intended to exclude other technical features, attachments, components or steps. Other objects, advantages and features of the invention will appear to those skilled in the art, partly from this description and partly from practice of the invention. The examples and drawings below are provided by way of example and are not intended to limit the invention.

Moreover, the present invention encompasses all possible combinations of the embodiments shown herein. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description that follows is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, together with all equivalents to what those claims assert, if properly described. Similar reference numbers in the drawings refer to identical or similar functions across various aspects.

Hereinafter, in order to enable those skilled in the art to easily practice the present invention, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 schematically shows a learning server 1000 that provides table data analysis information based on explainable artificial intelligence according to an embodiment of the present invention.

Referring to FIG. 1, the learning server 1000 includes a memory 1001 that stores instructions for providing table data analysis information based on explainable artificial intelligence and a memory 1001 that stores instructions stored in the memory based on explainable artificial intelligence. Thus, it may include a processor 1002 that provides table data analysis information. At this time, the learning server 1000 may include a personal computer (PC), a mobile computer, etc.

Specifically, the learning server 1000 is typically a computing device (e.g., a device that may include a computer processor, memory, storage, input and output devices, and other components of a conventional computing device; electronic devices such as routers, switches, etc. the desired system using a combination of a communication device (electronic information storage systems, such as network attached storage (NAS) and storage area network (SAN)) and computer software (i.e., instructions that cause a computing device to function in a particular way) Performance may be achieved.

Additionally, the processor of the computing device may include hardware components such as a Micro Processing Unit (MPU) or Central Processing Unit (CPU), cache memory, and data bus. Additionally, the computing device may further include an operating system and a software component of an application that performs a specific purpose.

However, this does not exclude the case where the computing device includes an integrated processor in which a medium, processor, and memory for implementing the present invention are integrated.

Meanwhile, in the present invention, it is possible to explain the machine learning model and the table data input to the machine learning model. It supports a server that generates table data masking information based on artificial intelligence and provides and displays the obtained table data masking information to the user terminal. The servers may be the same or separate servers, and in the present invention, they are collectively referred to as learning servers.

The process by which the learning server 1000 configured as above provides table data analysis information based on explainable artificial intelligence will be described below with reference to FIGS. 2 to 9.

First, Figure 2 is a flowchart schematically showing a process for providing table data analysis information based on explainable artificial intelligence according to an embodiment of the present invention.

Referring to FIG. 2, the overall process for obtaining table data analysis information according to an embodiment of the present invention will be described. First table data sets to verify each of the first machine learning model to the nth machine learning model. A state in which each of the n-th table data sets is input to each of the first to n-th machine learning models, thereby generating first to n-th learning states corresponding to each of the first to n-th table data sets. In, the learning server 1000 causes the user terminal to display the first machine learning model selection UI to the nth machine learning model selection UI corresponding to each of the first to nth machine learning models as the first machine learning model selection UI to the nth machine learning model selection UI on the screen of the user terminal. Display in area 1 can be supported (S110). For reference, here n is an integer greater than 1.

And, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected among the first machine learning model selection UI to the nth machine learning model selection UI, the learning server 1000 causes the user terminal to In the learning result verification tab corresponding to the k machine learning model selection UI, the k_1 partial table data set included in the k table data set input to the k machine learning model to the k_1 partial table corresponding to each of the k_m table data sets. The data set selection UI to the k_mth partial table data set selection UI may be supported to be displayed in a second area on the screen of the user terminal (S120). For reference, here k is an integer greater than or equal to 1 and less than n, and m is an integer greater than or equal to 1.

Next, when the k_ith partial table data set selection UI corresponding to the k_ith partial table data set is selected among the k_1th partial table data set selection UI to the k_mth partial table data set selection UI, the learning server performs an explainable artificial intelligence. Based on intelligence, in the First classification label data to p-th classification label data corresponding to each of the data and first classification prediction data to p-th classification prediction data corresponding to each of the first table row data to p-th table row data as table data analysis information. , it can be supported to display in the third area on the screen of the user terminal (S130). For reference, here i is an integer greater than or equal to 1 and less than or equal to m, and p is an integer greater than or equal to 1.

In the above, the process by which the learning server 1000 provides table data analysis information based on explainable artificial intelligence was briefly described, and this will be described in more detail below using FIGS. 3 to 9.

First, Figure 3 schematically shows how the first machine learning model selection UI to the nth machine learning model selection UI are displayed in the first area 210 on the screen of the user terminal according to an embodiment of the present invention. .

According to FIG. 3, each of the first table data set to the nth table data set for verifying each of the first machine learning model to the nth machine learning model is input to each of the first machine learning model to the nth machine learning model, thereby In a state where the first learning state to the nth learning state corresponding to each of the 1 table data set to the nth table data set are created, the learning server 1000 causes the user terminal to select the input data set of each machine learning model. The first machine learning model selection UI to the nth machine learning model selection UI corresponding to each of the first to nth machine learning models that can be selected by the user in the project tab 201 where the learning status is listed is displayed on the user terminal. It can support display in the first area 210 on the screen.

Here, in the case of FIG. 3, the first machine learning model selection UI to the fifth machine learning model selection UI (211, 212, 213, 214, 215) are displayed in the first area 210 on the first page of the project tab 201. shows what is displayed on the screen of the user terminal as an example.

In addition, each of the first machine learning model selection UI to the nth machine learning model selection UI according to the present invention corresponds to the first to nth table data sets of each of the first to nth machine learning models. The first to nth running states can be displayed. For example, referring to FIG. 3, the project name (Name) naming each machine learning model learning as the first learning state to the nth learning state, and the name of the data set used for learning each machine learning model (DataSet) , the progress status of learning of each machine learning model (Status), each project creation date (Created), etc. are displayed on each of the first machine learning model selection UI to the nth machine learning model selection UI to help the user make a selection. You can.

Next, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected among the first machine learning model selection UI to the nth machine learning model selection UI, the learning server 1000 causes the user terminal to In the learning result verification tab corresponding to the kth machine learning model selection UI, the k_1th part included in the kth table data set input to the kth machine learning model is the k_1th part corresponding to each of the k_th table data sets to the k_mth table data set. The table data set selection UI to the k_mth partial table data set selection UI may be supported to be displayed in a second area on the screen of the user terminal.

Accordingly, Figure 4 shows the k_1th part corresponding to each of the k_1th partial table data set to the k_mth partial table data set included in the kth table data set input to the kth machine learning model according to an embodiment of the present invention. It schematically shows how the table data set selection UI to the k_mth partial table data set selection UI is displayed in the second area 220 on the screen of the user terminal.

In the case of FIG. 4, the k_1th partial table data set and the k_2th partial table data included in the kth table data set are displayed in the second area 220 in the verification tab 202 where the running status for each partial table data set is listed. As an example, the k_1th partial table data set selection UI and the k_2nd partial table data set selection UI (221, 222) corresponding to each set are displayed on the screen of the user terminal. Referring to FIGS. 3 and 4 together, while the third machine learning model selection UI 213 is selected as the kth machine learning model selection UI in FIG. 3, FIG. 4 shows the corresponding third machine in the verification tab 202. A second area ( 220) is shown.

Here, if each of the first table data set to the nth table data set is an entire data set used in machine learning or deep learning, it is included in the kth table data set, which is one of the first table data set to the nth table data set. The k_1th partial table data set to the k_mth partial table data set may be a partial data set obtained by dividing and grouping the kth table data set, and the learning results of the machine learning model can be confirmed for each partial data set. For example, if a specific table data set consists of 1000 rows of data, divide it into the first partial table data set to the tenth partial table data set so that each partial table data set consists of 100 rows of data samples. Therefore, it will be possible to distribute the amount of computation required for learning a machine learning model as needed or to perform analysis of the learning results for each partial table data set.

In addition, the learning server 1000 according to the present invention allows the user terminal to select the k_1th partial table data set corresponding to each of the k_1th partial table data set to the k_mth table data set used for learning the kth machine learning model. It may be possible to display the k_1th partial learning state to the k_mth partial learning state included in the kth learning state generated for the kth machine learning model, respectively, on the to k_mth partial table data set selection UI.

That is, as shown in FIG. 4, the learning server 1000 causes the user terminal to enter a learning model code (Model Name), partial data set name (Dataset Nm), and data count (Data) as the k_1th partial learning state to the k_mth partial learning state, respectively. k_1 part containing at least one of Count (e.g., number of table data rows), running start time (Start Time), running end time (End Time), running accuracy evaluation value (Accuracy, F1 Score, etc.), and whether running is completed (Status) The k_mth part of the table data set selection UI may be supported for display on the table data set selection UI.

Meanwhile, the present invention is not limited to this, and as shown in FIG. 4, the second area 220 is divided into a running COMPLETION column, a running PROCESSING column, and a running column according to the running progress status for each partial table data set. Divided by the IN QUEUE column, the user terminal can select the k_1th partial table data set selection UI to the k_mth partial table data set selection UI corresponding to each of the k_1th partial table data set to the k_mth table data set. It can be supported to display in the corresponding column.

Next, Figures 5 and 6 show the k_ith partial table data corresponding to the k_ith partial table data set among the k_1th partial table data set selection UI to the k_mth partial table data set selection UI according to an embodiment of the present invention. When the set selection UI is selected, table data analysis information for the first to pth table row data included in the k_i partial table data set in the XAI tab 203 corresponding to the k_i partial table data set selection UI. It schematically shows how it is displayed in the third area 230 on the screen of the user terminal.

According to FIG. 5, when the k_ith partial table data set selection UI corresponding to the k_ith partial table data set is selected among the k_1th partial table data set selection UI to the k_mth partial table data set selection UI, the learning server 1000 A, based on explainable artificial intelligence, first table row data to p-th table row data included in the k_i partial table data set in the XAI tab 203 corresponding to the k_i partial table data set selection UI, first First classification label data to p-th classification label data (quality) corresponding to each of the table row data to the p-th table row data, and first classification prediction data to the p-th corresponding to each of the first table row data to the p-th table row data. p classification prediction data (inference_res), as table data analysis information, may be supported to be displayed in a third area on the screen of the user terminal.

In addition, when the k_i th partial table data set selection UI is selected, the learning server 1000 causes the user terminal to use the first table row data to the j th table row data for each of the first table row data to the p th table row data. may be supported to be displayed in a third area on the screen of the user terminal. Here, j is an integer greater than or equal to 1.

Specifically, referring to FIG. 6, for the selected k_i partial table data set selection UI 221, each table row data 231 and the first table row data 231 corresponding to the XAI tab corresponding thereto. Table row data to jth table row data, classification label data (quality; 232), and classification prediction data (inference_res; 233) may be displayed in the third area of the screen of the user terminal.

Here, the classification label data for each table row data is the correct answer data for the task (e.g., classification) that the machine learning model performs on the table row data, and the corresponding classification prediction data for each table row data is It may be the result of a task performed by a machine learning model on each table row data (for example, a classification result). In the case of the example disclosed in the drawing of the present invention, for the wine quality data set (Wine Quality Dataset) input as a table data set, each wine corresponding to each table row data is classified as white wine (0) or red wine (1) It shows as an example a machine learning model that performs a classification task. Accordingly, the classification label data represents the actual type of wine, and the classification prediction data represents the type of wine predicted by the machine learning model for each input table row data. The type is displayed. In addition, as table row data for measuring wine quality used to classify wine in each table row data, the present invention includes acetic acid concentration (volatile_acidity), density (density), acidity (pH), alcohol concentration (alcohol), etc. are listed and displayed.

Accordingly, referring to FIGS. 5 and 6, when the k_i-th partial table data set selection UI is selected, the learning server 1000 converts the first classification label data to the p-th classification label data into the first classification prediction data to the p-th classification label data. Compared to the classification prediction data, for specific table row data, which is each of the first table row data to the pth table row data, the user terminal is prompted to determine whether the specific classification label data corresponding to the specific table row data matches the specific classification prediction data. In this case, the specific classification label data and the specific classification prediction data are displayed as a first visual effect, and if the specific classification label data corresponding to the specific table row data does not match the specific classification prediction data, the specific classification label data and the specific classification prediction data are displayed as a first visual effect. It may be supported to display data with a second visual effect that is different from the first visual effect. For example, according to the embodiment of FIGS. 5 and 7, when specific classification label data corresponding to specific table row data matches specific classification prediction data, the specific classification label data and the specific classification prediction data are displayed as a first visual effect. It is emphasized with a colored underline, and when the specific classification label data corresponding to specific table row data does not match the specific classification prediction data, the specific classification label data and the specific classification prediction data are displayed in a different color from the first color as a second visual effect. It can be emphasized with a two-color underline, but the present invention is not limited to this.

Additionally, according to FIGS. 5 and 6, the learning server 1000 allows the user terminal to adjust the number of table row data, corresponding table row data, classification label data, and classification prediction data displayed on each page on the screen. You can provide a drop-down menu for adjusting the number of data rows on the screen so that when you click on it, the number of rows that can be changed is displayed.

For the k_i partial table data set selection UI selected as above, at least one table row data, table row data, classification label data, and classification prediction data corresponding to each table row data are displayed in the third area 230 of the screen of the user terminal. In the displayed state, when specific table row data is selected, the learning server 1000 may display additional information about the selected specific table row data in the fourth area of the screen of the user terminal.

Accordingly, Figure 7 shows the first table row included in the specific table row data when selecting specific table row data among the first to p-th table row data in the third area according to an embodiment of the present invention. It schematically shows how analysis information about data from the jth table row is displayed in the fourth area 240 on the screen of the user terminal.

First, when specific table row data is selected among the first to pth table row data, the learning server 1000 generates a plurality of classification classes to be predicted by the kth machine learning model based on explainable artificial intelligence. and a plurality of prediction probabilities 241 predicted by the kth machine learning model for each classification class may be displayed in the fourth area 240 on the screen of the user terminal. For example, in the example of FIG. 7, for data on a specific wine corresponding to the first table row data (i.e., selected specific table row data) included in the selected k_i partial table data set, the wine is classified as white wine. You can see that the predicted probability of classifying wine as 0.99 and the predicted probability of classifying it as red wine are displayed as 0.01.

Furthermore, when specific table row data is selected from among the first to p-th table row data, the learning server 1000 selects a plurality of table row data from among the first to j-th table row data based on explainable artificial intelligence. Table row data with high contribution to at least one of the classification classes are arranged in descending order to determine first to The first to xth contributions corresponding to can be additionally displayed as a graph 242 in the fourth area 240 on the screen of the user terminal. Here, x is an integer less than or equal to j.

For example, in the example of FIG. 7, for data on a specific wine corresponding to the first table row data (i.e., selected specific table row data), the top three that contributed the most to predicting the wine as a white wine As table row data, acetic acid concentration (volatile_acidity), density (density), and sulfate amount (sulphates) are determined as the first contributing table row data to the third contributing table row data, and each table row data is used to classify the wine as a white wine. The contribution to the inference was displayed as a horizontal bar graph for each.

For reference, just because specific table row data (e.g., data for a specific wine) is inferred into a specific classification class (e.g., white wine), there is no need to limit the first to x contributions to only the contribution to that specific classification class. does not exist. For example, in the case of the first to third contributing table row data corresponding to the second table row data shown in FIG. 7, one of them, the second contributing table row data, is the amount of sulfate in the second table row data. The contribution to predicting a specific wine as a red wine is displayed in a horizontal bar graph.

Meanwhile, in addition to the prediction probability 241 and the graph 242 for contribution, the learning server 1000, when specific table row data is selected among the first table row data to the p th table row data, based on explainable artificial intelligence, Preprocessing is performed on each of the first table row data to the jth table row data to generate the first table row preprocessing data to the jth table row preprocessing data 243 and added to the fourth area 240 on the screen of the user terminal. It can be supported to be displayed as .

Accordingly, the first contribution to the xth contribution may be generated with reference to the first table row preprocessing data to the jth table row preprocessing data. For example, looking at the first table row data in FIG. 7, it can be seen that the acetic acid concentration (volatile_acidity) before preprocessing is 0.7, but the preprocessing data value obtained by performing preprocessing is 0.40, and accordingly, the acetic acid It can be seen that the contribution to classification class prediction of concentration was calculated according to the standard of volatile_acidity > 0.37, referring to the value after preprocessing of 0.40. Looking at another example of the first table row data, you can see that the amount of sulfates before performing preprocessing is 0.56, but the preprocessing data value obtained by performing preprocessing is 0.14, and accordingly, the amount of sulfates is 0.14. It can be seen that the contribution to classification class prediction was calculated according to the standard of 0.13 < sulphates <= 0.16, referring to the value after preprocessing of 0.14.

Furthermore, the learning server 1000 visualizes the effect of changes in the first to j table row data displayed for each of the first to p-th table row data on the inference process of the machine learning model. For this purpose, it is possible to support the user terminal to display the increase/decrease in the table row data value and the corresponding change in prediction probability in a graph.

Accordingly, first, Figure 8 shows the first table row table of contents selection UI to j table row table of contents corresponding to each of the first to j table row data in the fourth area 240 according to an embodiment of the present invention. It schematically shows how a single item relationship graph 251 is displayed in the fifth area 250 on the screen of the user terminal by selecting one of the selection UIs 244.

Specifically, according to FIG. 8, the learning server 1000 first causes the user terminal to display the first to jth table row data for each of the first to pth table row data. It may be supported to display the corresponding first table row table of contents selection UI to j table row table of contents selection UI 244 in the fourth area on the screen of the user terminal.

Next, the learning server 1000 causes the user terminal to select the first specific table row table of contents selection UI 244_1 among the first table row table of contents selection UI to the jth table row table of contents selection UI. It may be supported to create a first graph creation option UI (245_1) in the table of contents selection UI.

Then, when the first graph creation option UI (245_1) is selected, the learning server 1000 selects first specific table row data corresponding to the first specific table row table of contents selection UI (244_1) based on explainable artificial intelligence. For , it can be supported to display in the fifth area 250 on the screen of the user terminal as a specific single item relationship graph 251 showing the relationship of contribution to each classification class according to the increase or decrease in the data value of the first specific table row. .

Looking at the example of FIG. 8, the acetic acid concentration (volatile_acidity) is selected as the first specific table row table of contents selection UI (244_1) among the plurality of table of contents selection UIs corresponding to the plurality of table row data for wine quality measurement used to classify wine. It can be seen that the table row table of contents selection UI for ) has been selected, and accordingly, the learning server 1000 causes the user terminal to create a specific single item relationship graph (251) showing the relationship of contribution to each classification class according to the increase or decrease in acetic acid concentration. ) can be generated following the first specific table row table of contents selection UI (244_1).

Accordingly, in response to the selection of the first graph creation option UI 245_1, the learning server 1000 predicts the change in acetic acid concentration (x-axis) and the resulting wine classification class based on explainable artificial intelligence. A specific single item relationship graph 251 showing a change in value (y-axis) may be displayed in the fifth area 250 on the screen of the user terminal. Looking at the example in Figure 8, when acetic acid concentration is viewed as a single item for measuring wine quality, it can be seen that as the acetic acid concentration increases, the probability of predicting a white wine increases.

Next, Figure 9 shows a first table row table of contents selection UI to j table row table of contents selection corresponding to each of the first to j table row data in the fourth area 240 according to an embodiment of the present invention. It schematically shows how a multi-item relationship graph 252 is displayed in the fifth area 250 on the screen of the user terminal by selecting two of the UIs.

Specifically, according to FIG. 9, the learning server 1000 first causes the user terminal to display the first to jth table row data for each of the first to pth table row data. It may be supported to display the corresponding first table row table of contents selection UI to j table row table of contents selection UI 244 in the fourth area on the screen of the user terminal.

Subsequently, the learning server 1000 causes the user terminal to select a second specific table row table of contents selection UI along with the first specific table row table of contents selection UI 244_1 among the first table row table of contents selection UI to the jth table row table of contents selection UI. When (244_2) is selected, the creation of one of a single item relationship graph or a multiple item relationship graph for at least one of the first specific table row table of contents selection UI 244_1 and the second specific table row table of contents selection UI 244_2 is selected. It may support creating a second graph creation option UI (245_2) that can be used.

Then, when a single item relationship graph is selected through the second graph creation option UI 245_2, the learning server 1000, based on explainable artificial intelligence, causes the user terminal to select the first specific table row table of contents UI ( For the first specific table row data corresponding to 244_1) or the second specific table row data corresponding to the second specific table row table of contents selection UI 244_2, each classification class according to the increase or decrease in the first specific table row data value. A screen of the user terminal using a first single item relationship graph showing the relationship of contributions to each other or a second single item relationship graph showing the relationship of contributions to each classification class according to an increase or decrease in the data value of a second specific table row as a single item relationship graph. It can support display in the fifth area 250 of the image.

Here, the method and form of generating the first single item relationship graph or the second single item relationship graph may be similar to the method and form of generating the specific single item relationship graph described in FIG. 8. For example, as shown in Figure 9, when acetic acid concentration (volatile_acidity) is selected as the first specific table row data value and density (density) is selected as the second specific table row data value, the first single item relationship graph includes the change in acetic acid concentration and Accordingly, a single item relationship graph showing a change in the predicted value for the wine classification class can be generated, and as a second single item relationship graph, a single item relationship graph showing the change in density and the corresponding change in the predicted value for the wine classification class can be generated. A graph may be created.

Meanwhile, when a multiple-item relationship graph is selected through the second graph creation option UI (245_2), the learning server 1000, based on explainable artificial intelligence, causes the user terminal to generate first specific table row data and second specific table row data. For table row data, a relationship graph showing the relationship of contribution to each classification class according to the increase or decrease of the first specific table row data value and the second specific table row data value is a multi-item relationship graph, and is displayed as a fifth item on the screen of the user terminal. Display in area 250 may be supported.

Looking at an example of this in FIG. 9, the acetic acid concentration as the first specific table row table of contents selection UI (244_1) among the plurality of table of contents selection UIs corresponding to the plurality of table row data for wine quality measurement used to classify wine. It can be seen that the table row table of contents selection UI for (volatile_acidity) is selected, and the table row table of contents selection UI for density (density) is selected as the second specific table row table of contents selection UI 244_2. In relation to this, Figure 9 shows, as an example, for the second specific table row table of contents selection UI 244_2 where the learning server 1000 is selected, the user terminal selects one of a single item relationship graph or a multiple item relationship graph for density. It can be seen that the second graph creation option UI (245_2) from which creation can be selected is supported to be displayed. Although not shown in FIG. 9, the learning server 1000 creates a single item relationship graph or multiple item relationship graph for acetic acid concentration as shown in FIG. 9 for the first specific table row table of contents selection UI 244_1 corresponding to acetic acid concentration. It may be possible to display a second graph creation option UI 245_2 that can select the creation of one of the item relationship graphs.

Here, in the example of FIG. 9, in response to the selection of a multi-item relationship graph in the second graph creation option UI 245_2, the learning server 1000 changes the acetic acid concentration (x-axis) based on explainable artificial intelligence. And the change in density (y-axis) and the resulting change in predicted value for the wine classification class (contour line) may be supported to be displayed in the fifth area 250 on the screen of the user terminal.

Looking at an example of the multi-item relationship graph 252 of FIG. 9, a bar graph is additionally displayed on the right side of the relationship graph, so that the closer the color on the graph is to the first color (e.g., yellow), the predicted value is 1 (i.e. red wine), and the closer the color on the graph is to the second color (e.g. purple), the closer the predicted value is to 0 (i.e. white wine). Based on this, the multi-item relationship graph 252 of FIG. 9 shows that the lower left area where the predicted value approaches 1 (i.e., the area where the number on the contour line increases) indicates that the lower the acetic acid concentration and density, the redder the wine to be analyzed. It shows that it is closer to wine, and the area in the upper right where the predicted value approaches 0 (i.e., the area where the number on the contour line becomes lower) shows that the higher the acetic acid concentration and density, the closer the wine being analyzed is to white wine. there is.

As above, in the present invention, based on explainable artificial intelligence technology, the degree to which numerical changes in specific features of table data contribute to the model's prediction and the impact on the model's prediction for each individual data point are displayed through a user-friendly UI/ By visualizing it with UX, we can provide an interface that is helpful for intuitive understanding and analysis.

Embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and perform program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the invention and vice versa.

In the above, the present invention has been described with specific details such as specific components and limited embodiments and drawings, but this is only provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , a person skilled in the art to which the present invention pertains can make various modifications and variations from this description.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all modifications equivalent to or equivalent to the scope of the claims fall within the scope of the spirit of the present invention. They will say they do it.

Claims (18)

In a method for providing table data analysis information based on explainable artificial intelligence (XAI),
(a) Each of the first table data set to the n-th table data set for verifying each of the first machine learning model to the n-th machine learning model is input to each of the first machine learning model to the n-th machine learning model, thereby In a state in which a first learning state to an n-th learning state corresponding to each of the first table data set to the n-th table data set is created, the learning server causes the user terminal to perform the first machine learning Supporting display of a first machine learning model selection UI to an nth machine learning model selection UI corresponding to each of the models to the nth machine learning model in a first area on the screen of the user terminal;
(b) Among the first machine learning model selection UI and the nth machine learning model selection UI, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected, the learning server is sent to the user terminal. In the learning result verification tab corresponding to the kth machine learning model selection UI, each of the k_1th partial table data set to the k_mth table data set included in the kth table data set input to the kth machine learning model is displayed. Supporting display of a k_1th partial table data set selection UI to a k_mth partial table data set selection UI in a second area on the screen of the user terminal; and
(c) When the k_ith partial table data set selection UI corresponding to the k_ith partial table data set is selected among the k_1th partial table data set selection UI and the k_mth partial table data set selection UI, the learning server, Based on the explainable artificial intelligence, first table row data to p-th table row data included in the k_i th partial table data set in the First classification label data to p-th classification label data corresponding to each of the row data to the p-th table row data, and first classification prediction data to p-th corresponding to each of the first table row data to the p-th table row data. supporting display of classification prediction data as the table data analysis information in a third area on the screen of the user terminal;
Including,
In step (c) above,
When the k_i th partial table data set selection UI is selected, the learning server causes the user terminal to use first table row data to j th table row data for each of the first table row data to the p th table row data. Supports display in the third area on the screen of the user terminal,
When specific table row data is selected from the first table row data to the p-th table row data, the learning server generates a plurality of classification classes to be predicted by the k-th machine learning model based on the explainable artificial intelligence. and a plurality of prediction probabilities predicted by the kth machine learning model for each classification class are supported to be displayed in the fourth area on the screen of the user terminal,
When the specific table row data is selected among the first table row data to the p-th table row data, the learning server operates based on the explainable artificial intelligence, (i) the first table row data to the j-th table row data. Among the table row data, table row data with a high contribution to at least one of the plurality of classification classes are arranged in descending order to determine first to x-th contributing table row data, where x is less than j. or the same integer, and (ii) adding the first contribution to the xth contribution corresponding to the first contribution table row data to the xth contribution table row data to the fourth area on the screen of the user terminal as a graph. A method characterized in that it supports display as. delete delete According to paragraph 1,
When the specific table row data is selected among the first table row data to the p-th table row data, the learning server operates based on the explainable artificial intelligence, (i) the first table row data to the j-th table row data. Performing preprocessing on each table row data to generate first table row preprocessing data to jth table row preprocessing data to be additionally displayed in the fourth area on the screen of the user terminal, (ii) the first table row preprocessing data A method characterized in that the first contribution to the xth contribution are calculated with reference to the first table row preprocessing data to the jth table row preprocessing data. In a method for providing table data analysis information based on explainable artificial intelligence (XAI),
(a) Each of the first table data set to the n-th table data set for verifying each of the first machine learning model to the n-th machine learning model is input to each of the first machine learning model to the n-th machine learning model, thereby In a state in which a first learning state to an n-th learning state corresponding to each of the first table data set to the n-th table data set is created, the learning server causes the user terminal to perform the first machine learning Supporting display of a first machine learning model selection UI to an nth machine learning model selection UI corresponding to each of the models to the nth machine learning model in a first area on the screen of the user terminal;
(b) Among the first machine learning model selection UI and the nth machine learning model selection UI, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected, the learning server is sent to the user terminal. In the learning result verification tab corresponding to the kth machine learning model selection UI, each of the k_1th partial table data set to the k_mth table data set included in the kth table data set input to the kth machine learning model is displayed. Supporting display of a k_1th partial table data set selection UI to a k_mth partial table data set selection UI in a second area on the screen of the user terminal; and
(c) When the k_ith partial table data set selection UI corresponding to the k_ith partial table data set is selected among the k_1th partial table data set selection UI and the k_mth partial table data set selection UI, the learning server, Based on the explainable artificial intelligence, first table row data to p-th table row data included in the k_i th partial table data set in the First classification label data to p-th classification label data corresponding to each of the row data to the p-th table row data, and first classification prediction data to p-th corresponding to each of the first table row data to the p-th table row data. supporting display of classification prediction data as the table data analysis information in a third area on the screen of the user terminal;
Including,
In step (c) above,
When the k_i th partial table data set selection UI is selected, the learning server causes the user terminal to use first table row data to j th table row data for each of the first table row data to the p th table row data. Supports display in the third area on the screen of the user terminal,
When specific table row data is selected from the first table row data to the p-th table row data, the learning server generates a plurality of classification classes to be predicted by the k-th machine learning model based on the explainable artificial intelligence. and a plurality of prediction probabilities predicted by the kth machine learning model for each classification class are supported to be displayed in the fourth area on the screen of the user terminal,
The learning server causes the user terminal to display a first table row table of contents corresponding to each of the first table row data to the j th table row data displayed for each of the first table row data to the p-th table row data. Supports displaying a selection UI to the jth table row table of contents selection UI in the fourth area on the screen of the user terminal,
The learning server causes the user terminal to: (i) when a first specific table row table of contents selection UI is selected among the first table row table of contents selection UI to the jth table row table of contents selection UI, the first specific table row table of contents selection UI; Generate a first graph creation option UI in the selection UI, and (ii) when the first graph creation option UI is selected, based on the explainable artificial intelligence, a first graph creation option corresponding to the first specific table row table of contents selection UI For specific table row data, supporting display in the fifth area on the screen of the user terminal as a specific single item relationship graph showing the relationship of contribution to each classification class according to the increase or decrease in the first specific table row data value. A method characterized by: In a method for providing table data analysis information based on explainable artificial intelligence (XAI),
(a) Each of the first table data set to the n-th table data set for verifying each of the first machine learning model to the n-th machine learning model is input to each of the first machine learning model to the n-th machine learning model, thereby In a state in which a first learning state to an n-th learning state corresponding to each of the first table data set to the n-th table data set is created, the learning server causes the user terminal to perform the first machine learning Supporting display of a first machine learning model selection UI to an nth machine learning model selection UI corresponding to each of the models to the nth machine learning model in a first area on the screen of the user terminal;
(b) Among the first machine learning model selection UI and the nth machine learning model selection UI, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected, the learning server is sent to the user terminal. In the learning result verification tab corresponding to the kth machine learning model selection UI, each of the k_1th partial table data set to the k_mth table data set included in the kth table data set input to the kth machine learning model is displayed. Supporting display of a k_1th partial table data set selection UI to a k_mth partial table data set selection UI in a second area on the screen of the user terminal; and
(c) When the k_ith partial table data set selection UI corresponding to the k_ith partial table data set is selected among the k_1th partial table data set selection UI and the k_mth partial table data set selection UI, the learning server, Based on the explainable artificial intelligence, first table row data to p-th table row data included in the k_i th partial table data set in the First classification label data to p-th classification label data corresponding to each of the row data to the p-th table row data, and first classification prediction data to p-th corresponding to each of the first table row data to the p-th table row data. supporting display of classification prediction data as the table data analysis information in a third area on the screen of the user terminal;
Including,
In step (c) above,
When the k_i th partial table data set selection UI is selected, the learning server causes the user terminal to use first table row data to j th table row data for each of the first table row data to the p th table row data. Supports display in the third area on the screen of the user terminal,
When specific table row data is selected from the first table row data to the p-th table row data, the learning server generates a plurality of classification classes to be predicted by the k-th machine learning model based on the explainable artificial intelligence. and a plurality of prediction probabilities predicted by the kth machine learning model for each classification class are supported to be displayed in the fourth area on the screen of the user terminal,
The learning server causes the user terminal to display a first table row table of contents corresponding to each of the first table row data to the j th table row data displayed for each of the first table row data to the p-th table row data. Supports displaying a selection UI to the jth table row table of contents selection UI in the fourth area on the screen of the user terminal,
The learning server causes the user terminal to select a second specific table row table of contents selection UI together with a first specific table row table of contents selection UI among the first table row table of contents selection UI and the jth table row table of contents selection UI, To generate a second graph creation option UI that can select the creation of one of a single item relationship graph or a multiple item relationship graph for at least one of the first specific table row table of contents selection UI and the second specific table row table of contents selection UI. Support,
The learning server, based on the explainable artificial intelligence, causes the user terminal to: (i) When the single item relationship graph is selected through the second graph creation option UI, the first specific table row table of contents selection UI; For the corresponding first specific table row data or the second specific table row data corresponding to the second specific table row table of contents selection UI, the contribution to each classification class according to the increase or decrease in the first specific table row data value A first single item relationship graph showing the relationship or a second single item relationship graph showing the relationship of contribution to each classification class according to the increase or decrease in the data value of the second specific table row is used as the single item relationship graph of the user terminal. Supports display in a fifth area on the screen, and (ii) when the multi-item relationship graph is selected through the second graph creation option UI, the first specific table row data and the second specific table row data , A relationship graph showing the relationship of contribution to each classification class according to an increase or decrease in the data value of the first specific table row and the data value of the second specific table row is displayed as the multi-item relationship graph on the screen of the user terminal. A method characterized by supporting display in a fifth area. According to paragraph 1,
In step (c) above,
When the k_i-th partial table data set selection UI is selected, the learning server compares the first classification label data to the p-th classification label data with the first classification prediction data to the p-th classification prediction data, For specific table row data, which is each of the 1st table row data to the pth table row data, the user terminal (i) when specific classification label data corresponding to the specific table row data matches specific classification prediction data, The specific classification label data and the specific classification prediction data are displayed as a first visual effect, and (ii) when the specific classification label data corresponding to the specific table row data does not match the specific classification prediction data, the specific classification label data is displayed as a first visual effect. A method characterized by displaying classification label data and the specific classification prediction data with a second visual effect different from the first visual effect. According to paragraph 1,
In step (b) above,
The learning server causes the user terminal to select the k_1th partial table data set selection UI corresponding to each of the k_1th partial table data set to the k_mth table data set used for learning the kth machine learning model. A method characterized in that it supports displaying the k_1th partial learning state to the k_mth partial learning state included in the kth learning state generated for the kth machine learning model, respectively, on the k_m partial table data set selection UI. According to clause 8,
The learning server allows the user terminal to use the k_1th partial running state to the k_mth partial running state, respectively, with a running model code, partial data set name, data count, running start time, running end time, running accuracy evaluation value, and running A method characterized in that it supports displaying at least one of completion status on the k_1th partial table data set selection UI or the k_mth partial table data set selection UI. In a learning server for providing table data analysis information based on explainable artificial intelligence (XAI),
at least one memory storing instructions; and
At least one processor configured to execute the instructions,
The processor is configured to: (1) each of a first table data set to an n-th table data set for verifying each of the first machine learning model to the n-th machine learning model; In a state in which a first learning state to an n-th learning state corresponding to each of the first table data set to the n-th table data set is created, the user terminal is prompted to perform the first machine learning A process for supporting display of a first machine learning model selection UI to an n-th machine learning model selection UI corresponding to each of the model to the nth machine learning model in a first area on the screen of the user terminal, (2) the first Among the machine learning model selection UI and the nth machine learning model selection UI, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected, the user terminal responds to the kth machine learning model selection UI. In the running result verification tab, select the k_1th partial table data set corresponding to each of the k_1th partial table data set to the k_mth table data set included in the kth table data set input to the kth machine learning model UI to k_m A process for supporting display of a partial table data set selection UI in a second area on the screen of the user terminal, and (3) the k_i among the k_1th partial table data set selection UI to the k_mth partial table data set selection UI, When the k_ith partial table data set selection UI corresponding to the partial table data set is selected, based on the explainable artificial intelligence, the k_ith partial table data set is selected in the XAI tab corresponding to the k_ith partial table data set selection UI. 1st table row data to pth table row data included in , first classification label data to pth classification label data corresponding to each of the first table row data to pth table row data, and the first table row data. Performing a process to support displaying first classification prediction data to pth classification prediction data corresponding to each of the pth table row data, as the table data analysis information, in a third area on the screen of the user terminal, ,
In process (3) above,
When the k_i th partial table data set selection UI is selected, the processor causes the user terminal to select first table row data to j th table row data for each of the first table row data to the p th table row data. Supports display in the third area on the screen of the user terminal,
When specific table row data is selected from the first table row data to the p-th table row data, the processor selects a plurality of classification classes to be predicted by the k-th machine learning model based on the explainable artificial intelligence. And supporting a plurality of prediction probabilities predicted by the kth machine learning model for each classification class to be displayed in a fourth area on the screen of the user terminal,
When the specific table row data is selected among the first table row data to the p-th table row data, the processor, based on the explainable artificial intelligence, (i) the first table row data to the j-th table Among the column data, table column data with a high contribution to at least one of the plurality of classification classes are arranged in descending order to determine first to xth contributing table column data, where x is less than j or are the same integer, and (ii) the first contribution to the xth contribution corresponding to the first contribution table row data to the A learning server that supports display. delete delete According to clause 10,
When the specific table row data is selected among the first table row data to the p-th table row data, the processor, based on the explainable artificial intelligence, (i) the first table row data to the j-th table Performing preprocessing on each row data to generate first table row preprocessing data to jth table row preprocessing data to be additionally displayed in the fourth area on the screen of the user terminal, (ii) the first table row preprocessing data A learning server that calculates the first contribution to the xth contribution by referring to table row preprocessing data or the jth table row preprocessing data. In a learning server for providing table data analysis information based on explainable artificial intelligence (XAI),
at least one memory storing instructions; and
At least one processor configured to execute the instructions,
The processor is configured to: (1) each of a first table data set to an n-th table data set for verifying each of the first machine learning model to the n-th machine learning model; In a state in which a first learning state to an n-th learning state corresponding to each of the first table data set to the n-th table data set is created, the user terminal is prompted to perform the first machine learning A process for supporting display of a first machine learning model selection UI to an n-th machine learning model selection UI corresponding to each of the model to the nth machine learning model in a first area on the screen of the user terminal, (2) the first Among the machine learning model selection UI and the nth machine learning model selection UI, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected, the user terminal responds to the kth machine learning model selection UI. In the running result verification tab, select the k_1th partial table data set corresponding to each of the k_1th partial table data set to the k_mth table data set included in the kth table data set input to the kth machine learning model UI to k_m A process for supporting display of a partial table data set selection UI in a second area on the screen of the user terminal, and (3) the k_i among the k_1th partial table data set selection UI to the k_mth partial table data set selection UI, When the k_ith partial table data set selection UI corresponding to the partial table data set is selected, based on the explainable artificial intelligence, the k_ith partial table data set is selected in the XAI tab corresponding to the k_ith partial table data set selection UI. 1st table row data to pth table row data included in , first classification label data to pth classification label data corresponding to each of the first table row data to pth table row data, and the first table row data. Performing a process to support displaying first classification prediction data to pth classification prediction data corresponding to each of the pth table row data, as the table data analysis information, in a third area on the screen of the user terminal, ,
In process (3) above,
When the k_i th partial table data set selection UI is selected, the processor causes the user terminal to select first table row data to j th table row data for each of the first table row data to the p th table row data. Supports display in the third area on the screen of the user terminal,
When specific table row data is selected from the first table row data to the p-th table row data, the processor selects a plurality of classification classes to be predicted by the k-th machine learning model based on the explainable artificial intelligence. And supporting a plurality of prediction probabilities predicted by the kth machine learning model for each classification class to be displayed in a fourth area on the screen of the user terminal,
The processor causes the user terminal to select a first table row table of contents corresponding to each of the first table row data to the j th table row data displayed for each of the first table row data to the p-th table row data. Supports displaying UI to jth table row table of contents selection UI in the fourth area on the screen of the user terminal,
The processor causes the user terminal to: (i) select the first specific table row table of contents selection UI among the first table row table of contents selection UI to the jth table row table of contents selection UI; Generate a first graph creation option UI in the UI, and (ii) when the first graph creation option UI is selected, based on the explainable artificial intelligence, a first specific table row table of contents selection UI For table row data, supporting display in the fifth area on the screen of the user terminal as a specific single item relationship graph showing the relationship of contribution to each classification class according to the increase or decrease of the first specific table row data value. Featured learning server. In a learning server for providing table data analysis information based on explainable artificial intelligence (XAI),
at least one memory storing instructions; and
At least one processor configured to execute the instructions,
The processor is configured to: (1) each of a first table data set to an n-th table data set for verifying each of the first machine learning model to the n-th machine learning model; In a state in which a first learning state to an n-th learning state corresponding to each of the first table data set to the n-th table data set is created, the user terminal is prompted to perform the first machine learning A process for supporting display of a first machine learning model selection UI to an n-th machine learning model selection UI corresponding to each of the model to the nth machine learning model in a first area on the screen of the user terminal, (2) the first Among the machine learning model selection UI and the nth machine learning model selection UI, when the kth machine learning model selection UI corresponding to the kth machine learning model is selected, the user terminal responds to the kth machine learning model selection UI. In the running result verification tab, select the k_1th partial table data set corresponding to each of the k_1th partial table data set to the k_mth table data set included in the kth table data set input to the kth machine learning model UI to k_m A process for supporting display of a partial table data set selection UI in a second area on the screen of the user terminal, and (3) the k_i among the k_1th partial table data set selection UI to the k_mth partial table data set selection UI, When the k_ith partial table data set selection UI corresponding to the partial table data set is selected, based on the explainable artificial intelligence, the k_ith partial table data set is selected in the XAI tab corresponding to the k_ith partial table data set selection UI. 1st table row data to pth table row data included in , first classification label data to pth classification label data corresponding to each of the first table row data to pth table row data, and the first table row data. Performing a process to support displaying first classification prediction data to pth classification prediction data corresponding to each of the pth table row data, as the table data analysis information, in a third area on the screen of the user terminal, ,
In process (3) above,
When the k_i th partial table data set selection UI is selected, the processor causes the user terminal to select first table row data to j th table row data for each of the first table row data to the p th table row data. Supports display in the third area on the screen of the user terminal,
When specific table row data is selected from the first table row data to the p-th table row data, the processor selects a plurality of classification classes to be predicted by the k-th machine learning model based on the explainable artificial intelligence. And supporting a plurality of prediction probabilities predicted by the kth machine learning model for each classification class to be displayed in a fourth area on the screen of the user terminal,
The processor causes the user terminal to select a first table row table of contents corresponding to each of the first table row data to the j th table row data displayed for each of the first table row data to the p-th table row data. Supports displaying UI to jth table row table of contents selection UI in the fourth area on the screen of the user terminal,
The processor causes the user terminal to select a second specific table row table of contents selection UI together with a first specific table row table of contents selection UI among the first table row table of contents selection UI and the j-th table row table of contents selection UI, Supports creating a second graph creation option UI that can select the creation of either a single item relationship graph or a multiple item relationship graph for at least one of the first specific table row table of contents selection UI and the second specific table row table of contents selection UI. And
The processor, based on the explainable artificial intelligence, causes the user terminal to: (i) when the single item relationship graph is selected through the second graph creation option UI, correspond to the first specific table row table of contents selection UI; For the first specific table row data or the second specific table row data corresponding to the second specific table row table of contents selection UI, the contribution to each classification class according to the increase or decrease in the first specific table row data value A first single item relationship graph showing a relationship or a second single item relationship graph showing a relationship of contribution to each classification class according to an increase or decrease in a second specific table row data value is used as the single item relationship graph in the user terminal. Supports display in a fifth area on the screen, and (ii) when the multi-item relationship graph is selected through the second graph creation option UI, for the first specific table row data and the second specific table row data, A relationship graph showing the relationship between the contribution to each classification class according to an increase or decrease in the first specific table row data value and the second specific table row data value is the multi-item relationship graph, and is displayed as the first specific table row data value on the screen of the user terminal. A learning server that supports display in 5 areas. According to clause 10,
In process (3) above,
When the k_i-th partial table data set selection UI is selected, the processor compares the first classification label data to the p-th classification label data with the first classification prediction data to the p-th classification prediction data, For specific table row data, which is each of the table row data and the pth table row data, the user terminal (i) when specific classification label data corresponding to the specific table row data matches specific classification prediction data, Displaying specific classification label data and the specific classification prediction data as a first visual effect, and (ii) if the specific classification label data corresponding to the specific table row data does not match the specific classification prediction data, the specific classification A learning server that displays label data and the specific classification prediction data as a second visual effect different from the first visual effect. According to clause 10,
In process (2) above,
The processor causes the user terminal to select the k_1th partial table data set UI to the k_mth table data set corresponding to each of the k_1th partial table data set to the k_mth table data set used for learning the kth machine learning model. A learning server that supports displaying the k_1th partial learning state to the k_mth partial learning state included in the kth learning state generated for the kth machine learning model, respectively, on the partial table data set selection UI. According to clause 17,
The processor causes the user terminal to use the k_1th partial running state to the k_mth partial running state, respectively, with a learning model code, partial data set name, data count, running start time, running end time, running accuracy evaluation value, and running completion. A learning server that supports displaying at least one of the k_1th partial table data set selection UI and the k_mth partial table data set selection UI.