JP7186411B1 - Information processing system, information processing method and information processing program - Google Patents

Abstract

An information processing system, an information processing method, and an information processing program are provided for suppressing a learning model generation process from becoming a black box. In an information processing system in which an information processing device, a user terminal, and a database communicate with each other through an electric communication line, a control unit 23 of the information processing device comprising a communication unit, a storage unit, and a control unit is configured to: An input data reading unit 232 that reads the first input data, and reads the second input data that is generated by performing a predetermined conversion process on the first input data and that is used to generate the learning model by the learning device. Data display for displaying at least points in common between the input data reading unit 232 and the first input data and the second input data and points of difference between the first input data and the second input data in an overviewable manner. a portion 237; The display mode of the points of difference is different from the display mode of the common points. [Selection drawing] Fig. 4

Description

The present invention relates to an information processing system, an information processing method, and an information processing program.

As prior art, the following documents regarding learning devices can be cited.

Japanese Unexamined Patent Application Publication No. 2021-177428

By the way, if the input data is not suitable for input to the learning device, the input data may be subjected to conversion processing in accordance with the learning device. The conversion process may cause differences between the input data entered by the user and the input data used to generate the learning model. As a result, the learning model generation process may become a black box.

(1) According to one aspect of the present invention, an information processing system is provided. This information processing system includes a control unit. The controller is configured to perform the following steps. The first input data reading step reads the first input data. In the second input data reading step, second input data generated by performing a predetermined conversion process on the first input data is read. The second input data is data used by the learner to generate a learning model. In the data display step, at least points in common between the first input data and the second input data and points of difference between the first input data and the second input data are displayed in an overviewable manner. The display mode of the points of difference is different from the display mode of the common points.

According to such an information processing system, it is possible to prevent the learning model generation process from becoming a black box.

1 is a configuration diagram showing an information processing system 1; FIG. 2 is a block diagram showing the hardware configuration of the information processing device 2; FIG. 3 is a block diagram showing the hardware configuration of the user terminal 3; FIG. An example of a functional unit included in the control unit 23 is shown. 4 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1; FIG. It is an example of the reception window 4 displayed on the display unit 34 . It is an example of the data window 5 and the conversion processing window 6 displayed on the display unit 34 . FIG. 10 is a diagram showing an example of the conversion processing window 6 displayed on the display unit 34 in the case of the second display mode 6b; 3 is a diagram showing an example of a model information display window 7 displayed on a display unit 34; FIG. It is an example of the model search window 8 and the model comparison window 9 displayed on the display unit 34 .

Embodiments of the present invention will be described below with reference to the drawings. Various features shown in the embodiments shown below can be combined with each other.

By the way, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium (Non-Transitory Computer-Readable Medium), or may be downloaded from an external server. It may be provided as possible, or may be provided so that the program is activated on an external computer and the function is realized on the client terminal (so-called cloud computing).

Further, in the present embodiment, the term “unit” may include, for example, a combination of hardware resources implemented by circuits in a broad sense and software information processing that can be specifically realized by these hardware resources. . In addition, various information is handled in the present embodiment, and these information are, for example, physical values of signal values representing voltage and current, and signal values as binary bit aggregates composed of 0 or 1. It is represented by high and low, or quantum superposition (so-called quantum bit), and communication and operation can be performed on a circuit in a broad sense.

A circuit in a broad sense is a circuit implemented by appropriately combining at least circuits, circuits, processors, memories, and the like. Application Specific Integrated Circuits (ASICs), programmable logic devices (e.g., Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and field It includes a programmable gate array (Field Programmable Gate Array: FPGA).

1. Hardware configuration This section describes the hardware configuration.

<Information processing system 1>
FIG. 1 is a configuration diagram showing an information processing system 1. As shown in FIG. The information processing system 1 includes an information processing device 2, a user terminal 3, and a database DB1. The information processing device 2, the user terminal 3, and the database DB1 are configured to be able to communicate with each other through an electric communication line. In one embodiment, information handling system 1 is comprised of one or more devices or components. If, for example, the information processing system 1 consists only of the information processing device 2 , the information processing system 1 can be the information processing device 2 . These constituent elements are described below.

<Information processing device 2>
FIG. 2 is a block diagram showing the hardware configuration of the information processing device 2. As shown in FIG. The information processing device 2 includes a communication section 21 , a storage section 22 and a control section 23 , and these components are electrically connected via a communication bus 20 inside the information processing device 2 . Each component will be further described.

The communication unit 21 is preferably a wired type communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but wireless LAN network communication, mobile communication such as 3G/LTE/5G, BLUETOOTH (registered trademark) Communication and the like may be included as desired. That is, it is more preferable to implement as a set of these communication means. That is, the information processing device 2 may communicate various information from the outside via the communication unit 21 and the network.

The storage unit 22 stores various information defined by the above description. For example, it can be used as a storage device such as a solid state drive (SSD) for storing various programs related to the information processing device 2 executed by the control unit 23, or as a temporary storage device related to program calculation. It can be implemented as a memory such as a Random Access Memory (RAM) that stores information (arguments, arrays, etc.) required for the . The storage unit 22 stores various programs, variables, etc. related to the information processing device 2 executed by the control unit 23 .

The control unit 23 processes and controls overall operations related to the information processing device 2 . The control unit 23 is, for example, a central processing unit (CPU) (not shown). The control unit 23 implements various functions related to the information processing device 2 by reading a predetermined program stored in the storage unit 22 . That is, information processing by software stored in the storage unit 22 can be specifically realized by the control unit 23 which is an example of hardware, and can be executed as each functional unit included in the control unit 23 . These are further detailed in the next section. Note that the control unit 23 is not limited to a single unit, and may be implemented to have a plurality of control units 23 for each function. A combination thereof may also be used.

<User terminal 3>
FIG. 3 is a block diagram showing the hardware configuration of the user terminal 3. As shown in FIG. The user terminal 3 includes a communication section 31 , a storage section 32 , a control section 33 , a display section 34 and an input section 35 . properly connected. Descriptions of the communication unit 31, the storage unit 32, and the control unit 33 are omitted because they are similar to the description of each unit in the information processing apparatus 2. FIG.

The display unit 34 may be included in the housing of the user terminal 3, or may be externally attached. The display unit 34 displays a screen of a graphical user interface (GUI) that can be operated by the user. For example, it is preferable to use a display device such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display according to the type of the user terminal 3 .

The input unit 35 may be included in the housing of the user terminal 3 or may be externally attached. For example, the input unit 35 may be integrated with the display unit 34 and implemented as a touch panel. With a touch panel, the user can input a tap operation, a swipe operation, or the like. Of course, a switch button, a mouse, a QWERTY keyboard, or the like may be employed instead of the touch panel. That is, the input unit 35 receives an operation input made by the user. The input is transferred as a command signal to the control unit 33 via the communication bus 30, and the control unit 33 can execute predetermined control and calculation as necessary.

<Database DB1>
Database DB1 stores external data D0. The external data D0 may be, for example, data accessible to the public or data accessible only to specific users. Also, the external data D0 may be data that can be accessed only by the user using the information processing system 1 . The database DB1 may be realized by a single storage device or may be realized by a plurality of storage devices. The contents represented by the external data D0 are arbitrary, such as hygiene observation results, climate observation results, statistical data, and the like.

2. Functional Configuration FIG. 4 shows an example of functional units provided in the control unit 23 . As shown in FIG. 4, the control unit 23 includes an input data receiving unit 231, an input data reading unit 232, a conversion processing identifying unit 233, a conversion processing presenting unit 234, an input data generating unit 235, and a display mode determining unit. A section 236 and a data display section 237 are provided.

The input data reception unit 231 receives input of the first input data D1. The first input data D<b>1 is data input to the information processing device 2 .

The input data reading unit 232 reads the input data D. FIG. The input data D includes first input data D1 and second input data D2, which will be described later.

The conversion process specifying unit 233 specifies a conversion process to be performed on the first input data D1.

The conversion processing presentation unit 234 causes the display unit 34 to present the conversion processing specified by the conversion processing specifying unit 233 .

The input data generator 235 generates the second input data D2 by performing conversion processing on the first input data D1. The second input data D2 can also be said to be the input data D obtained by converting the first input data D1.

The display mode determination unit 236 determines the display mode of the information on the input data D on the display unit 34 .

The data display unit 237 causes the display unit 34 to display information about the input data D in the display mode determined by the display mode determination unit 236 .

3. Details of Information Processing In this section, information processing executed in the information processing system 1 described above will be described. Note that the information processing may include any exception handling not shown in the activity diagram. Exception processing includes interruption of the information processing and omission of each processing. The selection or input performed in the information processing may be based on the user's operation, or may be automatically performed without depending on the user's operation.

FIG. 5 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. As shown in FIG. As shown in FIG. 5, the input data receiving unit 231 receives input of first input data D1 in activity A001. The processing executed in activity A001 corresponds to the first input data receiving step. In this embodiment, the user can input the first input data D1. The input of the input data D1 is not limited to that by the user. For example, the input of the input data D1 may be performed by any member configured to be able to communicate with the control unit 23, for example, the control unit 33.

The first input data D1 includes at least held data held by the user. Note that the first input data D1 may include data held by a person other than the user, such as data stored in the database DB1. The first input data D1 includes at least structured data. Structured data is data standardized to have a predetermined structure. Note that the first input data D1 may include data other than structured data, such as unstructured data or semi-structured data. Unstructured data is any form of data that does not have a standardized structure like structured data. Semi-structured data consists of a combination of unstructured data and tags that can identify the unstructured data. Formats of semi-structured data include, for example, graph type, key-value type, document type, and column type.

Next, the process proceeds to activity A002, and the input data reading unit 232 reads the first input data D1 received at activity A001.

Next, in activity A003, the control unit 23 identifies a plurality of learners ML according to the received first input data D1. The learning device ML uses the first input data D1 to generate a learning model M1. Learning model M1 produces at least one output y1 based on at least one input x1. The input x1 is also called an explanatory variable. The output y1 is also called an evaluation function or an evaluation parameter. At this time, the control unit 23 may accept the user's selection of the learning device ML identified in activity A002.

After accepting the selection of the learning device ML, the process proceeds to activity A004, and the control unit 23 accepts the selection of the analysis method. The analysis method may be predetermined, specified in activity A003, or specified according to the selected learner ML. Analytical techniques optionally include, for example, at least one of classification analysis, regression analysis, and time series analysis. Any algorithm such as supervised learning, unsupervised learning, or reinforcement learning can be used as the analysis method. The control unit 23 may receive selection of at least one of the plurality of analysis methods to be used for generating the learning model M1.

After receiving the selection of the analysis method, the process proceeds to activity A005, and the conversion process specifying unit 233 specifies the conversion process based on the first input data D1 and the learning device ML. Specifically, the conversion process specifying unit 233 specifies the conversion process based on the first input data D1 read in activity A002 and the learner ML specified or selected in activity A003. do. The conversion process specifying unit 233 specifies a conversion process according to the selected learning device ML. The control unit 23 may further specify conversion processing based on the specified or selected analysis method. For example, when the analysis method selected in activity A004 is the time series analysis, the conversion process specified by the control unit 23 is the conversion process of combining a plurality of inputs x1 representing time series into one, each data point If the time intervals are different, conversion processing that complements, deletes, or corrects data points so as to adjust the time intervals, calendar information, weather information, or demographic information as external data D0 as first input data D1 , and includes transformation processing for associating with the input x1 representing the time series.

The conversion process is a process of converting the first input data D1 input to the input data receiving unit 231 into the second input data D2 in a form that can be input to the specified learning device ML. The transformation process includes at least one of deleting data points included in the first input data D1 and adding data points to the first input data D1. Specifically, the conversion processing can include processing such as deletion of part of the first input data D1, complementation of missing values, deletion of outliers, and the like. The conversion processing may include arbitrary processing such as normalization of the first input data D1. The conversion process is also called data shaping.
The conversion process may include a combination process, a separation process, a correction, and the like of a plurality of inputs x1 included in the first input data D1. For example, if the first input data D1 has years, months, and days representing time series as different inputs x1, the conversion process may include combining these multiple inputs x1 into one input. . Further, in the conversion process, when a plurality of first input data D1 are received by a plurality of input data receiving units 231, the combining process of these first input data D1 may be performed. As another example, the conversion process may include additional processing of the input x1, addition of feature amounts, and the like. For example, the input data generator 235 may acquire arbitrary external data D0 from the database DB1 according to the first input data D1 and add the external data D0 to the first input data D1. The input data generation unit 235 may add feature amounts based on the added external data D0.
The conversion processing includes processing that can be automatically executed by the control unit 23 based on the contents of the first input data D1. Note that the conversion processing may include processing that can be executed based on designation by the user. Further, in this embodiment, the conversion processing includes automatic conversion processing specified by comparing the first input data D1 with predetermined conversion conditions. The conversion conditions include, for example, the capacity of the first input data D1, whether or not the variance of the statistical information of the first input data D1 is less than a threshold, and whether or not there is an outlier based on the statistical information of the first input data D1. is. Conversion conditions also suggest the need for conversion processing to be performed.
The conversion process also includes a process of converting the format of the first input data D1 input to the input data reception unit 231 into a format suitable for each of the specified learning devices ML. The format of the first input data D1 includes, for example, the name of the first input data D1, character code, line feed code, writing language, delimiter, and the like.
For example, when the character code of the input data D that can be input to the specified learning device ML is UTF-8, but the character code of the first input data D1 is Shift-JIS, the input data generation unit 235 executes conversion processing for converting the first input data D1 into second input data D2 whose character code is changed to UTF-8.
As another example, the conversion process may include a process of giving a name to each of the second input data D2 that can be input to the identified plurality of learners ML. At this time, it is preferable that the conversion process gives the second input data D2 a name by which the control unit 23 can uniquely identify the second input data D2. This facilitates management of the second input data D2. Note that the name may be assigned by changing the name of the first input data D1.
As another example, when the first input data D1 includes a BOM (Byte Order Mark), the conversion process may include removal of the BOM. This makes it easier to suppress unintended errors.
As another example, the conversion process may include a process of removing inputs x1 included in the first input data D1 that are not used to generate the learning model M1. The input data generation unit 235 may determine whether or not the input x1 is used to generate the learning model M1, for example, based on the format and distribution of the input x1. This reduces the volume of the second input data D2, thereby shortening the time required to generate the learning model M1. Note that specifying the conversion process may include specifying that there is no conversion process to be performed, for example, when there is no need to perform the conversion process.
The control unit 23 may further specify conversion processing based on the specified or selected analysis technique. For example, when the selected analysis method is time series analysis, the conversion process specified by the control unit 23 is a conversion process that combines a plurality of inputs x1 representing time series into one, and the time interval between each data point is if different, combining calendar information, weather information, or demographic information as external data D0 with the first input data D1; It includes conversion processing associated with the input x1 representing the time series.

The control unit 23 may specify the conditions under which the conversion process is performed, that is, the conversion conditions. For example, the control unit 23 may specify the conversion condition based on at least the first input data D1 and the learner ML specified or selected in the activity A003. For example, the control unit 23 identifies conversion conditions based on statistical information indicating the first input data D1. The statistical information about the first input data D1 includes, for example, data point distribution, mean value, variance, standard deviation, maximum value, minimum value, median value, mode value, re-likelihood, covariance, correlation coefficient, Including R2 value and so on. When the conversion process is removal of outliers, the control unit 23 determines that the data point is an outlier if the absolute value of the difference between the data point and the average value is twice the standard deviation or more. In this case, the conversion condition is that the absolute value of the difference between a certain data point and the average value is at least twice the standard deviation.

Next, the process proceeds to activity A006, and the conversion process presenting unit 234 generates visual information IFc1 configured so that at least one of the conversion processes specified in activity A005 can be selected. For convenience of explanation, the visual information IFc1 generated in the activity A006 is hereinafter referred to as the selected visual information IFc1. The selected visual information IFc1 is displayed on the display section 34. FIG. The user can select conversion processing by operating the visual selection information IFc1. At this time, the control unit 23 accepts the user's selection of the conversion process specified in activity A005. Any user interface such as a check box, a drum roll, or a pull-down can be adopted as a specific aspect of the visual selection information IFc1. Hereinafter, for convenience of explanation, the user interface is simply referred to as UI.

Next, the process proceeds to activity A007, and the input data generation unit 235 generates second input data D2 by performing conversion processing on the first input data D1. Specifically, the input data generator 235 performs the conversion process specified or selected in activity A005 on the first input data D1 read out in activity A002, thereby obtaining the second Generate input data D2. The second input data D2 is data used by the learner ML to generate the learning model M1. The second input data D2 can also be said to be input data D generated by performing a predetermined conversion process on the first input data D1.

Next, the process proceeds to activity A008, and the input data reading unit 232 reads the second input data. In this embodiment, the input data reading unit 232 reads the second input data D2 generated in activity A007. The second input data D2 read by the input data reading unit 232 in activity A008 is not limited to that generated in activity A007. For example, even if the second input data D2 is generated by a member different from the input data generation unit 235, for example, the control unit 33 of the user terminal 3, the second input data D2 generated by a device existing outside the information processing system 1 can be used. 2 input data D2.

Next, the process proceeds to activity A009, and the display mode determination unit 236 determines the display mode for displaying the information on the conversion process specified or selected in activity A005 on the display unit . The information about the conversion process includes, for example, specific contents of the conversion process, changes in the first input data D1 due to the conversion process, second input data D2 generated by the conversion process, and the like. Information about the conversion process includes common points 511 between the input data D1 and the second input data D2, and different points 512 between the first input data D1 and the second input data D2. In this embodiment, the display mode determining unit 236 determines at least one display mode of the common points 511 and the different points 512 according to the conversion process. Any display mode can be used as long as the common points 511 and the different points 512 can be distinguished. Therefore, the display mode of the different points 512 is different from the display mode of the common points 511 . In this display mode, for example, the common points 511 and the different points 512 can be distinguished by colors, outlines, marks, or the like. In this embodiment, the color corresponding to the point of difference 512 is different from the color corresponding to the point of commonality 511 .

Next, the process proceeds to activity A010, and the data display unit 237 causes the display unit 34 to display information regarding the conversion process. Specifically, the data display unit 237 causes the display unit 34 to display the information on the conversion process in the display mode determined in activity A009. Specifically, the data display unit 237 displays at least common points 511 between the first input data D1 and the second input data D2, and differences 512 between the first input data D1 and the second input data D2. , are displayed on the display unit 34 so that they can be viewed. For example, the data display unit 237 causes the display unit 34 to display the difference 512 in a color different from at least the common point 511 . This makes it easier for the user to intuitively recognize the contents of the conversion process performed on the first input data D1. It should be noted that the second input data D2 does not have to be actually generated at this stage, and may be expected to be generated based on the first input data D1 and conversion processing, for example. In this embodiment, the data display unit 237 causes the display unit 34 to visually display the distribution of at least one of the first input data D1 and the second input data D2. The distribution is displayed visually using any method, such as a histogram, line graph, pie chart, or bubble chart. The points of difference 512 and the points of commonality 511 are displayed at least in the distribution. The data display unit 237 may cause the display unit 34 to display the conditions for conversion processing in a recognizable manner. This suppresses conversion processing from becoming a black box. Furthermore, the data display unit 237 may display the conditions for the conversion process in a recognizable manner based on at least the first input data D1 and the specified learning device ML. In the present embodiment, the conditions under which conversion processing is performed correspond to the conversion conditions described above.

Next, the process proceeds to activity A011, and the control unit 23 inputs the second input data D2 to the learner ML specified or selected in activity A003. When there are a plurality of learning devices ML, the control unit 23 inputs the second input data D2 to each of the plurality of learning devices ML. Thereby, each learning device ML generates a learning model M1 based on the first input data D1. Specifically, the learning device ML uses the second input data D2 to generate the learning model M1. At this time, the control unit 23 may designate a learning algorithm in the learning device ML based on the selected analysis method. Note that the learning device ML may be stored in an arbitrary member included in the information processing system 1 or may be stored in an external device that can communicate with the information processing system 1 via an electric communication line. Note that the illustration of the external device is omitted.

Next, the process proceeds to activity A012, and the control unit 23 acquires the learning model M1 generated from each of the learning devices ML.

Next, the process proceeds to activity A013, and the control unit 23 causes the display unit 34 to display the model information IF1. Specifically, based on the first input data D1, the control unit 23 causes the display unit 34 to display the model information IF1 in a manner in which each learning model can be compared. For example, the control unit 23 causes the display unit 34 to display the model information IF1 generated using the selected learner ML among the specified learners ML.

The model information IF1 is information about the learning model M1. For example, the model information IF1 can include the name and capacity of the first input data D1 used to generate the learning model M1, the date and time when the learning model M1 was generated, and the like. The name of the first input data D1 is, for example, the file name of the first input data D1. In this embodiment, the model information IF1 includes at least accuracy information regarding the prediction accuracy of the learning model M1. When the analysis method is regression analysis, the accuracy information is, for example, the coefficient of determination (R2 score), the mean squared error (MSE), the mean absolute error (MAE), the mean squared deviation (RMSE: Root Mean Squared Error (RMSE), Root Mean Squared Error (RMSE), Logarithmic Mean Squared Logarithmic Error (RMLSE), Mean Absolute Percentage Error (MAPE). When the analysis method is classification analysis, the accuracy information includes accuracy, recall, precision, specificity, F value (F-measure), weighted F value, Matthews Including indicators such as the correlation coefficient (MCC: Matthews Correlation Coefficient), the kappa coefficient (Kappa), the log loss (Logloss), AUC: Area Under the Curve, PR-AUC: Area Under the Precision-Recall curve. The accuracy information is not limited to that used for binary classification, and may be used for other class classification greater than two. When the analysis method is time series analysis, the accuracy information is, for example, Coefficient of Variation, Dynamic Time Warping MAE, MAPE, Symmetric Mean Absolute Percentage Error (SMAPE: Symmetric MAPE), Weighted SMAPE, Mean Absolute Scaled Error (MASE), MARRE: Mean Absolute Ranged Relative Error, Overall Percentage error, R2, rho-risk RMSLE. The accuracy information may include statistical values of each parameter, such as MAE median and MAE mean. An index for one analysis method may be used as an index for another analysis method.

The control unit 23 causes the display unit 34 to display the difference between the first input data D1 and the second input data D2 in a recognizable manner. For example, the control unit 23 displays the difference 512 between the first input data D1 and the second input data D2 in a manner different from the common point 511 between the first input data D1 and the second input data D2. Let For example, the control unit 23 displays both in a mode in which at least one of color, shape, and pattern is different. As another example, the control unit 23 may display a predetermined mark such as an arrow in association with the difference 512 between the first input data D1 and the second input data D2.

Based on the first input data D1, the control unit 23 causes the display unit 34 to display the model information IF1 related to the learning model M1 generated by the specified learning device ML in a manner that enables comparison for each learning model M1. The control unit 23 of the present embodiment displays at least the accuracy information for each generated learning model M1 so that it can be compared. For example, the control unit 23 causes the display unit 34 to display the model information IF1 related to each learning model M1 in a viewable manner.

4. Example of Content Displayed on Display Unit 34 In this section, an example of content displayed on the display unit 34 based on the above information processing will be described. In this embodiment, a case where the user predicts the sales price of a product using the first input data D1 will be described. A reception window 4, a data window 5, a conversion processing window 6, a model information display window 7, a model search window 8, and a model comparison window 9 can be displayed on the display unit 34 of the present embodiment.

4-1. An example of reception window 4 First, the details of the reception window 4 will be described. FIG. 6 is an example of the reception window 4 displayed on the display unit 34. As shown in FIG. As shown in FIG. 6 , the reception window 4 includes an input data reception area 41 , a learning device selection area 42 , an analysis method selection area 43 and an reception operation display area 44 .

The input data reception area 41 displays a UI for receiving input of the first input data D1. The input data reception area 41 includes an import button 411 and a data name display area 412 .

The user inputs the first input data D<b>1 to the input data receiving unit 231 by operating the import button 411 . At this time, the input data receiving unit 231 receives input of the first input data D1 by the user based on the operation of the import button 411 by the user. After that, the input data reading unit 232 reads the input first input data D1.

The data name display area 412 displays the name of the input first input data D1.

The learner selection area 42 displays a UI that allows selection of the learner ML used to generate the learning model M1. The learning device ML displayed in the learning device selection area 42 is specified by the control unit 23 according to the first input data D1 read by the input data reading unit 232 . For example, the control unit 23 identifies the learning device ML displayed in the learning device selection area 42 according to the data size, format, and identifier of the first input data D1. The learning device selection area 42 of the present embodiment enters an active state in which the learning device ML can be selected when the input data reading unit 232 reads the input of the first input data D1.
The learning device selection area 42 includes a prediction target selection area 421 , a plurality of learning device display areas 422 , a learning device selection display area 423 and a first reception operation button 424 .

The prediction target selection area 421 is configured to be able to designate a parameter that is the output y1 of the learning model M1. The prediction target selection area 421 can be realized by arbitrary correspondence such as a pull tab, a list, and a button. In FIG. 6, the sales price is specified as the output y1. Note that the designated output y1 is not limited to one, and may be multiple.

A learning device display area 422 displays a UI for selecting the learning device ML identified in activity A002. For example, in the learner display area 422, information that can distinguish between a plurality of learners ML is displayed. The distinguishable information may include arbitrary information such as the name, type, and algorithm of the learning device ML. If the number of identified learners ML is smaller than the number of learner display areas 422, a part of the learner display area 422 may display that there is no information on the learners ML.

The learning device selection display area 423 displays whether or not the learning device ML corresponding to each of the learning device display areas 422 is selected. A specific aspect of the learning device selection display area 423 is arbitrary as long as the user can visually grasp it. For example, in the learning device selection display area 423, presence/absence of a check in a check box, change in color, change in shade, change in frame line, and the like are displayed.

The first reception operation button 424 is a UI that allows the user to decide whether or not to select an analysis method, which will be described later, in the selected learning device ML. When an operation is performed not to select the analysis method, for example, instead of the learning device selection area 42, the input data reception area 41 becomes active, and the first input data D1 can be received again.

On the other hand, when an operation for selecting an analysis method is performed on the first reception operation button 424, the analysis method selection area 43 becomes active. The analysis method selection area 43 displays a UI that allows the user to select an analysis method. The analysis method selection area 43 includes an analysis method selection button 431 , a model name display area 432 and a second reception operation button 433 .

The analysis method selection button 431 is configured to accept selection of an analysis method according to user's operation. The analysis method selection button 431 is configured to accept, for example, a user's click operation, tap operation, and flick operation. In this embodiment, a plurality of analysis method selection buttons 431 exist according to the number of set analysis methods. The display mode of the selected analysis method selection button 431 may be different from the display mode of the analysis method selection button 431 that is not selected. This makes it easier for the user to grasp which analysis method is selected.

The model name display area 432 displays the first input data D1 received in the input data receiving area 41, the learning device ML selected in the learning device selection area 42, and the analysis method selected in the analysis method selection area 43. and the name of the learning model M1 generated based on . Note that the model name display area 432 may be configured so that the user can enter the name of the learning model M1.

The second reception operation button 433 is configured so that the user can decide whether or not to generate the learning model M1. When it is determined not to generate the learning model M1 based on the operation of the second reception operation button 433, instead of the analysis method selection area 43, the input data reception area 41 or the learning device selection area 42 becomes active.

4-2. Example of Data Window 5 On the other hand, when it is determined to generate the learning model M1 based on the operation of the second reception operation button 433, the data window 5 and the conversion processing window 6 are displayed on the display unit 34. FIG. FIG. 7 shows an example of the data window 5 and the conversion processing window 6 displayed on the display unit 34. As shown in FIG.

The data window 5 is configured to display information about the first input data D1. The information about the first input data D1 includes, for example, the name of the first input data D1, the number of data points of the first input data D1, the capacity, the contents of the data points included in the first input data D1, and the like. is. The data window 5 includes a variable name display area 50 , an aggregation graph display area 51 , an aggregation information display area 52 and an individual information display area 53 .

In the variable name display area 50, information that can identify the input x1 included in the first input data D1 is displayed. For example, the variable name display area 50 causes the variable name display area 50 to display information corresponding to the names of the first input data D1 to the input x1.

Input data visual information is displayed in the total graph display area 51 . The input data visual information is a visual display of information related to the first input data D1. The input data visual information includes visual information regarding the distribution of at least one of the first input data D1 and the second input data D2. Input data visual information is displayed using, for example, histograms, line graphs, pie charts, or bubble charts. Input data visual information may be displayed using a combination of these. For example, the input data visual information includes statistical information of the first input data D1. Specifically, the input data visual information includes statistical information of the first input data D1 for each input x1. In this embodiment, the input data visual information is displayed in the summary information display area 52 as a histogram of the distribution of data points of the first input data D1. Also, the input data visual information may be displayed in the total information display area 52 as a combination of these display modes. For example, the histogram and the line graph may be displayed in the total information display area 52 so as to be viewable. Further, the histogram and the line graph may be superimposed and displayed on the total information display area 52 .
In the total graph display area 51, a difference 512 between the first input data D1 and the second input data D2 is displayed in a recognizable manner. In this embodiment, the display mode of the difference 512 between the first input data D1 and the second input data D2 is the same as the display mode of the common point 511 between the first input data D1 and the second input data D2. different. For example, the control unit 23 displays both in a mode in which at least one of color, shape, and pattern is different. As another example, the control unit 23 may display a predetermined mark such as an arrow in association with the difference 512 between the first input data D1 and the second input data D2. In the present embodiment, the common points 511 and the different points 512 are displayed in the tabulation graph display area 51 by the input data visual information so as to be viewable. The common points 511 and the different points 512 are displayed at least in the distribution. Also, the different point 512 is displayed in a color different from that of the common point 511 .

In the total information display area 52, statistical information regarding the first input data D1 is displayed. The statistical information displayed in the aggregated information display area 52 is, for example, maximum value, minimum value, average value, and standard deviation. Also, the statistical information may display the number of missing values in the first input data D1. The statistical information may be displayed as numerical values, character strings, or as visual information such as histograms.

In the individual information display area 53, information on data points included in the first input data D1 is displayed. Specifically, in the individual information display area 53, information on data points for each input x1 is displayed. Although the display mode in the individual information display area 53 is arbitrary, for example, information on data points for each input x1 is displayed in a table format.

4-3. Concerning the Conversion Processing Window 6 In the conversion processing window 6, at least information regarding the conversion processing to be performed on the first input data D1 is displayed. In this embodiment, the conversion processing window 6 is displayed so as to be viewable with the data window 5 , but may be displayed separately from the data window 5 . The display modes of the conversion processing window 6 include a first display mode 6a and a second display mode 6b. In the first display mode 6a, the conversion processing window 6 includes a first input data information display area 61, a generation condition display area 62, an automatic conversion processing display area 63, a processing condition display area 64, and a first A process execution button 65 , a manual conversion transfer button 66 and a process save button 67 are included.

Information about the first input data D1 or the second input data D2 is displayed in the first input data information display area 61 . In this embodiment, information about the second input data D2 generated by the conversion process is displayed. The information about the second input data D2 is, for example, the capacity of the second input data D2, the size of the second input data D2, the difference in capacity between the first input data D1 and the second input data D2, and the like. is.

The generation condition display area 62 displays the generation conditions of the learning model M1. The conditions for generating the learning model M1 are, for example, the prediction target selected in the prediction target selection area 421, the learning device ML selected in the learning device display area 422, the algorithm used in the learning device ML, and the like.

The automatic conversion processing display area 63 displays the content of the automatic conversion processing specified by the control section 23 . The content of the conversion processing includes, for example, deletion of part of the first input data D1, complementation of missing values, deletion of outliers, normalization of the first input data D1, and the like. Partial deletion of the first input data D1 means deletion of the input x1 that is unlikely to be used as an explanatory variable. Such an input x1 is, for example, the ID number of each data point. Note that the portion of the first input data D1 displayed in the data window 5 that has been changed by the conversion process is indicated by the indicator L1. The indicator L1 suggests a change part by conversion processing, for example, based on differences in color, shape, and pattern. The indicator L1 may indicate the area corresponding to the changed portion with a different outline from other areas.

The processing condition display area 64 displays conversion conditions for the automatic conversion processing displayed in the automatic conversion processing display area 63 . Specifically, in the processing condition display area 64, the conversion conditions corresponding to each automatic conversion process are displayed for each automatic conversion process.

The first process execution button 65 is a UI for causing the control unit 23 to execute the automatic conversion process displayed in the automatic conversion process display area 63 . The user can cause the control unit 23 to execute the automatic conversion process by operating the first process execution button 65 .

The manual conversion transition button 66 is used to transition the display mode of the conversion processing window 6 from the first display mode 6a to the second display mode 6b according to the user's operation. A second display mode 6b is a display mode in which the user can manually specify conversion processing. FIG. 8 is a diagram showing an example of the conversion processing window 6 displayed on the display section 34 in the case of the second display mode 6b. By operating the manual conversion transition button 66 , the conversion processing window 6 transitions to a display mode including a manual conversion processing designation area 661 and a manual conversion processing saving area 662 . At this time, the processing condition display area 64 and the first processing execution button 65 may be hidden. As a result, it is possible to reduce the display of low importance for the user's manual designation of the conversion process, and improve the convenience of operation.

The manual conversion process designation area 661 is configured so that the user can designate a conversion process different from the automatic conversion process. Hereinafter, for convenience of explanation, the conversion process specified in the manual conversion process specification area 661 will be referred to as manual conversion process. In this embodiment, the manual conversion processing designation area 661 displays the selected visual information IFc1. The user can select whether or not to execute at least part of the automatic conversion processing displayed in the automatic conversion processing display area 63 . For example, if the automatic conversion process includes outlier removal, the manual conversion process specification area 661 displays outlier candidates to be removed in the automatic conversion process. The user designates those candidates to be removed in the conversion process. If the conversion process is to complement missing values, the manual conversion process designation area 661 displays missing value candidates to be complemented by the automatic conversion process. The user designates one to be complemented by the conversion process from among the candidates. These designations can be realized, for example, by operating visual information such as check boxes, sliders, and buttons included in the manual conversion processing storage area 662 . A data point targeted for manual conversion processing may be configured to be distinguishable from other data points by an indicator L1. Note that the manual conversion process designation area 661 may be configured so that conversion processes other than the automatic conversion process can be designated.

The second input data D2 generated by the conversion process including the designated manual conversion process may change. In this case, common points 511 and differences 512 between the second input data D2 generated by the conversion process including the manual conversion process and the first input data D1 may be displayed in the total graph display area 51. . Further, in the summary graph display area 51, the points of difference of the second input data D2 before and after the manual conversion process may be displayed in a manner different from the points of commonality 511 and points of difference 512 described above. This allows the user to visually grasp the content of the specified manual conversion process. Moreover, it is preferable that the display is performed in conjunction with the specification of the manual conversion process. As a result, it becomes easy to grasp the influence of the designation of the manual conversion process on the first input data D1.

The manual conversion process saving area 662 is a UI that allows the user to decide whether or not to save the manual conversion process based on the user's operation. If a decision is made not to save the manual conversion process, the manual conversion process is discarded. After that, the display mode of the conversion processing window 6 transitions from the second display mode 6b to the first display mode 6a. On the other hand, if a decision is made to save the manual conversion process, the manual conversion process is updated as the conversion process. After that, the display mode of the conversion processing window 6 transitions from the second display mode 6b to the first display mode 6a.

The process save button 67 is a UI that allows the user to decide whether or not to save the contents of the conversion process based on the user's operation. When it is determined not to save the contents of the conversion process, the control unit 23 does not perform the conversion process on the first input data D1 and ends the information processing. At this time, the control unit 23 may cause the display unit 34 to display the reception window 4 again.

On the other hand, when it is determined to save the contents of the conversion process, the conversion process is executed on the first input data D1. Thereby, the second input data D2 is generated. Note that the generated second input data D<b>2 may be stored in the storage unit 22 . If manual conversion processing is specified, the conversion processing includes at least manual conversion processing. In this case, the conversion processing may include automatic conversion processing. This improves the reproducibility of a series of conversion processing including automatic conversion processing and manual conversion processing. On the other hand, if manual conversion processing is not specified, the conversion processing does not include manual conversion processing. The content of the conversion process may be stored in the storage unit 22 or the like, for example. After that, by inputting the second input data D2 to each of the identified learners ML based on the learning conditions, the learning model M1 is generated from each of the learners ML. After that, the model information display window 7 for the learning model M1 is displayed on the display section 34 .

4-4. Model Information Display Window 7 Next, an example of the model information display window 7 displayed on the display unit 34 will be described. FIG. 9 is a diagram showing an example of the model information display window 7 displayed on the display unit 34. As shown in FIG. The model information display window 7 displays information about the generated learning model M1. In this embodiment, information about one of the generated learning models M1 is displayed. The model information display window 7 includes a second input data information display area 71 , a model information display area 72 and a simulation execution button 73 .

The second input data information display area 71 displays information about the input data input to the learning device ML. In this embodiment, information about the second input data D2 is displayed. For example, the second input data information display area 71 displays the capacity, size, etc. of the second input data D2.

The model information display area 72 displays model information IF1 regarding the generated learning model M1. In this embodiment, the model information display area 72 includes at least accuracy information about the prediction accuracy of the learning model M1. The model information display area 72 includes a plurality of accuracy information display areas 721 , a contribution display area 722 and a contribution list display button 725 .

Accuracy information of the learning model M1 is displayed in the accuracy information display area 721 . In this embodiment, different accuracy information is displayed in each of the accuracy information display areas 721 . Specifically, the coefficient of determination, the mean square error, and the mean square deviation are individually displayed in each of the accuracy information display areas 721 . The accuracy information display area 721 can display a numerical value of each accuracy information, a meaning of each accuracy information, an evaluation method of each accuracy information, an improvement method of each accuracy information, and the like.

The contribution degree display area 722 displays the explanatory variable of the learning model M1, that is, the degree of contribution to the output y1 for each input x1 used to generate the learning model M1. The contribution is derived, for example, based on the coefficient for each input x1 in the learning model M1. The contribution includes an increase contribution and a decrease contribution. The increase contribution is a component of the contribution of the input x1 that contributes to the increase of the output y1. The decrease contribution is a component of the contribution of the input x1 that contributes to the decrease of the output y1. In this case, the coefficient for each input x1 in the learning model M1 may include a component corresponding to increased contribution and a component corresponding to decreased contribution. The contribution display area 722 includes an increased contribution display area 723 in which the increased contribution is displayed and a decreased contribution display area 724 in which the decreased contribution is displayed.

The increased contribution display area 723 and the decreased contribution display area 724 display the increased contribution and the decreased contribution so as to be distinguishable. Also, the increased contribution display area 723 and the decreased contribution display area 724 display the increased contribution and the decreased contribution so as to be comparable. For example, the increased contribution display area 723 and the decreased contribution display area 724 display the increased contribution and the decreased contribution as horizontal bar graphs so that they can be compared and viewed. In this embodiment, the contribution degree display area 722 displays the contribution degree of some of all the inputs x1. Specifically, in the contribution degree display area 722, among all the inputs x1, those with a predetermined ordinal number, such as the fifth, are displayed in descending order of contribution degree. This makes it easier for the user to recognize the input x1 that tends to affect the output y1.

The contribution degree list display button 725 is a UI for increasing the display number of the contribution degree of the input x1 based on the user's operation when only a part of the contribution degree of all the inputs x1 is displayed. Based on the operation of the contribution list display button 725, the contribution of all the inputs x1 may be displayed.

The simulation execution button 73 is a UI for executing a predictive simulation using the learning model M1 based on user's operation. For example, a predictive simulation is a search for an input x1 corresponding to an output y1 that satisfies a given condition. The predetermined condition is, for example, that the output y1 is equal to or greater than a predetermined threshold, or that the output y1 is the maximum or the maximum in a predetermined number of trials. When the input x1 is input to the learning model M1, the output y1 is obtained. At this time, by changing the input x1 within a predetermined domain, the output y1 changes according to the change in the input x1. Thus, the input x1 is derived when the output y1 satisfies a predetermined condition. For example, if the output y1 is the sales price and the predetermined condition is to maximize the sales price, the user can obtain the input x1 that maximizes the sales price through the predictive simulation.

4-5. Model Search Window 8 and Model Comparison Window 9 The control unit 23 may cause the display unit 34 to display the model search window 8 and the model comparison window 9 . FIG. 10 shows an example of the model search window 8 and the model comparison window 9 displayed on the display unit 34. As shown in FIG.

The model search window 8 includes a UI that allows searching for previously generated learning models M1. Specifically, the model search window 8 includes a search condition input area 81 , a search result display area 82 , and a search window exit button 83 .

The search condition input area 81 is configured to accept search conditions used for searching. The search condition is arbitrary, for example, the name of the learning model M1, the algorithm, keywords such as the name of the output y1, the learning condition of the learning model M1, the time when the learning model M1 was generated, and the like. In addition, the search condition input area 81 is configured to be able to search for the learning model M1 based on the received search condition based on the user's operation.

The search result display area 82 displays search results based on the search conditions received by the search condition input area 81 . In the search result display area 82, past learning models M1 that match the search conditions are displayed in a listable manner. At least part of the model information IF1 of the past learning model M1 may be displayed in the search result display area 82 so as to be visible to the user. This improves the listability of search results. The past learning model M1 displayed in the search result display area 82 is configured to be designated by the user. Designation of the learning model M1 by the user is visibly displayed by an indicator such as a check box. Hereinafter, for convenience of explanation, the learning model M1 specified in the search result display area 82 will be referred to as a specified learning model M2.

The search window end button 83 is a UI for ending the search of the past learning model M1 based on the user's operation.

In the model comparison window 9, the model information IF1 of the specified learning model M2 is displayed so as to be comparable. The model comparison window 9 includes a comparison model display area 91 , a parameter selection area 92 , a comparison result display area 93 and a simulation execution button 94 .

In the comparison model display area 91, at least part of the model information IF1 of the learning model M1 specified in the search result display area 82 is displayed. The comparison model display area 91 is configured so that the user can designate one of the designated learning models M2 displayed in the comparison model display area 91 .

The parameter selection area 92 is configured so that the user can select parameters used for generating and evaluating the designated learning model M2. In this embodiment, two parameters are selectable. Parameters used for generating and evaluating the designated learning model M2 include various accuracy information included in the model information IF1, learning conditions for the designated learning model M2, and the like. Hereinafter, for convenience of explanation, the parameter selected in the parameter selection area 92 will be referred to as the selected parameter.

In the comparison result display area 93, visual information that enables a list of selected parameters of the specified learning model M2 is displayed. Visual information is arbitrary, for example, a scatter diagram, a histogram, a correlation diagram, a three-dimensional plot diagram, or the like. This makes it possible to easily compare the accuracy of the designated learning model M2.

The simulation execution button 94 is a UI for executing a predictive simulation using the designated learning model M2 based on the user's operation. The specified learning model M2 used in the predictive simulation is the specified learning model M2 specified in the comparative model display area 91, for example.

In this embodiment, the model comparison window 9 is displayed so as to be viewable with the model search window 8 . This facilitates comparison between the search result and the designated learning model M2.

5. Others Regarding the information processing system 1 according to the above-described embodiment, the following aspects may be adopted.

The first input data D1 and the second input data D2 may each be stored in the database DB1 as the external data D0. These external data D0 may be provided to other users under predetermined conditions.

The control unit 23 displays the history of conversion processing performed on the first input data D1, the so-called conversion processing version, in at least one of the data window 5, the conversion processing window 6, and the model information display window 7. You may let This makes it easy to analogize the relationship between the conversion process and the accuracy information. Also, the control unit 23 may manage versions of conversion processing.

For example, when the first input data D1 does not satisfy a predetermined quality condition, the control unit 23 may cause the display unit 34 to display a warning. The quality conditions are, for example, the number of data points of the first input data D1, the capacity, the ratio of outliers, and the like. When the quality condition is not satisfied, for example, when the number of data points in the first input data D1 is less than a predetermined value, or when the number of outliers in the first input data D1 is greater than a predetermined reference number. be. The warning may be at least one of the reception window 4, data window 5, conversion processing window 6, model information display window 7, model search window 8, and model comparison window 9, or other windows. Note that the warning is not limited to being displayed on the display unit 34, and can be realized in any form such as sound, vibration, or light.

The information processing device 2 may be in an on-premise form or in a cloud form. The cloud-type information processing apparatus 2 may provide the above functions and processes in the form of, for example, SaaS (Software as a Service) or cloud computing.

In the above embodiment, the information processing device 2 performs various types of storage and control, but instead of the information processing device 2, a plurality of external devices may be used. That is, the first input data D1, the second input data D2, and the learning model M1 may be distributed and stored in a plurality of external devices using blockchain technology or the like.

It may be provided in each aspect described below.

(2) In the information processing system, further, in the conversion process specifying step, the conversion process is specified based on the first input data and the learning device.

According to such a configuration, it is possible to reduce the trouble of the user himself/herself preparing input data suitable for the learning device in advance.

(3) In the information processing system, further, in the conversion processing presenting step, visual information configured to allow selection of at least one of the specified conversion processing is generated.

According to such a configuration, the user can arbitrarily select the conversion process necessary for generating the learning model from among the identified conversion processes. Therefore, it is possible to improve the reproducibility of conversion processing by the user.

(4) In the information processing system, the data display step causes the points of difference to be displayed in colors different from at least the points of commonness.

According to such a configuration, it becomes easy for the user to intuitively grasp the difference between before and after the conversion processing based on the color.

(5) In the information processing system, the display mode determining step further determines a display mode of at least one of the common points and the different points according to the conversion process.

According to such a configuration, the user can visually grasp the content of the conversion processing performed on the data from the display mode of the difference between the two data. Therefore, it becomes easier for the user to grasp the mode of conversion processing.

(6) In the information processing system, the conversion process includes at least one of deletion of data points included in the first input data and addition of data points to the first input data. including, things.

According to such a configuration, since it is possible to cope with excess or deficiency of data points included in the first input data, the versatility of the information processing system is improved.

(7) In the information processing system, in the data display step, the distribution of at least one of the first input data and the second input data is visually displayed, and the points of difference and the points of commonality are: , at least appearing in said distribution,

According to such a configuration, it becomes easy for the user to visually grasp the difference due to the conversion processing from the data distribution state.

(8) In the information processing system, the first input data receiving step further receives input of the first input data by a user.

According to such a configuration, it is possible to visually grasp the details of the conversion process performed on the first input data input by the user. Therefore, convenience can be improved.

(9) In the information processing system, further, in the second input data generating step, the second input data is generated by performing the conversion process on the first input data.

With such a configuration, the second input data can be generated from the first input data. And it can be visually grasped as a point of difference. Therefore, convenience can be improved.

(10) An information processing method, comprising steps of the information processing system.

(11) An information processing program that causes a computer to execute each step of the information processing system.
Of course, this is not the only case.

Furthermore, the following points should also be noted.

The movement to use deep learning (DL) and other machine learning (machine learning, ML) technologies in various situations is accelerating, creating a situation that can be called a kind of boom. However, contrary to this excitement, 85% of ML introduction projects have failed, 10% of companies are able to utilize ML and AI (Artificial Intelligence) technology, and only 17% of information companies. It is said.

There are various causes for this. First, it is not easy to understand what problems ML and AI are effective against. Second, what kind of data should be prepared to use ML, and how to process and preprocess data. What to do depends on experience and intuition. Third, it is not easy to prepare a large amount of data. Fourth, it is not easy to understand how to build an ML or AI model. Moreover, it depends on experience and intuition. Fifth, it is difficult to understand why the desired output can be obtained from DL, which is a method of ML. and the fact that it is not possible to obtain

As mentioned above, there are various obstacles to the successful utilization of ML, but on the other hand, there are many ML services and AI services on the Internet, and it is a chaotic situation where you do not know which one to use. There is also

In addition, in order to use the above ML and AI services, many parameters must be entered, and it is difficult to understand the meaning of the parameters, and there is a reality that only ML and AI experts can use them. . In other words, ML services and AI services were not democratized services that could be used by non-specialists.

In view of the above-mentioned situation, it is possible to use ML services without specialized knowledge, and if you prepare input data, you can use ML services in 3 steps, and provide analysis of the obtained results, Furthermore, it is an object of the present invention to create an environment in which anyone can use ML services by providing a technique that can also perform prediction. This democratizes ML services.

A technical idea for solving the above problems is to provide a wrapping interface system for connecting to many ML (hereinafter referred to as AutoML) services existing on the Internet. As a result, data preparation such as data collection, preprocessing, and uploading (step 1), model construction and parallel execution of multiple MLs (step 2), performance comparison of each ML and introduction to actual work (step 3) It is possible to introduce ML in the following three steps.

For this purpose, the account setting, parameter input procedure, etc. are first unified, and then format conversion is applied to each AutoML. This makes it possible to create an account in 3 steps, which used to take 10 to 15 steps.

Next, collect internal and external data. The access points to internal and external data required for this are connected automatically or with user intervention and assistance, and the data is collected.

Next, the input data is processed. This includes, but is not limited to, data cleansing such as conversion to a unified format such as date data, processing data items with many missing points, and applying preprocessing including statistical processing from the original data. and transforming it into data suitable for the purpose, and using queries to perform data extraction, data merging, etc.

At this time, if necessary, the processed data may be displayed for confirmation and correction.

Next, prepare the ML model. It is not limited to the use of various ML services that exist on the Internet, but also a method of constructing an original model without programming based on a GUI (Graphical User Interface), and modifying various ML services that exist on the Internet to create an original model. This is done by a method of constructing and a method of importing an ML model that has already been constructed but is not open to the public on the Internet into the system of the present invention.

It also provides functionality that enables ML and statistical analysis without programming. In addition, a function is also provided to recommend from the input data what kind of template should be used to build a model with high accuracy.

The data input to the ML may be divided into learning data and prediction data, and the prediction data may be input to the ML that has been learned using the learning data. The prediction data is used hereafter for ML performance comparisons, etc.

Once the input data and the ML model have been prepared, training is started using the learning data.
At this time, multiple MLs may be executed in parallel.

After training, predictive data is injected into each ML service automatically or by user's instruction to obtain results. If multiple AutoMLs are executed in parallel at this time, the performance can be compared immediately.

View the results for each AutoML service. This includes graphical visualizations showing the level of prediction accuracy (coefficient of determination) for each ML service, comparison of item contributions, etc.

The above data collection, display and comparison of results are repeated, and when the user decides that it can be put into actual work, the operation is started.

In operation, it is within the scope of the present invention that the results are utilized by the application program by inquiring from the application program to this system using a prepared API (Application Program Interface), or that the results are displayed directly from this system. .

ML platforms including Kubeflow may be used for automation of operations.

The above series of operations may be pipelined to simplify the operations. Pipelining reduces the degree of freedom, but improves the overall outlook and allows non-experts to handle it. If it is necessary to increase the degree of freedom, specialized item settings may be performed by opening a detailed screen or the like.

Furthermore, it is possible to record and view various changes and operation histories along the flow of the pipeline, and it is possible to show the grounds for selecting ML services and models.

After the ML model to be adopted and the data preprocessing method required for it are decided, it is introduced into the actual business. For example, if you want a sales forecast as an ML output, you can directly see the screen displayed by the system of the present invention, or if necessary, request information provision from the application program via the API provided by the system of the present invention. may be issued and thereby displayed on the screen of the application program.

Therefore, in order to solve the above problems, an information processing method according to a first aspect includes a first step of unifying initial setting work including at least one of account setting and parameter input procedure setting; a second step of applying a format conversion to connect to possible automated machine learning services; a third step in which internal and external data is collected; a fourth step of processing said collected data; a fifth step of preparing an automated machine learning service to be used; splitting the data processed in said fourth step into learning data and prediction data; preparing said learning data in said fifth step; and a sixth step of learning by an automatic machine learning service performed by.

In order to solve the above problems, an information processing apparatus according to a second aspect includes a centralizing unit capable of centralizing initial setting work including at least one of account setting and parameter input procedure setting; A format conversion unit that performs format conversion for connecting to an automatic machine learning service that can exist in the a preparation unit that prepares for an automatic machine learning service to be performed, the data processed by the data processing unit is divided into learning data and prediction data, and the learning data is prepared by the preparation unit for automatic machine learning and a learning unit for learning by the service.

According to the above two aspects, even if you are not an expert in machine learning technology such as deep learning, you can select and / or build a learning model as long as you prepare learning data, and from the results of a plurality of learning models Performance can be compared, the optimum model can be selected from among multiple learning models, and it can be applied to actual business operations to support operations after application.

As a third aspect, in the second aspect, the collected and prepared input data may be converted according to many automatic machine learning services existing on the Internet. According to this aspect, it is possible to omit the process of preparing different input data for each machine learning service. It should be noted that this third mode can also be used in a superimposed manner with respect to the first mode.

As a fourth aspect, in the second aspect, simple format conversion of input data, data cleansing including processing of missing data or redundant/unnecessary data, extraction of feature values from original data, preprocessing including statistical processing may be applied to perform data transformations including conversion to data suitable for the purpose, data extraction using queries, and/or data merging. According to this aspect, simple format conversion of input data, data cleansing including processing of missing data or redundant/unnecessary data, extraction of feature values from original data, preprocessing including statistical processing are applied to achieve the purpose Data conversion, including conversion to suitable data, data extraction using queries, or data merging, can be performed by giving simple instructions. It should be noted that this fourth aspect can also be used in a superimposed manner with respect to the first aspect.

As a fifth aspect, in the second aspect, a list part that can list a plurality of machine learning services or machine learning models existing on the Internet, and a plurality of machine learning services or machine learning models listed in the list part A selection unit that selects connection to one of the above, and data input to a plurality of machine learning services or machine learning models collectively by being selected by the selection unit, parallel execution, acquisition of results, and and an execution unit that executes at least one of the comparison. It should be noted that this fifth mode can also be used in a superimposed manner with respect to the first mode.

As a sixth aspect, in the second aspect, the learning section and/or the preparation section may be performed by graphical user interface means. According to this aspect, in addition to selecting the above machine learning service, the user can build his own machine learning model using a graphical user interface-based technique and/or exists as public information. You can import a machine learning model that It should be noted that this sixth aspect can also be used in a superimposed manner with respect to the first aspect.

As a seventh aspect, in the second aspect, when the processed data is divided into the learning data and the prediction data in the learning unit, the input data to machine learning is compared in performance with the learning data. and/or may be divided into prediction data. According to this aspect, input data to the machine learning can be divided into learning data and performance comparison and/or prediction data for use. It should be noted that this seventh mode can also be used in a superimposed manner with respect to the first mode.

As an eighth aspect, the second aspect may further include an index providing unit that provides an index for comparing the performance of a plurality of machine learning services or machine learning models executed with the same input data. It should be noted that this eighth mode can also be used in a superimposed manner with respect to the first mode.

As a ninth aspect, in the eighth aspect, at least one of the coefficient of determination, mean absolute error, mean square deviation, item contribution, model prediction and actual comparison, residual histogram, as the indicator A metric may be presented to compare the performance of the machine learning service and the machine learning model. It should be noted that the ninth mode can also be used in a superimposed manner with respect to the mode in which the eighth mode is superimposed on the first mode.

As a tenth aspect, the second aspect may further include a selection unit that selects from a plurality of machine learning services and the result of each machine learning model out of the plurality. According to this aspect, the optimum one can be selected from a plurality of machine learning services and the results of the respective machine learning models out of the plurality and applied to actual business. It should be noted that this tenth aspect can also be used in a superimposed manner with respect to the first aspect.

As an eleventh aspect, in the second aspect, a maintenance and management unit capable of maintaining and/or managing the accuracy of the machine learning service and the machine learning model may be further provided. According to this aspect, a function is provided that can maintain and/or manage the accuracy of the machine learning service and the machine learning model after actual launch.
It should be noted that the eleventh aspect can also be used in a superimposed manner with respect to the first aspect.

As a twelfth aspect, in the second aspect, collection and preparation of the data, parallel execution of multiple machine learning services and machine learning models, performance comparison of machine learning services and machine learning models, input to actual business A pipeline section that pipelines the operations may be further provided. According to this aspect, the operation of collecting and preparing the above data, parallel execution of a plurality of machine learning services and machine learning models, performance comparison of machine learning services and machine learning models, and input to actual work is pipelined. , can improve the overall outlook. The twelfth mode can also be used in a superimposed manner with respect to the first mode.

As a thirteenth aspect, the twelfth aspect may further include a user intervention unit that allows a user to intervene as necessary at various intermediate stages of the pipelined processing. . According to this aspect, the user may intervene as necessary to perform detailed settings and operations at various intermediate stages of the pipelined processing. The thirteenth mode can also be used in a superimposed manner with respect to the mode in which the twelfth mode is superimposed on the first mode.

As a fourteenth aspect, in the second aspect, further comprising a data requesting unit for requesting the data from an application program via an application program interface in order to obtain the processing result of the machine learning service or the machine learning model It may be provided. According to this aspect, in order to obtain the processing results of a machine learning service or a machine learning model that has been put into actual business, an application program requests data from the system of the present invention via the application program interface. The application program may perform processing including display. It should be noted that the fourteenth aspect can also be used in a superimposed manner with respect to the first aspect.

As a fifteenth aspect, in the second aspect, the screen in at least one of the unification unit, the format conversion unit, the data collection unit, the data processing unit, the preparation unit, and the learning unit is a data screen. At least one of the screens related to collection and preparation, the screen related to selection/construction/execution of machine learning models, the screen related to performance comparison of each learning model, and the screen to decide the selection of machine learning models and introduce them to actual work. You may have a screen transition including. According to this aspect, information is exchanged with the user using a plurality of screens in each step from preparation and upload of data to comparison of performance between a plurality of MLs and introduction to actual work. is a screen related to data collection and preparation (preprocessing, uploading, etc.), a screen related to selection, construction, and execution of machine learning models (model construction, ML execution), a screen related to performance comparison of each learning model, and a machine Since it has screen transitions including screens for determining the selection of learning models and introducing them to actual work, the learning process can be designed in the screen transition definition. The fifteenth aspect can also be used in a superimposed manner with respect to the first aspect.

Further, in order to solve the above problems, a program according to a sixteenth aspect causes a computer to convert collected learning data to match each machine learning service or machine learning model without intervention of an expert. a data cleansing unit that processes missing, redundant, and unnecessary data; a feature value extraction unit that extracts feature values from the original data; and preprocessing including statistical processing. A data combining/splitting unit that converts data into data suitable for the purpose, extracts data using queries, and converts data including data combining; a normalization/standardization unit that normalizes and standardizes data; A service model selection section that selects learning services and machine learning models, a no-code development section for building machine learning models, a simulation section that executes multiple machine learning services and machine learning models in parallel, and results A model evaluation section that displays and compares, a model selection section that selects the optimal machine learning service or machine learning model, and an input/operation section that puts the selected model into actual work and operates it. It is characterized by functioning as a support part that supports.

According to the above aspect, the data processing/converting unit that converts the collected learning data so as to match each machine learning service or machine learning model without the intervention of an expert, and the processing of missing data, duplicated/unnecessary data, etc. a data cleansing unit including a data cleansing unit, a feature extraction unit that extracts feature values from the original data, a conversion to data suitable for the purpose by applying preprocessing including statistical processing, and data extraction and data combination using queries Data joining/splitting part that performs data transformation including normalization/standardization part that normalizes/standardizes data Service/model selection part that selects multiple machine learning services and machine learning models, without programming A no-code development department that builds its own machine learning model, a simulation department that runs multiple machine learning services and machine learning models in parallel, a model evaluation department that displays and compares results, and an optimal machine learning service or machine learning model. can function as a model selection unit that selects the model, an input/operation unit that inputs the selected model into actual work and operates it, and a support unit that supports a series of these.

A seventeenth aspect can also be implemented as a recording medium storing the program according to the sixteenth aspect.

Finally, while various embodiments of the present disclosure have been described, they have been presented by way of example and are not intended to limit the scope of the invention. The novel embodiment can be embodied in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. The embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

1: information processing system 2: information processing device 3: user terminal 4: reception window 5: data window 6: conversion processing window 6a: first display mode 6b: second display mode 7: model information display window 8: model Search window 9: Model comparison window 20: Communication bus 21: Communication unit 22: Storage unit 23: Control unit 30: Communication bus 31: Communication unit 32: Storage unit 33: Control unit 34: Display unit 35: Input unit 41: Input Data reception area 42 : Learning device selection area 43 : Analysis method selection area 44 : Reception operation display area 50 : Variable name display area 51 : Total graph display area 52 : Total information display area 53 : Individual information display area 61 : First Input data information display area 62 : Generation condition display area 63 : Automatic conversion process display area 64 : Processing condition display area 65 : First process execution button 66 : Manual conversion transfer button 67 : Process save button 71 : Second input data Information display area 72 : Model information display area 73 : Simulation execution button 81 : Search condition input area 82 : Search result display area 83 : Search window end button 91 : Comparison model display area 92 : Parameter selection area 93 : Comparison result display area 94 : Simulation execution button 231 : Input data receiving unit 232 : Input data reading unit 233 : Conversion processing specifying unit 234 : Conversion processing presenting unit 235 : Input data generating unit 236 : Display mode determining unit 237 : Data display unit 411 : Import button 412 : Data name display area 421 : Prediction target selection area 422 : Learning device display area 423 : Learning device selection display area 424 : First reception operation button 431 : Analysis method selection button 432 : Model name display area 433 : Second reception Operation button 511 : common point 512 : difference point 661 : manual conversion processing designation area 662 : manual conversion processing storage area 721 : accuracy information display area 722 : contribution display area 723 : increase contribution display area 724 : decrease contribution display area 725: Contribution list display button A001: Activity A002: Activity A003: Activity A004: Activity A005: Activity A006: Activity A007: Activity A008: Activity A009: Activity A010: Activity A011: Activity D: Input data D0: External data D1: First input data D2: Second input data DB1: Database IF1: Model information IFc1: Selection visual information L1: Indicator M1: Learning model M2: Designation Learning model ML: learning device x1: input y1: output

Claims (10)

An information processing system,
Equipped with a control unit,
The control unit is configured to perform the following steps,
reading first input data in a first input data reading step;
a conversion process identifying step, based on the read first input data and a predetermined conversion condition, identifying a predetermined conversion process to be performed on the first input data;
In the second input data reading step,IdentifiedPreviouschange of recordreading second input data generated by performing conversion processing, wherein the second input data is data used for generating a learning model by a learning device,
In the data display step, at least points in common between the first input data and the second input data and points of difference between the first input data and the second input data are displayed in an overviewable manner. ,
The display mode of the points of difference is different from the display mode of the common points.the law of nature,
In the data display step, the conversion process is displayed in a form that can be listed with the conversion conditions. thing. In the information processing system according to claim 1 ,
Furthermore, in the conversion process presenting step, visual information is generated so that at least one of the specified conversion processes can be selected. In the information processing system according to claim 1 or claim 2 ,
In the data display step, the different points are displayed in colors different from at least the common points. In the information processing system according to any one of claims 1 to 3 ,
Furthermore, in the display mode determining step, a display mode of at least one of the common points and the different points is determined according to the conversion processing. In the information processing system according to any one of claims 1 to 4 ,
The transforming process includes at least one of deleting data points included in the first input data and adding data points to the first input data. In the information processing system according to any one of claims 1 to 5 ,
In the data display step, the distribution of at least one of the first input data and the second input data is visually displayed;
wherein the points of difference and the points of commonality are represented at least in the distribution In the information processing system according to any one of claims 1 to 6 ,
Further, the first input data receiving step receives input of the first input data by a user. In the information processing system according to any one of claims 1 to 7 ,
Furthermore, in the second input data generating step, the second input data is generated by performing the conversion process on the first input data. An information processing method,
An object comprising each step of the information processing system according to any one of claims 1 to 8 . An information processing program,
A computer that executes each step of the information processing system according to any one of claims 1 to 8 .

Images