JP2024014158A - Information processing system, information processing method, and information processing program - Google Patents

Abstract

The present invention provides an information processing system, an information processing method, and an information processing program that grasp the influence of a pseudo correlation between an input and an output that may cause a decrease in the accuracy of a learning model. A method obtains a correlation between a learning result of a learning model using a first data set, a data region that is part of the first data set as an input, and an output of the learning model. The method includes an obtaining step A2 and a specifying step A5 of specifying a pseudo-correlation region based on the first data set and the correlation. The pseudo-correlation area is a data area that has a pseudo-correlation with the output of the learning model, among data areas that have a correlation. The method further includes a generation step A11 of performing at least one of modification and annotation on the pseudo-correlation region to generate a second data set, and a display processing step A12 of displaying information regarding the second data set. ,including. [Selection diagram] Figure 5

Description

The present invention relates to an information processing system, an information processing method, and an information processing program.

Patent Document 1 discloses a feature amount acquisition device, a display device, a feature amount acquisition method, a similar image search method, a display method, and a program that acquire feature amounts suitable for similar image search.

The feature acquisition device has a plurality of layers, and processes input data based on image data of an input image in which a first object and a second object surrounding the first object are captured in the plurality of layers. In the CNN discriminator that outputs the result of identification, an activation calculation unit derives the degree of influence of a unit in a certain layer among the plurality of layers on the classification result as an activation degree, and the derived activation degree and the input image. Based on the image data of and a feature amount obtaining unit that obtains the feature amount of the input image so that the feature amount is smaller than the feature amount of the high activity image region.

Japanese Patent Application Publication No. 2022-33429

By the way, a learning model identifies a correlation between an input and an output based on data included in a data set, and performs various tasks such as prediction, determination, and classification based on the identified correlation. However, the correlation may actually include a pseudo correlation that has no correlation in reality. A spurious correlation may cause a decrease in the accuracy of the learning model. Therefore, it is necessary to understand the influence of such pseudo correlation.

According to one aspect of the present invention, an information processing system is provided. This information processing system includes at least one processor capable of executing a program to perform the following steps. In the acquisition step, a learning result of the learning model using the first data set, a correlation between a data region that is part of the first data set as an input, and an output of the learning model are acquired. In the identifying step, a pseudo-correlation area is identified based on the first data set and the correlation. The pseudo-correlation area is a data area that has a pseudo-correlation with the output of the learning model, among the correlated data areas. In the generation step, at least one of modification and annotation is performed on the pseudo-correlation region to generate a second data set.

According to this configuration, it becomes easier for the user to understand the influence of the pseudo correlation region on the learning results using the first data set.

1 is a configuration diagram showing an information processing system 1. FIG. 2 is a block diagram showing the hardware configuration of an information processing device 2. FIG. 3 is a block diagram showing the hardware configuration of a user terminal 3. FIG. 2 is a diagram showing an example of a functional unit included in a processor 23. FIG. 2 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. FIG. FIG. 7 is a diagram showing an example of learning data D including a pseudo-correlation region R2, which is displayed on the display unit 34 as a result of display processing of activity A7. 3 is a diagram showing an example of data processing for learning data D. FIG.

Embodiments of the present invention will be described below with reference to the drawings. Various features shown in the embodiments described below can be combined with each other.

By the way, the program for implementing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be downloaded from an external server. The program may be provided in a manner that allows the program to be started on an external computer and the function thereof is realized on the client terminal (so-called cloud computing).

Furthermore, in this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by circuits in a broad sense and software information processing that can be concretely implemented by these hardware resources. . In addition, various types of information are handled in this embodiment, and these information include, for example, the physical value of a signal value representing voltage and current, and the signal value as a binary bit collection consisting of 0 or 1. It is expressed by high and low levels or quantum superposition (so-called quantum bits), and communication and calculations can be performed on circuits in a broad sense.

Further, a circuit in a broad sense is a circuit realized by at least appropriately combining a circuit, a circuit, a processor, a memory, and the like. That is, Application Specific Integrated Circuit (ASIC), programmable logic device (for example, Simple Programmable Logic Device (SPLD)), Complex Programmable Logic Device (Complex Pr) ogrammable Logic Device: CPLD), and fields This includes a field programmable gate array (FPGA) and the like.

1. Hardware configuration This section explains the hardware configuration.

<Information processing system 1>
FIG. 1 is a configuration diagram showing an information processing system 1. As shown in FIG. The information processing system 1 includes an information processing device 2, a user terminal 3, and a database DB1. The information processing device 2, the user terminal 3, and the database DB1 are configured to be able to communicate through a telecommunications line. In one embodiment, information handling system 1 is comprised of one or more devices or components. For example, if the information processing system 1 is composed of only the information processing device 2, the information processing system 1 can be the information processing device 2. These components will be explained below.

<Database DB1>
Database DB1 stores various data sets DS. The data set DS may include open data such as MNIST, Fashion-MNIST, ImageNet, etc., or may be provided in a limited manner to some users.

<Information processing device 2>
FIG. 2 is a block diagram showing the hardware configuration of the information processing device 2. As shown in FIG. The information processing device 2 includes a communication unit 21, a storage unit 22, and a processor 23, and these components are electrically connected via a communication bus 20 inside the information processing device 2. Each component will be further explained.

Although the communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., it is also suitable for wireless LAN network communication, mobile communication such as 3G/LTE/5G, and BLUETOOTH (registered trademark). Communication etc. may be included as necessary. That is, it is more preferable to implement it as a set of these plurality of communication means. That is, the information processing device 2 may communicate various information from the outside via the communication unit 21 and the network.

The storage unit 22 stores various information defined by the above description. This may be used, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the information processing device 2 executed by the processor 23, or as a temporary storage device related to program calculations. It can be implemented as a memory such as a random access memory (RAM) that stores necessary information (arguments, arrays, etc.). The storage unit 22 stores various programs, variables, etc. related to the information processing device 2 executed by the processor 23.

The processor 23 processes and controls overall operations related to the information processing device 2 . The processor 23 is, for example, a central processing unit (CPU) not shown. The processor 23 implements various functions related to the information processing device 2 by reading predetermined programs stored in the storage unit 22. That is, information processing by software stored in the storage unit 22 is specifically implemented by the processor 23, which is an example of hardware, and can be executed as each functional unit included in the processor 23. These will be explained in more detail in the next section. Note that the processor 23 is not limited to a single processor, and may be implemented so as to have a plurality of processors 23 for each function. It may also be a combination thereof.

<User terminal 3>
FIG. 3 is a block diagram showing the hardware configuration of the user terminal 3. The user terminal 3 includes a communication section 31 , a storage section 32 , a processor 33 , a display section 34 , and an HMI device 35 , and these components are electrically connected via the communication bus 30 inside the user terminal 3 . It is connected to the. Descriptions of the communication unit 31, storage unit 32, and processor 33 are omitted because they are similar to the descriptions of each unit in the information processing device 2.

The display unit 34 may be included in the user terminal 3 housing, or may be externally attached. The display unit 34 displays a graphical user interface (GUI) screen that can be operated by the user. This is preferably implemented by using display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display depending on the type of user terminal 3, for example.

HMI device 35 is a human machine interface device. The HMI device 35 may be included in the housing of the user terminal 3 or may be attached externally. For example, the HMI device 35 may be integrated with the display section 34 and implemented as a touch panel. With a touch panel, the user can input tap operations, swipe operations, and the like. Of course, a switch button, a mouse, a QWERTY keyboard, a voice recognition device, a gesture detection device, a line of sight detection device, a biological signal detection device, an imaging device, etc. may be used instead of the touch panel. That is, the HMI device 35 accepts operation inputs made by the user. In response, the HMI device 35 transfers a signal corresponding to the operational input to the processor 33 via the communication bus 30. The processor 33 can perform predetermined control and calculations as necessary. It can also be said that the HMI device 35 includes an input unit configured to accept input from the user.

2. Functional Configuration of Information Processing Device 2 FIG. 4 is a diagram illustrating an example of functional units included in the processor 23. As shown in FIG. 4, the processor 23 includes an acquisition section 231, a similarity calculation section 232, a specification section 233, a display processing section 234, a reception section 235, and a generation section 236. This section provides an overview of these functional units. Details of each functional unit will be explained together with information processing described later.

The acquisition unit 231 is configured to be able to acquire information from the user terminal 3 or other devices. The acquisition unit 231 acquires the dataset DS and the learning model M trained using the dataset DS, for example, from an information source such as the database DB1. The acquisition unit 231 also reads out various information stored in a storage area that is at least a part of the storage unit 22 and writes the read information to a work area that is at least a part of the storage unit 22. , is configured to be able to acquire various information. The storage area is, for example, an area of the storage unit 22 that is implemented as a storage device such as an SSD. The work area is, for example, an area implemented as a memory such as a RAM. Note that the acquisition by the acquisition unit 231 includes acquiring the output results of each functional unit included in the processor 23.

The similarity calculation unit 232 is configured to be able to calculate various pieces of information based on the acquisition results obtained by the acquisition unit 231. The similarity calculation unit 232 is configured to be able to calculate, for example, the similarity between a plurality of learning data D included in the data set DS, the similarity between a plurality of classes into which the learning data D is classified, and the like.

The identifying unit 233 is configured to be able to identify various pieces of information based on the acquisition results of the acquiring unit 231, the calculation results of the similarity calculation unit 232, and the like.

The display processing unit 234 is configured to be able to display various information. The information can be presented to the user via the display unit 34 of the user terminal 3 or another device. In such a case, for example, the display processing unit 234 controls the display unit 34 of the user terminal 3 to display visual information such as a screen, an image including a still image or a moving image, an icon, a message, and the like. The display processing unit 234 may generate only rendering information for displaying visual information on the user terminal 3. Note that the display processing unit 234 may present the output information to the user without going through the user terminal 3 or other device users.

The reception unit 235 is configured to be able to accept various specifications from the user. The designation may be input through the user terminal 3 or directly into the information processing device 2, for example.

The generation unit 236 is configured to be able to generate various types of information. For example, the generation unit 236 generates the dataset DS again based on the acquired dataset DS and the received specification.

The learning unit 237 is configured to be able to perform learning of the learning model M using the acquired data set DS. In addition, in order to cause a device other than the processor 23 to perform learning, the learning unit 237 issues a command to start learning the learning model M using the data set DS and information necessary for the learning, such as hyperparameters. It is also possible to transmit learning conditions such as the number of times of learning and the like.

3. About Information Processing In this section, information processing executed in the information processing system 1 described above will be explained.

3.1. About the flow of information processing FIG. 5 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. As shown in FIG. Note that the information processing may include any exception processing not shown. Exception handling includes interruption of the information processing and omission of each process. The selection or input performed in the information processing may be based on a user's operation, or may be automatically performed without depending on a user's operation.

[Activity A1]
In activity A1, the processor 33 receives an instruction to start learning from the user, and transmits the instruction to the processor 23 of the information processing device 2. The instruction includes information for specifying the data set DS used for learning. For example, the instruction includes the data set DS that the user sends as the data set DS. Hereinafter, for convenience of explanation, the processor 23 of the information processing device 2 will be simply referred to as the processor 23, and the processor 33 of the user terminal 3 will be simply referred to as the processor 33.

[Activity A2]
Next, in activity A2, the processor 23 acquires the data set DS based on the transmitted instruction. The data set DS may be input by the user or may be stored in the database DB. If the processor 23 has acquired a dataset DS in the past, it may acquire the dataset DS acquired in the past. Hereinafter, for convenience of explanation, the data set DS acquired by the processor 23 in activity A2 will be referred to as a first data set DS1. The first data set DS1 includes at least one data. The data may include any type of data, such as numerical data, time series data, audio data, image data, or combinations thereof. Hereinafter, for convenience of explanation, the data included in the first data set DS1 will be referred to as learning data D. The learning data D is used as teacher data input to the learning model M.

[Activity A3]
Next, the process proceeds to activity A3, where the processor 23 executes learning of the learning model M using the data set DS. Here, the processor 23 executes learning of the learning model M using the first data set DS1 acquired in activity A2. Thereby, the acquisition unit 231 acquires the learning result of the learning model M using the first data set DS1. The learning model M outputs a predetermined result by inputting the learning data D included in the data set DS. The specific aspect of the result output by the learning model M is arbitrarily determined according to the learning conditions, such as a prediction result by regression analysis, a classification result into another class, etc.

The processor 23 ends the learning when the evaluation index of the learning model M satisfies a predetermined condition. Although the evaluation index for the learning model M is arbitrary, it is preferably an index indicating the degree of applicability of the learning model M, such as a kappa coefficient, a coefficient of determination, an Akaike information criterion (AIC), or an R2 score. The predetermined condition is, for example, that at least one of these evaluation indicators is less than a predetermined threshold. Note that the learning end condition is not limited to this and is arbitrary. For example, the processor 23 may terminate learning when repeated processing has been performed for a predetermined number of epochs. Further, when the learning termination condition is satisfied, the processor 23 may accept a user's selection as to whether or not to terminate the learning.

The mode of learning is appropriately set depending on the type of learning data D, the task to be executed by learning model M, and the like. The learning results of this embodiment may include arbitrary information such as learning conditions, number of times of learning, and learning accuracy. The specific algorithm for learning the learning model M is arbitrary, such as supervised learning, unsupervised learning, and reinforcement learning. In the case of supervised learning, linear regression, logistic regression, random forest, boosting, support vector machine, neural network, etc. can be adopted as a learning algorithm for learning model M. In particular, when the learning data D includes image data, it is preferable to use a convolutional neural network (CNN). When the learning data D includes time-series data, it is preferable to use a recurrent neural network (RNN). On the other hand, in the case of unsupervised learning, the k-means method, principal component analysis, etc. can be adopted as the learning algorithm for the learning model M. The learning may be performed by the processor 23 itself or by a device other than the processor 23.

The learning result of this embodiment includes class information regarding the class to which each of the data (learning data D in this embodiment) classified by the learning model M and included in the first data set DS1 belongs. For example, a class may be expressed as a binary value (good or defective), but it may also be expressed as three or more indicators (for example, large, medium, small, etc.) that indicate the degree of deterioration. It's okay. Therefore, the learning model M of this embodiment can function as a classifier or a determiner that particularly classifies input data for each class. When the learning model M is generated by supervised learning, such a class is generated based on the label given to the dataset DS as supervised data. Moreover, when the learning model M is generated by unsupervised learning, the class is generated based on the label obtained as a learning result of the dataset DS.

[Activity A4]
Next, the process proceeds to activity A4, where the similarity calculation unit 232 calculates the similarity between classes based on the acquired class information. For example, the similarity calculation unit 232 determines a representative value of the learning data D belonging to each class, calculates the distance between the representative values in the feature space, and calculates the similarity between the classes based on the distance. . The similarity calculation unit 232 determines that the shorter the distance between two classes, the more similar the two classes are. The representative value is arbitrary, such as the centroid value or median value of the distribution of the learning data D, for example. Further, the specific form of the distance is arbitrary, such as Manhattan distance, Euclidean distance, and cosine similarity. The specific form of the degree of similarity is arbitrary, and the distance itself may be used as the degree of similarity, or the distance may be subjected to arbitrary calculation processing. Note that the similarity between classes is not limited to being calculated based on distance, and may be calculated based on a confusion matrix, for example.

[Activity A5]
Next, the process proceeds to activity A5, and the specifying unit 233 identifies the data region R that is part of the first data set DS1 and the learning model M that receives the learning data D as input, based on the calculated similarity. Identify the correlation between the output and The identification unit 233 identifying the correlation is one aspect of the acquisition unit 231 acquiring the correlation. Hereinafter, for convenience of explanation, a region showing a correlation between an input and an output specified in the data region R of the learning data D will be referred to as a correlation region R1. The identifying unit 233 identifies a correlation region R1 as a correlation between the data region R and the output of the learning model M that receives the learning data D as input. The correlation region R1 may include a pseudo correlation region R2 and a true correlation region R3.

<Pseudo correlation region R2>
The pseudo-correlation region R2 has a pseudo-correlation with the output of the learning model M, out of the correlated data region R. A pseudo-correlation is a state in which two events (input and output in this embodiment) are presumed to have a causal relationship due to the presence of a latent variable, even though there is no direct correlation between them. . A pseudo correlation is also called an apparent correlation.

<True correlation area R3>
The true correlation region R3 is a data region R that has a direct correlation with the output of the learning model M, among the correlated data regions R. Direct correlation means that there is a direct correlation between two events (input and output). It can also be said that the true correlation region R3 is a correlation region R1 other than the pseudo correlation region R2. Whether the correlation region R1 is a pseudo correlation region R2 or a true correlation region R3 may be determined from, for example, theory, a plurality of experimental facts, empirical rules, etc.

[Activity A6]
Next, the process proceeds to activity A6, and the specifying unit 233 specifies the pseudo correlation region R2 based on the first data set DS1 and the correlation. Specifically, the identifying unit 233 identifies the pseudo correlation region R2 from the identified correlation region R1. For example, the identifying unit 233 uses a previously learned determiner to determine whether the identified correlation region R1 is a pseudo correlation region R2 or a true correlation region R3. The learning of the determiner is performed using, for example, supervised data prepared in advance. The supervised data includes learning data D to which the correlation region R1 is linked, and a label indicating whether the linked correlation region R1 is a pseudo correlation region R2 or a true correlation region R3. The determiner receives such supervised data as input and outputs whether it is a pseudo correlation region R2 or a true correlation region R3. The specific mode of learning is arbitrary, and for example, the same mode as described in the learning of the learning model M can be mentioned.

[Activity A7]
Next, the process proceeds to activity A7, and the display processing unit 234 executes display processing. Thereby, the display processing unit 234 causes the display unit 34 to display the identified pseudo-correlation area R2 in a manner different from the data area R other than the pseudo-correlation area R2. The pseudo-correlation region R2 may be displayed in any manner as long as it can be visually distinguished by the user. For example, the pseudo-correlation region R2 may be surrounded by an outline unlike the other data regions R, or may be displayed in a different color from the other data regions R. The display processing unit 234 of this embodiment displays the learning results of the learning model M on the display unit 34 through display processing. Although the display mode of the learning results is arbitrary, the display processing unit 234 displays the learning results on the display unit 34 using a confusion matrix, a scatter diagram of the dataset DS, a total value of the learning data D for each class, etc. . The aggregated value of the learning data D includes, for example, the distribution of feature amounts for each class. In particular, when the learning data D includes image data, the display processing unit 234 causes the display unit 34 to display the learning results as an activation map. Any method can be used to generate the activation map, and examples thereof include CAM, Grad-CAM, Guided Grad-CAM, and the like.

[Activity A8]
Next, the process proceeds to activity A8, and the processor 33 causes the display unit 34 to display the result of the display process in activity A7. Thereby, the user can grasp the learning results of the learning model M, the data region R included in the learning data D used for learning the learning model M, especially the pseudo correlation region R2.

When the learning of the learning model M ends, this information processing ends. On the other hand, if learning of the learning model M is to be continued, the process proceeds to activity A9. The decision as to whether to end or continue learning the learning model M may be made by the user based on the learning results displayed in activity A8, or may be made automatically by the processor 23 based on the learning results.

[Activity A9]
In activity A9, the processor 33 transmits at least one designation of changes and annotations to the pseudo-correlation region R2 in the learning data D, input by the user. Here, the modification includes specific data processing performed on the pseudo-correlation region R2. Further, the processor 33 may transmit information specifying the pseudo correlation region R2 in which the data processing is performed. The annotation specifies how to use the pseudo correlation region R2 in learning using the learning model M. The annotation may be metadata indicating that the pseudo-correlation region R2 is to be excluded from the learning target by the learning model M, or may be weighting of the pseudo-correlation region R2 smaller than other regions in learning by the learning model M. It may also be metadata that specifies. Further, the annotation may be metadata specifying that the number of times the pseudo correlation region R2 is learned is less than a predetermined value in learning by the learning model M. Further, the annotation may indicate data processing to be performed before the learning data D is input to the learning model M. Note that data processing may include at least one of full processing and partial processing.

The overall processing is data processing performed on the entire learning data D including the pseudo-correlation region R2. The overall process includes, for example, a process of removing the learning data D including the pseudo-correlation region R2 from the first data set DS1, a pattern process uniformly performed on the entire learning data D, and the like. The pattern processing includes, for example, convolution of a predetermined pattern signal, subtraction of a predetermined noise pattern, and the like.

The partial processing is data processing performed only on the pseudo correlation region R2 in the learning data D. The partial processing includes, for example, deletion of the pseudo-correlation region R2, replacement with a prescribed pattern, mask processing, and the like. Furthermore, the partial processing may include any processing performed on at least a portion of the pseudo-correlation region R2. For example, the partial processing is not limited to data processing on the entire pseudo-correlation region R2, but may include data processing on a part of the pseudo-correlation region R2.

In particular, when the learning data D includes image data, the data processing includes image processing. Image processing includes overall processing such as deletion of the entire image including the pseudo-correlation region R2, pattern addition processing (for example, style transfer processing) for the entire learning data D including the pseudo-correlation region R2, and deletion of the pseudo-correlation region in the learning data D. This includes partial processing such as deletion of R2 and mask processing for the pseudo correlation region R2.

The specific manner of specifying the pseudo correlation region R2 for performing the data processing is arbitrary, such as a keyboard operation or a touch operation on the HMI device 35. Further, the specification is not limited to the entire one pseudo-correlation region R2, but may include specification of a partial region included in the pseudo-correlation region R2. An operation on the HMI device 35 generates a response from the HMI device 35.

[Activity A10]
Next, the process proceeds to activity A10, and the accepting unit 235 accepts the designation of at least one of changes and annotations to the pseudo correlation region R2 in the learning data D, transmitted from the processor 33.

[Activity A11]
Next, the process proceeds to activity A11, where at least one of data processing and annotation is performed on the pseudo correlation region R2 based on the designation received in activity A10. Thereby, the generation unit 236 generates the second data set DS2. In particular, when the first data set DS1 includes image data, the generation unit 236 generates the second data set DS2 by performing image processing as data processing on the image data including the pseudo-correlation region R2. generate. Image processing may include at least one of the above-mentioned overall processing and partial processing. For example, when the first data set DS1 includes image data, the generation unit 236 performs partial image processing (that is, partial processing included in image processing) on the pseudo correlation region R2.

After the processing of activity A11, the processing returns to activity A3, and the learning unit 237 performs learning of the learning model M using the second data set DS2. As a result, a learning model M with less influence of pseudo correlation can be obtained than when learning is performed using the first data set DS1. Therefore, the reliability of the learning model M can be improved. Thereafter, processing from activity A4 to activity A8 is performed. As a result, the processor 23 can perform data processing on the pseudo correlation region R2 on the learning model M using the second data set DS2 in the same way as the previous information processing on the first data set DS1. It is configured.

[Activity A12]
Note that after the processing of activity A11, in activity A12, the display processing unit 234 causes the display unit 34 to display information regarding the second data set DS2. The information regarding the second data set DS2 includes, for example, the content of the data processing performed on the learning data D, the results of the data processing, the difference between the first data set DS1 and the second data set DS2, and the like. This makes it easier for the user to understand the learning course of the learning model M.

3.2. Details of Information Processing In this section, details of the above information processing when the learning data D is image data will be explained. The learning data D of this embodiment is an electron microscope image (SEM image) of the electrode material, and includes an image of crystal domains of the electrode material. Each learning data D is associated with the degree of deterioration of the electrode material (for example, high degree of deterioration, medium degree of deterioration, and small degree of deterioration). The first data set DS1 includes the plurality of learning data D described above. The learning model M that has been trained based on the first data set DS1 outputs the degree of deterioration of the electrode material by inputting an electron microscope image of the electrode material. At this time, the specifying unit 233 specifies the data region R to be used as the material for determining the degree of deterioration to be output as the correlation region R1. Thereafter, the specifying unit 233 specifies whether the correlation region R1 is a pseudo correlation region R2 or a true correlation region R3, and the display processing unit 234 displays the correlation region R1 together with the learning results in activity A7 and activity A8. The display section 34 displays information related to the above.

FIG. 6 is a diagram showing an example of the learning data D including the pseudo correlation region R2, which is displayed on the display unit 34 as a result of the display processing of the activity A7. As a result of the processing from activity A3 to activity A6, the identifying unit 233 determines that the learning data D includes two correlation regions R1, one of the two correlation regions R1 is a pseudo correlation region R2, and the correlation region R1 is It is specified that the other is the true correlation region R3. The pseudo-correlation region R2 identified in this embodiment is, for example, a region where separator marks remain due to contact between the electrode material and the separator.

Each of the identified correlation regions R1 is surrounded by a rectangular frame as an annotation. Thereby, the correlation region R1 is displayed so as to be visible to the user. Specifically, the identified pseudo-correlation region R2 is surrounded by a solid-line rectangular frame, and the true correlation region R3 is surrounded by a broken-line rectangular frame. Note that the data areas R other than the correlation area R1 are not surrounded by such a rectangular frame. Such a display mode allows the user to visually recognize which region in the learning data D contributes to the determination of the deterioration state as the correlation region R1, and which correlation region R1 is the pseudo-correlation region R2. . Note that the processor 23 may be configured to be able to change whether the identified correlation region R1 is a pseudo correlation region R2 or a true correlation region R3 by a user's operation. In other words, the identifying unit 233 may identify candidates for the pseudo-correlation region R2 from the correlation region R1. This makes it easy to modify the identification result of the pseudo correlation region R2 based on the user's empirical rules. This operation can be realized using any input to the HMI device 35, such as mouse operation, touch operation, keyboard operation, voice, line of sight, and gesture. Due to this change, the display mode of the rectangular frame surrounding the correlation region R1 may be changed. Thereafter, data processing is performed on the learning data D including the identified pseudo-correlation region R2 based on the input mode specified in the activity A9 and the activity A10.

Next, details of data processing of the activity A11 on the learning data D will be explained. FIG. 7 is a diagram illustrating an example of data processing for learning data D. The data processing in this embodiment is image processing on image data included in the learning data D, particularly mask processing. Through the image processing, the generation unit 236 generates at least one learning data D in which the identified pseudo-correlation region R2 is masked, and a second data set DS2 including the learning data D. Note that the generation unit 236 performs the data processing only on the pseudo correlation region R2 among the identified correlation regions R1, and does not perform the data processing on the true correlation region R3. The learning model M is retrained using the second data set DS2 that includes the learning data D that has been subjected to such mask processing. Therefore, the second data set DS2 can be said to be a data set DS in which the input mode to the learning model M has been changed by mask processing. Note that the contents of the second data set DS2 including the learning data D after the data processing are displayed to the user via the display unit 34 in activity A12.

4. Others The above information processing mode is just an example, and is not limited to this.

The method for identifying the pseudo-correlation region R2 in activity A6 is not limited to using a determiner, but may be any method. The specifying unit 233 uses a relational expression or a lookup table that expresses the correlation between the input and the output that has been obtained in advance, and determines whether the output of the learning model M with respect to the input is a predetermined threshold value between the input and the output based on the relational expression. It may be determined whether the correlation region R1 is a pseudo correlation region R2 or a true correlation region R3 based on whether the correlation region R1 is larger than the above. Further, the identifying unit 233 may identify the pseudo correlation region R2 based on the user's operation on the correlation region R1. The user's input includes, for example, an operation indicating that the correlation region R1 is a pseudo correlation region R2, and an operation indicating that the correlation region R1 is a true correlation region R3. Further, the learning of the determiner may be performed as supervised data using learning data D in which whether the correlation region R1 is a pseudo correlation region R2 or a true correlation region R3 is specified by the operation of the user. .

If there is a learning model M trained using the first dataset DS1, the acquisition unit 231 acquires the learning model M, and performs transfer learning using the second dataset DS2 on the acquired learning model M. You may do so.

The data included in the first data set DS1 is not limited to the learning data D. For example, the data included in the first data set DS1 may be evaluation data. The evaluation data is, for example, data used to evaluate the performance of the learning model M. In other words, the use of the data included in the first data set DS1 is arbitrary, and the data may be used as teacher data or used to evaluate the learning model M.

If a response from the HMI device 35 is generated in activity A9, the acquisition unit 231 may acquire the response from the HMI device 35. In this case, the identifying unit 233 may further identify the pseudo correlation region R2 based on the response from the HMI device 35 acquired by the acquiring unit 231, the first data set DS1, and the correlation. For example, the processor 23 learns the determiner used to identify the pseudo correlation region R2 in activity A5 based on the response from the HMI device 35 acquired by the acquisition unit 231, and uses the learned determiner to The pseudo correlation region R2 may also be specified. In this case, the processor 23 treats the second data set DS2 generated based on the response from the HMI device 35 as the first data set DS1, and trains the determiner using the first data set DS1. It's okay.

The information processing device 2 may be in an on-premises form or may be in a cloud form. The cloud-based information processing device 2 may provide the above-mentioned functions and processing, for example, in the form of SaaS (Software as a Service) or cloud computing.

In the embodiment described above, the information processing device 2 performs various storage and control operations, but instead of the information processing device 2, a plurality of external devices may be used. That is, various information and programs may be distributed and stored in a plurality of external devices using blockchain technology or the like.

The embodiment described above is not limited to the information processing system 1, and may be an information processing method or an information processing program. The information processing method includes each step of the information processing system 1. The information processing program causes at least one computer to execute each step of the information processing system 1.

The information processing system 1 and the like may be provided in each of the following aspects.

(1) An information processing system, comprising at least one processor capable of executing a program so as to perform each of the following steps; A correlation between a data region that is part of the first data set as an input and an output of the learning model is obtained, and in the specifying step, based on the first data set and the correlation, A pseudo-correlation area is identified, where the pseudo-correlation area is one of the data areas having the correlation that has a pseudo-correlation with the output of the learning model, and in the generation step, the pseudo-correlation area is at least one of modification and annotation to the data to generate a second data set.

According to such a configuration, the learning results of the learning model differ depending on the input correspondence of the pseudo-correlation region depending on whether the first data set is used or the second data set is used. Therefore, by using the second data set when learning the learning model, it becomes easier for the user to understand the influence of the pseudo-correlation region on the learning results using the first data set.

(2) In the information processing system according to (1) above, in the acquiring step, a response from the HMI device is further acquired, and in the identifying step, the acquired response from the HMI device and the first The pseudo-correlation area is identified based on the data set and the correlation.

According to such a configuration, it becomes easier to reflect the user's empirical rules in identifying the pseudo-correlation area via the response from the HMI device.

(3) In the information processing system according to (1) or (2) above, the accepting step further accepts a user's designation of at least one of the change and the annotation.

According to such a configuration, the user's subjective judgment regarding the pseudo-correlation area, such as whether or not the identified pseudo-correlation area truly has pseudo-correlation and how strong the pseudo-correlation is, is This can be reflected in the input mode of the pseudo-correlation area. Therefore, by learning the learning model using the second data set generated in this way, it is possible to obtain learning results that are more consistent with the user's subjectivity.

(4) In the information processing system according to any one of (1) to (3) above, in the display processing step, the identified pseudo-correlation area is displayed in a manner different from the data area other than the pseudo-correlation area. Something to display.

According to such a configuration, it becomes easier for the user to visually determine which data area is a pseudo-correlation area.

(5) In the information processing system according to any one of (1) to (4) above, the learning result includes class information regarding a class to which data included in the data set belongs, which is classified by the learning model. Further, in the similarity calculation step, the similarity between the classes is calculated based on the obtained class information, and in the identification step, the correlation is further specified based on the calculated similarity. Something to do.

According to such a configuration, a pseudo correlation region is identified based on a quantitative index of similarity between classes, so that the objectivity of the identification result is improved.

(6) In the information processing system according to any one of (1) to (5) above, when the first data set includes image data, in the generation step, the image including the pseudo-correlation area The second data set is generated by performing image processing on the data.

According to such a configuration, it becomes easier for the user to visually understand the difference between the first data set and the second data set, and therefore it becomes easier for the user to understand the influence of the pseudo-correlation area.

(7) In the information processing system according to (6) above, in the generation step, the second data set is generated by performing partial image processing on the pseudo-correlation area.

According to such a configuration, it is possible to reduce the possibility that image processing will affect an area that is not a pseudo-correlation area, especially a data area that has a true correlation. Therefore, it becomes easier for the user to more accurately grasp the influence of the pseudo-correlation area.

(8) The information processing system according to any one of (1) to (7) above, further comprising, in the learning step, learning the learning model using the second data set.

According to such a configuration, it is possible to obtain a learning model with higher reliability than when using the first data set.

(9) An information processing method, comprising each step of the information processing system described in any one of (1) to (8) above.

(10) An information processing program that causes at least one computer to execute each step of the information processing system described in any one of (1) to (8) above.
Of course, this is not the case.

Finally, although various embodiments according to the present disclosure have been described, these are presented as examples and are not intended to limit the scope of the invention. The new embodiment can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. The embodiment and its modifications are included within the scope and gist of the invention, and are included within the scope of the invention described in the claims and its equivalents.

1: Information processing system 2: Information processing device 3: User terminal 20: Communication bus 21: Communication unit 22: Storage unit 23: Processor 30: Communication bus 31: Communication unit 32: Storage unit 33: Processor 34: Display unit 35: HMI device 231: Acquisition unit 232: Similarity calculation unit 233: Specification unit 234: Display processing unit 235: Reception unit 236: Generation unit 237: Learning unit DB1: Database R: Data area R1: Correlation area R2: Pseudo correlation area R3 : Correlation area

Claims (10)

An information processing system,
at least one processor capable of executing a program such that each of the following steps is performed;
In the acquisition step,
Learning results of the learning model using the first data set,
obtaining a correlation between a data region that is part of the first data set as an input and the output of the learning model;
In the identifying step, a pseudo-correlation region is identified based on the first data set and the correlation, and here, the pseudo-correlation region is defined as a region between the data region having the correlation and the output of the learning model. It has a pseudo correlation,
In the generation step, at least one of modification and annotation is performed on the pseudo-correlation region to generate a second data set. The information processing system according to claim 1,
In the obtaining step, further obtaining a response from the HMI device,
In the identifying step, the pseudo-correlation area is identified based on the obtained response from the HMI device, the first data set, and the correlation. The information processing system according to claim 1,
Furthermore, in the receiving step, a designation of at least one of the change and the annotation is received by the user. The information processing system according to claim 1,
In the display processing step, the identified pseudo-correlation area is displayed in a manner different from the data area other than the pseudo-correlation area. The information processing system according to claim 1,
The learning result includes class information regarding a class to which data included in the dataset belongs, classified by the learning model,
Furthermore, in the similarity calculation step, the similarity between the classes is calculated based on the acquired class information,
In the identifying step, the correlation is further identified based on the calculated similarity. The information processing system according to claim 1,
When the first data set includes image data,
In the generation step, the second data set is generated by performing image processing on the image data including the pseudo-correlation area. The information processing system according to claim 6,
In the generation step, the second data set is generated by performing partial image processing on the pseudo-correlation area. The information processing system according to claim 1,
Furthermore, in the learning step, the learning model is trained using the second data set. An information processing method,
A method comprising each step of the information processing system according to any one of claims 1 to 8. An information processing program,
A device that causes at least one computer to execute each step of the information processing system according to any one of claims 1 to 8.