US20220374648A1 - Computer-readable recording medium storing learning support program, learning support method, and learning support device - Google Patents

Computer-readable recording medium storing learning support program, learning support method, and learning support device Download PDF

Info

Publication number
US20220374648A1
US20220374648A1 US17/863,511 US202217863511A US2022374648A1 US 20220374648 A1 US20220374648 A1 US 20220374648A1 US 202217863511 A US202217863511 A US 202217863511A US 2022374648 A1 US2022374648 A1 US 2022374648A1
Authority
US
United States
Prior art keywords
clusters
distance
training
distance metric
metric model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US17/863,511
Other languages
English (en)
Inventor
Yuji MIZOBUCHI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujitsu Ltd
Original Assignee
Fujitsu Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujitsu Ltd filed Critical Fujitsu Ltd
Assigned to FUJITSU LIMITED reassignment FUJITSU LIMITED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MIZOBUCHI, YUJI
Publication of US20220374648A1 publication Critical patent/US20220374648A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G06K9/6215—
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06K9/6218—
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0454—
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G06N5/045—Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence

Definitions

  • the embodiments discussed herein are related to a learning support program, a learning support method, and a learning support device.
  • a non-transitory computer-readable recording medium stores a training support program for causing a computer to execute a process including: calculating a first embedded vector for each of clusters obtained by clustering samples included in training data, by inputting the samples that represent the clusters to a first distance metric model; performing training of a second distance metric model from the first distance metric model, based on labels set in pairs of the samples included in the training data; calculating a second embedded vector for each of the clusters, by inputting the samples that represent the clusters to the second distance metric model; detecting pairs of the clusters that are likely to be integrated when the training is performed with a greater number of epochs than a number of epochs at a time of the training of the second distance metric model, based on the first embedded vector of each of the clusters and the second embedded vector of each of the clusters; and outputting one of the pairs of the clusters in which a similarity label is not set, among the pairs of the clusters
  • FIG. 1 is a diagram illustrating a configuration example of a system according to a first embodiment
  • FIG. 2 is a diagram illustrating an example of a multi-class classification model
  • FIG. 3 is a diagram illustrating an example of the structure of a Siamese Network
  • FIG. 4 is a diagram illustrating an example of a metric space
  • FIG. 5 is a diagram illustrating an example of the metric space
  • FIG. 6 is a diagram illustrating an example of document data
  • FIG. 7 is a diagram illustrating an example of word string extraction results
  • FIG. 8 is a diagram illustrating an example of a label setting screen
  • FIG. 9 is a diagram illustrating an example of embedded vectors
  • FIG. 10 is a diagram illustrating an example of the embedded vectors
  • FIG. 11 is a diagram illustrating an example of an inquiry screen
  • FIG. 12 is a diagram illustrating an example of the label setting screen
  • FIG. 13 is a flowchart illustrating a procedure of training support processing according to the first embodiment.
  • FIG. 14 is a diagram illustrating a hardware configuration example of a computer.
  • a learning support program, a learning support method, and a learning support device according to the present application will be described below with reference to the attached drawings. Note that the present embodiments do not limit the disclosed technique. Additionally, each of the embodiments may be suitably combined within a range without causing contradiction between processing contents.
  • FIG. 1 is a diagram illustrating a configuration example of a system according to a first embodiment.
  • a system 1 illustrated in FIG. 1 provides a function corresponding to any machine learning task such as class classification, merely as one aspect.
  • the system 1 may include a server device 10 and a client terminal 30 . These server device 10 and client terminal 30 are connected so as to be capable of communicating with each other via a network NW.
  • the network NW may be an optional type of communication network such as the Internet or a local area network (LAN) regardless of whether the network NW is wired or wireless.
  • LAN local area network
  • the server device 10 is an example of a computer that executes the above-mentioned class classification.
  • the server device 10 may correspond to the learning support device.
  • the server device 10 may be implemented by installing a classification program that achieves the function corresponding to the above-mentioned class classification on any computer.
  • the server device 10 may be implemented as a server that provides a function relating to the above-mentioned class classification on-premises.
  • the server device 10 may also be implemented as a software as a service (SaaS) type application to provide a function corresponding to the above-mentioned class classification as a cloud service.
  • SaaS software as a service
  • the client terminal 30 corresponds to an example of a computer that is provided with a function corresponding to the above-mentioned class classification.
  • a desktop computer such as a personal computer, or the like corresponds to the client terminal 30 .
  • the client terminal 30 may be an optional computer such as a laptop computer, a mobile terminal device, or a wearable terminal.
  • Examples of a function in which the above-mentioned class classification task can be implemented include a similarity discrimination function between documents that discriminates similarity or dissimilarity between two documents.
  • the above-mentioned similarity discrimination function between documents allows the construction of a failure isolation graph in which similar case samples between trouble events and trouble causes are associated from past case collections relating to operational management of information technology (IT) services, or the like.
  • the failure isolation graph constructed in this manner may achieve a function of outputting a recommendation of the trouble cause corresponding to the trouble event when dealing with a trouble, or the like.
  • Distance metric learning represented by the Mahalanobis distance learning is known as an example of a method of learning the feature amount space.
  • distance metric learning a transformation is learned in which the degree of similarity between samples in the input space is made to correspond to the distance in the feature amount space.
  • the original space is distorted such that the distance between samples belonging to the same class becomes closer and the distance between samples belonging to different classes becomes farther.
  • feature amount space is referred to as a metric space or an embedded space in some cases.
  • a Mahalanobis distance d M(x, x′) indicated by following formula (1) is defined, and learning is performed with the component of M as a design variable.
  • Such an optimization problem of M is equivalent to learning a transformation L that makes the Euclidean distance between samples correspond to the degree of similarity between samples. From this, the transformation L can be learned by solving the problem of minimizing the loss function in following formula (2).
  • ⁇ pull (L) is denoted by following formula (3)
  • ⁇ push (L) is denoted by following formula (4).
  • “j (arrow symbol) i” means that x j is in the neighborhood of x i .
  • ⁇ pull (L) and ⁇ push (L) k neighborhood instances j of a certain instance i are used based on the k-nearest neighbor method.
  • ⁇ pull (L) indicated in above formula (3) a penalty is given when the distance between instances having the same label is large.
  • ⁇ push (L) indicated in above formula (4) a penalty is given when the distance between instances having different labels is small.
  • the degree of importance of the feature amount there is a method of learning the diagonal component of M in the Mahalanobis distance. For example, from the aspect of coping with the scalability of learning data and feature amount, by altering the elements other than the diagonal component of M in the Mahalanobis distance to zero, the relationship between different feature amounts is ignored, and the degree of importance of each feature amount is learned.
  • a method using a decision tree is known as a method for finding the degree of importance of the feature amount.
  • the decision tree is generated by repeating the selection of a feature amount that divides nodes.
  • the degree of importance is calculated for each feature amount.
  • the degree of importance FI(f j ) of the j-th feature amount f j can be calculated by computing the sum of information gains I at all the nodes as in following formula (5).
  • the “information gains I” in above formula (5) refer to the amount of information obtained when division from a parent node to child nodes happens.
  • the “parent node” mentioned here refers to a node before being divided by the branch of the feature amount, whereas the “child node” refers to a node after being divided by the branch of the feature amount.
  • the amount of information I(D p , f) obtained when division by the branch of the feature amount happens in the decision tree can be denoted by following formula (6).
  • “f” refers to the feature amount selected for the branch.
  • “D p ” refers to the parent node.
  • “D left ” refers to a child node on the left side after branching
  • “D right ” refers to a child node on the right side after branching.
  • N p refers to the number of samples in the parent node.
  • N left refers to the number of samples of the child node on the left side
  • “D right ” refers to the number of samples of the child node on the right side.
  • I ⁇ ( D p , f ) I ⁇ ( D p ) - N left N p ⁇ I ⁇ ( D left ) - N right N p ⁇ I ⁇ ( D right ) ( 6 )
  • a feature amount that maximizes such an amount of information which is a feature amount capable of decreasing the impureness between the parent node and the child node to the maximum, is selected for the branch of the decision tree.
  • the Gini coefficient, entropy, and the like can be used as the above amount of information.
  • a Gini coefficient I G (t) can be calculated by following formula (7).
  • an entropy I H (t) can be calculated by following formula (8).
  • the clustering distance learning device performs the following processing instead of the algorithm described in E. P. Xing, A. Y. Ng, M. I. Jordan and S. Russell: “Distance metric learning, with application to clustering with side-information”, Neural Information Processing Systems (NIPS) (2003), which performs iterative arithmetic operation until the Mahalanobis distance matrix A converges.
  • NIPS Neural Information Processing Systems
  • the Mahalanobis distance matrix A is computed in accordance with following formula (9) obtained by such a formulation.
  • A ⁇ n ⁇ k r nk ( x n - ⁇ k ) ⁇ ( x n - ⁇ k ) T ( tr ⁇ ( ⁇ n , k ⁇ n ′ , k ′ r nk ⁇ r n ′ ⁇ k ′ ( x n - ⁇ k ) ⁇ ( x n - ⁇ k ) T ⁇ ( x n ′ - ⁇ k ′ ) ⁇ ( x n ′ - k ′ ) T ) ) 1 / 2 ( 9 )
  • x n and “x n ” refer to the feature amount of the instances.
  • ⁇ k ” and “ ⁇ k ,” refer to the center of the cluster k or cluster k′.
  • r nk ” and “r nk ” denote the correspondence between the instance and the cluster.
  • “r nk ” is denoted by “1” when the instance x n belongs to the cluster k, while it is denoted by “0” in other cases.
  • r n′k′ is denoted by “1” when the instance x n ′ belongs to the cluster k′, while it is denoted by “0” in other cases.
  • tr(A T A) refers to the constraint condition for the matrix A.
  • the above distance metric learning is not limited to the example of learning the linear transformation to the feature amount space as in the above-described Mahalanobis distance learning, and a non-linear transformation to the feature amount space can also be learned by applying a neural network to the distance definition part of the class classification model.
  • FIG. 2 is a diagram illustrating an example of the multi-class classification model.
  • FIG. 2 illustrates an example of a multi-class classification model 2 that predicts the label of the class to which an instance of input data belongs.
  • model learning is performed in which the distance between hidden vectors output from the hidden layer (intermediate layer) of the i-th layer becomes closer between the learning samples.
  • the hidden vector input to any hidden layer included in the multi-class classification model 2 that has finished learning such as the hidden layer of the n-1-th layer or the n-th layer illustrated in FIG. 2 , may be regarded as being transformed to a position on the metric space corresponding to the label of the class to which the input data belongs. From this, the hidden vector input to any hidden layer can be used as an embedded vector.
  • the Siamese Network is known as an example of the deep metric learning.
  • the Siamese Network learns a function that maps input data over an appropriate metric space non-linearly while performing dimensional reduction based on similar or dissimilar pairs.
  • FIG. 3 is a diagram illustrating an example of the structure of the Siamese Network.
  • the Siamese Network inputs a pair of two samples assigned with the similarity or dissimilarity label to two neural networks NN 1 and NN 2 .
  • the parameters and the layer structure are shared between the neural networks NN 1 and NN 2 to which the pair of two samples are input in this manner.
  • the distance between the samples found from the embedded vector output by the neural network NN 1 and the embedded vector output by the neural network NN 2 is output as the degree of similarity.
  • parameters of the neural networks NN 1 and NN 2 that bring the distance of the similar pair closer while bringing the distance of the dissimilar pair farther based on the similarity or dissimilarity label are learned.
  • a model in which embedding into the metric space, which is distance metric, is performed such as the neural networks NN 1 and NN 2 of the Siamese Network as an example, will be described as a “distance metric model” in some cases.
  • fine-tune or the like is performed in some cases to cause a distance metric model that has finished learning to relearn using new data.
  • the label set among pieces of data of the learning samples at the time of relearning of the distance metric model mentioned thus far adversely affects the model after relearning in some cases.
  • FIG. 4 is a diagram illustrating an example of the metric space.
  • a metric space S 1 embedded by the distance metric model before relearning and a metric space S 2 embedded by the distance metric model after relearning are illustrated side by side in order from the left.
  • clustering results obtained by clustering each sample of the learning data used for relearning, based on the embedded vectors before relearning or after relearning are illustrated.
  • the metric space S 1 before relearning includes six clusters, namely, a cluster C 1 to a cluster C 6 .
  • the embedding into the metric space S 2 is relearned.
  • the distance between the clusters C 1 and C 2 is made closer than the distance in the metric space S 1 because of the relearning based on the similarity label set in the pair of the clusters C 1 and C 2 .
  • the distance between the clusters C 3 and C 4 is also made closer than the distance in the metric space S 1 .
  • the integration of these clusters C 3 and C 4 is occasionally not intended by the model designer, and an unexpected model may be obtained by relearning.
  • setting the similarity or dissimilarity label to all pairs of clusters at the time of relearning may occasionally not be deemed to be realistic from the viewpoint of resources, and there is an aspect in which setting labels is desired to be kept to the minimum.
  • the present embodiment provides a learning support function that outputs a pair of clusters that come closer to each other and do not have the similarity label set, based on the embedded vectors output by a distance metric model at each of time points before relearning and in the relearning process.
  • a learning support function that outputs a pair of clusters that come closer to each other and do not have the similarity label set, based on the embedded vectors output by a distance metric model at each of time points before relearning and in the relearning process.
  • FIG. 5 is a diagram illustrating an example of the metric space.
  • a metric space S 11 embedded by the distance metric model before relearning a metric space S 12 embedded by the distance metric model in the relearning process
  • a metric space S 13 embedded by the distance metric model after relearning are illustrated side by side in order from the left.
  • clustering results obtained by clustering each sample of the learning data used for relearning, based on the embedded vectors before relearning, in the relearning process, or after relearning are illustrated.
  • the metric space S 11 before relearning includes six clusters, namely, a cluster C 1 to a cluster C 6 .
  • the above-mentioned learning support function performs relearning in a state in which the similarity label is set in the pair of the clusters C 1 and C 2 among the above six clusters.
  • the above-mentioned learning support function performs relearning having a number of epochs in the relearning process smaller than a demanded number of epochs such as the number of epochs at which the value of the loss function converges or the number of epochs at which the correct answer rate of the test data reaches a fixed value, an example of which is one epoch.
  • a demanded number of epochs such as the number of epochs at which the value of the loss function converges or the number of epochs at which the correct answer rate of the test data reaches a fixed value, an example of which is one epoch.
  • the above-mentioned learning support function detects a pair of clusters that come closer to each other and do not have the similarity label set, based on the embedded vectors output by the distance metric model at each of time points before relearning and in the relearning process.
  • an embedded vector EV of a cluster representative is found for each of the clusters C 1 to C 6 .
  • the embedded vector EV of the cluster representative the average of the embedded vectors obtained by inputting samples belonging to the cluster to the distance metric model before relearning or in the relearning process can be used.
  • the above-mentioned learning support function calculates a moving direction of the cluster.
  • the moving direction of the cluster can be found by computation of subtracting the embedded vector of the cluster representative before relearning from the embedded vector of the cluster representative in the relearning process.
  • the above-mentioned learning support function extracts a pair of clusters in which the moving directions of the two clusters exist on a substantially the same straight line, in accordance with following formula (10).
  • “delta_EV_cluster1” refers to the moving direction of the cluster C 1 .
  • delta_EV_cluster2 refers to the moving direction of the cluster C 2 .
  • ⁇ 1 refers to a threshold value. A pair of clusters satisfying such formula (10) is extracted.
  • the above-mentioned learning support function calculates the distance between clusters at each of time points before relearning and in the relearning process.
  • the distance between clusters can be found by computing the Euclidean distance or cosine distance of the embedded vectors of the cluster representatives for each pair of clusters.
  • the above-mentioned learning support function calculates the amount of change in the distance between clusters between before relearning and in the relearning process.
  • the amount of change between before relearning and in the relearning process can be found by computation of dividing the distance between clusters in the relearning process by the distance between clusters before relearning. A pair of clusters in which the amount of change calculated between before relearning and in the relearning process in this manner is less than a predetermined threshold value such as ⁇ 2 is extracted.
  • a pair of clusters in which the similarity label is not set is detected as inquiry targets that are likely to be integrated after relearning.
  • the integration suitability of the pair of clusters may be accepted.
  • the pair of the clusters C 1 and C 2 and the pair of the clusters C 3 and C 4 are narrowed down using ⁇ 1 and ⁇ 2.
  • the pair of the clusters C 1 and C 2 in which the similarity label is set is excluded from the inquiry targets.
  • the pair of the clusters C 3 and C 4 in which the similarity label is not set are detected as inquiry targets.
  • Such a pair of the clusters C 3 and C 4 are output to the client terminal 30 or the like, and the integration suitability of the pair of clusters, such as stop of relearning or resetting of labels as an example, is accepted. For example, a request to stop relearning is accepted.
  • the dissimilarity label is set in the pair of the clusters C 3 and C 4 , and additionally, the similarity label is set in the pair of the clusters C 5 and C 6 . This makes it possible to adjust the environment for performing relearning while suppressing the integration of clusters that is not intended by the model designer.
  • the above-mentioned learning support function performs relearning having the demanded number of epochs, based on the reset labels. For example, relearning is performed in a state in which the similarity label is set in the pair of the clusters C 1 and C 2 , the dissimilarity label is set in the clusters C 3 and C 4 , and additionally, the similarity label is set in the clusters C 5 and C 6 .
  • the distance metric model before relearning may be used, or alternatively, the distance metric model in the relearning process may be used.
  • the embedding into the metric space S 13 is relearned.
  • the distance between the clusters C 1 and C 2 is made closer than the distances in the metric space S 11 and the metric space S 12
  • the distance between the clusters C 5 and C 6 is made closer than the distances in the metric space S 11 and the metric space S 12 .
  • FIG. 1 illustrates an example of the functional configuration of the server device 10 according to the first embodiment.
  • the server device 10 includes a communication interface 11 , a storage unit 13 , and a control unit 15 .
  • the server device 10 includes a communication interface 11 , a storage unit 13 , and a control unit 15 .
  • FIG. 1 Note that, while solid lines denoting relations of data exchange are illustrated in FIG. 1 , only a minimum part is illustrated for convenience of explanation.
  • input and output of data relating to each processing unit are not limited to the illustrated example, and input and output of data other than those illustrated, for example, input and output of data between a processing unit and another processing unit, between a processing unit and data, and between a processing unit and an external device, may be performed.
  • the communication interface 11 is an interface that performs control of communication with another device, which is, for example, the client terminal 30 .
  • a network interface card such as a local area network (LAN) card may be adopted for the communication interface 11 .
  • the communication interface 11 accepts a label setting, a relearning execution instruction, a relearning stop instruction, or the like from the client terminal 30 .
  • the communication interface 11 for example, transmits a pair of clusters as inquiry targets as to the integration to the client terminal 30 .
  • the storage unit 13 is a functional unit that stores data to be used in various programs including an operating system (OS) executed by the control unit 15 .
  • OS operating system
  • the above programs may correspond to not only a learning support program in which the above learning support function is modularized, but also packaged software in which the learning support program is packaged in the above-mentioned classification program, and the like.
  • the storage unit 13 may correspond to an auxiliary storage device in the server device 10 .
  • a hard disk drive (HDD), an optical disc, a solid state drive (SSD), or the like corresponds to the auxiliary storage device.
  • a flash memory such as an erasable programmable read only memory (EPROM) may also correspond to the auxiliary storage device.
  • EPROM erasable programmable read only memory
  • the storage unit 13 stores first model data 13 M 1 and learning data 14 as an example of data used in the program executed by the control unit 15 .
  • first model data 13 M 1 and learning data 14 data referenced by the above learning support program, examples of which include relearning conditions such as the demanded number of epochs and the number of epochs in the relearning process, may be stored in the storage unit 13 .
  • the first model data 13 M 1 is data of the distance metric model before relearning.
  • the “distance metric model before relearning” mentioned here may correspond to a distance metric model that has finished learning using learning data different from the learning data used for relearning, as a mere example.
  • the first model data 13 M 1 not only the model layer structure such as neurons and synapses in each layer among the input layer, hidden layer, and output layer forming the Siamese Network, but also the model parameters such as the weight and bias of each layer are stored in the storage unit 13 .
  • the learning data 14 is data used for relearning of the distance metric model.
  • the learning data 14 may include document data 14 A and label data 14 B.
  • the document data 14 A is data of documents.
  • the “documents” mentioned here may correspond to an example of samples input to the distance metric model.
  • FIG. 6 is a diagram illustrating an example of the document data 14 A.
  • FIG. 6 exemplifies ten documents, namely, a document D 1 to a document D 10 , as a mere example.
  • the cluster C 1 contains the documents D 1 to D 3 .
  • the cluster C 2 contains the documents D 4 to D 6 .
  • the cluster C 3 contains the documents D 7 and D 8 .
  • the cluster C 4 contains the documents D 9 and D 10 .
  • FIG. 6 illustrates text data as an example of the document data 14 A, but the text data is transformed to a numerical expression that can be input to the distance metric model, such as a vector expression, as preprocessing of input to the distance metric model.
  • a numerical expression that can be input to the distance metric model, such as a vector expression, as preprocessing of input to the distance metric model.
  • Bag of words or the like can be used for such transformation to the numerical expression.
  • the following processing is performed for each of the documents D 1 to D 10 .
  • the word string of the content word is extracted from the word string of the sentence obtained by applying the morphological analysis to the text of the natural language.
  • FIG. 7 is a diagram illustrating an example of word string extraction results.
  • FIG. 7 illustrates the word string extraction results for each of the documents D 1 to D 10 illustrated in FIG. 6 .
  • function words are excluded from the word strings corresponding to the sentences of the documents D 1 to D 10 , and additionally, particular expressions such as date and time are excluded as stop words.
  • the word strings of the content words are extracted.
  • a dictionary of all the documents from the document D 1 to the document D 10 is generated. For example, a dictionary containing words such as “monitor”, “AP server”, “DB server”, “failure”, “error”, “occur”, “VEO000481436” and “VEO000481437” is generated.
  • a vector such as ⁇ monitor: 1, AP server: 1, DB server: 0, failure: 1, error: 1, occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇ is obtained.
  • a vector such as ⁇ monitor: 1, AP server: 1, DB server: 0, failure: 0, error: 1, occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇ is obtained.
  • a vector such as ⁇ monitor: 1, AP server: 0, DB server: 1, failure: 1, error: 1, occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇ is obtained.
  • a vector such as ⁇ monitor: 1, AP server: 0, DB server: 1, failure: 0, error: 1, occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇ is obtained.
  • a vector such as ⁇ monitor: 1, AP server: 1, DB server: 0, failure: 1, error: 1, occur: 1, VEO000481436: 0, VEO000481437: 1 ⁇ is obtained.
  • a vector such as ⁇ monitor: 1, AP server: 0, DB server: 1, failure: 1, error: 1, occur: 1, VEO000481436: 0, VEO000481437: 1 ⁇ is obtained.
  • the eight-dimensional vectors of the documents D 1 to D 10 obtained by such preprocessing may be input to the distance metric model.
  • the vectors of the documents D 1 to D 10 can be generically referred to without distinction from each other, the vectors of the documents D 1 to D 10 will be described as “document vectors” in some cases.
  • the label data 14 B is data relating to labels set in pairs of clusters.
  • the label data 14 B may be generated by accepting a label setting from the client terminal 30 .
  • the label setting can be accepted via a label setting screen 200 illustrated in FIG. 8 .
  • FIG. 8 is a diagram illustrating an example of the label setting screen 200 .
  • a clustering result of the embedded vectors obtained by inputting the vectors of the documents D 1 to D 10 to the distance metric model before relearning is displayed on the label setting screen 200 .
  • the clusters C 1 to C 4 are displayed on the label setting screen 200 .
  • the cluster C 1 contains the documents D 1 to D 3 .
  • the cluster C 2 contains the documents D 4 to D 6 .
  • the cluster C 3 contains the documents D 7 and D 8 . Furthermore, the cluster C 4 contains the documents D 9 and D 10 . Along with displaying these clusters C 1 to C 4 , the label setting screen 200 displays the distances of the embedded vectors between the documents in the cluster.
  • An operation of assigning the similarity label to a pair of clusters is accepted on such a label setting screen 200 .
  • the similarity label can be set in a pair of documents by a drag-and-drop operation.
  • FIG. 8 an example of setting the similarity label in the pair of the clusters C 1 and C 2 by dragging the document D 1 belonging to the cluster C 1 to drop the dragged document D 1 onto the document D 4 belonging to the cluster C 2 is illustrated on the label setting screen 200 .
  • the label set in the pair of clusters in this manner is saved as the label data 14 B.
  • the designated pair of documents does not necessarily have to be regarded as a pair of clusters, and a label may be set in a pair of documents.
  • the label setting an example of accepting the setting of the similarity label is given here, but as a matter of course, the setting of the dissimilarity label may also be accepted.
  • an example of accepting the label setting by user operation is given here, but if the label setting can also be acquired via the network NW, the label setting may be acquired from an internal or external storage including a removable disk and the like.
  • the control unit 15 is a functional unit that performs overall control of the server device 10 .
  • control unit 15 may be implemented by a hardware processor such as a central processing unit (CPU) or a micro-processing unit (MPU). While a CPU and an MPU are exemplified as an example of the processor here, it may be implemented by any processor regardless of whether it is a versatile type or a specialized type. Additionally, the control unit 15 may also be achieved by a hard wired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the control unit 15 By executing the learning support program described above, the control unit 15 virtually achieves the processing units illustrated in FIG. 1 on a work area of a random access memory (RAM) such as a dynamic random access memory (DRAM) implemented as a main storage device (not illustrated).
  • RAM random access memory
  • DRAM dynamic random access memory
  • the program running on the server device 10 is not limited to this.
  • packaged software in which the learning support program is packaged in the above-mentioned classification program may be executed.
  • control unit 15 includes an acceptance unit 15 A, a first calculation unit 15 B, a learning unit 15 C, a second calculation unit 15 D, a third calculation unit 15 E, and a detection unit 15 F.
  • the acceptance unit 15 A is a processing unit that accepts a request for relearning.
  • the acceptance unit 15 A accepts a request for relearning such as fine-tune, by accepting a pressing operation on a learn button 200 A arranged on the label setting screen 200 illustrated in FIG. 8 from the client terminal 30 . Then, when a request for relearning is accepted, the acceptance unit 15 A reads the first model data 13 M 1 and the learning data 14 from the storage unit 13 .
  • the first calculation unit 15 B is a processing unit that calculates the embedded vector before relearning.
  • the first calculation unit 15 B performs first distance metric processing that calculates the embedded vector of the representative of each cluster of the learning data 14 , using the distance metric model before relearning. For example, the first calculation unit 15 B inputs the vector of each sample of the learning data 14 to the distance metric model before relearning loaded into a work area of a memory (not illustrated) in accordance with the first model data 13 M 1 read from the storage unit 13 . This causes the distance metric model before relearning to output the embedded vector.
  • the document vector that is a sample specified as the representative of the cluster is input to the input layer of the distance metric model before relearning.
  • the distance metric model is assumed to have “eight” input layers, which is the number of words in the documents D 1 to D 10 , and two output layers.
  • the samples specified as the representatives of the respective clusters C 1 to C 4 of the learning data are individually assumed as the document D 1 , the document D 4 , the document D 7 , and the document D 9 .
  • the vector of the document D 1 ⁇ monitor: 1, AP server: 1, DB server: 0, failure: 1, error: 1, occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇ is input to the input layer of the distance metric model before relearning.
  • the output layers of the distance metric model before relearning output the embedded vector [ ⁇ 5, ⁇ 5] of the document D 1 before relearning.
  • the embedded vector [ ⁇ 5, 5] of the document D 4 before relearning is obtained.
  • the embedded vector [5, 3] of the document D 7 before relearning is obtained.
  • the embedded vector [5, ⁇ 3] of the document D 9 before relearning is obtained.
  • the input and output in the above first distance metric processing are as follows.
  • D 1 , D 2 ⁇ Monitor: 1, AP Server: 1, DB Server: 0, Failure: 1, Error: 1, Occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇
  • D 3 ⁇ Monitor: 1, AP Server: 1, DB Server: 0, Failure: 0, Error: 1, Occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇
  • D 4 , D 5 ⁇ Monitor: 1, AP Server: 0, DB Server: 1, Failure: 1, Error: 1, Occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇
  • D 6 ⁇ Monitor: 1, AP Server: 0, DB Server: 1, Failure: 0, Error: 1, Occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇
  • D 7 , D 8 ⁇ Monitor: 1, AP Server: 1, DB Server: 0, Failure: 1, Error: 1, Occur: 1, VEO000481436: 0, VEO000481437: 1 ⁇
  • D 9 , D 10 ⁇ Monitor: 1, AP Server: 0, DB Server: 1, Failure: 1, Error: 1, Occur: 1, VEO000481436: 0, VEO000481437: 1 ⁇
  • FIG. 9 is a diagram illustrating an example of the embedded vectors.
  • the embedded vectors of the documents D 1 , D 4 , D 7 , and D 9 which are specified as the representatives of the clusters C 1 to C 4 embedded by the distance metric model before relearning, are mapped.
  • the document D 1 specified as the representative of the cluster C 1 is embedded in [ ⁇ 5, ⁇ 5]
  • the document D 4 specified as the representative of the cluster C 2 is embedded in [ ⁇ 5, 5].
  • the document D 7 specified as the representative of the cluster C 3 is embedded in [5, 3]
  • the document D 9 specified as the representative of the cluster C 4 is embedded in [5, ⁇ 3].
  • the learning unit 15 C is a processing unit that performs relearning of the distance metric model that has finished learning.
  • the learning unit 15 C uses the learning data 14 to perform distance metric learning of the distance metric model defined by the first model data 13 M 1 , which is relearning. For example, the learning unit 15 C performs the following processing for each pair of documents obtained by combining two of the documents D 1 to D 10 . For example, the learning unit 15 C relearns parameters of the Siamese Network that bring the distance between a similar pair closer while bringing the distance between a dissimilar pair farther, based on the similarity or dissimilarity label set in the pair of documents.
  • the learning unit 15 C is assumed to use the similarity label set in the pair of the clusters C 1 and C 2 to update the parameters of the Siamese Network.
  • the above combinations may include a pair of the documents D 1 and D 4 , a pair of the documents D 1 and D 5 , and a pair of the documents D 1 and D 6 .
  • the above combinations may include a pair of the documents D 2 and D 4 , a pair of the documents D 2 and D 5 , and a pair of the documents D 2 and D 6 .
  • the above combinations may include a pair of the documents D 3 and D 4 , a pair of the documents D 3 and D 5 , and a pair of the documents D 3 and D 6 .
  • the learning unit 15 C is not restricted to repeat relearning using the learning data 14 until the demanded number of epochs such as the number of epochs at which the value of the loss function converges or the number of epochs at which the correct answer rate of the test data reaches a fixed value.
  • the learning unit 15 C performs relearning having a number of epochs in the relearning process smaller than the demanded number of epochs, such as one epoch.
  • the input and output in the above distance metric learning processing are as follows.
  • the parameters and the like of the distance metric model in the relearning process obtained by relearning of the learning unit 15 C in this manner are stored as second model data 13 M 2 in a work area of the memory referenced by the control unit 15 .
  • the second model data 13 M 2 may be saved in any storage such as a storage area included in the storage unit 13 .
  • the second calculation unit 15 D is a processing unit that calculates the embedded vector in the relearning process.
  • the second calculation unit 15 D performs second distance metric processing that calculates the embedded vector of the representative of each cluster of the learning data 14 , using the distance metric model in the relearning process. For example, the second calculation unit 15 D inputs the vector of each sample of the learning data 14 to the distance metric model in the relearning process loaded into a work area of a memory (not illustrated) in accordance with the second model data 13 M 2 described above. This causes the distance metric model in the relearning process to output the embedded vector.
  • the document vector that is a sample specified as the representative of the cluster is input to the input layer of the distance metric model in the relearning process.
  • the parameters of the distance metric model are different but the layer structure is common between before relearning and in the relearning process.
  • the vector of the document D 1 . ⁇ monitor: 1, AP server: 1, DB server: 0, failure: 1, error: 1, occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇ is input to the input layer of the distance metric model in the relearning process.
  • the output layers of the distance metric model in the relearning process output the embedded vector [ ⁇ 5, ⁇ 4] of the document D 1 in the relearning process.
  • the embedded vector [ ⁇ 5, 4] of the document D 4 in the relearning process is obtained.
  • the embedded vector [5, 2] of the document D 7 in the relearning process is obtained.
  • the embedded vector [5, ⁇ 2] of the document D 9 in the relearning process is obtained.
  • the input and output in the above second distance metric processing are as follows.
  • D 1 ⁇ Monitor: 1, AP Server: 1, DB Server: 0, Failure: 1, Error: 1, Occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇
  • D 4 ⁇ Monitor: 1, AP Server: 0, DB Server: 1, Failure: 1, Error: 1, Occur: 1, VEO000481436: 1, VEO000481437: 0 ⁇
  • D 7 ⁇ Monitor: 1, AP Server: 1, DB Server: 0, Failure: 1, Error: 1, Occur: 1, VEO000481436: 0, VEO000481437: 1 ⁇
  • D 9 ⁇ Monitor: 1, AP Server: 0, DB Server: 1, Failure: 1, Error: 1, Occur: 1, VEO000481436: 0, VEO000481437: 1 ⁇
  • FIG. 10 is a diagram illustrating an example of the embedded vectors.
  • the embedded vectors of the documents D 1 , D 4 , D 7 , and D 9 which are specified as the representatives of the clusters C 1 to C 4 embedded by the distance metric model before relearning, are mapped by black circles.
  • the embedded vectors of the documents D 1 , D 4 , D 7 , and D 9 which are specified as the representatives of the clusters C 1 to C 4 embedded by the distance metric model in the relearning process, are mapped by white circles. As illustrated in FIG.
  • the document D 1 specified as the representative of the cluster C 1 is embedded in [ ⁇ 5, ⁇ 5] before relearning, while it is embedded in [ ⁇ 5, ⁇ 4] in the relearning process.
  • the document D 4 specified as the representative of the cluster C 2 is embedded in [ ⁇ 5, 5] before relearning, while it is embedded in [ ⁇ 5, 4] in the relearning process.
  • the document D 7 specified as the representative of the cluster C 3 is embedded in [5, 3] before relearning, while it is embedded in [5, 2] in the relearning process.
  • the document D 9 specified as the representative of the cluster C 4 is embedded in [5, ⁇ 3] before relearning, while it is embedded in [5, ⁇ 2] in the relearning process.
  • the third calculation unit 15 E is a processing unit that calculates a movement parameter between clusters between before relearning and in the relearning process.
  • the third calculation unit 15 E calculates the moving direction of the cluster by computation of subtracting the embedded vector of the cluster representative before relearning from the embedded vector of the cluster representative in the relearning process.
  • the input and output when the moving direction of the cluster is calculated in this manner are as follows.
  • the third calculation unit 15 E calculates the magnitude of a travel angle between the clusters.
  • the input and output when the travel angle between clusters is calculated in this manner are as follows.
  • the third calculation unit 15 E calculates the amount of change in the distance between clusters, based on the embedded vectors of the cluster representatives in the relearning process and the embedded vectors of the cluster representatives before relearning.
  • the amount of change between before relearning and in the relearning process can be found by computation of dividing the distance between clusters in the relearning process by the distance between clusters before relearning. The input and output when the amount of change in the distance between clusters is calculated in this manner are as follows.
  • before_distance refers to the distance between clusters before relearning
  • after_distance refers to the distance between clusters in the relearning process.
  • the detection unit 15 F is a processing unit that detects a pair of clusters that are likely to be integrated after relearning.
  • a pair of clusters that are likely to be integrated after relearning will be described as an “integrated cluster pair” in some cases.
  • the detection unit 15 F may correspond to an example of an output unit.
  • the detection unit 15 F detects the integrated cluster pair based on at least one or a combination of the magnitude of the travel angle between clusters calculated by the third calculation unit 15 E and the amount of change in the distance between clusters calculated by the third calculation unit 15 E.
  • the detection unit 15 F can detect the integrated cluster pair under an AND condition between the magnitude of the travel angle between clusters and the amount of change in the distance between clusters. For example, the detection unit 15 F detects, as the integrated cluster pair, a pair of clusters in which the magnitude of the travel angle between clusters is less than the predetermined threshold value ⁇ 1 and the amount of change in the distance between clusters is less than the predetermined threshold value ⁇ 2.
  • ⁇ 1 is assumed as 0.01
  • ⁇ 2 is assumed as 0.9
  • the pair of the clusters C 1 and C 2 and the pair of the clusters C 3 and C 4 are detected as the integrated cluster pairs.
  • the pair of the clusters C 1 and C 2 , the pair of the clusters C 1 and C 3 , the pair of the clusters C 2 and C 4 , and the pair of the clusters C 3 and C 4 are detected as the integrated cluster pairs.
  • the detection unit 15 F excludes a pair of clusters in which the similarity label is set, among pairs of clusters detected as the integrated cluster pairs, from the target for inquiry. For example, in the case of the example of the label setting screen 200 illustrated in FIG. 8 , since the similarity label is set in the pair of the clusters C 1 and C 2 in the label data 14 B, the pair of the clusters C 1 and C 2 is excluded, and the pair of the clusters C 3 and C 4 is extracted.
  • FIG. 11 is a diagram illustrating an example of an inquiry screen.
  • FIG. 11 illustrates an example in which an inquiry screen 400 including the integrated cluster pair detected as the inquiry targets is displayed in a pop-up manner in front of the label setting screen 200 .
  • the inquiry screen 400 displays a continue button 400 A and an interrupt button 400 B.
  • relearning is continued by the learning unit 15 C with the above label data 14 B set, up to the demanded number of epochs without interrupting relearning in the relearning process.
  • relearning can be interrupted in the relearning process. In addition to such interruption, the label can be reset on the label setting screen.
  • FIG. 12 is a diagram illustrating an example of the label setting screen.
  • FIG. 12 illustrates a label setting screen 210 displayed after the interrupt button 400 B of the inquiry screen 400 illustrated in FIG. 11 is operated. Resetting of the similarity or dissimilarity label can be accepted through an operation of associating a pair of documents on the label setting screen 210 illustrated in FIG. 12 .
  • the label setting screen 210 illustrated in FIG. 12 an example of setting the dissimilarity label in the pair of the clusters C 3 and C 4 by dragging the document D 7 belonging to the cluster C 3 to drop the dragged document D 7 onto the document D 9 belonging to the cluster C 4 is illustrated.
  • the label reset in the pair of clusters in this manner is overwritten and saved in the label data 14 B. Thereafter, when an operation on a learn button 210 A is accepted, the learning unit 15 C can be caused to execute relearning based on the reset label. Note that, when an operation on a cancel button 210 B is accepted, relearning is canceled.
  • the following behavior may be expected.
  • the label setting screen 200 illustrated in FIG. 8 illustrates an example in which the similarity label is set in the pair of the clusters C 1 and C 2 with the intention of integrating the clusters C 1 and C 2 .
  • Such label setting diminishes the importance, in cluster formation, of the words “AP server” appearing in the documents D 1 to D 3 and the words “DB server” appearing in the documents D 4 to D 6 .
  • the importance of “VEO000481436” and “VEO000481437” will increase in embedding into the metric space. From these facts, the possibility that the integration of the pair of the clusters C 1 and C 2 occurs due to the label setting in the pair of the clusters C 1 and C 2 increases.
  • FIG. 13 is a flowchart illustrating a procedure of learning support processing according to the first embodiment. As a mere example, this processing is started when a request for relearning is accepted, or the like. As illustrated in FIG. 13 , the acceptance unit 15 A reads the first model data 13 M 1 and the learning data 14 from the storage unit 13 (step S 101 ).
  • the first calculation unit 15 B performs the first distance metric processing that calculates the embedded vector of the representative of each cluster of the learning data 14 , using the distance metric model before relearning defined in the first model data 13 M 1 (step S 102 A).
  • the embedded vector of the cluster representative before relearning calculated in step S 102 A is output to the third calculation unit 15 E from the first calculation unit 15 B (step S 103 A).
  • step S 102 B In parallel with above step S 102 A and above step S 103 A, processing in following step S 102 B to following step S 105 B is executed.
  • step S 102 B the learning unit 15 C uses the learning data 14 to perform distance metric learning of the distance metric model defined by the first model data 13 M 1 , which is relearning.
  • the number of times the learning data 14 is relearned the number of epochs in the relearning process, which is smaller than the demanded number of epochs, is applied.
  • the parameters and the like of the distance metric model in the relearning process are output to the second calculation unit 15 D from the learning unit 15 C as the second model data 13 M 2 (step S 103 B).
  • the second calculation unit 15 D performs the second distance metric processing that calculates the embedded vector of the representative of each cluster of the learning data 14 , using the distance metric model in the relearning process defined in the second model data 13 M 2 (step S 104 B).
  • the embedded vector of the cluster representative in the relearning process calculated in step S 104 B is output to the third calculation unit 15 E from the second calculation unit 15 D (step S 105 B).
  • the third calculation unit 15 E calculates the movement parameters between clusters, such as the magnitude of the travel angle between clusters and the amount of change in the distance between clusters, based on the embedded vectors of the cluster representatives before relearning and the embedded vectors of the cluster representatives in the relearning process (step S 106 ).
  • the movement parameters between clusters calculated in step S 106 are output to the detection unit 15 F from the third calculation unit 15 E (step S 107 ).
  • the detection unit 15 F detects a pair of clusters that are likely to be integrated after relearning, based on at least one or a combination of the magnitude of the travel angle between clusters and the amount of change in the distance between clusters (step S 108 ).
  • a pair of cluster in which the similarity label is not set is output to a predetermined output destination such as the client terminal 30 as an example (step S 109 ).
  • the server device 10 provides the learning support function that detects a pair of clusters that come closer to each other and do not have the similarity label set, based on the embedded vectors output by the distance metric model at each of time points before relearning and in the relearning process. Consequently, according to the server device 10 according to the present embodiment, since the integration of the pair of clusters that is not intended by the model designer is suppressed, adversely affecting the distance metric model after relearning may be suppressed.
  • each of the illustrated constituent elements in each of the devices does not necessarily have to be physically configured as illustrated in the drawings.
  • specific modes of distribution and integration of the individual devices are not restricted to those illustrated, and all or some of the devices may be configured by being functionally or physically distributed and integrated in any unit depending on various loads, usage status, and the like.
  • the acceptance unit 15 A, the first calculation unit 15 B, the learning unit 15 C, the second calculation unit 15 D, the third calculation unit 15 E, or the detection unit 15 F may be connected by way of a network as an external device of the server device 10 .
  • different devices each may include the acceptance unit 15 A, the first calculation unit 15 B, the learning unit 15 C, the second calculation unit 15 D, the third calculation unit 15 E, or the detection unit 15 F and may be connected to a network to cooperate with each other, whereby the above-described function of the server device 10 may be achieved.
  • various kinds of processing described in the embodiments above may be achieved by a computer such as a personal computer or a workstation executing a program prepared in advance.
  • a computer such as a personal computer or a workstation executing a program prepared in advance.
  • FIG. 14 is a diagram illustrating a hardware configuration example of a computer.
  • a computer 100 includes an operation unit 110 a , a speaker 110 b , a camera 110 c , a display 120 , and a communication unit 130 .
  • the computer 100 includes a CPU 150 , a read-only memory (ROM) 160 , an HDD 170 , and a RAM 180 . These units 110 to 180 are each connected via a bus 140 .
  • ROM read-only memory
  • the HDD 170 stores a learning support program 170 a that has functions similar to the functions of the acceptance unit 15 A, the first calculation unit 15 B, the learning unit 15 C, the second calculation unit 15 D, the third calculation unit 15 E, and the detection unit 15 F indicated in the above first embodiment.
  • This learning support program 170 a may be integrated or separated in a similar manner to the respective constituent elements of the acceptance unit 15 A, the first calculation unit 15 B, the learning unit 15 C, the second calculation unit 15 D, the third calculation unit 15 E, and the detection unit 15 F illustrated in FIG. 1 .
  • all the data indicated in the first embodiment described above does not necessarily have to be stored in the HDD 170 , and it is sufficient if only data for use in processing is stored in the HDD 170 .
  • the CPU 150 reads the learning support program 170 a from the HDD 170 and then loads the read learning support program 170 a into the RAM 180 .
  • the learning support program 170 a functions as a learning support process 180 a as illustrated in FIG. 14 .
  • This learning support process 180 a loads various kinds of data read from the HDD 170 into an area allocated to the learning support process 180 a in a storage area included in the RAM 180 and executes various kinds of processing using this various kinds of loaded data.
  • examples of the processing to be executed by the learning support process 180 a include the processing illustrated in FIG. 13 and the like. Note that all the processing units indicated in the first embodiment described above do not necessarily have to run on the CPU 150 , and it is sufficient if only a processing unit corresponding to processing to be executed is virtually achieved.
  • each program may be stored in a “portable physical medium” such as a flexible disk, which is what is called an FD, a compact disc read only memory (CD-ROM), a digital versatile disc (DVD), a magneto-optical disk, or an integrated circuit (IC) card to be inserted into the computer 100 . Then, the computer 100 may acquire and execute each program from these portable physical media.
  • each program may be stored in another computer, server device, or the like connected to the computer 100 via a public line, the Internet, a LAN, a wide area network (WAN), or the like, and the computer 100 may acquire each program from them to execute the program.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
US17/863,511 2020-01-16 2022-07-13 Computer-readable recording medium storing learning support program, learning support method, and learning support device Pending US20220374648A1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/001382 WO2021144938A1 (ja) 2020-01-16 2020-01-16 学習支援プログラム、学習支援方法及び学習支援装置

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/001382 Continuation WO2021144938A1 (ja) 2020-01-16 2020-01-16 学習支援プログラム、学習支援方法及び学習支援装置

Publications (1)

Publication Number Publication Date
US20220374648A1 true US20220374648A1 (en) 2022-11-24

Family

ID=76864067

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/863,511 Pending US20220374648A1 (en) 2020-01-16 2022-07-13 Computer-readable recording medium storing learning support program, learning support method, and learning support device

Country Status (3)

Country Link
US (1) US20220374648A1 (https=)
JP (1) JP7287505B2 (https=)
WO (1) WO2021144938A1 (https=)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240005483A1 (en) * 2020-09-29 2024-01-04 Oxford University Innovation Limited Method of training a machine learning model, method of assessing ultrasound measurement data, method of determining information about an anatomical feature, ultrasound system
US20240184758A1 (en) * 2022-12-01 2024-06-06 Siemens Mobility GmbH Method of assessing input-output datasets using local complexity values and associated data structure
US20240211796A1 (en) * 2022-12-22 2024-06-27 Microsoft Technology Licensing, Llc Explanation of emergent semantics in embedding spaces via analogy

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7314331B1 (ja) 2022-01-28 2023-07-25 俊二 有賀 Ａｉを用いた類似デザイン判定システム
CN116361679B (zh) * 2023-06-02 2023-08-11 青岛豪迈电缆集团有限公司 基于数据驱动的电缆寿命智能预测方法及系统

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190130073A1 (en) * 2017-10-27 2019-05-02 Nuance Communications, Inc. Computer assisted coding systems and methods

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5889019B2 (ja) * 2012-02-06 2016-03-22 キヤノン株式会社 ラベル付加装置、ラベル付加方法及びプログラム

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190130073A1 (en) * 2017-10-27 2019-05-02 Nuance Communications, Inc. Computer assisted coding systems and methods

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240005483A1 (en) * 2020-09-29 2024-01-04 Oxford University Innovation Limited Method of training a machine learning model, method of assessing ultrasound measurement data, method of determining information about an anatomical feature, ultrasound system
US20240184758A1 (en) * 2022-12-01 2024-06-06 Siemens Mobility GmbH Method of assessing input-output datasets using local complexity values and associated data structure
US12411824B2 (en) * 2022-12-01 2025-09-09 Siemens Mobility GmbH Method of assessing input-output datasets using local complexity values and associated data structure
US20240211796A1 (en) * 2022-12-22 2024-06-27 Microsoft Technology Licensing, Llc Explanation of emergent semantics in embedding spaces via analogy

Also Published As

Publication number Publication date
WO2021144938A1 (ja) 2021-07-22
JPWO2021144938A1 (https=) 2021-07-22
JP7287505B2 (ja) 2023-06-06

Similar Documents

Publication Publication Date Title
US20220374648A1 (en) Computer-readable recording medium storing learning support program, learning support method, and learning support device
US11341424B2 (en) Method, apparatus and system for estimating causality among observed variables
US11048870B2 (en) Domain concept discovery and clustering using word embedding in dialogue design
EP3467723A1 (en) Machine learning based network model construction method and apparatus
US12327206B2 (en) Content analysis and retrieval using machine learning
JP6603828B1 (ja) フレーズ生成モデル学習装置、フレーズ生成装置、方法、及びプログラム
US11868859B1 (en) Systems and methods for data structure generation based on outlier clustering
WO2014073206A1 (ja) 情報処理装置、及び、情報処理方法
Piao et al. Rebalance Weights AdaBoost‐SVM Model for Imbalanced Data
US20250148822A1 (en) Systems for Generating Indications of Relationships between Electronic Documents
US20220414490A1 (en) Storage medium, machine learning method, and machine learning device
KR20240087607A (ko) 데이터 분석을 위한 테이블 분석 처리 방법 및 그를 위한 장치
Chang et al. A word embedding-based approach to cross-lingual topic modeling
US11853702B2 (en) Self-supervised semantic shift detection and alignment
US20250103920A1 (en) Decision Making Using Integrated Machine Learning Models and Knowledge Graphs
Ortner et al. Robust and sparse multigroup classification by the optimal scoring approach
Gong Analysis of internet public opinion popularity trend based on a deep neural network
US20250190712A1 (en) Methods and systems for transferring knowledge from large language models to small language models using out-of-distribution feedbacks
Ahmadi et al. Explainability of point cloud neural networks using smile: Statistical model-agnostic interpretability with local explanations
Gladence et al. A novel technique for multi-class ordinal regression-APDC
Zhang Anomaly detecting and ranking of the cloud computing platform by multi-view learning
CN111768214A (zh) 产品属性的预测方法、系统、设备和存储介质
WO2020167156A1 (ru) Способ отладки обученной рекуррентной нейронной сети
KR20220158571A (ko) 문서 구조 분석 학습 장치 및 방법, 문서 구조 분석 장치 및 방법
Wang et al. Information Theoretic Weighted Fuzzy Clustering Ensemble.

Legal Events

Date Code Title Description
AS Assignment

Owner name: FUJITSU LIMITED, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MIZOBUCHI, YUJI;REEL/FRAME:060493/0095

Effective date: 20220705

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION COUNTED, NOT YET MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: ALLOWED -- NOTICE OF ALLOWANCE NOT YET MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED