JP2021039757A - Classifying apparatus, learning apparatus, classifying method, learning method, control program, and recording medium - Google Patents

Abstract

To realize a technique for carrying out classification processing more properly.SOLUTION: A classifying apparatus (1) according to an embodiment of the present invention has an acquiring unit (5) for acquiring input data of a plurality of kinds, a first classifier (9) for mapping the input data to feature vectors in a feature amount space, and a second classifier (11) for outputting information relating to the plurality of kinds with using the feature vectors in the feature amount space as input. The second classifier (11) is input with feature vectors except for a common part among the feature vectors in the feature amount space corresponding to the respective input data of the plurality of kinds.SELECTED DRAWING: Figure 1

Description

The present invention relates to a classification device, a learning device, a classification method, a learning method, a control program, and a recording medium.

Conventionally, a classification device using a neural network is known. As an example, such a classification device classifies the authenticity of an object. A convolutional neural network (CNN) for determining whether a plant is healthy or sick is also known (Non-Patent Document 1).

Toda, Y., & Okura, F. (2019). How Convolutional Neural Networks Diagnose Plant Disease. Plant Phenomics, 2019, 1-14. URL (https://doi.ort/10.34133/2019/9237136)

By the way, in the classification apparatus, there is a first problem that it is preferable to perform the classification process more preferably.

In addition, the above-mentioned conventional technique has the following second problem. That is, since the learning model is usually a black box, the problem (interpretability problem) that the user cannot appropriately interpret (or explain) what features of the object are learned and classified. ). Further, there is a problem that it is not possible to confirm whether or not the algorithm of the learning model has properly learned the characteristics of the object.

One aspect of the present invention aims to realize a technique for solving at least one of the first problem and the second problem.

In order to solve at least the first problem, the classification device according to one aspect of the present invention has an acquisition unit that acquires input data that can take a plurality of types, and a feature vector that obtains the input data in a feature quantity space. The second classifier includes a first classifier that maps to the above, a second classifier that receives a feature vector in the feature quantity space as an input and outputs information about the type, and the second classifier includes the plurality of. A feature vector other than the common part of the feature vector in the feature amount space corresponding to each type of input data is input.

In order to solve at least the second problem, the learning device according to one aspect of the present invention maps input data that can take a plurality of types to a feature vector in a feature space including a plurality of different subspaces. The learning unit includes a learning unit for learning the first classifier, the learning unit includes a decoder that inputs a feature vector in the feature quantity space, and uses the first type of data as the input data of the first type. When inputting to the classifier, the difference between the data output by the decoder and the input data is small after masking the subspace corresponding to the second type data-specific feature in the feature amount space. In the case of masked learning in which the first classifier and the decoder are trained, and the second type of data is input to the first classifier as the input data, the feature quantity space. Of these, the first classifier is designed so that the difference between the data output by the decoder and the input data is small after masking the subspace corresponding to the unique feature of the first type of data. And learning with a mask to train the decoder.

In order to solve at least the second problem, the learning device according to one aspect of the present invention learns a first classifier that maps input data that can take a plurality of types to a feature vector in a feature space. It is provided with a learning unit to be used and a third classifier that receives a feature vector in the feature quantity space as an input and outputs information regarding the type, and the learning unit corresponds to each of the input data of the plurality of types. When the feature vector included in the common part of the feature vector in the feature quantity space is input to the third classifier, the classification accuracy of the third classifier is lowered, so that the first classifier is used. To learn.

In order to solve at least the first problem, the classification method according to one aspect of the present invention is a classification method executed by an apparatus, which includes an acquisition step of acquiring input data that can take a plurality of types, and the above-mentioned. The above includes a first classification step of mapping input data to a feature vector in the feature amount space, and a second classification step of inputting the feature vector in the feature amount space and outputting information on the type. In the second classification step, feature vectors other than the common portion of the feature vectors in the feature quantity space corresponding to each of the plurality of types of input data are input.

In order to solve at least the second problem, the learning method according to one aspect of the present invention maps input data that can take a plurality of types to a feature vector in a feature space including a plurality of different subspaces. The learning step includes a learning step for training the first classifier, and when the first type of data is input to the first classifier as the input data, the second type of the feature quantity space is included. After masking the subspace corresponding to the data-specific feature of, the first is such that the difference between the data output by the decoder that inputs the feature vector in the feature quantity space and the input data becomes small. In the masked learning for training the classifier 1 and the decoder, when the second type of data is input to the first classifier as the input data, the first of the feature quantity spaces is described. After masking the subspace corresponding to the unique feature of the species data, the data output by the decoder that inputs the feature vector in the feature amount space and the input data are reduced so that the difference is small. Masked learning is performed to train the first classifier and the decoder.

In order to solve at least the second problem, the learning method according to one aspect of the present invention learns a first classifier that maps input data that can take a plurality of types to a feature vector in a feature space. In the learning step, the feature vector included in the common part of the feature vector in the feature quantity space corresponding to each of the plurality of types of input data is converted into the feature vector in the feature quantity space. Is input to the third classifier that outputs information about the type, the first classifier is trained so that the classification accuracy of the third classifier is lowered.

According to one aspect of the present invention, it is possible to realize a technique for solving at least one of the first problem and the second problem.

It is a functional block diagram of the classification apparatus which concerns on embodiment. It is a flowchart which shows the flow of the learning process which concerns on embodiment. It is a conceptual diagram which shows the unsupervised learning of the 1st classifier and the decoder which concerns on embodiment. FIG. 5 is a conceptual diagram showing conditional learning of a first classifier and a decoder according to an embodiment, followed by supervised learning of a second classifier. It is a figure which shows an example of the image generated by adding the feature of a disease state or the feature of a healthy state to the input image which concerns on embodiment. It is a conceptual diagram which shows the learning example of the 1st classifier, the decoding machine, and the 3rd classifier which concerns on embodiment. It is a conceptual diagram which shows the supervised learning of the 2nd classifier which concerns on embodiment. It is a flowchart which shows the flow of the classification process which concerns on embodiment. It is a conceptual diagram which shows the learning example of the 1st classifier, the decoding machine, and the 3rd classifier which concerns on the appendix of embodiment. It is a conceptual diagram which shows the supervised learning of the 2nd classifier which concerns on the appendix of embodiment. It is a conceptual diagram which shows the interpretability improvement processing example which concerns on the appendix of the embodiment. It is a figure which shows the input image example to be input to the classification apparatus (learning apparatus) which concerns on the appendix of embodiment, and the generated image example. It is a conceptual diagram explaining the processing example for determining the feature amount which had a big influence on the classification result.

[Embodiment]
Hereinafter, one embodiment of the present invention will be described in detail. The classification device according to one aspect of the present invention is a device that classifies input data such as images into any of a plurality of types.

[1. Classification device configuration example]
FIG. 1 is a functional block diagram of the classification device (learning device) 1 according to the present embodiment. As shown in FIG. 1, the classification device 1 includes a control unit 3, an output unit 17, and a storage unit 19.

The control unit 3 is a control device that controls the entire classification device 1, and also functions as an acquisition unit 5, a classification unit 7, a learning unit 13, and a decoder 15.

The acquisition unit 5 acquires input data that can take a plurality of types. The mode of the input data acquired by the acquisition unit 5 does not limit the present embodiment. In the following, the input data will be described as image data relating to an organism, and the plurality of types will be described as including the type relating to whether the organism is healthy or ill. Further, in the present embodiment, as the image data relating to the living thing, the image data of the leaves of the plant will be described as an example, but for example, an image showing a part of the body of an animal or an insect may be used. Further, the plurality of types is not limited to two types, and may be three or more types as described later. As another aspect, for example, the input data is image data related to an industrial product or the like, and the plurality of types include a type relating to whether the industrial product or the like is a normal non-defective product or a defective product. There may be. Further, as another aspect, the input data may be voice data, and the plurality of types may include types classified according to the source of the voice.

Further, the acquisition source from which the acquisition unit 5 acquires the input data is not particularly limited, and may be an external device connected via the Internet, a storage unit 19 inside the classification device 1, or the like.

(Classification unit 7)
The classification unit 7 has a configuration for classifying the input data into any of a plurality of types, and includes a first classifier 9, a second classifier 11, and a third classifier 12. Further, the classification process by the classification unit 7 and the decoding process by the decoder 15 described later are defined by the learning model corresponding to the parameter set stored in the storage unit 19. Further, the learning model in the present embodiment includes at least one of a model for associating the input data with the feature vector in the feature amount space described later and a model for associating the vector in the feature amount space with the output data. Is included.

(First classifier 9)
The first classifier 9 maps the input data acquired by the acquisition unit 5 to a feature vector (latent variable) in the feature space including a plurality of different subspaces. The specific configuration of the first classifier 9 is not limited to this embodiment, but as an example, a network on the encoder side included in an autoencoder (AE: Autoencoder) can be used. Further, as the first classifier 9, the network on the encoder side included in the variational autoencoder (VAE) may be used. The details of the feature space will be described later.

(Second classifier 11)
The second classifier 11 takes the feature vector in the feature space as an input and outputs information on the type. The specific configuration of the second classifier 11 does not limit the present embodiment, but as an example, a fully connected neural network can be used. The specific example of the information regarding the type output by the second classifier 11 does not limit the present embodiment, but as an example, the leaves of the plant shown by the image to be classified are in a healthy state. It is information showing the estimation result as to whether the patient is in a state of illness or illness.

The learning unit 13 performs various learning of the classification unit 7 and the decoder 15. The details of the learning process by the learning unit 13 will be described later. In addition to the configuration shown in FIG. 1, the decoder 15 may have a configuration included in the learning unit 13.

Hereinafter, the data of the first kind will be described as an image showing the leaves of a plant in a healthy state, and the data of the second kind will be described as an image showing the leaves of a plant in a diseased state.

(Third classifier 12)
The third classifier 12 is used to determine whether the definition of the subspace of the feature space has been made favorably. The third classifier 12 takes the feature vector in the feature space as an input and outputs information on the type, and the details of the processing will be described later.

(Decoder 15)
The decoder 15 takes the feature vector in the feature space as an input and outputs the decoded image. The specific configuration of the decoder 15 is not limited to this embodiment, but as an example, a network on the decoder side included in an autoencoder (AE: Autoencoder) can be used. Further, the decoder 15 may be configured to use the network on the decoder side included in the variational autoencoder (VAE).

The output unit 17 is a display panel or speaker that outputs the classification result by the classification unit 7, or a communication module that outputs the classification result to an external display device or the like. The storage unit 19 is a storage device that stores various data, and stores various functions and parameter sets for defining a learning model.

The classification device 1 may be provided with an input interface such as a keyboard for the user to perform an instruction operation on the classification device 1.

[2. Learning process example]
Hereinafter, an example of the learning process in the classification device 1 will be described. In the learning process of this example, the following steps are generally executed.

(1) Unsupervised learning of the first classifier 9 and the decoder 15 is performed in advance using the learning data.

(2) Conditional learning of the first classifier 9 and the decoder 15 is performed using the first type data or the data to which the second type data label is added, and at the same time, the intersection of the feature vectors is performed. When the feature vector included in is input to the third classifier 12, conditional learning of the first classifier 9 is performed so that the classification accuracy of the third classifier 12 becomes low.

(3) While fixing the parameters of the learning model used by the encoder included in the first classifier 9, supervised learning of the second classifier 11 is performed using the output of the encoder.

FIG. 2 is a flowchart showing a flow of learning processing by the classification device 1 according to the present embodiment.

In step S101, the acquisition unit 5 acquires an image of a plant leaf as input data used for learning. In addition, at least a part of the above image includes a labeled image showing whether the leaves contained in the image are in a healthy state or in a diseased state.

In step S102, the learning unit 13 performs unsupervised learning of the first classifier 9 and the decoder 15 using the image acquired by the acquisition unit 5. FIG. 3 is a conceptual diagram showing unsupervised learning in this step S102. In the unsupervised learning here, the image acquired by the acquisition unit 5 is input to the first classifier 9, and the difference between the image output by the decoder 15 and the image acquired by the acquisition unit 5 becomes smaller. As described above, it is a process of updating the parameters of the first classifier 9 and the decoder 15.

Since unsupervised learning is executed in this step S102, it is not necessary that the above label is added to the image used for learning.

By the above unsupervised learning, feature vectors corresponding to each of the input data are generated in the feature space. Further, by performing the above unsupervised learning, a situation may occur in which the feature vectors corresponding to the input data are ubiquitous due to the difference in the type in the feature space in response to the difference in the type of the input data.

The learning unit 13 trains the first classifier 9 and the decoder 15 so that the difference between the image output by the decoder 15 and the image input to the first classifier 9 becomes small. In the unsupervised learning in step S102, the mask described later is not performed.

Further, by the process of this step S102, the weights of the parameters related to the learning of the first classifier 9 and the decoder 15 can be adjusted according to the tendency of the input data.

In step S103, the learning unit 13 performs conditional learning of the first classifier 9 and the decoder 15 using the labeled image acquired by the acquisition unit 5. As a result, a plurality of subspaces corresponding to different sets of feature vectors are defined in the feature space. In addition, it is required whether each feature vector and each subspace corresponds to the feature of a leaf in a healthy state or the feature of a leaf in a diseased state. Note that the learning unit 13 may apply a hostile generation network (GAN) to the images generated by the first classifier 9 and the decoder 15 and the input images in order to generate results that are easy for the user to understand. Good.

By performing the conditional learning in this step S103, each feature vector exists in any of a plurality of subspaces different from each other depending on the type. Here, the plurality of subspaces different from each other include a subspace corresponding to a feature vector indicating the characteristics of a leaf in a healthy state and a subspace corresponding to a feature vector indicating the characteristics of a leaf in a diseased state. .. Further, these subspaces are common parts corresponding to the characteristics regardless of whether they are in a healthy state or a diseased state, and have common parts corresponding to the characteristics of the leaves themselves.

From another side,
The subspace corresponding to the feature vector that represents the characteristics of a healthy leaf is
-The subspace corresponding to the feature vector showing the characteristics peculiar to the leaf in the healthy state (subspace peculiar to the healthy state) and
-It is a feature regardless of whether it is in a healthy state or a diseased state, and is composed of a subspace (common subspace) corresponding to the common feature of the leaf itself.
The subspace corresponding to the feature vector that characterizes the diseased leaf is
-Subspaces (subspaces specific to the diseased state) corresponding to feature vectors that show the characteristics unique to the diseased leaf,
-It is a feature regardless of whether it is in a healthy state or a diseased state, and is composed of a subspace (common subspace) corresponding to the common feature of the leaf itself.

Supplementally, the subspace corresponding to the feature vector indicating the characteristics of the leaves in the healthy state is the sum space of the subspace peculiar to the healthy state and the intersection subspace. The subspace corresponding to the feature vector indicating the characteristics of the leaf in the diseased state is the sum space of the subspace peculiar to the diseased state and the intersection.

FIG. 4 is a conceptual diagram showing the conditional learning. In the example of FIG. 4, the feature vector 21 shows features specific to the diseased leaf. Further, the feature vector 23 shows a feature peculiar to a leaf in a healthy state. In addition, the feature vector 25 shows the characteristics of the leaf itself that are common to the leaves in the healthy state and the diseased state. The learning process in the second classifier 11 shown in FIG. 4 is executed following the learning process in the first classifier 9 and the decoder 15. The process will be described later in step S104. In this step S103, when the learning unit 13 performs the conditional learning of the first classifier 9, the feature vector included in the common part of each subspace, the feature vector peculiar to the leaf in the healthy state, and the feature vector peculiar to the leaf in the healthy state are added. Masks are used to prevent the diseased leaves from containing unique feature vectors and to reflect each feature in the corresponding feature vectors. The learning unit 13 masks a space other than the subspace corresponding to the type of the input image, and then uses the feature vector as an input so that the difference between the image output by the decoder 15 and the original input image becomes small. The first classifier 9 and the decoder 15 are trained. Here, the mask means that the component of the feature vector corresponding to the subspace to be masked is set to 0 or randomly, or the node connected to the corresponding feature vector is blocked in the first classifier 9. Means processing. At this time, a method in which the independence of each feature is maintained is desirable. When inputting an image of a healthy leaf into the first classifier 9, the learning unit 13 masks a subspace of the feature space formed by a feature vector peculiar to the diseased leaf. Then, masked learning is performed so that the first classifier 9 and the decoder 15 are trained so that the difference between the data output by the decoder 15 and the input image becomes small.

This is because the learning unit 13 erases or replaces the portion of the image showing the leaf-specific feature in the diseased state with a random image component, or in the first classifier 9, the feature vector. It shows that the learning process is performed by blocking the node connected to. At this time, a method in which the independence of each feature is maintained is desirable. By learning as described above, when an image of a healthy leaf is input to the first classifier 9, the classification accuracy via the subspace formed by the feature vector of the diseased leaf is improved. Learning that does not improve can be realized.

Further, when the learning unit 13 inputs an image of a diseased leaf to the first classifier 9, it masks a subspace of the feature space formed by a feature vector unique to the healthy leaf. Then, masked learning is performed so that the first classifier 9 and the decoder 15 are trained so that the difference between the data output by the decoder 15 and the input image becomes small.

This is because the learning unit 13 erases or replaces the portion of the image showing the leaf-specific features in a healthy state with random image components, or in the first classifier 9, the feature vector. It shows that the learning process is performed by blocking the node connected to. At this time, a method in which the independence of each feature is maintained is desirable. By learning as described above, when the image of the diseased leaf is input to the first classifier 9, the learning is performed so that the feature vector of the healthy leaf is not updated so much. It can be realized.

Also, in masked learning, the feature vector corresponding to the feature regardless of whether it is in a healthy state or a sick state does not include information corresponding to the feature peculiar to the healthy state or the sick state. Therefore, the first classifier 9 and the decoder 15 and the third classifier 12 are trained as described later so that the classification accuracy via the subspace formed by the feature vector is not improved.

More generally speaking, in masked learning, when a certain type of data is input to the first classifier 9, the subspace corresponding to the unique feature of the certain type of data is not masked, and the certain type is not masked. Data output by the decoder 15 after masking the subspaces that are common to the characteristics of the data of the above and the characteristics of the data of other types and masking the subspaces that correspond to the characteristics unique to the data of other types. The first classifier 9 and the decoder 15 are trained so that the difference from the input data becomes small.

The feature space may have a high dimension such as 256 to 4096, but if the feature space as an example is schematically shown as three dimensions,
-Feature vector corresponding to the first type data = (x1, y1, 0)
-Feature vector corresponding to the second type data = (0, y2, z2)
A feature vector like this is generated. Here, x1 and y1 are, for example, the main components in the feature vector corresponding to the first type data, and y2 and z2 are, for example, the main components in the feature vector corresponding to the second type data. .. Note that the component indicated by "0" refers to a non-major component, and even if it is not exactly zero, "the value is sufficiently small compared to other components" or "a large number of input vectors are input. It has a property such as "it rarely has a component that is not 0".

In the schematic example above,
-Corresponding subspace of type 1 data (X, Y, 0)
= Subspace unique to type 1 data (X, 0, 0) + Intersection (0, Y, 0)
-Corresponding subspace (0, Y, Z) of type 2 data
= Subspace unique to type 2 data (0, 0, Z) + intersection subspace (0, Y, 0)
It has the correspondence.

In addition, the first classifier 9 maps the image of a leaf in a healthy state to a feature vector in a certain subspace among a plurality of subspaces in the process of learning processing, and the first classifier 9 maps the image of the leaf in the diseased state. The image is mapped to a feature vector in another subspace different from the one subspace among the plurality of subspaces.

Further, by performing the masked learning in this step S103, in the classification process described later, a certain type of data as input data is converted into a feature vector in a certain subspace among a plurality of subspaces in the feature amount space. It is possible to perform a process of mapping and mapping other types of data as input data to a feature vector in another subspace different from the certain subspace among the plurality of subspaces.

FIG. 5 shows a leaf having the characteristic of the diseased state and a leaf having the characteristic of the diseased state, which is output by masking the characteristic of the leaf in the healthy state of the original input image or the characteristic of the leaf in the diseased state and performing learning. It is a figure which shows an example of the leaf which has the characteristic of a healthy state. This can be rephrased as the learning unit 13 adding the characteristics of leaves in a healthy state or the characteristics of leaves in a diseased state to the original input image. By confirming the output image illustrated in FIG. 5, the learning unit 13 can suitably determine whether or not the learning is properly performed by the user.

Further, in this step S103, it can be confirmed that the learning unit 13 further learns the third classifier 12 as follows, and for example, the definition regarding the subspace of the feature amount space is suitably improved. .. FIG. 6 is a conceptual diagram showing an example of such learning processing.

(Learning phase of the third classifier 12)
The feature vector in the common subspace is input to the third classifier 12, and the first classifier 9 and the decoder 15 are trained so that the correct answer rate of the output of the third classifier 12 is low.

In other words, the learning unit 13
When the feature vector included in the intersection of the feature vectors in the feature quantity space corresponding to each of the plurality of types of input data is input to the third classifier 12, the classification of the third classifier 12 is performed. To reduce accuracy and, as mentioned above,
The difference between the image output by the decoder 15 and the image input to the first classifier 9 is reduced.
The first classifier 9 and the decoder 15 are trained.

The learning phase of the third classifier 12 will be described in more detail as follows.

Regarding the first classifier (encoder) 9 and the decoder (decoder) 15, if there is a difference between the input image to the first classifier 9 and the generated image output by the decoder 15, the difference (error A). The parameters of the first classifier 9 and the decoder 15 are changed so that (also referred to as) becomes smaller. By repeating the above processing using various input images, it becomes possible to generate an image that looks exactly like the input image (the above processing is also referred to as processing A).

Further, in the process according to the present embodiment, in addition to the above process, a third feature vector 25 included in the intersection of the feature vectors in the feature quantity space corresponding to each of the plurality of types of input data is used. Input to the classifier 12. Then, if the classification result by the third classifier 12 and the correct answer (input label) are different, an error (also referred to as error B) is sent to the first classifier 9. Then, the parameters of the decoder 15 are changed so that the error A becomes small, while the error A becomes small and the error B becomes large for the first classifier (encoder) 9. (The above process is also called process B).

In this way, process A and process B are performed at the same time.

In this way, the learning unit 13 has features unique to the first type of data in the intersection of the subspace corresponding to the first kind of data and the subspace corresponding to the second kind of data in the feature quantity space. The first classifier and the decoder 15 are trained so as not to include any of the data-specific features of the second type and the second type.

In step S104, the learning unit 13 performs supervised learning of the second classifier 11 by inputting the feature vector peculiar to the leaf in the healthy state and the feature vector peculiar to the leaf in the diseased state. Here, the feature vector in the space other than the intersection of the plurality of subspaces in the feature quantity space is input to the second classifier 11. In other words, among the components of the feature vector in the feature quantity space, each component in the space other than the common portion of the plurality of subspaces is input to the second classifier 11.

By inputting a feature vector in a space other than the intersection to the second classifier 11, the classification accuracy by the second classifier 11 can be improved.

FIG. 7 is a conceptual diagram showing the above-mentioned supervised learning. In the example of FIG. 7, the “encoder” has already been learned in step S103, and the parameters at that time are fixed and used. “Fully Connected” refers to the fully connected network of the second classifier 11. Further, "Decision" is information on the type output by the second classifier 11, and indicates information indicating whether the leaves shown in the image are in a healthy state or in a diseased state.

According to the configuration described above with reference to the flowchart of FIG. 2, it is difficult to explain or interpret that it is difficult to confirm whether or not the algorithm of the learning model has properly learned the characteristics of the object. The problem can be solved.

The first classifier 9 and the decoder 15 can generate pseudo input data used for learning. In step S102 and subsequent steps, the images generated by the first classifier 9 and the decoder 15 as the variational autoencoder may be used for the learning process.

Further, as described above with reference to the flowchart of FIG. 2, the classification device 1 also functions as a learning device 1 for learning the neural network. The learning device 1 is first input data that can take a plurality of types, and maps input data such as an image showing a plant leaf to a feature vector in a feature quantity space including a plurality of different subspaces. A learning unit 13 for learning the classifier 9 is provided, and the learning unit 13 is provided with a decoder 15 for inputting a feature vector in the feature quantity space, and as the input data, an image showing a leaf of a plant in a healthy state. When inputting first-class data such as, etc. to the first classifier 9, the decoder 15 outputs after masking the subspace corresponding to the second-class data-specific feature in the feature quantity space. In the masked learning in which the first classifier 9 and the decoder 15 are trained so that the difference between the data to be input and the input data becomes small, the leaves of a diseased plant are used as the input data. When the second type data such as the image to be shown is input to the first classifier 9, the partial space corresponding to the first type data-specific feature in the feature quantity space is masked and then decoded. In order to reduce the difference between the data output by the device 15 and the input data, the first classifier 9 and the decoder 15 are trained to perform learning with a mask.

Further, in the learning method using the learning device 1 or the like, the first classifier 9 that maps the input data that can take a plurality of types to the feature vector in the feature quantity space including a plurality of different subspaces is trained. The learning step includes a learning step, and when the first type of data is input to the first classifier 9 as the input data, the learning step is a portion of the feature quantity space corresponding to the feature peculiar to the second type of data. After masking the space, the first classifier 9 and the decoder 15 so that the difference between the data output by the decoder 15 that inputs the feature vector in the feature quantity space and the input data becomes small. In the case of learning with a mask for learning the above, when the second type of data is input to the first classifier 9 as the input data, it corresponds to the feature peculiar to the first type of data in the feature amount space. The first classifier 9 and decoding are performed so that the difference between the data output by the decoder 15 that inputs the feature vector in the feature quantity space and the input data is small after masking the subspace. It can be said that the configuration is such that learning with a mask is performed so that the vessel 15 is trained.

<Additional notes related to learning processing examples>
In the above-mentioned learning processing example, it is not essential to perform unsupervised learning of the first classifier 9 and the decoder 15 in advance using the learning data. That is, the classification device 1 may be configured to acquire an image of a plant leaf in step S101 and then perform masked learning in step S103.

[3. Classification processing example]
Hereinafter, an example of the input data classification process executed by the classification device 1 that has performed the above-mentioned learning will be described. FIG. 8 is a flowchart showing the flow of the classification process by the classification device 1 according to the present embodiment.

In step S201, the acquisition unit 5 acquires an image of a plant leaf as input data to be classified.

In step S202, the first classifier 9 maps the image acquired by the acquisition unit 5 to the feature vector in the feature space. At this time, the subspaces in the feature space to which the image is mapped differ from each other depending on whether the image is an image of a leaf in a healthy state or an image of a leaf in a diseased state.

In step S203, the second classifier 11 is a feature vector mapped by the first classifier 9, and the leaf of the plant indicated by the image to be classified is displayed by inputting the feature vector in the feature amount space. , Outputs information indicating whether it is in a healthy state or in a sick state.

In step S204, the output unit 17 causes the information output by the second classifier 11 to be displayed on, for example, a display or the like owned by the output unit 17.

[4. Importance evaluation process]
In order to further improve the interpretability of the classification process according to the present embodiment, the classification device 1 may be configured to perform a series of processes related to the importance evaluation listed below.

(4-1: Transformation of the coordinate system or feature vector of the feature space)
The control unit 3 of the classification device 1 converts the feature vector in the coordinate system of the feature space or the feature space, and inputs the converted feature space or feature vector to the second classifier 11. The user can easily confirm to what extent which region in the quantity space or which component in the feature vector affects the estimation result output by the second classifier 11.

That is, in general, if the feature vector before conversion is expressed as X, the feature vector after conversion is expressed as X', and the function that converts the vector is expressed as f.
X'= f (X)
By transforming the feature vector with the above and inputting the converted feature vector to the second classifier 11 or the decoder, which region in the feature space or which component in the feature vector is included in the estimation result. The user can easily confirm whether or not the influence is exerted. In addition, it is possible to easily confirm what kind of weight and non-linearity contribute to what kind of output.

Here, the specific example and the determination method of the function f do not limit the present embodiment, but for example, an example of the function f includes a conversion function in various methods such as backpropagation.

Further, in order to further improve the interpretability, as the above function f, a function in which the feature vector before conversion is added to X and at least a part of the estimation result output by the second classifier 11 is used as an argument is adopted. You may. As an example, when the component indicating a disease in the estimation result output by the second classifier 11 is expressed as Dout,
X'= f (X, Dout)
The feature vector may be converted by the above, and the feature vector before and after the conversion may be input to the decoder, respectively.

As a result, the user can easily confirm which component in the disease feature vector affects the estimation result of the disease to what extent. In addition, it is possible to easily confirm what kind of weight and non-linearity contribute to the estimation result of disease.

As an example of such conversion of the feature vector, as will be described later, the converted feature vector is generated by changing the component on which a relatively large weighting coefficient is multiplied among each component of the feature vector. Can be mentioned.

(4-2: Output of evaluation result)
The output unit 17 of the classification device 1 input the output image (also referred to as output image A1) of the decoder when the feature vector before conversion is input to the decoder and the feature vector after conversion to the decoder. The image (also referred to as a difference image) that visualizes the difference from the output image (also referred to as the output image A2) of the decoder in the case may be displayed.

As an example, the control unit 3 of the classification device 1 generates a difference image between the output image A1 and the output image A2 or a heat map image using the output image A1 and the output image A2 as a difference image, and the output unit 17 It may be configured to display different images in.

Further, when displaying these different images, it is preferable to display the parameters that characterize the conversion function f together. As a result, the user can easily confirm what kind of conversion is performed and how much the output image is affected.

[5. Differences from conventional technology]
In "2. Learning processing example" and "3. Classification processing example", the differences between the above-described configuration according to the present embodiment and the prior art will be supplemented. As an existing method for confirming which feature in the input data corresponds to which classification, for example, the following example can be given.
-Visualization of negative layer activation The activation of each layer and each filter can be used to identify where activation is for a particular image.
-Visual confirmation of features Increase activation by repeatedly adding gradation to the image. Finally, the user can see the changed part in the image.
・ Visualization of semantic dictionary
Use the global mean pooling of the last layer of the CNN and multiply it by only one linear layer to classify. After identifying the important channels in the global average pooling layer, visualizing the features makes it possible to identify important areas in a particular class image.
-Attention map There are several common algorithms that are classified as attention maps (general gradient, GRAD-CAM, induced backpropagation, integral gradient, etc.).

In general they use backpropagation gradients to account for the activation of a given class. As an example, GRAD-CAM can identify important regions in a particular class image by using global mean pooling + reverse gradient.

Most of the existing methods as described above are used to describe the results from classification networks such as AlexNet and VGG16 in the classification process such as determining whether a plant has a disease. However, the above-mentioned existing method has a problem that it is not possible to know which feature in the input data is extracted from the filter in the upper layer of the neural network. Also, in the above existing method, additional training is required to confirm which feature was used to give class activation. In the semantic dictionary method, the top-level neurons involved in the activation of each class can be visualized as a feature visualization map, but it is difficult to identify human-understandable features from the feature visualization map. Attention maps allow the creation of feature visualization maps for each class without the need for additional training, but similar to the above, it is difficult to identify user-understandable features from the feature maps. In addition, some methods, even the attention map method, require additional training. In addition, the above-mentioned existing method focuses only on features that depend on the size of a part of a leaf and the like, and cannot consider features that do not depend on the size such as colors that are difficult to express in a feature map.

On the other hand, according to the configuration according to the present embodiment, it is possible to confirm which feature included in the image corresponds to which classification, and what features are common to the plurality of classifications. Further, in the above configuration, since the training pipeline is used, no additional training is required.

Further, in the existing method, the entire neural network corresponding to the classification unit 7 in the present embodiment is trained collectively. On the other hand, the configuration according to the present embodiment is different from the existing method in that the first classifier 9 and the second classifier 11 are trained step by step.

Further, comparing the configuration according to the present embodiment with Fader Networks, when training Fader Networks, conditional training is performed in which each element of the encoder output vector is changed to 1-0. On the other hand, the configuration according to this embodiment uses different parts of the vector for the class, and instead of using 1-0, the class value is passed directly. Further, in the configuration according to the present embodiment, a classifier is used instead of the class vector, but this part is not included in Fader Networks. Therefore, the pipeline of algorithms in the configuration according to this embodiment can be classified using explainable features. Then, more class-specific information can be sent to the decoder layer. Since it is necessary to confirm the correct characteristics represented by the encoder, activation of the lower layer encoder in the decoder is not used in the configuration according to the present embodiment.

[6. Application example of the present application]
For example, unsupervised learning of a variational auto-encoder is performed using plant leaf images downloaded from the Plant Village Dataset, a plant image database that is open to the public on the Internet. The input image size is 256 × 256 pixels, the output of the encoder is an average value of 4096, and a standard deviation of 4096. The structure of the encoder is 6 convolution layers + 3 fully connected layers, and vice versa for the decoder, which is 3 fully connected layers + 6 deconvolution layers. Of the above learned variational autoencoders, the 4096-dimensional feature vector (latent variable) is defined as 1/4 as sound, 1/2 as common, and 1/4 as disease feature. Conditional learning of the above learned variational autoencoder is performed using the leaf image to which the healthy or disease class information is given. Here, in order to prevent the common feature portion of the latent variable from including the health and disease features, the encoder is learned so that the classification accuracy of the three-layer CNN by the health and disease classifier does not improve. In addition, in order to reflect the healthy characteristics in the healthy characteristics of the latent variable and the disease characteristics in the disease characteristics, the disease characteristics of the latent variables are random in the case of healthy leaves, and the healthy characteristics are random in the case of diseased leaves. Change to to learn encoders and decoders. In addition, a Generative Adversarial Network (GAN) is applied to the images and input images generated by these encoders and decoders to produce results that are easy for the user to understand. Supervised learning of a three-layer CNN health / disease classifier is performed by inputting a 2048-dimensional latent variable that connects the sound feature part and the disease feature part of the learned encoding.

[7. Example〕
In the configuration described above in "6. Application example of the present application", healthy tomato leaves and leaves suffering from tomato plague, healthy tomato leaves and leaves suffering from tomato mosaic disease, and healthy bell pepper leaves and bell pepper spots. When the classification process in the classification device 1 of the present embodiment was executed using three types of data sets of leaves affected by bacterial disease as input data, the average value of the classification accuracy in the above three sets was 94%. It was also confirmed that the feature amounts used included leaf color, shape, surface texture, and the like.

[8. Appendix 1 of the embodiment]
As described above, the classification device (learning device) 1 according to the present embodiment maps the acquisition unit 5 that acquires the input data that can take a plurality of types and the input data to the feature vector in the feature quantity space. A first classifier 9 and a second classifier 11 that inputs a feature vector in the feature quantity space and outputs information about the type are provided, and the second classifier 11 includes the plurality of types. It can be expressed that a feature vector other than the common part of the feature vector in the feature quantity space corresponding to each of the input data of is input.

In the classification device (learning device) 1 according to the present embodiment, as described above, the second classifier 11 has a common feature vector in the feature quantity space corresponding to each of the plurality of types of input data. Since the feature vector other than the part is input, more suitable classification processing can be realized. For example, the classification accuracy is improved as compared with the configuration in which the feature vector of the common portion is also input to the second classifier 11. ..

Further, in the classification device (learning device) 1 according to the present embodiment, as described above, the acquisition unit 5 acquires input data that can take a plurality of types. Here, the plurality of types are not limited to two types, and may be three or more types as described later.

Therefore, in the classification device (learning device) 1 according to the present embodiment, not only the classification into two classes (for example, the classification regarding whether the plant is healthy or diseased) but also the classification into multiple classes (for example, plants). It is possible to preferably perform the classification process not only for whether the disease is healthy or ill, but also for what kind of illness the disease is related to).

Further, as described above, the classifier (learning device) 1 according to the present embodiment learns the first classifier 9 that maps input data that can take a plurality of types to a feature vector in the feature amount space. A learning unit 13 and a third classifier 12 that receives a feature vector in the feature quantity space as input and outputs information about the type are provided, and the learning unit 13 corresponds to each of the input data of the plurality of types. When the feature vector included in the common part of the feature vector in the feature quantity space is input to the third classifier 12, the classification accuracy of the third classifier 12 is lowered, so that the first classifier is used. 9 is to be learned.

In the classification device (learning device) 1 according to the present embodiment, as described above, the learning unit 13 is included in the intersection of the feature vectors in the feature quantity space corresponding to each of the plurality of types of input data. When the feature vector is input to the third classifier 12, the first classifier 9 is trained so that the classification accuracy of the third classifier 12 becomes low, so that the interpretability can be improved. ..

[9. Appendix 2 of the embodiment]
In the above-described embodiment, the feature vector 21 showing the characteristic peculiar to the diseased leaf and the feature vector 23 showing the characteristic peculiar to the healthy leaf are distinguished and processed. The matters described in this specification are not limited.

As an example, the classification device (learning device) 1 according to the present embodiment has a feature vector 21 showing a feature peculiar to a diseased leaf and a feature vector 23 showing a feature peculiar to a healthy leaf. Instead of distinguishing them, they may be combined and treated as a classifiable feature vector 24.

Even in such a configuration, the classification device (learning device) 1 according to the present embodiment uses the acquisition unit 5 for acquiring input data that can take a plurality of types and the input data as a feature vector in the feature quantity space. A first classifier 9 for mapping and a second classifier 11 for inputting a feature vector in the feature quantity space and outputting information on the type are provided, and the second classifier 11 includes the plurality of the above. A feature vector other than the common part of the feature vector in the feature quantity space corresponding to each of the input data of the type of is input.

FIG. 9 is a conceptual diagram showing an example of learning processing when such a configuration is used, and is a drawing corresponding to FIG. As shown in FIG. 9, in the learning process according to this example, the feature vector in the feature space is the classifiable feature vector 24 and the common feature showing the features of the leaf itself common to the leaves in the healthy state and the diseased state. It is composed of a vector 25.

In other words, in the learning process according to the example, the feature space is
-The subspace corresponding to the classifiable feature vector (classifiable subspace) and
-It is a feature regardless of whether it is in a healthy state or a diseased state, and is composed of a subspace (common subspace) corresponding to the common feature of the leaf itself.

Further, in the learning process according to this example, learning of the first classifier 9, the decoder 15, and the third classifier 12 is performed without masking each subspace described in step S103. Since other configurations in the learning process of the first classifier 9, the decoder 15, and the third classifier 12 have been described in the above-described embodiment, the description will be omitted here.

Even in such a configuration, the classifier (learning device) 1 according to the present embodiment learns the first classifier 9 that maps input data that can take a plurality of types to a feature vector in the feature amount space. A learning unit 13 and a third classifier 12 that receives a feature vector in the feature quantity space as input and outputs information about the type are provided, and the learning unit 13 corresponds to each of the input data of the plurality of types. When the feature vector included in the common part of the feature vector in the feature quantity space is input to the third classifier 12, the classification accuracy of the third classifier 12 is lowered, so that the first classifier is used. Learn 9

Further, FIG. 10 is a conceptual diagram showing supervised learning by the second classifier 11 when the above configuration is used, and is a drawing corresponding to FIG. 7. In the example of FIG. 10, "Fully Connected" indicates a fully connected network included in the second classifier 11. Further, "Decision" is information on the type output by the second classifier 11, and indicates information indicating whether the leaves shown in the image are in a healthy state or in a diseased state. Here, the state of illness can be classified into multi-class such as illness A, illness B, illness C, and so on. Since other configurations in the learning process of the second classifier 11 have been described in the above-described embodiment, a new description will be omitted here.

According to the classification device (learning device) 1 configured as described above, it is not an alternative classification of healthy or illness, but for example, healthy, illness A, illness B, illness C, ... Like classification, multi-stage classification (multi-class classification) can be easily performed.

[10. Appendix 3 of the embodiment]
In the above-described embodiment, the importance evaluation process for further improving the interpretability of the classification process has been described, but a specific processing example for improving the interpretability of the classification process will be described below.

FIG. 11 is a conceptual diagram showing an example of interpretability improvement processing according to this example. The classification device (learning device) 1 according to the present embodiment performs the processing of this example as an example as follows.

(Step S301)
First, for a certain input image, the first classifier 9 sets a reference feature point (feature vector) 39 in the classifiable feature space as shown by reference numeral 31 in FIG.

(Step S302)
Subsequently, for a plurality of input images of the same class, the first classifier 9 sets a plurality of points in the vicinity of the reference feature point 39 in the classifiable feature space as shown by reference numeral 33 in FIG. Set (generate). The generation of a plurality of points in this step is generated by Monte Carlo sampling as an example.

(Step S303)
Subsequently, as shown by reference numeral 35 in FIG. 11, the second classifier 11 is a cluster to which the feature vectors in the feature quantity space corresponding to each of the plurality of types of input data belong in the classifiable feature space. To determine the hyperplane (separated hyperplane) in the feature space that defines the boundaries of clusters adjacent to each other.

In the example shown by reference numeral 35 in FIG. 11, the determined separation hyperplane 41 separates the clusters classified as healthy and the clusters classified as diseased.

(Step S304)
Subsequently, the second classifier 11 determines one or a plurality of feature vectors located in the vicinity of the hyperplane determined in step S303, and determines the determined one or a plurality of feature vectors in the decoder 15 ("" in FIG. 11 ". Input to the trained decoder "). The feature vector input to the decoder 15 may have a configuration that can be specified by the user.

In the example shown by reference numeral 37 in FIG. 11, a diseased leaf image is generated by inputting the feature point 39 (classified as a disease) set in step S301 into the decoder 15 (“learned decoder” in FIG. 11). Further, the feature vector located in the vicinity of the feature point 39 (near the separation hyperplane 41) and classified into a type (class) different from the feature point 39 is input to the decoder 15 to be sound. Generate a leaf image. In the example shown by reference numeral 37 in FIG. 11, more specifically, as a feature vector located in the vicinity of the feature point 39 (near the separation hyperplane 41), the feature of the sound point closest to the reference feature point 39. Vectors are used.

(Step S305)
Subsequently, the output unit 17 displays the image data output by the decoder 15 on the display panel (display unit). As an example, the output unit 17 displays the healthy leaf image and the diseased leaf image generated by the decoder 15 in a comparable manner.

The classification device (learning device) 1 configured as described above is an image generated from feature vectors classified into different types, and displays images classified into different types (classes).

FIG. 12 is a diagram showing an example of an input image to be input to the classification device (learning device) 1 and an example (display example) of the generated diseased leaf image and healthy leaf image. As shown in FIG. 12, the classification device (learning device) 1 generates a healthy leaf image that is barely classified as healthy from the input healthy leaf image, or is barely classified as a disease from the input healthy leaf image. It is possible to generate a diseased leaf image.

Further, for a plurality of types of teacher images, as shown by reference numeral 33 in FIG. 11, a plurality of clusters are generated by generating feature points in the classifiable feature space (teacher image is healthy, disease A). , Illness B, Illness C, 4 or more clusters; even though it is healthy, the clusters may be divided into two). By linearly interpolating the feature vector toward the feature point of each center of gravity and outputting it by the decoder 15 as shown by reference numeral 37 in FIG. 11, a typical healthy leaf, a typical diseased A leaf, and a typical one are output. Since it is possible to generate an image that gradually changes to a disease B leaf, etc., it is possible to visualize the features used for classification by the model as shown in FIG.

In this way, the user can easily grasp the criteria by which the classification device (learning device) 1 classifies. Therefore, according to the classification device (learning device) 1 configured as described above, the interpretability can be improved.

Further, the classification device (learning device) 1 according to this example may further perform the following processing in order to determine the feature amount that has a greater influence on the classification result. FIG. 13 is a conceptual diagram illustrating a processing example for determining a feature amount that has a greater influence on the classification result.

(Step S401)
The second classifier 11 is trained using the plurality of points generated in step S302 (see reference numeral 51 in FIG. 13) as teacher data. Alternatively, the classification device (learning device) 1 is configured to include another classifier, and the other classifier is trained using a plurality of points generated in S302 as teacher data. Here, in this example, a linear classifier can be used as the second classifier 11 or another classifier.

Reference numeral 52 in FIG. 13 indicates a set of weighting factors W ^{c in the second classifier 11 or other classifier thus learned.}
W ^c = [W ₁₁ , W ₁₂ , W ₁₃ , ..., W ₂₁ , W ₂₂ , W ₂₃ , ...]
And the feature vector X ^c
X ^c = [X ₁ , X ₂ , X ₃ , ...]
^{And the vector Y c} showing the classification result by the second classifier 11 or another classifier.
Y ^c = [Y ₁ , Y ₂ ]
Shows the relationship with.

(Step S402)
Subsequently, the second classifier 11 or another classifier is a component (also referred to as an important component) of the ^{feature vector X c on} which a relatively large coefficient is multiplied among the weighting coefficients included in the weighting coefficient ^{set W c.} To identify. As an example, the second classifier 11 or another classifier identifies the component of the ^{feature vector X c} to which the coefficient is most multiplied among the weighting coefficients included in the weighting coefficient ^{set W c.}

The method of specifying the important component of the feature vector is not limited to the above example. For example, a _{component (X b ) having a larger coefficient × component value (W ab} × X _b value) is selected as an important component. May be specified as. In general, even if the value of the coefficient is relatively large, when the value of the component to be multiplied is close to 0, the contribution of the ^{component to the classification result vector Y c is small.}

(Step S403)
Subsequently, the second classifier 11 or another classifier generates the converted feature vector by changing the value of the important component specified in step S402. Then, the control unit 3 generates a decoded image by inputting the generated feature vector after conversion into the decoder 15 (“learned decoder” in reference numeral 53 in FIG. 13). Here, it is preferable that the value of the important component is changed a plurality of times in multiple steps. As an example, it is preferable to change the important components in the feature vector little by little to generate each decoded image. Reference numeral 53 in FIG. 13 indicates a plurality of image examples generated in multiple stages in this way. The “learned encoder” in reference numeral 53 in FIG. 13 refers to the first classifier 9.

As described above, in this example, the converted feature vector obtained by changing the relatively important component among the components of one or a plurality of feature vectors located in the vicinity of the hyperplane is transmitted to the decoder 15. Let me enter.

According to the classification device (learning device) 1 configured in this way, the decoded images obtained by changing the important components of the feature vector can be compared, so that the user can use the classification device (learning device) 1. It is possible to easily grasp what kind of standard is used for classification. Therefore, according to the classification device (learning device) 1 configured as described above, the interpretability can be improved.

[Example of realization by software]
The control block (particularly the acquisition unit 5, the classification unit 7, the learning unit 13 and the decoder 15) of the classification device (learning device) 1 is realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. It may be realized by software.

In the latter case, the classification device 1 includes a computer that executes instructions of a program that is software that realizes each function. The computer includes, for example, one or more processors and a computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. Further, a GPU (Graphics Processing Unit) may be used in combination to speed up the processing. As the recording medium, a "non-temporary tangible medium", for example, a ROM (Read Only Memory) or the like, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (Random Access Memory) for expanding the above program may be further provided. Further, the program may be supplied to the computer via an arbitrary transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. It should be noted that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present invention.

1 Classification device (learning device)
3 Control unit 5 Acquisition unit 7 Classification unit 9 First classifier 11 Second classifier 12 Third classifier 13 Learning unit 15 Decoder 17 Output unit (display unit)
19 Memory

Claims (19)

An acquisition unit that acquires input data that can take multiple types,
A first classifier that maps the input data to a feature vector in the feature space,
A second classifier that takes a feature vector in the feature space as an input and outputs information about the type is provided.
A classifier characterized in that a feature vector other than a common portion of the feature vector in the feature quantity space corresponding to each of the plurality of types of input data is input to the second classifier. The feature space includes a plurality of subspaces different from each other.
The first classifier is
A certain type of data as the input data is mapped to a feature vector in a certain subspace among the plurality of subspaces.
The classification according to claim 1, wherein the data of another type as the input data is mapped to a feature vector in another subspace different from the certain subspace among the plurality of subspaces. apparatus. A learning unit that trains the first classifier,
A third classifier that takes a feature vector in the feature space as an input and outputs information about the type is provided.
When the learning unit inputs the feature vector included in the common portion of the feature vector in the feature quantity space corresponding to each of the plurality of types of input data to the third classifier, the third classifier The classification device according to claim 1 or 2, wherein the first classifier is trained so that the classification accuracy of the classifier is lowered. The learning unit
To train the first classifier by performing conditional learning to input the labeled input data indicating whether the data is the first type data or the second type data to the first classifier. The classification device according to claim 3, which is characterized. A decoder that inputs a feature vector in the feature space is provided.
The learning unit
When the first type of data is input to the first classifier, the decoder outputs after masking the subspace corresponding to the feature unique to the second type of data in the feature quantity space. This is supervised learning in which the first classifier and the decoder are trained so that the difference between the data to be input and the input data is small.
When the second type of data is input to the first classifier, the decoder outputs after masking the subspace corresponding to the feature unique to the first type of data in the feature quantity space. The classification device according to claim 4, wherein learning with a mask for training the first classifier and the decoder is performed so that the difference between the data to be input and the input data is small. The learning unit has features unique to the first type of data and a common portion of the subspace corresponding to the first type of data and the subspace corresponding to the second type of data in the feature amount space. The classification device according to claim 5, wherein the first classifier is trained so as not to include any of the second type data-specific features. Prior to the masked learning, the learning unit performs the first classifier and the first classifier so that the difference between the data output by the decoder and the input data is small without masking the feature space. The classification device according to claim 5 or 6, wherein the decoder is trained. The second classifier is a cluster to which a feature vector in the feature space corresponding to each of the plurality of types of input data belongs, and is in the feature space that defines boundaries of clusters adjacent to each other. Determine the hyperplane,
The classification device is
A decoder in which one or more feature vectors located near the hyperplane determined by the second classifier, or a feature vector specified by the user, is input.
The classification device according to any one of claims 1 to 4, further comprising a display unit for displaying data output by the decoder. Among the components of one or a plurality of feature vectors located in the vicinity of the hyperplane, the converted feature vector obtained by changing a relatively important component is input to the decoder. The classification device according to claim 8. The input data is image data relating to an organism, and the plurality of types include any one of claims 1 to 9 relating to whether the organism is healthy or ill. The classification device described in the section. A learning unit for learning a first classifier that maps input data that can take multiple types to a feature vector in a feature space containing a plurality of different subspaces is provided.
The learning unit
A decoder that inputs a feature vector in the feature space is provided.
When first-class data is input to the first classifier as the input data, the decoding device after masking the subspace corresponding to the second-class data-specific feature in the feature quantity space. This is masked learning in which the first classifier and the decoder are trained so that the difference between the data output by the data and the input data is small.
When the second type of data is input to the first classifier as the input data, the partial space corresponding to the feature peculiar to the first type of data is masked in the feature quantity space, and then the said. A learning device characterized in that learning with a mask for learning the first classifier and the decoder is performed so that the difference between the data output by the decoder and the input data becomes small. A learning unit that trains a first classifier that maps input data that can take multiple types to a feature vector in the feature space.
A third classifier that takes a feature vector in the feature space as an input and outputs information about the type is provided.
When the learning unit inputs the feature vector included in the common portion of the feature vector in the feature quantity space corresponding to each of the plurality of types of input data to the third classifier, the third classifier A learning device characterized in that the first classifier is trained so that the classification accuracy of the classifier is lowered. A classification method performed by the device,
An acquisition step to acquire input data that can take multiple types, and
A first classification step of mapping the input data to a feature vector in the feature space,
It includes a second classification step in which a feature vector in the feature space is input and information about the type is output.
In the second classification step, a classification method characterized in that a feature vector other than a common portion of the feature vector in the feature quantity space corresponding to each of the plurality of types of input data is input. Includes a learning step that trains a first classifier that maps input data that can be of multiple types to feature vectors in a feature space that includes multiple subspaces that differ from each other.
The learning step
When first-class data is input to the first classifier as the input data, the feature quantity space is masked with a subspace corresponding to the feature specific to the second-class data, and then the feature quantity is used. Masked learning that trains the first classifier and the decoder so that the difference between the data output by the decoder that inputs the feature vector in space and the input data becomes small.
When the second type of data is input to the first classifier as the input data, the partial space corresponding to the feature peculiar to the first type of data is masked in the feature quantity space, and then the above. Performs supervised learning to train the first classifier and the decoder so that the difference between the data output by the decoder that inputs the feature vector in the feature space and the input data becomes small. A learning method characterized by. It includes a learning step that trains a first classifier that maps input data that can be of multiple types to a feature vector in the feature space.
In the learning step, the feature vector included in the common portion of the feature vector in the feature quantity space corresponding to each of the plurality of types of input data is input, and the feature vector in the feature quantity space is input, and the type is described. A learning method characterized in that the first classifier is trained so that the classification accuracy of the third classifier becomes low when the information is input to the third classifier that outputs information about the third classifier. A control program for operating a computer as the classification device according to claim 1, wherein the computer functions as the acquisition unit, the first classifier, and the second classifier. A control program for operating a computer as a learning device according to claim 11 or 12, wherein the computer functions as a learning unit. A computer-readable recording medium on which the control program according to claim 16 is recorded. A computer-readable recording medium on which the control program according to claim 17 is recorded.

Images