JP2023078763A - Classification device configured to execute classification processing using machine learning model, method, and computer program - Google Patents

Abstract

To provide a technique for improving accuracy of classification processing.SOLUTION: A machine learning model including one or more known classes and one or more dummy classes, is trained with a first training data group for the known classes and a second training data group for the dummy classes. A classification processing unit determines a class of data to be classified as "unknown" when a result of classifying the data to be classified using the machine learning model is a dummy class.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to a classifying device, method, and computer program that perform classifying processing using a machine learning model.

Patent Documents 1 and 2 disclose what is called a capsule network as a vector neural network type machine learning model using vector neurons. A vector neuron means a neuron whose inputs and outputs are vectors. A capsule network is a machine learning model whose network nodes are vector neurons called capsules. Vector neural network type machine learning models such as capsule networks can be used to classify input data.

U.S. Pat. No. 5,210,798 International Publication No. 2019/083553

However, in the prior art, there are cases where the class classification result of data to be classified that should be determined as unknown is erroneously determined to be a known class, so there has been a demand to improve the accuracy of class classification processing.

According to a first aspect of the present disclosure, there is provided a classifying device that performs classifying processing of data to be classified using a machine learning model including a vector neural network having a plurality of vector neuron layers. This class classification device includes a class classification processing unit that executes the class classification process. The machine learning model has one or more known classes and one or more dummy classes, and the machine learning model includes a first training data group for the known classes and a set of dummy classes for the dummy classes. Learning is executed using the second teacher data group of . The class classification processing unit determines that the class of the data to be classified is unknown when the class classification result of the data to be classified using the machine learning model is the dummy class.

According to a second aspect of the present disclosure, a method is provided for performing class classification processing of data to be classified using a machine learning model including a vector neural network having multiple vector neuron layers. This method includes (a) a step of inputting the data to be classified into the machine learning model to obtain a classification result, wherein the machine learning model includes one or more known classes and one or more dummy classes. and wherein the machine learning model is trained using a first training data group for the known class and a second training data group for the dummy class; (b) determining the class of the data to be classified as unknown when the class classification result is the dummy class;

According to a third aspect of the present disclosure, there is provided a computer program that causes a processor to perform class classification processing of data to be classified using a machine learning model that includes a vector neural network having multiple vector neuron layers. This computer program includes (a) a process of inputting the data to be classified into the machine learning model to obtain a classification result, wherein the machine learning model includes one or more known classes and one or more dummy classes. a class, wherein the machine learning model is trained using a first training data group for the known class and a second training data group for the dummy class; and (b) determining the class of the data to be classified as unknown when the class classification result is the dummy class.

1 is a block diagram showing a class classification system according to a first embodiment; FIG. Explanatory drawing which shows the structural example of a machine-learning model. FIG. 4 is an explanatory diagram showing the class structure of a machine learning model according to the first embodiment; 4 is a flow chart showing a processing procedure of a preparatory step in the first embodiment; FIG. 4 is an explanatory diagram showing how a patch image is created from a sample image; Explanatory drawing which shows a characteristic spectrum. Explanatory drawing which shows the structure of a known-feature spectrum group. 4 is a flow chart showing a processing procedure of a class classification step in the first embodiment; FIG. 9 is an explanatory diagram showing the class configuration of a machine learning model in the second embodiment; 9 is a flow chart showing a processing procedure of a preparatory step in the second embodiment; FIG. 4 is an explanatory diagram showing clustering processing of sample images; Explanatory drawing which shows the grouped sample image. FIG. 4 is an explanatory diagram showing a teacher data group composed of patch images extracted from sample images; FIG. 11 is an explanatory diagram showing a method of creating a dummy class teacher data group according to the second embodiment; 9 is a flow chart showing a processing procedure of a class classification step in the second embodiment; FIG. 11 is an explanatory diagram showing the class structure of a machine learning model according to the third embodiment; FIG. 11 is an explanatory diagram showing a method of creating a dummy class teacher data group according to the third embodiment; FIG. 11 is an explanatory diagram showing a method of creating a dummy class teacher data group in the fourth embodiment; 11 is a flow chart showing a processing procedure of a preparatory step in the fifth embodiment; FIG. 12 is an explanatory diagram showing a performance preference selection screen in the fifth embodiment; FIG. Explanatory drawing which shows the 1st calculation method of the similarity degree according to class. Explanatory drawing which shows the 2nd calculation method of the similarity degree according to class. Explanatory drawing which shows the 3rd calculation method of the similarity according to class.

A. First embodiment:
FIG. 1 is a block diagram showing a class classification system according to the first embodiment. This class classification system includes an information processing device 100 and a camera 400 . The camera 400 is for taking an image of an article to be sorted. As the camera 400, a camera that captures a color image may be used, or a camera that captures a monochrome image or a spectral image may be used. In the present embodiment, images captured by the camera 400 are used as teacher data and data to be classified, but data other than images may be used as teacher data and data to be classified. In this case, instead of the camera 400, a data acquisition device to be classified according to the type of data is used.

The information processing apparatus 100 has a processor 110 , a memory 120 , an interface circuit 130 , and an input device 140 and a display device 150 connected to the interface circuit 130 . A camera 400 is also connected to the interface circuit 130 . For example, without limitation, processor 110 not only has the ability to perform the processing detailed below, but also displays data obtained by the processing and data generated during the processing on display device 150. It also has the function to

The processor 110 functions as a learning execution unit 112 that executes learning of a machine learning model, and a class classification processing unit 114 that executes class classification processing for data to be classified. The class classification processor 114 includes a similarity calculator 310 and a class discriminator 320 . The learning execution unit 112 and the class classification processing unit 114 are realized by the processor 110 executing computer programs stored in the memory 120 . However, the learning execution unit 112 and the class classification processing unit 114 may be realized by hardware circuits. A processor in the present disclosure is a term that also includes such hardware circuitry. Also, the one or more processors that perform the learning process and the classifying process may be processors included in one or more remote computers connected via a network.

The memory 120 stores a machine learning model 200, a first training data group TD1 for known classes, a second training data group TD2 for dummy classes, and a known feature spectrum group GKSp. The machine learning model 200 is used for processing by the classification processor 114 . A configuration example and operation of the machine learning model 200 will be described later. The teacher data groups TD1 and TD2 are collections of labeled data used for learning of the machine learning model 200. FIG. In this embodiment, the teacher data groups TD1 and TD2 are sets of image data. The known feature spectrum group GKSp is a set of feature spectra obtained when training data is input again to the machine learning model 200 that has already been learned. A feature spectrum will be described later.

FIG. 2 is an explanatory diagram showing the configuration of the machine learning model 200. As shown in FIG. This machine learning model 200 has an input layer 210 , an intermediate layer 280 and an output layer 260 . Intermediate layers 280 include convolutional layer 220 , primary vector neuron layer 230 , first convolutional vector neuron layer 240 , and second convolutional vector neuron layer 250 . Output layer 260 is also referred to as "classification vector neuron layer 260". Of these layers, the input layer 210 is the lowest layer and the output layer 260 is the highest layer. In the following description, each layer of the intermediate layer 280 and the output layer 260 will be called "Conv layer 220", "PrimeVN layer 230", "ConvVN1 layer 240", "ConvVN2 layer 250", and "ClassVN layer 260", respectively.

Although two convolution vector neuron layers 240 and 250 are used in the example of FIG. 2, the number of convolution vector neuron layers is arbitrary and the convolution vector neuron layers may be omitted. However, it is preferable to use one or more convolutional vector neuron layers.

An image having a size of 32×32 pixels is input to the input layer 210 . The configuration of each layer other than the input layer 210 can be described as follows.
Conv layer 220: Conv[32,4,2]
- PrimeVN layer 230: PrimeVN[16,1,1]
ConvVN1 layer 240: ConvVN1[12,3,2]
ConvVN2 layer 250: ConvVN2[8,4,1]
ClassVN layer 260: ClassVN[M,4,1]
・Vector dimension VD: VD=16
In the description of each of these layers, the string before parentheses is the layer name, and the numbers inside the parentheses are the number of channels, surface size of the kernel, and stride, respectively. For example, the layer name of the Conv layer 220 is “Conv”, the number of channels is 32, the kernel surface size is 4×4, and the stride is 2. These descriptions are shown under each layer in FIG. The hatched rectangle drawn in each layer represents the surface size of the kernel used in calculating the output vector of the adjacent upper layer. In this embodiment, since the input data is image data, the surface size of the kernel is also two-dimensional. Note that the parameter values used in the description of each layer are examples and can be changed arbitrarily.

The input layer 210 and the Conv layer 220 are layers composed of scalar neurons. The other layers 230-260 are layers composed of vector neurons. A vector neuron is a neuron whose input and output are vectors. In the above description, the dimensions of the output vectors of the individual vector neurons are 16 and constant. In the following, the term "node" is used as a superordinate concept for scalar neurons and vector neurons.

In FIG. 2, for the Conv layer 220, a first axis x and a second axis y defining the plane coordinates of the node array and a third axis z representing depth are shown. It also shows that the Conv layer 220 has sizes of 15, 15, and 32 in the x, y, and z directions. The size in the x direction and the size in the y direction are called "resolution". The size in the z direction is the number of channels. These three axes x, y, and z are also used as coordinate axes indicating the position of each node in other layers. However, in FIG. 2, illustration of these axes x, y, and z in layers other than the Conv layer 220 is omitted.

As is well known, the resolution W1 after convolution is given by the following equation.
W1=Ceil{(W0-Wk+1)/S} (A1)
Here, W0 is the resolution before convolution, Wk is the surface size of the kernel, S is the stride, and Ceil{X} is a function for rounding up the decimal part of X.
The resolution of each layer shown in FIG. 2 is an example when the resolution of the input data is 32, and the actual resolution of each layer is appropriately changed according to the size of the input data.

The ClassVN layer 260 has M channels. M is the number of classes discriminable using the machine learning model 200 . In this embodiment, M is an integer of 2 or greater. The M channels of the ClassVN layer 260 output decision values Class_1 to Class_M for the M classes. Typically, the class having the largest value among these determination values Class_1 to Class_M is determined as the corresponding class of the data to be classified. Further, when the largest value among the determination values Class_1 to Class_M is less than a predetermined threshold value, it is possible to determine that the class of the data to be classified is unknown.

In FIG. 2, partial regions Rn in each layer 220, 230, 240, 250, 260 are also depicted. The suffix “n” of the partial region Rn is the code for each layer. For example, partial region R220 indicates a partial region in Conv layer 220. FIG. The “partial region Rn” is specified by a plane position (x, y) defined by the position of the first axis x and the position of the second axis y in each layer, and a plurality of channels along the third axis z is a region containing The partial region Rn has dimensions of “Width”דHeight”דDepth” corresponding to the first axis x, the second axis y, and the third axis z. In this embodiment, the number of nodes included in one “partial region Rn” is “1×1×number of depths”, that is, “1×1×number of channels”.

As shown in FIG. 2 , a feature spectrum Sp, which will be described later, is calculated from the output of the ConvVN2 layer 250 and input to the similarity calculation section 310 . The similarity calculator 310 uses this feature spectrum Sp and a known feature spectrum group GKSp created in advance to calculate the similarity for each class, which will be described later. It is also possible to determine the corresponding class of the data to be classified using this similarity for each class.

In the present disclosure, the vector neuron layer used for similarity calculation is also called "specific layer". A vector neuron layer other than the ConvVN2 layer 250 may be used as the specific layer, and any number of vector neuron layers greater than or equal to one may be used. The configuration of the feature spectrum and the method of calculating the similarity using the feature spectrum will be described later.

FIG. 3 is an explanatory diagram showing the class configuration of the machine learning model 200 in the first embodiment. A single machine learning model 200 is used to classify data Di to be classified. In the first embodiment, the M classes of the machine learning model 200 consist of M−1 known classes and one dummy class. That is, the M class determination values Class_1 to Class_M are composed of M-1 known class determination values Known_1 to Known_Mk and one dummy class determination value Dummy_1. where Mk=M−1. Since M is an integer of 2 or more, the number of known classes M−1 is 1 or more. Note that when the number of dummy classes is m, m may be 1, or may be 2 or more.

FIG. 4 is a flow chart showing the processing procedure of the machine learning model preparation process. In step S110, the learning executing unit 112 generates a plurality of sample images by photographing a plurality of samples using the camera 400. FIG. In step S120, the learning executing unit 112 applies preprocessing to the sample image to create a patch image. As preprocessing, for example, processing such as resolution adjustment and data normalization (min-max normalization) can be used.

FIG. 5 is an explanatory diagram showing how a patch image is created from a sample image. Here, a plurality of small-sized patch images PD are cut out from the sample image SD. This example shows an example in which the sample image SD is divided into a plurality of patch images PD by dividing lines indicated by dashed lines. Each of these individual patch images PD is used as an image for teacher data. A larger number of patch images PD may be extracted from one sample image SD by setting the stride of the clipping position of the patch image PD smaller than the size of one side of the patch image PD. In this disclosure, the sample image SD is also called "sample data SD", and the patch image PD is also called "patch data PD". Note that the sample image SD may be used as it is as the image for the teacher data without extracting the patch image PD.

In this embodiment, a plurality of patch images PD extracted from one sample image SD are used as an image group belonging to one class. Since the number of known classes is Mk in the machine learning model 200 shown in FIG. 3, Mk sample images SD are captured in step S110, and a plurality of patch images PD are extracted from each sample image SD in step S120. be.

In step S130, the learning execution unit 112 creates the first training data group TD1 for the known class by assigning labels to the patch images PD. In this embodiment, Mk teacher data groups are created by assigning one of Mk labels 1 to Mk to each patch image PD. These labels correspond to Mk known classes Known_1 to Known_Mk of the machine learning model 200 shown in FIG. In this disclosure, "label" and "class" mean the same thing.

At step S140, the learning execution unit 112 creates a second training data group TD2 for the dummy class. In the machine learning model 200 shown in FIG. 3, the number of dummy classes is one, and the second teacher data group TD2 is created for the dummy class Dummy_1. The second training data group TD2 is created so as to improve the accuracy of class classification of known classes. Therefore, it is preferable that the images included in the second training data group TD2 are created without the characteristics of the images included in the first training data group TD1. Specifically, for example, it is assumed that numbers 0 to 9 are set as known classes and handwritten numbers are classified into classes. In this case, images of handwritten characters such as alphabets can be used as the images of the second training data group TD2 for the dummy class. In this way, handwritten digits can be correctly classified into one of classes 0 to 9, and handwritten characters can be prevented from being erroneously discriminated as digits, so that it is possible to improve the classification accuracy of handwritten digits. As an effect of the dummy class, it can be expected that the ability of the machine learning model 200 to distinguish between the input data is improved by increasing the variations of the learning data. In other words, the performance of determining that input data not included in the dummy class, such as Greek letters and symbols other than the alphabet, as unknown is improved.

In step S150, the learning execution unit 112 executes learning of the machine learning model 200 using the first training data group TD1 for the known class and the second training data group TD2 for the dummy class. After learning is completed, the trained machine learning model 200 is stored in the memory 120 .

In step S160, the learning execution unit 112 re-inputs a plurality of teacher data to the machine learning model 200 that has already been trained to generate a known feature spectrum group GKSp. The known feature spectrum group GKSp is a set of feature spectra described below.

FIG. 6 is an explanatory diagram showing a feature spectrum Sp obtained by inputting arbitrary input data to the learned machine learning model 200. As shown in FIG. As shown in FIG. 2, in this embodiment a feature spectrum Sp is created from the output of the ConvVN2 layer 250 . The horizontal axis of FIG. 6 is the position of the vector elements for the output vectors of the nodes included in one partial region R250 of the ConvVN2 layer 250. In FIG. The position of this vector element is represented by a combination of the output vector element number ND and the channel number NC at each node. In this embodiment, since the vector dimension is 16 (the number of elements of the output vector output by each node), the element numbers ND of the output vector are 16 from 0 to 15. FIG. Also, since the number of channels in the ConvVN2 layer 250 is eight, there are eight channel numbers NC from 0 to 7. In other words, this feature spectrum Sp is obtained by arranging a plurality of element values of the output vector of each vector neuron included in one partial region R250 over a plurality of channels along the third axis z.

The vertical axis in FIG. 6 indicates the feature value _CV at each spectral position. In this example, the feature value _CV is the value _VND of each element of the output vector. Statistical processing such as centering to the mean value 0 may be performed on the characteristic value _CV . As the characteristic value _CV , a value obtained by multiplying the value _VND of each element of the output vector by a normalization factor described later may be used, or the normalization factor may be used as it is. In the latter case, the number of feature values _CV included in the feature spectrum Sp is equal to the number of channels, which is eight. Note that the normalization coefficient is a value corresponding to the vector length of the output vector of that node.

The number of feature spectra Sp obtained from the output of the ConvVN2 layer 250 for one input data is equal to the number of planar positions (x, y) of the ConvVN2 layer 250, that is, the number of partial regions R250, so 16. be.

The learning execution unit 112 re-inputs teacher data to the machine learning model 200 that has already been trained, calculates the feature spectrum Sp shown in FIG. 6, and registers it in the memory 120 as a known feature spectrum group GKSp.

FIG. 7 is an explanatory diagram showing the configuration of the known feature spectrum group GKSp. In this example, the set of known feature spectra GKSp obtained from the output of ConvVN2 layer 250 is shown. As the known feature spectrum group GKSp, it suffices if the one obtained from the output of at least one vector neuron layer is registered, and the known feature spectrum group obtained from the output of the ConvVN1 layer 240 or the ClassVN layer 260 is registered. may be made.

Each record of the known feature spectrum group GKSp has a parameter i indicating the order of labels or classes, a parameter k indicating the order of subregions Rn in the layer, a parameter q indicating the data number, and a known feature spectrum KSp. contains. The known feature spectrum KSp is the same as the feature spectrum Sp in FIG.

The class parameter i takes values from 1 to M. The parameter k of the partial region Rn takes a value indicating which of the plurality of partial regions Rn included in the specific layer, that is, which plane position (x, y). Since the ConvVN2 layer 250 has 16 partial regions R250, k=1-16. The data number parameter q indicates the number of teacher data with the same label, and takes values from 1 to max1 for class 1 and from 1 to maxM for class M.

The plurality of teacher data used in step S160 need not be the same as the plurality of teacher data used in step S150. However, even in step S160, if some or all of the multiple teaching data used in step S150 are used, there is an advantage that there is no need to prepare new teaching data.

FIG. 8 is a flow chart showing the processing procedure of the class classification process using the learned machine learning model 200. As shown in FIG. In step S<b>210 , the class classification processing unit 114 uses the camera 400 to photograph the classification target product, thereby generating classification target data. In step S220, the class classification processing unit 114 performs preprocessing on the data to be classified as necessary. This preprocessing preferably applies the same preprocessing as that performed in step S120 of FIG. The data to be classified after preprocessing is created as an image having the same size as the patch image PD shown in FIG. Note that preprocessing can be omitted. In step S<b>230 , the class classification processing unit 114 reads the learned machine learning model 200 and the known feature spectrum group GKSp from the memory 120 .

In step S<b>240 , the class classification processing unit 114 inputs data to be classified to the machine learning model 200 . In step S250, the class classification processing unit 114 obtains the feature spectrum Sp shown in FIG. 6 using the output of the ConvVN2 layer 250, which is the specific layer. In step S260, the similarity calculator 310 calculates a similarity using the feature spectrum Sp obtained in step S250 and the known feature spectrum group GKSp shown in FIG. As the degree of similarity, either the degree of similarity by class or the maximum degree of similarity without considering the class can be used.

The similarity S(Class) for each class can be calculated using, for example, the following equation.
S(Class)=max[G{Sp,KSp(Class,k)}] (A2)
Here, "Class" is the ordinal number for the class, G {a, b} is a function for obtaining the similarity between a and b, Sp is the feature spectrum obtained according to the data to be classified, KSp (Class, k) is the specific , k is the ordinal number of the known feature spectrum, and max[X] is the logical operation that takes the maximum value of X. As the function G{a,b} for obtaining the similarity, for example, a cosine similarity or a similarity using a distance such as the Euclidean distance can be used. The similarity S(Class) is the maximum similarity calculated between the feature spectrum Sp and each known feature spectrum KSp(Class,k) corresponding to a specific class. Such similarity S(Class) is obtained for each of M classes. The similarity S(Class) represents the extent to which data to be classified is similar to the features of each class. This degree of similarity S(Class) can also be used as descriptive information regarding the result of classification of data to be classified. Another method of calculating similarity for each class will be described later.

The maximum similarity S(All) that does not consider classes can be calculated using, for example, the following equation.
S(All)=max[G{Sp,KSp(k)}] (A3)
where KSp(k) denotes the k-th one of all known feature spectra. This maximum similarity S(All) is the maximum value of similarities between the feature spectrum Sp and all known feature spectra KSp. Since the known feature spectrum KSp(k) that gives the maximum similarity S(All) can be specified, the label, ie, the class can be specified from the known feature spectrum group GKSp shown in FIG. This maximum similarity S(All) can also be used as descriptive information for explaining the result of classifying whether data to be classified belongs to known data or unknown data.

In step S270, the class determination unit 320 determines the applicable class of the data to be classified. This determination can be made using at least one of the judgment values Class_1 to Class_M output from the ClassVN layer 260 and the degree of similarity obtained in step S250. For example, only the judgment values Class_1 to Class_M output from the ClassVN layer 260 may be used to determine the applicable class of the data to be classified. In this case, steps S250 and S260 described above may be omitted. Alternatively, the corresponding class of the data to be classified may be determined using only the similarity S(Class) for each class. In this case, when the similarity S(Class) of a certain class is equal to or greater than a predetermined threshold value, it can be determined that the data to be classified belongs to that class. On the other hand, if the similarity S(Class) for all classes is less than the threshold, it can be determined that the data to be classified belongs to an unknown class. When the maximum similarity S(All) is used as the similarity, if the maximum similarity S(All) is equal to or greater than the threshold, the class of the known feature spectrum that gives the maximum similarity S(All) is treated as the class of unclassified data. If it is less than the threshold, it can be determined as an unknown class. The maximum similarity S(All) can also be displayed as explanatory information explaining whether the data to be classified belongs to known data or unknown data. Furthermore, both the similarity and the judgment values Class_1 to Class_M of the ClassVN layer 260 may be used to determine the corresponding class of the data to be classified. For example, when the corresponding class determined from the similarity and the corresponding class determined from the determination values Class_1 to Class_M of the ClassVN layer 260 match, it is possible to determine that the data to be classified belongs to that class. can. If the corresponding class determined from the degree of similarity does not match the corresponding class determined from the determination values Class_1 to Class_M of the ClassVN layer 260, the data to be classified is determined to belong to an unknown class. be able to.

The class determination unit 320 determines the class of the data to be classified as unknown when the class classification result of the data to be classified is a dummy class. As a result, it is possible to correctly determine data to be classified that does not correspond to a known class as unknown.

As described above, in the first embodiment, class classification processing is executed using the machine learning model 200 having known classes and dummy classes, so the accuracy of class classification processing can be improved. Moreover, by using dummy classes, it is possible to expect an effect that the ability of the machine learning model 200 to discriminate differences in input data is improved by increasing variations in learning data.

B. Second embodiment:
FIG. 9 is an explanatory diagram showing the class structure of the machine learning model in the second embodiment. The device configuration of the class classification system in the second embodiment is almost the same as that shown in FIG. As shown in FIG. 9, the classification system of the second embodiment includes N machine learning models 200_1 to 200_N. N is an integer of 2 or more. Each of these machine learning models 200_1 to 200_N has the same configuration as the machine learning model 200 shown in FIG. However, the number of classes of individual machine learning models 200_1 to 200_N may be different values. Additional codes “_1” to “_N” added to the end of the machine learning model codes are added to distinguish the N machine learning models. When there is no need to distinguish from each other, these symbols "_1" to "_N" are omitted and simply referred to as "machine learning model 200". Also, hereinafter, the j-th machine learning model among the N is referred to as "machine learning model 200_j".

The j-th machine learning model 200_j has j_M classes. These j_M classes consist of j_M-1 known classes and one dummy class. That is, the judgment values Class_j_1 to Class_j_M of the j_M classes are composed of the judgment values Known_j_1 to Known_j_Mk of the j_Mk known classes and the judgment value Dummy_j_1 of one dummy class. where j_Mk=j_M−1. j_M is an integer of 2 or more, and the number of known classes j_Mk is 1 or more. When the number of dummy classes is m, m may be 1, or may be 2 or more.

FIG. 10 is a flow chart showing the processing procedure of the machine learning model preparation process in the second embodiment. The only difference from the preparation process in the first embodiment shown in FIG. 4 is that step S115 is added between step S110 and step S120, and step S140 is replaced with step S145. Other steps are almost the same as in FIG.

In step S115, the learning executing unit 112 executes processing for dividing the plurality of sample images into N groups. The number N of groups corresponds to the number N of machine learning models 200 shown in FIG.

FIG. 11 shows an example of dividing a plurality of sample images into two groups GP_1 and GP_2. Grouping can be performed, for example, by clustering processing using the k-means method. In the example of FIG. 11, a plurality of sample images are classified into two groups GP_1 and GP_2 centered on respective centers of gravity G1 and G2. Grouping may be performed using a method other than the k-means method. For example, a plurality of sample images may be arranged and classified into a plurality of groups according to their ordinal numbers. The number N of groups can be set to any value of 2 or more.

FIG. 12 is an explanatory diagram showing sample images grouped into N groups. The j-th group GP_j includes multiple sample images SD_j_1 to SD_j_Mk. Here, j_Mk is the number of known classes in the j-th machine learning model 200_j shown in FIG.

The N groups of sample images thus grouped are used in steps S120 and S130 of FIG. 10 to create training data for the N machine learning models 200_1 to 200_N. That is, patch images are created from the sample images in step S120, and labels are assigned to the patch images in step S130, thereby creating the first training data group for the known class. The processing contents of steps S120 and S130 are the same as in the first embodiment.

FIG. 13 is an explanatory diagram showing a first training data group composed of patch images extracted from the sample images shown in FIG. 12 in the second embodiment. The j-th first training data group TD_j includes a plurality of patch image groups PD_j_1 to PD_j_Mk. Each of the multiple patch image groups PD_j_1 to PD_j_Mk includes multiple patch images. The first patch image group PD_1_1 is extracted from the first sample image SD_1_1 shown in FIG. The same applies to other patch image groups.

In step S145, the learning execution unit 112 uses the teacher data group of the known class to create a second teacher data group for the dummy class. In the class classification system shown in FIG. 9, the number of dummy classes in each machine learning model 200_j is one, and a second teacher data group is created for the dummy class Dummy_j_1. When the number m of dummy classes is 2 or more, a second teacher data group is created for each dummy class.

FIG. 14 is an explanatory diagram showing a method of creating a second training data group for a dummy class in the second embodiment. The second training data group Dummy_PD_1_1 for the dummy class in the first machine learning model 200_1 is N−1 first training data groups TD_2 to TD_N for the other N−1 machine learning models 200_2 to 200_N. data selected from Also, the second teacher data group Dummy_PD_1_1 is selected so as to include a part of each of the N-1 first teacher data groups TD_2 to TD_N. This selection is preferably made randomly. In order to ensure that the second teacher data group Dummy_PD_1_1 includes a part of each of the N−1 first teacher data groups TD_2 to TD_N, the number of patch images forming the second teacher data group Dummy_PD_1_1 is , N is preferably set to a sufficiently large number. Similarly, the second teacher data group Dummy_PD_j_1 for the dummy class in the j-th machine learning model 200_j other than the first is also the N-1 first teacher data groups for the other N-1 machine learning models. , and are selected so as to contain a portion of each of the N−1 first teacher data groups.

Even when the number m of dummy classes is 2 or more, teacher data for those dummy classes can be similarly created. That is, the second teacher data group for the m dummy classes of each machine learning model 200 is selected from among the N-1 first teacher data groups for the other N-1 machine learning models. Selected data, selected to include a portion of each of the N−1 first teacher data groups.

If the second training data group Dummy_PD_j_1 for the dummy class is created as described above, in each machine learning model 200_j, data to be classified that should be determined as a known class in other machine learning models is determined as a dummy class. can do. As a result, it is possible to prevent erroneous determination as a known class in two or more machine learning models, thereby improving the accuracy of class classification processing for the entire class classification system. Since the processing after step S150 in FIG. 10 is almost the same as in the first embodiment, the description thereof is omitted.

FIG. 15 is a flow chart showing the processing procedure of the class classification process in the second embodiment. The only difference from the class classification process in the first embodiment shown in FIG. 8 is that step S235 is added between step S230 and step S240, and that processing after step S280 is added. , other steps are almost the same as in FIG.

At step S235, the class determination unit 320 selects one of the N machine learning models 200_1 to 200_N. In steps S240 to S270, class classification processing is performed using one selected machine learning model.

In step S280, the class determination unit 320 determines whether or not the processing of steps S235 to S270 has been completed for all of the N machine learning models 200. If not completed, the process returns to step S235, and if completed, the process proceeds to step S300.

In step S235, the data to be classified may be classified into one of the groups based on the same criteria as in step S115 of FIG. 10 described above, and a machine learning model using the teacher data of that group may be selected. In this case, step S280 is omitted.

In step S300, the class determination unit 320 determines whether the class classification result in any of the N machine learning models 200 is a known class. If there is a class classification result of known class, it is determined in step S310 that the data to be classified corresponds to the known class. On the other hand, if all the class classification results by the N machine learning models are dummy classes, or if the degree of similarity is less than the threshold, the class of the data to be classified is determined to be unknown in step S320. In step S330, the class classification result of the data to be classified is output.

As described above, in the second embodiment, similarly to the first embodiment, the machine learning model 200 having known classes and dummy classes is used to execute the class classification process, so the accuracy of the class classification process is improved. be able to. Also, as in the first embodiment, by increasing the variation of learning data, an effect can be expected that the machine learning model 200 can improve its ability to distinguish between different input data. In particular, in the second embodiment, as shown in FIG. 9, when classifying into many known classes using N machine learning models 200, the accuracy of class classification can be improved. Furthermore, in the second embodiment, as shown in FIG. 14, the second teacher data group for the dummy class of each machine learning model 200 is N The second teacher data group for the dummy class can be easily created because the data is selected from the -1 first teacher data group.

C. Third embodiment:
FIG. 16 is an explanatory diagram showing the class configuration of the machine learning model in the third embodiment. The device configuration of the class classification system in the third embodiment is almost the same as that shown in FIG. As shown in FIG. 16, the classification system of the third embodiment also includes N machine learning models 200_1 to 200_N, like the second embodiment. N is an integer of 2 or more.

The j-th machine learning model 200_j has j_M classes. These j_M classes are composed of j_Mk known classes and N-1 dummy classes. That is, the judgment values Class_j_1 to Class_j_M of the j_M classes are composed of the judgment values Known_j_1 to Known_j_Mk of the j_Mk known classes and the judgment values Dummy_j_1 to Dummy_j_N-1 of the N-1 dummy classes. Here, the total number of classes j_M is 2 or more, and the number j_Mk of known classes is 1 or more. Since N is an integer of 2 or more, the number of dummy classes N-1 is 1 or more. In a typical example, N is 3 or more, and the number N-1 of dummy classes is 2 or more.

FIG. 17 is an explanatory diagram showing a method of creating a second training data group for dummy classes in the third embodiment. Here, only the method of creating the second training data group Dummy_PD_1 for the dummy class in the first machine learning model 200_1 is shown. The second teacher data group Dummy_PD_1 is N-1 data selected from the N-1 first teacher data groups TD_2 to TD_N for the other N-1 machine learning models 200_2 to 200_N. Contains the groups Dummy_PD_1_2 to Dummy_PD_1_N. These N-1 data groups Dummy_PD_1_2 to Dummy_PD_1_N are selected so as to correspond to the N-1 first teacher data groups TD_2 to TD_N. Selection from the individual first teacher data groups is preferably performed at random. Similarly, the second teacher data group Dummy_PD_j_1 for the dummy class in the j-th machine learning model 200_j other than the first is N-1 first teacher data for the other N-1 machine learning models 200. Each is selected from among the group.

The preparation process and the classifying process in the third embodiment are almost the same as those in the second embodiment described with reference to FIGS. 10 and 15, so description thereof will be omitted.

The third embodiment described above also has substantially the same effects as the second embodiment described above. In addition, in the third embodiment, as shown in FIG. 17, N−1 second teacher data groups for N−1 dummy classes of the individual machine learning models 200 are used for the other N− Since it is selected so as to correspond to the N−1 first training data groups for one machine learning model 200, it is possible to easily obtain the second training data group for the dummy class of the N machine learning models 200. can be created in

D. Fourth embodiment:
FIG. 18 is an explanatory diagram showing a method of creating the second training data group for the dummy class in the fourth embodiment. The configuration of the class classification system in the fourth embodiment is the same as in the third embodiment shown in FIG. 16, and includes N machine learning models 200_1 to 200_N. Also, as shown in FIG. 16, the j_M classes of the j-th machine learning model 200_j are composed of j_Mk known classes and N-1 dummy classes. is the same as

FIG. 18 shows only the method of creating the second training data group Dummy_PD_1 for the dummy class in the first machine learning model 200_1. In the fourth embodiment, an average sample image SD_j_ave is created by averaging j_Mk sample images SD_j_1 to SD_j_Mk that make up each sample image group GP_j before creating the second training data group. As a result, N average sample images SD_1_ave to SD_N_ave are obtained. Note that the sample image group GP_j is the same as that shown in FIG. 12, and is the image group used to extract the first training data group for the known class of the machine learning model 200_j. That is, each piece of first training data included in the first training data group is data extracted from sample data having a size larger than that of the first training data.

The second training data group Dummy_PD_1 for the dummy class consists of N−1 average sample images SD_2_ave to SD_N_ave corresponding to the other N−1 machine learning models 200_2 to 200_N.
, N-1 data groups Dummy_PD_1_2 to Dummy_PD_1_N each extracted from . These N-1 data groups Dummy_PD_1_2 to Dummy_PD_1_N are extracted so as to correspond to N-1 average sample images SD_1_ave to SD_N_ave. A second teacher data group Dummy_PD_j for the dummy class in the j-th machine learning model 200_j other than the first is also created in the same manner.

The preparation process and the classifying process in the fourth embodiment are almost the same as those in the second embodiment described with reference to FIGS. 10 and 15, so description thereof will be omitted.

The fourth embodiment described above also has substantially the same effect as the second or third embodiment described above. In addition, in the fourth embodiment, as shown in FIG. 18, N−1 second teacher data groups for N−1 dummy classes of individual machine learning models 200 are combined with other N−1 Since it is extracted from the average sample image obtained by averaging the sample image group used to create the first teacher data group for the N machine learning models 200, the first training data group for the N machine learning models 200 2 A training data group can be easily created.

In the second to fourth embodiments described above, the second teacher data group for the dummy class in the j-th machine learning model 200_j is the first teacher data group for the known classes in the other N-1 machine learning models. They are common in that they are created using a data group or a sample image group. In this way, data to be classified that is determined to belong to a known class by any one machine learning model is determined to be a dummy class by the other N−1 machine learning models. Therefore, it is possible to reduce the possibility of being erroneously determined to be a known class in two or more machine learning models, so it is possible to improve the accuracy of class classification processing.

According to experiments conducted by the inventors, among the above-described second to fourth embodiments, the second embodiment is the most superior in terms of processing speed. In terms of classification accuracy, the third embodiment is the best, and the second embodiment is the second best. A user can select either embodiment in consideration of these performances.

E. Fifth embodiment:
FIG. 19 is a flow chart showing the processing procedure of the machine learning model preparation process in the fifth embodiment. The only difference from the preparation process in the second embodiment shown in FIG. 10 is that step S142 is added between step S130 and step S145, and other steps are substantially the same as in FIG.

In the fifth embodiment, the number of dummy classes in the N machine learning models 200 and the method of creating the second training data group for the dummy classes are determined according to the user's designation in step S142. In step S142, the learning execution unit 112 receives performance preferences regarding the performance of the machine learning model 200 from the user.

FIG. 20 is an explanatory diagram showing a screen WD for selecting performance preferences. This screen WD has, as the performance of the machine learning model 200, an option CH1 of "emphasis on speed" and an option CH2 of "emphasis on accuracy". The user can select either of these two options CH1, CH2. For example, when the user selects the option CH1 of "emphasis on speed", the learning execution unit 112 performs the second class described with reference to FIGS. Select the configuration and processing of the embodiment. On the other hand, when the user selects the option CH2 of "accuracy-oriented", the learning execution unit 112 performs the third class described with reference to FIGS. Select the configuration and processing of the embodiment. In this way, it is possible to select an appropriate number of dummy classes and a method of creating the second training data group according to performance preferences indicating which of performances such as processing speed and processing accuracy should be prioritized.

Note that the number of dummy classes corresponding to the two options CH1 and CH2 and the method of creating the second training data group for the dummy classes are not limited to the above-described second and third embodiments, and may be of other forms. You can adopt anything. Also, options other than the two options CH1 and CH2 described above may be used as options that can be selected in the performance preference. For example, performance with a good balance between processing speed and processing accuracy may be provided as another option.

F. Calculation method of similarity by class:
For example, any one of the following methods can be adopted as a method for calculating the similarity by class described above.
(1) A first calculation method M1 for obtaining similarity by class without considering the correspondence between the feature spectrum Sp and the partial region Rn in the group of known feature spectra GKSp
(2) A second calculation method M2 for obtaining the similarity by class between the corresponding partial regions Rn of the feature spectrum Sp and the group of known feature spectra GKSp.
(3) A third calculation method M3 for obtaining similarity by class without considering partial regions Rn at all
Hereinafter, methods for calculating similarities by class from the output of the ConvVN2 layer 250 will be sequentially described according to these calculation methods M1, M2, and M3.

FIG. 21 is an explanatory diagram showing the first calculation method M1 for class similarity. In the first calculation method M1, first, from the output of the ConvVN2 layer 250, which is a specific layer, the local similarity S(i,k) indicating the similarity to each class i for each partial region k is calculated according to the formula described later. be done. In the machine learning model 200 of FIG. 2, the number of partial regions R250 in the ConvVN2 layer 250 is 16, so the partial region parameter k takes a value of 1-16. From these local similarities S(i,k), one of the three class similarities Sclass(i) shown on the right side of FIG. 21 is calculated.

In the first calculation method M1, the local similarity S(i,k) is calculated using the following equation.
S(i,k)=max[G{Sp(k), KSp(i,k=all,q=all)}] (C1)
here,
i is a parameter indicating the class,
k is a parameter indicating the partial region Rn;
q is a parameter indicating a data number;
G{a,b} is a function for obtaining the similarity between a and b,
Sp(k) is the feature spectrum obtained from the output of a specific subregion k of a specific layer, depending on the data to be classified;
KSp(i, k=all, q=all) is the known feature spectrum group GKSp shown in FIG. feature spectrum,
max[X] is a logical operation that takes the maximum of the X values.
As the function G{a,b} for obtaining the degree of similarity, for example, a formula for obtaining cosine similarity or a formula for obtaining similarity according to distance can be used.

The three types of similarity by class Sclass(i) shown on the right side of FIG. It is obtained by taking the value. Which of the maximum, average, and minimum calculations is used depends on the intended use of the classification process. Which of these three types of calculations is used is preset by the user experimentally or empirically.

As described above, in the first calculation method M1 of similarity by class,
(1) The similarity between the feature spectrum Sp obtained from the output of a specific partial region k of a specific layer and all known feature spectra KSp associated with the specific layer and each class i according to the data to be classified Finding a certain local similarity S(i,k),
(2) For each class i, find the class similarity Sclass(i) by taking the maximum value, average value, or minimum value of the local similarities S(i,k) for a plurality of partial regions k.
According to this first calculation method M1, the similarity by class Sclass(i) can be obtained by a relatively simple calculation and procedure.

FIG. 22 is an explanatory diagram showing the second calculation method M2 of similarity by class. In the second calculation method M2, the local similarity S(i,k) is calculated using the following equation instead of the above equation (C1).
S(i,k)=max[G{Sp(k), KSp(i,k,q=all)}] (C2)
here,
KSp(i,k,q=all) is a known feature spectrum of all data numbers q in a specific subregion k of a specific layer associated with class i in the known feature spectrum group GKSp shown in FIG. is.

In the first calculation method M1 described above, the known feature spectrum KSp (i, k=all, q=all) in all partial regions k of the specific layer was used, whereas in the second calculation method M2 , only the known feature spectrum KSp(i,k,q=all) for the same subregion k as the subregion k of the feature spectrum Sp(k) is used. Other methods in the second calculation method M2 are the same as the first calculation method M1.

In the second class similarity calculation method M2,
(1) Depending on the data to be classified, the feature spectrum Sp obtained from the output of a specific subregion k of a specific layer and all known feature spectra associated with the specific subregion k of the specific layer and each class i Find the local similarity S(i,k), which is the similarity with KSp,
(2) Calculate class-wise similarity Sclass(i) by taking the maximum value, average value, or minimum value of local similarities S(i,k) for a plurality of partial regions k for each class i. .
The class similarity Sclass(i) can also be obtained by the second calculation method M2 through relatively simple calculations and procedures.

FIG. 23 is an explanatory diagram showing the third calculation method M3 of similarity by class. In the third calculation method M3, the class similarity Sclass(i) is calculated from the output of the ConvVN2 layer 250, which is the specific layer, without obtaining the local similarity S(i,k).

The class similarity Sclass(i) obtained by the third calculation method M3 is calculated using the following equation.
Sclass(i)=max[G{Sp(k=all), KSp(i,k=all,q=all)}] (C3)
here,
Sp(k=all) is the feature spectrum obtained from the output of all sub-regions k of a particular layer, depending on the data to be classified. The class similarity Sclass(i) given by the formula (C3) is substantially the same as the class similarity S(Class) given by the formula (A2) described in the first embodiment.

As described above, in the third calculation method M3 of similarity by class,
(1) Similarity by class, which is the similarity between all feature spectra Sp obtained from the output of a specific layer according to the data to be classified and all known feature spectra KSp associated with the specific layer and each class i Obtain Sclass(i) for each class.
According to this third calculation method M3, the similarity by class Sclass(i) can be obtained by a simpler calculation and procedure.

All of the three calculation methods M1 to M3 described above are methods of calculating similarity by class using the output of one specific layer. However, one or more of the vector neuron layers 240, 250, and 260 shown in FIG. 2 can be used as specific layers to calculate the similarity by class. When using a plurality of specific layers, for example, it is preferable to use the minimum value among a plurality of similarities by class obtained from a plurality of specific layers as the final similarity.

G. How to compute the output vector of each layer of the machine learning model:
The calculation method of the output of each layer shown in FIG. 2 is as follows.

Each node in the PrimeVN layer 230 takes the scalar output of the 1x1x32 nodes in the Conv layer 220 as a 32-dimensional vector and multiplies this vector by the transformation matrix to obtain the vector output of that node. This transformation matrix is an element of a kernel with a surface size of 1×1 and is updated as the machine learning model 200 learns. Note that it is also possible to integrate the processing of the Conv layer 220 and the PrimeVN layer 230 into one primary vector neuron layer.

ここで、
Ｍ^Ｌ _ｉは、下位層Ｌにおけるｉ番目のノードの出力ベクトル、
Ｍ^Ｌ＋１ _ｊは、上位層Ｌ＋１におけるｊ番目のノードの出力ベクトル、
ｖ_ｉｊは、出力ベクトルＭ^Ｌ＋１ _ｊの予測ベクトル、
Ｗ^Ｌ _ｉｊは、下位層Ｌの出力ベクトルＭ^Ｌ _ｉから予測ベクトルｖ_ｉｊを算出するための予測行列、
ｕ_ｊは、予測ベクトルｖ_ｉｊの和、すなわち線形結合、である和ベクトル、
ａ_ｊは、和ベクトルｕ_ｊのノルム|ｕ_ｊ|を正規化することによって得られる正規化係数であるアクティベーション値、
Ｆ（Ｘ）は、Ｘを正規化する正規化関数である。 When PrimeVN layer 230 is referred to as "lower layer L" and its upper adjacent ConvVN1 layer 240 is referred to as "upper layer L+1", the output of each node in upper layer L+1 is determined using the following equation: .

here,
M ^L _i is the output vector of the i-th node in the lower layer L,
M ^L+1 _j is the output vector of the j-th node in the upper layer L+1;
v _ij is the prediction vector of the output vector M ^L+1 _j ;
W ^L _ij is a prediction matrix for calculating the prediction vector v _ij from the output vector M ^L _i of the lower layer L;
u _j is the sum vector, i.e. the linear combination, of the prediction vectors v _ij ;
a _j is the activation value, the normalization factor obtained by normalizing the norm |u _j | of the sum vector u _j ;
F(X) is the normalization function that normalizes X.

ここで、
ｋは、上位層Ｌ＋１のすべてのノードに対する序数、
βは、任意の正の係数である調整パラメーターであり、例えばβ＝１である。 As the normalization function F(X), for example, the following formula (E3a) or formula (E3b) can be used.

here,
k is the ordinal number for all nodes in the upper layer L+1,
β is a tuning parameter that is any positive coefficient, eg β=1.

In the above equation (E3a), the activation value a _j is obtained by normalizing the norm |u _j | of the sum vector u _j for all the nodes of the upper layer L+1 with the softmax function. On the other hand, in equation (E3b), the activation value a _j is obtained by dividing the norm |u _j | of the sum vector u _j by the sum of the norms |u _j | for all nodes in the upper layer L+1. As the normalization function F(X), functions other than the formulas (E3a) and (E3b) may be used.

The ordinal number i in the above equation (E2) is conveniently assigned to the nodes of the lower layer L used to determine the output vector M ^L+1 _j of the j-th node in the upper layer L+1, and values from 1 to n are Take. Also, the integer n is the number of nodes in the lower layer L used to determine the output vector M ^L+1 _j of the j-th node in the upper layer L+1. Therefore, the integer n is given by the following equation.
n=Nk×Nc (E5)
Here, Nk is the surface size of the kernel, and Nc is the number of channels in the PrimeVN layer 230, which is the lower layer. In the example of FIG. 2, Nk=9 and Nc=16, so n=144.

One kernel used to determine the output vector of the ConvVN1 layer 240 has 3×3×16=144 elements with a surface size of kernel size 3×3 and a depth of 16 channels in the lower layer. , and each of these elements is the prediction matrix W ^L _ij . Also, to generate the 12-channel output vectors of the ConvVN1 layer 240, 12 sets of these kernels are required. Therefore, the number of kernel prediction matrices W ^L _ij used to determine the output vectors of the ConvVN1 layer 240 is 144×12=1728. These prediction matrices W ^L _ij are updated as the machine learning model 200 learns.

As can be seen from the above equations (E1) to (E4), the output vector M ^L+1 _j of each node in the upper layer L+1 is obtained by the following calculation.
(a) Multiply the output vector M ^L _i of each node in the lower layer L by the prediction matrix W ^L _ij to obtain a prediction vector v _ij ;
(b) find the sum vector _{uj ,} which is the sum, i.e., the linear combination, of the prediction vectors vij obtained from each node of the lower layer L;
(c) normalizing the norm |u _j | of the sum vector u _j to find the activation value a _j , which is a normalization factor;
(d) Divide the sum vector u _j by the norm |u _j | and multiply it by the activation value a _j .

Note that the activation value a _j is a normalization factor obtained by normalizing the norm |u _j | with respect to all nodes in the upper layer L+1. Therefore, the activation value _aj can be considered as an index indicating the relative output strength of each node among all nodes in the upper layer L+1. The norms used in equations (E3), (E3a), (E3b), and (4) are typically L2 norms representing vector lengths. At this time, the activation value a _j corresponds to the vector length of the output vector M ^L+1 _j . Since the activation value _aj is only used in the above equations (E3) and (E4), it need not be output from the node. However, it is also possible to configure the upper layer L+1 to output the activation value _aj to the outside.

The configuration of the vector neural network is almost the same as that of the capsule network, and the vector neurons of the vector neural network correspond to the capsules of the capsule network. However, the calculations according to the above equations (E1) to (E4) used in the vector neural network are different from those used in the capsule network. The biggest difference between the two is that in the capsule network, the prediction vector v _ij on the right side of the above equation (E2) is multiplied by a weight, and the weight is searched for by repeating dynamic routing multiple times. be. On the other hand, in the vector neural network of this embodiment, since the output vector M ^L+1 _j is obtained by calculating the above-mentioned equations (E1) to (E4) once in order, there is no need to repeat the dynamic routing, and the operation has the advantage of being faster. In addition, the vector neural network of the present embodiment requires a smaller amount of memory for calculation than the capsule network. There is also an advantage that it can be done with

A vector neural network is similar to a capsule network in that it uses nodes whose inputs and outputs are vectors. Therefore, the advantages of using vector neurons are also shared with capsule networks. In addition, the plurality of layers 220 to 260 represent features of larger regions as they go higher, and features of smaller regions as they go lower, which is the same as an ordinary convolutional neural network. Here, the "feature" means a characteristic part included in the input data to the neural network. Vector neural networks and capsule networks are superior to ordinary convolutional neural networks in that the output vector of a node contains spatial information representing the spatial information of the feature represented by that node. That is, the vector length of the output vector of a certain node represents the existence probability of the feature represented by that node, and the vector direction represents spatial information such as the direction and scale of that feature. Therefore, the vector directions of the output vectors of two nodes belonging to the same layer represent the positional relationship of their respective features. Alternatively, it can be said that the vector directions of the output vectors of the two nodes represent variations of the feature. For example, for a node corresponding to the "eyes" feature, the direction of the output vector could represent variations in the fineness of the eyes, how they are hung, and so on. In a normal convolutional neural network, it is said that spatial information of features disappears due to pooling processing. As a result, vector neural networks and capsule networks have the advantage of being superior to ordinary convolutional neural networks in their ability to identify input data.

The advantages of vector neural networks can also be considered as follows. That is, the vector neural network has the advantage that the output vectors of the nodes represent the features of the input data as coordinates in a continuous space. Therefore, the output vectors can be evaluated such that if the vector directions are close, the features are similar. In addition, even if the feature included in the input data is not covered by the teacher data, there is an advantage that the feature can be determined by interpolation. On the other hand, a conventional convolutional neural network suffers from chaotic compression due to pooling processing, and thus has the disadvantage that the features of input data cannot be expressed as coordinates in a continuous space.

The output of each node of the ConvVN2 layer 250 and the ClassVN layer 260 is also determined in the same way using the formulas (E1) to (E4) described above, so detailed description is omitted. The ClassVN layer 260, which is the highest layer, has a resolution of 1×1 and M channels.

The output of the ClassVN layer 260 is transformed into multiple decision values Class_1 through Class_M for multiple classes. These judgment values are values normalized by a softmax function. Specifically, for example, from the output vector of each node of the ClassVN layer 260, the vector length of the output vector is calculated, and the vector length of each node is normalized by the softmax function. to obtain the decision value for each class. As described above, the activation value a _j obtained by the above equation (E3) is a value corresponding to the vector length of the output vector M ^L+1 _j and is normalized. Therefore, the activation value _aj at each node of the ClassVN layer 260 may be output and used as it is as the judgment value for each class.

In the above-described embodiment, as the machine learning model 200, a vector neural network that obtains an output vector by calculating the above equations (E1) to (E4) was used. A capsule network disclosed in Japanese Patent Publication No. 2009/083553 may be used.

・Other forms:
The present disclosure is not limited to the embodiments described above, and can be implemented in various forms without departing from the scope of the present disclosure. For example, the present disclosure can also be implemented in the following aspects. The technical features in the above embodiments corresponding to the technical features in each form described below are used to solve some or all of the problems of the present disclosure, or to achieve some or all of the effects of the present disclosure. In order to achieve the above, it is possible to appropriately replace or combine them. Also, if the technical features are not described as essential in this specification, they can be deleted as appropriate.

(1) According to the first aspect of the present disclosure, there is provided a classifying device that performs classifying processing of data to be classified using a machine learning model including a vector neural network having a plurality of vector neuron layers. This class classification device includes a class classification processing unit that executes the class classification process. The machine learning model has one or more known classes and one or more dummy classes, and the machine learning model includes a first training data group for the known classes and a set of dummy classes for the dummy classes. Learning is executed using the second teacher data group of . The class classification processing unit determines that the class of the data to be classified is unknown when the class classification result of the data to be classified using the machine learning model is the dummy class.
According to this classifying device, the classifying process is executed using a machine learning model having one or more known classes and one or more dummy classes, so it is possible to improve the accuracy of the classifying process.

(2) In the class classification device, when N is an integer of 2 or more, the class classification device has N machine learning models, and the class classification processing unit includes the N machine learning models. The class of the data to be classified may be determined to be unknown when all the class classification results by the model are the dummy class.
According to this classifying device, it is possible to improve the accuracy of classifying even when classifying into many known classes using N machine learning models.

(3) In the above class classification device, the second teacher data group for the dummy class of each machine learning model is N-1 for the other N-1 machine learning models. It is data selected from the first teacher data group, and may be selected so as to include a part of each of the N−1 first teacher data groups.
According to this classifying device, the processing time required for classifying can be shortened. In addition, it is possible to easily create a second training data group for dummy classes of N machine learning models.

(4) In the class classification device, each of the N machine learning models may have N-1 dummy classes. The N-1 second teacher data groups for the N-1 dummy classes of the individual machine learning models are N-1 for the other N-1 machine learning models. wherein the N-1 second teacher data groups are selected so as to correspond to the N-1 first teacher data groups It can be assumed that there is
According to this classifying device, the accuracy of classifying can be improved. In addition, it is possible to easily create a second training data group for dummy classes of N machine learning models.

(5) In the class classification device, each of the N machine learning models has the N-1 dummy classes, and the individual first teacher data included in the first teacher data group are: Data extracted from sample data having a size larger than that of the first teacher data may be used. The N-1 second teacher data groups for the N-1 dummy classes of the individual machine learning models are N-1 for the other N-1 machine learning models. Data extracted from N-1 average sample data obtained by averaging the N-1 sample data groups used to extract the first teacher data group of the N-1 The second training data group may be extracted so as to correspond to the N−1 average sample data.
According to this class classification device, it is possible to easily create a second training data group for dummy classes of N machine learning models.

(6) The class classification device includes a learning execution unit that executes learning of the N machine learning models, and the learning execution unit is selected by the user for the performance of the N machine learning models. receiving a performance preference, and selecting one of a plurality of options for the number of dummy classes in the N machine learning models and a method for creating the second training data group according to the performance preference. may be
According to this classifying device, it is possible to create an appropriate dummy class according to performance preferences indicating which of performances such as processing speed and processing accuracy should be prioritized.

(7) The class classification device may further include a memory for storing a group of known feature spectra obtained from the output of a specific layer of the machine learning model when a plurality of teacher data are input to the machine learning model. good. The class classification processing unit performs (a) a process of reading the known feature spectrum group from the memory, and (b) a feature obtained from the output of the specific layer when the data to be classified is input to the machine learning model. and (c) using the similarity to determine the class of the data to be classified. may be
According to this classifying apparatus, the class of data to be classified can be determined with high accuracy using the degree of similarity between the feature spectrum and the known feature spectrum group.

(8) In the class classification device, the class classification processing unit, when the class classification result of the data to be classified using the machine learning model is the dummy class, or when the similarity is less than a threshold , the class of the data to be classified may be determined as unknown.
According to this classifying device, it can be determined with high accuracy whether the class of data to be classified is unknown.

(9) In the above classifying device, the specific layer includes vector neurons arranged on a plane defined by two axes, a first axis and a second axis, on a third axis in a direction different from the two axes. may have a configuration arranged as a plurality of channels along the . (i) a plurality of element values of an output vector of a vector neuron at a planar location of one of said specific layers arranged across said plurality of channels along said third axis; (ii) a characteristic spectrum of the second type obtained by multiplying each element value of the characteristic spectrum of the first type by an activation value corresponding to the vector length of the output vector; (iii) the specific and a third type of feature spectrum arranged over the plurality of channels along the third axis.
According to this classifying device, the feature spectrum can be easily obtained.

(10) According to the second aspect of the present disclosure, there is provided a method of performing class classification processing of data to be classified using a machine learning model including a vector neural network having multiple vector neuron layers. This method includes (a) a step of inputting the data to be classified into the machine learning model to obtain a classification result, wherein the machine learning model includes one or more known classes and one or more dummy classes. and wherein the machine learning model is trained using a first training data group for the known class and a second training data group for the dummy class; (b) determining the class of the data to be classified as unknown when the class classification result is the dummy class;

(11) According to a third aspect of the present disclosure, there is provided a computer program that causes a processor to execute class classification processing of data to be classified using a machine learning model that includes a vector neural network having a plurality of vector neuron layers. . This computer program includes (a) a process of inputting the data to be classified into the machine learning model to obtain a classification result, wherein the machine learning model includes one or more known classes and one or more dummy classes. a class, wherein the machine learning model is trained using a first training data group for the known class and a second training data group for the dummy class; and (b) determining the class of the data to be classified as unknown when the class classification result is the dummy class.

The present disclosure can also be implemented in various forms other than those described above. For example, it can be realized in the form of a computer program for realizing the function of the classifying device, a non-transitory storage medium in which the computer program is recorded, or the like.

DESCRIPTION OF SYMBOLS 100... Information processing apparatus, 110... Processor, 112... Learning execution part, 114... Classification process part, 120... Memory, 130... Interface circuit, 140... Input device, 150... Display device, 200... Machine learning model, 210... Input layer 220 Convolution layer 230 Primary vector neuron layer 240 First convolution vector neuron layer 250 Second convolution vector neuron layer 260 Classification vector neuron layer 280 Intermediate layer 310 Similarity calculation Section 320 Class determination section 400 Camera

Claims (11)

A classifier that executes class classification processing of data to be classified using a machine learning model including a vector neural network having a plurality of vector neuron layers,
A class classification processing unit that executes the class classification process,
The machine learning model has one or more known classes and one or more dummy classes,
The machine learning model is trained using a first teacher data group for the known class and a second teacher data group for the dummy class,
The class classification processing unit determines that the class of the data to be classified is unknown when the class classification result of the data to be classified using the machine learning model is the dummy class.
Class classifier. The classifying device according to claim 1,
When N is an integer of 2 or more, the classifying device has N machine learning models,
The class classification processing unit determines that the class of the data to be classified is unknown when all the class classification results by the N machine learning models are the dummy class.
Class classifier. The classifying device according to claim 2,
The second teacher data group for the dummy class of each machine learning model is selected from among the N-1 first teacher data groups for the other N-1 machine learning models. and selected to include a portion of each of said N-1 first training data groups. The classifying device according to claim 2,
each of the N machine learning models has N-1 dummy classes;
The N-1 second teacher data groups for the N-1 dummy classes of the individual machine learning models are N-1 for the other N-1 machine learning models. wherein the N-1 second teacher data groups are selected so as to correspond to the N-1 first teacher data groups class classifier. The classifying device according to claim 2,
each of the N machine learning models has N-1 dummy classes;
each of the first training data included in the first training data group is data extracted from sample data having a size larger than that of the first training data;
The N-1 second teacher data groups for the N-1 dummy classes of the individual machine learning models are N-1 for the other N-1 machine learning models. Data extracted from N-1 average sample data obtained by averaging the N-1 sample data groups used to extract the first teacher data group of the N-1 The class classification device, wherein the second training data group is extracted so as to correspond to the N-1 average sample data. The classifying device according to claim 2,
A learning execution unit that executes learning of the N machine learning models,
The learning execution unit receives performance preferences selected by a user for the performance of the N machine learning models, and according to the performance preferences, the number of the dummy classes in the N machine learning models. and a class classification device that selects one of a plurality of options for the method of creating the second teacher data group. The classifying device according to any one of claims 1 to 6,
A memory for storing a known feature spectrum group obtained from the output of a specific layer of the machine learning model when a plurality of teacher data is input to the machine learning model,
The class classification processing unit,
(a) reading the known feature spectrum group from the memory;
(b) a process of calculating the similarity between the feature spectrum obtained from the output of the specific layer when the data to be classified is input to the machine learning model and the group of known feature spectra;
(c) a process of determining the class of the data to be classified using the similarity;
A classifier configured to perform a A classifying device according to claim 7,
When the class classification result of the data to be classified using the machine learning model is the dummy class, or when the similarity is less than a threshold, the class classification processing unit identifies the class of the data to be classified as unknown. determine that
Class classifier. A classifying device according to claim 7 or 8,
In the specific layer, vector neurons arranged on a plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes. having a configuration that
The feature spectrum is
(i) a first type feature spectrum in which a plurality of element values of an output vector of a vector neuron at a planar position of one of said specific layers are arranged over said plurality of channels along said third axis;
(ii) a second-type feature spectrum obtained by multiplying each element value of the first-type feature spectrum by an activation value corresponding to the vector length of the output vector;
(iii) a third type of feature spectrum arranged across the plurality of channels along the third axis of the activation values at a planar location of one of the particular layers;
A classifier that is any of A method for classifying data to be classified using a machine learning model including a vector neural network having multiple vector neuron layers, comprising:
(a) a step of inputting the data to be classified into the machine learning model to obtain a classification result, wherein the machine learning model has one or more known classes and one or more dummy classes; and the machine learning model is trained using a first training data group for the known class and a second training data group for the dummy class;
(b) determining the class of the data to be classified as unknown when the class classification result is the dummy class;
A method, including A computer program that causes a processor to perform class classification processing of data to be classified using a machine learning model that includes a vector neural network having a plurality of vector neuron layers,
Said computer program comprises:
(a) A process of inputting the data to be classified into the machine learning model to obtain a classification result, wherein the machine learning model has one or more known classes and one or more dummy classes. and the machine learning model is trained using a first training data group for the known class and a second training data group for the dummy class;
(b) a process of determining that the class of the data to be classified is unknown when the class classification result is the dummy class;
a computer program that causes said processor to execute

Images