WO2021210413A1 - Discrimination target classification method, program employed in same, and discriminating device - Google Patents

Discrimination target classification method, program employed in same, and discriminating device Download PDF

Info

Publication number
WO2021210413A1
WO2021210413A1 PCT/JP2021/014130 JP2021014130W WO2021210413A1 WO 2021210413 A1 WO2021210413 A1 WO 2021210413A1 JP 2021014130 W JP2021014130 W JP 2021014130W WO 2021210413 A1 WO2021210413 A1 WO 2021210413A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
multidimensional
multidimensional feature
feature vector
discrimination
Prior art date
Application number
PCT/JP2021/014130
Other languages
French (fr)
Japanese (ja)
Inventor
貴之 石黒
星野　仁
Original Assignee
株式会社Ｒｏｘｙ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｒｏｘｙ filed Critical 株式会社Ｒｏｘｙ
Priority to JP2022515303A priority Critical patent/JPWO2021210413A1/ja
Publication of WO2021210413A1 publication Critical patent/WO2021210413A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to a method for classifying discriminating objects, a program used for the classification method, and a discriminating device.
  • Patent Document 1 uses a convolutional neural network to perform a calculation by a convolutional layer on an image to be discriminated, and features from the first fully connected layer after the calculation of the convolutional layer is completed.
  • a method of extracting a quantity (multidimensional feature vector), executing a multivariate analysis using the extracted multidimensional feature vector, and classifying the discrimination target based on the result of the multivariate analysis is described.
  • the multidimensional feature vector extracted from the first fully connected layer after the calculation of the convolution layer is completed is not always effective as a feature quantity used for classification of the discrimination target, and in terms of improving the classification accuracy, it is still possible. There is room for improvement.
  • the present invention has been made in view of the above, and one of the objects of the present invention is to provide a technique that contributes to improvement of classification accuracy.
  • the classification method of the discrimination target (a) the data of the discrimination target is acquired, and (b) a feature extractor using a neural network including deep learning is used to obtain the data of the discrimination target. A one-dimensional feature vector is extracted, and (c) the extracted first multidimensional feature vector is subjected to a second multidimensional lower dimension than the first multidimensional feature vector by a feature converter using a plurality of fully connected layers. It is converted into a feature vector, and (d) based on the converted second multidimensional feature vector, the discrimination target is classified by a classifier using statistical machine learning.
  • the "data to be discriminated” in the present invention typically corresponds to the image data to be discriminated, and the image data to be discriminated literally corresponds to this, but has been learned. In the learning stage of the model, it is a concept that includes sample data with a teacher signal.
  • the "statistical machine learning device” in the present invention refers to machine learning other than machine learning using a neural network including deep learning, such as gradient boosting, support vector machine, random forest, neural network, and Gaussian normalization. , Sansamble inspection, etc. correspond to this.
  • the classification of the discrimination target is accurate. It can be realized well. It should be noted that a feature extractor for extracting the first multidimensional feature vector, which is a variety of feature quantities of the data to be discriminated, and a plurality of full couplings for converting the first multidimensional feature vector into a low-dimensional second multidimensional feature vector. Since the feature converter using the layer and the classifier that classifies the discrimination target based on the second multidimensional feature vector converted by the feature converter have different configurations, it is necessary to improve the classification accuracy of the discrimination target.
  • Learning of feature extractors and feature converters and learning of classifiers can be performed separately. That is, when the cause of the low classification accuracy is the low feature extraction / conversion accuracy by the feature extractor and the feature converter, the feature extractor and the feature converter are trained and the classification accuracy by the classifier is performed. If it is caused by the low level, only the classifier can be trained, and the classification accuracy can be improved efficiently.
  • a program for classifying the discrimination target is configured.
  • the program is for causing one or a plurality of computers to execute each step of the classification method of the discrimination target according to the present invention in any of the above-described aspects.
  • the program may be recorded on a computer-readable recording medium such as a hard disk, ROM, SSD, flash memory (USB memory, SD card, etc.), floppy disk, CD, DVD, or a transmission medium.
  • a computer-readable recording medium such as a hard disk, ROM, SSD, flash memory (USB memory, SD card, etc.), floppy disk, CD, DVD, or a transmission medium.
  • a communication network such as the Internet or LAN, or may be exchanged in any other manner.
  • each step of the discrimination target classification method according to the present invention in any of the above-described embodiments. Is executed, so that the same action and effect as the above-described method for classifying the discrimination target according to the present invention, for example, the effect that the classification of the discrimination target can be accurately realized can be obtained.
  • a data acquisition unit for acquiring data to be discriminated and a neural network including deep learning for extracting a first multidimensional feature vector from the acquired data to be discriminated are used.
  • a feature extractor and a plurality of fully connected layers that convert the first multidimensional feature vector extracted by the feature extractor into a second multidimensional feature vector having a lower dimension than the first multidimensional feature vector are used. It includes a feature converter and a classifier using statistical machine learning that classifies discrimination targets using a second multidimensional feature vector.
  • the "data to be discriminated" in the present invention typically corresponds to the image data to be discriminated, and the image data to be discriminated literally corresponds to this, but feature extraction.
  • the "statistical machine learning device" in the present invention refers to machine learning other than machine learning using a neural network including deep learning, such as gradient boosting, support vector machine, random forest, neural network, and Gaussian normalization. , Sansamble inspection, etc. correspond to this.
  • the classification of the discrimination target is accurate. It can be realized well.
  • a feature extractor that extracts the first multidimensional feature vector, which is a variety of feature quantities of the data to be discriminated, and a plurality of all that convert the first multidimensional feature vector into a low-dimensional second multidimensional feature vector. Since the feature converter using the coupling layer and the classifier that classifies the discrimination target based on the second multidimensional feature vector converted by the feature converter have different configurations, the classification accuracy of the discrimination target is improved.
  • the learning of the feature extractor and the feature converter, and the learning of the classifier can be carried out separately. That is, when the cause of the low classification accuracy is the low feature extraction accuracy by the feature extractor and the feature converter, the feature extractor and the feature converter are trained and the classification accuracy by the classifier is low. If this is the case, only the classifier can be trained, and the classification accuracy can be improved efficiently.
  • a storage unit that stores the second multidimensional feature vector and a distance value between each of the plurality of second multidimensional feature vectors stored in the storage unit are stored. It also has a calculation department for calculation. Then, in the learning stage, the feature extractor and the feature converter learn by using the error backpropagation method and the gradient descent method based on the distance value calculated by the calculation unit.
  • the classification of the discrimination target is effective, and the classification of the discrimination target is performed using the second multidimensional feature vector extracted and converted in the learning stage of the feature extractor and the feature converter. Therefore, the classification accuracy is correct. Can be further improved.
  • the classification accuracy can be improved.
  • the computer 1 that functions as the discrimination device according to the present embodiment is configured as a microprocessor centered on the CPU 2, and has a ROM 4 that stores various processing programs and temporarily stores data.
  • RAM 6, GPU 8 that performs calculation processing and matrix calculation processing necessary for performing image processing hard disk (HDD) 10, which is a large-capacity memory that stores various data including various application programs (simply called applications) and image data, It is provided with an input / output interface (I / F) 12 for inputting / outputting data to / from an external device such as a camera 70.
  • I / F input / output interface
  • the computer 1 as the discrimination device according to the present embodiment will be described as a device that discriminates whether the discrimination target is an OK product or an NG product.
  • the computer 1 is an example of an implementation configuration corresponding to the "discrimination device" in the present invention.
  • the computer 1 is provided with an input device 14 such as a keyboard and a mouse for which a user inputs various commands, a display 60 for displaying various information, and the like.
  • the CPU 2, ROM 4, RAM 6, GPU 8, HDD 10, I / F 12, input device 14, display 60, and the like are electrically connected by a bus 80 so that various control signals and data can be exchanged with each other.
  • the computer 1 has a function of executing an operation corresponding to the input operation when the user inputs the cursor or the like displayed on the display 60 via the input device 14. Further, the computer 1 functions as a discrimination device according to the present embodiment by executing various processes by an application stored in the HDD 10, specifically, a discrimination application that executes the discrimination processing of the discrimination target. .. In the present embodiment, the discriminating device is configured to be feasible by the computer 1, but it may be realized as a dedicated device.
  • the computer 1 includes the above-mentioned hardware resources such as CPU2, ROM4, RAM6, GPU8, HDD10, I / F12, input device 14, and display 60, a discrimination application, and the present embodiment.
  • Image display control unit 20, image acquisition unit 22, area designation unit 24, feature extractor 26, feature converter 27, classifier 28, distance calculation in collaboration with software such as the visualization program related to The device 29, the two-dimensional graphed data generation unit 30, the storage unit 33, and the like are configured as functional blocks. In other words, each of these parts (image display control unit 20, image acquisition unit 22, area designation unit 24, feature extractor 26, feature converter 27, classifier 28, distance calculator 29, two-dimensional graphing data generation unit).
  • the feature extractor 26, the feature converter 27, and the display 60 are examples of an implementation configuration corresponding to the “visualization device” in the present invention.
  • the image display control unit 20 displays a predetermined window 62 on the screen 61 of the display 60 as shown in FIG. Further, the image display control unit 20 teaches based on the sample image data 34a with a teacher signal (see FIG. 2) selected by the user inputting the cursor or the like displayed on the display 60 via the input device 14.
  • a sample image with a signal is displayed on the window 62
  • a discrimination target image is displayed on the window 62 based on the discrimination target image data 34b (see FIG. 2) acquired by the image acquisition unit 22, and the area designation unit 24 is displayed.
  • the designated area image is displayed on the window 62 based on the designated area image data 34c (see FIG. 2) acquired by.
  • the image display control unit 20 displays a two-dimensional graph on the window 62 based on the model two-dimensional graphed data 36a (see FIG. 2) stored in the storage unit 33, or generates two-dimensional graphed data.
  • the discrimination two-dimensional graph is displayed on the window 62 based on the discrimination two-dimensional graphing data 36b generated by the unit 30. Further, the image display control unit 20 displays the discrimination result by the classifier 28 in the window 62.
  • the sample image with a teacher signal is an image used for learning the feature extractor 26 and the feature converter 27.
  • the discrimination target image typically corresponds to an image of a target whose classification (OK product or NG product) is unknown, but verification and learning of the learning results of the feature extractor 26 and the feature converter 27
  • it preferably includes an image of a target whose classification (OK product or NG product) is known, for example, a sample image with a teacher signal arbitrarily selected by the user from a plurality of sample images with a teacher signal.
  • the image acquisition unit 22 acquires sample image data 34a with a teacher signal and image data 34b to be discriminated taken by the camera 70 (see FIG. 1), and uses these acquired image data as the image display control unit 20 and the feature extractor 26. , Supply to the storage unit 33 and the like.
  • the area designation unit 24 is used when an arbitrary area of the sample image with a teacher signal or the image to be discriminated displayed on the window 62 is clicked or dragged by an input operation via the user's input device 14.
  • 34c of the image data (hereinafter referred to as “designated area image data”) 34c is acquired, and the acquired designated area image data 34c (see FIG. 2) is supplied to the image display control unit 20, the feature extractor 26, the storage unit 33, and the like. ..
  • the feature extractor 26 extracts the first multidimensional feature vectors 90a, 90b, 90c from the plurality of sample image data 34a with teacher signals, the discriminant target image data 34b, and the designated area image data 34c, and extracts the first multidimensional.
  • the feature vectors 90a, 90b, 90c are supplied to the feature converter 27.
  • the feature extractor 26 is configured to use a convolutional neural network (CNN) as shown in FIGS. 3 and 4.
  • CNN convolutional neural network
  • the first multidimensional feature vectors 90a, 90b, and 90c are examples of implementation configurations corresponding to the "first multidimensional feature vector" in the present invention, respectively.
  • a plurality of sample image data 34a with teacher signals and discrimination targets are used using a predetermined filter (not shown).
  • the so-called convolution process for extracting the features of the image data 34a, 34b, 34c from the image data 34b and the designated area image data 34c without losing the features is performed a plurality of times, and then flattened into a column vector.
  • the feature converter 27 has a plurality of fully connected layers 27a and 27b, and the first multidimensional extracted by the feature extractor 26 using the fully connected layers 27a and 27b.
  • the process of lowering the dimension of the feature vectors 90a, 90b, 90c is executed.
  • the first multidimensional feature vectors 90a, 90b, 90c are converted into the first multidimensional feature vectors 90a, 90b, 90c by Fully Connected processing, and the second multidimensional feature vectors 92a, which are lower in dimension than the first multidimensional feature vectors 90a, 90b, 90c, It is converted into 92b and 92c and supplied to the classifier 28, the distance calculator 29, the two-dimensional graphing data purification unit 30, and the storage unit 33.
  • the first multidimensional feature vectors 90a, 90b, 90c are obtained from the first multidimensional feature vectors 90a, 90b, 90c by pooling processing such as Global max Polling or Global Average Polling instead of the Fully Connected processing. May be converted into low-dimensional second multidimensional feature vectors 92a, 92b, 92c.
  • the second multidimensional feature vectors 92a, 92b, and 92c are stored in the storage unit 33, respectively.
  • the second multidimensional feature vectors 92a, 92b, and 92c are examples of the implementation configurations corresponding to the "second multidimensional feature vector" in the present invention.
  • the feature extractor 26 and the feature converter 27 include the first multidimensional feature vectors 90a, 90b, 90c and the second multidimensional feature vectors 92a, 92b, which can accurately discriminate the classification of the discrimination target. Learning is performed in advance so that 92c can be obtained, and the learned model 35 is stored in the storage unit 33. Twice
  • the classifier 28 functions when discriminating the classification of the discriminating target.
  • the classification of the discrimination target that is, whether the discrimination target is an OK product or an NG product, is based on the second multidimensional feature vectors 92a, 92b, 92c converted by the feature converter 27. Is determined, and the determination result is supplied to the image display control unit 20.
  • the classification of the discrimination target is that the second multidimensional feature vectors 92a, 92b, 92c to be discriminated by the arithmetic parameters machine-learned using the second multidimensional feature vectors 92a, 92b, 92c are OK products.
  • a one-dimensional numerical value indicating the degree of certainty is calculated, and if it is equal to or more than the set threshold value, it is classified as an OK product, and if it is less than the set threshold value, it is classified as an NG product.
  • the classifier 28 is configured to use so-called statistical machine learning such as gradient boosting, support vector machine, random forest, neural network, Gaussian normalization, and ensemble inspection.
  • the distance calculator 29 functions when generating the trained model 35, and when verifying the trained model 35 and improving the accuracy.
  • the distance calculator 29 calculates the distance between the second multidimensional feature vectors 92a of the sample image data 34a with a plurality of teacher signals, and uses the calculated distance as the distance data 38. It is stored in the storage unit 33. The calculated distance is fed back to the feature extractor 26 and the feature converter 27 (see FIG. 4A).
  • the distance calculator 29 uses the second multidimensional feature vector 92a of the sample image data 34a with a plurality of teacher signals, the discriminant image data 34b, and the designation. The distance between the second multidimensional feature vectors 92b and 92c of the area image data 34c is calculated, and the calculated distance is stored in the storage unit 33 as the distance data 38.
  • the feedback to the feature extractor 26 and the feature converter 27 is the distance between the second multidimensional feature vectors 92a between the OK products and the second multidimensional feature vector 92a of the OK product and the NG product.
  • the configuration was made by modifying the parameters of the vessel 27. In such parameter modification (learning of the feature extractor 26 and the feature converter 27), the second multidimensional feature vector 92a is visualized (two-dimensional graphing), and the trained model 35 (feature extractor 26 and the feature converter 27) is modified.
  • the trained model 35 is generated and stored in the storage unit 33.
  • the learning of the trained model 35 and the learning of the feature extractor 26 and the feature converter 27 are synonymous.
  • deep metric learning is used for modifying the parameters of the feature extractor 26 and the feature converter 27. That is, the first multidimensional feature vector 90a is extracted by the feature extractor 26, and the extracted first multidimensional feature vector 90a is converted into the second multidimensional feature vector 92a by the feature converter 27, and the converted second multi.
  • the two-dimensional graphing data generation unit 30 generates the model two-dimensional graphing data 36a based on the second multidimensional feature vector 92a of the plurality of sample image data 34a with teacher signals, and also generates the discrimination target image data 34b and Based on the second multidimensional feature vectors 92b and 92c of the designated area image data 34c, the discriminant two-dimensional graphed data 36b is generated.
  • the two-dimensional graphing data 36a for the model and the two-dimensional graphing data 36b for discrimination are generated by using the second multidimensional feature vectors 92a, 92b, and 92c as a set of the feature quantities of the number of dimensions.
  • the second multidimensional feature vectors 92a, 92b, 92c are n-dimensional column vectors (f 1 , f 2 , f 3 , ..., f n-1 , f n ), n feature quantities f.
  • Data as a set of 1 , f 2 , f 3 , ..., f n-1 , f n , that is, as shown in FIG.
  • the sample image data 34a with a plurality of teacher signals stored in the storage unit 33 by the user For the purpose of verifying the learning result of the trained model 35 (feature extractor 26 and feature converter 27) and improving the learning accuracy, the sample image data 34a with a plurality of teacher signals stored in the storage unit 33 by the user.
  • the two-dimensional graphing data generation unit 30 has the arbitrary one teacher signal from the model two-dimensional graphed data 36a.
  • a model generated based on the second multidimensional feature vector data 92a hereinafter referred to as "arbitrary second multidimensional feature vector 92a'" of the sample image data 34a (hereinafter referred to as "arbitrary sample image data 34a'").
  • Two-dimensional graphing data 36a for use hereinafter, referred to as "selected two-dimensional graphing data 36a'" is extracted.
  • the storage unit 33 is secured in at least one of the RAM 6 and the HDD 10, and is the sample image data 34a with a teacher signal taken by the camera 70 (see FIG. 1), the image data 34b to be discriminated, and the designation acquired by the area designation unit 24.
  • the multidimensional feature vectors 92a, 92b, 92c and the distance data 38 calculated by the distance calculator 29 are stored.
  • FIG. 6 is a main flowchart showing an example of the classification determination routine.
  • the feature extractor 26 (trained feature extractor 26) using the trained model 35 performs the discrimination target image data 34b, the arbitrary sample image data 34a', or the designated area.
  • Image data 34c is read (step S10), and a process of extracting the first multidimensional feature vectors 90a, 90b, 90c from the read discrimination target image data 34b, arbitrary sample image data 34a', or designated area image data 34c is executed. (Step S12).
  • the feature converter 27 (trained feature converter 27) using the trained model 35 uses the first multidimensional feature vectors 90a, 90b, 90c extracted by the feature extractor 26 as the second multidimensional feature vector.
  • the process of converting to 92a, 92b, 92c is executed (step S14).
  • the classifier 28 uses the second multidimensional feature vectors 92a, 92b, 92c converted by the feature converter 27 to discriminate not only the image of the target whose classification is unknown but also the arbitrary sample image.
  • the process of classifying the images (designated area images) of arbitrary areas of these images specified by the user is executed (step S16), and the discrimination result (OK product or NG product) according to the classification is output. (Step S18), this routine is terminated.
  • the first multidimensional feature vectors 90a, 90b, 90c are converted into the second multidimensional feature vectors 92a, 92b, 92c by the feature converter 27 learned in advance using deep metric learning, and thus discriminated. It is possible to reduce the dimension without losing the features that are effective for classifying the object.
  • the learning of the trained model 35 (feature extractor 26 and feature converter 27) using deep metric learning is visualized and the trained model 35 (feature extractor 26 and feature conversion) is performed.
  • the second multidimensional feature vector 92a which is effective for classifying the discrimination target, is performed until the learning situation (individuality) of the vessel 27) is visually confirmed and the learning situation (individuality) becomes a desired state. It can be appropriately converted to 92b and 92c.
  • FIG. 7A is an explanatory diagram showing a state in which the trained model 35 is in an appropriate learning situation (individuality), and FIG. 7B is a state in which the learning situation (individuality) of the trained model 35 is not appropriate.
  • the reference numeral "Og” is a graph generated by the two-dimensional graphing data generation unit 30 based on the second multidimensional feature vector 92a of the sample image data 34a with a plurality of teacher signals of the OK product.
  • the symbol “Ng” is a graph generated by the two-dimensional graphing data generation unit 30 based on the second multidimensional feature vector 92a of the sample image data 34a with a plurality of teacher signals of the NG product.
  • 7 (a) and 7 (b) are one-dimensional numerical indexes such as a correct answer rate, a precision rate, and a recall rate that are generally used when evaluating the learning status (individuality) of the trained model 35. All are visualizations of the trained model 35 which is the same or has similar numerical indexes.
  • the trained model 35 when the trained model 35 is in an appropriate learning situation (individuality), the two-dimensional graph Og for the model and the two-dimensional graph Ng for the model are displayed clearly separated, and the boundary is displayed.
  • the second multidimensional feature vectors 92a, 92b, 92c, which are effective for classifying the discrimination target, are appropriate by using the feature converter 27 using the trained model 35, which is clarified and the learning situation (individuality) is appropriate. Can be obtained.
  • the two-dimensional graph Og for the model and the two-dimensional graph Ng for the model are displayed close to each other, and the boundary becomes unclear. If the feature converter 27 using the trained model 35 whose situation (individuality) is not appropriate is used, the second multidimensional feature vectors 92a, 92b, 92c effective for classifying the discrimination target cannot be appropriately obtained.
  • the trained model 35 By visualizing the learning status (individuality) of the trained model 35 in this way, it is possible to evaluate whether or not the trained model 35 is in a learning status (individuality) that can appropriately discriminate the classification of the discrimination target.
  • An appropriate trained model 35 can be generated with a smaller amount of sample data than in the past. Then, by being able to generate an appropriate trained model 35, it is possible to widen the setting range of the threshold value appropriate for the classification of the discrimination target by the classifier 28. As a result, it is possible to suppress erroneous discrimination due to disturbance, for example, the influence of external light when acquiring an image to be discriminated, and it is possible to stably obtain an accurate discriminant result.
  • the second multidimensional feature vector 92b to be discriminated in addition to the two-dimensional graphs Og and Ng for the model, the second multidimensional feature vector 92b to be discriminated and the second multidimensional feature of the region specified by the user. Since the state of the vector 92c is displayed as a two-dimensional graph Dg for discrimination, it is possible to visualize the discrimination status of the discrimination target and the classification of the designated area. As a result, it is possible to visually confirm the discrimination status of the discrimination target and the discrimination status of the portion of concern by the user (for example, when there is a scratch on the discrimination target, the scratch can be discriminated as NG.
  • the learning of the feature extractor 26 and the feature converter 27, or the learning of the classifier 28 can be redone.
  • the classifier 28 classifies the discrimination target using the second multidimensional feature vectors 92a, 92b, 92c having features effective for the classification of the discrimination target, the classification of the discrimination target can be realized with high accuracy.
  • the classification of the discrimination target by the classifier 28 is performed by the second multidimensional of the discrimination target based on the calculation parameters machine-learned using the plurality of second multidimensional feature vectors 92a stored in the storage unit 33.
  • a one-dimensional numerical value indicating the certainty that the feature vectors 92b and 92c are OK products is calculated and compared with the set threshold value.
  • the classification of the discrimination target is performed.
  • the cause of the erroneous determination can be easily identified, and the countermeasure can be appropriately taken.
  • the shapes of the discriminant two-dimensional graph Dg and the selected two-dimensional graph Cg are separated from the model two-dimensional graph Ng, and it is determined that the product is NG despite the different shapes.
  • the shapes of the discriminant two-dimensional graph Dg and the selected two-dimensional graph Cg are different from the model two-dimensional graph Og, and despite the different shapes, the product is OK.
  • the two-dimensional graph Dg for discrimination is a graph generated by the two-dimensional graphing data generation unit 30 based on the second multidimensional feature vectors 92b and 92c of the discrimination target image data 34b and the designated area image data 34c. ..
  • the selected two-dimensional graph Cg is a graph generated by the two-dimensional graphing data generation unit 30 based on the selected two-dimensional graphing data 36a'.
  • the first sample image data 34a with a plurality of teacher signals, the discrimination target image data 34b, and the designated area image data 34c are used by using the trained feature extractor 26.
  • the multidimensional feature vectors 90a, 90b, 90c are extracted, and the extracted first multidimensional feature vectors 90a, 90b, 90c are used with the trained feature converter 27 to obtain the first multidimensional feature vectors 90a, 90b, 90c.
  • the second multidimensional feature vectors 92a, 92b, 92c which are lower in dimension and effective for classifying the discrimination target, and to classify the discrimination target based on the converted second multidimensional feature vectors 92a, 92b, 92c.
  • the classification of the discrimination target can be realized with high accuracy.
  • the first sample image data 34a with a plurality of teacher signals to be discriminated, the discriminant target image data 34b, and the designated area image data 34c have various feature quantities.
  • the feature extractor 26 for extracting the multidimensional feature vectors 90a, 90b, 90c and the first multidimensional feature vectors 90a, 90b, 90c are converted into the low-dimensional second multidimensional feature vectors 92a, 92b, 92c with the feature converter 27. Since the classifier 28 that classifies the discrimination target based on the second multidimensional feature vectors 92a, 92b, 92c converted by the feature converter 27 has a different configuration, the feature is improved in improving the classification accuracy of the discrimination target.
  • Learning of the extractor 26 and the feature converter 27 and learning of the classifier can be carried out separately. That is, when the cause of the low classification accuracy is the low feature extraction / conversion accuracy by the feature extractor 26 and the feature converter 27, the feature extractor 26 and the feature converter 27 are trained and classified. When it is caused by the low classification accuracy of the device 28, only the classifier 28 can be trained, and the classification accuracy can be efficiently improved.
  • the discriminating device has been described as a device that discriminates whether the discriminating target is an OK product or an NG product, but the present invention is not limited to this.
  • the discriminating device may be applied to a device that discriminates which of a plurality of classifications the discriminating target is classified into.
  • the vertical axis has column numbers 1, 2, 3, ..., N-1, n
  • the horizontal axis is the feature quantity.
  • a coordinate system Cartesian coordinate system
  • these feature quantities f 1 , f 2 , f 3 , ..., f n A two-dimensional graph in which -1 and f n are plotted and connected by a curve or a straight line is used, but the present invention is not limited to this.
  • the input device 14 has been described as a keyboard and a mouse, but may include a pointing device such as a touch panel, a button, a dial, a touch sensor, a touch pad, and the like.
  • the large-capacity memory for storing various data including image data is HDD 10, but the present invention is not limited to this.
  • a large-capacity memory for storing various data including image data a flash memory (USB memory, SD card, etc.), SSD, floppy disk, CD, DVD, or the like may be applied.
  • various applications including a visualization program are stored in the HDD 10, but the present invention is not limited to this.
  • various applications including a visualization program may be configured to be distributed from another computer to the computer 1 as a discrimination device according to the present embodiment via a transmission medium, for example, a communication network such as the Internet or LAN.
  • the present embodiment shows an example of a mode for carrying out the present invention. Therefore, the present invention is not limited to the configuration of the present embodiment.

Abstract

[Problem] To improve classification accuracy. [Solution] First multidimensional feature vectors 90a, 90b, 90c are extracted from a plurality of sets of teaching signal attached sample image data 34a, discrimination target image data 34b, or specified region image data 34c, using a trained feature extractor 26, the extracted first multidimensional feature vectors 90a, 90b, 90c are converted, using a trained feature converter 27, into second multidimensional feature vectors 92a, 92b, 92c having fewer dimensions than the first multidimensional feature vectors 90a, 90b, 90c and effective for classification of a discrimination target, and classification of the discrimination target is performed on the basis of the converted second multidimensional feature vectors 92a, 92b, 92c. As a result, classification of the discrimination target can be implemented with high accuracy.

Description

判別対象の分類方法、これに用いるプログラム、および、判別装置Classification method of discrimination target, program used for this, and discrimination device
　本発明は、判別対象の分類方法、これに用いるプログラム、および、判別装置に関するものである。 The present invention relates to a method for classifying discriminating objects, a program used for the classification method, and a discriminating device.
　特開２０１９－２１１２８８号公報（特許文献１）には、畳み込みニューラルネットワークを用いて判別対象の画像について畳み込み層による計算を行い、当該畳み込み層の計算が終了した後の最初の全結合層から特徴量（多次元特徴ベクトル）を抽出し、抽出した多次元特徴ベクトルを用いて多変量解析を実行して、当該多変量解析の結果に基づいて判別対象の分類を行う方法が記載されている。 Japanese Patent Application Laid-Open No. 2019-21128 (Patent Document 1) uses a convolutional neural network to perform a calculation by a convolutional layer on an image to be discriminated, and features from the first fully connected layer after the calculation of the convolutional layer is completed. A method of extracting a quantity (multidimensional feature vector), executing a multivariate analysis using the extracted multidimensional feature vector, and classifying the discrimination target based on the result of the multivariate analysis is described.
　当該分類方法では、多次元特徴ベクトルの抽出（畳み込みニューラルネットワークを利用）と、判別対象の分類（多変量解析を利用）と、を別構成で実現している。これにより、良品・不良品などの分類基準を容易に変更することができる。 In the classification method, extraction of a multidimensional feature vector (using a convolutional neural network) and classification of a discrimination target (using multivariate analysis) are realized by different configurations. As a result, the classification criteria for non-defective products and defective products can be easily changed.
特開２０１９－２１１２８８号公報JP-A-2019-21128
　しかしながら、畳み込み層の計算が終了した後の最初の全結合層から抽出した多次元特徴ベクトルは、判別対象の分類に用いる特徴量としては、必ずしも有効ではなく、分類精度の向上という点において、なお改良の余地がある。 However, the multidimensional feature vector extracted from the first fully connected layer after the calculation of the convolution layer is completed is not always effective as a feature quantity used for classification of the discrimination target, and in terms of improving the classification accuracy, it is still possible. There is room for improvement.
　本発明は、上記に鑑みてなされたものであり、分類精度の向上に資する技術を提供することを目的の一つとする。 The present invention has been made in view of the above, and one of the objects of the present invention is to provide a technique that contributes to improvement of classification accuracy.
　本発明に係る判別対象の分類方法の好ましい形態によれば、（ａ）判別対象のデータを取得し、（ｂ）深層学習を含むニューラルネットワークを用いた特徴抽出器によって、判別対象のデータから第１多次元特徴ベクトルを抽出し、（ｃ）抽出した第１多次元特徴ベクトルを、複数の全結合層を用いた特徴変換器によって、第１多次元特徴ベクトルよりも次元の低い第２多次元特徴ベクトルに変換し、（ｄ）変換された第２多次元特徴ベクトルに基づいて、統計的機械学習を用いた分類器によって判別対象を分類する。ここで、本発明における「判別対象のデータ」とは、典型的には判別対象の画像データがこれに該当し、文字通りに判別する必要がある対象の画像データがこれに該当するが、学習済モデルの学習段階においては、教師信号付きサンプルデータを含む概念である。さらに、本発明における「統計的機械学習器」とは深層学習を含むニューラルネットワークを用いた機械学習以外の機械学習、例えば、勾配ブースティングや、サポートベクターマシーン、ランダムフォレスト、ニューラルネットワーク、ガウス正規化、サンサンブル検査などがこれに該当する。 According to a preferable form of the classification method of the discrimination target according to the present invention, (a) the data of the discrimination target is acquired, and (b) a feature extractor using a neural network including deep learning is used to obtain the data of the discrimination target. A one-dimensional feature vector is extracted, and (c) the extracted first multidimensional feature vector is subjected to a second multidimensional lower dimension than the first multidimensional feature vector by a feature converter using a plurality of fully connected layers. It is converted into a feature vector, and (d) based on the converted second multidimensional feature vector, the discrimination target is classified by a classifier using statistical machine learning. Here, the "data to be discriminated" in the present invention typically corresponds to the image data to be discriminated, and the image data to be discriminated literally corresponds to this, but has been learned. In the learning stage of the model, it is a concept that includes sample data with a teacher signal. Further, the "statistical machine learning device" in the present invention refers to machine learning other than machine learning using a neural network including deep learning, such as gradient boosting, support vector machine, random forest, neural network, and Gaussian normalization. , Sansamble inspection, etc. correspond to this.
　本発明によれば、特徴変換器によって変換された特徴であって、判別対象の分類に有効な低次元の第２多次元特徴ベクトルを、判別対象の分類に用いるため、判別対象の分類を精度良く実現できる。なお、判別対象のデータが有する多様な特徴量である第１多次元特徴ベクトルを抽出する特徴抽出器および第１多次元特徴ベクトルを低次元の第２多次元特徴ベクトルに変換する複数の全結合層を用いた特徴変換器と、当該特徴変換器によって変換された第２多次元特徴ベクトルに基づき判別対象を分類する分類器と、が別構成であるため、判別対象の分類精度の向上に際し、特徴抽出器および特徴変換器の学習と、分類器の学習と、を分けて実施することができる。即ち、分類精度の低さの原因が、特徴抽出器および特徴変換器による特徴抽出・変換精度の低さに起因する場合には、特徴抽出器および特徴変換器を学習させ、分類器による分類精度の低さに起因する場合には分類器のみを学習させることができ、分類精度の向上を効率良く行うことができる。 According to the present invention, since the low-dimensional second multidimensional feature vector, which is a feature converted by the feature converter and is effective for classifying the discrimination target, is used for the classification of the discrimination target, the classification of the discrimination target is accurate. It can be realized well. It should be noted that a feature extractor for extracting the first multidimensional feature vector, which is a variety of feature quantities of the data to be discriminated, and a plurality of full couplings for converting the first multidimensional feature vector into a low-dimensional second multidimensional feature vector. Since the feature converter using the layer and the classifier that classifies the discrimination target based on the second multidimensional feature vector converted by the feature converter have different configurations, it is necessary to improve the classification accuracy of the discrimination target. Learning of feature extractors and feature converters and learning of classifiers can be performed separately. That is, when the cause of the low classification accuracy is the low feature extraction / conversion accuracy by the feature extractor and the feature converter, the feature extractor and the feature converter are trained and the classification accuracy by the classifier is performed. If it is caused by the low level, only the classifier can be trained, and the classification accuracy can be improved efficiently.
　本発明に係るプログラムの好ましい形態によれば、判別対象を分類するためのプログラムが構成される。当該プログラムは、上述したいずれかの態様の本発明に係る判別対象の分類方法の各ステップを１又は複数のコンピュータに実行させるためのものである。当該プログラムは、コンピュータが読み取り可能な記録媒体、例えば、ハードディスクやＲＯＭ、ＳＳＤ、フラッシュメモリ（ＵＳＢメモリ、ＳＤカードなど）、フロッピーディスク、ＣＤ、ＤＶＤなどに記録されていても良いし、伝送媒体、例えば、インターネットやＬＡＮなどの通信網を介してあるコンピュータから別のコンピュータへ配信されても良いし、あるいは、その他如何なる態様で授受されても良い。 According to the preferred form of the program according to the present invention, a program for classifying the discrimination target is configured. The program is for causing one or a plurality of computers to execute each step of the classification method of the discrimination target according to the present invention in any of the above-described aspects. The program may be recorded on a computer-readable recording medium such as a hard disk, ROM, SSD, flash memory (USB memory, SD card, etc.), floppy disk, CD, DVD, or a transmission medium. For example, it may be distributed from one computer to another computer via a communication network such as the Internet or LAN, or may be exchanged in any other manner.
　本発明によれば、プログラムを一つのコンピュータに実行させるか又は複数のコンピュータに各ステップを分担して実行させることによって、上述したいずれかの態様の本発明に係る判別対象の分類方法の各ステップが実行されるため、上述した本発明に係る判別対象の分類方法と同様の作用効果、例えば、判別対象の分類を精度良く実現できるという効果を得ることができる。 According to the present invention, by causing one computer to execute the program or having a plurality of computers execute each step in a shared manner, each step of the discrimination target classification method according to the present invention in any of the above-described embodiments. Is executed, so that the same action and effect as the above-described method for classifying the discrimination target according to the present invention, for example, the effect that the classification of the discrimination target can be accurately realized can be obtained.
　本発明に係る判別装置の好ましい形態によれば、判別対象のデータを取得するデータ取得部と、取得した判別対象のデータから第１多次元特徴ベクトルを抽出する深層学習を含むニューラルネットワークを用いた特徴抽出器と、当該特徴抽出器により抽出された第１多次元特徴ベクトルを、当該第１多次元特徴ベクトルよりも低次元の第２多次元特徴ベクトルに変換する複数の全結合層を用いた特徴変換器と、第２多次元特徴ベクトルを用いて判別対象を分類する統計的機械学習を用いた分類器と、を備えている。ここで、本発明における「判別対象のデータ」とは、典型的には判別対象の画像データがこれに該当し、文字通りに判別する必要がある対象の画像データがこれに該当するが、特徴抽出器および特徴変換器の学習段階においては、教師信号付きサンプルの画像データを含む概念である。また、本発明における「統計的機械学習器」とは深層学習を含むニューラルネットワークを用いた機械学習以外の機械学習、例えば、勾配ブースティングや、サポートベクターマシーン、ランダムフォレスト、ニューラルネットワーク、ガウス正規化、サンサンブル検査などがこれに該当する。 According to a preferred embodiment of the discrimination device according to the present invention, a data acquisition unit for acquiring data to be discriminated and a neural network including deep learning for extracting a first multidimensional feature vector from the acquired data to be discriminated are used. A feature extractor and a plurality of fully connected layers that convert the first multidimensional feature vector extracted by the feature extractor into a second multidimensional feature vector having a lower dimension than the first multidimensional feature vector are used. It includes a feature converter and a classifier using statistical machine learning that classifies discrimination targets using a second multidimensional feature vector. Here, the "data to be discriminated" in the present invention typically corresponds to the image data to be discriminated, and the image data to be discriminated literally corresponds to this, but feature extraction. In the learning stage of the device and the feature converter, it is a concept that includes image data of a sample with a teacher signal. Further, the "statistical machine learning device" in the present invention refers to machine learning other than machine learning using a neural network including deep learning, such as gradient boosting, support vector machine, random forest, neural network, and Gaussian normalization. , Sansamble inspection, etc. correspond to this.
　本発明によれば、特徴変換器によって変換された特徴であって、判別対象の分類に有効な低次元の第２多次元特徴ベクトルを、判別対象の分類に用いるため、判別対象の分類を精度良く実現できる。なお、判別対象のデータが有する多様な特徴量である第１多次元特徴ベクトルを抽出する特徴抽出器と、第１多次元特徴ベクトルを低次元の第２多次元特徴ベクトルに変換する複数の全結合層を用いた特徴変換器と、当該特徴変換器によって変換された第２多次元特徴ベクトルに基づき判別対象を分類する分類器と、が別構成であるため、判別対象の分類精度の向上に際し、特徴抽出器および特徴変換器の学習と、分類器の学習と、を分けて実施することができる。即ち、分類精度の低さの原因が、特徴抽出器および特徴変換器による特徴抽出精度の低さに起因する場合には、特徴抽出器および特徴変換器を学習させ、分類器による分類精度の低さに起因する場合には分類器のみを学習させることができ、分類精度の向上を効率良く行うことができる。 According to the present invention, since the low-dimensional second multidimensional feature vector, which is a feature converted by the feature converter and is effective for classifying the discrimination target, is used for the classification of the discrimination target, the classification of the discrimination target is accurate. It can be realized well. A feature extractor that extracts the first multidimensional feature vector, which is a variety of feature quantities of the data to be discriminated, and a plurality of all that convert the first multidimensional feature vector into a low-dimensional second multidimensional feature vector. Since the feature converter using the coupling layer and the classifier that classifies the discrimination target based on the second multidimensional feature vector converted by the feature converter have different configurations, the classification accuracy of the discrimination target is improved. , The learning of the feature extractor and the feature converter, and the learning of the classifier can be carried out separately. That is, when the cause of the low classification accuracy is the low feature extraction accuracy by the feature extractor and the feature converter, the feature extractor and the feature converter are trained and the classification accuracy by the classifier is low. If this is the case, only the classifier can be trained, and the classification accuracy can be improved efficiently.
　本発明に係る判別装置の更なる形態によれば、第２多次元特徴ベクトルを記憶する記憶部と、当該記憶部に記憶された複数の第２多次元特徴ベクトルについて各々の間の距離値を算定する算定部と、をさらに備えている。そして、特徴抽出器および特徴変換器は、学習段階においては、算定部により算定された距離値に基づき誤差逆伝播法および勾配降下法を用いて学習する。 According to a further form of the discriminating device according to the present invention, a storage unit that stores the second multidimensional feature vector and a distance value between each of the plurality of second multidimensional feature vectors stored in the storage unit are stored. It also has a calculation department for calculation. Then, in the learning stage, the feature extractor and the feature converter learn by using the error backpropagation method and the gradient descent method based on the distance value calculated by the calculation unit.
　本形態によれば、判別対象の分類に有効、かつ、特徴抽出器および特徴変換器の学習段階において抽出および変換された第２多次元特徴ベクトルを用いて判別対象の分類を行うため、分類精度のより一層の向上を図ることができる。 According to this embodiment, the classification of the discrimination target is effective, and the classification of the discrimination target is performed using the second multidimensional feature vector extracted and converted in the learning stage of the feature extractor and the feature converter. Therefore, the classification accuracy is correct. Can be further improved.
　本発明によれば、分類精度の向上を図ることができる。 According to the present invention, the classification accuracy can be improved.
本発明の実施の形態に係る判別装置として機能するコンピュータ１の構成の概略を示す構成図である。It is a block diagram which shows the outline of the structure of the computer 1 which functions as the discriminating device which concerns on embodiment of this invention. 本発明の実施の形態に係る判別装置として機能するコンピュータ１の機能構成を示す機能ブロック図である。It is a functional block diagram which shows the functional structure of the computer 1 which functions as the discrimination apparatus which concerns on embodiment of this invention. 特徴抽出器２６の構成の概略を示す構成図である。It is a block diagram which shows the outline of the structure of the feature extractor 26. 学習済モデル３５の生成および判別対象の分類の概略を示す説明図である。It is explanatory drawing which shows the outline of the generation of the trained model 35 and the classification of the discriminant object. 第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃの二次元グラフ化の概略を示す説明図である。It is explanatory drawing which shows the outline of the 2D graphing of the 2nd multidimensional feature vector 92a, 92b, 92c. 分類判別ルーチンの一例を示すフローチャートである。It is a flowchart which shows an example of the classification discrimination routine. 同じ数値指標を有する学習済モデル３５の学習状況（個性）を示す説明図である。It is explanatory drawing which shows the learning situation (individuality) of the trained model 35 which has the same numerical index. ウィンドウ６２にモデル用二次元グラフＯｇ，Ｎｇ、および、判別用二次元グラフＤｇ，選択二次元グラフＣｇが表示された状態を示す説明図である。It is explanatory drawing which shows the state which the 2D graph Og, Ng for a model, the 2D graph Dg for discrimination, and the selection 2D graph Cg are displayed in the window 62. 分類器２８の演算パラメータあるいは閾値の設定が適切でない場合の判別用二次元グラフＤｇ，選択二次元グラフＣｇの状態を示す説明図である。It is explanatory drawing which shows the state of the 2D graph Dg for discrimination, and the selection 2D graph Cg when the setting of the calculation parameter or the threshold value of the classifier 28 is not appropriate.
　次に、本発明を実施するための最良の形態を実施例を用いて説明する。 Next, the best mode for carrying out the present invention will be described with reference to examples.
　本実施の形態に係る判別装置として機能するコンピュータ１は、図１に示すように、ＣＰＵ２を中心とするマイクロプロセッサとして構成されており、各種処理プログラムを記憶するＲＯＭ４、一時的にデータを記憶するＲＡＭ６、画像処理を行う際に必要な計算処理や行列演算処理を行うＧＰＵ８、各種アプリケーションプログラム（単にアプリケーションと称する）や画像データを含む各種データを記憶する大容量メモリであるハードディスク（ＨＤＤ）１０、カメラ７０などの外部機器とのデータの入出力を行う入出力インターフェイス（Ｉ／Ｆ）１２などを備えている。なお、以下では説明の便宜上、本実施の形態に係る判別装置としてのコンピュータ１が、判別対象がＯＫ品であるかＮＧ品であるかの判別を行う装置として説明する。コンピュータ１は、本発明における「判別装置」に対応する実施構成の一例である。 As shown in FIG. 1, the computer 1 that functions as the discrimination device according to the present embodiment is configured as a microprocessor centered on the CPU 2, and has a ROM 4 that stores various processing programs and temporarily stores data. RAM 6, GPU 8 that performs calculation processing and matrix calculation processing necessary for performing image processing, hard disk (HDD) 10, which is a large-capacity memory that stores various data including various application programs (simply called applications) and image data, It is provided with an input / output interface (I / F) 12 for inputting / outputting data to / from an external device such as a camera 70. In the following, for convenience of explanation, the computer 1 as the discrimination device according to the present embodiment will be described as a device that discriminates whether the discrimination target is an OK product or an NG product. The computer 1 is an example of an implementation configuration corresponding to the "discrimination device" in the present invention.
　コンピュータ１は、ユーザが各種指令を入力するキーボードおよびマウス等の入力装置１４や、各種情報を表示するディスプレイ６０などを備えている。ＣＰＵ２やＲＯＭ４、ＲＡＭ６、ＧＰＵ８、ＨＤＤ１０、Ｉ／Ｆ１２、入力装置１４、ディスプレイ６０などは、バス８０によって電気的に接続され、互いに各種制御信号やデータのやり取りができるように構成されている。 The computer 1 is provided with an input device 14 such as a keyboard and a mouse for which a user inputs various commands, a display 60 for displaying various information, and the like. The CPU 2, ROM 4, RAM 6, GPU 8, HDD 10, I / F 12, input device 14, display 60, and the like are electrically connected by a bus 80 so that various control signals and data can be exchanged with each other.
　当該コンピュータ１は、ディスプレイ６０に表示されたカーソル等をユーザが入力装置１４を介して入力操作すると、その入力操作に応じた動作を実行する機能を有している。また、当該コンピュータ１は、ＨＤＤ１０に格納されたアプリケーション、具体的には、判別対象の判別処理を実行する判別アプリケーションによって各種処理が実行されることで、本実施の形態に係る判別装置として機能する。なお、本実施の形態では、判別装置としてコンピュータ１によって実現可能な構成としたが、専用装置として実現しても良い。 The computer 1 has a function of executing an operation corresponding to the input operation when the user inputs the cursor or the like displayed on the display 60 via the input device 14. Further, the computer 1 functions as a discrimination device according to the present embodiment by executing various processes by an application stored in the HDD 10, specifically, a discrimination application that executes the discrimination processing of the discrimination target. .. In the present embodiment, the discriminating device is configured to be feasible by the computer 1, but it may be realized as a dedicated device.
　また、コンピュータ１には、図２に示すように、ＣＰＵ２やＲＯＭ４、ＲＡＭ６、ＧＰＵ８、ＨＤＤ１０、Ｉ／Ｆ１２、入力装置１４、ディスプレイ６０などの前述したハードウェア資源と、判別アプリケーションや本実施の形態に係る可視化プログラムといったソフトウェアと、の一方または双方の協働により、画像表示制御部２０や、画像取得部２２、領域指定部２４、特徴抽出器２６、特徴変換器２７、分類器２８、距離算定器２９、二次元グラフ化データ生成部３０、記憶部３３等が機能ブロックとして構成されている。換言すれば、これらの各部（画像表示制御部２０、画像取得部２２、領域指定部２４、特徴抽出器２６、特徴変換器２７、分類器２８、距離算定器２９、二次元グラフ化データ生成部３０、記憶部３３）は、ＨＤＤ１０からＲＡＭ６上に展開されたアプリケーションを実行するＣＰＵ２からの命令によって、図１に示す各構成要素（ＣＰＵ２やＲＯＭ４、ＲＡＭ６、ＧＰＵ８、ＨＤＤ１０、Ｉ／Ｆ１２、入力装置１４、ディスプレイ６０など）が単独あるいは協働して動作することにより実現される機能であると言うことができる。なお、画像表示制御部２０や、画像取得部２２、領域指定部２４、特徴抽出器２６、特徴変換器２７、分類器２８、距離算定器２９、二次元グラフ化データ生成部３０、記憶部３３等は、アドレスバスやデータバスなどのバスライン８２によって電気的に接続されている。特徴抽出器２６、特徴変換器２７およびディスプレイ６０（後述するウィンドウ６２）は、本発明における「可視化装置」に対応する実施構成の一例である。 Further, as shown in FIG. 2, the computer 1 includes the above-mentioned hardware resources such as CPU2, ROM4, RAM6, GPU8, HDD10, I / F12, input device 14, and display 60, a discrimination application, and the present embodiment. Image display control unit 20, image acquisition unit 22, area designation unit 24, feature extractor 26, feature converter 27, classifier 28, distance calculation in collaboration with software such as the visualization program related to The device 29, the two-dimensional graphed data generation unit 30, the storage unit 33, and the like are configured as functional blocks. In other words, each of these parts (image display control unit 20, image acquisition unit 22, area designation unit 24, feature extractor 26, feature converter 27, classifier 28, distance calculator 29, two-dimensional graphing data generation unit). 30; 14, display 60, etc.) can be said to be a function realized by operating independently or in cooperation. The image display control unit 20, the image acquisition unit 22, the area designation unit 24, the feature extractor 26, the feature converter 27, the classifier 28, the distance calculator 29, the two-dimensional graph data generation unit 30, and the storage unit 33. Etc. are electrically connected by a bus line 82 such as an address bus or a data bus. The feature extractor 26, the feature converter 27, and the display 60 (window 62 described later) are examples of an implementation configuration corresponding to the “visualization device” in the present invention.
　画像表示制御部２０は、判別アプリケーションが起動されると、図１に示すように、ディスプレイ６０の画面６１に所定のウィンドウ６２を表示する。また、画像表示制御部２０は、ディスプレイ６０に表示されたカーソル等をユーザが入力装置１４を介して入力操作することにより選択された教師信号付きサンプル画像データ３４ａ（図２参照）に基づいて教師信号付きサンプル画像をウィンドウ６２上に表示させたり、画像取得部２２により取得される判別対象画像データ３４ｂ（図２参照）に基づいて判別対象画像をウィンドウ６２上に表示させたり、領域指定部２４により取得される指定領域画像データ３４ｃ（図２参照）に基づいて指定領域画像をウィンドウ６２上に表示させる。 When the discrimination application is started, the image display control unit 20 displays a predetermined window 62 on the screen 61 of the display 60 as shown in FIG. Further, the image display control unit 20 teaches based on the sample image data 34a with a teacher signal (see FIG. 2) selected by the user inputting the cursor or the like displayed on the display 60 via the input device 14. A sample image with a signal is displayed on the window 62, a discrimination target image is displayed on the window 62 based on the discrimination target image data 34b (see FIG. 2) acquired by the image acquisition unit 22, and the area designation unit 24 is displayed. The designated area image is displayed on the window 62 based on the designated area image data 34c (see FIG. 2) acquired by.
　さらに、画像表示制御部２０は、記憶部３３に記憶されたモデル用二次元グラフ化データ３６ａ（図２参照）に基づいて二次元グラフをウィンドウ６２上に表示させたり、二次元グラフ化データ生成部３０によって生成される判別用二次元グラフ化データ３６ｂに基づいて判別用二次元グラフをウィンドウ６２上に表示させたりする。また、画像表示制御部２０は、分類器２８による判別結果をウィンドウ６２に表示する。なお、教師信号付きサンプル画像とは、特徴抽出器２６および特徴変換器２７の学習を行うために使用する画像である。また、判別対象画像とは、典型的には、分類（ＯＫ品　ｏｒ　ＮＧ品）が不明な対象の画像がこれに該当するが、特徴抽出器２６および特徴変換器２７の学習結果の検証や学習精度の向上を目的として分類（ＯＫ品　ｏｒ　ＮＧ品）が既知の対象の画像、例えば、複数の教師信号付きサンプル画像からユーザが任意に選択した教師信号付きサンプル画像を好適に包含する。 Further, the image display control unit 20 displays a two-dimensional graph on the window 62 based on the model two-dimensional graphed data 36a (see FIG. 2) stored in the storage unit 33, or generates two-dimensional graphed data. The discrimination two-dimensional graph is displayed on the window 62 based on the discrimination two-dimensional graphing data 36b generated by the unit 30. Further, the image display control unit 20 displays the discrimination result by the classifier 28 in the window 62. The sample image with a teacher signal is an image used for learning the feature extractor 26 and the feature converter 27. Further, the discrimination target image typically corresponds to an image of a target whose classification (OK product or NG product) is unknown, but verification and learning of the learning results of the feature extractor 26 and the feature converter 27 For the purpose of improving accuracy, it preferably includes an image of a target whose classification (OK product or NG product) is known, for example, a sample image with a teacher signal arbitrarily selected by the user from a plurality of sample images with a teacher signal.
　画像取得部２２は、カメラ７０（図１参照）によって撮影された教師信号付きサンプル画像データ３４ａや判別対象画像データ３４ｂを取得し、取得したこれら画像データを画像表示制御部２０や特徴抽出器２６、記憶部３３などに供給する。 The image acquisition unit 22 acquires sample image data 34a with a teacher signal and image data 34b to be discriminated taken by the camera 70 (see FIG. 1), and uses these acquired image data as the image display control unit 20 and the feature extractor 26. , Supply to the storage unit 33 and the like.
　領域指定部２４は、ユーザの入力装置１４を介した入力操作によって、ウィンドウ６２上に表示された教師信号付きサンプル画像や判別対象画像の任意の領域がクリックやドラッグなどされた際に、当該領域の画像データ（以下、「指定領域画像データ」という）３４ｃを取得し、取得した指定領域画像データ３４ｃ（図２参照）を画像表示制御部２０や特徴抽出器２６、記憶部３３などに供給する。 The area designation unit 24 is used when an arbitrary area of the sample image with a teacher signal or the image to be discriminated displayed on the window 62 is clicked or dragged by an input operation via the user's input device 14. 34c of the image data (hereinafter referred to as “designated area image data”) 34c is acquired, and the acquired designated area image data 34c (see FIG. 2) is supplied to the image display control unit 20, the feature extractor 26, the storage unit 33, and the like. ..
　特徴抽出器２６は、複数の教師信号付きサンプル画像データ３４ａや、判別対象画像データ３４ｂ、指定領域画像データ３４ｃから第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを抽出し、抽出した第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを特徴変換器２７に供給する。ここで、本実施の形態では、特徴抽出器２６には、図３および図４に示すように、畳み込みニューラルネットワーク（ＣＮＮ）を用いる構成とした。第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃは、それぞれ本発明における「第１多次元特徴ベクトル」に対応する実施構成の一例である。 The feature extractor 26 extracts the first multidimensional feature vectors 90a, 90b, 90c from the plurality of sample image data 34a with teacher signals, the discriminant target image data 34b, and the designated area image data 34c, and extracts the first multidimensional. The feature vectors 90a, 90b, 90c are supplied to the feature converter 27. Here, in the present embodiment, the feature extractor 26 is configured to use a convolutional neural network (CNN) as shown in FIGS. 3 and 4. The first multidimensional feature vectors 90a, 90b, and 90c are examples of implementation configurations corresponding to the "first multidimensional feature vector" in the present invention, respectively.
　特徴抽出器２６による第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃの抽出は、図３に示すように、所定のフィルタ（図示せず）を用いて複数の教師信号付きサンプル画像データ３４ａや判別対象画像データ３４ｂ、指定領域画像データ３４ｃから、これら画像データ３４ａ，３４ｂ，３４ｃの特徴を失うことなく特徴を抽出する所謂畳み込み処理を複数回行った後、列ベクトルにフラット化することにより行う。 In the extraction of the first multidimensional feature vectors 90a, 90b, 90c by the feature extractor 26, as shown in FIG. 3, a plurality of sample image data 34a with teacher signals and discrimination targets are used using a predetermined filter (not shown). The so-called convolution process for extracting the features of the image data 34a, 34b, 34c from the image data 34b and the designated area image data 34c without losing the features is performed a plurality of times, and then flattened into a column vector.
　特徴変換器２７は、図４に示すように、複数の全結合層２７ａ，２７ｂを有しており、当該全結合層２７ａ，２７ｂを用いて、特徴抽出器２６によって抽出された第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを低次元化する処理を実行する。具体的には、全結合（Ｆｕｌｌｙ　Ｃｏｎｎｅｃｔｅｄ）処理によって第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを当該第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃよりも低次元の第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに変換し、分類器２８や距離算定器２９、二次元グラフ化データ精製部３０、記憶部３３に供給する。なお、全結合（Ｆｕｌｌｙ　Ｃｏｎｎｅｃｔｅｄ）処理に替えてＧｌｏｂａｌ　ｍａｘ　ＰｏｏｌｉｎｇやＧｌｏｂａｌ　Ａｖｅｒａｇｅ　Ｐｏｏｌｉｎｇなどのプーリング処理によって、第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを当該第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃよりも低次元の第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに変換しても良い。第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃは、それぞれ記憶部３３に記憶される。第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃは、本発明における「第２多次元特徴ベクトル」に対応する実施構成の一例である。 As shown in FIG. 4, the feature converter 27 has a plurality of fully connected layers 27a and 27b, and the first multidimensional extracted by the feature extractor 26 using the fully connected layers 27a and 27b. The process of lowering the dimension of the feature vectors 90a, 90b, 90c is executed. Specifically, the first multidimensional feature vectors 90a, 90b, 90c are converted into the first multidimensional feature vectors 90a, 90b, 90c by Fully Connected processing, and the second multidimensional feature vectors 92a, which are lower in dimension than the first multidimensional feature vectors 90a, 90b, 90c, It is converted into 92b and 92c and supplied to the classifier 28, the distance calculator 29, the two-dimensional graphing data purification unit 30, and the storage unit 33. It should be noted that the first multidimensional feature vectors 90a, 90b, 90c are obtained from the first multidimensional feature vectors 90a, 90b, 90c by pooling processing such as Global max Polling or Global Average Polling instead of the Fully Connected processing. May be converted into low-dimensional second multidimensional feature vectors 92a, 92b, 92c. The second multidimensional feature vectors 92a, 92b, and 92c are stored in the storage unit 33, respectively. The second multidimensional feature vectors 92a, 92b, and 92c are examples of the implementation configurations corresponding to the "second multidimensional feature vector" in the present invention.
　なお、特徴抽出器２６および特徴変換器２７は、判別対象の分類の判別を精度良く行うことができるような第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃおよび第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃを得ることができるように、予め学習を行って、学習済モデル３５として記憶部３３に記憶させておく。　　　 The feature extractor 26 and the feature converter 27 include the first multidimensional feature vectors 90a, 90b, 90c and the second multidimensional feature vectors 92a, 92b, which can accurately discriminate the classification of the discrimination target. Learning is performed in advance so that 92c can be obtained, and the learned model 35 is stored in the storage unit 33. Twice
　分類器２８は、図４（ｂ）に示すように、判別対象の分類の判別を行う際に機能する。図４（ｂ）に示すように、特徴変換器２７によって変換された第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに基づいて、判別対象の分類、即ち、判別対象がＯＫ品であるかＮＧ品であるかの判別を行い、当該判別結果を画像表示制御部２０に供給する。判別対象の分類は、具体的には、第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃを用いて機械学習した演算パラメータにより判別対象の第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃがＯＫ品である確信度を示す一次元の数値を演算し、設定した閾値以上であればＯＫ品であると分類し、設定した閾値未満であればＮＧ品であると分類する。本実施の形態では、分類器２８には、勾配ブースティングや、サポートベクターマシーン、ランダムフォレスト、ニューラルネットワーク、ガウス正規化、アンサンブル検査などの所謂統計的機械学習を用いる構成とした。 As shown in FIG. 4B, the classifier 28 functions when discriminating the classification of the discriminating target. As shown in FIG. 4B, the classification of the discrimination target, that is, whether the discrimination target is an OK product or an NG product, is based on the second multidimensional feature vectors 92a, 92b, 92c converted by the feature converter 27. Is determined, and the determination result is supplied to the image display control unit 20. Specifically, the classification of the discrimination target is that the second multidimensional feature vectors 92a, 92b, 92c to be discriminated by the arithmetic parameters machine-learned using the second multidimensional feature vectors 92a, 92b, 92c are OK products. A one-dimensional numerical value indicating the degree of certainty is calculated, and if it is equal to or more than the set threshold value, it is classified as an OK product, and if it is less than the set threshold value, it is classified as an NG product. In the present embodiment, the classifier 28 is configured to use so-called statistical machine learning such as gradient boosting, support vector machine, random forest, neural network, Gaussian normalization, and ensemble inspection.
　距離算定器２９は、学習済モデル３５を生成する際、および、当該学習済モデル３５の検証や精度の向上を図る際に機能する。学習済モデル３５を生成する際には、距離算定器２９は、複数の教師信号付きサンプル画像データ３４ａの第２多次元特徴ベクトル９２ａ間の距離を算定し、算定した当該距離を距離データ３８として記憶部３３に記憶する。算定した当該距離は、特徴抽出器２６および特徴変換器２７にフィードバックされる（図４（ａ）参照）。一方、学習済モデル３５の検証や精度の向上を図る際には、距離算定器２９は、複数の教師信号付きサンプル画像データ３４ａの第２多次元特徴ベクトル９２ａと、判別対象画像データ３４ｂや指定領域画像データ３４ｃの第２多次元特徴ベクトル９２ｂ，９２ｃと、の間の距離を算定し、算定した当該距離を距離データ３８として記憶部３３に記憶する。 The distance calculator 29 functions when generating the trained model 35, and when verifying the trained model 35 and improving the accuracy. When generating the trained model 35, the distance calculator 29 calculates the distance between the second multidimensional feature vectors 92a of the sample image data 34a with a plurality of teacher signals, and uses the calculated distance as the distance data 38. It is stored in the storage unit 33. The calculated distance is fed back to the feature extractor 26 and the feature converter 27 (see FIG. 4A). On the other hand, when verifying the trained model 35 and improving the accuracy, the distance calculator 29 uses the second multidimensional feature vector 92a of the sample image data 34a with a plurality of teacher signals, the discriminant image data 34b, and the designation. The distance between the second multidimensional feature vectors 92b and 92c of the area image data 34c is calculated, and the calculated distance is stored in the storage unit 33 as the distance data 38.
　本実施の形態では、特徴抽出器２６および特徴変換器２７へのフィードバックは、ＯＫ品同士の第２多次元特徴ベクトル９２ａ間の距離と、ＯＫ品の第２多次元特徴ベクトル９２ａおよびＮＧ品の第２多次元特徴ベクトル９２ａ間の距離と、が相対的に最適化されるように、Ｔｒｉｐｌｅｔ　ｌｏｓｓ関数などの損失関数を用いて誤差逆伝搬法および勾配降下法により、特徴抽出器２６および特徴変換器２７のパラメータの修正を行うことにより行う構成とした。このようなパラメータの修正（特徴抽出器２６および特徴変換器２７の学習）は、第２多次元特徴ベクトル９２ａの可視化（二次元グラフ化）を行って、学習済モデル３５（特徴抽出器２６および特徴変換器２７）の学習状況（個性）を把握したうえで、当該学習状況（個性）が所望の状態となるまで実施される。こうして、学習済モデル３５が生成され、記憶部３３に記憶される。なお、本実施の形態では、学習済モデル３５の学習と、特徴抽出器２６および特徴変換器２７の学習と、は同義である。 In the present embodiment, the feedback to the feature extractor 26 and the feature converter 27 is the distance between the second multidimensional feature vectors 92a between the OK products and the second multidimensional feature vector 92a of the OK product and the NG product. The feature extractor 26 and feature conversion by the error backpropagation method and the gradient descent method using a loss function such as the Triplet loss function so that the distance between the second multidimensional feature vectors 92a is relatively optimized. The configuration was made by modifying the parameters of the vessel 27. In such parameter modification (learning of the feature extractor 26 and the feature converter 27), the second multidimensional feature vector 92a is visualized (two-dimensional graphing), and the trained model 35 (feature extractor 26 and the feature converter 27) is modified. After grasping the learning situation (individuality) of the feature converter 27), it is carried out until the learning situation (individuality) becomes a desired state. In this way, the trained model 35 is generated and stored in the storage unit 33. In the present embodiment, the learning of the trained model 35 and the learning of the feature extractor 26 and the feature converter 27 are synonymous.
　ここで、本実施の形態では、特徴抽出器２６および特徴変換器２７のパラメータの修正には、ディープメトリックラーニングを用いる構成とした。即ち、特徴抽出器２６によって第１多次元特徴ベクトル９０ａを抽出し、抽出した第１多次元特徴ベクトル９０ａを特徴変換器２７によって第２多次元特徴ベクトル９２ａに変換すると共に、変換した第２多次元特徴ベクトル９２ａ間の距離を距離算定器２９によって算定して、算定した当該距離に基づき誤差逆伝搬法および勾配降下法により特徴抽出器２６および特徴変換器２７のパラメータの修正を行う一連の処理にディープメトリックラーニングを適用した。 Here, in the present embodiment, deep metric learning is used for modifying the parameters of the feature extractor 26 and the feature converter 27. That is, the first multidimensional feature vector 90a is extracted by the feature extractor 26, and the extracted first multidimensional feature vector 90a is converted into the second multidimensional feature vector 92a by the feature converter 27, and the converted second multi. A series of processes in which the distance between the dimensional feature vectors 92a is calculated by the distance calculator 29, and the parameters of the feature extractor 26 and the feature converter 27 are corrected by the error back propagation method and the gradient descent method based on the calculated distance. Deep metric learning was applied to.
　二次元グラフ化データ生成部３０は、複数の教師信号付きサンプル画像データ３４ａの第２多次元特徴ベクトル９２ａに基づいて、モデル用二次元グラフ化データ３６ａを生成すると共に、判別対象画像データ３４ｂや指定領域画像データ３４ｃの第２多次元特徴ベクトル９２ｂ，９２ｃに基づいて、判別用二次元グラフ化データ３６ｂを生成する。 The two-dimensional graphing data generation unit 30 generates the model two-dimensional graphing data 36a based on the second multidimensional feature vector 92a of the plurality of sample image data 34a with teacher signals, and also generates the discrimination target image data 34b and Based on the second multidimensional feature vectors 92b and 92c of the designated area image data 34c, the discriminant two-dimensional graphed data 36b is generated.
　具体的には、第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃを次元数の特徴量の集合としてモデル用二次元グラフ化データ３６ａや判別用二次元グラフ化データ３６ｂを生成する。例えば、第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃがｎ次元列ベクトル（ｆ１，ｆ２，ｆ３，・・・，ｆｎ－１，ｆｎ）である場合、ｎ個の特徴量ｆ１，ｆ２，ｆ３，・・・，ｆｎ－１，ｆｎの集合としたデータ、即ち、図５に示すように、縦軸に列番号１，２，３，・・・，ｎ－１，ｎをとり、横軸に特徴量ｆ１，ｆ２，ｆ３，・・・，ｆｎ－１，ｆｎをとった座標系（直交座標系）に、これら特徴量ｆ１，ｆ２，ｆ３，・・・，ｆｎ－１，ｆｎをプロットして曲線ないし直線で繋いだ二次元グラフを表示可能なデータを生成する。 Specifically, the two-dimensional graphing data 36a for the model and the two-dimensional graphing data 36b for discrimination are generated by using the second multidimensional feature vectors 92a, 92b, and 92c as a set of the feature quantities of the number of dimensions. For example, when the second multidimensional feature vectors 92a, 92b, 92c are n-dimensional column vectors (f 1 , f 2 , f 3 , ..., f n-1 , f n ), n feature quantities f. Data as a set of 1 , f 2 , f 3 , ..., f n-1 , f n , that is, as shown in FIG. 5, column numbers 1, 2, 3, ..., N on the vertical axis. In a coordinate system (Cartesian coordinate system) with -1, n and feature quantities f 1 , f 2 , f 3 , ..., f n-1 , f n on the horizontal axis, these feature quantities f 1 , Plot f 2 , f 3 , ..., f n-1 , f n to generate data that can display a two-dimensional graph connected by a curve or a straight line.
　なお、学習済モデル３５（特徴抽出器２６および特徴変換器２７）の学習結果の検証や学習精度の向上を目的として、ユーザが記憶部３３に記憶された複数の教師信号付きサンプル画像データ３４ａの中から任意の一の教師信号付きサンプル画像データ３４ａを選択した場合には、二次元グラフ化データ生成部３０は、モデル用二次元グラフ化データ３６ａの中から、当該任意の一の教師信号付きサンプル画像データ３４ａ（以下、「任意サンプル画像データ３４ａ’」という）の第２多次元特徴ベクトルデータ９２ａ（以下、「任意の第２多次元特徴ベクトル９２ａ’」という）に基づいて生成されたモデル用二次元グラフ化データ３６ａ（以下、「選択二次元グラフ化データ３６ａ’」という）を抽出する。 For the purpose of verifying the learning result of the trained model 35 (feature extractor 26 and feature converter 27) and improving the learning accuracy, the sample image data 34a with a plurality of teacher signals stored in the storage unit 33 by the user. When any one sample image data 34a with a teacher signal is selected from the data, the two-dimensional graphing data generation unit 30 has the arbitrary one teacher signal from the model two-dimensional graphed data 36a. A model generated based on the second multidimensional feature vector data 92a (hereinafter referred to as "arbitrary second multidimensional feature vector 92a'") of the sample image data 34a (hereinafter referred to as "arbitrary sample image data 34a'"). Two-dimensional graphing data 36a for use (hereinafter, referred to as "selected two-dimensional graphing data 36a'") is extracted.
　記憶部３３は、ＲＡＭ６およびＨＤＤ１０の少なくとも一方に確保され、カメラ７０（図１参照）によって撮影された教師信号付きサンプル画像データ３４ａおよび判別対象画像データ３４ｂや、領域指定部２４によって取得された指定領域画像データ３４ｃ、学習済モデル３５、二次元グラフ化データ生成部３０によって生成されたモデル用二次元グラフ化データ３６ａおよび判別用二次元グラフ化データ３６ｂ、特徴変換器２７によって変換された第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃ、および、距離算定器２９によって算定された距離データ３８を記憶する。 The storage unit 33 is secured in at least one of the RAM 6 and the HDD 10, and is the sample image data 34a with a teacher signal taken by the camera 70 (see FIG. 1), the image data 34b to be discriminated, and the designation acquired by the area designation unit 24. Area image data 34c, trained model 35, model two-dimensional graphing data 36a generated by the two-dimensional graphing data generation unit 30, discriminant two-dimensional graphing data 36b, second converted by the feature converter 27. The multidimensional feature vectors 92a, 92b, 92c and the distance data 38 calculated by the distance calculator 29 are stored.
　次に、本実施の形態に係る判別装置としてのコンピュータ１の動作、特に、判別対象の分類を判別する際の動作について説明する。図６は、分類判別ルーチンの一例を示すメインフローチャートである。 Next, the operation of the computer 1 as the discrimination device according to the present embodiment, particularly the operation when discriminating the classification of the discrimination target will be described. FIG. 6 is a main flowchart showing an example of the classification determination routine.
　分類判別ルーチンが実行されると、まず、学習済モデル３５を利用した特徴抽出器２６（学習済の特徴抽出器２６）が、判別対象画像データ３４ｂ、任意サンプル画像データ３４ａ’、または、指定領域画像データ３４ｃを読み込み（ステップＳ１０）、読み込んだ判別対象画像データ３４ｂ、任意サンプル画像データ３４ａ’、または、指定領域画像データ３４ｃから第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを抽出する処理を実行する（ステップＳ１２）。 When the classification discrimination routine is executed, first, the feature extractor 26 (trained feature extractor 26) using the trained model 35 performs the discrimination target image data 34b, the arbitrary sample image data 34a', or the designated area. Image data 34c is read (step S10), and a process of extracting the first multidimensional feature vectors 90a, 90b, 90c from the read discrimination target image data 34b, arbitrary sample image data 34a', or designated area image data 34c is executed. (Step S12).
　続いて、学習済モデル３５を利用した特徴変換器２７（学習済の特徴変換器２７）が、特徴抽出器２６が抽出した第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに変換する処理を実行する（ステップＳ１４）。そして、分類器２８が、特徴変換器２７が変換した第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃを用いて、判別対象（判別が必要な分類が未知の対象の画像のみならず、任意サンプル画像、および、ユーザによって指定されたこれらの画像の任意の領域の画像（指定領域画像））を分類する処理を実行し（ステップＳ１６）、当該分類による判別結果（ＯＫ品　ｏｒ　ＮＧ品）を出力して（ステップＳ１８）、本ルーチンを終了する。 Subsequently, the feature converter 27 (trained feature converter 27) using the trained model 35 uses the first multidimensional feature vectors 90a, 90b, 90c extracted by the feature extractor 26 as the second multidimensional feature vector. The process of converting to 92a, 92b, 92c is executed (step S14). Then, the classifier 28 uses the second multidimensional feature vectors 92a, 92b, 92c converted by the feature converter 27 to discriminate not only the image of the target whose classification is unknown but also the arbitrary sample image. , And, the process of classifying the images (designated area images) of arbitrary areas of these images specified by the user is executed (step S16), and the discrimination result (OK product or NG product) according to the classification is output. (Step S18), this routine is terminated.
　ここで、第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃは、ディープメトリックラーニングを用いて予め学習された特徴変換器２７によって、第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに変換されるため、判別対象の分類に有効な特徴を失うことなく低次元化することができる。 Here, the first multidimensional feature vectors 90a, 90b, 90c are converted into the second multidimensional feature vectors 92a, 92b, 92c by the feature converter 27 learned in advance using deep metric learning, and thus discriminated. It is possible to reduce the dimension without losing the features that are effective for classifying the object.
　なお、本実施の形態では、ディープメトリックラーニングを用いた学習済モデル３５（特徴抽出器２６および特徴変換器２７）の学習は、可視化を行って、学習済モデル３５（特徴抽出器２６および特徴変換器２７）の学習状況（個性）を視覚的に確認しながら、当該学習状況（個性）が所望の状態となるまで実施されるため、判別対象の分類に有効な第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに適切に変換することができる。 In the present embodiment, the learning of the trained model 35 (feature extractor 26 and feature converter 27) using deep metric learning is visualized and the trained model 35 (feature extractor 26 and feature conversion) is performed. The second multidimensional feature vector 92a, which is effective for classifying the discrimination target, is performed until the learning situation (individuality) of the vessel 27) is visually confirmed and the learning situation (individuality) becomes a desired state. It can be appropriately converted to 92b and 92c.
　図７（ａ）は、学習済モデル３５が適切な学習状況（個性）にある状態を示す説明図であり、図７（ｂ）は、学習済モデル３５の学習状況（個性）が適切でない状態を示す説明図である。図中、符号「Ｏｇ」は、ＯＫ品の複数の教師信号付きサンプル画像データ３４ａの第２多次元特徴ベクトル９２ａに基づいて、二次元グラフ化データ生成部３０によって生成されたグラフであり、図中、符号「Ｎｇ」は、ＮＧ品の複数の教師信号付きサンプル画像データ３４ａの第２多次元特徴ベクトル９２ａに基づいて、二次元グラフ化データ生成部３０によって生成されたグラフである。図７（ａ）および図７（ｂ）は、学習済モデル３５の学習状況（個性）の評価を行う際に一般的に用いられる正解率や適合率、再現率などといった一次元の数値指標のいずれもが同じ、あるいは、当該数値指標が近い学習済モデル３５について可視化したものである。 FIG. 7A is an explanatory diagram showing a state in which the trained model 35 is in an appropriate learning situation (individuality), and FIG. 7B is a state in which the learning situation (individuality) of the trained model 35 is not appropriate. It is explanatory drawing which shows. In the figure, the reference numeral "Og" is a graph generated by the two-dimensional graphing data generation unit 30 based on the second multidimensional feature vector 92a of the sample image data 34a with a plurality of teacher signals of the OK product. Among them, the symbol "Ng" is a graph generated by the two-dimensional graphing data generation unit 30 based on the second multidimensional feature vector 92a of the sample image data 34a with a plurality of teacher signals of the NG product. 7 (a) and 7 (b) are one-dimensional numerical indexes such as a correct answer rate, a precision rate, and a recall rate that are generally used when evaluating the learning status (individuality) of the trained model 35. All are visualizations of the trained model 35 which is the same or has similar numerical indexes.
　図７に示すように、学習済モデル３５が適切な学習状況（個性）にあると、モデル用二次元グラフＯｇと、モデル用二次元グラフＮｇと、が明確に離隔されて表示され、境界が明確となり、このような学習状況（個性）が適切な学習済モデル３５を利用した特徴変換器２７を用いることにより、判別対象の分類に有効な第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃを適切に得ることができる。 As shown in FIG. 7, when the trained model 35 is in an appropriate learning situation (individuality), the two-dimensional graph Og for the model and the two-dimensional graph Ng for the model are displayed clearly separated, and the boundary is displayed. The second multidimensional feature vectors 92a, 92b, 92c, which are effective for classifying the discrimination target, are appropriate by using the feature converter 27 using the trained model 35, which is clarified and the learning situation (individuality) is appropriate. Can be obtained.
　一方、学習済モデル３５の学習状況（個性）が適切でないと、モデル用二次元グラフＯｇと、モデル用二次元グラフＮｇと、が近接して表示され、境界が不明確となり、このような学習状況（個性）が適切でない学習済モデル３５を利用した特徴変換器２７を用いると、判別対象の分類に有効な第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃを適切に得ることができない。 On the other hand, if the learning situation (individuality) of the trained model 35 is not appropriate, the two-dimensional graph Og for the model and the two-dimensional graph Ng for the model are displayed close to each other, and the boundary becomes unclear. If the feature converter 27 using the trained model 35 whose situation (individuality) is not appropriate is used, the second multidimensional feature vectors 92a, 92b, 92c effective for classifying the discrimination target cannot be appropriately obtained.
　このように、学習済モデル３５の学習状況（個性）を可視化することにより、学習済モデル３５が判別対象の分類を適切に判別できる学習状況（個性）にあるか否かを評価することができ、従来に比べて少量のサンプルデータで適切な学習済モデル３５を生成することができる。そして、適切な学習済モデル３５を生成することができることによって、分類器２８による判別対象の分類に適切な閾値の設定範囲を広くとることができる。これにより、外乱、例えば、判別対象の画像を取得する際の外光の影響などによる誤判別を抑制でき、正確な判別結果を安定して得ることができる。 By visualizing the learning status (individuality) of the trained model 35 in this way, it is possible to evaluate whether or not the trained model 35 is in a learning status (individuality) that can appropriately discriminate the classification of the discrimination target. , An appropriate trained model 35 can be generated with a smaller amount of sample data than in the past. Then, by being able to generate an appropriate trained model 35, it is possible to widen the setting range of the threshold value appropriate for the classification of the discrimination target by the classifier 28. As a result, it is possible to suppress erroneous discrimination due to disturbance, for example, the influence of external light when acquiring an image to be discriminated, and it is possible to stably obtain an accurate discriminant result.
　もとより、本実施の形態では、図８に示すように、モデル用二次元グラフＯｇ，Ｎｇに加えて、判別対象の第２多次元特徴ベクトル９２ｂやユーザによって指定された領域の第２多次元特徴ベクトル９２ｃの状態を判別用二次元グラフＤｇとして表示するため、判別対象や指定された領域の分類の判別状況を可視化することができる。これにより、判別対象の判別状況やユーザ自ら気になる箇所の判別状況を視覚的に確認することができると共に（例えば、判別対象に傷などがあった場合に、当該傷をＮＧとして判別できているのかなど）、当該気になる箇所の判別状況に応じて（ＯＫ品であるのにＮＧであると判別していた場合や、これとは逆に、ＮＧ品であるのにＯＫ品であると判別していた場合）、特徴抽出器２６および特徴変換器２７の学習、あるいは、分類器２８の学習をやり直すことができる。 Of course, in the present embodiment, as shown in FIG. 8, in addition to the two-dimensional graphs Og and Ng for the model, the second multidimensional feature vector 92b to be discriminated and the second multidimensional feature of the region specified by the user. Since the state of the vector 92c is displayed as a two-dimensional graph Dg for discrimination, it is possible to visualize the discrimination status of the discrimination target and the classification of the designated area. As a result, it is possible to visually confirm the discrimination status of the discrimination target and the discrimination status of the portion of concern by the user (for example, when there is a scratch on the discrimination target, the scratch can be discriminated as NG. Depending on the discrimination situation of the part of concern (whether it is OK, etc.), it is determined that it is NG even though it is an OK product, or conversely, it is an OK product even though it is an NG product. The learning of the feature extractor 26 and the feature converter 27, or the learning of the classifier 28 can be redone.
　このような判別対象の分類に有効な特徴を有する第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃを用いて、分類器２８が判別対象の分類を行うため、判別対象の分類を精度良く実現できる。 Since the classifier 28 classifies the discrimination target using the second multidimensional feature vectors 92a, 92b, 92c having features effective for the classification of the discrimination target, the classification of the discrimination target can be realized with high accuracy.
　なお、分類器２８による判別対象の分類は、本実施の形態では、記憶部３３に記憶された複数の第２多次元特徴ベクトル９２ａを用い機械学習した演算パラメータにより、判別対象の第２多次元特徴ベクトル９２ｂ，９２ｃがＯＫ品である確信度を示す一次元の数値を演算し、設定した閾値と比較することにより行う構成とした。 In the present embodiment, the classification of the discrimination target by the classifier 28 is performed by the second multidimensional of the discrimination target based on the calculation parameters machine-learned using the plurality of second multidimensional feature vectors 92a stored in the storage unit 33. A one-dimensional numerical value indicating the certainty that the feature vectors 92b and 92c are OK products is calculated and compared with the set threshold value.
　また、本実施の形態では、学習済モデル３５の学習状況（個性）を可視化すると共に、特徴抽出器２６および特徴変換器２７と、分類器２８と、を別構成として有するため、判別対象の分類の判定において誤判定した場合に、当該誤判定の原因を簡易に特定することができると共に、その対処を適切に行うことができる。 Further, in the present embodiment, since the learning situation (individuality) of the trained model 35 is visualized and the feature extractor 26, the feature converter 27, and the classifier 28 are provided as separate configurations, the classification of the discrimination target is performed. When an erroneous determination is made in the determination of the above, the cause of the erroneous determination can be easily identified, and the countermeasure can be appropriately taken.
　例えば、図９（ａ）に示すように、判別用二次元グラフＤｇや選択二次元グラフＣｇの形状がモデル用二次元グラフＮｇと離れており、異なる形状にも関わらず、ＮＧ品と判断された場合や、図９（ｂ）に示すように、判別用二次元グラフＤｇや選択二次元グラフＣｇの形状がモデル用二次元グラフＯｇと離れており、異なる形状にも関わらず、ＯＫ品と判断された場合、即ち、ＯＫ品およびＮＧ品の特徴は捉えることができているが、ＯＫ品およびＮＧ品の分類を誤っているような場合には、分類器２８の演算パラメータの機械学習や閾値の設定が適切でないと判断することができ、この場合、分類器２８の演算パラメータの機械学習や閾値の設定をし直すことで対応できる。これにより、簡単かつ迅速に判定装置の判定精度を向上することができ、かつ、学習済モデル３５の信頼性の向上を図ることができる。なお、判別用二次元グラフＤｇは、判別対象画像データ３４ｂや指定領域画像データ３４ｃの第２多次元特徴ベクトル９２ｂ，９２ｃに基づいて、二次元グラフ化データ生成部３０によって生成されるグラフである。また、選択二次元グラフＣｇは、選択二次元グラフ化データ３６ａ’に基づいて、二次元グラフ化データ生成部３０によって生成されるグラフである。 For example, as shown in FIG. 9A, the shapes of the discriminant two-dimensional graph Dg and the selected two-dimensional graph Cg are separated from the model two-dimensional graph Ng, and it is determined that the product is NG despite the different shapes. In this case, or as shown in FIG. 9B, the shapes of the discriminant two-dimensional graph Dg and the selected two-dimensional graph Cg are different from the model two-dimensional graph Og, and despite the different shapes, the product is OK. If it is judged, that is, if the characteristics of the OK and NG products can be grasped, but the classification of the OK and NG products is incorrect, machine learning of the arithmetic parameters of the classifier 28 or It can be determined that the setting of the threshold value is not appropriate, and in this case, it can be dealt with by machine learning the calculation parameter of the classifier 28 or resetting the setting of the threshold value. As a result, the determination accuracy of the determination device can be easily and quickly improved, and the reliability of the trained model 35 can be improved. The two-dimensional graph Dg for discrimination is a graph generated by the two-dimensional graphing data generation unit 30 based on the second multidimensional feature vectors 92b and 92c of the discrimination target image data 34b and the designated area image data 34c. .. The selected two-dimensional graph Cg is a graph generated by the two-dimensional graphing data generation unit 30 based on the selected two-dimensional graphing data 36a'.
　以上説明した本実施の形態に係る判別装置によれば、学習済の特徴抽出器２６を用いて複数の教師信号付きサンプル画像データ３４ａや、判別対象画像データ３４ｂ、指定領域画像データ３４ｃから第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを抽出し、抽出した第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを、学習済の特徴変換器２７を用いて当該第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃよりも低次元かつ判別対象の分類に有効な第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに変換し、変換した第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに基づいて判別対象の分類を行うため、判別対象の分類を精度良く実現できる。 According to the discrimination device according to the present embodiment described above, the first sample image data 34a with a plurality of teacher signals, the discrimination target image data 34b, and the designated area image data 34c are used by using the trained feature extractor 26. The multidimensional feature vectors 90a, 90b, 90c are extracted, and the extracted first multidimensional feature vectors 90a, 90b, 90c are used with the trained feature converter 27 to obtain the first multidimensional feature vectors 90a, 90b, 90c. In order to convert to the second multidimensional feature vectors 92a, 92b, 92c, which are lower in dimension and effective for classifying the discrimination target, and to classify the discrimination target based on the converted second multidimensional feature vectors 92a, 92b, 92c. , The classification of the discrimination target can be realized with high accuracy.
　また、本実施の形態に係る判別装置によれば、判別対象である複数の教師信号付きサンプル画像データ３４ａや、判別対象画像データ３４ｂ、指定領域画像データ３４ｃが有する多様な特徴量である第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを抽出する特徴抽出器２６および当該第１多次元特徴ベクトル９０ａ，９０ｂ，９０ｃを低次元の第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに特徴変換器２７と、当該特徴変換器２７によって変換された第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃに基づき判別対象を分類する分類器２８と、が別構成であるため、判別対象の分類精度の向上に際し、特徴抽出器２６および特徴変換器２７の学習と、分類器の学習と、を分けて実施することができる。即ち、分類精度の低さの原因が、特徴抽出器２６および特徴変換器２７による特徴抽出・変換精度の低さに起因する場合には、特徴抽出器２６および特徴変換器２７を学習させ、分類器２８による分類精度の低さに起因する場合には分類器２８のみを学習させることができ、分類精度の向上を効率良く行うことができる。 Further, according to the discrimination device according to the present embodiment, the first sample image data 34a with a plurality of teacher signals to be discriminated, the discriminant target image data 34b, and the designated area image data 34c have various feature quantities. The feature extractor 26 for extracting the multidimensional feature vectors 90a, 90b, 90c and the first multidimensional feature vectors 90a, 90b, 90c are converted into the low-dimensional second multidimensional feature vectors 92a, 92b, 92c with the feature converter 27. Since the classifier 28 that classifies the discrimination target based on the second multidimensional feature vectors 92a, 92b, 92c converted by the feature converter 27 has a different configuration, the feature is improved in improving the classification accuracy of the discrimination target. Learning of the extractor 26 and the feature converter 27 and learning of the classifier can be carried out separately. That is, when the cause of the low classification accuracy is the low feature extraction / conversion accuracy by the feature extractor 26 and the feature converter 27, the feature extractor 26 and the feature converter 27 are trained and classified. When it is caused by the low classification accuracy of the device 28, only the classifier 28 can be trained, and the classification accuracy can be efficiently improved.
　本実施の形態では、判別装置が、判別対象がＯＫ品であるかＮＧ品であるかの判別を行う装置として説明したが、これに限らない。例えば、判別装置が、判別対象を複数の分類のうちのいずれに分類されるかの判別を行う装置に適用しても良い。 In the present embodiment, the discriminating device has been described as a device that discriminates whether the discriminating target is an OK product or an NG product, but the present invention is not limited to this. For example, the discriminating device may be applied to a device that discriminates which of a plurality of classifications the discriminating target is classified into.
　本実施の形態では、第２多次元特徴ベクトル９２ａ，９２ｂ，９２ｃのグラフ化として、縦軸に列番号１，２，３，・・・，ｎ－１，ｎをとり、横軸に特徴量ｆ１，ｆ２，ｆ３，・・・，ｆｎ－１，ｆｎをとった座標系（直交座標系）に、これら特徴量ｆ１，ｆ２，ｆ３，・・・，ｆｎ－１，ｆｎをプロットして曲線ないし直線で繋いだ二次元グラフを用いたが、これに限らない。 In the present embodiment, as a graph of the second multidimensional feature vectors 92a, 92b, 92c, the vertical axis has column numbers 1, 2, 3, ..., N-1, n, and the horizontal axis is the feature quantity. In a coordinate system (Cartesian coordinate system) that takes f 1 , f 2 , f 3 , ..., f n-1 , f n , these feature quantities f 1 , f 2 , f 3 , ..., f n A two-dimensional graph in which -1 and f n are plotted and connected by a curve or a straight line is used, but the present invention is not limited to this.
　本実施の形態では、入力装置１４は、キーボードおよびマウスとして説明したが、タッチパネル等のポインティングデバイスや、ボタン、ダイヤル、タッチセンサ、タッチパッド等を含んでいても良い。 In the present embodiment, the input device 14 has been described as a keyboard and a mouse, but may include a pointing device such as a touch panel, a button, a dial, a touch sensor, a touch pad, and the like.
　本実施の形態では、画像データを含む各種データを記憶する大容量メモリをＨＤＤ１０としたが、これに限らない。画像データを含む各種データを記憶する大容量メモリとして、フラッシュメモリ（ＵＳＢメモリ、ＳＤカードなど）や、ＳＳＤ、フロッピーディスク、ＣＤ、ＤＶＤなどを適用しても良い。 In the present embodiment, the large-capacity memory for storing various data including image data is HDD 10, but the present invention is not limited to this. As a large-capacity memory for storing various data including image data, a flash memory (USB memory, SD card, etc.), SSD, floppy disk, CD, DVD, or the like may be applied.
　本実施の形態では、可視化プログラムを含む各種アプリケーションをＨＤＤ１０に記憶する構成としたが、これに限らない。例えば、可視化プログラムを含む各種アプリケーションは、伝送媒体、例えば、インターネットやＬＡＮなどの通信網を介して他コンピュータから本実施の形態に係る判別装置としてのコンピュータ１へ配信される構成としても良い。 In the present embodiment, various applications including a visualization program are stored in the HDD 10, but the present invention is not limited to this. For example, various applications including a visualization program may be configured to be distributed from another computer to the computer 1 as a discrimination device according to the present embodiment via a transmission medium, for example, a communication network such as the Internet or LAN.
　本実施形態は、本発明を実施するための形態の一例を示すものである。したがって、本発明は、本実施形態の構成に限定されるものではない。 The present embodiment shows an example of a mode for carrying out the present invention. Therefore, the present invention is not limited to the configuration of the present embodiment.
１　　　　　　コンピュータ１（判別装置）
２　　　　　　ＣＰＵ
４　　　　　　ＲＯＭ
６　　　　　　ＲＡＭ
８　　　　　　ＧＰＵ
１０　　　　　ＨＤＤ
１２　　　　　入出力インターフェイス
１４　　　　　入力装置
２０　　　　　画像表示制御部
２２　　　　　画像取得部
２４　　　　　領域指定部
２６　　　　　特徴抽出器（特徴抽出器）
２７　　　　　特徴変換器（特徴変換器）
２７ａ　　　　全結合層（複数の全結合層）
２７ｂ　　　　全結合層（複数の全結合層）
２８　　　　　分類器（分類器）
２９　　　　　距離算定器
３０　　　　　二次元グラフ化データ生成部
３３　　　　　記憶部
３４ａ　　　　教師信号付きサンプル画像データ
３４ｂ　　　　判別対象画像データ
３４ｃ　　　　指定領域画像データ
３５　　　　　学習済モデル（学習済モデル）
３６ａ　　　　モデル用二次元グラフ化データ
３６ｂ　　　　判別用二次元グラフ化データ
３８　　　　　距離データ
６０　　　　　ディスプレイ
６２　　　　　ウィンドウ
８０　　　　　バス
８２　　　　　バスライン
９０ａ　　　　第１多次元特徴ベクトル（第１多次元特徴ベクトル）
９０ｂ　　　　第１多次元特徴ベクトル（第１多次元特徴ベクトル）
９０ｃ　　　　第１多次元特徴ベクトル（第１多次元特徴ベクトル）
９２ａ　　　　第２多次元特徴ベクトル（第２多次元特徴ベクトル）
９２ａ’　　　任意の第２多次元特徴ベクトル（第２多次元特徴ベクトル）
９２ｂ　　　　第２多次元特徴ベクトル（第２多次元特徴ベクトル）
９２ｃ　　　　第２多次元特徴ベクトル（第２多次元特徴ベクトル）
ｆｎ　　　　　特徴量（特徴量）
ｎ　　　　　　列番号
Ｏｇ　　　　　ＯＫ品のモデル用二次元グラフ
Ｎｇ　　　　　ＮＧ品のモデル用二次元グラフ
Ｄｇ　　　　　判別用二次元グラフ
Ｃｇ　　　　　選択二次元グラフ
  1 Computer 1 (discrimination device)
2 CPU
4 ROM
6 RAM
8 GPU
10 HDD
12 Input / output interface 14 Input device 20 Image display control unit 22 Image acquisition unit 24 Area designation unit 26 Feature extractor (feature extractor)
27 Feature converter (feature converter)
27a Fully bonded layer (multiple fully bonded layers)
27b Fully bonded layer (multiple fully bonded layers)
28 Classifier (classifier)
29 Distance calculator 30 Two-dimensional graphed data generation unit 33 Storage unit 34a Sample image data with teacher signal 34b Discrimination target image data 34c Designated area image data 35 Trained model (trained model)
36a Two-dimensional graphed data for model 36b Two-dimensional graphed data for discrimination 38 Distance data 60 Display 62 Window 80 Bus 82 Bus line 90a First multidimensional feature vector (first multidimensional feature vector)
90b 1st multidimensional feature vector (1st multidimensional feature vector)
90c 1st multidimensional feature vector (1st multidimensional feature vector)
92a Second multidimensional feature vector (second multidimensional feature vector)
92a'Any second multidimensional feature vector (second multidimensional feature vector)
92b Second multidimensional feature vector (second multidimensional feature vector)
92c 2nd multidimensional feature vector (2nd multidimensional feature vector)
fn feature amount (feature amount)
n Column number Og Two-dimensional graph for OK product model Ng Two-dimensional graph for NG product model Dg Two-dimensional graph for discrimination Cg Selected two-dimensional graph

Claims (4)

  1. 　（ａ）判別対象のデータを取得し、
    　（ｂ）深層学習を含むニューラルネットワークを用いた特徴抽出器によって、前記判別対象のデータから第１多次元特徴ベクトルを抽出し、
    　（ｃ）抽出した前記第１多次元特徴ベクトルを、複数の全結合層を用いた特徴変換器によって、前記第１多次元特徴ベクトルよりも次元の低い第２多次元特徴ベクトルに変換し、
    　（ｄ）変換された前記第２多次元特徴ベクトルに基づいて、統計的機械学習を用いた分類器によって前記判別対象を分類する
    　判別対象の分類方法。     (A) Acquire the data to be discriminated and
    (B) A first multidimensional feature vector is extracted from the data to be discriminated by a feature extractor using a neural network including deep learning.
    (C) The extracted first multidimensional feature vector is converted into a second multidimensional feature vector having a lower dimension than the first multidimensional feature vector by a feature converter using a plurality of fully connected layers.
    (D) A method for classifying the discriminant target by a classifier using statistical machine learning based on the converted second multidimensional feature vector.
  2. 　判別対象を分類するためのプログラムであって、
    請求項１に記載の判別対象の分類方法の各ステップを１又は複数のコンピュータに実行させるためのプログラム。     It is a program for classifying the discrimination target,
    A program for causing one or more computers to execute each step of the classification method of the discrimination target according to claim 1.
  3. 　判別対象のデータを取得するデータ取得部と、
    　取得した前記判別対象のデータから第１多次元特徴ベクトルを抽出する深層学習を含むニューラルネットワークを用いた特徴抽出器と、
    　該特徴抽出器により抽出された前記第１多次元特徴ベクトルを、該第１多次元特徴ベクトルよりも低次元の第２多次元特徴ベクトルに変換する複数の全結合層を用いた特徴変換器と、
    　前記第２多次元特徴ベクトルを用いて前記判別対象を分類する統計的機械学習を用いた分類器と、
    　を備える判別装置。     The data acquisition unit that acquires the data to be discriminated, and
    A feature extractor using a neural network including deep learning that extracts the first multidimensional feature vector from the acquired data to be discriminated, and
    A feature converter using a plurality of fully connected layers that converts the first multidimensional feature vector extracted by the feature extractor into a second multidimensional feature vector having a lower dimension than the first multidimensional feature vector. ,
    A classifier using statistical machine learning that classifies the discrimination target using the second multidimensional feature vector, and
    A discriminating device comprising.
  4. 　第２多次元特徴ベクトルを記憶する記憶部と、
    　該記憶部に記憶された複数の第２多次元特徴ベクトルについて各々の間の距離値を算定する算定部と、
    をさらに備えており、
    　前記特徴抽出器および前記特徴変換器は、学習段階においては、前記算定部により算定された前記距離値に基づき誤差逆伝播法および勾配降下法を用いて学習する
    　請求項３に記載の判別装置。
     
          A storage unit that stores the second multidimensional feature vector,
    A calculation unit that calculates the distance value between each of the plurality of second multidimensional feature vectors stored in the storage unit, and a calculation unit.
    Is further equipped with
    The discrimination device according to claim 3, wherein the feature extractor and the feature converter learn by using an error backpropagation method and a gradient descent method based on the distance value calculated by the calculation unit in the learning stage.
PCT/JP2021/014130 2020-04-17 2021-04-01 Discrimination target classification method, program employed in same, and discriminating device WO2021210413A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2022515303A JPWO2021210413A1 (en) 2020-04-17 2021-04-01

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-073809 2020-04-17
JP2020073809 2020-04-17

Publications (1)

Publication Number Publication Date
WO2021210413A1 true WO2021210413A1 (en) 2021-10-21

Family

ID=78084280

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/014130 WO2021210413A1 (en) 2020-04-17 2021-04-01 Discrimination target classification method, program employed in same, and discriminating device

Country Status (2)

Country Link
JP (1) JPWO2021210413A1 (en)
WO (1) WO2021210413A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014203135A (en) * 2013-04-01 2014-10-27 キヤノン株式会社 Signal processor, signal processing method, and signal processing system
WO2018170421A1 (en) * 2017-03-17 2018-09-20 Magic Leap, Inc. Room layout estimation methods and techniques
JP2019507935A (en) * 2016-03-11 2019-03-22 エヌイーシー ラボラトリーズ アメリカ インクＮＥＣ Ｌａｂｏｒａｔｏｒｉｅｓ Ａｍｅｒｉｃａ， Ｉｎｃ． Deep deformation network for object landmark detection
JP2020060879A (en) * 2018-10-05 2020-04-16 オムロン株式会社 Learning device, image generator, method for learning, and learning program

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014203135A (en) * 2013-04-01 2014-10-27 キヤノン株式会社 Signal processor, signal processing method, and signal processing system
JP2019507935A (en) * 2016-03-11 2019-03-22 エヌイーシー ラボラトリーズ アメリカ インクＮＥＣ Ｌａｂｏｒａｔｏｒｉｅｓ Ａｍｅｒｉｃａ， Ｉｎｃ． Deep deformation network for object landmark detection
WO2018170421A1 (en) * 2017-03-17 2018-09-20 Magic Leap, Inc. Room layout estimation methods and techniques
JP2020060879A (en) * 2018-10-05 2020-04-16 オムロン株式会社 Learning device, image generator, method for learning, and learning program

Also Published As

Publication number Publication date
JPWO2021210413A1 (en) 2021-10-21

Similar Documents

Publication Publication Date Title
Krause et al. Using visual analytics to interpret predictive machine learning models
JP6547275B2 (en) INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, AND PROGRAM
US20020165839A1 (en) Segmentation and construction of segmentation classifiers
US20200103353A1 (en) Information processing apparatus, information processing method, and storage medium
JP7213701B2 (en) Machining equipment, machining method, machining program, and inspection equipment
JP2020042519A (en) Abnormality detection device, abnormality detection method, and abnormality detection program
JP6950504B2 (en) Abnormal candidate extraction program, abnormal candidate extraction method and abnormal candidate extraction device
Nair et al. A life cycle on processing large dataset-LCPL
JP6602013B2 (en) Information processing apparatus, information processing method, and program
JP2023139296A (en) Signal processing method, signal processing apparatus, and signal processing program
WO2021210413A1 (en) Discrimination target classification method, program employed in same, and discriminating device
JP2020047010A (en) Information estimation device and information estimation method
EP3923228B1 (en) Data analysis method, device and program
WO2021210414A1 (en) Visualization method, program using same, visualization device, and discrimination device provided with same
Hu et al. Shape-driven coordinate ordering for star glyph sets via reinforcement learning
WO2020167156A1 (en) Method for debugging a trained recurrent neural network
Chinnasamy et al. An outlier based bi-level neural network classification system for improved classification of cardiotocogram data
Schwenke et al. Abstracting Local Transformer Attention for Enhancing Interpretability on Time Series Data.
Neshatian et al. Genetic programming for performance improvement and dimensionality reduction of classification problems
JP2018151913A (en) Information processing system, information processing method, and program
Aazi et al. Feature selection for multiclass support vector machines
WO2022172445A1 (en) Information processing device, information processing method and information processing program
KR102236802B1 (en) Device and method for feature extraction of data for diagnostic models
WO2024004127A1 (en) Information processing system
Nanaware et al. Exploratory data analysis using dimension reduction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21789349

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022515303

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21789349

Country of ref document: EP

Kind code of ref document: A1