JP2009250722A - Defect learning apparatus, defect learning method and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the determination performance of defects more than before, regarding a defect learning apparatus, defect learning method and computer program. <P>SOLUTION: As for feature quantities f1-f50 of a first group, a learning model A is formed, by generating a plurality of partial feature quantity spaces for associating the group of values of the feature quantities f1-f50 of the first group with defect types "C1" and "C2". Next, the learning model A is applied to each of the data of feature quantities f51-f100 of a second group. As a result, out of the data of the feature quantities f51-f100 of the second group, the defect data, for which the learning model A cannot specify the defect type "C1" or "C2", are defined as discard class data. In the feature quantities for which the learning model A cannot specify the defect type "C1" or "C2", a learning model B is formed, by generating a plurality of partial feature quantity spaces for associating the group of the values of the feature quantity with the defect types "C1" and "C2". <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a wrinkle learning device, a wrinkle learning method, and a computer program, and is particularly suitable for imaging a wrinkle inspection object and performing a wrinkle inspection.

For example, in production lines such as steel plates, harmful wrinkles may occur in the material to be rolled. For this reason, the image obtained by detecting and imaging the reflected light obtained by illuminating the surface of the material to be rolled is processed to extract the feature amount for the image of the cocoon candidate, and based on the extracted feature amount Inspecting the presence and type of harmful flaws occurring in the material to be rolled is performed. Features include dimensional features such as heel length, width, and area, geometric features such as roundness and aspect ratio (aspect ratio), and average and minimum image density of the heel. There are light and dark feature values such as value and maximum value. By extracting such feature amounts from the image, it is possible to determine the type, grade (score), and the like of the bag.
In Patent Document 1, a feature classification extracted as described above is applied to a shape classification by applying a recognition method using a neural network, and an authentication method using a decision tree is further applied for each feature. The classification is described.

Japanese Patent Laid-Open No. 5-280959

However, in the technique described in Patent Document 1, the threshold value set in advance in each node of the decision tree (tree) is compared with the feature amount of the input image, and the course is selected based on the result. When we reach the end of the tree, we are going to categorize it into the corresponding category.
As described above, in the technique described in Patent Document 1, the wrinkles represented in the image are classified into one of the classes according to the determination condition predetermined in the tree. Therefore, if the discrimination conditions are set finely in order to improve the discrimination performance of wrinkles, it takes time to process, and if the discrimination conditions are set coarsely, the discrimination performance of wrinkles will deteriorate, but there are various forms of hail Therefore, it is difficult to set the discrimination condition appropriately. Therefore, the technique described in Patent Document 1 has a problem that it is difficult to improve the discrimination performance of wrinkles existing in various forms.

  The present invention has been made in view of such problems, and an object thereof is to improve the discrimination performance of wrinkles as compared with the conventional art.

  The wrinkle learning apparatus according to the present invention includes wrinkle data that includes wrinkle-related feature amounts from an image of an inspection object, and wrinkle data acquired by the acquisition means, wherein And a first determination unit that determines a combination of feature value values that can separate the plurality of species, and a feature amount determined to be capable of separating the plurality of species by the first determination unit. First generation means for generating a learning model from a partial feature amount space for associating a set of values with the plurality of soot seeds, and soot data obtained by the obtaining means, which has already been used Extraction means for arranging bag data different from the data in a plurality of partial feature amount spaces generated by the first generation unit, and extracting data of a kind corresponding to the bag data from each partial feature amount space; In the extraction means The ratio of the number of specific types of data extracted from each partial feature amount space by the extraction means to the total number of types of data extracted from each partial feature amount space is compared with a threshold, Corresponding to the data of the species extracted by the extraction unit when the determination unit determines whether or not the class related to the species can be specified, and the determination unit determines that the class cannot be specified Second determination means for determining a combination of feature value values capable of separating the plurality of soot species, using the soot data in the plurality of soot types as rejection class data, A second generation unit that generates a learning model from a partial feature amount space for associating a set of feature value values determined to be able to separate a plurality of species with the plurality of species; The Characterized in that it.

  The wrinkle learning method of the present invention includes an acquisition step of acquiring wrinkle data including a feature amount related to a wrinkle from an image of an inspection object, and wrinkle data acquired by the acquisition step, wherein the wrinkle data in a plurality of types Using the first determination step of determining a combination of feature value values that can separate the plurality of species, and the feature amount determined to be capable of separating the plurality of species by the first determination step. A first generation step for generating a learning model from a partial feature space for associating a set of values with the plurality of types, and the selection data acquired by the acquisition step, The soot data different from the data is arranged in the plurality of partial feature amount spaces generated by the first generation step, and the soot type data corresponding to the soot data is extracted from each partial feature amount space. A ratio of the number of specific types of data extracted from each partial feature amount space by the extraction step to the total number of types of data extracted from each partial feature amount space by the extraction step; A determination step for comparing the threshold value to determine whether or not the class relating to the species can be specified, and if it is determined by the determination step that the class cannot be specified, the extraction step extracts A second data for identifying a combination of feature value values that can separate the plurality of species using the species data of the plurality of species as the rejection class data. A set of feature values determined to be capable of separating a plurality of species by the discrimination step and the second discrimination step are associated with the plurality of species. Wherein the the partial feature space and a second generation step of generating a learning model.

  The computer program of the present invention includes an acquisition step of acquiring wrinkle data including a feature amount related to wrinkles from an image of an inspection object, and wrinkle data acquired by the acquisition step, wherein wrinkle data in a plurality of types And a first determination step for determining a combination of feature value values capable of separating the plurality of species, and a feature value determined to be capable of separating the plurality of species by the first determination step. A first generation step of generating a learning model from a partial feature amount space for associating a set of a plurality of species with the plurality of species, and the cocoon data acquired by the acquisition step,疵 data different from the 疵 data are arranged in the plurality of partial feature amount spaces generated by the first generation step, and the varieties of data corresponding to the 疵 data from each partial feature amount space are arranged. The number of specific types of data extracted from each partial feature amount space by the extraction step with respect to the total number of species data extracted from each partial feature amount space by the extraction step. A step of determining whether or not a class relating to the species can be specified by comparing the ratio of the ratio and a threshold value, and if it is determined that the class cannot be specified by the determination step, the extraction疵 data corresponding to the species data extracted in the step, and using the 疵 data in multiple species as rejection class data, discriminating combinations of feature values that can separate the multiple species A set of feature values determined to be able to separate a plurality of species by the second discrimination step, and the plurality of species Characterized in that to execute a second generation step of generating a learning model from the partial feature space for attaching respond to computer.

  According to the present invention, cocoon data including a feature quantity related to cocoon is arranged in a partial feature quantity space (learning model), and cocoon data corresponding to the cocoon data is extracted from each of a plurality of partial feature quantity spaces. Then, the ratio of the number of the extracted specific species data to the total number of the extracted species data is compared with the threshold value to determine whether or not the class relating to the species can be identified. If it is determined that the class cannot be specified as a result of this determination, the defect data corresponding to the extracted defect data is set as the rejection class data, and the partial feature space (learning model) is used by using the defect data. Is generated. Therefore, the learning model can be strengthened, and the wrinkle discrimination performance can be improved as compared with the prior art.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating an example of a schematic configuration of a surface defect inspection system. In addition, in this embodiment, as shown in FIG. 1, after finishing rolling by the finishing mill 7 provided with the work roll 7a, the intermediate roll 7b, and the backup roll 7c as a wrinkle inspection object which performs a wrinkle inspection. The strip steel plate 1 will be described as an example. Moreover, in the following description, the strip steel plate 1 is abbreviated as a steel plate 1.

In FIG. 1, the steel plate 1 is conveyed (moved) in the longitudinal direction of the steel plate 1 (the direction of the arrow in FIG. 1) by the conveying rolls 2a to 2c. Further, the rotation of the transport rolls 2a to 2c is detected by the rotary encoder 3, respectively.
The surface wrinkle inspection system installed in such a rolled steel sheet production line includes a lighting device 4, an imaging device 5, a wrinkle determination device 6, and a wrinkle learning device 9.

  The illuminating device 4 includes, for example, a light emitting unit that emits light having high luminance, such as a halogen lamp, and a condensing unit such as a condensing lens. The light emitted from the halogen lamp or the like is condensed linearly by the condensing unit, so that the surface of the steel plate 1 is illuminated linearly in the width direction. In the present embodiment, the illuminating device 4 irradiates a linear light beam longer than the length in the width direction of the steel plate 1 so that the entire width direction of the steel plate 1 is surely illuminated in a linear shape. Yes.

The imaging device 5 has a function of capturing an image inspection range of the steel plate 1 illuminated by the illumination device 4 and generating an image signal. As shown in FIG. 1, the imaging device 5 is disposed at a position facing the illumination device 4 across the wrinkle inspection range of the steel plate 1, that is, on an optical path of light irradiated from the illumination device 4 and reflected by the steel plate 1. The The imaging device 5 may be either an area camera (sensor) that reads an image in frame units or a line camera (sensor) that reads an image in line units, or a camera such as a CCD image sensor or a CMOS image sensor. But it ’s okay. The captured image may be either a black and white gray image or a color image.
Here, the case where a line camera that outputs a monochrome grayscale image is used as the imaging device 5 will be described as an example. Specifically, the imaging device 5 captures a grayscale image of a wrinkle inspection range that is elongated in the width direction of the steel plate 1, and outputs, for example, black and white 256 gradation information as a captured image.

  The rotary encoder 3 outputs a control timing signal (pulse signal) according to the conveying speed of the steel plate 1 so that the imaging device 5 can continuously capture the surface of the moving steel plate 1 with a predetermined spatial resolution without any gap. 5 is output. The imaging device 5 repeats imaging and transfer of the captured image according to the control timing signal output from the rotary encoder 3.

  The wrinkle determination device 6 and the wrinkle learning device 9 are, for example, PCs (Personal Computers). The eyelid determination device 6 and the eyelid learning device 9 include a CPU, ROM, RAM, hard disk, user interface including a keyboard and a mouse, a data input / output control device, and the like. The CPU, ROM, RAM, hard disk, user interface, and data input / output control device are each connected to a communication bus and can communicate with each other. In this embodiment, it is assumed that wrinkle determination device 6 and wrinkle learning device 9 are connected to each other so as to be communicable via, for example, a LAN (Local Area Network).

  The hard disk, which is one of the storage media of the eyelid determination device 6, is used in an inspection application program for determining the type of eyelids (an eyelid inspection) and the application program for the inspection as will be described later. Data is stored. In addition, the hard disk of the eyelid determination device 6 also stores image data captured by the imaging device 5 and processed by the data input / output control device. Here, the data input / output control device performs processing for configuring the image data for each line output from the imaging device 5 into, for example, image data having the number of pixels of 2048 (width) × 512 (length). , Transfer to RAM. Here, for convenience of explanation, one pixel represents a captured image of a 1 mm × 1 mm region on the surface of the steel plate 1. That is, an image composed of 2048 (width) × 512 (length) pixels represents a captured image having a width of 2048 mm and a length of 512 mm of the steel plate 1.

  On the hard disk, which is one of the storage media of the heel learning device 9, a heel learning application program for generating and updating a discrimination model (learning model) for performing the heel inspection by the heel determination device 6, and its heel learning Data used in the application program is stored. In addition, the hard disk of the wrinkle learning device 9 also stores “haze data including one or a plurality of feature amounts related to wrinkles” obtained by the wrinkle determination device 6 as described later. The wrinkle determination device 6 inputs the latest learning model obtained by the wrinkle learning device 9 and performs a wrinkle inspection according to the learning model. As described above, as the feature amount, for example, a dimensional feature amount such as the length, width, and area of the heel, a shape feature amount such as a circularity, an aspect ratio (aspect ratio), and the like, There are shaded features such as the average value, minimum value, and maximum value of image density.

The CPU and the like of the wrinkle determination device 6 and the wrinkle learning device 9 activate the flaw detection application program and the wrinkle learning application program based on the operation of the user interface by the user. Then, the CPU or the like executes an inspection application program and an eyelid learning application program to execute processes for performing eyelid inspection and learning model generation / update.
The display devices 8 and 10 include, for example, an LCD (Liquid Crystal Display) and the like, and are for displaying images based on the processes executed by the eyelid determination device 6 and the eyelid learning device 9.
Although FIG. 1 shows the case where only one surface of the steel sheet 1 is subjected to the defect inspection, in practice, the other surface of the steel sheet 1 is also subjected to the defect inspection.

FIG. 2 is a block diagram illustrating an example of a functional configuration of the wrinkle determination device 6.
In FIG. 2, the image input unit 21 inputs image data for each line imaged by the imaging device 5, and uses, for example, image data (2048 × 1 pixel) for a plurality of lines continuously input, for example, Image data having the number of pixels of 2048 (width) × 512 (length) is configured. Then, the image input unit 21 stores the image data in the RAM.
The image input unit 21 is realized by using, for example, a data input / output control device included in the eyelid determination device 6.

The shading correction unit 22 performs shading correction on the image data so that the entire image data obtained by the image input unit 21 has uniform brightness. Then, the shading correction unit 22 stores the image data subjected to the shading correction in a RAM (storage medium).
The shading correction unit 22 is realized by using, for example, the CPU, ROM, RAM, and hard disk of the information processing apparatus 6.

The binarization unit 23 performs binarization processing on the image data that has undergone shading correction. Specifically, for example, the binarization unit 29 determines whether or not the value of each pixel of the image data subjected to the shading correction is larger than a threshold set in advance in the hard disk or the like. Then, the value of the pixel larger than the threshold is set to “1”, the value of the other pixel is set to “0”, and a binary image having only “0” or “1” as the pixel value (luminance value) is generated. .
The binarization part 23 is implement | achieved by using CPU, ROM, RAM, and a hard disk of the wrinkle determination apparatus 6, for example.

The labeling / feature amount calculation unit 24 performs a labeling process and a feature amount calculation on the binary image obtained by the binarization unit 23, and extracts a wrinkle candidate. Here, the labeling process is a process performed before the feature amount calculation, and for each figure (connected figure) that is a single figure or geometrically connected in the binary image to form one lump. This is a process of assigning different numbers (labels).
The feature amount calculation (feature extraction) is a process for obtaining a feature amount related to the shape of a figure. The feature quantity obtained by this feature quantity calculation is, for example, “width / length / length / width ratio (aspect ratio) / area / circularity / maximum length / perimeter” of the rectangle circumscribing the candidate Candidate luminance distribution (average value, minimum value, maximum value, etc. of luminance) ”or the like is a combination of one or two or more selected. The labeling / feature quantity calculation unit 24 uses the soot data including the “one or more feature quantities related to the soot” thus obtained as the “position on the steel plate 1” where the soot having the feature quantity exists. The data is stored in the bag data storage unit 25 in association with the data (that is, linked).
The labeling / feature amount calculation unit 24 is realized by using, for example, the CPU, ROM, RAM, and hard disk of the wrinkle determination device 6. Moreover, the bag data storage part 25 is implement | achieved by using the hard disk of the bag determination apparatus 6, for example.
As long as the feature amount can be obtained, the contents of the process for obtaining the feature amount are not limited to those described above.

As will be described later, when the learning model 9 generates a learning model, the learning model 9 transmits the learning model to the learning apparatus 6. When the learning model acquisition unit 26 of the heel determination device 6 receives the learning model from the heel learning device 9, the learning model acquisition unit 26 outputs the learning model to the heel determination unit 27.
The learning model acquisition unit 26 is realized by using, for example, the data input / output control device of the eyelid determination device 6.

The wrinkle determination unit 27 applies the “latest learning model” output from the learning model acquisition unit 26 to the wrinkle candidate wrinkle data stored in the wrinkle data storage unit 25, and finally Make a correct habit determination. For example, a roll that affects the quality of the product is determined to be harmful. In addition, the type of cocoon and the grade (grade) of the cocoon are also determined as the cocoon type. On the other hand, if oil, water droplets, light stains, etc. are attached to the surface of the steel sheet and do not affect the quality of the product, it is determined to be harmless. Then, the eyelid determination unit 27 associates the result of the eyelid inspection with the eyelid data stored in the eyelid data storage unit 25.
The details of the learning model will be described together with the description of the eyelid learning device 9.

As described above, when the eyelid inspection is performed on the image data in a predetermined range, the eyelid determination unit 27 determines the result of the eyelid inspection (the determination result of harmful or harmless, ) And the feature amount obtained by the labeling / feature amount calculation unit 24 are generated and displayed on the display device 8. In the present embodiment, the wrinkle determination unit 27 generates wrinkle detection image data (map) in which an image simulating a heel is superimposed on an image simulating the steel plate 1. Thereby, an inspector can be made to recognize visually what kind of wrinkle has occurred in which place of the steel plate 1.
The wrinkle determination unit 27 is realized by using, for example, the CPU, ROM, RAM, and hard disk of the wrinkle determination device 6.

The iron data output determination unit 28 is stored in the iron data storage unit 25 based on the operation state and date / time of the rolled steel plate production line input from a process computer or the like that controls and monitors the rolled steel plate production line. It is determined whether or not it is time to output the heel data to the heel learning device 9. In the present embodiment, the saddle data output determination unit 28, every time half a year has elapsed from a predetermined date, the saddle data on the first coil of the day (steel rolled up after being rolled in the rolled steel plate production line) Is determined to be output to the kite learning device 9. However, the timing at which the soot data is output to the soot learning device and the content of the soot data to be output are not limited to this. For example, every time a coil is manufactured, wrinkle data about the coil may be output to the wrinkle learning device 9, or a limit of 100 may be provided continuously. Moreover, you may make it output the wrinkle data about the last coil of the day to the wrinkle learning apparatus 9 for every day.
The bag data output determination unit 28 is realized by using, for example, the CPU, ROM, RAM, and hard disk of the bag determination device 6.

When the heel data output determination unit 28 determines that the heel data is to be output to the heel learning device 9, the heel data output unit 29 is set in advance among the heel data stored in the heel data storage unit 25. The obtained wrinkle data is transmitted to the wrinkle learning device 9. As described above, in this embodiment, the kite data output unit 29 transmits the kite data on the first coil of the day to the kite learning device 9 every half year after a predetermined date and time.
Further, as described above, each wrinkle data stored in the wrinkle data storage unit 25 is associated with a result of a wrinkle inspection such as a wrinkle type by the wrinkle determination unit 27. Therefore, in this embodiment, the eyelid data output unit 29 transmits the result of the eyelid inspection associated with the eyelid data to the eyelid learning device 9 together with the eyelid data.
The heel data output unit 29 is realized by using, for example, a data input / output control device of the heel determination device 6.

FIG. 3 is a block diagram illustrating an example of a functional configuration of the heel learning device 9.
In FIG. 3, the heel data acquisition unit 31 receives and stores the heel data transmitted from the heel determination device 6. When the wrinkle data is output to either the supervised learning unit 32 or the unsupervised learning unit 33 described later, the wrinkle data is deleted.
The heel data acquisition unit 31 is realized by using, for example, the CPU, ROM, RAM, hard disk, and data input / output control device of the heel determination device 6.

The wrinkle learning device 9 of the present embodiment performs two types of learning, supervised learning and unsupervised learning. Supervised learning is a method for generating (learning) a learning model by assigning “correct data (teacher data) related to species” to the kite data acquired by the kite data acquisition unit 31. It is. On the other hand, unsupervised learning refers to “six kinds of data” obtained by applying a learning model generated (learned) by the heel learning device 9 itself instead of the data set by the inspector. The learning model is generated (learned) by giving to the bag data acquired in 31.
Here, the correct answer data is obtained, for example, as follows. First, an inspector visually confirms a large number of images including various types of defects and actual steel irons corresponding to these images, and determines the type and grade of the defects. Based on the content determined in this way, “information on the cocoon type (type of cocoon, grade, etc.)” in the image is registered as correct data in the cocoon learning apparatus (computer).

In the present embodiment, the supervised learning unit 32 performs supervised learning, and the unsupervised learning unit 33 performs unsupervised learning.
First, the supervised learning performed by the supervised learning unit 32 will be described in detail.

When the data point arrangement unit 32a receives a plurality of pieces of bag data related to the bottle type from the bag data acquisition unit 31, the data point arrangement unit 32a divides the plurality of bottle data into three groups of bag data, and the first group of bag data Based on the above, image data representing a plurality of feature amounts relating to the species is generated. In the present embodiment, for example, it is assumed that the total number of bag data is 15000. For example, 7000 pieces of eyelid data in the first group, 5000 pieces of eyelid data in the second group, and 3000 pieces of eyelid data in the third group are set.
The data point arrangement unit 32a causes the display device 10 to display an image based on the image data generated for the first group of eyelid data. When this image is displayed, the inspector uses the user interface of the wrinkle determination device 6 to input correct data for a plurality of wrinkle data relating to the wrinkle type. The correct answer data is, for example, a determination result by an inspector such as the type and grade of the bag. The data point arrangement unit 32a gives the input correct data in association with the corresponding wrinkle data for each of the first group of wrinkle data.

The data point arrangement unit 32a includes a plurality of feature amounts included in the bag data input from the bag data acquisition unit 31 with each data point (value) of the “plurality of bag data related to the bottle type” to which the correct data is assigned. Are arranged in a coordinate space (feature amount space) with each feature amount as an axis. If a partial feature amount space to be described later has already been generated, the data point arrangement unit 32a also arranges the data points of the eyelid data used when generating the partial feature amount space in the coordinate space.
FIG. 4 is a diagram illustrating an example of a state in which a plurality of data points of the cocoon data regarding the cocoon type are arranged in the coordinate space having each feature amount as an axis. In the following description, “arranging the data points of the eyelid data in the coordinate space” is referred to as “placement of the eyelid data in the coordinate space” as necessary. Further, in the following description, it is assumed that a plurality of soot data relating to the soot types “C1” and “C2” are input from the soot data acquisition unit 31 as correct data, and the type of feature quantity is 150. Suppose it is a seed. Further, in FIG. 4, it is assumed that the feature quantities f2 and f7 are standardized for convenience of explanation.

  When the data point arrangement unit 32b arranges the data points of the basket data to which the correct data of the basket types “C1” and “C2” are assigned in the coordinate space by the data point placement unit 32a, Based on the arrangement position, it is determined whether or not the regions of the feature value values of the eyelid data overlap each other. As a result of the determination, if the regions of the feature value values of the cocoon data to which the correct data of the cocoon types “C1” and “C2” are not overlapped with each other, the coordinate space in which the cocoon data is arranged The combination of the feature value values of the axes is determined as a combination of feature value values that can completely separate the types “C1” and “C2” that are correct data.

  That is, in the example shown in FIG. 4, in the coordinate space having the feature quantities f2 and f7 as axes, the eyelid data relating to the eyelid type “C1” represented by white circles and the eyelid relating to the eyelid type “C2” represented by black circles. Data and are arranged. The positions of the eyelid data of the eyelid types “C1” and “C2” do not match in the coordinate space. Therefore, the varieties “C1” and “C2” can be completely separated by the areas of the respective values of the feature amounts f2 and f7. That is, the species “C1” and “C2” can be specified by a combination of the values of the feature amounts f2 and f7.

  As shown in FIG. 4, the feature amount combination determination unit 32 b determines that the feature value value areas of the cocoon data to which the correct data of the cocoon types “C1” and “C2” are not overlapped with each other. The combination of the values of the feature values of the axes in the coordinate space where the bag data is arranged is temporarily stored. Here, the feature amount combination determining unit 32b performs such an operation on, for example, the first group of feature amounts f1 to f50 by dividing 150 types of feature amounts into three. In the present embodiment, the following processing is performed by dividing into the first, second, and third groups as the feature amount groups. The ratio of the number of feature amounts belonging to each group is described below. It is not necessary to limit to the ratio illustrated in FIG.

The learning model generation unit 32c, based on the combination of the values of the feature amounts temporarily stored by the feature amount combination determination unit 32b and the types “C1” and “C2” that are correct answer data, A learning model is generated from a partial feature amount space for associating a set of values (that is, a region in the coordinate space) with a species.
FIG. 5 is a diagram conceptually illustrating an example of a learning model. FIG. 6 is a diagram conceptually illustrating an example of how a learning model is generated.
In FIG. 5, as an example, the learning model 51 is generated as a set of the partial feature amount space 1, the partial feature amount space 2,..., The partial feature amount space L (L is an integer of 3 or more). Show.
Here, the partial feature amount space 1 is for associating a set of values of the feature amounts f2 and f7 with the species “C1” and “C2”, and the feature amounts f2 and f7 are coordinate axes. The partial feature amount space 2 is for associating a set of values of the feature amounts f1, f3, and f7 with the species “C1” and “C2”, and uses the feature amounts f1, f3, and f7 as coordinate axes. The partial feature amount space L is for associating the feature amounts f4, f6, and f9 with the species “C1” and “C2”, and uses the feature amounts f4, f6, and f9 as coordinate axes.
At this stage, as shown in FIG. 6A, among the value pairs of the feature amounts f1 to f50, the feature value set that completely separates the species “C1” and “C2” and the species A learning model A is generated as a set of a plurality (L) of partial feature amount spaces for associating “C1” and “C2”.

  Next, as shown in FIG. 6B, the soot seed extracting unit 32 d (the data point arranging unit 32 a among the soot data “C1” and “C2” obtained by the soot data obtaining unit 31. The learning model A is applied to each of the second group of cocoon data (among the three groups of cocoon data divided in step 1) (that is, each partial feature amount space included in the learning model A of the second group of cocoon data) And determining whether the second group of eyelid data belongs to the species “C1” or “C2” from the position of the data point of each eyelid data in each of the partial feature amount spaces 1 to L). Extract the resulting information.

The rejection class determination unit 32e determines the number of seeds “C1” and “C2” that are determination results based on the partial feature amount spaces 1 to L extracted by the seed type extraction unit 32d for each of the second group of soot data. Based on the result of applying the learning model A, it is determined whether or not the species of the second group of soot data can be specified. FIG. 7 is a diagram conceptually illustrating an example of a process for determining the type “C1” and “C2” of each type of data.
That is, as shown in FIG. 7, in this embodiment, the seeds for the total number of seeds “C1” and “C2”, which are the judgment results by the partial feature amount spaces 1 to L, extracted by the seed type extracting unit 32d. When the ratio of the number determined to be “C1” is equal to or greater than a certain threshold, for example, 90% or more, the rejection class determination unit 32e indicates that the second group of soot data to which the learning model A is applied is the so-called “C1” ”(Determined that the species can be identified). Further, the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2”, which is the determination result by the partial feature amount spaces 1 to L extracted by the seed type extraction unit 32d, is less than a certain threshold value. For example, when it is 10% or less, the rejection class determination unit 32e determines that the second group of soot data to which the learning model A is applied belongs to the soot type C2 (determined that the soot type can be specified) To do). Then, the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” that are the determination results by the partial feature amount spaces 1 to L extracted by the seed type extraction unit 32d is within a certain range. For example, when it is larger than 10% and smaller than 90%, the rejection class determination unit 32e determines that the second group of soot data to which the learning model A is applied belongs to the sow type “C1”. It is determined that it does not belong (determined that the species cannot be identified).

  Among the second group of soot data, the soot data determined by the rejection class judging unit 32e to identify the soot type is another class that does not belong to any class in the learning model A (in the following explanation, this class Class (referred to as rejection class) data (in the following description, referred to as rejection data as required). The data point arrangement unit 32a generates image data representing the feature amount (feature amount for which the species cannot be specified) indicated by the rejection data. Then, the data point arrangement unit 32a causes the display device 10 to display an image based on the image data. When this image is displayed, the inspector inputs correct data for the rejection data using the user interface of the eyelid determination device 6. The data point arrangement unit 32a assigns the input correct data in association with the corresponding rejection data for each of the second group of bag data (see FIG. 6B). Here, either correct data “C1” or “C2” is given to the reject data.

  There may be data that is determined to be rejected data regardless of whether the type “C1” or “C2” is assigned as correct data. The data point arrangement unit 32a uses the data points of the rejection data to which any one of the types “C1” and “C2” is assigned as correct data based on the values of the feature amounts f51 to f100 of the second group. It is arranged in a coordinate space (feature amount space) with the amount as an axis. In other words, the data point arrangement unit 32a, for all combinations of the second group of feature values f51 to f100, includes reject data (second group of cocoon data for which the species cannot be identified) in a coordinate space having the combination as each axis. Place data points (see FIG. 4). When a partial feature amount space to be described later has already been generated, the data point arrangement unit 32a also arranges the data points of the eyelid data used when generating the partial feature amount space in the coordinate space.

  When the data point of the rejection data is arranged in the coordinate space by the data point arrangement unit 32a, the feature amount combination determination unit 32b determines the feature value value areas of the rejection data from each other based on the respective arrangement positions. It is determined whether or not there are duplicates. As a result of this determination, if the area of feature value values of the reject data does not overlap with each other, the combination of the feature value values of the axes in the coordinate space where the reject data is arranged is the correct data. A certain kind “C1” or “C2” is determined as a combination of feature value values that can be completely separated.

  When the feature quantity combination determination unit 32b determines that the feature value value areas of the reject data to which correct data of the species “C1” and “C2” are assigned do not overlap each other, the reject data is arranged. A combination of values of each feature amount (a feature amount for which the species cannot be specified) of the axis in the coordinate space is temporarily stored. The feature amount combination determination unit 32b performs such an operation on the second group of feature amounts f51 to f100 out of 150 types.

  The learning model generation unit 32c, based on the combination of the values of the feature amounts temporarily stored by the feature amount combination determination unit 32b and the types “C1” and “C2” that are correct answer data, A learning model B is generated from a partial feature amount space for associating a set of values with a seed type (see FIG. 6B). Since the learning model B generation method is the same as the learning model A generation method, a detailed description of the learning model B generation method is omitted.

  Next, as shown in FIG. 6 (c), the soot seed extracting unit 32 d performs the third group of soot data among the plurality of soot data related to the soot seed obtained by the soot data obtaining unit 31. By applying the learning model A (that is, by placing the third group of wrinkle data in each partial feature amount space included in the learning model A), in each of the partial feature amount spaces 1 to L, the data points of each wrinkle data Information of the determination result as to which of the species “C1” or “C2” belongs to

  The rejection class determination unit 32e determines the number of seeds “C1” and “C2” that are the determination results based on the partial feature amount spaces 1 to L extracted by the seed type extraction unit 32d for each of the third group of soot data. Based on the result of applying the learning model A, it is determined whether or not the species of the third group of soot data can be specified. As described above, the number of types determined as the type “C1” for the total number of types “C1” and “C2”, which are the determination results of the partial feature amount spaces 1 to L extracted by the type extraction unit 32d. When the ratio is within a certain range, for example, greater than 10% and smaller than 90%, the rejection class determination unit 32e determines that the species cannot be identified from the third group of soot data to which the learning model A is applied. If not, the rejection class determination unit 32e determines that the species can be identified from the third group of soot data to which the learning model A is applied.

Next, the soot seed extraction unit 32d applies the learning model B to each of the soot data (rejection data) determined by the rejection class determination unit 32e that the soot seed cannot be specified in the third group of soot data. (That is, the third group of wrinkle data is arranged in each partial feature amount space included in the learning model B), and in each of the partial feature amount spaces, from the position of the data point of each wrinkle data, Information on the result of determination as to whether the soot data belongs to the soot type “C1” or “C2” is extracted.
The rejection class determination unit 32e is based on the number of seeds “C1” and “C2”, which are judgment results based on the partial feature space, extracted by the seed type extraction unit 32d for each of the third group of soot data. As a result of applying the learning model B, it is determined whether or not the species of the third group of soot data can be specified. In the present embodiment, the ratio of the number determined to be the seed “C1” to the total number of the seeds “C1” and “C2”, which are the determination results based on the partial feature amount space extracted by the seed type extraction unit 32d, If the threshold value is equal to or higher than a certain threshold, for example, 90% or higher, the rejection class determination unit 32e determines that the third group of soot data to which the learning model B is applied belongs to the soot type “C1”. Can be identified). Further, the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2”, which is the determination result using the partial feature amount space extracted by the seed type extraction unit 32d, is equal to or less than a certain threshold value. For example, when it is 10% or less, the rejection class determination unit 32e determines that the third group of soot data to which the learning model B is applied belongs to the soot type C2 (determines that the soot type can be specified). ). Then, the ratio of the number of seeds “C1” to the total number of seeds “C1” and “C2”, which is the determination result using the partial feature amount space extracted by the seed type extraction unit 32d, is within a certain range. For example, when it is larger than 10% and smaller than 90%, the rejection class determination unit 32e indicates that the third group of soot data to which the learning model B is applied belongs to the sow species “C1” even if it belongs to the sow species “C1”. It is determined that it is not a thing (determined that the species cannot be identified).

  In the third group of soot data, the soot data determined by the reject class determining unit 32e that the soot type cannot be specified becomes reject data that does not belong to any class in the learning model B. The data point arrangement unit 32a generates image data representing the feature amount (feature amount for which the species cannot be specified) indicated by the rejection data. Then, the data point arrangement unit 32a causes the display device 10 to display an image based on the image data. When this image is displayed, the inspector inputs correct data for the rejection data using the user interface of the eyelid determination device 6. The data point arrangement unit 32a assigns the input correct data in association with the corresponding rejection data for each of the third group of wrinkle data (see FIG. 6C). In this case, either correct data “C1” or “C2” is given to the reject data.

  As described above, there may be data that is determined to be rejected data regardless of whether the type “C1” or “C2” is given as correct data. The data point arrangement unit 32a uses the data points of the rejection data to which any of the types “C1” and “C2” is assigned as correct data based on the values of the feature amounts f101 to f150 of the third group. It is arranged in a coordinate space (feature amount space) with the amount as an axis. In other words, the data point arrangement unit 32a, for all combinations of the third group of feature values f101 to f150, displays rejection data (third group of cocoon data for which no species can be identified) in the coordinate space with the combination as each axis. Place data points (see FIG. 4). When a partial feature amount space to be described later has already been generated, the data point arrangement unit 32a also arranges the data points of the eyelid data used when generating the partial feature amount space in the coordinate space.

  When the data point of the rejection data is arranged in the coordinate space by the data point arrangement unit 32a, the feature amount combination determination unit 32b determines the feature value value areas of the rejection data from each other based on the respective arrangement positions. It is determined whether or not there are duplicates. As a result of this determination, if the area of feature value values of the reject data does not overlap with each other, the combination of the feature value values of the axes in the coordinate space where the reject data is arranged is the correct data. A certain kind “C1” or “C2” is determined as a combination of feature value values that can be completely separated.

  When the feature quantity combination determination unit 32b determines that the feature value value areas of the reject data to which correct data of the species “C1” and “C2” are assigned do not overlap each other, the reject data is arranged. A combination of values of each feature amount (a feature amount for which the species cannot be specified) of the axis in the coordinate space is temporarily stored. The feature amount combination determination unit 32b performs such an operation on the third group of the feature amounts f101 to f150 out of 150 types.

  The learning model generation unit 32c, based on the combination of the values of the feature amounts temporarily stored by the feature amount combination determination unit 32b and the types “C1” and “C2” that are correct answer data, A learning model C is generated from a partial feature amount space for associating a set of values with a seed type (see FIG. 6C). Since the generation method of the learning model C is the same as the generation method of the learning models A and B, a detailed description of the generation method of the learning model C is omitted.

  However, in the present embodiment, when the soot data is applied to the learning model C, either the soot type “C1” or “C2” is always obtained from the soot data. Specifically, in the present embodiment, the number of types determined as the type “C1” for the total number of types “C1” and “C2”, which are the determination results based on the partial feature space extracted by the type extraction unit 32d. When the ratio is equal to or higher than a certain threshold, for example, 50% or higher, the rejection class determination unit 32e determines that the soot data to which the learning model C is applied belongs to the soot type “C1”. On the other hand, there is a ratio of the number determined as the seed “C1” to the total number of the seeds “C1” and “C2”, which is the determination result using the partial feature amount space extracted by the seed type extraction unit 32d. When it is smaller than the threshold, for example, smaller than 50% (less than 50%), the rejection class determination unit 32e determines that the soot data to which the learning model C is applied belongs to the soot type “C2”. .

As described above, in the present embodiment, as shown in FIG. 6D, three learning models A to C are generated. The generated learning model (partial feature amount space) is output to the learning model update unit 34 and the learning model output unit 35.
When the learning model (partial feature amount space) is input, the learning model update unit 34 rewrites the already stored learning model (partial feature amount space) to the input learning model (partial feature amount space). FIG. 8 is a diagram conceptually illustrating an example of how the learning model (partial feature amount space) is updated.

In the example shown in FIG. 8, the partial feature amount space 2 is deleted and the partial feature amount space L + 1 is added by this learning. That is, in the bag data acquired when the previous learning model A1 is generated, the bag types “C1” and “C2” can be completely separated by the feature amounts f1, f3, and f7, but the current learning model A2 is generated. If the soot data acquired at this time is added, the soot species “C1” and “C2” cannot be completely separated by the feature amounts f1, f3, and f7. In addition, in the bag data acquired when the previous learning model A1 is generated, the bag types “C1” and “C2” cannot be completely separated by the feature amounts f4, f6, and f10. In the soot data acquired at the time of generation, the soot species “C1” and “C2” can be completely separated by the feature amounts f4, f6, and f10.
In addition, when a learning model (partial feature amount space) is input, the learning model output unit 35 transmits the learning model to the wrinkle determination device 6.

  Further, when the learning model (partial feature amount space) is updated by the learning model update unit 34, the autonomous learning determination unit 36 of the partial feature amount spaces included in the learning models A to C is not updated. Count the number of quantity spaces. Then, for example, the autonomous learning determination unit 36 determines whether or not the absolute value of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is equal to or less than a threshold value. As a result of this determination, if the absolute value of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is less than or equal to the threshold value, the autonomous learning determination unit 36 turns on the autonomous learning flag. The learning method is automatically switched from supervised learning to unsupervised learning. On the other hand, if the absolute value of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is not less than or equal to the threshold value, the autonomous learning determination unit 36 keeps the autonomous learning flag off, Try to continue supervised learning. The autonomous learning flag is stored in a RAM, a register, or the like included in the kite learning device 9.

FIG. 9 is a diagram illustrating an example of the relationship between the number of partial feature amount spaces that have not been updated and the learning execution date and time.
In the present embodiment, the kite learning device 9 acquires the kite data from the kite determination device 6 every six months. Therefore, in FIG. 9, the distance in the horizontal axis direction between the plots corresponds to six months. It will be a thing.
In FIG. 9, supervised learning is performed until the absolute value Δ of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is equal to or less than the threshold date and time x. After the date x, unsupervised learning is performed. Thereafter, supervised learning is performed again after the date and time y when the absolute value Δ of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is larger than the threshold value.

  Note that FIG. 9 shows a case where the number counted in the current supervised learning is significantly reduced at the date and time y than the number counted in the previous supervised learning. It is possible that the number counted in the current supervised learning may be significantly larger than the number counted in the previous supervised learning. In such a case, unsupervised learning is continued. However, in the present embodiment, for the sake of explanation, when unsupervised learning is performed, the number counted in the current supervised learning is greatly increased from the number counted in the previous supervised learning. I will explain that there is no.

  The supervised learning unit 32, the learning model update unit 34, and the autonomous learning determination unit 36 are realized by using, for example, the CPU, ROM, RAM, and hard disk of the eyelid determination device 6. The learning model output unit 35 is realized by using, for example, a data input / output control device of the wrinkle determination device 6.

Here, with reference to FIG. 6 (d), an example of a method for discriminating the species in such learning models A to C will be described.
First, the learning model A is applied to each of the input bag data, and information on either of the bag types “C1” and “C2” is obtained from each partial feature amount space included in the learning model A. Then, when the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is 90% or more, for example, it is determined that the input soot data belongs to the seed “C1”. For example, if it is 10% or less, it is determined that the input soot data belongs to the soot type “C2” (learning model A → class determination in FIG. 6D).

  On the other hand, when the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is, for example, larger than 10% and smaller than 90%, the determination result in the learning model A indicates that the input soot data Since it is unclear whether it belongs to the species “C1” or “C2”, the learning model B is applied to each of the inputted cage data. As a result, when the ratio of the number of seeds “C1” to the total number of seeds “C1” and “C2” is 90% or more, for example, the input soot data belongs to the seed “C1”. For example, if it is 10% or less, it is determined that the input soot data belongs to the soot type “C2” (learning model B → class determination in FIG. 6D).

On the other hand, when the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is, for example, larger than 10% and smaller than 90%, the determination result in the learning model A indicates that the input soot data Since it is unknown which of the species “C1” and “C2” belongs to, the learning model C is applied to each of the inputted cage data. As a result, if the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is, for example, 50% or more, the input soot data is determined to belong to the seed “C1”. If it is smaller than 50%, it is determined that the input soot data belongs to the soot type “C2” (learning model C → class determination in FIG. 6D).
In the present embodiment, the wrinkle determination unit 27 of the wrinkle determination device 6 performs the determination of the wrinkle type as described above.

Next, unsupervised learning performed by the unsupervised learning unit 33 will be described in detail.
First, the soy seed extraction unit 33a applies the learning model A stored in the learning model update unit 34 to each of the soot data acquired by the soot data acquisition unit 31 (each of the soot data is learned as the learning model A). Information on any of the species “C1” and “C2” is extracted from each partial feature space.

  Next, the rejection class determination unit 33b applied the learning model A based on the number of seeds “C1” and “C2” that are the determination results based on the partial feature amount space extracted by the seed type extraction unit 33a. It is determined whether the cocoon data belongs to the cocoon species “C1”, the cocoon species “C2”, or does not belong to any of the cocoon species “C1” or “C2”.

  The seed type data adding unit 33c associates and assigns the data of the type “C1” to the type data determined to belong to the type “C1” by the rejection class determination unit 33b. Moreover, the soot seed data giving unit 33c gives the soot data “C2” in association with the soot data determined to belong to the soot type “C2” by the rejection class determining unit 33b.

  The data point arrangement unit 33d sets the data points of the cocoon data to which any one of the cocoon types “C1” and “C2” is assigned based on the values of the 150 types of feature values f1 to f150 as axes. Is arranged in a coordinate space (feature value space). That is, the data point arrangement unit 33d arranges the data points of the eyelid data in the coordinate space having the combinations as the respective axes for all combinations of the feature amounts f1 to f150 (see FIG. 4). The data point arrangement unit 32a also arranges the data points of the eyelid data used when generating the existing partial feature amount space in the coordinate space.

  When the eyelid data is placed in the coordinate space by the data point placement unit 33d, the feature amount combination determining unit 33e overlaps the feature value value areas of the eyelid data based on the respective placement positions. It is determined whether or not. As a result of this determination, if the regions of the feature value values of the eyelid data do not overlap each other, the combination of the feature value values of the axes of the coordinate space where the eyelid data is arranged is C1 ”and“ C2 ”are determined as combinations that can be completely separated. Then, the feature amount combination determination unit 33e temporarily stores a combination of values of each feature amount of the axis of the coordinate space in which the bag data is arranged. The feature amount combination determination unit 32b performs such an operation on the feature amounts f1 to f150.

The learning model generation unit 33f determines each feature based on the combination of the values of the feature values temporarily stored by the feature value combination determination unit 33e and the species “C1” and “C2” assigned to the bag data. A learning model A is generated from a partial feature amount space for associating a pair of quantity values with a seed type.
As described above, when the rejection class determination unit 33b determines that the soot data to which the learning model A is applied does not belong to any of the soot types “C1” and “C2”, the soot data is Rejection data. The soot seed extraction unit 33 applies the learning model B stored in the learning model update unit 34 to each of the soot data (rejection data) (that is, the soot data that is the rejection data is included in the learning model B). In each partial feature space, the determination result as to whether the bag data belongs to the type “C1” or “C2” from the position of the data point of each bag data Extract information.

  Next, the rejection class determination unit 33b applied the learning model B based on the numbers of the seeds “C1” and “C2” that are the determination results based on the partial feature amount space extracted by the seed type extraction unit 33a. It is determined whether the cocoon data belongs to the cocoon species “C1”, the cocoon species “C2”, or does not belong to any of the cocoon species “C1” or “C2”.

  The species data giving unit 33c associates and assigns the data of the species “C1” to the species data determined to belong to the species “C1” by the rejection class determination unit 33b. Moreover, the soot seed data giving unit 33c gives the soot data “C2” in association with the soot data determined to belong to the soot type “C2” by the rejection class determining unit 33b.

  The data point arrangement unit 33d sets the data points of the cocoon data to which any one of the cocoon types “C1” and “C2” is assigned based on the values of the 150 types of feature values f1 to f150 as axes. Is arranged in a coordinate space (feature value space). That is, the data point arrangement unit 33d arranges the data points of the eyelid data in the coordinate space having the combinations as the axes for all the combinations of the 150 types of feature quantities f1 to f150 (see FIG. 4). The data point arrangement unit 32a also arranges the data points of the eyelid data used when generating the existing partial feature amount space in the coordinate space.

  When the eyelid data is placed in the coordinate space by the data point placement unit 33d, the feature amount combination determining unit 33e overlaps the feature value value areas of the eyelid data based on the respective placement positions. It is determined whether or not. As a result of the determination, if the regions of the feature value values of the eyelid data do not overlap with each other, the combination of feature value values corresponding to the respective axes in the coordinate space where the eyelid data is arranged is The seeds “C1” and “C2” are determined as combinations that can be completely separated. Then, the feature amount combination determination unit 33e temporarily stores a combination of values of each feature amount of the axis of the coordinate space in which the bag data is arranged. The feature amount combination determination unit 32b performs such an operation on 150 types of feature amounts f1 to f150.

  The learning model generation unit 33f determines each feature amount based on the combination of the feature value values temporarily stored by the feature amount combination determination unit 33e and the seed types “C1” and “C2” assigned to the soot data. As described above, the learning model B is generated from the partial feature amount space for associating the value set and the species.

  As described above, when the rejection class determination unit 33b determines that the soot data to which the learning model B is applied does not belong to any of the soot species “C1” or “C2”, the soot data is Rejection data. The soot seed extraction unit 33 applies the learning model C stored in the learning model update unit 34 to each of the soot data (rejection data) (that is, the soot data that is the rejection data is included in the learning model C). From the position of the data point of each defect data (rejection data) in each partial feature space, the defect data (rejection data) is of type “C1”, “C2” The information of the determination result belonging to which is extracted.

  Next, the rejection class determination unit 33b applied the learning model C based on the numbers of the seeds “C1” and “C2” that are the determination results based on the partial feature amount space extracted by the seed type extraction unit 33a. It is determined whether the cocoon data belongs to the cocoon species “C1” or the cocoon species “C2”.

  The species data giving unit 33c associates and assigns the data of the species “C1” to the species data determined to belong to the species “C1” by the rejection class determination unit 33b. Moreover, the soot seed data giving unit 33c gives the soot data “C2” in association with the soot data determined to belong to the soot type “C2” by the rejection class determining unit 33b.

  The data point arrangement unit 33d uses the feature values f1 to f150 based on the values of 150 feature values f1 to f150 as the data points of the bag data to which any of the types “C1” and “C2” is assigned. Is arranged in a coordinate space (feature value space). That is, the data point arrangement unit 33d arranges the data points of the eyelid data in the coordinate space having the combinations as the respective axes for all combinations of the feature amounts f1 to f150 (see FIG. 4). The data point arrangement unit 32a also arranges the data points of the eyelid data used when generating the existing partial feature amount space in the coordinate space.

  When the eyelid data is placed in the coordinate space by the data point placement unit 33d, the feature amount combination determining unit 33e overlaps the feature value value areas of the eyelid data based on the respective placement positions. It is determined whether or not. As a result of this determination, if the regions of the feature value values of the eyelid data do not overlap each other, the combination of the feature value values of the axes of the coordinate space where the eyelid data is arranged is C1 ”and“ C2 ”are determined as combinations that can be completely separated. Then, the feature amount combination determination unit 33e temporarily stores a combination of values of each feature amount of the axis of the coordinate space in which the bag data is arranged. The feature amount combination determination unit 32b performs such an operation on the feature amounts f1 to f150.

  The learning model generation unit 33f determines each feature amount based on the combination of the feature value values temporarily stored by the feature amount combination determination unit 33e and the seed types “C1” and “C2” assigned to the soot data. As described above, the learning model C is generated from the partial feature amount space for associating the value set and the species.

  As described above, according to the present embodiment, in supervised learning, learning models A to C are generated by adding “correct data relating to species” input by an inspector to the kite data, whereas unsupervised learning is performed. Then, the learning models A to C are generated by assigning the soot data obtained by applying the soot data to the already generated learning models A to C to the soot data. In other words, in unsupervised learning, the kite learning device 9 can autonomously generate (learn) a learning model without helping humans.

  When unsupervised learning is performed, three learning models A to C are generated as in the case of supervised learning, and the generated learning model (partial feature amount space) is connected to the learning model update unit 34. It is output to the learning model output unit 35. Then, the learning model update unit 34 rewrites the already stored learning model (partial feature amount space) to the input learning model (partial feature amount space) (see FIG. 8). In addition, when a learning model (partial feature amount space) is input, the learning model output unit 35 transmits the learning model to the wrinkle determination device 6.

  Furthermore, when the absolute value of the difference between the number counted in the current unsupervised learning and the number counted in the previous supervised learning is equal to or less than a threshold value, the autonomous learning determination unit 36 The autonomous learning flag is kept on and the unsupervised learning is continued. On the other hand, if the absolute value of the difference between the number counted in the previous unsupervised learning and the number counted in the current unsupervised learning is not less than or equal to the threshold value, the autonomous learning determination unit 36 turns off the autonomous learning flag and unsupervised Automatically switches the learning method from learning to supervised learning.

Next, an example of the operation of the eyelid determination device 6 will be described with reference to the flowchart of FIG.
First, in step S <b> 1, the image input unit 21 determines whether image data is input from the imaging device 5. If the result of this determination is that image data has not been input from the imaging device 5, the operation proceeds to step S21 to be described later.
On the other hand, when image data is input from the imaging device 5, the process proceeds to step S <b> 2, and the labeling / feature amount calculation unit 24 performs processing on the image data processed by the shading correction unit 22 and the binarization unit 23. After performing the labeling process, the feature amount is calculated.

Next, in step S <b> 3, the eyelid determining unit 27 determines whether or not learning models A to C are acquired from the eyelid learning device 9. As a result of this determination, if the learning models A to C are not acquired, the wrinkle determination device 6 determines that it is in an initial state or the like, and proceeds to step S16 described later.
On the other hand, if learning models A to C have been acquired, the process proceeds to step S4. In step S4, the wrinkle determination unit 27 applies the acquired learning model A to each piece of wrinkle data calculated in step S2, and determines the vaginal type from each partial feature amount space included in the learning model A. Information on either “C1” or “C2” is obtained.

Next, in step S5, the wrinkle determination unit 27 determines the ratio of the number of types “C1” to the total number of types “C1” and “C2”, which is the execution result of step S4. As a result of the determination, if the ratio of the number of types “C1” to the total number of types “C1” and “C2” is 90% or more, the process proceeds to step S6, and the type determining unit 27 determines in step S2. It is determined that the calculated soot data belongs to the soot type “C1”. And it progresses to step S16 mentioned later.
Further, when the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is 10% or less, the process proceeds to step S7, and the soot determining unit 27 calculates the soot calculated in step S2. It is determined that the data belongs to the species “C2”. And it progresses to step S16 mentioned later.

  On the other hand, if the ratio of the number of seeds “C1” to the total number of seeds “C1” and “C2” is greater than 10% and smaller than 90%, the process proceeds to step S8. In step S8, the wrinkle determination unit 27 applies the acquired learning model B to each piece of wrinkle data calculated in step S2, and determines the wrinkle type from each partial feature amount space included in the learning model B. Information on either “C1” or “C2” is obtained.

Next, in step S9, the wrinkle determination unit 27 determines the ratio of the number of types “C1” to the total number of types “C1” and “C2”, which is the execution result of step S8. As a result of the determination, when the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is 90% or more, the process proceeds to step S10, and the eyelid determination unit 27 performs the process in step S2. It is determined that the calculated soot data belongs to the soot type “C1”. And it progresses to step S16 mentioned later.
If the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is 10% or less, the process proceeds to step S11, and the soot determining unit 27 calculates the soot calculated in step S2. It is determined that the data belongs to the species “C2”. And it progresses to step S16 mentioned later.

  On the other hand, if the ratio of the number of species “C1” to the total number of species “C1” and “C2” is greater than 10% and less than 90%, the process proceeds to step S12. In step S12, the wrinkle determination unit 27 applies the acquired learning model C to each of the wrinkle data calculated in step S2, and determines the vaginal type from each partial feature amount space included in the learning model C. Information on either “C1” or “C2” is obtained.

Next, in step S13, the wrinkle determination unit 27 determines the ratio of the number of types “C1” to the total number of types “C1” and “C2”, which is the execution result of step S12. As a result of the determination, if the ratio of the number of types “C1” to the total number of types “C1” and “C2” is 50% or more, the process proceeds to step S14, and the type determining unit 27 determines in step S2. It is determined that the calculated soot data belongs to soot type C1. And it progresses to step S16 mentioned later.
On the other hand, if the ratio of the number of seeds “C1” to the total number of seeds “C1” and “C2” is less than 50%, the process proceeds to step S15, and the eyelid determination unit 27 calculates the soot calculated in step S2. It is determined that the data belongs to the species “C2”. Then, the process proceeds to step S16.

  In step S16, the eyelid determining unit 27 stores the eyelid data calculated in step S2 in the eyelid data storage unit 25. At this time, if any of the species “C1” or “C2” is obtained in the learning models A to C (when the process proceeds from step other than step S3 to step S16), the determination is made. The data of the types “C1” and “C2” that are present and the bag data calculated in step S2 are stored in the bag data storage unit 25 in association with each other.

  Next, in step S <b> 17, the wrinkle determination unit 27 determines whether or not it is time to display the wrinkle type by determining whether or not the wrinkle determination has been performed on the image data in a predetermined range. As a result of this determination, if it is the timing for performing the wrinkle determination on the image data in a predetermined range and displaying the vase type, the process proceeds to step S18. In step S 18, the wrinkle determination unit 27 generates wrinkle detection image data (map) obtained by superimposing images imitating a wrinkle and displays the data on the display device 8. Then, the process proceeds to step S19.

On the other hand, if the wrinkle determination is not performed for the image data in the predetermined range and it is not the time to display the wrinkle type, step S18 is omitted and the process proceeds to step S19.
In step S 19, the heel data output determination unit 28 determines whether it is time to output the heel data stored in the heel data storage unit 25 to the heel learning device 9.

  If it is determined that it is time to output the heel data to the heel learning device 9, the process proceeds to step S20. In step S <b> 20, the heel data output unit 29 transmits the heel data obtained when preset among the heel data stored in the heel data storage unit 25 to the heel learning device 9. As described above, in this embodiment, every time half a year has passed from a predetermined date and time, the kite data for the first coil of the day is transmitted to the kite learning device 9. Then, the process returns to step S1.

If it is determined in step S1 that image data has not been input from the imaging device 5, the process proceeds to step S21. In step S21, the learning model acquisition unit 26 determines whether a learning model has been acquired from the kite learning device 9. If the result of this determination is that a learning model has not been acquired, processing returns to step S1.
On the other hand, if the learning model has been acquired, the process proceeds to step S22, where the learning model acquisition unit 26 has already acquired a learning model of the same type as the learning model determined to have been acquired in step S21. Determine.

As a result of this determination, if a learning model of the same type as the learning model determined to have been acquired in step S21 has not been acquired, the process proceeds to step S23. In step S23, the wrinkle determination unit 27 newly registers the learning model acquired in step S21. Then, the process returns to step S1.
On the other hand, if a learning model of the same type as the learning model determined to have been acquired in step S21 has already been acquired, the process proceeds to step S24. In step S24, the eyelid determination unit 27 rewrites the acquired learning model with the learning model acquired in step S21. Then, the process returns to step S1.

Next, an example of the operation of the eyelid learning apparatus 9 will be described with reference to the flowchart of FIG.
First, in step S <b> 101, the eyelid data acquisition unit 31 waits until the eyelid data is acquired from the eyelid determination device 6. When the bag data is acquired, the process proceeds to step S102. As described above, in the present embodiment, a plurality of pieces of eyelid data including 150 types of feature values related to the species “C1” and “C2” are acquired. In step S102, the autonomous learning determination unit 36 determines whether or not unsupervised learning can be performed by determining whether or not the autonomous learning flag is turned on. As a result of the determination, if the autonomous learning flag is turned on and unsupervised learning can be performed, the process proceeds to step S154 in FIG. 11-5 described later.
As described above, in the present embodiment, an acquisition unit is realized by executing the process of step S101.

On the other hand, if the autonomous learning flag is off and unsupervised learning cannot be performed, it is determined that supervised learning is performed, and the process proceeds to step S103. In step S103, the data point arrangement unit 32a divides the plurality of eyelid data determined to have been acquired in step S101 into three groups of eyelid data, and an image based on the first group of eyelid data is included. The correct data displayed on the display device 10 and input by the inspector to the displayed image is attached in association with the corresponding bag data.
Next, in step S104, the data point arrangement unit 32a selects one of the combinations of the first group of feature quantities f1 to f50, and a coordinate space (feature quantity space) around each feature quantity of the selected combination. In step S103, the data points of the “plurality of cocoon data relating to the species” to which the correct data of the species “C1” and “C2” are assigned are arranged. If the partial feature amount space has already been generated, the data point arrangement unit 32a also arranges the data points of the eyelid data used when generating the partial feature amount space in the coordinate space.

Next, in step S105, the feature amount combination determination unit 32b, based on the arrangement positions of the data points in step S104, the feature amount of the cocoon data to which correct data of the cocoon types “C1” and “C2” are assigned. It is determined whether or not the value areas overlap with each other. As a result of the determination, if the regions of the feature value values of the cocoon data to which the correct data of the cocoon types “C1” and “C2” are not overlapped with each other, the process proceeds to step S108 described later.
On the other hand, if the regions of the feature value values of the cocoon data to which the correct data of the cocoon types “C1” and “C2” are added, the process proceeds to step S106. When the process proceeds to step S106, the learning model generation unit 32c refers to the learning model update unit 34, and sets the value of each feature value of the axis of the coordinate space in which the eyelid data is arranged in step S104 and the kind “C1”, “ It is determined whether or not the learning model A includes a partial feature amount space for associating with “C2”. As a result of the determination, if the partial feature amount space is included in the learning model A, the process proceeds to step S107.

In step S107, the learning model update unit 34 deletes the partial feature amount space from the learning model A and updates the learning model A. And it progresses to step S110 mentioned later.
On the other hand, the learning model A does not include a partial feature amount space for associating a pair of feature values of the axes of the coordinate space in which the eyelid data is arranged in step S104 with the species “C1” and “C2” In this case, step S107 is omitted and the process proceeds to step S110 described later.

  If it is determined in step S105 that the regions of the feature value values of the cocoon data to which the correct data of the cocoon types “C1” and “C2” are not overlapped with each other, the process proceeds to step S108. In step S108, the learning model generation unit 32c refers to the learning model update unit 34, and sets the value of each feature quantity of the axis of the coordinate space in which the eyelid data is arranged in step S104 and the kind “C1”, “ It is determined whether or not a partial feature amount space for associating with “C2” is included in the learning model A. As a result of the determination, if the partial feature amount space is not included in the learning model A, the process proceeds to step S109.

In step S109, the learning model updating unit 34 adds the partial feature amount space to the learning model A and updates the learning model A. Then, the process proceeds to step S110.
On the other hand, when the partial feature amount space is included in the learning model A, step S109 is omitted and the process proceeds to step S110.

  As described above, in this embodiment, the placement unit is realized by executing the process of step S104, the first determination unit is realized by executing the process of step S105, and the processes of steps S106 to S109 are executed. By doing so, the first generating means and updating means are realized.

In step S110, the feature amount combination determining unit 32b determines whether all combinations of the first group of feature amounts f1 to f50 have been selected. As a result of the determination, if all the combinations of the feature amounts f1 to f50 of the first group are not selected, the process returns to step S104, and step S104 is performed until all the combinations of the feature amounts f1 to f50 of the first group are selected. -Repeat the process of step S110. Then, the process proceeds to step S111.
In the present embodiment, all combinations of the first group of feature quantities f1 to f50 are selected, but this is not necessarily required. For example, 2 to n (n is an integer greater than or equal to 3 (for example, 8)) combinations among the combinations of the feature amounts f1 to f50 of the first group may be selected.

If it progresses to step S111, the learning model production | generation part 32c will confirm the content of the newest learning model A based on the result of step S104-S110.
Next, in step S112, the learning model output unit 35 outputs the data of the learning model A determined in step S111 to the wrinkle determination device 6.
Next, in step S113 of FIG. 11-2, the soot seed extraction unit 32d applies the learning model A to each of the second group of soot data among the three soot data groups divided in step S103. (Arranged in each partial feature amount space included in the learning model A), information on any of the species “C1” and “C2” is extracted from each partial feature amount space.
Thus, in this embodiment, an extraction means is implement | achieved by performing the process of step S113.

Next, in step S114, the rejection class determination unit 32e extracts the species “C1” and “C2” that are the execution results of step S113 extracted by the species extraction unit 32d for each of the second group of soot data. Based on the number, it is determined whether or not the species of the second group of soot data can be specified as a result of applying the learning model A. Specifically, in the present embodiment, the ratio of the number of grape seeds “C1” to the total number of grape seeds “C1” and “C2” extracted by the grape seed extraction unit 32d is greater than 10% and smaller than 90%. Moreover, the rejection class determination unit 32e determines that the sow seed cannot be specified.
As described above, in this embodiment, the determination unit is realized by executing the process of step S114.
If the result of this determination is that the species cannot be identified, the process proceeds to step S115. When the process proceeds to step S115, the rejection class determination unit 32e extracts the second group of soot data determined to be unable to specify the soot type as the reject data. Then, the process proceeds to step S116.

On the other hand, when the species can be specified, step S115 is omitted and the process proceeds to step S116.
In step S116, the soy seed extraction unit 32d determines whether or not the processes in steps S113 to S116 have been performed for all of the second group of soot data. If the result of this determination is that the processing of steps S113 to S116 has not been performed for all of the second group of soot data, the process returns to step S113, and the processing of steps S113 to S116 for all of the second group of soot data. Steps S113 to S116 are repeated until. Then, the process proceeds to step S117.

In step S117, the data point arrangement unit 32a determines whether or not at least one rejection data is extracted in step S115. If at least one rejection data is not extracted in step S115 as a result of this determination, the process proceeds to step S149 in FIG.
On the other hand, if at least one rejection data is extracted in step S115, the process proceeds to step S118. In step S118, the data point arrangement unit 32a displays on the display device 10 an image representing the feature amount (feature amount that cannot specify the species) indicated by the rejection data, and the inspector inputs the displayed image. The correct answer data is given in association with the corresponding rejection data.

  Next, in step S119, the data point arrangement unit 32a selects one of the combinations of the second group of feature quantities f51 to f100, and a coordinate space (feature quantity space) around each feature quantity of the selected combination. In step S118, the data points of the rejection data (second group of cocoon data in which the cocoon type cannot be specified) to which correct data of the cocoon types “C1” and “C2” are assigned are arranged. If the partial feature amount space has already been generated, the data point arrangement unit 32a also arranges the data points of the eyelid data used when generating the partial feature amount space in the coordinate space.

Next, in step S120, the feature amount combination determination unit 32b, based on the result of arrangement of the data points in step S119, reject data to which correct data of types “C1” and “C2” is assigned (type 3 It is determined whether or not the regions of the feature value values of the second group (疵 data that cannot be specified) overlap each other. As a result of this determination, the feature value value areas of the reject data (second group of cocoon data in which the cocoon type cannot be specified) to which correct data of the cocoon types “C1” and “C2” are assigned do not overlap each other. In this case, the process proceeds to step S123 described later.
On the other hand, when the regions of the feature values of the rejection data (second group of cocoon data in which the cocoon type cannot be specified) with the correct data of the cocoon types “C1” and “C2” are mutually overlapped The process proceeds to step S121. In step S121, the learning model generation unit 32c refers to the learning model update unit 34, and sets the value of each feature value of the axis of the coordinate space in which the eyelid data is arranged in step S119 and the kind “C1”, “ It is determined whether or not the learning model B includes a partial feature amount space for associating with “C2”. As a result of this determination, if the partial feature amount space is included in the learning model B, the process proceeds to step S122.

In step S122, the learning model update unit 34 deletes the partial feature amount space from the learning model B and updates the learning model B. And it progresses to step S125 mentioned later.
On the other hand, the learning model B does not include a partial feature amount space for associating the feature value values of the axes of the coordinate space in which the eyelid data are arranged in step S119 with the species “C1” and “C2”. In this case, step S122 is omitted and the process proceeds to step S125 described later.

  In step S120, if the regions of the feature values of the rejection data (the second group of cocoon data in which the cocoon type cannot be specified) to which the correct data of the cocoon types “C1” and “C2” are assigned do not overlap each other If it is determined, the process proceeds to step S123. When the process proceeds to step S123, the learning model generation unit 32c refers to the learning model update unit 34, and sets the value of each feature value of the axis of the coordinate space in which the eyelid data is arranged in step S119 and the kind “C1”, “ It is determined whether or not the learning model B includes a partial feature amount space for associating with “C2”. As a result of the determination, if the partial feature amount space is not included in the learning model B, the process proceeds to step S124.

In step S124, the learning model update unit 34 adds the partial feature amount space to the learning model B and updates the learning model B. Then, the process proceeds to step S125.
On the other hand, when the partial feature amount space is included in the learning model B, step S124 is omitted and the process proceeds to step S125.

  As described above, in the present embodiment, the placement unit is realized by executing the process of step S119, the second determination unit is realized by executing the process of step S120, and the processes of steps S121 to S124 are executed. By doing so, the second generating means and updating means are realized.

In step S125, the feature amount combination determination unit 32b determines whether all combinations of the second group of feature amounts f51 to f100 have been selected. As a result of the determination, if all the combinations of the feature amounts f51 to f100 of the second group are not selected, the process returns to step S119, and step S119 is performed until all the combinations of the feature amounts f51 to f100 of the second group are selected. -Repeat the process of step S125. Then, the process proceeds to step S126.
In the present embodiment, all combinations of the second group of feature quantities f51 to f100 are selected, but this is not always necessary. For example, 2 to n (n is an integer greater than or equal to 3 (for example, 8)) combinations may be selected from among the combinations of the feature amounts f51 to f100 of the second group.

In step S126, the learning model generation unit 32c determines the content of the latest learning model B based on the results of steps S119 to S125.
Next, in step S127, the learning model output unit 35 outputs the data of the learning model B determined in step S126 to the wrinkle determination device 6.
Next, in step S128 of FIG. 11-3, the sow seed extraction unit 32d applies the learning model A to each of the third group of soot data among the three soot data groups divided in step S103. (The third group of cocoon data is arranged in each partial feature amount space included in the learning model A), and information on any of the cocoon types “C1” and “C2” is extracted from each partial feature amount space.

Next, in step S129, the rejection class determination unit 32e extracts the species “C1” and “C2” that are the execution results of step S128 extracted by the species extraction unit 32d for each of the third group of soot data. Based on the number, it is determined whether or not the species of the third group of soot data can be specified as a result of applying the learning model A.
If the result of this determination is that the species cannot be identified, the process proceeds to step S115. When the process proceeds to step S115, the rejection class determination unit 32e extracts the third group of soot data determined to be unable to specify the soot type as the reject data. Then, the process proceeds to step S131.

On the other hand, if the species can be specified, step S130 is omitted and the process proceeds to step S131.
In step S131, the soy seed extraction unit 32d determines whether or not the processes in steps S128 to S131 have been performed on all the soot data in the third group. As a result of the determination, if the processes of steps S128 to S131 are not performed for all the third group of eyelid data, the process returns to step S128, and the processes of steps S128 to S131 are performed for all of the third group of eyelid data. Steps S128 to S131 are repeated until. Then, the process proceeds to step S132.

In step S132, the data point arrangement unit 32a determines whether or not at least one rejection data is extracted in step S130. As a result of the determination, if at least one rejection data is not extracted in step S130, the process proceeds to step S149 in FIG.
On the other hand, when at least one rejection data is extracted in step S130, the process proceeds to step S133. In step S133, the data point arrangement unit 32a displays on the display device 10 an image representing the feature amount (feature amount that cannot identify the species) indicated by the rejection data, and the inspector inputs the displayed image. The correct answer data is given in association with the corresponding rejection data.

Next, in step S134, the soot seed extraction unit 32d applies the learning model B (rejection class determination) to each of the soot data (rejection data) determined by the rejection class determination unit 32e to be unable to identify the soot seed. The soot data determined that the soot type cannot be specified by the part 32e is arranged in each partial feature amount space included in the learning model B), and any of the soot types “C1” and “C2” is selected from each partial feature amount space. Extract information.
Thus, in the present embodiment, the second extraction unit is realized by executing the process of step S134.

Next, in step S135, the rejection class determination unit 32e extracts the species “C1”, which is the execution result of step S134, extracted by the species extraction unit 32d for each of the species data determined to be unable to identify the species. Based on the number of “C2”, as a result of applying the learning model B, it is determined whether or not the soot species of the soot data can be specified. Specifically, in the present embodiment, the ratio of the number of grape seeds “C1” to the total number of grape seeds “C1” and “C2” extracted by the grape seed extraction unit 32d is greater than 10% and smaller than 90%. Moreover, the rejection class determination unit 32e determines that the sow seed cannot be specified.
Thus, in the present embodiment, the second determination unit is realized by executing the process of step S135.
As a result of this determination, if the species cannot be identified, the process proceeds to step S136. If it progresses to step S136, the rejection class determination part 32e will extract the 3rd group of soot data determined that a sow seed | species cannot be specified as rejection data. Then, the process proceeds to step S137.

On the other hand, if the species can be specified, step S136 is omitted and the process proceeds to step S137.
If it progresses to step S137, the grape seed extraction part 32d will determine whether the process of step S133-S137 was performed about all the rejection data extracted by step S130. As a result of this determination, if the processing of steps S133 to S137 has not been performed for all of the rejection data extracted in step S130, the process returns to step S133, and all of the rejection data extracted in step S130 is processed in step S128. Steps SS133 to S137 are repeated until the processing of S131 is performed. Then, the process proceeds to step S138.

In step S138, the data point arrangement unit 32a determines whether or not at least one rejection data is extracted in step S136. As a result of the determination, if at least one rejection data is not extracted in step S136, the process proceeds to step S149 in FIG.
On the other hand, if at least one rejection data is extracted in step S136, the process proceeds to step S139. In step S139, the data point arrangement unit 32a displays on the display device 10 an image representing the feature amount (feature amount that cannot specify the species) indicated by the rejection data, and the inspector inputs the displayed image. The correct answer data is given in association with the corresponding rejection data.

  Next, in step S140 of FIG. 11-4, the data point arrangement unit 32a selects one of the combinations of the third group of feature quantities f101 to f150, and a coordinate space with each feature quantity of the selected combination as an axis. In the (feature amount space), the respective data points of the reject data (third group of trap data for which the trap type cannot be identified) to which the correct data of the trap types “C1” and “C2” are assigned in step S139 are arranged. If the partial feature amount space has already been generated, the data point arrangement unit 32a also arranges the data points of the eyelid data used when generating the partial feature amount space in the coordinate space.

Next, in step S141, the feature amount combination determination unit 32b, based on the result of the arrangement of the data points in step S140, reject data (species of “C1” and “C2”). It is determined whether or not the region of the feature value values of the third group (疵 data that cannot be specified) overlap each other. As a result of this determination, the feature value areas of the rejection data (third group of cocoon data in which the cocoon type cannot be identified) to which correct data of the cocoon types “C1” and “C2” are assigned do not overlap each other. In this case, the process proceeds to step S144 described later.
On the other hand, when the areas of the feature value values of the rejection data (third group of cocoon data for which the cocoon type cannot be identified) to which the correct data of the cocoon types “C1” and “C2” are assigned overlap each other The process proceeds to step S142. When the process proceeds to step S142, the learning model generation unit 32c refers to the learning model update unit 34, and sets the value of each feature value of the axis of the coordinate space in which the eyelid data is arranged in step S140 and the kind “C1”, “ It is determined whether or not the learning model C includes a partial feature amount space for associating with “C2”. As a result of this determination, if the partial feature amount space is included in the learning model C, the process proceeds to step S143.

In step S143, the learning model update unit 34 deletes the partial feature amount space from the learning model C and updates the learning model C. And it progresses to step S146 mentioned later.
On the other hand, when the partial feature amount space is not included in the learning model C, step S143 is omitted and the process proceeds to step S146 described later.

  In step S141, the feature value value areas of the reject data (third group cocoon data in which the cocoon type cannot be identified) to which the correct data of the cocoon types “C1” and “C2” are assigned do not overlap each other. If it is determined, the process proceeds to step S145. When the process proceeds to step S145, the learning model generation unit 32c refers to the learning model update unit 34, and sets the value of each feature amount of the axis of the coordinate space in which the eyelid data is arranged in step S140 and the kind “C1”, “ It is determined whether or not the learning model C includes a partial feature amount space for associating with “C2”. As a result of the determination, if the partial feature amount space is not included in the learning model C, the process proceeds to step S145.

In step S145, the learning model update unit 34 updates the learning model C by adding the partial feature amount space to the learning model C. Then, the process proceeds to step S146.
On the other hand, when the partial feature amount space is included in the learning model C, step S145 is omitted and the process proceeds to step S146.

  As described above, in the present embodiment, the placement unit is realized by executing the process of step S140, the third determination unit is realized by executing the process of step S141, and the processes of steps S142 to S145 are executed. Thus, the third generation unit and the update unit are realized.

In step S146, the feature amount combination determination unit 32b determines whether all combinations of the third group of feature amounts f101 to f150 have been selected. As a result of the determination, if all combinations of the third group of feature quantities f101 to f150 have not been selected, the process returns to step S140, and step S140 is continued until all of the third group of feature quantities f101 to f150 are selected. Step S146 is repeated. Then, the process proceeds to step S147.
In the present embodiment, all combinations of the third group of feature amounts f101 to f150 are selected, but this is not necessarily required. For example, 2 to n (n is an integer greater than or equal to 3 (for example, 8)) combinations among the combinations of the feature amounts f101 to f150 of the third group may be selected.

In step S147, the learning model generation unit 32c determines the contents of the latest learning model C based on the results of steps S140 to S147.
Next, in step S148, the learning model output unit 35 outputs the data of the learning model C determined in step S147 to the wrinkle determination device 6. Then, the process proceeds to step S149.
Proceeding to step S149, the autonomous learning determination unit 36 has not been updated among the partial feature amount spaces included in the learning models A to C based on the results of steps S107, S109, S122, S124, S143, and S145. The number of partial feature space is counted. And the autonomous learning determination part 36 determines whether the absolute value of the difference of the number counted in this learning and the number counted in the last learning is below a threshold value.
As a result of this determination, the autonomous learning determination unit 36, for example, if the absolute value of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is not less than or equal to the threshold value, It progresses to step S152 mentioned later.

  On the other hand, if the absolute value of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is equal to or less than the threshold, it is determined that unsupervised learning can be performed, and the process proceeds to step S150. . In step S150, the autonomous learning determination unit 36 determines whether or not the autonomous learning flag is turned off. As a result of this determination, if the autonomous learning flag is turned off, the process proceeds to step S151. In step S151, the autonomous learning determination unit 36 turns on the autonomous learning flag. And the process by the flowchart of FIG. 11 is complete | finished.

On the other hand, when the autonomous learning flag is turned on, step S151 is omitted and the process according to the flowchart of FIG. 11 is terminated.
If it is determined in step S149 that the absolute value of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is not less than or equal to the threshold value, it is determined that unsupervised learning cannot be performed. Then, the process proceeds to step S152. In step S152, the autonomous learning determination unit 36 determines whether or not the autonomous learning flag is turned on. If the result of this determination is that the autonomous learning flag is on, processing proceeds to step S153. In step S153, the autonomous learning determination unit 36 turns off the autonomous learning flag. And the process by the flowchart of FIG. 11 is complete | finished.
On the other hand, when the autonomous learning flag is turned off, step S153 is omitted and the process according to the flowchart of FIG. 11 is terminated.

If it is determined in step S102 in FIG. 11A that the autonomous learning flag is on and unsupervised learning can be performed, the process proceeds to step S154 in FIG. 11-5. In step S154, the cocoon seed extraction unit 33a applies the learning model A stored in the learning model update unit 34 to each of the cocoon data acquired by the cocoon data acquisition unit 31 (each of the cocoon data Are placed in each partial feature amount space included in the learning model A), and information on any of the species “C1” and “C2” is extracted from each partial feature amount space.
As described above, in the present embodiment, the extraction unit is realized by executing the process of step S154.

Next, in step S155, the rejection class determination unit 33b determines the type “C1” based on the number of types “C1” and “C2” that are the execution results of step S154 extracted by the type extraction unit 33a. ], The ratio of the number of the seeds “C1” to the total number of “C2” is determined.
Thus, in the present embodiment, the determination unit is realized by executing the process of step S155.
As a result of the determination, when the ratio of the number of the species “C1” to the total number of the species “C1” and “C2” is 90% or more, the process proceeds to step S156, and the rejection class determination unit 33b determines the learning model. It is determined that the soot data to which A is applied belongs to the soot type “C1”. And it progresses to step S158 mentioned later.
Further, when the ratio of the number of types “C1” to the total number of types “C1” and “C2” is 10% or less, the process proceeds to step S157, and the rejection class determination unit 33b applies the learning model A. It is determined that the soot data belongs to the soot type “C2”. And it progresses to step S158 mentioned later.
Further, when the ratio of the number of seeds “C1” to the total number of seeds “C1” and “C2” is larger than 10% and smaller than 90%, the process proceeds to step S174 in FIG. 11-6.

In step S158, the species data giving unit 33c associates and assigns the data of the species “C1” to the species data determined to belong to the species “C1” in step S156, and performs step S157. Then, the data of the species “C2” is associated with the grape data determined to belong to the species “C2”. Thereby, the cocoon data for which the species “C1” and “C2” are specified by the learning model A are associated with the data of the species “C1” and “C2”.
Next, in step S159, the data point arrangement unit 33d selects one of the combinations of the feature amounts f1 to f150, and in step S158, the coordinate point (feature amount space) having each selected feature amount as an axis is selected. The data points of the cocoon data to which the data of the seeds “C1” and “C2” are assigned are arranged. Note that the data point arrangement unit 33d also arranges the data points of the eyelid data used when generating the existing partial feature amount space in the coordinate space.

Next, in step S160, the feature amount combination determination unit 33e, based on the result in step S159, sets the feature value value areas of the cocoon data to which the data of the cocoon types “C1” and “C2” are assigned to each other. It is determined whether or not there are duplicates. As a result of this determination, if the regions of the feature value values of the eyelid data do not overlap each other, the process proceeds to step S163 described later.
On the other hand, when the regions of the feature value values of the eyelid data overlap with each other, the process proceeds to step S161. When the process proceeds to step S161, the learning model generation unit 33f refers to the learning model update unit 34, and sets the feature value values of the axis of the coordinate space in which the eyelid data is arranged in step S159 and the kind “C1”, “ It is determined whether or not the learning model A includes a partial feature amount space for associating with “C2”. As a result of this determination, when the partial feature amount space is included in the learning model A, the process proceeds to step S162.

In step S162, the learning model update unit 34 deletes the partial feature amount space from the learning model A and updates the learning model A. And it progresses to step S165 mentioned later.
On the other hand, when the partial feature amount space is not included in the learning model A, step S162 is omitted and the process proceeds to step S165 described later.

  If it is determined in step S160 that the feature value areas of the cocoon data to which the cocoon species “C1” and “C2” data are assigned do not overlap with each other, the process proceeds to step S163. When the process proceeds to step S163, the learning model generation unit 33f refers to the learning model update unit 34, and sets the value of each feature value of the axis of the coordinate space in which the eyelid data is arranged in step S159 and the kind “C1”, “ It is determined whether or not a partial feature amount space for associating with “C2” is included in the learning model A. As a result of the determination, if the partial feature amount space is not included in the learning model A, the process proceeds to step S164.

In step S164, the learning model update unit 34 adds the partial feature amount space to the learning model A and updates the learning model A. Then, the process proceeds to step S165.
On the other hand, when the partial feature amount space is included in the learning model A, step S164 is omitted and the process proceeds to step S165.

  As described above, in the present embodiment, the placement unit is realized by executing the processing of step S159, the first determination unit is realized by executing the processing of step S160, and the processing of steps S161 to S164 is executed. By doing so, the first generating means and updating means are realized.

In step S165, the feature amount combination determination unit 33e determines whether all combinations of the feature amounts f1 to f150 have been selected. As a result of the determination, if all the combinations of the feature amounts f1 to f150 are not selected, the process returns to step S159, and the processes of steps S159 to S165 are repeated until all the combinations of the feature amounts f1 to f150 are selected. Then, the process proceeds to step S166.
In the present embodiment, all combinations of the feature amounts f1 to f150 are selected, but it is not always necessary to do so. For example, all the combinations of 2 to n (n is an integer of 3 or more (for example, 8)) among the combinations of the feature amounts f1 to f150 may be selected.

In step S166, the learning model generation unit 33f determines the content of the latest learning model A based on the results of steps S159 to S165.
Next, in step S167, the learning model output unit 35 outputs the data of the learning model A determined in step S166 to the wrinkle determination device 6.
Next, in step S168, the soy seed extraction unit 33a determines whether or not all the soot data acquired by the soot data acquisition unit 31 has been processed. If the result of this determination is that processing has not been performed for all of the eyelid data acquired by the eyelid data acquisition unit 31, the process returns to step S154, and the processing of step S154 described above is performed on the remaining eyelid data. .
On the other hand, when processing has been performed for all of the eyelid data acquired by the eyelid data acquisition unit 31, the process proceeds to step S169 in FIG. 11-7. Proceeding to step S169, the autonomous learning determination unit 36, among the partial feature amount spaces included in the learning models A to C, based on the results of steps S162, S164 and steps S182, S184, S197, S199 described later, Count the number of partial feature spaces that have not been updated. And the autonomous learning determination part 36 determines whether the absolute value of the difference of the number counted in this learning and the number counted in the last learning is below a threshold value.
As a result of this determination, the autonomous learning determination unit 36, for example, if the absolute value of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is not less than or equal to the threshold value, It progresses to step S172 mentioned later.

  On the other hand, if the absolute value of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is less than or equal to the threshold, it is determined that unsupervised learning can be performed, and the process proceeds to step S170. . In step S170, the autonomous learning determination unit 36 determines whether or not the autonomous learning flag is turned off. If the result of this determination is that the autonomous learning flag is off, processing proceeds to step S171. In step S171, the autonomous learning determination unit 36 turns on the autonomous learning flag. And the process by the flowchart of FIG. 11 is complete | finished.

On the other hand, when the autonomous learning flag is turned on, step S171 is omitted and the processing according to the flowchart of FIG. 11 is terminated.
If it is determined in step S170 that the absolute value of the difference between the number counted in the current supervised learning and the number counted in the previous supervised learning is not less than or equal to the threshold value, it is determined that unsupervised learning cannot be performed. Then, the process proceeds to step S172. In step S172, the autonomous learning determination unit 36 determines whether or not the autonomous learning flag is turned on. If the result of this determination is that the autonomous learning flag is on, processing proceeds to step S173. In step S173, the autonomous learning determination unit 36 turns off the autonomous learning flag. And the process by the flowchart of FIG. 11 is complete | finished.
On the other hand, when the autonomous learning flag is turned off, step S173 is omitted and the process according to the flowchart of FIG. 11 is terminated.

If it is determined in step S155 that the ratio of the number of seeds “C1” to the total number of seeds “C1” and “C2” is greater than 10% and less than 90%, the step of FIG. 11-6 is performed. The process proceeds to S174. Proceeding to step S174, the culm seed extraction unit 33a determines that the ratio of the number of cultivar “C1” to the total number of cultivar “C1” and “C2” is greater than 10% and smaller than 90%. In model A, the learning model B stored in the learning model update unit 34 is applied to each of the bag data for which the bottle types “C1” and “C2” cannot be identified (each of the bag data is used as the learning model B). Arranged in each included partial feature amount space), information on any of the species “C1” and “C2” is extracted from each partial feature amount space.
Thus, in the present embodiment, the second extraction unit is realized by executing the process of step S174.

Next, in step S175, the rejection class determination unit 33b determines the type “C1” based on the number of types “C1” and “C2”, which are the execution results of step S174, extracted by the type extraction unit 33a. ], The ratio of the number of the seeds “C1” to the total number of “C2” is determined.
Thus, in the present embodiment, the second determination unit is realized by executing the process of step S175.
As a result of the determination, when the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is 90% or more, the process proceeds to step S176, and the rejection class determination unit 33b determines the learning model. It is determined that the soot data to which B is applied belongs to the soot type “C1”. And it progresses to step S178 mentioned later.
Further, when the ratio of the number of types “C1” to the total number of types “C1” and “C2” is 10% or less, the process proceeds to step S177, and the rejection class determination unit 33b applies the learning model B. It is determined that the soot data belongs to the soot type “C2”. And it progresses to step S178 mentioned later.
Further, when the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is larger than 10% and smaller than 90%, the process proceeds to step S189 in FIG. 11-7.

When the process proceeds to step S178, the soot seed data adding unit 33c attaches the soot type "C1" data to the soot data determined to belong to the soot type "C1" in step S176, and in step S177, the soot seed type is assigned. The cocoon data “C2” is associated with the cocoon data determined to belong to “C2”. Thereby, the cocoon data for which the species “C1” and “C2” are specified by the learning model B are associated with the data of the species “C1” and “C2”.
Next, in step S179, the data point arrangement unit 33d selects one of the combinations of the feature quantities f1 to f150, and in step S178, the coordinate space (feature quantity space) having each selected feature quantity as an axis is selected. The data points of the eyelid data to which the data of the seeds “C1” and “C2” are assigned are arranged. Note that the data point arrangement unit 33d also arranges the data points of the eyelid data used when generating the existing partial feature amount space in the coordinate space.

Next, in step S180, the feature amount combination determination unit 33e, based on the result in step S159, sets the feature value value areas of the cocoon data to which the varieties “C1” and “C2” data are assigned to each other. It is determined whether or not there are duplicates. As a result of this determination, if the regions of the feature value values of the eyelid data do not overlap with each other, the process proceeds to step S183 described later.
On the other hand, if the regions of the feature value values of the eyelid data overlap each other, the process proceeds to step S181. When the process proceeds to step S181, the learning model generation unit 33f refers to the learning model update unit 34, and sets the value of each feature quantity of the axis of the coordinate space in which the eyelid data is arranged in step S179 and the kind “C1”, “ It is determined whether or not a partial feature amount space for associating with “C2” is included in the learning model B. As a result of this determination, if the partial feature amount space is included in the learning model B, the process proceeds to step S182.

In step S182, the learning model update unit 34 deletes the partial feature amount space from the learning model B and updates the learning model B. And it progresses to step S185 mentioned later.
On the other hand, when the partial feature amount space is not included in the learning model B, step S182 is omitted and the process proceeds to step S185 described later.

  If it is determined in step S180 that the feature value areas of the cocoon data to which the cocoon species “C1” and “C2” data are assigned do not overlap with each other, the process proceeds to step S183. When the process proceeds to step S183, the learning model generation unit 33f refers to the learning model update unit 34, and sets the value of each feature amount of the coordinate space axis in which the eyelid data is arranged in step S179 and the kind “C1”, “ It is determined whether or not the learning model B includes a partial feature amount space for associating with “C2”. As a result of the determination, if the partial feature amount space is not included in the learning model B, the process proceeds to step S184.

In step S184, the learning model update unit 34 adds the partial feature amount space to the learning model B and updates the learning model B. Then, the process proceeds to step S185.
On the other hand, if the partial feature amount space is included in the learning model B, step S184 is omitted and the process proceeds to step S185.

  As described above, in the present embodiment, the placement unit is realized by executing the process of step S179, the second determination unit is realized by executing the process of step S180, and the processes of steps S181 to S184 are executed. By doing so, the second generating means and updating means are realized.

In step S185, the feature amount combination determination unit 33e determines whether all combinations of the feature amounts f1 to f150 have been selected. As a result of the determination, if all the combinations of the feature amounts f1 to f150 have not been selected, the process returns to step S179, and the processes of steps S179 to S185 are repeated until all the combinations of the feature amounts f1 to f150 are selected. . Then, the process proceeds to step S186.
In the present embodiment, all combinations of the feature amounts f1 to f150 are selected, but it is not always necessary to do so. For example, all the combinations of 2 to n (n is an integer of 3 or more (for example, 8)) among the combinations of the feature amounts f1 to f150 may be selected.

In step S186, the learning model generation unit 33f determines the content of the latest learning model B based on the results of steps S179 to S185.
Next, in step S187, the learning model output unit 35 outputs the learning model B data determined in step S186 to the wrinkle determination device 6.
Next, in step S188, the soy seed extraction unit 33a determines whether or not the processing has been performed for all of the soot data acquired by the soot data acquisition unit 31. If the result of this determination is that processing has not been performed for all of the eyelid data acquired by the eyelid data acquisition unit 31, the process returns to step S154 in FIG. Execute the process.

  On the other hand, when processing has been performed for all of the eyelid data acquired by the eyelid data acquisition unit 31, the process proceeds to step S169 in FIG. 11-7 described above. In step S169, the autonomous learning determination unit 36 determines the partial feature amount space included in the learning models A to C based on the results of steps S162 and S164 described above, steps S182 and S184, and S197 and S199 described later. Among them, the number of partial feature amount spaces that have not been updated is counted. In steps S170 to S173, the autonomous learning determination unit 36 turns off the autonomous learning flag when performing supervised learning, and turns on the autonomous learning flag when performing unsupervised learning. finish.

If it is determined in step S175 that the ratio of the number of seeds “C1” to the total number of seeds “C1” and “C2” is greater than 10% and less than 90%, the step of FIG. 11-7 is performed. The process proceeds to S189. Proceeding to step S189, the soot seed extraction unit 33a determines that the ratio of the number of soot seeds “C1” to the total number of soot seeds “C1” and “C2” is greater than 10% and less than 90%. In the models A and B, the learning model C stored in the learning model update unit 34 is applied to each of the bag data for which the bottle types “C1” and “C2” could not be identified (each of the bag data is a learning model). C) is placed in each partial feature amount space included in C), and information on any of the species “C1” and “C2” is extracted from each partial feature amount space.
As described above, in the present embodiment, the third extraction unit is realized by executing the process of step S189.

Next, in step S190, the rejection class determination unit 33b determines the type “C1” based on the number of types “C1” and “C2”, which are the execution results of step S189, extracted by the type extraction unit 33a. ], The ratio of the number of the seeds “C1” to the total number of “C2” is determined.
Thus, in the present embodiment, the third determination unit is realized by executing the process of step S190.
As a result of the determination, when the ratio of the number of the seeds “C1” to the total number of the seeds “C1” and “C2” is 50% or more, the process proceeds to step S191, and the rejection class determination unit 33b determines the learning model. It is determined that the soot data to which C is applied belongs to the soot type “C1”. Then, the process proceeds to step S193.
On the other hand, when the ratio of the number of species “C1” to the total number of species “C1” and “C2” is lower than 50%, the process proceeds to step S192, and the rejection class determination unit 33b applies the learning model C. It is determined that the soot data belongs to the soot type “C2”. Then, the process proceeds to step S193.

Proceeding to step S193, the species data giving unit 33c gives the species “C1” data to the species data determined to belong to the species C1 in step S191, and the species “C2” in step S192. ”Is assigned to the cocoon data determined to belong to“ 疵 ”. Thereby, the cocoon data for which the species “C1” and “C2” are specified by the learning model B are associated with the data of the species “C1” and “C2”.
Next, in step S194, the data point arrangement unit 33d selects one of the combinations of the feature quantities f1 to f150, and in step S193, the coordinate space (feature quantity space) having each selected feature quantity as an axis is selected. The data points of the cocoon data to which the data of the seeds “C1” and “C2” are assigned are arranged. Note that the data point arrangement unit 33d also arranges the data points of the eyelid data used when generating the existing partial feature amount space in the coordinate space.

Next, in step S195, the feature amount combination determination unit 33e, based on the result in step S194, sets the regions of the feature amount values of the cocoon data to which the data of the cocoon types “C1” and “C2” are attached to each other. It is determined whether or not there are duplicates. As a result of this determination, if the regions of the feature value values of the eyelid data do not overlap each other, the process proceeds to step S198 described later.
On the other hand, if the regions of the feature value values of the eyelid data overlap each other, the process proceeds to step S196. When the process proceeds to step S196, the learning model generation unit 33f refers to the learning model update unit 34, and sets the value of each feature value of the axis of the coordinate space in which the eyelid data is arranged in step S194 and the kind “C1”, “ It is determined whether or not the learning model C includes a partial feature amount space for associating with “C2”. As a result of this determination, if the partial feature amount space is included in the learning model C, the process proceeds to step S197.

In step S197, the learning model update unit 34 deletes the partial feature amount space from the learning model C and updates the learning model C. And it progresses to step S200 mentioned later.
On the other hand, when the partial feature amount space is not included in the learning model C, step S197 is omitted and the process proceeds to step S200 described later.

  If it is determined in step S195 that the feature value areas of the cocoon data to which the cocoon species “C1” and “C2” data are assigned do not overlap with each other, the process proceeds to step S198. When the process proceeds to step S198, the learning model generation unit 33f refers to the learning model update unit 34, and sets the value of each feature value of the axis of the coordinate space in which the eyelid data is arranged in step S194 and the kind “C1”, “ It is determined whether or not the learning model C includes a partial feature amount space for associating with “C2”. As a result of the determination, if the partial feature amount space is not included in the learning model C, the process proceeds to step S199.

In step S199, the learning model update unit 34 adds the partial feature amount space to the learning model C and updates the learning model C. Then, the process proceeds to step S200.
On the other hand, if the partial feature amount space is included in the learning model C, step S199 is omitted and the process proceeds to step S200.

  As described above, in the present embodiment, the placement unit is realized by executing the process of step S194, the third determining unit is realized by executing the process of step S195, and the processes of steps S196 to S199 are executed. Thus, the third generation unit and the update unit are realized.

In step S200, the feature amount combination determination unit 33e determines whether all combinations of the feature amounts f1 to f150 have been selected. As a result of the determination, if all the combinations of the feature amounts f1 to f150 are not selected, the process returns to step S194, and the processes of steps S194 to S200 are repeatedly performed until all the combinations of the feature amounts f1 to f150 are selected. . Then, the process proceeds to step S201.
In the present embodiment, all combinations of the feature amounts f1 to f150 are selected, but it is not always necessary to do so. For example, all the combinations of 2 to n (n is an integer of 3 or more (for example, 8)) among the combinations of the feature amounts f1 to f150 may be selected.

In step S201, the learning model generation unit 33f determines the content of the latest learning model C based on the results of steps S194 to S200.
Next, in step S202, the learning model output unit 35 outputs the data of the learning model C determined in step S201 to the wrinkle determination device 6.
Next, in step S <b> 203, the soy seed extraction unit 33 a determines whether or not all the soot data acquired by the soot data acquisition unit 31 has been processed. If the result of this determination is that processing has not been performed for all of the eyelid data acquired by the eyelid data acquisition unit 31, the process returns to step S154 in FIG. Execute the process.

  On the other hand, when processing has been performed for all of the eyelid data acquired by the eyelid data acquisition unit 31, the process proceeds to step S169 described above. Proceeding to step S169, the autonomous learning determination unit 36, among the partial feature amount spaces included in the learning models A to C, based on the results of steps S162, S164, S182, S184 and steps S197, S199 described above, Count the number of partial feature spaces that have not been updated. In steps S170 to S173, the autonomous learning determination unit 36 turns off the autonomous learning flag when performing supervised learning, and turns on the autonomous learning flag when performing unsupervised learning. finish.

  As described above, in the present embodiment, first, for the first group of feature quantities f1 to f50, the value groups of the first group of feature quantities f1 to f50 are associated with the species “C1” and “C2”. A plurality of partial feature amount spaces are generated to form (learn) the learning model A. Next, the learning model A is applied to each of the data of the feature amounts f51 to f100 of the second group. As a result, among the data of the feature quantities f51 to f100 of the second group, the soot data for which the soot species “C1” and “C2” cannot be specified in the learning model A is set as the rejection class data. In the learning model A, learning is performed by generating a plurality of partial feature amount spaces for associating the feature type values that cannot identify the species “C1” and “C2” with the species “C1” and “C2”. Model B is formed (relearned). Furthermore, the learning models A and B are sequentially applied to the data of the feature amounts f101 to f150 of the third group. As a result, among the data of the feature amounts f101 to f150 of the third group, the trap data that cannot identify the trap types “C1” and “C2” in both the learning model A and the learning model B are set as rejection class data. A plurality (preferably three or more) are associated with a pair of feature values that cannot identify the types “C1” and “C2” in the learning model A and the learning model B and the types “C1” and “C2”. ) To generate a learning model C (relearning).

  As described above, in the upper learning model (learning model A, B) constructed by the partial feature amount space, the upper learning model (learning model A , B) is applied to a lower learning model (learning models B and C) constructed by a partial feature amount space constituted by feature amounts different from those of B), and the species “C1” and “C2” are specified. . Therefore, if the species “C1” and “C2” can be identified by the higher-level learning model, it is possible to determine the soot at an early stage. On the other hand, when the upper learning model cannot identify the species “C1” and “C2”, the lower learning model identifies the species “C1” and “C2” (learning model B, By strengthening the learning model (by C), the species “C1” and “C2” can be identified as accurately as possible. Therefore, it is possible to improve the discrimination performance of wrinkles as compared with the conventional one while suppressing an increase in processing load.

  Further, in the present embodiment, when the learning models A to C are updated, the inspection is performed based on the number of partial feature amount spaces that have not changed before and after the update among the partial feature amount spaces included in the learning models A to C. The supervised learning that generates the learning model by assigning the "correct data on the species" set by the staff to the learning data, and the "type data" obtained by applying the learning model Thus, it is automatically determined whether to perform unsupervised learning for generating a learning model. Therefore, it is possible to give a universal index for determining whether or not to generate a learning model, and there is no need to regularly provide correct answer data (teacher data) as in the past. It is possible to reduce the time and labor for giving (teacher data), and to easily maintain the wrinkle discrimination performance in the wrinkle determination device 6 for a long time. As a result, the scissors determination device 6 operates stably, and the function of the scissors determination device 6 is exerted on the entire steel plate 1 that is the inspection object over a long period of time.

In the present embodiment, the case where the seeds are the seeds “C1” and “C2” has been described as an example, but the number of seeds is not limited to two. Moreover, the content of a cocoon kind is not limited to the kind and grade of a cocoon.
In the present embodiment, learning models A to C are generated (learned), but the number of learning models is not limited to three as long as the number is two or more. In addition, the determination conditions for specifying the species in each learning model A to C are not limited to steps S5, S9, and S13 in FIG. However, the last generated learning model (learning model C in the present embodiment) is such that the feature quantity belongs to one of the species “C1” and “C2” (that is, the rejection class cannot be made). A discrimination condition is generated.

  In this embodiment, when supervised learning is performed, the eyelid data acquired by the eyelid data acquisition unit 31 of the eyelid learning device 9 is divided into the first group to the third group of eyelid data, and the first group to the third group. Group wrinkle data was used for generation (learning) of learning models A to C, respectively. However, it is not always necessary to do this, and for the generation (learning) of learning models A to C, the eyelid data acquisition unit 31 may individually acquire the eyelid data.

  In the present embodiment, when generating (learning) the learning models A to C, combinations of different feature value values are extracted as feature value values that can completely separate the grape species “C1” and “C2”. I tried to do it. That is, when generating (learning) the learning model A, a combination of feature value values that can completely separate the species “C1” and “C2” from the feature values f1 to f50 of the first group is extracted. When generating (learning) the learning model B, a combination of feature value values that can completely separate the species “C1” and “C2” is extracted from the feature values f51 to f100 of the second group. When generating (learning) the learning model C, a combination of feature value values that can completely separate the species “C1” and “C2” is extracted from the feature values f101 to f150 of the third group. I tried to do it. However, this is not always necessary.

  For example, when generating (learning) the learning models A to C, combinations of feature value values that partially overlap are extracted as feature values that can completely separate the varieties “C1” and “C2”. Also good. For example, when generating (learning) the learning model A, a combination of feature value values that can completely separate the species “C1” and “C2” is extracted from the feature values f1 to f60 of the first group. Then, when generating (learning) the learning model B, a combination of feature value values that can completely separate the species “C1” and “C2” from the feature values f51 to f110 of the second group is extracted. When generating (learning) the learning model C, a combination of feature value values that can completely separate the species “C1” and “C2” is extracted from the feature values f101 to f150 of the third group. You may make it do.

  In the present embodiment, the absolute difference between the number of “non-updated partial feature amount spaces” counted in the current learning and the number of “partial feature amount spaces not updated” counted in the previous learning is absolute. It is determined whether or not the value is equal to or less than a threshold value, and whether to perform supervised learning or unsupervised learning is determined. However, this is not always necessary. For example, it may be determined whether or not the number of “partial feature amount spaces not updated” counted in the current learning is equal to or greater than a threshold value. Further, the number of “updated partial feature amount spaces” may be counted instead of the number of “partial feature amount spaces that have not been updated”.

  In the present embodiment, the case where the eyelid determination device 6 and the eyelid learning device 9 are connected to each other via a LAN has been described as an example, but this is not necessarily required. For example, the eyelid determination device 6 and the eyelid learning device 9 may not be able to communicate with each other. In this case, the learning model obtained by the heel learning device 9 is stored in a removable disk or the like, the inspector inserts the removable disk storing the learning model into the heel determination device 6, and the learning model is stored in the heel determination device 6. You just have to install it. Moreover, you may comprise the wrinkle determination apparatus 6 and the wrinkle learning apparatus 9 as one apparatus.

In the present embodiment, the description has been made on the premise of a learning model using a partial feature amount space, but the concept of a rejection class can also be introduced in a decision tree or SVM (Support Vector Machine).
First, the decision tree will be described. For example, it is assumed that a determination condition is set in the decision tree when the value of the feature value fx included in the cocoon data is 1 <fx <100, and it is determined that the cocoon type “C1”. In this case, instead of this determination condition, when the value of the feature quantity fx included in the bag data is 10 <fx <90, it is determined that the bag type is “C1”, and 1 <fx ≦ 10 or 90 A determination condition for determining that the class is a rejection class when ≦ fx <100 is set. If it is determined that the class is a rejection class, a decision tree having a determination condition different from the determination condition of the determination tree is generated (learned) using the trap data.
Next, SVM will be described. For example, 疵 data in which the distance D from the identification plane of the two feature values is within the range of −X <D <X (X is a positive number) is set as the rejection class data, and the 疵 data becomes the rejection class data. Using the data, an SVM different from the SVM is generated (learned). In addition, the defect data in the boundary surface of the support vectors of the species “C1” and “C2” may be used as the rejection class data.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment described above, whether to perform supervised learning or unsupervised learning is determined based on the number of partial feature amount spaces in the learning model. In contrast, in this embodiment, it is determined whether to perform supervised learning or unsupervised learning based on the distance between learning models. In this way, the present embodiment and the first embodiment described above are mainly different in criteria for performing supervised learning or unsupervised learning. Therefore, in the description of the present embodiment, the same parts as those in the first embodiment described above are denoted by the same reference numerals as those in FIGS.

FIG. 12 is a diagram conceptually illustrating an example of a distance between learning models.
In the example shown in FIG. 12, the direction of that shown in FIG. 12 (a), than that shown in FIG. 12 (b), a (a is a natural number) learning model P ^a which is generated th, (a + 1) -th The distance D from the learning model P ^{a + 1} generated in ( ¹⁾ is large. Learning model P ^a, the distance D between the P a ^{+ 1,} for example, it can be determined and the center of gravity of the learning model P ^a, the distance between the learning model P ^{a + 1} of the center of gravity.
Here, learning model P ^a, an embodiment of a method of calculating the distance D between the P a ^{+ 1} will be described.
First, the n-dimensional feature amount space F is represented by the following equation (1), where f is the feature amount.
F = {f ₁ , f ₂ ,..., F _n } (1)

An m-dimensional partial feature amount space G _k obtained by extracting m feature amounts f from the n-dimensional feature amount space F is expressed by the following equation (2).
G _k = {fθ ₁ , fθ ₂ ,..., Fθ _m }
However, 1 ≦ m ≦ n, k = {1, 2, 2 ⁿ −1} (2)

In this case, a set of partial feature space that is generated a second (learning model) P ^a is assumed to consist of L _a number of partial feature amount space F. Then, the set P ^a of the partial feature amount space is expressed as the following equation (3). Here, a function ξ representing whether or not a certain feature amount is included in an arbitrary partial feature amount space is defined as the following equation (4). Then, the set of partial feature amount spaces is expressed as the following equation (5).

Then, using equation (5), the center of gravity μ ^a of the set of partial feature amount spaces is expressed as the following equation (6). Then, the distance D between the centroids is expressed by the following equation (7).

  In this embodiment, for example, in step S149 of FIG. 11-4 and step S169 of FIG. 11-7, it is determined whether or not the distance D between the centroids obtained by the equation (7) is equal to or less than a threshold value. . When the distance D between the centroids is equal to or smaller than the threshold value, unsupervised learning is performed. When the distance D is larger than the threshold value, supervised learning is performed.

FIG. 13 is a diagram illustrating an example of the relationship between the distance between the centroids of the learning model, the species match rate, and the learning execution date and time.
In FIG. 13, a line represented by a one-dot chain line represents a distance between the centroids of the learning model. Further, the line connected by white circles is the species match rate when supervised learning is performed, and the line connected by black circles is the species match rate when unsupervised learning is performed. Note that the soot type match rate indicates the rate at which the soot type determined by the soot determining device 6 matches the actual soot type.

As can be seen from FIG. 13, when the distance between the centroids of the learning model is large and the date and time v is switched from supervised learning to unsupervised learning, the species match rate decreases. On the other hand, when switching from supervised learning to unsupervised learning at a date and time w where the distance between the centroids of the learning model is small, the species match rate hardly changes between supervised learning and unsupervised learning.
As described above, the same effect as described in the first embodiment can also be obtained by using the distance between learning models as an index for determining switching between supervised learning and unsupervised learning. Note that the distance between the learning models is not limited to that described above.

The embodiment of the present invention described above can be realized by a computer executing a program. Further, a means for supplying the program to the computer, for example, a computer-readable recording medium such as a CD-ROM recording such a program, or a transmission medium for transmitting such a program may be applied as an embodiment of the present invention. it can. A program product such as a computer-readable recording medium that records the program can also be applied as an embodiment of the present invention. The programs, computer-readable recording media, transmission media, and program products are included in the scope of the present invention.
In addition, the above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

It is the figure which showed the 1st Embodiment of this invention and showed an example of schematic structure of a surface flaw inspection system. It is a block diagram which shows the 1st Embodiment of this invention and shows an example of the function structure which a wrinkle determination apparatus has. It is a block diagram which shows the 1st Embodiment of this invention and shows an example of the function structure which a heel learning apparatus has. It is a figure which shows the 1st Embodiment of this invention and shows an example of a mode that the data point of the some soot data regarding a soot seed | species was arrange | positioned in the coordinate space centering on each feature-value. It is a figure which shows the 1st Embodiment of this invention and shows an example of a learning model notionally. It is a figure which shows the 1st Embodiment of this invention and shows an example of a mode that a learning model is produced | generated. It is a figure which shows the 1st Embodiment of this invention and shows an example of the process which identifies a sow seed | species conceptually. It is a figure which shows the 1st Embodiment of this invention and shows an example of a mode that a learning model (partial feature-value space) is updated. It is a figure which shows the 1st Embodiment of this invention and shows an example of the relationship between the number of the partial feature-value space which was not updated, and a learning execution date. It is a flowchart which shows the 1st Embodiment of this invention and demonstrates an example of the operation | movement of a wrinkle determination apparatus. It is a flowchart which shows the 1st Embodiment of this invention and demonstrates an example of the operation | movement of a wrinkle learning apparatus. It is a flowchart which shows the 1st Embodiment of this invention and follows FIG. It is a flowchart which shows the 1st Embodiment of this invention and follows FIG. 11-2. It is a flowchart which shows the 1st Embodiment of this invention and follows FIG. 11-3. It is a flowchart which shows the 1st Embodiment of this invention and follows FIG. It is a flowchart which shows the 1st Embodiment of this invention and follows FIG. 11-5. It is a flowchart which shows the 1st Embodiment of this invention and follows FIGS. 11-6. It is a figure which shows the 2nd Embodiment of this invention and shows an example of the distance between learning models notionally. It is a figure which shows the 2nd Embodiment of this invention and shows an example of the relationship between the distance between the gravity centers of a learning model, the sow seed match rate, and a learning execution date.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Steel plate 2 Conveyance roll 3 Rotary encoder 4 Illumination device 5 Imaging device 6 疵 determination device 7 Finishing mill 8, 10 Display device 9 疵 Learning device

Claims (11)

Acquisition means for acquiring wrinkle data including a characteristic amount related to wrinkles from the image of the inspection object;
First discriminating means for discriminating a combination of feature value values capable of separating the plurality of soot species, using soot data in a plurality of soot species, which is the soot data obtained by the obtaining means;
A learning model is generated from a partial feature amount space for associating a plurality of feature value values determined to be separable by the first determining means with a plurality of feature types. Generating means;
The wrinkle data acquired by the acquisition means and different from the wrinkle data already used is arranged in a plurality of partial feature amount spaces generated by the first generation means, and each partial feature amount An extraction means for extracting cocoon data corresponding to the cocoon data from the space;
The ratio of the number of specific types of data extracted from each partial feature amount space by the extraction unit to the total number of types of data extracted from each partial feature amount space by the extraction unit is compared with a threshold value. And determining means for determining whether or not a class relating to the species can be specified;
If it is determined that the class cannot be specified by the determination means, the defect data corresponding to the kind data extracted by the extraction means, and the defect data in a plurality of kinds as the rejection class data And a second determination unit that determines a combination of feature value values that can separate the plurality of species.
A learning model is generated from a partial feature amount space for associating a set of feature value values determined to be able to separate a plurality of species by the second discrimination means and the plurality of species. And a generation means. The feature value of the cocoon data acquired by the acquisition unit and different from the already used data is arranged in the learning model generated by the nth generation unit (n is an integer of 2 or more). A second to n-th extraction means for extracting the cocoon species data corresponding to the cocoon data;
Comparing the ratio of the number of specific types of data extracted by the second to nth extraction means to the total number of data of the specific types extracted by the second to nth extraction means and the threshold value A second to n-th determining means for determining whether or not a class relating to the species can be specified;
When it is determined that the class cannot be specified by the second to nth determining means, the cocoon data corresponding to the cocoon type data extracted by the second to nth extracting means, Third to (n + 1) discriminating means for discriminating combinations of feature value values capable of separating the plurality of soot species, using soot data of the soot species as rejection class data;
A learning model is obtained from a partial feature amount space for associating a set of feature values determined to be able to separate a plurality of species by the third to (n + 1) discrimination means and the plurality of variants. The wrinkle learning device according to claim 1, further comprising third to (n + 1) generation means for generating.   The wrinkle learning according to claim 1, wherein the determination unit determines a combination of values of feature amounts different from the already used feature amounts among the feature amounts acquired by the acquisition unit. apparatus.   4. The apparatus according to claim 1, further comprising an updating unit that stores the learning model generated by the generating unit in a storage medium and updates the learning model stored in the storage medium. The kite learning device described. The cocoon data acquired by the acquisition means, and the value of the feature value of the cocoon data in which the kind data is given as correct data, the two or more feature values of the plurality of feature values as axes Having an arrangement means for arranging in a coordinate space,
In the coordinate space in which the feature value values of the eyelid data are arranged by the arrangement means, the determining means is a feature amount constituting an axis of the coordinate space in which the feature value value areas of the eyelid data do not overlap each other. The wrinkle learning apparatus according to claim 1, wherein a combination of values is discriminated. An acquisition step of acquiring wrinkle data including a feature amount related to wrinkles from the image of the inspection object;
A first determination step of determining a combination of feature value values capable of separating the plurality of seeds using the seed data in the plurality of seeds, which is the eyelid data acquired in the acquisition step;
The first determination step generates a learning model from a partial feature amount space for associating a set of feature value values determined to be able to separate a plurality of species and the plurality of variants. Generation step;
The wrinkle data acquired by the acquisition step, which is different from the wrinkle data already used, is arranged in the plurality of partial feature amount spaces generated by the first generation step, and each partial feature amount An extraction step for extracting cocoon data corresponding to the cocoon data from the space;
The ratio of the number of specific types of data extracted from each partial feature amount space by the extraction step to the total number of types of data extracted from each partial feature amount space by the extraction step is compared with a threshold value. And a determination step for determining whether or not a class relating to the species can be specified;
If it is determined that the class cannot be specified by the determination step, the defect data corresponding to the defect data extracted by the extraction step, and the defect data in a plurality of defect types as the rejection class data A second determination step of determining a combination of feature value values that can be used to separate the plurality of species;
A learning model is generated from a partial feature amount space for associating a set of feature values determined to be able to separate a plurality of species by the second discrimination step and the plurality of variants. And a generation step. The feature value of the eyelid data obtained by the obtaining step, which is different from the already used data, is arranged in the learning model generated by the nth generation step (n is an integer of 2 or more). And the 2nd-nth extraction step which extracts the data of the soot kind corresponding to the said soot data,
Comparing the ratio of the number of data of a specific grape type extracted by the second to nth extraction steps to the total number of the species data extracted by the second to nth extraction steps, and a threshold value Then, the second to nth determination steps for determining whether or not the class relating to the species can be specified;
When it is determined that the class cannot be specified by the second to nth determination steps, the cocoon data corresponding to the varieties of data extracted by the second to nth extraction steps, Third to (n + 1) discrimination steps for discriminating combinations of feature values that can separate the plurality of soot species using soot data of the soot species as rejection class data;
A learning model is extracted from the partial feature amount space for associating a set of feature values determined to be able to separate a plurality of variants by the third to (n + 1) discrimination steps and the plurality of variants. The third to (n + 1) generation steps to be generated are included in the learning method according to claim 6.   The said learning step discriminate | determines the combination of the value of the feature-value different from the feature-value already used among the feature-values acquired by the said acquisition step, The spear learning of Claim 6 or 7 characterized by the above-mentioned. Method.   9. The method according to claim 6, further comprising an update step of storing the learning model generated by the generation step in a storage medium and updating the learning model stored in the storage medium. The method of learning cocoons described. The cocoon data acquired by the acquisition step, wherein the value of the feature value of the cocoon data in which the cocoon type data is given as the correct answer data, the two or more feature values of the plurality of feature values as axes A placement step for placing in a coordinate space,
In the determination step, among the coordinate spaces in which the feature value values of the eyelid data are arranged in the placement step, the feature amount constituting the axis of the coordinate space in which the feature value value areas of the eyelid data do not overlap each other The combination of values is discriminated, The wrinkle learning method according to any one of claims 6 to 9. An acquisition step of acquiring wrinkle data including a feature amount related to wrinkles from the image of the inspection object;
A first determination step of determining a combination of feature value values capable of separating the plurality of seeds using the seed data in the plurality of seeds, which is the eyelid data acquired in the acquisition step;
The first determination step generates a learning model from a partial feature amount space for associating a set of feature value values determined to be able to separate a plurality of species and the plurality of variants. Generation step;
The wrinkle data acquired by the acquisition step, which is different from the wrinkle data already used, is arranged in the plurality of partial feature amount spaces generated by the first generation step, and each partial feature amount An extraction step for extracting cocoon data corresponding to the cocoon data from the space;
The ratio of the number of specific types of data extracted from each partial feature amount space by the extraction step to the total number of types of data extracted from each partial feature amount space by the extraction step is compared with a threshold value. And a determination step for determining whether or not a class relating to the species can be specified;
If it is determined that the class cannot be specified by the determination step, the defect data corresponding to the defect data extracted by the extraction step, and the defect data in a plurality of defect types as the rejection class data A second determination step of determining a combination of feature value values that can be used to separate the plurality of species;
A learning model is generated from a partial feature amount space for associating a set of feature value values determined to be capable of separating a plurality of species and the plurality of species by the second determining step. A computer program for causing a computer to execute a generating step.

Images