JP2011192032A - Flaw learning device, flaw learning method and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To select data suited to the updating of a learning model from the data of many flaws when updating the learning model for discriminating a flaw kind from the feature amount of the flaw generated in a flaw inspection object using the data of the feature amount of the flaw for which the correct answer data of the flaw kind are given. <P>SOLUTION: The learning model is generated as a set of a plurality (L pieces) of partial feature amount spaces for making the set of the range of values of feature amounts for completely separating the flaw kinds "CL1", "CL2" and the flaw kinds "CL1", "CL2" correspond to each other. Then, the learning model is updated using only the flaw data with low reliability over the discrimination result of a flaw kind in the learning model in the feature amount space among the inputted flaw data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a wrinkle learning device, a wrinkle learning method, and a computer program, and is particularly suitable for automatically discriminating the type of wrinkle formed on the surface of an inspection object from an image. .

  In a production line such as a steel plate, harmful wrinkles may occur in the material to be rolled. For this reason, the image obtained by detecting and imaging the reflected light obtained by illuminating the surface of the material to be rolled is processed to extract the feature amount for the image of the wrinkle candidate, and based on the extracted feature amount Inspecting the presence and type of harmful flaws occurring in the material to be rolled is performed. Features include dimensional features such as heel length, width, and area, shape features such as circularity and aspect ratio (aspect ratio), and average and minimum image density of the heel. There are light and dark feature values such as value and maximum value. By extracting such feature amounts from the image, it is possible to discriminate the type, grade (score), and the like (these are collectively referred to as “species”).

  In an inspection apparatus that performs such an inspection of wrinkles, learning using teacher data is performed to construct a learning model that outputs the seeds from the feature values (performed supervised machine learning). Here, data (images) of a large number of cocoons are collected, and for each of the cocoons, the inspector determines the cocoon type, and the determined cocoon type data is registered as correct data for the cocoon. Is called teacher data. It is generally known that collecting such teacher data requires an enormous cost (see Patent Document 1 and Non-Patent Document 1).

  In conventional supervised machine learning, it is necessary to create a learning model for data distributed as widely as possible in the feature space, so a large amount of teacher data is used. In particular, in facilities that operate for a long period of time, such as an inspection device for defects in a production line for steel plates, it is said that it is necessary to periodically collect defects data and provide it with teacher data for learning. It was. However, there is a limit to the number of wrinkle data (images) that can actually be handled, and when this exceeds the limit, the amount of calculation increases and learning cannot be performed. Also, for example, when creating a learning model for discriminating between species A and species B, if the coordinator who creates the learning model erroneously includes data unrelated to species A and B in the data of the species, The reliability of the resulting learning model will be impaired. As described above, selection of appropriate wrinkle data is an indispensable element for realizing highly accurate machine learning.

Conventionally, however, it has been necessary to perform selection of cocoon data in the sense of a coordinator and prepare all types of cocoon data within a range that can be calculated. In order to assess (judge) the defect data, pass the steel sheet through the re-inspection line, observe the steel sheet in detail, and compare the observed defect of the steel sheet with the image of the defect displayed by the defect inspection device. Therefore, it is necessary to give correct data to the bag. Therefore, several hours to several tens of pieces of soot data are required for several hours and re-inspection line operation. For this reason, as described above, a great deal of cost has been spent collecting teacher data.
Therefore, in Patent Document 1, data that deviates from the probability distribution followed by the majority of data is considered to be data that is unlikely to occur, the data is excluded as a “statistical outlier”, and the data is identified as abnormal (incorrect). ing. Then, correct data is assigned to data excluding the data to perform supervised learning.

JP 2003-5970 A

Translated by Morio Onoe, “Pattern Identification”, New Technology Communications, February 2003, p14

However, for example, it is conceivable that data having a “statistical outlier” exists in the vicinity of the identification boundary of a plurality of species in the feature amount space. In such a case, the data cannot be simply determined as an “abnormal value”.
In addition, if the learning model is updated sequentially, the data of the kite to be newly learned has a “statistical outlier” that is different from the conventional data in the feature amount space due to a change in the operating conditions, etc. There is. In such a case, if the data is judged as an abnormal value with a simple probability distribution of feature values, the steel sheet flaw inspection apparatus that sequentially creates a learning model excludes important data. It will end up.
The present invention has been made in view of such problems, and a learning model for discriminating the species from the feature amount of the kite occurring in the kite inspection object is provided with correct data of the kite type. An object of the present invention is to enable selection of data suitable for updating a learning model from a large number of wrinkle data when updating using the data of the feature amount of the wrinkles.

  The wrinkle learning device according to the present invention includes an acquisition unit that acquires a plurality of wrinkle data each including a plurality of features of a wrinkle as data items from an image of an inspection object, and a variety of correct answers for each of the wrinkle data An assigning means for giving data based on an operation of the input device by a user, and a feature value of the wrinkle data obtained by the obtaining means, wherein a plurality of wrinkles to which correct data is given by the giving means Based on the result of arranging a plurality of feature values of each piece of data in a feature amount space having each of the plurality of feature amounts as respective coordinate axes, the type of the heel data is determined from the plurality of feature value values of the heel data. In the feature amount space having each coordinate axis as a plurality of feature amounts of the wrinkle data among the wrinkle data acquired by the generation unit that generates a learning model for determination and the acquisition unit, An extraction unit that extracts cocoon data having a low certainty with respect to the determination result of the cocoon type in the learning model, and an update unit that updates the learning model generated by the generation unit, and the grant unit includes the update unit When the learning model is updated by the above, the correct answer data of the cocoon data extracted by the extracting unit among the cocoon data acquired by the acquiring unit is obtained based on the operation of the input device by the user. The update means is a feature value value of the heel data extracted by the extraction means, and a plurality of feature value values of the heel data to which correct data is assigned by the grant means, Based on the result of arranging the values of the plurality of feature amounts of the eyelid data used for generating the learning model in the feature amount space having each of the plurality of feature amounts as coordinate axes, And updates the model.

  According to the wrinkle learning method of the present invention, an acquisition step of acquiring a plurality of wrinkle data each including a plurality of features of a wrinkle as a data item from an image of an inspection object, and a variety of correct answers for each of the wrinkle data A granting step for granting data based on an operation of the input device by a user, and a feature value of the trap data acquired by the acquisition step, wherein a plurality of traps to which correct data is given by the granting step Based on the result of arranging a plurality of feature values of each piece of data in a feature amount space having each of the plurality of feature amounts as respective coordinate axes, the type of the heel data is determined from the plurality of feature value values of the heel data. In the feature amount space in which each of the plurality of feature amounts of the wrinkle data among the wrinkle data acquired by the generating step and generating the learning model for discrimination An extraction process for extracting cocoon data having a low certainty for the determination result of the cocoon type in the learning model, and an updating process for updating the learning model generated by the generating process, wherein the adding process includes the updating process When the learning model is updated by the above, among the cocoon data obtained by the obtaining step, the cocoon data extracted by the extraction step is used to obtain various kinds of correct answer data based on the operation of the input device by the user. And the updating step is a feature value value of the heel data extracted by the extraction step, and a plurality of feature value values of the heel data to which correct data is given by the adding step, and Based on the result of arranging the values of the plurality of feature amounts of the eyelid data used for generating the learning model in the feature amount space having each of the plurality of feature amounts as coordinate axes, And updates the model.

  The computer program according to the present invention includes an acquisition step of acquiring a plurality of wrinkle data each including a plurality of features of a wrinkle as data items from an image of an inspection object, and a variety of correct answer data for each of the wrinkle data And a plurality of wrinkle data to which correct answer data has been given by the giving step. Based on the result of arranging each of the plurality of feature values in the feature amount space with each of the plurality of feature amounts as each coordinate axis, the type of the cocoon data is determined from the plurality of feature value values of the heel data A learning model for generating a learning model, and among the wrinkle data acquired by the acquisition step, a plurality of feature amounts of the wrinkle data are in a feature amount space having each coordinate axis And an extraction step for extracting cocoon data having a low certainty with respect to a discrimination result of the cocoon type in the learning model, and an updating step for updating the learning model generated by the generation step, and the adding step When the learning model is updated by the updating step, the user inputs the correct answer data for the cocoon data extracted by the extracting step among the cocoon data acquired by the acquiring step. The update step is a feature value value of the wrinkle data extracted by the extraction step, and the update step includes a plurality of feature amounts of the wrinkle data to which correct data is given by the addition step. The values and the values of the plurality of feature values of the eyelid data that have already been used for generating the learning model into a feature amount space with each of the plurality of feature values as coordinate axes. Based on the result of the location, and updates the learning model.

  According to the present invention, from the acquired wrinkle data, the wrinkle data having a low certainty for the discrimination result of the wrinkle type in the learning model is extracted in the feature amount space having each of the plurality of feature amounts of the wrinkle data as coordinate axes. The learning model is updated using the extracted soot data. Therefore, data suitable for updating the learning model can be selected from a large number of bag data. Therefore, the work of the inspector for updating the learning model can be reduced, and the reliability of the learning model can be improved.

It is the figure which showed embodiment of this invention and showed an example of schematic structure of the surface flaw inspection system. It is a block diagram which shows embodiment of this invention and shows an example of the function structure which a wrinkle determination apparatus has. It is a block diagram which shows embodiment of this invention and shows an example of the function structure which a heel learning apparatus has. It is a figure which shows embodiment of this invention and shows an example of a mode that the data point of the some cocoon data regarding a kind is arrange | positioned in the coordinate space which uses each feature-value as a coordinate axis. It is a figure which shows embodiment of this invention and shows an example of a learning model notionally. It is a figure which shows embodiment of this invention and shows an example of a mode that a learning model (partial feature-value space) is updated. It is a figure which shows embodiment of this invention and shows an example of the process which determines the seed | species "CL1" and "CL2" of each eyelid data. It is a figure which shows embodiment of this invention and shows the example of arrangement | positioning of the wrinkle data in the feature-value space which uses each of feature-value fi and fj as a coordinate axis. It is a figure which shows embodiment of this invention and shows an example of the relationship between the vote rate with respect to the kind "CL1", and the number (data number) of soot data. It is a flowchart which shows embodiment of this invention and demonstrates an example of operation | movement of a wrinkle determination apparatus. It is a flowchart which shows embodiment of this invention and demonstrates an example of the operation | movement of a wrinkle learning apparatus. It is a figure which shows embodiment of this invention and shows an example of the number of data (data number) used for the production | generation / update of the learning model in each month. It is a figure which shows the implementation type | system | group of this invention, and shows an example of the soot seed | species coincidence rate in each month.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating an example of a schematic configuration of a surface defect inspection system. In the present embodiment, as shown in FIG. 1, the strip steel plate 1 is immersed in a molten metal plating bath 12 through a snout 11, and the molten metal plating bath 12 is passed through a folding roll 13 in the bath and a support roll 14 in the bath. Is pulled up vertically. Then, gas is blown from the wiping nozzles 15a and 15b to the front and back surfaces of the strip steel plate 1 pulled up from the molten metal plating bath 12 to adjust the basis weight of plating. In this embodiment, the case where the plated steel plate 10 manufactured with such a plating manufacturing line is a flaw inspection object will be described as an example. Moreover, in the following description, the plated steel plate 10 is abbreviated as the steel plate 10.

In FIG. 1, the steel plate 10 is conveyed (moved) in the longitudinal direction of the steel plate 10 (the direction of the arrow in FIG. 1) by the conveying rolls 2a and 2b. The rotations of the transport rolls 2a and 2b are detected by the rotary encoder 3, respectively.
The surface wrinkle inspection system installed in such a plating production line includes a lighting device 4, an imaging device 5, a wrinkle determination device 6, and a wrinkle learning device 7.

  The illuminating device 4 includes, for example, a light emitting unit that emits light having high luminance, such as a halogen lamp, and a condensing unit such as a condensing lens. The light emitted from the halogen lamp or the like is condensed linearly by the condensing unit, so that the surface of the steel plate 10 is illuminated linearly in the width direction. In the present embodiment, the illuminating device 4 irradiates a linear luminous flux longer than the length in the width direction of the steel plate 10 so that the entire width direction of the steel plate 10 is surely illuminated in a linear shape. Yes.

The imaging device 5 has a function of capturing an image inspection range of the steel plate 10 illuminated by the illumination device 4 and generating an image signal. As shown in FIG. 1, the imaging device 5 is disposed on the optical path of light that is irradiated from the illumination device 4 and reflected by the steel plate 10. The imaging device 5 may be either an area camera (sensor) that reads an image in units of frames or a line camera (sensor) that reads an image in units of lines, such as a CCD image sensor or a CMOS image sensor. You can use a camera. The captured image may be a black and white grayscale image or a color image.
Here, the case where a line camera that outputs a monochrome grayscale image is used as the imaging device 5 will be described as an example. Specifically, the imaging device 5 captures a grayscale image in the wrinkle inspection range that is elongated in the width direction of the steel plate 10, and outputs, for example, black and white 256 gradation information as a captured image.

  The rotary encoder 3 outputs a control timing signal (pulse signal) corresponding to the conveying speed of the steel plate 10 so that the imaging device 5 can continuously capture the surface of the moving steel plate 10 with a predetermined spatial resolution without any gap. 5 is output. The imaging device 5 repeats imaging and transfer of the captured image according to the control timing signal output from the rotary encoder 3.

  The wrinkle determination device 6 and the wrinkle learning device 7 are, for example, PCs (Personal Computers). The eyelid determination device 6 and the eyelid learning device 7 include a CPU, ROM, RAM, hard disk, user interface including a keyboard and a mouse, a data input / output control device, and the like. The CPU, ROM, RAM, hard disk, user interface, and data input / output control device are each connected to a communication bus and can communicate with each other. In this embodiment, it is assumed that wrinkle determination device 6 and wrinkle learning device 7 are connected to each other so as to be communicable via, for example, a LAN (Local Area Network).

  The hard disk, which is one of the storage media of the eyelid determination device 6, is used in an inspection application program for determining the type of eyelids (an eyelid inspection) and the application program for the inspection as will be described later. Data is stored. In addition, the hard disk of the eyelid determination device 6 also stores image data captured by the imaging device 5 and processed by the data input / output control device. Here, the data input / output control device performs processing for configuring the image data for each line output from the imaging device 5 into, for example, image data having the number of pixels of 2048 (width) × 512 (length). And transfer to RAM. Here, for convenience of explanation, one pixel represents a captured image of a 1 mm × 1 mm region on the surface of the steel plate 1. That is, an image composed of 2048 (width) × 512 (length) pixels represents a captured image having a width of 2048 mm and a length of 512 mm of the steel plate 1.

  The hard disk, which is one of the storage media of the heel learning device 7, has a heel learning application program for generating and updating a discrimination model (learning model) for performing the heel inspection by the heel determination device 6, and its heel learning. Data used in the application program is stored. The hard disk of the wrinkle learning device 7 also stores “haze data including a plurality of feature amounts relating to wrinkles as data items” obtained by the wrinkle determination device 6 as described later. The wrinkle determination device 6 inputs the latest learning model obtained by the wrinkle learning device 7 and performs a wrinkle inspection according to the learning model. As described above, as the feature amount, for example, a dimensional feature amount such as the length, width, and area of the heel, a shape feature amount such as a circularity, an aspect ratio (aspect ratio), and the like, There are shaded features such as the average value, minimum value, and maximum value of image density.

The CPU and the like of the wrinkle determination device 6 and the wrinkle learning device 7 activate the flaw detection application program and the wrinkle learning application program based on the operation of the user interface by the user. Then, the CPU or the like executes an inspection application program and an eyelid learning application program to execute processes for performing eyelid inspection and learning model generation / update.
The display devices 8 and 9 include, for example, an LCD (Liquid Crystal Display) and the like, and are for displaying images based on processes executed by the eyelid determination device 6 and the eyelid learning device 7, respectively.
In addition, in FIG. 1, although the case where a flaw test | inspection is performed only about one surface (front surface) of the steel plate 10 is shown, a flaw inspection is actually performed also about the other surface (back surface) of the steel plate 10.

FIG. 2 is a block diagram illustrating an example of a functional configuration of the wrinkle determination device 6.
In FIG. 2, the image input unit 21 inputs image data for each line imaged by the imaging device 5, and uses, for example, image data (2048 × 1 pixel) for a plurality of lines continuously input, for example, Image data having the number of pixels of 2048 (width) × 512 (length) is configured. Then, the image input unit 21 stores the image data in the RAM.
The image input unit 21 is realized, for example, when the data input / output control device included in the eyelid determination device 6 executes the processing described above.

The shading correction unit 22 performs shading correction on the image data so that the entire image data obtained by the image input unit 21 has uniform brightness. Then, the shading correction unit 22 stores the image data subjected to the shading correction in a RAM (storage medium).
The shading correction unit 22 is realized, for example, when the CPU of the wrinkle determination device 6 executes a program for performing shading correction.

The binarization unit 23 performs binarization processing on the image data that has undergone shading correction. Specifically, the binarization unit 23 determines whether, for example, the value of each pixel of the image data subjected to the shading correction is larger than a threshold set in advance in the hard disk or the like. Then, the value of the pixel larger than the threshold is set to “1”, the value of the other pixel is set to “0”, and a binary image having only “0” or “1” as the pixel value (luminance value) is generated. .
The binarization unit 23 is realized, for example, when the CPU of the eyelid determination device 6 executes a program for performing binarization processing.

The labeling / feature amount calculation unit 24 performs a labeling process and a feature amount calculation on the binary image obtained by the binarization unit 23, and extracts a wrinkle candidate. Here, the labeling process is a process performed before the feature amount calculation, and for each figure (connected figure) that is a single figure or geometrically connected in the binary image to form one lump. This is a process of assigning different numbers (labels).
The feature amount calculation (feature extraction) is a process for obtaining a feature amount related to the shape of a figure. The feature amount calculated by this feature amount calculation is, for example, “a rectangle circumscribing the heel candidate“ width / length / length / width ratio (aspect ratio) / area / circularity / maximum length / perimeter ””, and It is one or a combination of two or more selected from “brightness candidate luminance distribution (average value, minimum value, maximum value, etc. of luminance)”. The labeling / feature quantity calculating unit 24 uses the soot data including the “a plurality of feature quantities relating to the soot” obtained as described above as the data item of the “position on the steel plate 10” where the soot having the feature quantity exists. The data is stored in the bag data storage unit 25 in association with the data (that is, linked).
The labeling / feature amount calculation unit 24 is realized, for example, when the CPU of the eyelid determination apparatus 6 executes a labeling process and a program for calculating the feature amount.
If the feature amount can be obtained, the contents of the process for obtaining the feature amount are not limited to those described above.

As will be described later, when the learning model 7 generates a learning model, the learning model 7 transmits the learning model to the determining apparatus 6. When the learning model acquisition unit 26 of the heel determination device 6 receives the learning model from the heel learning device 7, the learning model acquisition unit 26 outputs the learning model to the heel determination unit 27.
The learning model acquisition unit 26 is realized, for example, when the data input / output control device of the eyelid determination device 6 executes the above-described processing.

The wrinkle determination unit 27 applies the latest learning model output from the learning model acquisition unit 26 to the wrinkle candidate's wrinkle data stored in the wrinkle data storage unit 25 to obtain the final wrinkle for the wrinkle candidate. Make a decision. For example, if the wrinkle candidate is a roll wrinkle that affects the quality of the product, the wrinkle determination unit 27 determines that the wrinkle candidate is harmful. The cocoon determination unit 27 also determines the type of cocoon and the grade (score) of the cocoon as the cocoon type. On the other hand, if the wrinkle candidate is “oil, water droplets, light dirt, etc.” attached to the surface of the steel plate 10 and does not affect the quality of the product, the wrinkle determination unit 27 will remove the wrinkle candidate harmlessly. Is determined. Then, the eyelid determination unit 27 associates the result of the eyelid inspection with the eyelid data stored in the eyelid data storage unit 25.
The details of the learning model will be described together with the description of the eyelid learning device 7.

When the image data input as described above is subjected to a defect inspection, the defect determination unit 27 determines the result of the defect inspection (determination of harmful defect or harmless defect, defect type (such as defect type, defect grade, etc.)). The wrinkle detection image data indicating the determination result) and the feature amount obtained by the labeling / feature amount calculation unit 24 is generated and displayed on the display device 8. In the present embodiment, the wrinkle determination unit 27 generates wrinkle detection image data (map) in which an image imitating a wrinkle is superimposed on an image imitating the steel plate 10. Thereby, an inspector can be made to recognize visually what kind of wrinkles have occurred in which place of the steel plate 10. FIG.
The wrinkle determination unit 27 is realized, for example, when the CPU of the wrinkle determination device 6 executes a program for wrinkle determination.

疵 Data output determination unit 28 疵 stores the 疵 data stored in 疵 data storage unit 25 based on the operation state and date / time of the plating production line input from a process computer that controls and monitors the plating production line. It is determined whether or not it is time to output to the learning device 7. In the present embodiment, the saddle data output determination unit 28 is configured for the first coil of the day (steel wound after being plated in the plating production line) every time one month has elapsed from a predetermined date. Is determined to be output to the heel learning device 7. However, the timing at which the heel data is output to the heel learning device 7 and the content of the heel data to be output are not limited to this. For example, every time a coil is manufactured, wrinkle data about the coil may be output to the wrinkle learning device 7, or a limit of 100 may be provided continuously. Moreover, you may make it output the wrinkle data about the last coil of the day to the wrinkle learning apparatus 7 for every day.
The soot data output determination unit 28 is realized, for example, when the CPU of the soot determining device 6 determines that it is time to execute the soot data output program and output the soot data.

When the heel data output determination unit 28 determines that the heel data is to be output to the heel learning device 7, the heel data output unit 29 is set in advance among the heel data stored in the heel data storage unit 25. The obtained wrinkle data is transmitted to the wrinkle learning device 7. As described above, in this embodiment, the kite data output unit 29 transmits the kite data on the first coil of the day to the kite learning device 7 every time one month has elapsed from a predetermined date and time.
Here, in this embodiment, the eyelid data output from the eyelid data output unit 29 is not subjected to the eyelid inspection by the eyelid determination unit 27, and the result of the eyelid inspection is not associated with the eyelid data. Shall.
The heel data output unit 29 is realized, for example, when the data input / output control device of the heel determination device 6 executes the processing described above.

FIG. 3 is a block diagram illustrating an example of a functional configuration of the heel learning device 7.
In FIG. 3, the eyelid data acquisition unit 31 receives and stores the eyelid data transmitted from the eyelid determination device 6. When the wrinkle data is output to any of the teacher data generation unit 32 and the uncertain data extraction unit 38 described later, the wrinkle data is deleted. Specifically, the kite data acquisition unit 31 determines whether or not a learning model is stored in the learning model storage unit 36, and if no learning model is stored in the learning model storage unit 36, The wrinkle data transmitted from the determination device 6 is output to the teacher data generation unit 32. On the other hand, when the learning model is stored in the learning model storage unit 36, the bag data acquisition unit 31 outputs the bag data transmitted from the bag determination device 6 to the uncertain data extraction unit 38.
The cocoon data acquisition unit 31 is realized, for example, when the data input / output control device of the cocoon learning device 7 receives the cocoon data and the CPU performs the above-described determination and output processing. Here, the bag data transmitted from the bag determination device 6 is received, but it is not always necessary to do so. For example, the bag data stored in the removable storage medium owned by the user may be acquired.

  In the wrinkle learning device 7 of this embodiment, supervised learning is performed. Supervised learning is a method for generating (learning) a learning model by assigning “correct data (teacher data) related to species” to the kite data acquired by the kite data acquisition unit 31. It is. Here, the correct answer data is obtained, for example, as follows. First, an inspector visually confirms (assesses) a large number of images including various defects and the actual steel defects corresponding to these images, and determines the type, grade, and the like (model) of the defects. Based on the content determined in this way, “information on the cocoon type (type of cocoon, grade, etc.)” in the image is registered in the cocoon learning device 7 (computer) as correct data.

When no learning model is stored in the learning model storage unit 36, the teacher data generation unit 32 causes the display device 10 to display an image based on all the wrinkle data acquired by the wrinkle data acquisition unit 31. When this image is displayed, the inspector uses the user interface of the wrinkle determination device 7 to input correct data for a plurality of wrinkle data relating to the wrinkle type. The teacher data generation unit 32 assigns the input correct answer data to each of all the wrinkle data acquired by the wrinkle data acquisition unit 31 in association with the wrinkle data corresponding to the correct answer data.
On the other hand, when the learning model is stored in the learning model storage unit 36, the teacher data generation unit 32 is extracted by the uncertain data extraction unit 38 from the bag data acquired by the bag data acquisition unit 31. The correct data input as described above is attached to each of the wrinkle data in association with the wrinkle data corresponding to the correct data.
In the teacher data generation unit 32, for example, the CPU of the kite learning device 7 reads the kite data stored in the RAM or the like, performs the display and input processing described above, and associates the kite data with the correct answer data. This can be realized by storing in a RAM or the like.

The feature amount space arrangement unit 33 uses a coordinate space (feature amount space) with each feature amount as a coordinate axis of each data point (value of feature amount) of the “plurality of pieces of cocoon data relating to the species” to which correct data is assigned. ). When a partial feature amount space to be described later has already been generated, the feature amount space arrangement unit 33 also arranges data points of the eyelid data used when generating the partial feature amount space in the coordinate space.
FIG. 4 is a diagram illustrating an example of a state in which a plurality of data points of the cocoon data relating to the cocoon type are arranged in the coordinate space having the feature amounts as coordinate axes. In the following description, “arranging the data points of the eyelid data in the coordinate space” is referred to as “placement of the eyelid data in the coordinate space” as necessary. Also, in the following description, it is assumed that a plurality of pieces of bag data relating to two types of bottle types “CL1” and “CL2” are input from the bag data acquisition unit 31 as correct answer data, and the type of feature amount is 20 Suppose it is a seed. Further, in FIG. 4, it is assumed that the feature quantities f2 and f7 are standardized for convenience of explanation.
For example, the feature amount space arranging unit 33 generates a coordinate space (feature amount space) based on a combination of all feature amounts set in advance in the HDD or the like by the CPU of the eyelid learning device 7, and stores the RAM in the coordinate space. This can be realized by performing processing for arranging the data points of the eyelid data read from the memory and storing the result in the RAM or the like.

  When the feature amount combination determining unit 34 arranges the data points of the eyelid data to which the correct answer data of the species “CL1” and “CL2” are placed in the coordinate space by the feature amount space arranging unit 33, the data point It is determined whether or not the regions of the feature value values of the wrinkle data overlap with each other based on the arrangement position of. As a result of this determination, if the regions of the feature value values of the eyelid data to which the correct data of the species “CL1” and “CL2” are not overlapped with each other, the data points (feature amount Value) is determined as a combination of feature value values that can completely separate the correct data “CL1” and “CL2”.

That is, in the example shown in FIG. 4, in the coordinate space having the feature amounts f2 and f7 as the respective coordinate axes, the eyelid data relating to the eyelid “CL1” represented by white circles and the eyelid “CL2” represented by black circles.疵 data about and are arranged. The positions of the eyelid data of the eyelid types “CL1” and “CL2” do not match in the coordinate space. Accordingly, the species “CL1” and “CL2” can be completely separated by the respective regions of the feature amounts f2 and f7. That is, the grape types “CL1” and “CL2” can be specified by the combination of the values of the feature amounts f2 and f7.
The feature amount combination determination unit 34, for example, correct data of the types “CL1” and “CL2” based on the data points of the eyelid data arranged in the coordinate space (feature amount space) by the CPU of the eyelid learning device 7. If it is determined that the areas of the feature value values of the wrinkle data to which the wrinkles are assigned do not overlap each other, the combination of the values of the coordinate axes (each feature amount) of the coordinate space in which the wrinkle data is arranged is stored in the RAM or the like. Can be realized.

The learning model generating / updating unit 35 determines each feature value based on the combination of the feature value values determined by the feature value combination determining unit 34 and the types “CL1” and “CL2” which are correct data. One or a plurality of partial feature amount spaces for associating a pair of value ranges (that is, a region in the coordinate space) and the species are generated. Then, the learning model generation / update unit 35 stores the generated partial feature amount space in the learning model storage unit 36 as a learning model. When the learning model is not generated (when the learning model is not stored in the learning model storage unit 36), the learning model generation / update unit 35 stores the generated learning model in the learning model storage unit 36 as it is.
FIG. 5 is a diagram conceptually illustrating an example of a learning model.
In FIG. 5, as an example, the learning model 51 is generated as a set of the partial feature amount space 1, the partial feature amount space 2,..., The partial feature amount space L (L is an integer of 3 or more). Show.
Here, the partial feature amount space 1 is for associating a set of value ranges of the feature amounts f2 and f7 with the species “CL1” and “CL2”, and each of the feature amounts f2 and f7 is defined as a coordinate axis. To do. The partial feature amount space 2 is for associating a set of value ranges of the feature amounts f1, f3, and f7 with the species “CL1” and “CL2”. The feature amounts f1, f3, and f7 are coordinate axes. And The partial feature amount space L is for associating a set of value ranges of the feature amounts f4, f6, and f9 with the species “CL1” and “CL2”. The feature amounts f4, f6, and f9 are coordinate axes. And

On the other hand, when the learning model is generated (when the learning model is stored in the learning model storage unit 36), the learning model generation / update unit 35 uses the already stored learning model (partial feature amount space). Rewrite the generated learning model (partial feature space).
FIG. 6 is a diagram conceptually illustrating an example of how the learning model (partial feature amount space) is updated.

In the example shown in FIG. 6, the partial feature amount space 2 is deleted and the partial feature amount space L + 1 is added by this learning. That is, in the bag data obtained when generating the previous learning model, the bag types “CL1” and “CL2” can be completely separated by the feature amounts f1, f3, and f7, but when generating the current learning model, When the acquired wrinkle data is added, the wrinkle types “CL1” and “CL2” cannot be completely separated by the feature amounts f1, f3, and f7. Further, in the bag data acquired when generating the previous learning model, the bag types “CL1” and “CL2” could not be completely separated by the feature amounts f5, f8, and f9, but the current learning model is generated. When the soot data acquired at this time is added, the soot species “CL1” and “CL2” can be completely separated by the feature amounts f5, f8, and f9.
In the learning model generation / update unit 35, for example, the CPU of the eyelid learning device 7 reads out data of a combination of values of each coordinate axis (each feature amount) in the coordinate space in which the eyelid data is arranged from the RAM or the like, and from the read data This can be realized by generating a partial feature amount space and storing it in the learning model storage unit 36 that can be realized using an HDD or the like.

When a new learning model is stored in the learning model storage unit 36, the learning model output unit 37 transmits the learning model to the wrinkle determination device 6.
The learning model output unit 37 is realized, for example, when the data input / output control device of the kite learning device 7 executes such data output processing.

Here, an example of learning using the learning model (the eyelid determination unit 25 of the eyelid determination device 6 shown in FIG. 2) will be described.
FIG. 7 is a diagram conceptually illustrating an example of processing for determining the type “CL1” and “CL2” of each type of data. As shown in FIG. 7, among the preset value pairs of the feature amounts f1 to f20, a feature value set that completely separates the types “CL1” and “CL2” and the type “ A learning model is generated as a set of plural (L) partial feature amount spaces for associating CL1 ”and“ CL2 ”with each other.

In the example illustrated in FIG. 7, the eyelid determination unit 25 of the eyelid determination device 6 applies a learning model to each of the candidate eyelid data stored in the eyelid data storage unit 25 (that is, the eyelid data is converted into the learning model). In each of the partial feature amount spaces 1 to L), from the position of the data point of each of the heel data, the heel data is assigned to any of the types “CL1” and “CL2”. Extract information about the determination result of belonging.
In the present embodiment, the number of types determined as the type “C1” with respect to the total number of types “CL1” and “CL2” (that is, the total number of partial feature amount spaces) that is the determination result of the partial feature amount spaces 1 to L. When the ratio is equal to or higher than a certain threshold, for example, 50% or higher, the wrinkle determination unit 25 of the wrinkle determination device 6 determines that the wrinkle data to which the learning model is applied belongs to the species “CL1”. On the other hand, the number of partial feature amount spaces determined to be the type “CL1” with respect to the total number of types “CL1” and “CL2” (the total number of partial feature amount spaces) that are the determination results of the partial feature amount spaces 1 to L. When the ratio is less than a certain threshold value, for example, less than 50%, the eyelid determination unit 25 of the eyelid determination device 6 determines that the eyelid data to which the learning model is applied belongs to the species “CL2”.
As described above, in this embodiment, when the soot data is applied to the learning model, either the soot type “CL1” or “CL2” is always obtained from the soot data.

  The uncertain data extraction unit 38 reads the learning model stored in the learning model storage unit 36. Then, the uncertain data extraction unit 38 applies the learning model to each of all the wrinkle data acquired by the wrinkle data acquisition unit 31. Then, the uncertain data extraction unit 38 determines whether the bag data belongs to the type “CL1” or “CL2” from the position of the data point of each bag data in each of the partial feature amount spaces. To extract. Then, the uncertain data extraction unit 38 determines the partial feature “CL1” with respect to the total number of types “CL1” and “CL2” (total number of partial feature amount spaces) that is the determination result of the partial feature amount space. The number of quantity spaces is derived as the vote rate for the species “CL1”. The uncertain data extraction unit 38, when the voting rate for the species “CL1” is, for example, 30% or more and 70% or less, the species data to which the learning model is applied is the species type in the learning model in the feature amount space. It is assumed that the certainty data with respect to the determination result is low, and the certain data is a target to which correct data is assigned. On the other hand, the uncertain data extraction unit 38 assigns correct data to the cocoon data to which the learning model is applied when the vote rate for the cocoon type “CL1” is, for example, less than 30% or exceeds 70%. Excluded from the target.

FIG. 8 is a diagram conceptually illustrating an example of arrangement of wrinkle data in a feature amount space having the feature amounts fi and fj as coordinate axes.
In FIG. 8, the eyelid data indicated by white circles indicates the eyelid data belonging to the eyelid type “CL1”, and the eyelid data indicated by black circles indicates the data belonging to the eyelet type “CL2”. As shown in FIG. 8, when wrinkle data that is a discrimination target of wrinkles is arranged in a region 81 where haze data indicated by white circles and haze data indicated by black circles are mixed. The certainty (reliability) of the discrimination result of the learning model as to whether the data belongs to the kind “CL1” or the kind “CL2” becomes relatively low. Since such drought data is useful for updating the learning model, correct data is given.

On the other hand, as shown in FIG. 8, the wrinkle data to be discriminated for the wrinkle type is arranged in the areas 82 and 83 where the wrinkle data indicated by the white circles and the haze data indicated by the black circles are not mixed. In this case, since the certainty data has a relatively high certainty with respect to the discrimination result of the kind in the learning model, the influence on the determination of the partial feature amount space is small. Therefore, it is less necessary to give correct data and use it for updating the learning model. In addition, as will be described later, when correct data is given to such cocoon data and used for updating the learning model, the identification performance of the cocoon species decreases (see the description of FIG. 13 described in [0066]).
Therefore, in the present embodiment, as described above, the uncertain data extraction unit 38, for all of the cocoon data acquired by the cocoon data acquisition unit 31, in the feature amount space, The wrinkle data arranged in the processing region with a low certainty factor is extracted as wrinkle data used for updating the learning data. In FIG. 8, the region 81 near the identification boundary 84 of the species “Cl1” and “CL2” in the feature amount space is a region where the reliability (that is, the certainty factor) of the determination result of the learning model is relatively low. Although a case has been described as an example, such a region may exist other than near the boundary.

FIG. 9 is a diagram illustrating an example of the relationship between the vote rate for the type “CL1” and the number of pieces of data (number of data). In FIG. 9, the number of bag data (number of data) is standardized for convenience of explanation. In FIG. 9, black circles indicate the cocoon data confirmed (assessed) that the cocoon type is “CL2”, and white circles indicate the cocoon data confirmed (assessed) that the cocoon type is “CL1”.
The graph shown in FIG. 9 is obtained as follows. First, a learning model is applied to the obtained soot data, and the number of partial feature amount spaces that output the soot type “CL1” as a determination result is divided by the total number of partial feature amount spaces, so that the type “CL1” is obtained. The voting rate is derived. Then, the inspector visually confirms (assesses) that the type of the soot data is “CL1” or “CL2”. The derivation of the vote rate for the type “CL1” and the confirmation of the type are performed for all the obtained type data, and the type of the vote determined to be “CL1” at each vote rate. The number of data is obtained and the black circles shown in FIG. 9 are plotted. Similarly, at each vote rate, the number of bag data assessed as “CL2” is determined, and the black circles shown in FIG. 9 are plotted.
In the example illustrated in FIG. 9, when the vote rate for the species “CL1” is in the range of 30% or more and 70% or less, the species data evaluated as “CL1” and the species is “ A relatively large amount of soot data assessed as “CL2” is included. Therefore, here, the uncertain data extraction unit 38 extracts the soot data within this range as the soot data used for updating the learning data. However, the range of the voting rate for the grape type “CL1” to be extracted as the bag data used for the update of the learning data is not limited to the range of 30% or more and 70% or less. In the feature amount space, Any range may be used as long as the main point is to extract the wrinkle data arranged in the region where the certainty for the wrinkle type discrimination result in the learning model is low (considered). Specifically, the performance of the learning model is evaluated as shown in FIG. 13, and a range in which the highest species match rate can be obtained may be set. Moreover, in order to extract uncertain data, it is not always necessary to create the graph shown in FIG. That is, it is not necessary for the inspector to visually confirm (assess) whether the type of the bag data is “CL1” or “CL2”, and only the determination result when the learning model is applied is used as the flag data. It is a means to extract. In the present embodiment, when there are a plurality of seed species output from each partial feature amount space for a certain seed type, the minimum condition is to extract the corresponding seed data. Furthermore, in this embodiment, there are a plurality of species that are output from each partial feature amount space for a certain species, and the presence rate of a particular species among the plurality of species is within a predetermined range. It is an additional condition to extract the wrinkle data when it is within.

As described above, when the learning model is stored in the learning model storage unit 36, correct data is assigned to the teacher data generation unit 32 only for the wrinkle data extracted by the uncertain data extraction unit 38.
For example, the uncertain data extraction unit 38 reads the wrinkle data stored in the RAM or the like while the CPU of the wrinkle learning device 7 reads out the learning model stored in the HDD or the like, and selects a learning model for the wrinkle data. This can be realized by storing, in a RAM or the like, soot data whose vote rate for the soot type “CL1” is within a preset range in the HDD or the like as soot data used for updating the learning model.

Next, an example of the operation of the eyelid determination device 6 will be described with reference to the flowchart of FIG.
First, in step S <b> 1, the image input unit 21 determines whether image data is input from the imaging device 5. If the result of this determination is that image data has not been input from the imaging device 5, the operation proceeds to step S14 to be described later.
On the other hand, when image data is input from the imaging device 5, the process proceeds to step S <b> 2, and the labeling / feature amount calculation unit 24 performs processing on the image data processed by the shading correction unit 22 and the binarization unit 23. After performing the labeling process, the feature amount is calculated. Then, the labeling / feature amount calculation unit 24 associates (that is, links) the wrinkle data including the calculated feature amount with the data of “position on the steel plate 10” where the wrinkle having the characteristic amount exists. And stored in the bag data storage unit 25.

Next, in step S <b> 3, the heel data output determination unit 28 determines whether it is time to output the heel data stored in the heel data storage unit 25 to the heel learning device 7. As described above, in this embodiment, every time one month elapses from a predetermined day, the hail data about the first coil of the day (steel iron wound after being plated on the plating production line) Is output to the kite learning device 7. If it is determined that it is time to output the heel data to the heel learning device 7, the process proceeds to step S4. On the other hand, if it is not time to output the heel data to the heel learning device 7, the process skips step S4 and proceeds to step S5 described later.
In step S4, the kite data output unit 29 transmits to the kite learning device 7 all kite data obtained for the first coil on a predetermined day among the kite data stored in the kite data storage unit 25. To do. Then, the process proceeds to step S5.

In step S5, the wrinkle determination unit 27 determines whether to apply the learning model to the wrinkle data obtained in step S2. For example, when the learning model is not acquired or when the bag data is output in step S4, it is determined that the learning model is not applied to the bag data obtained in step S2. As a result of this determination, when the learning model is applied to the eyelid data obtained in step S2, the process proceeds to step S6. On the other hand, when the learning model is not applied to the eyelid data obtained in step S2, the process returns to step S1.
Generally, when collecting wrinkle data to be assessed in the initial adjustment stage of the apparatus, it is determined in step S5 that the learning model is not applied. In step S6, the wrinkle determination unit 27 selects one of the wrinkle data obtained in step S2.
Next, in step S <b> 7, the eyelid determination unit 27 applies the acquired learning model to the eyelid data selected in step S <b> 6, and determines the species “CL <b> 1” from each partial feature amount space included in the learning model. , “CL2” is obtained.

Next, in step S8, the eyelid determining unit 27 sets the number of species “CL1” to the total number of species “CL1” and “CL2” (that is, the total number of partial feature amount spaces) that is the execution result of step S4. Determine the percentage. As a result of this determination, when the ratio of the number of the seeds “C1” to the total number of the seeds “CL1” and “CL2” is 50% or more, the process proceeds to step S9, and the eyelid determination unit 27 performs step S6. It is determined that the selected soot data belongs to the soot type “CL1”. And it progresses to step S11 mentioned later.
On the other hand, if the ratio of the number of seeds “CL1” to the total number of seeds “CL1” and “CL2” is less than 50%, the process proceeds to step S10, and the eyelid determination unit 27 selects the soot selected in step S6. It is determined that the data belongs to the species “CL2”. Then, the process proceeds to step S11.

In step S11, the eyelid determination unit 27 associates the eyelid data selected in step S6 with the data of the eyelid type “CL1” or “CL2” determined in steps S9 and S10 for the eyelid data. And stored in the bag data storage unit 25.
Next, in step S12, the wrinkle determination unit 27 determines whether or not the wrinkle type has been determined for all of the wrinkle data obtained in step S2. If the result of this determination is that the soot type has not been determined for all the soot data, the process returns to step S6, and the soot determining unit 27 selects one unselected soot data. Then, the processes in steps S6 to S12 are repeated until the type of the soot is determined for all the soot data.
As described above, when all types of bag data obtained from the image data input in step S1 are determined, the process proceeds to step S13. In step S <b> 13, the wrinkle determination unit 27 generates wrinkle detection image data (map) obtained by superimposing images imitating a wrinkle and displays it on the display device 8. And the process by the flowchart of FIG. 10 is complete | finished.

If it is determined in step S1 that image data has not been input from the imaging device 5, the process proceeds to step S14. In step S <b> 14, the learning model acquisition unit 26 determines whether a learning model has been acquired from the kite learning device 7. If the result of this determination is that a learning model has not been acquired, processing returns to step S1.
On the other hand, if the learning model has been acquired, the process proceeds to step S15, where the learning model acquisition unit 26 has already registered a learning model of the same type as the learning model determined to have been acquired in step S14. Determine.

As a result of this determination, if a learning model of the same type as the learning model determined to have been acquired in step S14 has not been registered, the process proceeds to step S16. In step S16, the eyelid determination unit 27 newly registers the learning model acquired in step S16. Then, the process returns to step S1.
On the other hand, if a learning model of the same type as the learning model determined to have been acquired in step S14 has already been registered, the process proceeds to step S17. In step S17, the eyelid determination unit 27 rewrites the registered learning model with the learning model determined to have been acquired in step S14. Then, the process returns to step S1.

Next, an example of the operation of the heel learning device 7 will be described with reference to the flowchart of FIG.
First, in step S <b> 101, the eyelid data acquisition unit 31 waits until it receives eyelid data transmitted from the eyelid determination device 6. In step S101, all the eyelid data obtained for the first coil on a predetermined day are received at a time. When the bag data is received, the process proceeds to step S102. In step S102, the eyelid data acquisition unit 31 determines whether or not a learning model is stored in the learning model storage unit 36. If the learning model is stored in the learning model storage unit 36 as a result of this determination, the process proceeds to step S110 described later.
On the other hand, when the learning model is not stored in the learning model storage unit 36, the process proceeds to step S103. In step S103, the teacher data generation unit 32 assigns correct data to each of all the bag data received in step S101.
Next, in step S104, the feature quantity space arrangement unit 33 selects one combination of a plurality of feature quantities.

Next, in step S105, the feature quantity space arrangement unit 33 selects each data point (feature quantity value) of the “plurality of soot data relating to the soot species” to which the correct answer data is assigned in step S104. Each feature amount is arranged in a coordinate space (feature amount space) having coordinate axes. Then, the feature amount space arrangement unit 33 determines whether or not the areas of the respective data points (feature amount values) of the bag data overlap each other. As a result of the determination, if the areas of the respective data points (feature value values) of the eyelid data overlap with each other, the process skips step S106 and proceeds to step S107 described later.
On the other hand, if the areas of the respective data points (feature value values) of the eyelid data do not overlap each other, the process proceeds to step S106. In step S106, the feature amount combination determination unit 34 completely sets the combinations of the data points (feature amount values) arranged in the coordinate space in step S105 as the correct data “CL1” and “CL2”. It is determined that the combination of feature value values that can be separated. Then, the feature amount combination determination unit 34 stores the combination of the data points (feature amount values) arranged in the coordinate space in step S105.

Next, in step S107, the feature amount space arrangement unit 33 determines whether all feature amount combinations have been selected. If the result of this determination is that not all feature value combinations have been selected, processing returns to step S104, and the feature value space layout unit 33 selects one unselected feature value combination. Then, the processes in steps S104 to S107 are repeated until the process for all combinations of feature amounts is completed.
When the process is completed for all combinations of feature amounts, the process proceeds to step S108. In step S108, the learning model generating / updating unit 35 determines each feature quantity based on the combination of the feature quantity values stored in step S106 and the types “CL1” and “CL2” that are correct answer data. One or a plurality of partial feature amount spaces for associating a set of values (that is, a region in the coordinate space) and a seed type with each other are generated. Then, the learning model generation / update unit 35 stores the generated partial feature amount space in the learning model storage unit 36 as a learning model.

Next, in step S109, the learning model output unit 37 transmits the learning model generated in step S108 to the wrinkle determination device 6. Then, the process returns to step S101.
If it is determined in step S102 that the learning model is stored in the learning model storage unit 36, the process proceeds to step S110. In step S110, the uncertain data extraction unit 38 selects one of all the wrinkle data received in step S101.
Next, in step S111, the uncertain data extraction unit 38 applies the learning model to the wrinkle data selected in step S110.
Next, in step S112, the uncertain data extraction unit 38 derives the vote rate VR for the kind “CL1” based on the result of applying the learning model in step S111, and votes for the kind “CL1”. It is determined whether or not the rate VR is 30% or more and 70% or less. As a result of this determination, if the vote rate VR for the species “CL1” is not 30% or more and 70% or less, the symbol data selected in step S110 is not the symbol data useful for updating the learning data. Step S113 is omitted and the process proceeds to step S114 described later.

On the other hand, if the vote rate VR for the species “CL1” is not less than 30% and not more than 70%, the process proceeds to step S113. If it progresses to step S113, the uncertain data extraction part 38 will memorize | store temporarily the haze data selected at step S110 as haze data useful for the update of learning data. Then, the process proceeds to step S114.
In step S114, the uncertain data extraction unit 38 determines whether all the wrinkle data received in step S101 has been selected. As a result of this determination, if not all the bag data received in step S101 have been selected, the process returns to step S110, and the uncertain data extraction unit 38 selects one unselected bag data. And the process of step S110-S114 is repeatedly performed until a process is complete | finished about all the eyelid data received by step S101.

When the processing is completed for all the eyelid data received in step S101, the process proceeds to step S115, and the teacher data generation unit 32 assigns correct data to each of the eyelid data stored in step S113.
Next, in step S116, the feature quantity space arrangement unit 33 selects one combination of a plurality of feature quantities.
Next, in step S117, the feature amount space arranging unit 33 selects each data point (feature amount value) of the “plurality of soot data relating to the soot type” to which the correct data is assigned in step S115. The feature amount is arranged in a coordinate space (feature amount space) having coordinate axes. As described above, in this step S117, in addition to the data point (feature value) of the wrinkle data to which the correct answer data is given in step S115 this time, the correct answer data is given in steps S103 and S115 so far. The data points (feature value) of the eyelid data are also arranged in the coordinate space (feature value space). Then, the feature amount space arrangement unit 33 determines whether or not the areas of the respective data points (feature amount values) of the bag data overlap each other. As a result of the determination, if the areas of the respective data points (feature value values) of the eyelid data do not overlap each other, the process proceeds to step S120 described later.

  On the other hand, if the areas of the respective data points (feature value values) of the eyelid data overlap each other, the process proceeds to step S118. In step S118, the feature amount combination determination unit 34 determines whether or not the combination of the feature amounts selected in step S116 is a combination registered as a partial feature amount space. If the result of this determination is that the combination of feature quantities selected in step S116 is not a combination registered as a partial feature quantity space, it is not necessary to delete the partial feature quantity space, so step S119 is omitted and will be described later. Proceed to step S122. On the other hand, if the combination of the feature amounts selected in step S116 is a combination registered as a partial feature amount space, the process proceeds to step S119, and the feature amount combination determination unit 34 determines that it is registered in step S118. The partial feature amount space thus obtained is deleted from the learning model. In the example shown in FIG. 6, the partial feature amount space 2 is deleted in this step S119. And it progresses to step S122 mentioned later.

  If it is determined in step S117 that the areas of the respective data points (feature value values) of the eyelid data do not overlap with each other, the process proceeds to step S120. In step S120, the feature amount combination determination unit 34 determines whether or not the combination of the feature amounts selected in step S116 is a combination registered as a partial feature amount space. If the result of this determination is that the combination of feature quantities selected in step S116 is a combination registered as a partial feature quantity space, there is no need to add a partial feature quantity space, so step S121 is omitted and will be described later. The process proceeds to step S122. On the other hand, if the combination of feature amounts selected in step S116 is not a combination registered as a partial feature amount space, the process proceeds to step S121, and the feature amount combination determination unit 34 determines that it is registered in step S118. Added partial feature space to the learning model. In the example shown in FIG. 6, the partial feature amount space L + 1 is added in this step S121. Then, the process proceeds to step S122.

In step S122, the feature amount space arrangement unit 33 determines whether all combinations of feature amounts have been selected. If the result of this determination is that not all feature quantity combinations have been selected, the process returns to step S116, and the feature quantity space layout unit 33 selects one unselected feature quantity combination. Then, the processes in steps S116 to S122 are repeated until the process is completed for all the feature value combinations.
When the processing is completed for all the feature value combinations, the process proceeds to step S123. In step S123, the learning model generation / update unit 35 updates the learning model based on the results of steps S119 and S121, and stores the updated learning model in the learning model storage unit 36.
Next, in step S124, the learning model output unit 37 transmits the learning model updated in step S123 to the wrinkle determination device 6. Then, the process returns to step S101.

FIG. 12 is a diagram illustrating an example of the number of heel data (number of data) used for generation / update of the learning model in each month. In FIG. 12, it is the result of investigating whether the wrinkles which have arisen in the plated steel plate 10 manufactured with the plating production line are two types of types "CL1" and "CL2." Here, 20 types of feature values are set. In addition, about 600 pieces of cocoon data are newly input to the cocoon learning device 7 in each month.
In FIG. 12, as in this embodiment, the blacked-out graph is used to update the learning model only for the wrinkle data having a low certainty with respect to the determination result of the wrinkle type in the learning model except the first month (January) Indicates the number of data when used. On the other hand, the open graph shows the number of data when all the wrinkle data is used for updating the learning model in all months. As shown in FIG. 12, according to the present embodiment, it can be seen that the more the learning model is updated, the more the number of wrinkle data used for the update can be reduced. In the example shown in FIG. 12, in August, according to the present embodiment, the number of wrinkle data used for updating is reduced to about 1/3 compared to the case where all the wrinkle data is used for updating the learning model. I was able to.

FIG. 13 is a diagram illustrating an example of the sow seed match rate in each month. FIG. 13 shows the result of processing the same soot data as that obtained from the result shown in FIG. Here, the soot seed match rate indicates the rate at which the result of determining the soot seed by the soot determining device 6 for a certain soot matches the result of visual inspection of the soot by the inspector.
In FIG. 13, as in the present embodiment, in the graph 131, except for the first month (January), only the wrinkle data with low confidence in the wrinkle type discrimination result in the learning model is used for updating the learning model. In this case, the species match rate is shown. On the other hand, the graph 132 shows the species match rate when all the cocoon data are used for updating the learning model in all the months. As shown in FIG. 13, it can be seen that, according to the present embodiment, it is possible to improve the identification performance of the grape species. This is presumably because when all the wrinkle data is used for updating the learning model, the learning model becomes over-learned and the generalization of the wrinkle data with the unknown hail species is inferior. In addition, if there is too much sputum data to which correct data is added, it is considered that there is a high possibility that an error will occur in correct data set by the inspector for the sputum data. In particular, in the feature amount space, if there is an error in the correct data set for the cocoon data that is placed in the region where the certainty of the discrimination result of the species in the learning model is high, the region with the low certainty This is considered to have a more adverse effect on the growth of the learning model than when there is an error in the correct answer data set for the arranged trap data.

  As described above, in this embodiment, a plurality of feature value ranges that completely separate the species “CL1” and “CL2” and a plurality of species for associating the species “CL1” and “CL2” with each other. A learning model is generated as a set of L (L) partial feature amount spaces. Then, the learning model is updated using only the cocoon data having a low certainty of the discrimination result of the cocoon type in the learning model in the feature amount space among the input cocoon data. Therefore, it is possible to automatically select soot data useful for updating the learning model. Thereby, the correct data setting work by the inspector can be reduced, and erroneous correct data can be prevented from being set in the bag data. Therefore, the kind discrimination | determination performance in the eyelid determination apparatus 6 can be improved (that is, the reliability of a learning model can be improved), and the performance of the eyelid determination apparatus 6 can be automatically maintained over a long period of time. Is possible. As a result, the wrinkle determination device 6 operates stably, and the function of the wrinkle determination device 6 is exhibited over the entire steel sheet in the long term.

In the present embodiment, the case where the plated steel sheet 10 manufactured in the plating production line is a flaw inspection object has been described as an example, but the flaw inspection object is any metal as long as it is a metal. There may be.
Further, in the present embodiment, the case where the type of grape seed is “2” and the type of feature amount is “20” has been described as an example, but the kind of grape seed may be 3 or more, The number of feature amounts may be any number as long as it is two or more.

  Further, in the present embodiment, the case where the bag data used when updating the learning model is extracted based on the vote rate for the bag type “CL1” has been described as an example. However, the criterion for extracting the eyelid data used when updating the learning model is not limited to this. For example, when an algorithm that discriminates the species by the distance D from the species identification boundary in the feature amount space, such as a support vector machine (SVM), is used as the learning model, for example, as follows. be able to. That is, within the range of −X <D <X (where X is a positive number) in the input soot data (“spot data whose kind of soup is unknown” to be used for determining whether or not to use in the learning model)疵 data in the feature amount space is 疵 data with low confidence in the discrimination result of the モ デ ル type in the learning model, and only this 疵 data among the input 疵 data is used for updating the learning model Can do. In addition, as a learning model, as in the nearest neighbor method, the species is unknown based on the species and number of learning data that exists in the vicinity of the species whose species is unknown (the species data given species). In the case of using an algorithm for determining the type of soot data, for example, the following can be performed. That is, in the k nearest neighbor method, k cases are found in order from the input soot data. The soot data input to the class occupying the largest number among the k cases is classified. Here, if the ratio of the number of classes CL1 and CL2 in the k data is close, it can be said that the certainty level is low. It is possible to obtain cocoon data having a low certainty for the discrimination result of the cocoon type in the learning model and use only the cocoon data among the inputted cocoon data for updating the learning model.

In the present embodiment, for example, an example of an acquisition unit is realized by using the eyelid data acquisition unit 31 of FIG. 2, and an example of an assignment unit is realized by using the teacher data generation unit 32, and feature amount space arrangement By using the unit 33, the feature amount combination determination unit 34, and the learning model generation / update unit 35, an example of a generation unit is realized. Further, for example, by using the uncertain data extraction unit 38, an example of an extraction unit is realized. Further, for example, an example of the updating unit is realized by using the feature amount space arranging unit 33, the feature amount combination determining unit 34, and the learning model generating / updating unit 35.
Further, for example, an example of an acquisition process is realized by executing the process of step S101 in FIG. 11, and an example of an adding process is realized by executing the processes of steps S103 and S115, and steps S104 to S108 are performed. By executing the processing, an example of the generation process is realized. Further, for example, an example of the extraction process is realized by executing the processes of steps S110 to S114 in FIG. 11, and an example of the update process is realized by executing the processes of steps S116 to S123.
An example of the specific type is the type “CL1”, and an example of the predetermined range is 30% ≦ VR ≦ 70% (VR is a vote rate).

The embodiment of the present invention described above can be realized by a computer executing a program. Further, a means for supplying the program to the computer, for example, a computer-readable recording medium such as a CD-ROM recording such a program, or a transmission medium for transmitting such a program may be applied as an embodiment of the present invention. it can. A program product such as a computer-readable recording medium that records the program can also be applied as an embodiment of the present invention. The programs, computer-readable recording media, transmission media, and program products are included in the scope of the present invention.
In addition, the embodiments of the present invention described above are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. Is. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

DESCRIPTION OF SYMBOLS 1 Strip | belt-shaped steel plate 2 Conveyance roll 3 Rotary encoder 4 Illumination device 5 Imaging device 6 疵 determination device 7 疵 Learning device 8, 9 Display device 10 Plated steel plate

Claims (5)

An acquisition means for acquiring a plurality of wrinkle data each including a plurality of feature amounts of wrinkles as data items from the image of the inspection object;
A granting unit that grants certain types of correct answer data to each of the trap data based on an operation of the input device by a user;
The feature value values of the wrinkle data acquired by the acquisition unit, the plurality of feature values of each of the plurality of wrinkle data to which the correct data is added by the adding unit, and the plurality of feature amounts respectively. Generating means for generating a learning model for discriminating the species of the eyelid data from the values of a plurality of feature amounts of the eyelid data based on the result of arrangement in the feature amount space as each coordinate axis;
Among the cocoon data acquired by the acquisition unit, cocoon data having a low certainty for the discrimination result of the cocoon type in the learning model is extracted in a feature amount space having each of the plurality of feature amounts of the heel data as coordinate axes. Extraction means;
Updating means for updating the learning model generated by the generating means,
When the updating unit updates the learning model by the updating unit, among the cocoon data acquired by the acquiring unit, the cocoon data extracted by the extracting unit is used as a kind of correct answer data. Based on the operation of the input device by the user,
The updating means is a feature value value of the wrinkle data extracted by the extracting means, and a plurality of feature value values of the wrinkle data to which correct data is given by the giving means, and generation of the learning model already The learning model is updated based on a result of arranging the values of the plurality of feature amounts of the wrinkle data used in the above in a feature amount space having each of the plurality of feature amounts as respective coordinate axes.疵 Learning device. The learning model is a plurality of partial feature amount spaces for associating a set of feature value ranges that can separate a plurality of species and the plurality of species, each of which includes Given a set of feature values, a plurality of partial features that output a single species as a discrimination result of the species based on the position in the partial feature space of the feature value set of the relevant data Have quantity space,
The extracting means provides a plurality of feature value values of the eyelid data acquired by the acquiring means to the plurality of partial feature amount spaces, so that a plurality of species that are output as discrimination results from the respective partial feature amount spaces are provided. And when the number of specific types of the plurality of types is within a predetermined range with respect to the total number of the partial feature amount spaces, the type of the type in the learning model is determined. The wrinkle learning device according to claim 1, wherein the wrinkle learning device extracts the wrinkle data having a low certainty about the result. An acquisition step of acquiring a plurality of wrinkle data each including a plurality of feature amounts of wrinkles as data items from the image of the inspection object;
A granting step for granting each kind of correct answer data, based on the operation of the input device by the user, to each of the saddle data;
The feature value values of the wrinkle data acquired by the acquisition step, wherein the plurality of feature values of each of the plurality of wrinkle data to which the correct data is given by the adding step A generation step for generating a learning model for discriminating the species of the cocoon data from the values of the plurality of feature values of the cocoon data based on the result of arrangement in the feature amount space as each coordinate axis;
Of the cocoon data acquired by the acquisition step, cocoon data having a low certainty with respect to the discrimination result of the cocoon type in the learning model is extracted in the feature amount space having each of the plurality of feature amounts of the heel data as coordinate axes. An extraction process;
Updating the learning model generated by the generating step, and
In the granting step, when updating the learning model by the updating step, among the cocoon data acquired by the obtaining step, among the cocoon data extracted by the extracting step, 疵 kinds of correct answer data, Based on the operation of the input device by the user,
The update step is a feature value value of the wrinkle data extracted by the extraction step, and a plurality of feature value values of the wrinkle data to which correct data is given by the adding step, and generation of the learning model already The learning model is updated based on a result of arranging the values of the plurality of feature amounts of the wrinkle data used in the above in a feature amount space having each of the plurality of feature amounts as respective coordinate axes.疵 Learning method. The learning model is a plurality of partial feature amount spaces for associating a set of feature value ranges that can separate a plurality of species and the plurality of species, each of which includes Given a set of feature values, a plurality of partial features that output a single species as a discrimination result of the species based on the position in the partial feature space of the feature value set of the relevant data Have quantity space,
In the extracting step, a plurality of types of soot species output as discrimination results from each partial feature amount space are provided by providing a set of feature value values of the soot data acquired in the acquiring step to the plurality of partial feature amount spaces. And when the number of specific types of the plurality of types is within a predetermined range with respect to the total number of the partial feature amount spaces, the type of the type in the learning model is determined. The cocoon learning method according to claim 3, wherein the cocoon learning method is extracted as cocoon data having a low certainty about the result. An acquisition step of acquiring a plurality of wrinkle data each including a plurality of feature amounts of wrinkles as data items from the image of the inspection object;
A granting step for granting each kind of correct answer data, based on the operation of the input device by the user, to each of the saddle data;
The feature value values of the wrinkle data acquired by the acquisition step, wherein the plurality of feature values of each of the plurality of wrinkle data to which the correct data is given by the adding step A generation step for generating a learning model for discriminating the species of the cocoon data from the values of the plurality of feature values of the cocoon data based on the result of arrangement in the feature amount space as each coordinate axis;
Of the cocoon data acquired by the acquisition step, cocoon data having a low certainty with respect to the discrimination result of the cocoon type in the learning model is extracted in the feature amount space having each of the plurality of feature amounts of the cocoon data as coordinate axes. An extraction process;
Updating the learning model generated by the generating step, and causing the computer to execute,
In the granting step, when updating the learning model by the updating step, among the cocoon data acquired by the obtaining step, among the cocoon data extracted by the extracting step, 疵 kinds of correct answer data, Based on the operation of the input device by the user,
The update step is a feature value value of the wrinkle data extracted by the extraction step, and a plurality of feature value values of the wrinkle data to which correct data is given by the adding step, and generation of the learning model already The learning model is updated based on a result of arranging the values of the plurality of feature amounts of the wrinkle data used in the above in a feature amount space having each of the plurality of feature amounts as respective coordinate axes. Computer program.

Images