JP2020177429A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

To make it possible to prevent deterioration in accuracy in identifying a non-defective item.SOLUTION: An information processing apparatus determines a parameter of a discriminator that estimates the state of the appearance of an object from data obtained by measuring the object, and comprises: acquisition means that acquires first estimation data estimated by inputting, to a first discriminator, first measurement data obtained by measuring the object in a first environment, and second estimation data estimated by inputting, to a second discriminator, second measurement data obtained by measuring the object in a second environment different from the first environment; and determination means that, based on training data including a label showing the state of the appearance of the object, the first estimation data, and the second estimation data, determines an updated parameter of the first discriminator. The determination means determines the parameter that is updated to make the difference between a result of estimation included in the first estimation data and a result of estimation included in the second estimation data smaller than a predetermined threshold.SELECTED DRAWING: Figure 1

Description

The present invention relates to determining parameters of a classifier that estimates the appearance of an object.

There is an increasing need for automation in visual inspection to determine the quality of parts and products. As an automation technique, for example, a technique for separating a non-defective product or a defective product from a video / image data obtained by capturing a signal by a line sensor, an area sensor, or the like by a pre-defined classifier is known. Specifically, a score (evaluation value) for determining whether the product is non-defective or defective is calculated by using a discriminator for the video / image data. The calculated score is compared with the threshold value as a criterion to identify non-defective products and defective products. Conventionally, development has been made for the purpose of eliminating variations in judgments and oversights by humans by automating inspections that have been performed manually. This is because the inspection is performed with high accuracy and the same standard.

However, even if the same part is inspected, the identification result may differ due to individual differences for each device, changes over time, changes in the surrounding environment such as lighting conditions, and the like. Therefore, it is necessary to determine so that there is no difference in the identification result in the device in which the individual difference for each device or the surrounding environment has changed.

As a method of determining the difference between the devices, for example, there is a method of Patent Document 1. The same defective product sample data is inspected by a plurality of devices, and the imaging parameter of the inspection device or the setting parameter of the device is adjusted so that the difference between the inspection result and the reference device is within the reference value.

Japanese Unexamined Patent Publication No. 2004-144685

However, in Patent Document 1, since adjustment is performed for the same defective product sample data, the parameters are suitable only for the same defective product sample data. As a result, there is a possibility of performance deterioration in which non-defective products and defective products that could be originally identified cannot be identified.

The purpose of this proposal is to suppress deterioration of the identification accuracy of non-defective products even when the parameters of the classifier are determined between a plurality of devices or in devices whose environment has changed.

The information processing device according to the present invention that solves the above problems is an information processing device that determines parameters of a classifier that estimates the appearance state of the object from the data obtained by measuring the object, and the object in the first environment. The first estimation data estimated by inputting the first measurement data obtained by measuring the above to the first classifier and the second measurement data obtained by measuring the object in a second environment different from the first environment are second. The second estimation data estimated by inputting to the classifier, the acquisition means for acquiring the acquisition means, the training data including the label indicating the appearance state of the object, the first estimation data, and the second estimation data. Based on this, the determination means for determining the updated parameters of the first classifier is provided, and the determination means defines a difference between the estimation results included in the first estimation data and the second estimation data. It is characterized in that the parameter updated to be smaller than the threshold value is determined.

It is possible to suppress deterioration of the identification accuracy of non-defective products even when the parameters of the classifier are determined between a plurality of devices or in devices whose environment has changed.

Block diagram showing an example of functional configuration of an information processing system The figure which shows the hardware configuration example of an information processing apparatus Flow chart explaining the processing executed by the information processing device Example of information stored in the second storage unit 107 Flow chart explaining the processing executed by the information processing device Flow chart explaining the processing executed by the information processing device Diagram showing an example of GUI Block diagram showing a functional configuration example of an information processing device Flow chart explaining the processing executed by the information processing device Block diagram showing a functional configuration example of an information processing device Conceptual diagram of a device that inspects the appearance of an object

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Embodiment 1>
In the first embodiment, a case where the discriminator parameter is determined by utilizing the difference in the discrimination result due to the individual difference between the two devices will be described as an example.

First, the outline of the present embodiment will be described. For the two inspection devices, the device 1 and the device 2, which perform the inspection using the predefined classifier, the classifier parameters are determined so as to reduce the individual difference between the devices. The concept will be described with reference to FIG. The information processing device 11 adjusts the parameters of each classifier of the inspection device that inspects the appearance state of the object. The imaging device 110 and the lighting device 14 are included in the inspection device 1. The object 15 is an object to be inspected, for example, a product or a part. Here, inspecting the appearance of an object for checking whether the surface of a finished product or part is scratched or dirty in a product production line will be described as a specific example. In this inspection, a product without scratches or stains is detected as a normal product, and a product with scratches or stains is detected as an NG product with an image. In the present embodiment, the appearance inspection of the object is actually carried out after completing the parameters of the classifier and the internal parameters of the device (calibration of the imaging device, etc.) for each inspection device in advance. During the inspection, the processing described below is carried out in order to confirm whether the accuracy of the classifier has deteriorated and whether there is a large difference in the identification results of each device. For devices that output incorrect estimation results, update the parameters of the classifier so that the estimation results of each device do not vary. First, the same sample (object) is used in the inspection device 1 and the inspection device 2, and an inspection using a classifier is performed for each device. The inspection result for the measurement data obtained by measuring the sample for each device is saved in the information processing device. The identification results obtained by inputting the measurement data obtained by measuring the same sample in each device into the classifier are compared. If there is a difference in the compared discrimination results, the parameters of the classifier in both or one device are determined so as to reduce the difference. By aligning the identification results indicating the appearance state of the object before and after the change, it is possible to estimate the normality even for unknown features. Hereinafter, specific embodiments will be described.

In the first embodiment, the appearance inspection of the same type of sample (object) is performed by two different inspection devices. Of the two inspection devices, the classifier 2 of the test device 2 is a model that has been sufficiently learned under the environment in which it is currently arranged, but the classifier 1 of the test device 1 has changed slightly due to, for example, a slight movement. In this state, learning is not enough. In such a case, it is necessary for the inspection device 1 to adjust all the device parameters and the like from 0, or to create training data again to relearn the classifier. However, by adjusting the parameters of the discriminator using the estimation result of the inspection device 2 and the training data used in the original learning, the parameters of the discriminator were determined between a plurality of devices or in a device in which the environment changed. Even in this case, deterioration of the identification accuracy of non-defective products can be suppressed.

FIG. 1 is a block diagram showing a functional configuration example of the information processing system 1000. The information processing system 1000 includes inspection devices 1 and 2 and an information processing device 11. Inspection devices 1 and 2 are devices that perform inspections for identifying the presence or absence of abnormalities in a sample (object). That is, the inspection devices 1 and 2 refer to inspection devices that estimate the state of an object from the measured data of the object. For example, it is used in a factory for visual inspection to check for scratches on a finished product.

The first storage unit 100 holds training data obtained by measuring a training sample used when the classifier of the inspection device is trained. The first storage unit 100 may be provided outside the inspection devices 1 and 2 or inside each device. Specifically, the training data is image data including an image obtained by capturing an image of the training sample and positive and negative labels (here, normal label and abnormal label). Here, the training data is used when the (first) classifier of the estimation unit 105 is trained, but any data may be used as long as it has a correct positive / negative label. The training data may be a color or monochrome image captured by the imaging device, a distance image measured by a distance sensor, or the like.

The image pickup devices 110 and 110'capture the sample (object). Here, mainly, in order to determine the difference in the identification results by the classifiers performed by the inspection devices 1 and 2, the data obtained by measuring the sample is stored as the first measurement data and the second measurement data, respectively. The first measurement data or the second measurement data may or may not have labels such as good / bad and defect type. The image (measurement data) obtained by imaging is input to the image acquisition unit 103 (103'). Depending on the type of inspection, a measuring device (for example, a distance sensor) other than the imaging device may be used.

The inspection device 1 is an inspection device having a data acquisition unit 102, an image acquisition unit 103, an extraction unit 104, an estimation unit 105, and a learning unit 106. The processing performed by each functional configuration of the inspection devices 1 and 2 will be described later. Further, although the inspection device 101'has the same configuration as that of the inspection device 101 in FIG. 1, the configuration may be different as long as it is an inspection device for identifying using a discriminator.

The information processing device 11 determines the parameters of the classifier using the inspection results of each inspection device. The acquisition unit 111 acquires the identification result of the classifier based on the measurement data (features extracted from) of the sample measured from each of the inspection device 1 and the inspection device 2. The acquisition unit 111 acquires the first data estimated by inputting (features) the first measurement data obtained by measuring the target object in the inspection device 1 to the first classifier. Similarly, the second data estimated by inputting (features) the second measurement data obtained by measuring the target object in the inspection device 2 to the second classifier is acquired from the estimation unit 105'. The association unit 112 associates the unique data that can uniquely identify the sample (object) with the first estimation data estimated by the estimation unit 105. Further, the association unit 112 associates the first estimated data estimated by the estimation unit 105 with the second estimated data based on the unique data. The comparison unit 113 compares the estimation results (first estimation data and second estimation data associated with the unique data) by the classifier for each inspection device stored in the second storage unit 117. The comparison unit 113 may be arranged in the information processing device 11, or may be arranged in each of the inspection devices 1 and 2. The determination unit 114 determines the parameters of the estimation unit 105 or the classifier of the estimation unit 105'. The parameter of the discriminator referred to here refers to the weighting coefficient of the network when the discriminator is a deep neural network. The output unit 115 outputs the comparison result of the identity of the estimated values such as the difference in the score and the difference in the identification result. The input unit 116 accepts input by the user and inputs the information input to each functional configuration. The second storage unit 117 stores the estimation results (that is, the first estimation data, the second estimation data, and the unique data) by the discriminator for each inspection device.

FIG. 2 is a diagram showing a hardware configuration example of the information processing device 11. The central processing unit (CPU) H01 uses the RAM H03 as a work memory to read and execute the OS and other programs stored in the ROM H02 and the storage device H04, and controls each configuration connected to the system bus H09 to control various configurations. Performs processing operations and logical judgments. The process executed by the CPU H01 includes the information processing of the embodiment. The storage device H04 is a hard disk drive, an external storage device, or the like, and stores programs and various data related to the image recognition process of the embodiment. The input unit H05 is an image pickup device such as a camera, an input device such as a button for inputting a user instruction, a keyboard, and a touch panel. The storage device H04 is connected to the system bus H09 via an interface such as SATA, and the input unit H05 is connected to the system bus H09 via a serial bus such as USB, but the details thereof will be omitted. Communication I / FH06 communicates with an external device by wireless communication. The display unit H07 is a display. The sensor H08 is an image sensor or a distance sensor. H08 corresponds to the imaging device 101 (101').

Hereinafter, the processing executed by each functional configuration of the inspection device will be described. The data acquisition unit 102 will be described. In this embodiment, it is premised that data obtained from the same sample is used between the devices. In this embodiment, data obtained from the same sample between the two devices is used. If the data used can be matched between the devices, the identification result of the classifier can be determined for each sample, so that the parameters of the classifier can be determined with higher accuracy.

The data acquisition unit 102 acquires unique data that can uniquely identify a sample (object) included in the first measurement data or the second measurement data. The unique data includes a serial number uniquely assigned to each sample, a distribution of feature amounts, an object fingerprint that appears uniquely to an object, and the like as information that can uniquely identify a sample. If the sample itself does not have information that can be uniquely identified, there is also a method of preparing a case for storing the sample and adding information that gives the case uniqueness. By doing so, it is possible to reliably associate the samples between the devices. In the present embodiment, a case where the serial number of each sample is read will be described as an example. As long as the method can identify the individual, a method using barcode authentication or a wireless method such as RFID may be used.

The image acquisition unit 103 acquires the first measurement data including the result of measuring the sample (target object). That is, an image for inspection is obtained by taking a sample for determining the difference and a sample to be inspected. As the image for inspection obtained here, the captured image data may be used as it is, or image conversion such as Haar wavelet conversion may be performed. The type of image can be changed according to the purpose of the inspection. For example, an RGB color image, a black-and-white image, or a grayscale grayscale image may be used. In addition to the image captured by the camera, an infrared photograph taken by an infrared camera or a distance image obtained by an active distance sensor typified by Lidar or ToF may be used. The image acquisition units 103 and 103'may be provided outside the inspection devices 1 and 2. Similarly, the image acquisition unit 103'in the inspection device 2 acquires an image (second measurement data) of the target object captured by the image pickup device 110'. It should be noted that the image pickup device 110 and the image pickup device 110'can not always take pictures under exactly the same conditions. The first measurement data and the second measurement data may be common data.

The extraction unit 104 extracts (image) features by a predetermined method based on the first measurement data. Data features are extracted based on the image data obtained by the image acquisition unit 103. As the feature to be extracted, a feature amount designed in advance by the user such as an average value or a dispersion value of pixel values may be used, or a feature amount automatically acquired by using a deep neural network or the like may be used. Is also good. In the present embodiment, the subsequent processing is carried out using the features extracted by the former. Similarly, the extraction unit 104'in the inspection device 2 extracts features from the second measurement data by a predetermined method.

The estimation unit 105 estimates the first estimation data (estimated value information) including the result of inputting the features extracted from the first measurement data into the classifier. The estimated value information obtained here is the sample data and the result estimated by the discriminator (OK / NG), and the score indicating the normal product quality calculated to determine whether the product is non-defective or defective (OK / NG). Evaluation value) is included. Further, as the classifier used here, a pre-defined classifier is used. As the method of defining the discriminator, for example, a method of designing / learning by a different method for each device may be used, or a method of designing / learning from a reference device by a method such as transfer learning may be used. good. Similarly, the estimation unit 105'in the inspection device 2 estimates the second data including the result of inputting the feature extracted from the second measurement data by the extraction unit 104'to the classifier.

The association unit 112 inputs the first estimation data estimated by the estimation unit 105 and the second measurement data by the estimation unit 105'to the (second) classifier of another device based on the unique data. It is associated with the second estimated data including the result. The association data including the result of the association is stored. That is, based on the sample-specific information (unique data) obtained by the data acquisition unit 102, the estimated value information (first estimated data, second estimated data) for each device of the sample obtained by the estimation units 105 and 105' ) Is linked and the associated data is stored.

The second storage unit 117 stores the result of associating the first estimated data with the second estimated data. FIG. 4 shows an example of the association data stored in the identification result storage unit 107.

The comparison unit 113 compares the difference in the identification results for each target object based on the association data. That is, the estimated value information for the same sample stored in the second storage unit 117 is compared. The estimated value information to be compared here includes a score (evaluation value) indicating the quality of the sample as a normal product, a binary identification result of a non-defective product, a defective product, and the like.

The determination unit 114 determines the parameters of the first classifier so that the first estimated data and the second estimated data are brought close to each other based on the training data used when the parameters of the first classifier were generated. The parameters of the classifier are determined so as to reduce the difference in the discrimination results for each target object. That is, based on the comparison result information output by the comparison unit 108, the parameters of the classifier used by the estimation unit 105 or the imaging parameters used by the image acquisition unit 103 are determined. The specific determination method will be described in the details of the process according to the flowchart of the embodiment.

The learning units 106 and 106'update the parameters of the discriminator that have been set so far to the new parameters determined by the determination unit 114.

FIG. 3 is a flowchart illustrating a process executed by the information processing apparatus. In the following description, the notation of the process (S) is omitted by adding S at the beginning of each process (step). Hereinafter, the details of the processing of the present embodiment will be described with reference to the flowchart. However, the information processing apparatus 101 does not necessarily have to perform all the steps described in this flowchart. The process shown in the flowchart of FIG. 3 is executed by the computer H01 of FIG. 2 according to the computer program stored in the storage device H04.

First, in S301, the acquisition unit 111 acquires unique data that can uniquely identify the sample (object) included in the first measurement data. That is, the serial number of each sample is read to identify the information that can uniquely identify the sample. For example, an individual can be identified by assigning an ID such as a barcode to the sample and reading them. In addition, instead of the acquisition unit 111, the data acquisition unit 102 (102') of the inspection device 1 (2) may acquire unique data that can uniquely identify the sample (object) included in the first measurement data. ..

In S302, the acquisition unit 111 acquires the first measurement data including the result of measuring the sample (object). Alternatively, the image acquisition unit 103 (103') in the inspection device 1 (2) acquires image data (first measurement data) obtained by capturing an image of the sample from the first storage unit 100. The first measurement data includes information such as the measured time and internal parameters of the measuring device in addition to the measurement result (image in the case of the imaging device) of measuring the sample. Similarly, the image acquisition unit 103'acquires the second measurement data in which the sample is measured. The first measurement data and the second measurement data may be the same data.

In S303, the acquisition unit 111 acquires a predetermined (image) feature extracted based on the first measurement data. Here, features designed in advance by the user are extracted from the image acquired in S302. The extraction unit 104 (104') in the inspection device 1 (2) may extract a predetermined feature based on the first measurement data or the second measurement data. In S304, the acquisition unit 111 acquires the first estimation data including the result of inputting the first measurement data into the classifier. The estimation unit 105 of the inspection device 1 estimates the first estimation data (estimated value information) including the result of inputting the first measurement data (including the feature extracted from the first measurement data) into the classifier. May be good. That is, identification is performed based on the feature amount extracted in S303, and estimated value information is obtained. In S305, the associating unit 112 associates the unique data that can uniquely identify the sample (object) with the first estimated data estimated by the estimation unit 105. Similarly, the unique data that can uniquely identify the sample (object) and the second estimation data including the estimation result that the second measurement data is input to the (second) classifier of another device by the estimation unit 105'. , Correspond.
In S306, the second storage unit 117 stores the first estimated data associated with the unique data. That is, the estimated value information (first estimated data) obtained in S304 is stored together with the information (unique data) obtained in S301. Similarly, the second estimation data, which is the estimation result in which the features of the second measurement data are input to the classifier, and the unique data corresponding to the second measurement data are stored in the second storage unit 117. Using the data inspected by the inspection devices 1 and 2 respectively, the flowcharts up to this point are carried out for the number of sample data for determining the difference. As a result, the second storage unit 117 stores the information for each device for comparing the identification results.

Next, in S307, the association unit 112 associates the first estimated data estimated by the estimation unit 105 with the second estimated data based on the unique data. In S308, the comparison unit 113 compares the difference in the estimation result for each object based on the difference in the estimation result in which the first estimation data and the second estimation data are associated with each other. That is, the estimated value information for the same sample stored in the second storage unit 117 is compared. Estimated value information (first estimated data and second estimated data) for the same sample of apparatus 1 and apparatus 2 is compared. Here, the score (evaluation value) will be described as the estimated value information to be compared. Compare the scores of device 1 and device 2 for the same sample. When there is a difference larger than a predetermined value, in S309, the determination unit 114 inputs the training data including the label indicating the state of the object, the first estimation data, and the second estimation data to the classifier, respectively, and the estimation result. The parameters of the first classifier are determined based on. At this time, the parameters are determined so that the difference between the estimation results included in the first estimation data and the second estimation data is smaller than a predetermined threshold value. That is, the parameters of the classifiers of both or one of the devices are determined so that the calculated score is equal to or less than the reference value. In addition, the updated parameters are determined based on the training data used when generating the parameters for the first classifier.

A specific method for determining the parameters will be described by taking the case of determining the parameters of the apparatus 1 as an example. For example, if the environment of the device 1 is different from the measurement at the time of learning to which the training data is applied, it is better to update the parameters of the classifier of the device 1. However, there are cases where it becomes impossible to identify an object that was originally identified when training is performed using only new data. In order to avoid such a case, when determining the parameters of the device 1, make sure that the determination is not suitable only for the measured data set of the newly added sample. Therefore, the normal value score (estimated result) of the measurement data (first measurement data) of the added sample saved in S305 is used with the normal value score used when the parameter of the classifier of the device 1 is generated. Determine the parameter θ ⁽¹⁾ . The feature amount of each sample of the number of samples N used when the classifier of the device 1 was generated is x _li (i = 1, 2, ..., N), and the score at the time of generation is s _1i (= f (x _1i ;). θ ⁽¹⁾ )), let the score of the parameter determination target be f ^ (x _1i ; θ ⁽¹⁾ ). Further, in each sample having the number of samples M used for determining the difference, the feature amount in the device 1 is _x'1j , and the feature amount in the device 2 is _x'2j (j = 1, 2, ..., M). When the score for each sample of the device 1 is f ^ ( _x'1j ; θ ⁽¹⁾ ) and the parameter of the device 2 is θ ⁽²⁾ , the score for each sample is f ^ ( _x'2i ; θ ^(2)). ). The parameter determination of the device 1 is carried out by the following equation.

Here, f (x; θ) is a model representing a discriminator, and any model such as a logistic regression model or a neural network model that can express parameters may be used. Also, loss represents a loss function. Here, the loss function is defined as a two-variable function of loss (A, B), and the parameters are updated so as to reduce the difference between A and B. In the case of the present embodiment, A and B each represent a score, and by reducing this difference, a similar score can be calculated for each device. The loss function to be used should be determined by the method used to model the classifier. Generally, there are square loss, L1 loss, Hoover loss and the like. α represents the contribution rate, and is a variable that determines whether to prioritize the result of the score for the sample or the score for the sample used at the time of generating the discriminator to determine the parameter. The first term in Equation 1 is to obtain the loss function from the score at the time of determination and the score at the time of generation for the training data used at the time of generating the classifier of the device 1. The second term obtains the loss function from the score for the sample of the device 1 and the score for the first storage unit of the device 2. The parameter is determined by obtaining the parameter θ ⁽¹⁾ that minimizes the first and second terms. At this time, as the search method for the parameter θ ⁽¹⁾ , a stochastic gradient descent method, a Momentum method, an Adaptive Moment Estimation method, or the like may be used.

Next, a determination method when determining the classifier parameters of both the device 1 and the device 2 will be described. When it is obvious that the first environment of the device 1 has changed, the parameters of the classifier of the device 1 may be adjusted as described above. However, it may be unclear which device's classifier parameters should be adjusted. In that case, the parameters of the discriminator may be adjusted so that a common result (score) can be obtained between the two devices. First, the formula for determining the classifier parameter of the device 2 is shown below.

Here, x _2k (k = 1, 2, ..., L) is the feature amount of each sample having the number of samples L used when the classifier of the device ₂ was generated, and s _2k is the score at the time of generation. When determining the classifier parameters of both the device 1 and the device 2 at the same time, first determine the classifier parameters of the device 1 using Equation 1. Next, the classifier parameter of the device 2 is determined from the mathematical formula 2 using the determined parameter of the device 1. Here, the parameters of the classifier of the device 1, the parameters of the classifier of the device 2, and the score (estimation result of the classifier) for the measurement data for each device in the parameters can be calculated. The parameter determination is repeated until the difference between the scores of the classifiers of the device 1 and the device 2 for each measurement data calculated here becomes equal to or less than the reference value.

By determining the parameters as described above, it is possible to reduce the difference in the identification results while suppressing the deterioration of the performance due to the parameter determination. The feature amount at the time of generating the classifier of each device used here and the score at the time of generation may be stored in the first storage unit 107, or may be stored for each device. This also applies to the following description.

Hereinafter, the modification of the first embodiment will be described.

In the above case, the score is determined to be equally close to the result of identifying the measurement data of the same sample (object) having a difference in score between the devices by the classifier. Determine the parameters of the discriminator by giving importance to the measurement data so that the score is closer to the measurement data with a large difference in score, or the scores are almost the same for some measurement data. In some cases, it is done. In that case, the second term of the mathematical formula 1 may be modified to give weight to how much the parameter is determined for each measurement data. FIG. 5 shows a flowchart in which weights are assigned to each measurement data. The parts common to S301 to S306 and S307 in FIG. 3 are omitted from the description.

In S407, when the weight for each measurement data is not defined, the determination unit 114 determines the weight for each measurement data based on the difference in scores or the like. The determination formula when the weight is defined for each measurement data is as follows.

Here, ω _j is a weighting coefficient of the classifier for each measurement data held in the first storage unit 100, and may be calculated automatically by the information processing device, or is explicitly determined by the user for each measurement data. You may. In the case of automatic calculation, for example, there is a method of using the difference in scores of the devices 1 and 2 for the same sample as a weight. By doing so, the weight for the sample having a large difference in score between the devices is increased, and it is determined that the score is closer. This makes it possible to determine the parameters in consideration of the importance of each sample.

Even if the score calculated for each device is different, there is a need that only the identification results of non-defective products and defective products should match. This case will be described as a next modification. In this case, the identification result is used as the estimated value information to be compared. The parameter θ ⁽¹⁾ and the threshold value t ₁ that serves as a reference for determining a non-defective product or a defective product are determined so that the identification results for the same sample between the devices match. In this case, it is necessary to decide whether to match the identification result with a non-defective product or a defective product. If the sample for determining the difference is not labeled as a good product or a defective product, the correct identification result is estimated based on the score for the same sample for each device and the threshold information for each device. Specifically, the difference between the score calculated by the device 1 and the threshold value of the device 1 is calculated for samples having different identification results between the devices. Similarly, in the device 2, the difference between the score and the threshold value of the device 2 is obtained. The calculation results of the device 1 and the device 2 are compared, and the decision is made with the larger difference as the correct identification result. At this time, a threshold value is added to the determination parameter of the equation 1, and the determination is made by the following equation.

Here, f ^ (x _1i ; θ ⁽¹⁾ , t ₁ ) represents the identification result at the time of parameter determination for the sample at the time of generating the classifier of the apparatus 1. s _1i represents the identification result at the time of generating the classifier for the sample, and f ^ ( _x'1j ; θ ⁽¹⁾ , t ₁ ) represents the identification result at the time of parameter determination in the device 1 of the first storage unit. f ^ ( _x'2j ; θ ⁽²⁾ , t ₂ ) represents the identification result at the time of parameter determination in the apparatus 2 in the sample, respectively. As a result, when the identification results differ between the devices, it is possible to determine the parameters so that the identification results match even if the calculated scores are different.

The sample (object) may be used in its entirety for the determination, or a part of the sample may be used for the determination. Similarly, all or some of the sample information used when generating the classifier may be used.

<Embodiment 2>
In the first embodiment, a method of determining the classifier parameter based on the individual difference between the two devices has been described. In the present embodiment, a method of determining the classifier parameter based on individual differences between a plurality of devices (n ≧ 3) will be described. Here, the processing performed by the determination unit 114 will be described by taking as an example the case where there are 1 to n (n ≧ 3) devices and the classifier parameters of m devices (m ≦ n) are determined.

When determining m devices U _a (a = 1, 2, ..., M) based on the score of each device, nm devices V _b (b = 1) that do not perform parameter adjustment of the classifier are not performed. , 2, ..., Nm) is treated as a reference device. Of the device U to perform the parameter adjustment of the discriminator, the case where the determination of device U ₁ as an example.

The integrated score used for the comparison of the device U ₁ is determined based on the scores of the nm reference devices V _b without parameter adjustment of the classifier. The method of calculating the integrated score at this time is arbitrary, and for example, an average value, a median value, or the like may be used. The integrated score may be calculated based on the score of the reference device V _b as described above, or may be calculated based on the scores of all the devices except the device U ₁ . Also, in the case of m = n, that is, for all devices and decisions subject because the reference device V _b does not exist, calculated total score based on all scoring apparatus except the apparatus U ₁ of the decision object To do. After calculating the integrated score, the score is compared with the score of the device U ₁ , and the classifier parameters are determined so that the score is equal to or less than the threshold value based on the difference in the comparison results. The formula for determining the parameters based on the score of the device U ₁ and the integrated score determined by the nm reference devices V _b that are not to be determined is shown below.

Here, _stj is an integrated score in each sample for difference determination determined based on the values of nm reference devices V _b . Other items are synonymous with Equation 1, so explanations are omitted.

Further, the identification result may be used as the estimated value information to be compared. When the identification result is used, the identification result to be matched for each sample may be determined by a majority vote. For example, when a sample is identified as a non-defective product, a defective product, and a non-defective product by three devices, the correct identification result is regarded as a non-defective product, and the classifier parameter of the device determined to be a defective product is determined. Further, as in the first embodiment, the identification result to be matched may be determined by the difference between the score and the threshold value of the device judged to be a non-defective product and the difference between the score and the threshold value of the device judged to be a defective product. The formula for determining the parameters based on the identification result is shown below.

Here, _stj represents the identification result of a non-defective product and a defective product in each sample for determining the difference. Since the other items are synonymous with the formula 4, the explanation is omitted.

As in the first embodiment, the sample may be used in full or the determination may be performed, or a part of the sample may be used to perform the determination. Further, as in the formula 3, the determination may be performed by weighting each sample.

<Embodiment 3>
In the first and second embodiments, a method of automatically determining the parameters of the classifier has been described. In the present embodiment, a method will be described in which the user confirms a sample having a different identification result of each device and determines the classifier parameter. Here, the case of the devices 2 to n (n ≧ 2) is assumed, but in the present embodiment, the case of n = 3 will be described as an example. When introducing a plurality of inspection devices for visual inspection, etc., it is necessary to make adjustments so that there is no difference between the aircraft. On the other hand, it is conceivable that multiple devices will be installed and the aircraft will differ due to deterioration over time or changes in the environment after operation. In that case, the characteristics of the samples are compared with multiple devices. It is possible to save the trouble of specifying the reference device by the user.
In that case, it is implemented with three or more devices. (Inspection) If there is a difference in the identification result for each inspection device in the device 1, device 2, and device 3, a list of samples with the difference. The captured image and the identification result are displayed on the GUI (Graphical User Interface). The user selects the correct identification result from the information displayed on the GUI, and determines the parameters of the classifier based on the result. A flowchart when the user selects the identification result is shown in FIG. 6 and described.

In the present embodiment, the image (first measurement data) taken by S302 is displayed when displaying by GUI. Therefore, in S305, when the second storage unit 117 stores the estimated value information (first estimated data), the image (first measurement data) obtained in S502 is also stored.

In S507, the output unit 112 outputs the first measurement data corresponding to the samples having a difference as a result of comparison in S506 to be displayed as a list in the GUI.

In S508, the input unit 201 inputs a correct identification label as a correction result on the GUI by the user for each first measurement data having a difference. In S509, the determination unit 114 determines the parameters of the discriminator using the mathematical formula 6 based on the correction result.

An example of the GUI displayed to the user is shown in FIG. 7 and described. FIG. 7A is a GUI displayed in S507, and displays a list of samples having different identification results. The difference data display unit 40 displays the IDs of the samples with differences, and the identification result display unit 41 displays the identification results of each device. When the user selects the display button 42 of the sample to be labeled, the screen transitions to the screen of FIG. 7B. The status display unit 43 indicates whether the measurement data of the sample is labeled as normal or defective, or is still labeled. The button 44 instructs the unlabeled measurement data to be automatically labeled.

On the screen of FIG. 7B, the display unit 202 displays the images taken by each device, and the result display unit 50 displays the identification result of each device. When the user assigns labels of non-defective products and defective products with the label assigning button 45 and selects the completion button 47, the transition to FIG. 7A is performed, and the status display unit 43 is changed from unfinished to finished. Here, when it is desired to continuously label a plurality of samples, by selecting the button 48, the unlabeled sample data that has not been labeled is displayed in FIG. 7B in the status display unit 43. Will be done. If you want to return to the previous sample, you can return to the previous sample with the label by selecting the button 46. In FIG. 7A, if the button 46 is selected, the label can be automatically attached by majority vote as in the second embodiment.

FIG. 8 shows a block diagram showing an example of the functional configuration of the information processing system. The same configuration as in FIG. 1 is omitted from the description.

The display device 22 displays the GUI output from the output unit 115. It also accepts input information input from the GUI by the user. The GUI example corresponds to FIGS. 7 (a) and 7 (b). The comparison unit 113 displays a list of differences in the identification results on the GUI and labels the correct identification results by the user. Based on the identification result information updated by the user, the determination unit 114 determines the classifier parameters. Further, the comparison unit 113 and the display device 22 may be arranged outside, or may be configured in the inspection devices 1 and 2.

<Embodiment 4>
In the embodiments so far, determination of the classifier parameter based on individual differences between a plurality of devices has been described. In the present embodiment, determination of the classifier parameter due to changes in the environment of one device, changes over time of the device itself, and the like will be described. Since it is possible to handle cases that cannot be handled by adjusting the parameters of the classifier, the applicable scenes are expanded. In addition, when the lighting conditions change drastically and there is a large difference in the images captured between the devices, it is better to adjust the imaging parameters instead of the classifier. The method described here is a method of supplementing the difference in the identification result of the device due to the environmental change such as the surrounding lighting conditions and the time-dependent change of the device itself by determining the parameters of the classifier without changing the surrounding environment and the shooting parameters. Is. When determining the classifier parameters in one device, it is necessary to store a part of the measurement data of the sample used when the classifier was generated in the first storage unit. Further, although the flowchart of the present embodiment is the same as that of the first embodiment, it is necessary to hold a plurality of estimated value information of the same device for the same sample in the second storage unit 117. In this case, information such as the date and time information and the sequence ID identified by the estimation unit 105 are also stored in the second storage unit 117. When the device at the time of generating the classifier is device 1, and the device after an environmental change or a predetermined time has elapsed is device 1', the score or identification result of device 1'is the score or identification of device 1. Determine the classifier parameters for device 1'so that they are close to or the same as the results. The measurement data in the first storage unit used here was used when the classifier was generated, and since the correct identification result is known, it is not necessary for the user to give the correct result using the GUI, and the identification result. There is no need to estimate. Therefore, it is possible to automatically determine the parameters of the classifier. Sample N (i = 1, 2, ..., N) used when the classifier of the device 1 was generated, measurement data P (j = 1, 2, ..., P) (P ≦ i) obtained from the sample. In, when determining the parameters based on the score, the following determination formula is used.

Here, s _1i is the score at the time of generating the classifier of the device 1 with respect to the measurement data of the sample used at the time of generating the classifier of the device --- 1, and s _1j is the score at the time of generating the classifier of the device 1 with respect to the measurement data of the device 1. It is a score. When determining the parameters based on the identification result, the following determination formula is used.

As in other embodiments, the sample may be used in full or the decision may be made, or a portion of the sample may be used to make the decision.

<Embodiment 5>
In the embodiments so far, a method of determining the classifier parameter in the inspection device has been described. In this embodiment, a method of determining an imaging parameter when photographing a sample with an inspection device will be described. A flowchart for determining the shooting parameters will be described with reference to FIG. The description of the process common to FIG. 3 will be omitted.

In S607, the comparison unit 113 determines, based on the comparison result of S606, whether the difference in the score for each sample between the devices is equal to or less than the set threshold value, or whether the identification results for each sample match. If there are samples that exceed the threshold value or the identification results do not match, the transition to S608 occurs.

In S608, the acquisition unit 111 determines the imaging parameters of the device such as the camera position, the illumination position, and the amount of light of the image acquisition unit involved in the photographing of the sample. Set and control using the determined shooting parameters, and shoot the sample (object) again. The shooting parameters are searched until the score difference is equal to or less than the threshold value in all the samples (objects) or the identification results match. Imaging / illumination control, parameter search, etc. are performed by control means (not shown). At this time, the user may set the identification result for each measurement data of the sample, and only the device different from the identification result taught by the user may be determined. The correct identification result may be judged by majority vote, and only the devices having different identification results may be determined. By adjusting the imaging parameters of the imaging device, the visual inspection can be performed more accurately.

<Embodiment 6>
In the above embodiments, the method of determining the imaging parameter and the discriminator parameter has been described, but both the imaging parameter and the discriminator parameter may be determined at the same time.

When a plurality of devices exist from 1 to n (n ≧ 2) and the photographing parameter and the classifier parameter of the device m (m ≦ n) are determined at the same time, the update formula of the formula 5 or the formula 6 is changed to U ₁ = m. It is used in place of. At this time, the parameter θ ^(m) to be determined includes, in addition to the parameters of the classifier, shooting parameters such as the camera position of the image acquisition unit related to the shooting of the sample, the illumination position, and the amount of light. These parameters are changed exploratoryly to determine the parameters. Although the method of determining the parameters for a plurality of devices is described in this embodiment, it can also be applied to the case of environmental change or change with time in one device. At this time, not only the imaging parameter and the classifier parameter but also the environmental change and the change with time may be used as parameters as environmental condition parameters. Parameters of environmental change and change with time include the amount of change in the feature amount at the time of generation and the time of environmental change with respect to the sample used at the time of classifier generation, and the amount of light in the surrounding environment. Therefore, when the environmental condition parameter is used for determination, it is necessary to store the environmental condition at the time of generating the classifier in the second storage unit 117. This environmental condition parameter is added to the classifier parameter and the photographing parameter, and the determination is made using the formula 5 and the formula 6. Further, in the present embodiment, since the number of parameters to be determined increases as compared with the other embodiments, a method such as determining the parameters by referring to the past parameter values and shooting conditions and searching the vicinity thereof is used. Search efficiently. Further, the parameters to be determined may be reduced by using a method of fixing and determining one or more of the classifier parameter, the photographing parameter, and the environmental condition parameter, a method of randomly determining the parameter to be determined, and the like.

<Embodiment 7>
In the embodiments so far, the case where the individual can be uniquely identified by using the identifier assigned to the sample (object) has been described. If the sample is labeled as non-defective, defective, defect type, etc., or if the sample can be classified according to the similarity of the distribution of features, etc., determine the parameters of the discriminator without uniquely identifying the sample. It may be carried out. In this embodiment, a method of determining the parameters of the classifier when the sample is not uniquely specified will be described. The configuration of this embodiment is shown in FIG. Since the sample is not uniquely specified, the data acquisition unit is not included as compared with FIG. Other than that, it is synonymous with Fig. 1, so the explanation is omitted.

When M samples for determining the difference can be classified into Q classes (j = 1, 2, ..., Q) according to the similarity of the distribution of the features, a plurality of devices 1 to n (n ≧ 2). An example of determining the parameter of the device m (m ≦ n) of (k = 1, 2, ..., N) will be described. First, the measurement data of the sample is identified by each device, and the class score _Skj is acquired for each class. The method for determining the class score may be, for example, an average value, a median value, a total value, or the like. When the number of sample measurement data classified into class 1 is E (E≤M), the feature amount in the device m is _x'mH (H = 1, 2, ..., E). _Assuming that the score for each sample measurement data of the device m is f ^ ( _x'mH ; θ ^(m) ), when the class score S _m1 in the device m is calculated as a total value, it is calculated by the following formula.

The parameters of the classifier are determined using the following formula so that the difference in scores for each class in the device m is equal to or less than the reference value.

Here, _stj is an integrated class score determined based on the class score of each class of the device other than the device m. The method of calculating the integrated class score may be, for example, an average value, a median value, a total value, or the like, as in the case of the class score. Other items are synonymous with Equation 1, so explanations are omitted.

As in the first embodiment, the determination may be made using the identification result for each class. In this case, the identification result for each class in each device is determined by a majority vote in each class. After that, the identification result to be determined for each class is determined. For example, for a certain class Q _j, is determined by the three devices non-defective, defective, if identified as good, correct identification result of class Q _j is a non-defective sample contained in the class Q _j is a good As described above, the parameters of the device m are determined. The formula for determining the parameters is as follows.

Here, s _mj is the identification result in the class j of the device m, and s _tj is the integrated identification result calculated from a device other than the device m.

The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiment is supplied to the system or device via a network for data communication or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or device reads and executes the program. Further, the program may be recorded and provided on a computer-readable recording medium.

1 Inspection device 110 Imaging device 11 Information processing device 2 Inspection device

Claims (16)

An information processing device that determines the parameters of a classifier that estimates the appearance of an object from the measured data of the object.
The object was measured in a second environment different from the first environment and the first estimated data estimated by inputting the first measurement data obtained by measuring the object in the first environment into the first classifier. An acquisition means for acquiring the second estimated data estimated by inputting the second measurement data to the second classifier, and
A determination means for determining updated parameters of the first classifier based on training data including a label indicating the appearance state of the object, the first estimation data, and the second estimation data.
The information processing apparatus is characterized in that the determination means determines the parameters updated so that the difference between the estimation results included in the first estimation data and the second estimation data becomes smaller than a predetermined threshold value. In the first classifier, parameters learned based on the training data obtained by measuring the object in an environment different from the first environment are set in advance.
The information processing according to claim 1, wherein the determination means determines the updated parameters of the first classifier so as to match the first estimated data with the corresponding second estimated data. apparatus. The first estimated data and the second estimated data include scores indicating that the appearance state of the object is normal.
The determination means determines the parameters of the first classifier so that the difference between the score included in the first estimation data and the score included in the second estimation data is smaller than a predetermined threshold value. The information processing apparatus according to claim 1 or 2. The determining means of the first classifier means that for data in which the difference between the score included in the first estimated data and the score included in the second estimated data is larger than a predetermined value, the difference becomes smaller. The information processing apparatus according to claim 3, wherein the parameters are determined. The determination means determines the parameters of the first classifier based on the score included in the first estimation data and the weighting according to the magnitude of the difference between the score included in the second estimation data. The information processing apparatus according to claim 3. The first estimation data and the second estimation data include binary labels indicating normal and abnormal appearance states of the object.
The determination means according to claim 1 or 2, wherein the determination means determines the parameters of the first classifier based on the first estimation data and the second estimation data so that the labels match. Information processing equipment. The acquisition means further acquires unique data that can uniquely identify each of the objects.
Further provided with an associating means for associating the first estimated data with the second estimated data for each object based on the unique data.
Based on the association between the first estimated data and the second estimated data by the associating means, the determining means reduces the difference between the corresponding first estimated data and the second estimated data. The information processing apparatus according to any one of claims 1 to 6, wherein the parameters of the first classifier are determined. The first estimation data is data estimated by inputting the first measurement data obtained by measuring the object in the first apparatus into the first classifier.
The second measurement data obtained by measuring an object by the second device is the data obtained by measuring the object after a lapse of time by the first device, according to any one of claims 1 to 7. The information processing device described. The acquisition means indicates the first estimation data estimated by inputting the first measurement data obtained by measuring the object in the first apparatus into the first classifier, and the first apparatus after a predetermined time has passed. The second estimation data estimated by inputting the second measurement data obtained by measuring the object in the second apparatus into the second classifier having the same parameters as the first classifier is acquired.
The determination means according to any one of claims 1 to 8, wherein the determination means determines the parameters of the second classifier based on the training data, the first estimation data, and the second estimation data. The information processing device described. An output means for outputting a display showing the difference between the first estimated data and the second estimated data, and
Further, an input means for inputting an instruction to modify the first estimated data or the second estimated data with respect to the display output by the output means is provided.
The information processing apparatus according to any one of claims 1 to 9, wherein the determination means determines a parameter of the first classifier based on the instruction input to the input means. Any of claims 1 to 10, wherein the determination means determines the parameters of the first classifier, and further determines the parameters of the second classifier based on the parameters of the first classifier. The information processing device according to item 1. Further provided with an extraction means for extracting features from the first measurement data and the second measurement data by a predetermined method.
The determination means of the first classifier is such that the difference between the estimation results is small for each class in which the first measurement data and the second measurement data are classified based on the characteristics extracted by the extraction means. The information processing apparatus according to any one of claims 1 to 11, wherein the parameters are determined. The acquisition means further acquires a plurality of estimated data estimated by inputting measurement data obtained by measuring the object in a plurality of environments into a plurality of classifiers.
In the determination means, the difference between the first estimation data and the estimation result included in the estimation data is based on the estimation data in the environment selected by the user among the plurality of estimation data acquired by the acquisition means. The information processing apparatus according to any one of claims 1 to 12, wherein the parameter is determined so as to be smaller than a predetermined threshold value. The determination means further determines the imaging parameters of the imaging device that images the object so that the difference between the estimation results included in the first estimation data and the second estimation data becomes smaller than a predetermined threshold value. The information processing apparatus according to any one of claims 1 to 13. A program for causing a computer to function as each means included in the information processing apparatus according to any one of claims 1 to 14. It is an information processing method that determines the parameters of a classifier that estimates the appearance state of an object from the measured data of the object.
The object was measured in a second environment different from the first environment and the first estimated data estimated by inputting the first measurement data obtained by measuring the object in the first environment into the first classifier. An acquisition process for acquiring the second estimated data estimated by inputting the second measurement data to the second classifier, and
It comprises a determination step of determining updated parameters of the first classifier based on training data including a label indicating the appearance state of the object, the first estimated data and the second estimated data.
The information processing method is characterized in that the determination step determines the parameters updated so that the difference between the estimation results included in the first estimation data and the second estimation data becomes smaller than a predetermined threshold value.

Images