JP2018190282A - Inspection device, inspection method and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an inspection method which is robust to variation elements such as noise, etc.SOLUTION: Target data representing a state of an inspection object and a plurality of processing parameters different from one another are input (S301), and respective feature measurement processes using a plurality of the processing parameters are performed for the target data (S302). Further, results of respective feature measurement processes are integrated, and a physical amount enabling comparison with a predetermined inspection standard is extracted from the integrated result as a first feature of the target data (S303). Further, a feature of the target data not depending on the inspection standard is extracted as a second feature (S304). Information for determining the quality of the inspection object is generated based on the extracted first feature and the second feature (S305).SELECTED DRAWING: Figure 12

Description

  The present invention relates to an apparatus and method for inspecting a state of an inspection object.

  A technique is known in which the characteristics of an inspection object are extracted by image processing or the like to determine whether the product is good or defective. As this type of prior art, for example, there is an observation apparatus disclosed in Patent Document 1. In this observation apparatus, a result of each image processing using a plurality of processing parameters performed on an image obtained by imaging an inspection target is compared with teaching data created in advance. Then, a processing parameter having a relatively high degree of coincidence is selected as a processing parameter. As a result, the processing parameter setting operation is easy and fast.

JP 2011-145275 A

  In the observation apparatus disclosed in Patent Document 1, the processing parameter is fixedly used at a specific value. For this reason, for example, when extracting the characteristics of the inspection target, there remains a problem that the determination of pass / fail is unstable because it is easily affected by slight changes in imaging conditions and noise.

  A main object of the present invention is to provide an inspection apparatus capable of performing a robust inspection against a variable element such as an environmental change or noise in which an inspection object is placed.

  An inspection apparatus to which the present invention is applied includes a data input unit that inputs target data representing a state of an inspection target, and a feature measurement unit that performs each feature measurement process using a plurality of different processing parameters on the target data And a first feature extraction means for extracting the first feature of the target data from the integrated result, and the state based on the extracted first feature And a discriminating means for discriminating whether the quality is good or bad.

  According to the present invention, since the results of feature measurement processing using a plurality of processing parameters are integrated, it is possible to perform feature extraction that is robust against the environment, noise, and fluctuation of the inspection target.

(A)-(d) is an illustration figure of the image which imaged the test object. The example figure of the inspection standard for being able to say that it is good quality conditions. FIG. The hardware block diagram of the inspection apparatus which concerns on 1st Embodiment. The functional block block diagram of an inspection apparatus. (A)-(c) is an illustration figure of the binarized image from which a threshold value differs. FIG. 6 is an exemplary diagram of a result of feature measurement processing. (A) is an illustration figure of teaching data, (b) is the figure which showed the example of the result of the characteristic measurement process. The illustration figure of the number of coincident samples of the result of a characteristic measurement process, and a visual determination result. The processing flow which shows the example of a learning procedure of an inspection standard dependence feature extraction part. The processing flow which shows the learning procedure of a discrimination | determination part. The processing flow which shows the example of a test | inspection procedure by 1st Embodiment. The processing flow which shows the example of a learning procedure of the test reference dependence feature extraction part by 2nd Embodiment. (A), (b) is an illustration figure of the characteristic measurement process in 3rd Embodiment. The illustration figure of the teaching data in 3rd Embodiment.

  Hereinafter, an embodiment will be described in which the present invention is applied to an inspection apparatus that measures the characteristics of the state of the inspection target and thereby determines whether the inspection target is a non-defective product or a defective product.

[First Embodiment]
An example in which the metal material whose surface is polished and the state of the surface is an inspection object will be described. There are various methods for measuring the characteristics of the state of the inspection target, but the first embodiment shows an example of measurement by image processing. In a metal material manufactured by polishing, a peculiar portion, for example, a point-like defect may be generated due to friction or corrosion in the manufacturing process. 1A to 1D are images obtained by imaging the surface of a metal material. In the image 110 of FIG. 1A, four point-like defects 111 to 114 are generated. Two point-like defects 121 and 122 are generated in the image 120 of FIG. In the image 130 of FIG. 1C, five point defects 131 to 135 are generated.

When inspecting a metal material in which defects in the manufacturing process are unavoidable, an inspection standard for determining whether or not a good product condition is satisfied is usually set. For example, as shown in FIG. 2, the metal material that satisfies the condition of being a non-defective product (non-defective product condition) has an area of point defects of less than 0.04 mm ² . That is, even if a point-like defect occurs in the images 110 to 140, it can be said to be a non-defective product if it is less than the area defined by the inspection standard. In the first embodiment, there are learning samples of non-defective products and defective products visually discriminated by an inspector in advance, and learning of various functions used in the inspection is performed using these learning samples. An example of a learning sample is shown in FIG.

In FIG. 3, the sample ID # 1 corresponds to the image 110 in FIG. Similarly, sample ID # 2 to sample ID # 4 correspond to the images 120 to 140 in FIGS. 1B to 1D, respectively. In the “good / bad” column, “good” or “bad” is recorded by visual inspection by the inspector. Points # 1 to # 6 are point-like defects measured by image processing, and six are recorded in descending order of area. For example, the point # 1 of the sample ID # 1 is the largest point defect 111 existing in the image 110, and the area thereof is 0.06 mm ² . Point # 2 is a point-like defect 112, and its area is 0.025 mm ² . Since there are only four point-like defects 111 to 114 in the image 110, the points # 5 and # 6 are “N / A”. In the example of the image 110, since the area of the point defect 111 is 0.06 mm ² , the area exceeds the allowable range of the non-defective condition. For this reason, the state of the metal material represented by the image 110 does not satisfy the non-defective product condition of the inspection standard in FIG. Two point-like defects 121 and 122 exist in the sample ID # 2, that is, the image 120 in FIG. 1B, but the areas are all within the allowable range. Therefore, the state of the metal material represented by the image 120 is “good”. Although there are five point-like defects 131 to 135 in the image 130 of FIG. 1C, all of them are within the allowable range. Therefore, the state of the metal material represented by the image 130 is “good”. On the other hand, the point defect is not present in the image 140 of FIG. Therefore, all the points # 1 to # 6 of the sample # 4 corresponding to the image 140 are recorded as “N / A”. However, in practice, a non-uniform portion of surface processing, that is, processing unevenness 146 occurs over a wide range. Therefore, the image 140 is recorded as “defective” by visual discrimination.

  Next, a configuration example of the inspection apparatus according to the first embodiment will be described. FIG. 4 is a hardware configuration diagram of the inspection apparatus. The inspection apparatus includes a computer having a CPU 10, a RAM 11, a ROM 12, an external storage 13, an input I / F 14, an output I / F 15, and a device I / F 16 as hardware components. “I / F” is an abbreviation for interface. These hardware components are connected to each other via a control bus B1. A CPU (Central Processing Unit) 10 executes the computer program of the present invention to cause the computer to operate as an inspection device. A RAM (Random Access Memory) 11 is a rewritable memory used as a temporary storage area such as a main memory and a work area of the CPU 10. A ROM (Read Only Memory) 12 is a read-only memory in which the computer program, device driver, and the like are stored.

  The external storage 13 is a semiconductor memory or a hard disk, and forms a database system that stores a plurality of candidate parameters (processing parameters), teaching data, input data, various processed data, and the like, which will be described later. The input I / F 14 transmits input data to the CPU 10. The output I / F 15 displays data output according to instructions from the CPU 10 on a display device such as a liquid crystal display. The device I / F 16 transmits control signals for starting and stopping operations to various devices according to instructions from the CPU 10. Examples of the various devices include an imaging device, an image sensor, a sound collecting microphone, a vibration sensor, and an acceleration sensor. The input data is data output from these devices. 1st Embodiment demonstrates the example at the time of using the imaging device which images the surface of a metal raw material as a device.

  FIG. 5 is a functional block configuration diagram of the inspection apparatus. These functional blocks are formed by the CPU 10 reading and executing the computer program. The inspection apparatus includes functional blocks of a data input unit 401, a feature measurement unit 402, a parameter input unit 403, an inspection criterion-dependent feature extraction unit 404, a constraint condition input unit 405, an inspection criterion-independent feature extraction unit 406, and a determination unit 407. .

  The data input unit 401 inputs target data representing the state of the inspection target at the time of inspection, and learning data having the same data structure as the target data instead of the target data at the time of learning. In the first embodiment, the target data is an image obtained by imaging a metal material to be inspected with an imaging device. This target data is a gray scale image. The target data is input every time the metal material to be inspected is changed, but all the images are the same resolution images taken under the same imaging conditions. The learning data is also a gray scale image and is an image having the same resolution as the target data.

  The feature measurement unit 402 performs each feature measurement process using a plurality of different processing parameters on the target data. The plurality of processing parameters are determined by learning, which will be described later, and are given from the parameter input unit 403 among those stored in the external storage 13. As processing parameters, for example, a threshold value used when binarizing an image, a kernel size or standard deviation of a LoG filter, a number of times of performing a morphological operation, or the like can be used. When these processing parameters change, the result of the feature measurement processing also changes. In the following description, an example in which a plurality of processing parameters are obtained by changing a threshold value for binarization will be described. However, a processing parameter other than the threshold value can be used as the changing processing parameter.

  The feature measurement unit 402 performs feature measurement processing, for example, according to the following procedure. First, a convolution operation using a LoG filter is performed on the target data, and processing for enhancing the blob region is performed. A LoG (Laplacian of Gaussian) filter is a filter that can extract image features at different scales. A blob region is a region where pixels having the same logical value are adjacent to each other. In this example, an area where the pixel density is minimum, that is, an area recognized as a point defect is emphasized. The feature measurement unit 402 binarizes the image in which the blob area is emphasized with a plurality of threshold values. For example, if the image 110 shown in FIG. 1A is the target data, the feature measurement unit 402 takes the absolute value of the density of each pixel after convolving a LoG filter on the image 110, and obtains a plurality of values. It binarizes with the threshold value. The images obtained in this way are called binary images. An example of a binarized image is shown in FIG. Each binarized image 610 shown in FIG. 6A is an image binarized using a threshold value T1. Similarly, the binarized images 620 and 630 shown in FIGS. 6B and 6C are images obtained by binarizing the image 110 using threshold values T2 and T3, respectively. The magnitude of the threshold is T1 <T2 <T3.

  The feature measurement unit 402 performs labeling on each blob area by processing the binarized images 610, 620, and 630 using, for example, a known Union-Find algorithm. Thereby, it is possible to identify the dot-like figure existing in the binarized image and its size. This point-like figure is called a “measurement point”. Each measurement point is recognized as a point defect. For example, the measurement point 611 in the binarized image 610 illustrated in FIG. 6A corresponds to the point defect 111 of the image 110 illustrated in FIG. Similarly, the measurement points 612 to 614 correspond to the point defects 112 to 114 of the image 110. The measurement point 615 is a measurement point that does not exist in the image 110 but appears due to the influence of noise mixed during image processing. The measurement point 611 in FIG. 6A, the measurement point 621 in FIG. 6B, and the measurement point 631 in FIG. 6C all correspond to the same point defect 111. However, the measurement points 611, 621, and 631 have different sizes (areas) due to differences in threshold values.

The feature measurement unit 402 calculates the area of the measurement point by counting the number of pixels corresponding to each measurement point. For example, when the image resolution is 20 pixels / mm, the area of one pixel is 0.0025 mm ² . When it is necessary to further reduce the noise, for example, the morphological operation on the binarized image is repeated. As a result, minute measurement points that do not exist originally (for example, the measurement points 615) can be erased, and it is possible to measure features that are robust against fluctuation elements such as noise.

  An example of the result of the feature measurement process by the feature measurement unit 402 is shown in FIG. In FIG. 7, “sample ID” is identification information of a labeled blob area. In the sample ID # 11, the area of the measurement point of the blob area obtained by processing the binarized image 610 with the threshold value T1 is recorded in association with it. Sample ID # 12 is recorded in association with the area of the measurement point of each blob region processed with threshold T2 and sample ID # 13 with threshold T3. Similarly, the area of the measurement point of each blob area obtained by processing the binarized image 620 with the threshold values T1 to T3 is recorded in association with the sample ID # 21 to sample ID # 23. In sample ID # 31 to sample ID # 33, the area of the measurement point of each blob region obtained by processing the binarized image 630 using threshold values T1 to T3 is recorded in association with each other. The areas of the measurement points are arranged in descending order of the areas obtained by processing with the threshold value T1. In the example of FIG. 7, there are six measurement points, but this number is arbitrary. If no measurement point is found, N / A is recorded instead of the area.

The inspection criterion-dependent feature extraction unit 404 integrates the results of the respective feature measurement processes using a plurality of processing parameters, and extracts image features that are target data from the integrated results. This feature is a physical quantity that enables comparison with a predetermined inspection standard, for example, the inspection standard illustrated in FIG. 2, and is referred to as a first characteristic for convenience. In the first embodiment, a feature vector obtained by combining the number of threshold values to be used as a dimension is used as the first feature. That is, feature measurement results corresponding to at least two of the three threshold values T1, T2, and T3 obtained from the feature measurement unit 402 are combined and output as a feature vector. For example, when the feature extraction results based on the threshold values T1 and T2 are combined, the feature vector is [0.05, 0.0225, 0.01, 0.01, 0.0025, 0, 0.0225, 0.01, 0.005, 0, 0, 0]. Which threshold is used to combine the feature measurement results is determined by learning using constraint conditions described later. The inspection criterion dependent feature extraction unit 404 outputs the feature vector derived in this way as a first feature extraction result.
Note that the information is not necessarily limited to a feature vector as long as the information represents a physical quantity that enables comparison with the inspection standard, such as the size (area) and the number of point defects, and other physical quantities are not limited to the first feature. You may use as an extraction result.

The inspection criterion-independent feature extraction unit 406 extracts the feature of the state of the inspection target different from the first feature from the target data without performing the feature measurement process. That is, feature extraction that does not depend on the inspection standard is performed. Such a feature is referred to as a “second feature” for convenience. This second feature is not a feature such as size (area) or number, but statistical information that statistically indicates the degree of peculiarity of the state to be inspected, such as non-uniformity of surface processing and a wide range of dents. It is. When extracting the second feature, first, the target data (image in this example) is divided into, for example, 16 parts, and at least the average, variance, skewness, and kurtosis of the image luminance values for each of the divided blocks. Calculate one or more. Then, the image statistic obtained in each block is used as a feature amount. The inspection criterion-independent feature extraction unit 406 outputs this as a second feature extraction result.
By extracting the second feature, when there is a specific part other than the defect defined by the physical quantity, the specific part can be recognized. Therefore, the accuracy of good product discrimination in the inspection object is improved.

The determination unit 407 determines pass / fail of the inspection target based on at least one of the first feature extraction result and the second feature extraction result, preferably both. For the pass / fail judgment, for example, the subspace method can be used, and the projection distance to the subspace learned by the non-defective sample can be used.
For example, if the first feature output from the inspection criterion dependent feature extraction unit 404 is xm and the second feature output from the inspection criterion independent feature extraction unit 406 is xf, the determination unit 407 combines them. The feature x = [xm, xf] is input. Note that the second feature xf is not always necessary, and the feature x may be the first feature xm.

At this time, a non-defective product score s (x) obtained by scoring the quality of the non-defective product to be inspected is expressed by the following Expression 1. Here, U represents a subspace learned with a non-defective sample.

The pass / fail judgment is not limited to the subspace method, and a k-nearest neighbor method, 1-class SVM, or the like can be used. Further, instead of learning using only non-defective samples, defective samples may be used as learning data, and two-class identification of non-defective or defective products may be performed. In the first embodiment, the determination unit 407 is described as determining whether it is good or bad. However, even if multi-class identification is performed to determine a plurality of quality grades, it is possible to identify whether the product is good or defective using the same method. It is.
Since the first feature extraction result is obtained by integrating the feature measurement results using each of a plurality of threshold values, it is possible to discriminate robustly against fluctuations in imaging conditions and noise compared to the case where a fixed threshold value is used. It can be carried out.

<Learning of inspection equipment>
The inspection device learns the functions of the inspection criterion-dependent feature extraction unit 404 and the determination unit 407 before inspecting the state of the inspection target. Learning data using defects is used for learning. In addition, various conditions can be used as constraints for learning. First, an example in which teaching data on the area and position of a point-like defect is given as in each sample shown in FIG.

FIG. 8A shows teaching data 810 for the image 110 shown in FIG. 1A, and FIG. 8B shows an example of characteristic measurement results in the binarized image 610 shown in FIG. In the teaching data 810, the coordinates and areas of the point-like defects 111 to 114 existing in the image 110 are associated with each other. First, the inspection criterion-dependent feature extraction unit 404 selects a portion having the closest coordinate distance to the point defect 111 from the measurement points 611 to 615 in FIG. In this example, the measurement point 611 is the closest, and the difference in area is 0.01 mm ² . Similarly, regarding the point defects 112 to 114, the difference between the closest measurement point and the area is obtained. For the measurement point 615, there is no corresponding teaching data. Therefore, the area of the measurement point 615 is recorded as a difference from the teaching data as it is. In this way, the difference in area between the measurement points 611 to 615 of the image 610 and the teaching data is obtained, and the mean square error of the difference in area between the points is obtained. This mean square error is taken as the evaluation value of the threshold T1 that generated the image 610. In this case, the lower the evaluation value, the better the evaluation. Similarly, evaluation values are obtained for the threshold values T2 and T3. A prescribed number of threshold values are selected in order from the one with the highest evaluation value corresponding to each threshold value thus obtained. It should be noted that all threshold values at which the error becomes a certain value or less may be selected without giving the specified number. The selected threshold value is used as a processing parameter (threshold value) input from the parameter input unit 403 at the time of inspection.

As another constraint condition in learning, the non-defective condition of the inspection standard shown in FIG. 2 can be used instead of the area of each point-like defect of the learning sample illustrated in FIG. In this case, the non-defective product or the defective product is determined by the area of the largest point defect. Referring to the result of feature measurement associated with sample ID # 11 in FIG. 7 (corresponding to sample # 1 in the learning sample in FIG. 3), the measurement point having the maximum area is measurement point # 1, and the area Is 0.05 mm ² . This area does not satisfy the non-defective condition of the inspection standard shown in FIG. Therefore, the sample ID # 11 is a defective product. This result matches the visual determination result (“bad”) for the sample ID # 1 shown in FIG. On the other hand, the maximum area of the result of the feature measurement associated with the sample ID # 12 of FIG. 7 (corresponding to the sample # 1 of the learning sample of FIG. 3 but with a different threshold) is 0.0225 mm ² . Therefore, it is determined as a non-defective sample and does not coincide with the visual determination result (“defective”) for the sample ID # 1 shown in FIG.
FIG. 9 shows an example of the number of coincident samples with the visual determination result when processing is performed with the respective threshold values. The number of matched samples can be used as an evaluation value for each threshold. In the example of FIG. 9, for example, the upper two threshold values T1 and T2 can be set as processing parameters input from the parameter input unit 403 at the time of detection.

Next, the learning procedure of the inspection criterion dependent feature extraction unit 404 will be described in detail with reference to FIG. The inspection apparatus inputs learning data from the data input unit 401 (S101). Also, a plurality of processing parameter candidates are input from the parameter input unit 403 (S102). Processing parameter candidates are referred to as “candidate parameters”. Then, the feature measurement unit 402 performs feature measurement of learning data corresponding to each of the plurality of candidate parameters. That is, the physical quantity that enables comparison with the inspection standard is extracted as the first learning feature of the learning data. For example, point-like defects are extracted and the area of each extracted defect is measured (S103). Thereafter, teaching data corresponding to the input learning data is input from the constraint condition input unit 405 (S104). Then, the result measured in S103 is compared with the teaching data input in S104 (S105). Comparison of the measured results is performed by calculating an error between the points described in the teaching data and the measurement points. Alternatively, it is confirmed whether the maximum area in the measurement result satisfies the non-defective product determination condition.
It is determined whether or not the processing of S101 to S105 has been completed for all learning data (S106), and if there is unprocessed learning data, the process returns to S101 (S106: N). When the processing is completed for all the learning data (S106: Y), an evaluation value for each candidate parameter is calculated (S107). Based on this evaluation value, some upper candidate parameters are determined as processing parameters (S108), and are temporarily stored in the database. Thereby, the learning of the inspection reference feature 404 is completed.

Next, the learning procedure of the determination unit 407 after the processing parameters are determined by the procedure of FIG. 10 will be described with reference to FIG. The inspection apparatus inputs learning data from the data input unit 401 (S201). The feature measurement unit 402 performs feature measurement processing on the learning data using the plurality of processing parameters determined in S108 (S202). Further, the inspection standard dependent feature extraction unit 404 calculates a feature xm obtained by integrating the feature measurement results (S203).
Also, feature extraction processing is performed on the learning data by the inspection criterion-independent feature extraction unit 406 to obtain a feature xf (S204). That is, the feature of the learning data different from the first learning feature is extracted from the learning data as the second learning feature without performing the feature measurement processing of the learning data. It is determined whether or not the processing of S201 to S204 is completed for all learning data (S205), and if there is unprocessed learning data, the process returns to S201 (S205: N). If there is no unprocessed learning data (S205: Y), the discrimination function is learned for all the learning data using the feature x = [xm, xf] obtained in S201 to S204 (S206). Thereafter, the learning of the determination unit 407 is completed.

<Inspection method>
Next, an inspection method for an inspection object using the inspection apparatus will be described. In the inspection apparatus, it is assumed that the inspection criterion-dependent feature extraction unit 404 and the determination unit 407 are learned as described above. FIG. 12 is an explanatory diagram of an inspection procedure executed by the inspection apparatus.
The inspection apparatus inputs target data from the data input unit 401 and also inputs a plurality of different processing parameters from the parameter input unit 403 (S301). The feature measurement unit 402 performs each feature measurement process using a plurality of processing parameters on the target data (S302). The inspection standard dependent feature extraction unit 404 integrates the results of the respective feature measurement processes, and extracts a physical quantity that enables comparison with a predetermined inspection standard from the integrated results as the first feature of the target data (S303). ). Further, the inspection criterion-independent feature extraction unit 406 extracts, as the second feature, the feature of the target data different from the first feature from the target data without performing the feature measurement process (S304). Thereafter, the determination unit 407 generates information for determining the quality of the inspection target state based on the extracted first feature and second feature, for example, the above-described non-defective product score (S305). When the non-defective score exceeds the predetermined score, it is determined that the inspection target is a non-defective product and the other items are defective products (S306).
Since the first feature extraction result is an integration of feature measurement results using each of a plurality of processing parameters, it is more robust against fluctuations in inspection conditions and noise than when fixed processing parameters are used. A determination can be made.

  In the first embodiment, an example in which the subspace method is used for quality determination has been described. However, the present invention is not limited to this, and a k-nearest neighbor method, 1-class SVM, or the like can also be used. Further, instead of learning using only non-defective samples, defective samples may be used as learning data, and two-class identification of non-defective or defective products may be performed. In the first embodiment, the determination unit 407 has been described as determining whether or not the product is good. However, even in the case of multi-class identification that determines a plurality of quality grades, it is possible to determine whether the product is good or defective using the same method. It is.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the second embodiment, an example will be described in which an estimated value of a physical quantity obtained by regression analysis is extracted as the first feature instead of the physical quantity measured in the first embodiment. A regression device using a neural network is used for the regression analysis. The hardware configuration of the inspection apparatus used for carrying out the inspection method is the same as that of the first embodiment shown in FIG. The functional block configuration is basically the same as that of the first embodiment shown in FIG. 5, but the operation of the inspection criterion-dependent feature extraction unit 404 is different from that of the inspection apparatus of the first embodiment. That is, the inspection criterion-dependent feature extraction unit 404 inputs a plurality of feature measurement results obtained from a plurality of processing parameters in the feature measurement unit 402 to a regressor using a neural network, and obtains an estimated value. Then, the estimated value is input to the determination unit 407 as the first feature.

For example, when the area of each point measured by a plurality of processing parameters in the feature measurement unit 402 is input, the area of each point estimated from the input is output as an estimated value. Here, let x be the input vector to the neural network and y be the output vector. The input vector x is a feature vector obtained by integrating the results processed by the feature measurement unit 402 using the processing parameters input from the parameter input unit 403 as feature measurement results. For example, it is assumed that thresholds T1, T2, and T3 are used as processing parameters and measurement points as shown in FIG. 7 are obtained. In this case, the input vector x is as follows from FIG.
x = [0.05, 0.0225, 0.01, 0.01, 0.0025, 0, 0.0225, 0.01, 0.005, 0, 0, 0, 0.01, 0.05, 0, 0, 0, 0]

The output y is an estimated value of the areas of measurement points # 1 to # 6 for each sample ID. This estimated value is a six-dimensional vector. When the input to the k-th neuron in the i-th layer of the neural network is x ⁽ⁱ⁾ _k and the output is h ⁽ⁱ⁾ _k , the relationship is expressed by the following equation 2.
h ⁽ⁱ⁾ _k = f ⁽ⁱ⁾ k (W ^{(i) T} _k X ⁽ⁱ⁾ _k ) (Expression 2)
In Equation 2, w ⁽ⁱ⁾ _k is a weight vector, and f ⁽ⁱ⁾ _k (•) is an activation function. Assuming that there are N neurons in the i-th layer, the vector h ⁽ⁱ⁾ summing up all the i-th layers is h ⁽ⁱ⁾ = [h ⁽ⁱ⁾ ₁ , h ⁽ⁱ⁾ ₂ ^{,. )} _N ].

In Equation 2, if using all the output of the i-1 th layer as ^{x _{(i) k,}} it is ^{_{x (i) k = h (}} i-1). When the final layer is the Mth layer, the output y is expressed by Equation 3.
y = [h ^(M) ₁ ,..., h ^(M) ₆ ] (Expression 3)

The six-dimensional vector y is handled in the same manner as the first feature xm described in the first embodiment, and is used for learning and inspection by the determination unit 407.
The inspection standard dependent feature extraction unit 404 learns weight vectors of all neurons using the teaching data. The input vector to the neural network at the time of learning is the result of processing the learning data with candidate parameters. The output is an estimate of the area of the point defect. An error between the output and the teaching data corresponding to the learning data is obtained, and the weight of each neuron is corrected using a known error propagation method or the like.

  As a modification of the second embodiment, the neural network does not output a 6-dimensional vector corresponding to the estimated value of the area of each point-like defect, but instead calculates the value of the difference between the point of the maximum area and the non-defective condition. It may be output. In this case, what is given as teaching data is a value of a difference between the area of the point having the largest area among the point-like defects in the learning data and the non-defective condition.

  As another modified example of the second embodiment, the neural network can also learn to output whether the non-defective product condition is satisfied, that is, the above-described non-defective product score, instead of the estimated value of the area of the point defect. In this case, only information indicating whether or not the learning data is non-defective is provided as teaching data.

As another modified example of the second embodiment, the inspection criterion-dependent feature extraction unit 404 uses the area other than the point defect area (physical quantity) calculated by the neural network or data other than the non-defective product score derived based on the area as the first item. It can also be used as a feature. That is, the intermediate processing value before the non-defective product score is derived can be used as the first feature. Specifically, instead of the last stage output y of the neural network, for example, i-th layer of all outputs integrated vector _{^{h (i) = [h (}} i) 1, h (i) 2, ..., h ( ⁱ⁾ _N ] can be handled in the same manner as the first feature xm, and can be used for learning and checking by the determination unit 407.

The inspection method of the second embodiment is the same as that of the first embodiment except that the inspection criterion-dependent feature extraction unit 404 is a regressor using a neural network. The inspection method in the second embodiment will be specifically described with reference to FIG. First, the learning process of the inspection standard dependent feature extraction unit 404 will be described. It is assumed that the learning data and the teaching data are divided into mini-batches that are a set of a plurality of data.
In the inspection apparatus, one mini-batch is selected by the data input unit 401 (S401), and one of the learning data included in the selected mini-batch is input (S402). The parameter input unit 403 inputs a plurality of processing parameters (S403). The feature measurement unit 402 performs feature measurement processing corresponding to each of the plurality of input processing parameters, extracts point-like defects, and measures the size of each extracted defect (S404).
The constraint condition input unit 405 inputs teaching data corresponding to the input learning data (S405). The inspection criterion-dependent feature extraction unit 404 calculates an estimated value by the current neural network using the measured size of each defect as an input vector, and obtains an error from the teaching data input in S405 (S406). The inspection criterion-dependent feature extraction unit 404 confirms whether or not the processing of S402 to S406 has been completed for all of the mini-batches selected in S401 (S407). If not completed (S407: N), the process returns to S402. If completed (S407: Y), the inspection criterion-dependent feature extraction unit 404 takes the average of the errors calculated in S402 to S406 for all of the mini-batches and updates the weight of each neuron by back propagation or the like. (S408). The inspection criterion-dependent feature extraction unit 404 confirms whether the processing of S401 to S408 has been completed for all mini-batches created in advance (S409), and if not completed (S409: N), returns to S401. If it has been completed (S409: Y), the learning process is terminated.

The learning process of the determination unit 407 is the same as the learning process of the determination unit 407 of the first embodiment shown in FIG. Below, a different point from 1st Embodiment is demonstrated.
In the feature measurement process of S203, the feature measurement unit 402 performs detection and measurement of defects defined in the inspection rule by using all the processing parameters used at the time of learning by the inspection criterion-dependent feature extraction unit 404. In the process of S204, the inspection criterion-dependent feature extraction unit 404 calculates a first feature from the result of the feature measurement process obtained in S203. The calculated first feature is an area of an estimated prescribed number of point-like defects. As another implementation, a value indicating the difference between the area of the largest point-like defect and the area of the point determined by the non-defective product condition or a score indicating the degree of non-defective product may be calculated. Furthermore, as another implementation, the output of the intermediate layer of the neural network may be a feature amount calculated by the inspection criterion-dependent feature extraction unit 404.

  Next, the inspection process of the second embodiment will be described. This inspection process is the same as the inspection process of the first embodiment shown in FIG. The difference is that the feature measurement processing in S302 performs detection and measurement of defects defined in the inspection rule using all the processing parameters used at the time of learning by the inspection criterion-dependent feature extraction unit 404.

  In the inspection standard dependent feature extraction unit 404, an example in which a neural network regressor is used has been described, but other methods may be used. For example, a linear regressor, SVRRF (Support Vector Regression, Random Forest), or the like can be used. In addition, when the neural network is composed of a plurality of weak regressors, such as Random Forest, a feature vector output by the inspection criterion-dependent feature extraction unit 404 is a vector whose elements are the outputs of the weak regressors. It can also be.

For example, it is assumed that the regression unit of the inspection criterion-dependent feature extraction unit 404 is expressed as the following Expression 4. In Expression 4, x is a vector whose elements are areas of points measured by a plurality of parameters in the feature measurement unit 402, and f (x) is a weak regressor constituting the regressor.

  The inspection criterion-dependent feature extraction unit 404 may output an estimated value y of the area of point-like defects as an output, or may output a vector having the outputs f of N weak regression units as elements. The above description is the same for the pass / fail discrimination function of the discriminator 407 instead of the regressor.

[Third Embodiment]
Next, an inspection apparatus according to the third embodiment of the present invention will be described. In the third embodiment, an example is shown in which the inspection target is a drive portion of the device, and the target data representing the state is acoustic data that indirectly represents the state of the inspection target. The determination of pass / fail is whether or not the device is operating normally. The hardware configuration of the inspection apparatus according to the third embodiment is the same as that of the first embodiment shown in FIG. 4, and the functional block configuration is the same as that of the second embodiment. Therefore, the same components or the same functions will be described with the same reference numerals as those of the inspection apparatus according to the first embodiment or the second embodiment.

  In the third embodiment, a one-minute test run is performed with a microphone attached to a specific location of the device. At this time, if the acoustic data acquired from the microphone represents an abnormal sound, it is determined that the normal operation is performed if the number of occurrences of the abnormal sound is within a specified value. That is, the inspection apparatus according to the third embodiment measures the number of occurrences of abnormal noise, which is an inspection standard-dependent feature, and determines whether the device is good or bad together with other acoustic feature amounts (operation sound that does not depend on the inspection standard-dependent feature). Do.

  In the inspection apparatus according to the third embodiment, acoustic data from a microphone attached to a specific portion of the device is input to the data input unit 401 as input data. When learning is performed by the inspection criterion-dependent feature extraction unit 404 and the quality determination unit 407, acoustic data recorded in advance is input as learning data. At the time of inspection, acoustic data acquired in real time in a state where the device is actually operated is input. However, it may be acoustic data recorded in advance.

The feature measurement unit 402 detects when an abnormal sound is included in the input acoustic data, and measures the number of times the abnormal sound is detected within a predetermined time. Examples of acoustic data are shown in FIGS. 14 (a) and 14 (b). The vertical axis of the graph is the absolute value | P | of the sound pressure P, and the horizontal axis is the time after the start of the trial run. When the absolute value of the sound pressure P exceeds a predetermined threshold, the inspection device recognizes that an abnormal sound has occurred. The predetermined threshold is given from the parameter input unit 403. An example will be described with reference to FIG.
Here, it is assumed that the threshold is Tp1. | P | exceeds the threshold value Tp1 at times t2 and t3. In this case, two abnormal sounds are detected. Similarly, abnormal noise is detected three times when the threshold value is Tp2, and five times when the threshold value is Tp3.

The inspection criterion-dependent feature extraction unit 404 outputs a value obtained by integrating the outputs of the feature measurement unit 402 corresponding to the three threshold values Tp1, Tp2, and Tp3 input from the parameter input unit 403 to the determination unit 407.
As in the second embodiment, the inspection criterion-dependent feature extraction unit 404 in the third embodiment is realized by a regressor using a neural network. However, other regressors can be used. For example, it is assumed that threshold values Tp1, Tp2, and Tp3 are input from the parameter input unit 403 as a plurality of processing parameters for the data in FIG. The outputs of the feature measurement unit 402 in this case are “2”, “3”, and “5” corresponding to the number of occurrences of abnormal noise in each case. At this time, the feature vector input to the inspection criterion-dependent feature extraction unit 404 is [2, 3, 5], and the output of the inspection criterion-dependent feature extraction unit 404 is the difference estimated by the regressor using this feature vector as an input. This is the number of sound detections.

  The inspection standard dependent feature extraction unit 404 performs learning based on teaching data. The teaching data is input from the constraint condition input unit 405. FIG. 15 shows an example of teaching data. In the teaching data, the ID of each sample, the determination result of pass / fail, and the number of times abnormal noise is determined by the inspector are recorded. The inspector listens to the operating sound measured from the microphone attached to the device, detects whether there is any abnormal sound, and records the number of times the abnormal sound is detected.

The inspection criterion-independent feature extraction unit 406 extracts an acoustic feature quantity that does not depend on teaching data from the input acoustic data. As the acoustic feature amount, for example, a power spectrum or a phase spectrum of short-time discrete Fourier transform can be used.
The discriminating unit 407 is a discriminator having the output of the measurement amount integration unit 413 and the output of the inspection reference independent feature extraction unit 406 as the feature amount, similar to the first embodiment and the second embodiment 2. Pass / fail judgment is performed.

  The procedure of the inspection method according to the third embodiment is substantially the same as the inspection method of the second embodiment. That is, first, the inspection criterion-dependent feature extraction unit 404 learns according to the processing flow of FIG. Further, the learning of the determination unit 407 is performed according to the processing flow of FIG. Then, the quality of the device to be inspected is determined according to the processing flow of FIG. The difference from the second embodiment is that the target data and learning data input to the data input unit 401 are acoustic data, and the processing parameter input from the parameter input unit 403 is a threshold for the absolute value of the sound pressure of the acoustic signal. It is a point. The point that the teaching data is divided into a plurality of mini-batches that are a set of a plurality of data is the same as in the second embodiment. In the third embodiment, the feature measurement unit 402 performs a process of counting the number of times that the absolute value of the sound pressure of the acoustic signal exceeds the threshold value (plural). Further, the inspection criterion-dependent feature extraction unit 404 calculates an estimated value by the current neural network using the count number for each threshold as an input vector. At the time of learning, an error between the estimated value and the teaching data is obtained.

  In the third embodiment, an example in which the target data representing the state of the drive portion of the device is acoustic data has been described. However, the number of abnormal vibrations caused by a failure of the device by providing a vibration sensor or acceleration sensor in the drive portion. Etc. may be measured. The target data in this case is vibration data that indirectly represents the state of the drive portion, and the processing parameter is, for example, an amplitude value (plural) of abnormal vibrations.

[Modification]
In the first to third embodiments, the example in which the quality of the inspection target is determined by the characteristic measurement process of the state of the inspection target has been described. However, the present invention can be used for classification of a plurality of grades. For example, it can be used to inspect the degree of product scratches in the manufacturing process. It can also be used to determine the culture state of cells and microorganisms. Further, it can be used to determine how good the skin condition is. The processing parameters in these cases are different threshold values depending on the grade, for example.

  The present invention also provides a computer program for realizing one or more functions of the first to third embodiments described above to a computer via a network or a storage medium, and one or more processors in the computer execute the program. It can also be realized by a process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (13)

Data input means for inputting target data representing the state of the inspection target;
Feature measurement means for performing each feature measurement process using a plurality of different processing parameters for the target data;
First feature extraction means for integrating the results of the respective feature measurement processes and extracting the first feature of the target data from the integrated results;
Discrimination means for discriminating whether the state is good or not based on the extracted first feature;
An inspection apparatus comprising: The first feature extraction means extracts a physical quantity that enables comparison with a predetermined inspection standard as the first feature.
The inspection apparatus according to claim 1. The first feature extraction means extracts an estimated value of the physical quantity by regression analysis instead of the physical quantity as the first feature.
The inspection apparatus according to claim 2. The estimated value is a non-defective score obtained by scoring the degree of quality of the state or an intermediate processing value before derivation of the non-defective score,
The inspection apparatus according to claim 3. A second feature extracting means for extracting a feature of the state different from the first feature as the second feature from the target data without using the feature measurement processing;
The determining means determines the quality of the state based on the extracted first feature and the second feature,
The inspection apparatus according to any one of claims 1 to 4. The second feature is statistical information that statistically represents the degree of peculiarity of the state,
The inspection apparatus according to claim 5. Learning means for learning the feature measurement processing using learning data having the same data structure as the target data input instead of the target data and a plurality of candidate parameters input instead of the plurality of processing parameters Further comprising:
The inspection apparatus according to any one of claims 1 to 6. The learning means determines some of the plurality of candidate parameters as the plurality of processing parameters according to the learning result,
The inspection apparatus according to claim 7. The target data is an image obtained by imaging the state of the inspection target, or acoustic data or vibration data that indirectly represents the state of the inspection target.
The inspection apparatus according to any one of claims 1 to 8. A method executed by a computer in which target data representing a state of an inspection target and a plurality of processing parameters different from each other are input,
Performing each feature measurement process using the plurality of processing parameters for the target data;
Integrating the results of the respective feature measurement processes, and extracting a physical quantity that enables comparison with a predetermined inspection standard from the integrated results as the first feature of the target data;
Extracting the feature of the target data different from the first feature as the second feature from the target data without performing the feature measurement process;
Generating information for determining the quality of the state based on the extracted first feature and the second feature;
An inspection method characterized by comprising: Inputting learning data having the same data structure as the target data before inputting the target data;
Inputting a plurality of candidate parameters that are candidates for the processing parameter;
A feature measurement process using each of the plurality of candidate parameters is performed on the learning data, and an evaluation value of each candidate parameter is calculated by comparing a result of each feature measurement process with a predetermined condition. An evaluation process;
Determining the plurality of processing parameters according to the evaluation value;
Further comprising:
The inspection method according to claim 10. Further inputting the learning data before inputting the target data;
A step of performing feature measurement processing using each of a plurality of processing parameters determined according to the evaluation value for the input learning data;
Integrating the results of the respective feature measurement processes, extracting a physical quantity that enables comparison with the inspection standard from the integrated results as a first learning feature of the learning data;
Extracting a feature of the learning data different from the first learning feature as the second learning feature from the learning data without performing the feature measurement process;
Learning a function of determining the quality of the state based on the extracted first learning feature and the second learning feature;
The inspection method according to claim 11, further comprising:   A computer program for causing a computer to execute the inspection method according to any one of claims 10 to 12.