JP5441728B2 - Sensory inspection device and sensory inspection method - Google Patents

Description

  The present invention relates to a sensory inspection apparatus and a sensory inspection method for automatically inspecting visual inspection of a display, abnormal sound inspection of various machines, and the like.

  In recent years, automation has progressed in the manufacturing process of various industrial products, and the inspection process is no exception. On the other hand, however, inspections that require sensual judgment, such as display unevenness and abnormal noise of mechanical parts, still rely on subjective inspections such as visual inspection and hearing by the operator. However, in the inspection by the subjectivity of the worker, the sensory inspection for a long time puts a burden on the worker, the variation due to individual differences, physical condition, time zone, etc. is large. It is difficult to maintain the uniformity of the test results because it takes time to grow and the processing ability is low. Therefore, even in sensory inspections, automation that does not depend on the subjectivity of workers is required.

As a method for automating such a sensory test, there is a method using pattern recognition.
Here, the inspection by pattern recognition will be described with reference to FIG.
FIG. 13 is a diagram showing a feature space in an inspection using conventional pattern recognition.

  In the inspection using pattern recognition, first, some feature amounts are extracted from the information obtained by detecting the inspection target with a detector, and plotted in a feature amount space with each feature amount as a coordinate axis, as shown in FIG. Then, it is classified into a non-defective product, a defective product, or a gray product (a product that cannot be judged as a good product or a defective product) depending on which region it belongs to when plotted. In FIG. 13, for simplicity, the feature amount is two, but it may be one or three or more. At this time, to determine which region is non-defective, defective, or gray, prepare samples in advance that are known to be good or bad, learn from the distribution when plotting them in the feature space, and divide the region I do.

  As a learning and classification method, a method using a neural network, an SVM (support-vector-machine, hereinafter abbreviated as SVM), a self-organizing map, an MT (Mahalanobis Taguchi, hereinafter abbreviated as MT) method, or the like is used. is there. In addition, a method using a neural network generates pseudo defects based on non-defective and defective samples and uses them as learning samples, thereby increasing the accuracy of determination with a small number of learning samples. (For example, see Patent Document 1).

Hereinafter, the quality area learning of the feature amount space in the conventional sensory test apparatus will be described with reference to FIGS.
FIG. 14 is a diagram showing a configuration of a learning device for a pass / fail region in a feature amount space in a conventional sensory inspection device, and shows a conventional block diagram described in Patent Document 1. In FIG. FIG. 15 is a diagram illustrating a state in which defective products are synthesized from non-defective products in a feature amount space in a conventional sensory inspection apparatus.

  In FIG. 14, it is possible to check the quality of the product using the neural network 101. The non-defective product learning is performed by reading a pre-stored good product image by the good product image input unit 102 and inputting it to the neural network 101 via the pre-processing unit 107 that performs filter processing and feature quantity extraction. Let students learn.

  In order to learn defective products, when the number of images of defective products is equal to or greater than the required learning number, a pre-stored defective product image is read and input to the neural network 101 via the preprocessing unit 107. Learning is performed as defective product data.

  However, since it is usually difficult to obtain the required number of original defective data, the defective image extraction unit 104 that extracts the difference data 108 from the defective image from the defective image, and the non-defective product input unit 102. A condition for determining how to synthesize the difference data 108 with a synthesis position and the like is determined based on the random number value of the random number generation unit 109 by reading an image and synthesizing the difference data 108 to generate a pseudo defect image. And a pseudo data condition setting unit 105 for instructing the pseudo defective image creating unit 106.

  Thereby, in the conventional configuration, as shown in FIG. 15, a large number of pseudo-defective images are created in the vicinity based on the actual defective image, and the number of learning samples is increased as shown in FIG. It was intended to improve the accuracy of the pass / fail judgment.

JP 2005-156334 A

However, even in the conventional configuration, the problem in the automation of inspection using pattern recognition cannot be sufficiently solved.
A problem in the automation of inspection using pattern recognition is that it is difficult to learn a boundary for making the same determination as a person because there are few effective samples for determining the boundary between a non-defective product and a defective product.

  This will be described in detail with reference to FIG. 16 showing the distribution of non-defective products and defective products in the conventional feature amount space. In general, since the variation of non-defective products has a normal distribution, there are many samples near the average value, and there are few non-defective products near the boundary. In addition, the frequency of occurrence of defective products is generally low, and even if a defect occurs, the sample is likely to deviate greatly from the boundary. When this is plotted in the feature amount space, it is as shown in FIG. In FIG. 16, since the interval between the non-defective product and the defective product is large, it is possible to form a large number of boundary lines for discriminating between the non-defective product and the defective product, and it is difficult to uniquely define the boundary line. I can't.

  Further, the invention relating to the pseudo-defective image automatic creation device and the image inspection device disclosed in Patent Document 1 supplements the number of defective products that are small, and will be described in the feature amount space. As shown in FIG. This is a method of increasing defective product data in the vicinity based on the data, but the number of samples that can be increased is limited to the periphery of the basic defective product data. For this reason, it cannot be expected that inspection accuracy will be greatly improved.

  SUMMARY OF THE INVENTION The present invention solves the above-described conventional problems, and an object thereof is to realize a sensory test that makes it possible to define a pass / fail judgment standard close to human judgment with a small number of learning samples.

  In order to achieve the above object, a sensory test apparatus according to the present invention includes a pseudo sample creation unit that creates a pseudo sample, a determination input unit that inputs a pass / fail result determined by a person for the pseudo sample, and the determination input. A pseudo sample feature quantity generating unit that generates a feature quantity near the boundary between a non-defective product and a defective product from the pass / fail result input in the unit and uses it as a feature quantity of the pseudo sample; and the pseudo sample created by the pseudo sample creating unit A SVM determination unit that creates a boundary line between a non-defective product and a defective product from the feature amount and performs a pass / fail determination by SVM analysis, and a determination comparison unit that compares the pass / fail result at the determination input unit with the pass / fail result at the SVM determination unit And a re-determination input unit that inputs a re-determination result re-determined by a person with respect to a pseudo sample having a different pass / fail result in the determination / comparison unit, and for the pass / fail result input in the re-determination input unit, the S A weighting processing unit for adding a sample by adding a weight according to a distance from a boundary line created by the M determination unit; and a sensory test unit for performing a pass / fail determination by adding the sample added by the weighting processing unit. It is characterized by that.

In order to achieve the above object, the sensory test method of the present invention includes a pseudo sample creating step of creating a pseudo sample, and a determination input step of inputting a pass / fail result when a person makes a judgment on the pseudo sample. From the pseudo sample feature value generating step of generating a feature value in the vicinity of the boundary between a non-defective product and a defective product from the pass / fail result input in the determination input step to be a feature value of the pseudo sample, and from the feature value of the pseudo sample The SVM determination process for determining pass / fail by SVM analysis for creating a boundary line between a non-defective product and a defective product, the determination comparison step for comparing the results of the determination input step and the SVM determination step, and the determination / comparison step are different. A re-determination input step for inputting a pass / fail result re-determined by a person for a pseudo sample, and a result input in the re-determination input step, created in the SVM determination step A weighting processing step of adding a pseudo sample by adding weighted according to the distance from the boundary line was, and characterized in that it has a functional inspection step of performing quality determination by adding the added samples by the weighting processing step To do.

  As described above, according to the present invention, it is possible to define a pass / fail judgment criterion that is close to human judgment with a small number of learning samples.

(A) A block diagram showing a pseudo sample generation system of the sensory test apparatus in the first embodiment of the present invention, (b) a block diagram showing a pattern recognition learning device of the sensory test apparatus in the first embodiment. Flowchart showing the flow of pseudo sample creation processing in the first embodiment (A) The figure which showed the example of the quality product image among the images which imaged LCD in this Embodiment 1 with the camera, (b) The example of inferior product image among the images which imaged LCD in this Embodiment 1 with the camera Figure showing The flowchart which shows the flow of the feature-value extraction in this Embodiment 1. (A) The figure explaining the feature-value of the initial sample in the feature-value space in this Embodiment 1, (b) The figure explaining the quality result of the initial-sample in the feature-value space in this Embodiment 1, (c) This The figure which shows the increase method of the pseudo sample in Embodiment 1, (d) The figure which shows the quality result after the pseudo sample increase in the feature-value space in this Embodiment 1. The block diagram which shows the structure of the pseudo sample preparation part in this Embodiment 1. FIG. The flowchart which shows the flow of the pattern recognition learning method in this Embodiment 1. The figure which illustrates the feature-value space mapping in this Embodiment 1. The figure which illustrates the result of having determined the pseudo sample on the feature-value space in this Embodiment 1 by the person The figure which illustrates the determination result of SVM on the feature-value space in this Embodiment 1. The figure explaining the weighting processing method on the feature-value space in this Embodiment 1. (A) The figure explaining the feature-value of the initial sample in the feature-value space in Embodiment 2 of this invention, (b) The figure explaining the quality result of the initial-sample in the feature-value space in this Embodiment 2, (c) ) A diagram showing a method for increasing pseudo samples in the second embodiment, and (d) a diagram showing a pass / fail result after increasing pseudo samples in the feature amount space in the second embodiment. Diagram showing feature space in inspection using pattern recognition The figure which shows the structure of the quality area learning apparatus of the feature-value space in the conventional sensory test apparatus. The figure which shows a mode that the inferior goods are synthesize | combined in the feature-value space in the conventional sensory test apparatus. Diagram showing the distribution of non-defective and defective products in the conventional feature space

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals, and description thereof is omitted as appropriate.
(Embodiment 1)
First, the sensory test apparatus and the sensory test method according to the first embodiment will be described with reference to FIGS.

  FIG. 1A is a block diagram showing a pseudo sample generation system of the sensory test apparatus according to Embodiment 1 of the present invention. FIG.1 (b) is a block diagram which shows the pattern recognition learning apparatus of the sensory test apparatus in Embodiment 1 of this invention. In the first embodiment, the pseudo sample generation system is a pseudo sample generation apparatus.

  In FIG. 1 (a), a non-defective product data input unit 1 reads pre-stored non-defective product images, audio data, etc., and extracts them through a feature amount extraction unit 2 that performs filtering and feature amount extraction. The feature amount is input to the feature amount space creation unit 3. The feature amount space creation unit 3 performs normalization and coordinate transformation based on the feature amount distribution, creates a feature amount space, and the feature amount space storage unit 4 stores the coordinate system. Further, the sample feature quantity generation unit 5 generates a feature quantity of an initial sample based on the feature quantity space created by the feature quantity space creation unit 3 and outputs it to the feature quantity space storage unit 4 and the pseudo sample creation unit 6.

  The pseudo sample creation unit 6 creates a pseudo sample by synthesizing the feature values of the initial sample generated by the sample feature value generation unit 5 with the good product data input from the good product data input unit 1. At this time, if the feature amount has a degree of freedom, the random number generation unit 7 may determine the feature amount within the range of the degree of freedom and use it for creating a pseudo sample. Then, the pseudo sample display unit 8 displays the pseudo sample created by the pseudo sample creation unit 6. In the judgment input unit 9, the inspector can input the quality of the displayed pseudo sample, and the result is sent to the feature amount space storage unit 4. The feature amount space storage unit 4 stores the feature value of the pseudo sample input from the sample feature amount generation unit 5 and the determination result input from the determination input unit 9 corresponding thereto as a set.

  The pseudo sample feature quantity generation unit 10 receives the feature quantity space stored in the feature quantity space storage unit 4 and the pseudo sample determination result, and draws a boundary line (a line representing a boundary between a good product and a defective product) by pattern recognition. A new sample feature quantity effective for the above is generated and input to the pseudo sample creation unit 6 and the feature quantity space storage unit 4. This feature amount is stored in the feature amount space storage unit 4 as a set together with the determination result in the same manner as the feature amount of the pseudo sample.

FIG. 2 is a flowchart showing the flow of the pseudo sample creation process in the first embodiment of the present invention.
As shown in FIG. 2, when creating a pseudo sample, first, the non-defective product data input unit 1 reads a plurality of non-defective product data detected by the detecting unit stored in advance (step S001). Next, the feature quantity extraction unit 2 extracts the feature quantity from each good product data (step S002). Then, the feature amount space creation unit 3 calculates the Mahalanobis distance based on the average value and variation of each good product data, thereby normalizing each feature amount, and the feature amount indicating the distribution of each normalized feature amount A space is created, and the feature amount space storage unit 4 stores the feature amount space (step S003). Next, the sample feature value generation unit 5 creates a plurality of samples whose Mahalanobis distance from the origin is a constant value as an initial sample (step S004). At this time, an initial sample in which the Mahalanobis distance is within a certain range may be created, and the certain range is preferably within a range of ± 10% from a certain value. Specifically, the Mahalanobis distance from the origin is preferably 2-5. In addition, when there are a plurality of initial samples, it is preferable that they are equally arranged on a hypersphere centered on the origin of the feature amount space. Subsequently, the pseudo sample creation unit creates standard good product data from the average value of the non-defective product data read in step S001, and synthesizes the feature quantity of the created initial sample with the feature quantity of the standard good product data to create a pseudo sample. (Step S005). Next, the created pseudo sample is displayed on the pseudo sample display unit 8, and the quality of the displayed pseudo sample is judged by a skilled inspector or a person in charge of quality assurance, and the result is input to the judgment input unit 9. (Step S006). Next, the input determination result and the feature amount of the pseudo sample corresponding to the determination result are added to the feature amount space (step S007).

  Here, when the number of samples in the feature amount space is less than the predetermined value N, a feature amount of an additional sample is newly generated from the stored feature amount of the sample. As the fixed value N of the number of samples, a number based on the experience of the inspector or a number required for design is set as appropriate (step S008). After step S008, the process returns to step S005, and a pseudo sample is created again based on the feature amount of the newly generated additional sample. When the number of samples is equal to or greater than a certain value N, the creation of the pseudo sample is completed. The pseudo sample creation is thus completed, and the pattern recognition learning can be performed (step S009).

Next, the learning operation of the present embodiment will be specifically described with reference to FIGS.
FIG. 3 is an example of an image obtained by capturing an image of an LCD (liquid crystal display) with a camera, and is a diagram showing a line luminance file in which a part thereof is viewed in both the X direction and the Y direction. 3A is a non-defective image, and FIG. 3B is a defective image when there is unevenness. In the case of the non-defective image in FIG. 3A, there is a large and smooth shading, but the local fluctuation is small. On the other hand, the defective product image of FIG. 3B has local fluctuations in addition to the same shading as the non-defective product image, which becomes defective as unevenness. The shading referred to here is a current situation where the center of the image is brighter than the edges, and the line luminance file of the entire image looks like a mountain. In addition, the portion of the luminance distribution that does not match the partially adjacent luminance flow that is locally generated with respect to the overall generated shading is made uneven.

  First, in order to obtain the average value and variation of standard good products, images acquired from n good products are read. In order to obtain the average value and the variation, it is better that the number of non-defective products is large. However, it is difficult to prepare a sample at the beginning of the start-up, and it takes time. 2 step S001).

  Next, the feature amount of the non-defective product is extracted. This process will be described with reference to FIG. FIG. 4 is a flowchart showing a flow of feature amount extraction in Embodiment 1 of the present invention. When extracting the feature amount of the non-defective product, first, the non-defective product data input unit 1 inputs a good product image (step S021). Next, an image obtained by removing local fluctuations from the non-defective image is obtained by passing through a low-pass filter (step S022). By subtracting this from the original non-defective image, it is possible to remove the influence of shading and obtain an image with only local fluctuations (step S023). Then, this image is binarized with a certain threshold value (step S024). After performing the binarization process, a plurality of uneven portions are labeled (step S025). Next, an area and a volume are calculated for each labeled part (step S026). Among these, the maximum luminance with the highest luminance is searched as the maximum unevenness (step S027). Finally, the maximum unevenness area Si (i = 1, 2,..., N) and the volume Vi (i = 1, 2,..., N) are output as feature amounts (step S028). The reason why only the largest unevenness is determined is that if it is a non-defective product level, other small unevenness is acceptable, and if it is a defective product level, at least larger unevenness can be determined as defective. Yes (see step S002 in FIG. 2).

In this way, when the feature amount of the non-defective product is extracted, the feature amount space creation unit 3 creates a feature amount space based on the Mahalanobis distance. When the average values of the area Si and the volume Vi obtained in step S002 are ms and mv, the standard deviations are σs and σv, respectively, and the feature quantities obtained by normalizing the area Si and the volume Vi are si and vi, respectively, si, vi is obtained by the following equation (equation 1).

次に、正規化した特徴量の相関行列Ｒを次式（式２）で求める。

Next, the correlation matrix R of the normalized feature quantity is obtained by the following formula (Formula 2).

ここで、ｒ１２、ｒ２１は次式（式３）で表される。

Here, r12 and r21 are expressed by the following formula (formula 3).

そして、相関行列Ｒの逆行列Ａを求める。

Then, an inverse matrix A of the correlation matrix R is obtained.

このとき、相関行列Ｒは対称行列のため、逆行列Ａも対称行列である。そのため、行列Ａを固有値分解すると、次式（式５）で表すことができる。

At this time, since the correlation matrix R is a symmetric matrix, the inverse matrix A is also a symmetric matrix. Therefore, when the matrix A is subjected to eigenvalue decomposition, it can be expressed by the following equation (Equation 5).

ここで、Λは固有値を対角上に並べた対角行列であり、Ｘ’はＸの転置行列である。次に、Λ^１／２をΛの各要素を１／２乗したものとして、特徴量ｓｉ、ｖｉを次式（式６）で変換する。

Here, Λ is a diagonal matrix in which eigenvalues are arranged diagonally, and X ′ is a transposed matrix of X. Next, assuming that [Lambda] ^{1/2 is obtained} by raising each element of [Lambda] to the 1/2 power, the feature quantities si and vi are converted by the following equation (Equation 6).

そして、この変換した特徴量ｓｉ’、ｖｉ’を特徴量空間とする。このとき、特徴量空間の原点からの距離｜｜ｘ’｜｜がマハラノビス距離となっている（図２のステップＳ００３参照）。

The converted feature quantities si ′ and vi ′ are set as a feature quantity space. At this time, the distance || x ′ || from the origin of the feature amount space is the Mahalanobis distance (see step S003 in FIG. 2).

  Next, a feature quantity of the initial sample is generated based on the feature quantity space. At this time, as shown in FIG. 5A, a plurality of samples are generated such that the distance from the origin of the feature amount space is a constant a. Here, when the defect rate of the process is known, a is a multiple of the standard deviation of the feature amount that is the defect rate. For example, if the defect rate is 0.3%, a = 3, and if the defect rate is 4.6%, a = 2. When the defect rate of the process is unknown, an estimated defect rate is used, or a value empirically obtained as a standard process capability is used, for example, a = 3. As a result, the distance from the origin of the feature amount space is the Mahalanobis distance, and the initial sample is generated at a distance corresponding to the defect rate, so the initial sample is close to the sample near the boundary between the non-defective product and the defective product. . Therefore, necessary samples can be secured and the number of samples necessary for learning can be reduced (see step S004 in FIG. 2).

  Next, a pseudo sample is created based on the feature amount of the generated initial sample. FIG. 6 is a block diagram showing the configuration of the pseudo sample creation unit in the first embodiment of the present invention. In FIG. 6, a non-defective image input unit 21 inputs n non-defective images, and an average image creating unit 22 averages the feature amounts of the n non-defective images to create a non-defective average image. The feature amount input unit 23 inputs the feature amounts si ′ and vi ′ generated by the initial sample feature amount generation unit 5 and the pseudo sample feature amount generation unit 10. The feature quantities si ′ and vi ′ are converted into the area Si and the volume Vi by the following formulas (formula 7) and (formula 8) in the feature quantity conversion unit 24.

特徴量は面積Ｓｉと体積Ｖｉのみしか決まっていないため、ムラの位置や形状には自由度がある。そこで、乱数発生部７で発生した乱数が乱数入力部２５に入力され、パラメータ決定部２６でムラの位置、形状を決定する。最後に、画像合成部２７で、良品の平均画像にムラを合成して擬似サンプルを作成する（図２のステップＳ００５参照）。

Since the feature amount is determined only by the area Si and the volume Vi, the position and shape of the unevenness have a degree of freedom. Therefore, the random number generated by the random number generation unit 7 is input to the random number input unit 25, and the parameter determination unit 26 determines the position and shape of the unevenness. Finally, the image synthesis unit 27 creates a pseudo sample by synthesizing the non-defective average image with the unevenness (see step S005 in FIG. 2).

  Next, the created pseudo sample is displayed on the pseudo sample display unit 8, and the inspector divides the evaluation into four stages: a non-defective product, a non-defective product near the boundary, a defective product near the boundary, and a defective product. The result is output to 9 (see step S006 in FIG. 2).

  Subsequently, the generated feature quantities si ′ and vi ′ and the determination result determined in step S006 are stored in the feature quantity space storage unit 4 as a set. The result at this time is a non-defective product, near the boundary with respect to a sample whose distance from the origin of the feature amount space is constant, as shown in the diagram explaining the pass / fail result of the initial sample in the feature amount space in FIG. The result shows the four-stage evaluation results of the non-defective product, the defective product near the boundary, and the defective product (see step S007 in FIG. 2).

  Since at least N samples are required to accurately determine the pass / fail judgment boundary, if the number of generated pseudo samples is less than N, a new pseudo sample is generated. A new pseudo sample is first generated by the pseudo sample feature value generation unit 10, and there are three methods for generating the feature value. This method will be described with reference to FIG. 5C showing a method for increasing pseudo samples. The first is a feature amount that is changed from the feature amount of the sample determined to be non-defective to the origin by a predetermined amount (the Mahalanobis distance is reduced). The second is a predetermined amount based on the feature amount of the sample that is determined to be defective. The feature amount was changed only in the direction opposite to the origin (the Mahalanobis distance was increased), and the third one was selected from the feature amounts of the samples that were determined to be non-defective products near the boundary and defective products near the boundary. This is a method of calculating the feature amount by averaging the magnitude and direction of the vector. As an averaging method, the distance and the direction from the origin of the feature amount can be synthesized at the same ratio, and for example, they can be synthesized at a ratio of 1/2. By generating a pseudo sample having such a feature amount, it is possible to increase the number of pseudo samples near the boundary between the non-defective product and the defective product (see step S008 in FIG. 2).

  The above cycle is repeated until the number of pseudo samples reaches N or more. As a result, as shown in the diagram showing the pass / fail result after the pseudo sample increase in the feature amount space in FIG. 5D, the number of samples near the boundary between the non-defective product and the defective product increases. And since the determination result of the non-defective product and the defective product used for the generation of the pseudo sample is a result determined by the person, the determination result of the non-defective product and the defective product of the generated pseudo sample is close to the result of the human determination. It is possible to accurately determine a boundary line between a non-defective product and a defective product as indicated by a broken line as a boundary line close to the above determination.

Finally, all the generated pseudo samples are displayed again and detected by the pattern recognition learning system (see step S009 in FIG. 2).
In the pattern recognition learning system, learning is performed using the pseudo sample created in this way.

In the present embodiment, the feature amount has been described in two dimensions of area and volume. However, the feature amount is not particularly limited to two dimensions, and may be m dimensions. In this case, m-number of yji a normalized feature value (j = 1,2, ..., m , i = 1,2, ..., n) Equation 2 as Formula 3 is expressed by the following equation (Equation 9), (formula 10).

なお、本実施の形態において、初期サンプルの特徴量を生成時の原点からの距離を一定値ａとして生成したが、定める距離にある程度の範囲のばらつきを持たせても良い。そうすることで、擬似サンプル生成時に異なるマハラノビス距離のサンプルを発生させやすくなる。

In this embodiment, the initial sample feature amount is generated with the distance from the origin at the time of generation as a constant value a. However, the determined distance may have a certain range of variation. By doing so, it becomes easy to generate samples of different Mahalanobis distances when generating pseudo samples.

Hereinafter, the learning apparatus and the learning method will be described with reference to FIGS.
In FIG. 1B, a detection unit 11 acquires an image or sound generated by a real sample, a pseudo sample generation device, or a pseudo sample generation method, and a CCD camera, a microphone, or the like is used. The feature quantity extraction unit 12 extracts a feature quantity from the image or sound acquired by the detection unit 11 and outputs the feature quantity to the feature quantity space creation unit 13. The feature amount space creation unit 13 performs normalization and coordinate conversion based on the feature amount distribution, creates a feature amount space, and the feature amount space storage unit 15 stores the coordinate system of the feature amount space. Further, the feature amount of the pseudo sample extracted by the feature amount space creation unit 13 is output to an SVM (support-vector-machine) determination unit 16. In addition, the determination input unit 14 inputs a result of whether a person has determined whether the created pseudo sample is good or bad. The feature amount space storage unit 15 stores the feature amount value input from the feature amount space creation unit 13 and the determination result input from the corresponding determination input unit 14 as a set. The SVM analysis is performed from the feature amount space of the pseudo sample input to the SVM determination unit 16, and the created pseudo sample is mechanically determined as pass / fail. The determination comparison unit 17 compares the determination result made by the person in the determination input unit 14 with the mechanical determination result of the SVM determination unit 16 in the determination comparison unit 17, and re-determination input unit only the pseudo samples having different pass / fail determinations 18 is read again, and the person makes a pass / fail judgment again.

  Here, consider again the case where the result of the person's determination of pass / fail is a non-defective product. In the weighting processing unit 19, the point P on the feature amount space of the re-determined non-defective product pseudo-sample and the barycentric point O of the determination space group of the non-defective product (good / bad result opposite to the re-determined quality) are connected with a straight line, When the intersection of the straight line and the boundary line of the SVM analysis pass / fail judgment space is Q, the length of the line segment OQ is L1, and the length of the line segment QP is L2, the length of the two line segments is Accordingly, the feature amount of the re-determined non-defective sample is weighted by the number of M × L2 / L1 (M is a weighting factor). Then, the pass / fail determination result is input again to the SVM determination unit 16, the SVM analysis is performed, and the determination comparison unit 17 checks whether the determination result of the SVM determination unit 16 and the determination result of the redetermination input unit 18 are different. If there is a mismatch, the same processing is repeated until they match. If the determination result of the learned pseudo sample becomes the same in the SVM determination unit 16 and the redetermination input unit 18, the learned pass / fail space is output to the pass / fail space output unit 20. The actual inspection is performed using the created pass / fail space, and the pass / fail is determined from the feature quantity of the actual sample and the inspection result is taught.

FIG. 7 is a flowchart showing the flow of the pattern recognition learning method according to Embodiment 1 of the present invention.
As shown in FIG. 7, first, the pseudo sample created by the pseudo sample creation unit 6 is read out to the detection unit 11 (step S011). Next, the feature amount extraction unit 12 extracts a feature amount from each sample (step S012). The feature amount space creation unit 13 maps the feature amount of each sample on the feature amount space, and stores it in the feature amount space storage unit 15 (step S013). Next, a skilled inspector or a quality assurance manager or the like performs a quality judgment on the read pseudo sample, and inputs the result to the judgment input unit 14 (step S014). Subsequently, the feature amount extracted from the pseudo sample is input to the SVM determination unit 16 to perform SVM analysis (step S015). The determination comparison unit 17 compares the determination result of the person made in step S014 with the determination result of SVM for each pseudo sample (step S016). If the determination results are different (including erroneous determination), the erroneously determined pseudo sample is determined again and reviewed. Here, in the case of erroneous determination, a case where a person has determined that the product is a non-defective product is an “overdetection” that is determined to be a defective product as a result of SVM analysis, and a case where a person has determined that the product is defective is a SVM analysis. As a result, there are two cases of “missing” which is judged as a non-defective product. At that time, the weighting processing unit assigns the weight of the redetermination result to the feature amount space with respect to the judgment result (step S017). Further, if the same processing is repeated and there is no erroneous determination, the same pass / fail space as the human judgment is created, so that the boundary line used for the inspection becomes clear and it is possible to shift to the inspection of the actual sample (step S018).

In the above description, the case of performing SVM determination has been described as an example, but the determination method is not specified, and other determination means for performing mechanical pass / fail determination can be used.
Next, the operation of this embodiment will be described using the example of the LCD shown in FIG.

First, a pseudo sample of the LCD screen is created and the image is read (see step S011 in FIG. 7).
Next, the image processing described with reference to FIG. 4 is performed to extract feature amounts. In this case, a spot on the LCD screen is detected. The maximum brightness value and area of the spot are extracted as features (see step S012 in FIG. 7). Next, as shown in FIG. 8, the pseudo sample created on the feature amount space is mapped (see step S013 in FIG. 7). FIG. 8 is a diagram illustrating feature amount space mapping in Embodiment 1 of the present invention.

  Next, the quality of the pseudo sample to which a skilled inspector, a person in charge of quality assurance, etc. are mapped is determined (see step S014 in FIG. 7). FIG. 9 is a diagram illustrating the result of the person determining the pseudo sample on the feature amount space in the first embodiment of the present invention, and shows the result of the person determining the quality. Next, the same pseudo sample mapped is determined by SVM analysis (see step S015 in FIG. 7). As shown in the diagram illustrating the determination result of SVM in the feature amount space in the first embodiment of the present invention in FIG. Next, the pass / fail determination result of the person and the pass / fail determination result of the SVM are compared, and an erroneous determination sample is output (see step S016 in FIG. 7). Next, as shown in FIG. 11, the point P of the re-determined missed pseudo sample and the barycentric point O of the determination space group of the defective product (good or bad content opposite to the re-determined point P) are connected with a straight line, The intersection of the straight line and the boundary line of the SVM analysis pass / fail judgment space is Q, the length of the line segment OQ is L1, and the length of the line segment QP is L2, how much the misjudgment point P is from the pass / fail boundary. When the distance is quantified, the quantified distance variable X is expressed by the following formula (formula 11).

ここで、重み付けする点数Ｙは、次式（式１２）となる。

Here, the weighted score Y is expressed by the following formula (formula 12).

ここで、Ｍは重み係数で、任意の定数とする。つまり、誤判定された擬似サンプルの特徴量に重み付けを付加することにより、再判定した結果の内容を同じ特徴量空間上にＹ点の数分追加して、擬似サンプルを特徴量空間上で増加させる。すなわち、再判定された擬似サンプルが１つ存在し、この擬似サンプルに重み付けする点数Ｙが３となった場合、合計で４つの擬似サンプルが生成されることになる。そのため、重み付けを行うことにより、特徴量空間上での擬似サンプルが３つ増加することになる。

Here, M is a weighting factor, and is an arbitrary constant. In other words, erroneous by the addition of only weighted the feature value of the determined pseudo sample, add a few minutes of Y point the contents of the re-determination result to the same feature quantity space, feature space on the pseudo sample Increase with. That is, when there is one re-determined pseudo sample and the number Y of points for weighting this pseudo sample is 3, a total of four pseudo samples are generated. Therefore, by performing weighting, three pseudo samples on the feature amount space are increased.

  As described above, even if a pseudo sample is mechanically added, the determination of the SVM can be performed correctly, and the boundary line of good / bad can be calculated without newly adding a pseudo sample for intentional learning by human hands. Yes (see step S017 in FIG. 7). In the case of FIG. 11 (when there is a misjudgment of an oversight), the SVM analysis is performed again, and repeated until the person and SVM judgments completely match. If there is no misjudgment, the judgment of the person is reflected there. A pass / fail space with a boundary is completed. Then, using the feature amount space, pattern recognition of an actual sample is performed, and a pass / fail judgment inspection is performed.

  In this way, by using the initial samples distributed at the Mahalanobis distance from the origin of the feature amount space, learning the pass / fail judgment boundary line that becomes the pass / fail judgment criterion, it is possible to learn using only the initial sample near the boundary. Therefore, it is possible to determine an optimal quality determination criterion with a small number of learning samples.

  In addition, when learning using a pseudo sample, by adding a weight to a false sample that has been erroneously determined, erroneous determination may be performed in a mechanical determination step without newly adding a pseudo sample by a human hand. Since the determination of the pseudo sample that has been performed can be performed again, it is possible to determine an optimal quality determination criterion with a small number of learning samples.

In the above description, the case where both of the above processes are performed has been described as an example. However, even if only one of the above processes is performed, it is possible to determine an optimal quality determination criterion with a small number of learning samples.
(Embodiment 2)
Next, features different from the first embodiment of the sensory test apparatus and sensory test method according to the second embodiment will be described with reference to FIG.

FIG. 12 is a diagram showing a feature amount space according to the second embodiment of the present invention. 12, the same components as those in FIG. 5 are denoted by the same reference numerals, and the description thereof is omitted.
FIG. 12A shows the feature amount of the initial sample. In selecting the initial sample, in the first embodiment, the initial sample whose distance from the origin is a is selected and the direction is random, but in the second embodiment of the present invention, the distance from the origin is In a, an initial sample whose intervals are equal to each other is selected.

FIG. 12B shows the determination result of the initial sample.
FIG. 12C is a diagram illustrating a method for generating a feature value by the pseudo sample feature value generation unit 10. Here, in the second embodiment, a new pseudo sample is generated along a straight line from the origin. If the determination result is a non-defective product, a new pseudo sample is created at a position far from the origin. Generate a sample.

  The completion condition is repeated until the number of pseudo samples is exceeded in the first embodiment, but the second embodiment is characterized by repeating until a non-defective product near the boundary and a defective product near the boundary are detected. The distribution of the pseudo samples at this time is illustrated in the figure showing the quality results after the pseudo samples increase in the feature amount space of FIG.

Since the processes other than those described here are the same as those in the first embodiment, description thereof will be omitted.
In the sensory inspection apparatus and method according to Embodiment 2 of the present invention, a random sample is not used, and a pseudo sample is generated until a pass / fail result near the boundary is actually detected instead of the number. It is possible to determine an optimum pass / fail judgment criterion with a small number of learning samples.

  INDUSTRIAL APPLICABILITY The present invention can accurately determine pass / fail judgment criteria with a small number of learning samples, and is useful for sensory inspection apparatuses and sensory inspection methods that automatically inspect visual inspection of displays and abnormal sound inspection of various machines. It is.

DESCRIPTION OF SYMBOLS 1 Good product data input part 2 Feature quantity extraction part 3 Feature quantity space creation part 4 Feature quantity space storage part 5 Initial sample feature quantity generation part 6 Pseudo sample creation part 7 Random number generation part 8 Pseudo sample display part 9 Judgment input part 10 Pseudo sample Feature amount generation unit 11 Detection unit 12 Feature amount extraction unit 13 Feature amount space creation unit 14 Determination input unit 15 Feature amount space storage unit 16 SVM determination unit 17 Determination comparison unit 18 Re-determination input unit 19 Weighting processing unit 20 Pass / fail space output unit DESCRIPTION OF SYMBOLS 21 Good product image input part 22 Average image creation part 23 Feature-value input part 24 Feature-value conversion part 25 Random-number input part 26 Parameter determination part 27 Image composition part 101 Neural network 102 Good-quality image input part 104 Defective image extraction part 105 Pseudo-data condition setting Unit 106 Pseudo-defective image creation unit 107 Pre-processing unit 108 Difference data 109 Random Generating unit

Claims (14)

A pseudo sample creation unit for creating a pseudo sample;
A determination input unit for inputting a pass / fail result determined by a person with respect to the pseudo sample;
A pseudo sample feature value generation unit that generates a feature value in the vicinity of a boundary between a non-defective product and a defective product from the pass / fail result input by the determination input unit, and sets the feature value of the pseudo sample as a feature value;
An SVM determination unit that creates a boundary line between a non-defective product and a defective product from the feature amount of the pseudo sample created by the pseudo sample creation unit, and performs pass / fail judgment by SVM analysis;
A determination comparison unit that compares the pass / fail result in the determination input unit with the pass / fail result in the SVM determination unit;
A re-determination input unit that inputs a pass / fail result that a person re-determines for a pseudo sample having a different pass / fail result in the determination comparison unit;
A weighting processing unit for adding a sample by adding a weight according to the distance from the boundary line created by the SVM determination unit to the pass / fail result input by the redetermination input unit;
A sensory test apparatus, comprising: a sensory test unit that adds a sample added by the weighting processing unit and performs pass / fail determination. A non-defective product data input unit for inputting non-defective product data.
A feature amount space creating unit that converts the feature amount extracted from the non-defective product data into a feature amount converted so that the distance from the origin of the feature amount space becomes the Mahalanobis distance;
The sensory test apparatus according to claim 1, further comprising: an initial sample generating unit that generates an initial sample such that a distance from the origin of the feature amount space has a constant value, and generates a pseudo sample from the initial sample. . The sensory test apparatus according to claim 2, wherein the initial sample has a distance from the origin of the feature amount space within a certain value ± 10%. The sensory test apparatus according to claim 3, wherein the initial sample has a distance from the origin of the feature space between 2 and 5. 5. The quality determination result input in the determination input unit is divided into four stages: a non-defective product, a non-defective product near the boundary line, a defective product near the boundary line, and a defective product. Sensory inspection device according to crab. The pseudo sample is a method of reducing the Mahalanobis distance of the feature quantity of the non-defective sample, a method of increasing the Mahalanobis distance of the feature quantity of the non-defective sample, based on the result of the sample judged to be good or bad by the judgment input unit, near the boundary line 6. The method according to claim 5, wherein the non-defective sample or the defective sample near the boundary line is synthesized by a method of synthesizing the distance and direction from the origin of the feature amount of two samples at the same ratio. Sensory test equipment. The ratio of combining the distance and the direction from the origin of the feature amount space of the feature amount of two samples of the non-defective sample near the boundary line or the defective sample near the boundary line is 1/2 each. Item 6. The sensory test apparatus according to Item 6. A pseudo sample creation process for creating a pseudo sample;
A determination input step of inputting a pass / fail result when a person determines the pseudo sample;
A pseudo sample feature value generating step for generating a feature value in the vicinity of a boundary between a non-defective product and a defective product from the pass / fail result input in the determination input step to be a feature value of the pseudo sample;
An SVM determination step for determining pass / fail by SVM analysis for creating a boundary line between a non-defective product and a defective product from the feature amount of the pseudo sample;
A determination comparison step for comparing the results of the determination input step and the SVM determination step;
A re-judgment input step for inputting a pass / fail result re-determined by a person for a pseudo sample having a different pass / fail judgment in the judgment comparison step;
A weighting process step of adding a pseudo sample by adding a weight according to the distance from the boundary line created in the SVM determination step to the result input in the redetermination input step;
A sensory test method comprising: a sensory test step of performing pass / fail judgment by adding the sample added in the weighting process step . A non-defective product data input process for inputting non-defective product data.
A feature amount space creating step for converting the feature amount extracted from the non-defective product data into a feature amount converted so that the distance from the origin of the feature amount space becomes the Mahalanobis distance;
9. The sensory inspection method according to claim 8 , further comprising an initial sample generation step of generating an initial sample such that a distance from the origin of the feature amount space has a constant value, and generating a pseudo sample from the initial sample. . The sensory inspection method according to claim 9 , wherein the initial sample has a distance from the origin of the feature amount space within a certain value ± 10%. The initial sample, the functional inspection method according to claim 1 0, wherein the distance from the origin of the feature space is between 2-5. The decision input step quality results input by the non-defective, the vicinity of the boundary line good, defective near the boundary line, of claim 8 according to claim 1 1, characterized in that is divided into four stages of the defective The sensory test method according to any one of the above. The pseudo sample is a method of reducing the Mahalanobis distance of the feature amount of the non-defective sample, a method of increasing the Mahalanobis distance of the feature amount of the defective sample based on the result of the sample determined in the determination input step, methods for synthesizing the distance and direction from the feature amounts of the origin of the two samples in the same proportion of defective samples in the vicinity of the non-defective or boundary, of claim 1 2, wherein the created either Sensory inspection method. According to claim 1 3, wherein the ratio of synthesizing the distance and direction from the feature amounts of the origin of the two samples of the defective samples in the vicinity of the non-defective or border in the vicinity of the boundary line is equal to or is halves Sensory inspection method.

Images