JP2007114843A - Quality deciding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a quality deciding device for easily updating the criterion of the an object to be inspected. <P>SOLUTION: This quality deciding device 1 is provided with an operation means 11 equipped with an image display part 21 for displaying an image obtained by photographing an object to be inspected and a quality decision result input part 23 for inputting the decision result of whether the object is non-defective or defective for the image displayed on the image display part 21; and a storage means 12 for storing the image and quality decision result in association with each other; and a control means 13 equipped with an image recognizing part 41 for calculating at least one featured value based on the image, a quality criterion generating part 42 for generating a quality criterion for deciding whether the object is non-defective or defective based on calculated featured values: and quality deciding part 43 for deciding whether or not the object is non-defective based on the generated quality criterion. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an apparatus for determining the quality of an inspection object, and more specifically, a quality determination criterion is automatically generated from an image obtained by photographing the inspection object, and the quality determination is performed using the quality determination criterion. Relates to the device to perform.

  In the production lines of various products, an inspection process for removing defective products including breakage and assembly defects from parts and finished products is provided in order to maintain and improve product quality. In the inspection process, for example, an inspector visually confirms an inspection object such as a part or a finished product that flows on the production line, or measures a shape or size of a specific part of the inspection object, an electrical characteristic, etc. Based on the measurement values obtained by performing various measurements, whether the inspection object is a good product or a defective product is determined. Further, in such an inspection process, an inspection person may visually check the object to be inspected by enlarging the captured image and determine the quality of the object to be inspected.

  However, it is very labor-intensive to visually inspect the images of the inspection objects one by one, especially when the number of inspection objects is very large. It was difficult to do. Therefore, in such inspections that require visual confirmation, there have been cases where the purpose of the inspection for maintaining and improving the quality cannot be sufficiently achieved. Therefore, the quality of the image is improved by analyzing the image and automatically determining whether it is a non-defective product by analyzing the image of the inspection object. At that time, for example, when a dimension or numerical standard after processing of a certain part is determined, the criterion for the quality determination is a numerical standard such as a dimension defined in such a design drawing. However, when visual inspection is replaced with image processing, it is not possible to simply quantify the pass / fail judgment criteria for inspections performed by sensory evaluation of visual workers or inspections without drawing standards (for example, the absence of foreign matter). There is. When performing pass / fail judgment using such image processing for such an inspection, the feature quantity to be noticed is detected from the non-defective sample image and the defective sample image in order to replace the pass / fail judgment criteria with numerical values, and the feature quantity It was necessary to determine the optimum threshold value for.

  On the other hand, the feature of the inspection object is calculated from the image data obtained by photographing the inspection object, the frequency distribution of the feature amount is calculated for each non-defective product and defective product, and the non-defective product and the defective product are calculated based on the frequency distribution. A method of automatically calculating a threshold value is known (see Patent Document 1).

  When the method described in Patent Document 1 is used, a feature amount is calculated based on an image obtained by photographing an inspection object, and the feature amount is compared with a predetermined threshold value to determine whether the product is a good product or a defective product. Therefore, it is possible to automatically determine the quality of an inspection object without the need for human inspection.

  However, for example, when a trial production of an inspection process is performed, there are cases in which the quality determination criteria have to be changed due to a change in the image taken of the inspection object. In such a case, it is desirable that the quality criteria can be easily updated. Furthermore, it is desirable that the validity of the calculated pass / fail judgment criterion can be easily verified.

JP-A-8-7104

  In view of the above problems, an object of the present invention is to provide a quality determination device that can easily update a quality determination criterion for an inspection object.

  Another object of the present invention is to provide a pass / fail judgment device that can easily verify the calculated pass / fail judgment criteria.

  Still another object of the present invention is to provide a quality determination device that can automatically perform quality determination of an inspection object with high accuracy.

  According to the first aspect of the present invention, the pass / fail determination apparatus according to the present invention is configured to display an image obtained by photographing an inspection object and an image displayed on the image display unit. The operation means having a pass / fail judgment result input unit for inputting a judgment result of whether the inspection object is a non-defective product or a defective product and a storage means for storing the image and the pass / fail judgment result in association with each other are provided. It is possible to easily add and modify the sample image used for generation, and it is possible to easily generate and update the pass / fail judgment criteria.

  According to a second aspect of the present invention, the storage means stores a first storage area for storing an image when the inspection object on the image is a non-defective product, and an image when the inspection object on the image is a defective product. By having the second storage area for storing the sample image, it is possible to associate the sample image with the pass / fail judgment result without having to separately create a file indicating the correspondence between the sample image and the pass / fail judgment result. Therefore, when using an image whose pass / fail judgment result is known in advance as a sample image, simply adding the image directly to each storage area, the added image and the inspection object photographed in the image The pass / fail judgment result can be associated.

  If the storage means has the first and second storage areas as described above, the pass / fail judgment result input section stores the storage location of the image displayed on the image display section. May be designated as the first storage area or the second storage area.

  According to the fourth aspect of the present invention, the pass / fail judgment result input unit has a first operation button for associating a judgment result with a non-defective product with an image displayed on the image display unit, and a judgment result with a defective product. By having the second operation button for associating, the sample image and the pass / fail judgment result can be associated with a simple operation.

  In addition to mechanically configured buttons, operation buttons are those that execute a predetermined function by an operation (mouse click, touch to touch panel, etc.) that specifies a specific area displayed on a monitor with a pointing device. Including.

  In addition, as described in claim 5, the operation means may further include a determination criterion generation instruction unit that starts generation of the pass / fail determination criterion. The generation of the pass / fail judgment criteria can be started at an arbitrary time, and the pass / fail judgment criteria can be easily generated and updated.

  Further, as described in claim 6, the operation unit includes a verification instruction unit that causes the pass / fail determination unit to perform pass / fail determination of the image stored in the storage unit, and the pass / fail determination result determined by the pass / fail determination unit. By further including a determination result display unit that displays “”, it is possible to easily verify the validity of the quality determination criteria once generated.

Hereinafter, the quality determination device according to the present invention will be described in detail with reference to the drawings.
The pass / fail determination apparatus 1 according to the present invention automatically generates pass / fail judgment criteria based on the sample image of the inspection object that is the target of pass / fail judgment and the result of judging the pass / fail by visually observing the sample image, It is a device that automatically determines the pass / fail of an inspection object using the pass / fail judgment criteria, and can easily add / delete sample images used to generate the pass / fail judgment criteria and update the pass / fail judgment criteria. Device.

FIG. 1 shows a configuration block diagram of a quality determination device 1 according to the present invention.
The pass / fail determination apparatus 1 according to the present invention includes an operation unit 11 that inputs a pass / fail determination result while observing a sample image of an inspection object, a storage unit 12 that stores the sample image in association with the pass / fail determination result, and a sample image. One or more feature amounts are extracted from the image, a pass / fail judgment criterion is generated based on the feature amount and the pass / fail judgment result associated with each sample image, and the image is taken using the generated pass / fail judgment criteria. A control means 13 for judging the quality of the inspection object and an imaging means 14 for photographing the inspection object are provided.

Hereinafter, each part will be described in detail.
In FIG. 2, the schematic block diagram of the operation means 11 is shown.
The operation means 11 includes a monitor such as a liquid crystal display and a pointing device such as a mouse and a touch panel. As shown in FIG. 2, the operation means 11 includes an image selection unit 21 that selects a sample image of an inspection object, an image display unit 22 that displays the selected sample image, and an inspection object that is reflected in the sample image. A pass / fail judgment result input unit 23 for inputting a judgment result as to whether the product is a non-defective product or a defective product, a judgment reference generation instruction unit 24 for instructing generation start of a pass / fail judgment criterion, and storage means using the generated pass / fail judgment criterion 12, a verification instruction unit 25 that performs a pass / fail determination on the sample image stored in the image 12, a determination result display unit 26 that displays a result of the pass / fail determination, and a criterion for determining whether the sample image is a non-defective product. And a determination reference adjusting unit 27 for adjusting.

The image selection unit 21 selects a sample image to be used for generation of a pass / fail judgment criterion from an image server connected to the storage unit 12 or the pass / fail judgment device 1 (to be described later) or another pass / fail judgment device. As shown in FIG. 2, for example, a list in which a desired image can be selected by an image name and an add button for confirming the selection or a cancel button for canceling the selection are configured.
The configuration of the image selection unit 21 is not limited to the above, and a known file selection technique can be used. Alternatively, a storage location (directory, folder, etc.) of the image to be selected may be set in advance, and images in the storage location may be automatically selected in order of name.

  The image display unit 22 displays the image selected by the image selection unit 21. Further, the image display unit 22 may have an image enlargement / reduction function in order to facilitate observation of the inspection object.

  The pass / fail judgment result input unit 23 inputs the result of the operator's judgment on whether the inspection object shown in the sample image is a non-defective product or a defective product by visual observation. Is stored in the storage means 12. When the operator determines that the object to be inspected is a non-defective product, the operator presses the OK button 231 (“press” is selected by clicking the mouse in the button display area when the pointing device is a mouse. In the case of a touch panel, this represents a selection operation of a specific area such as touching a button display area). When the OK button 231 is pressed, the displayed sample image is moved to a non-defective image storage location in the storage unit 12 to be described later. On the other hand, when the operator determines that the inspection object is defective, the operator presses the NG button 232. When the NG button 232 is pressed, the displayed sample image is moved to the storage location for the defective product image. When the Delete button 233 is pressed, the sample image already associated with the pass / fail judgment result is deleted from the storage location for the non-defective product image or the storage location for the defective product image, and is no longer used for generating the pass / fail judgment criteria.

As described above, since the storage locations of the sample images are divided into different locations for the non-defective product and the defective product, the sample image can be stored in the storage unit 12 in association with the determination result.
The pass / fail judgment result input unit 23 is not limited to the above-described one. For example, a sample image can be stored in a non-defective product storage location or stored by a file move (or copy) operation between directories or folders using a drag / drop operation with a mouse. You may make it move to the storage place of a good article.

  The determination reference generation instruction unit 24 is configured as an operation button as shown in FIG. 2. When the operation button is pressed, each sample image stored in the control unit 13 in association with the pass / fail determination result is stored in the storage unit 12. The calculation of the feature amount is started. And the control means 13 produces | generates the quality determination criteria based on those feature-values.

  As shown in FIG. 2, the verification instruction unit 25 is configured as an operation button. By pressing this operation button, the control unit 13 calculates a feature amount for each sample image stored in the storage unit 12. And pass / fail is determined using the feature amount and the generated pass / fail criterion. The operation means 11 displays each sample image on which the pass / fail determination has been performed on the image display unit 22 in the order of the pass / fail determination, and simultaneously displays the pass / fail determination result for the sample image on the determination result display unit 26.

  The determination result display unit 26 displays the result of automatically determining pass / fail using the generated pass / fail criterion for the sample image displayed on the image display unit 22 and the feature amount detected by the control means 13. To do. Then, it can be confirmed whether or not the generated pass / fail criterion is appropriate.

  The determination criterion adjustment unit 27 enables adjustment so that the criterion for determining whether or not the product is non-defective is strict or gradual with respect to the quality determination criterion generated by the control unit 13. As an example, as illustrated in FIG. 2, the determination reference adjustment unit 27 changes the degree of adjustment with a slide bar. As will be described later, the degree of adjustment added here is taken into consideration in the generation of the pass / fail criterion. By making such adjustments, it is possible to optimize the pass / fail judgment criteria according to the risk of misjudgment, such as reducing the probability of erroneously judging a defective product as a non-defective product.

Next, the storage unit 12 will be described.
The storage unit 12 is composed of a nonvolatile semiconductor memory, a hard disk, or an optical recording medium such as a DVD or CD, and stores the sample image in association with the pass / fail judgment result as described above. In order to associate the sample image with the pass / fail judgment result, the sample image when the inspection target is a non-defective product and the sample image when the inspection target is a defective product are stored in different storage locations. Specifically, a non-defective product directory (or folder) 31 and a defective product directory (or folder) 32 are provided in the storage unit 12, and sample images corresponding to good products are stored in the good product directory 31. 32 stores a sample image corresponding to the defective product. Alternatively, instead of providing a directory, the storage unit 12 may be divided into two partitions.

  Thus, by separating the sample image corresponding to the non-defective product from the sample image corresponding to the non-defective product, the sample image can be easily associated with the pass / fail judgment result. In addition, when a sample image whose quality is known in advance is added, it is only necessary to directly place the sample image in each storage location, and the labor of inputting the quality determination result while referring to the image by the operation means 11 is saved. be able to.

Next, the control means 13 will be described.
As shown in FIG. 1, the control means 13 uses an image recognition unit 41 that extracts one or more feature amounts from a sample image and uses the obtained feature amounts and pass / fail judgment results for use in setting pass / fail judgment criteria. A pass / fail judgment reference generating unit 42 for generating pass / fail judgment criteria, and a pass / fail judgment means 43 for judging pass / fail of the inspection object using the obtained pass / fail judgment criteria, a central processing unit (CPU) of a personal computer, It consists of a microprocessor such as a numerical operation processor, a semiconductor memory such as a ROM and a RAM, and a program for operating them.

  When the determination criterion generation instruction unit 24 of the operation unit 11 instructs the image recognition unit 41 to generate a determination criterion, the image recognition unit 41 reads each sample image stored in association with the pass / fail determination result from the storage unit 12, and A specific area used for determining whether the object is a non-defective product or a defective product is recognized, and various measured values obtained from the specific area are calculated as feature amounts. The calculated feature values are stored in the storage unit 12 in groups of good and defective products.

  A known image recognition technique can be used to calculate the feature amount. For example, if the area of interest of the inspection object is different in brightness or color tone from the surrounding area, binarization processing is performed to extract the area of interest, and the number of pixels included in the area is defined as the area. It can be an amount. Alternatively, the center of gravity, maximum length, aspect ratio, and the like of the region extracted by the binarization process may be used as the feature amount. Furthermore, in order to investigate whether dust or the like is attached to a specific part of the inspection object, an edge detection process or the like along the outer shape of the inspection object is performed, and the inspection object occupied in the sample image The region may be specified, and an average pixel value of a portion corresponding to a partial region in the region occupied by the inspection target may be used as the feature amount.

  Further, for sample images taken under various environments (for example, under different lighting conditions), in order to eliminate the influence of the environmental conditions, pre-processing may be performed before calculating the feature amount. As such pre-processing, for example, processing for adding or subtracting a certain pixel value to all pixels included in the sample image so that the average density of each sample image matches, or for noise removal Filter processing using a Gaussian filter or the like is included.

  The calculated feature values are aggregated for each non-defective product and defective product, and are stored in the storage unit 12 as a table for associating, for example, a sample image name with a feature value calculated from the sample image.

Next, the quality determination criterion generation unit 42 will be described.
The pass / fail judgment standard generating unit 42 sets an identification boundary for judging whether the product is a non-defective product or a defective product based on a set of feature amounts calculated by the image recognition unit 41 from an image in which an inspection object is captured. As an example of such an identification boundary, when there are n types of feature amounts, an average value vector Uav = (u1, u2,..., Un) of feature amounts obtained from sample images corresponding to non-defective products, Mean value vectors Vav = (v1, v2,..., Vn) of the feature values obtained from the corresponding sample images are obtained, orthogonal to the straight line connecting these average value vectors Uav and Vav, and each of Uav and Vav, etc. The surface s1 located at the distance can be used as an identification boundary between a good product and a defective product. In this case, when a feature quantity vector that is a set of feature quantities obtained for an arbitrary image is present on the average value vector Uav side with respect to the surface s1, the inspection object shown in the image is determined to be a non-defective product. Is done.

  In addition, as another example of the identification boundary that becomes the pass / fail judgment criterion, when a normal distribution is assumed for any feature amount, if the variance is different between the case of the non-defective product and the case of the defective product, It is preferable that the surface s2 represented by the discriminant function having the Mahalanobis distance from the above average value vectors Uav and Vav to be an equal distance is used as an identification boundary. This is because the quality distribution can be taken into account because the distribution of the feature amount is taken into account.

  The calculated identification boundary is represented by a discriminant function using, for example, the feature amount calculated by the image recognition unit 41 as an input variable. When the output value obtained by inputting each feature amount calculated by the image recognition unit 41 to the discriminant function is larger than a determination value Tj (for example, Tj = 0), the inspection object is determined to be non-defective, and conversely If it is equal to or less than the determination value Tj, it is determined as a defective product.

The determination value Tj is determined in consideration of the degree of adjustment input in the determination reference adjustment unit 27 of the operation unit 11 described above. For example, when the operator does not change anything in the generated pass / fail judgment criteria, the judgment value Tj is a predetermined value D. In addition, when the criterion for determining that the product is non-defective is strictly set in the determination criterion adjustment unit 27, a value corresponding to the degree of adjustment is added to the predetermined value D. On the other hand, in the determination reference adjustment unit 27, when the reference for determining non-defective product is set gently, the value corresponding to the degree of adjustment is subtracted from the predetermined value D. As is clear from the above, as the determination value Tj increases or decreases, the criterion for determining a non-defective product is changed.
Note that each coefficient and determination value Tj of the obtained discriminant function are stored in the storage unit 12.

  As another example, the pass / fail criterion generation unit 14 can determine a discrimination boundary between a non-defective product and a defective product by using a three-layer perceptron model neural network or nonparametric learning such as a support vector machine. . In this case, a set of feature amounts (area, center of gravity, aspect ratio, etc. of the specific region) calculated from a plurality of sample images is input, and a pass / fail judgment result for each set of feature amounts (for example, “1” for a non-defective product). In the case of a defective product, learning is performed using the value “−1” as the pass / fail judgment result) as an output. Then, learning is repeated until a correct output is obtained for each set of feature amounts. The system learned in this way is used as a pass / fail criterion.

  When such non-parametric learning is used, even when a good product and a defective product cannot be linearly separated based on the dimension of the number of feature quantities, an identification boundary that can perform a pass / fail judgment with high accuracy can be set.

Next, the quality determination unit 43 will be described.
The pass / fail determination means 43 is a determination that represents the feature amount calculated by the image recognition unit 41 with respect to the image obtained by capturing the inspection object acquired from the imaging means 14 and the identification boundary stored in the storage means 12. Based on the coefficient of the function, the output value of the discriminant function is calculated. When the output value is larger than the determination value Tj, the inspection object shown in the image is determined as a non-defective product. On the other hand, when the output value is equal to or less than the above-described determination value Tj, the inspection object shown in the image is determined as a defective product.

  The determination result is displayed on the determination result display unit 26 of the operation means 11. Further, the determination result may be transmitted from the control means 13 to an equipment having an inspection process in which the quality determination device 1 according to the present invention is used, and used for selection of a non-defective product and a defective product.

  The imaging means 14 is composed of a CCD camera or the like, and images the inspection object so that the difference between the non-defective product and the defective product can be clearly distinguished on the image. The photographed image is stored in the storage unit 12 or used as a sample image for generating a pass / fail judgment standard, or used for pass / fail judgment of an inspection object as part of a manufacturing process or an inspection process.

  Next, a procedure for generating a pass / fail criterion in the pass / fail determination apparatus 1 according to the present invention will be described with reference to a flowchart.

FIG. 3 shows a flowchart of a procedure for generating a pass / fail criterion.
First, for each sample image used for generating the pass / fail judgment criteria, the operating means 11 visually determines whether the inspection object photographed in the sample image is a non-defective product or a defective product, and associates the result with the sample image. It memorize | stores in the memory | storage means 12 (step S01).

  When the pass / fail judgment results are associated with all the sample images, the image recognition unit 41 calculates a feature amount from each sample image (step S02). The calculated feature quantities are aggregated after being distinguished by non-defective products and defective products, and stored in the storage unit 12.

  Then, the pass / fail judgment criterion generating unit 42 calculates, for each calculated feature amount, statistics such as the average value, variance, and covariance of each of the non-defective product and the defective product, and generates pass / fail judgment criteria as a discrimination function, or A pass / fail criterion is generated by learning a system such as a neural network (step S03). The generated discriminant function is stored in the storage unit 12 (step S04).

  As described above, the pass / fail determination apparatus 1 according to the present invention can easily add / update the sample image used for generation of the pass / fail determination criteria, and as a result, the usage status of the apparatus. Accordingly, it is possible to easily generate / update the pass / fail judgment criteria.

  In addition, embodiment mentioned above is for demonstrating this invention, and this invention is not limited to these embodiment.

  For example, in the above-described embodiment, the storage unit 12 associates the sample image and the pass / fail judgment result by separately storing the sample image corresponding to the non-defective product and the sample image corresponding to the defective product. A table describing the pass / fail judgment result in association with the sample image name may be created and stored.

  In the above-described embodiment, the image recognition unit 41 calculates the feature amount for each sample image after an instruction is given through the pass / fail criterion generation instruction unit 24 of the operation unit 11. At the time when the pass / fail judgment result of the sample image is input, the feature amount for the sample image may be calculated. With such a configuration, since the calculation of the feature amount has already been completed when the pass / fail judgment criterion is generated, the time required to generate the pass / fail criterion can be shortened.

  Further, when a sample image is selected by the image selection unit 21 in the operation unit 11 to make a pass / fail judgment, the image recognition unit 41 immediately calculates a feature amount for the selected sample image. Also good. Then, the calculated feature value may be displayed on the determination result display unit 26. By adopting such a configuration, it is possible to refer to the value of the feature amount when judging the quality of the inspection object photographed in the sample image by visual inspection.

  Also, the pass / fail judgment means 43 may be configured as hardware different from other means, and may be a pass / fail judgment apparatus independent of the pass / fail judgment criteria generation. With such a configuration, for example, when performing a pass / fail determination on a plurality of production lines, a pass / fail determination criterion is generated in advance based on the present invention, and the generated pass / fail determination criteria are passed to each manufacturing line. By installing and using the determination apparatus, it is possible to easily apply the same quality determination criteria to a plurality of production lines.

It is a block diagram of the configuration of the quality determination device according to the present invention. It is a schematic block diagram of an operation means. It is a flowchart which shows the operation | movement procedure at the time of quality determination criteria production | generation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Pass / fail judgment apparatus 11 Operation means 12 Storage means 13 Control means 14 Imaging means 21 Image selection part 22 Image display part 23 Pass / fail judgment result input part 24 Judgment reference generation instruction part 25 Verification instruction part 26 Judgment result display part 27 Judgment standard adjustment part 27 31 Directory for non-defective product 32 Directory for defective product 41 Image recognition unit 42 Pass / fail judgment reference generation unit 43 Pass / fail judgment unit

Claims (6)

An image display unit for displaying an image obtained by photographing an inspection object, and a quality determination result input unit for inputting a determination result as to whether the inspection object is a non-defective product or a defective product for the image displayed on the image display unit Operating means comprising:
Storage means for storing the image and the pass / fail judgment result in association with each other;
An image recognition unit that calculates at least one feature quantity based on the image; and a pass / fail judgment that generates a pass / fail judgment criterion for determining whether the inspection object is a non-defective product or a defective product based on the feature quantity A control unit having a reference generation unit and a pass / fail determination unit that determines whether the inspection object is a non-defective product based on the pass / fail determination criteria;
A pass / fail judgment device characterized by comprising:   The storage means stores a first storage area for storing the image when the inspection object on the image is a non-defective product, and a second storage area for storing the image when the inspection object on the image is a defective product. The quality determination apparatus according to claim 1, comprising:   The quality determination apparatus according to claim 2, wherein the quality determination result input unit specifies whether the storage location of the image displayed on the image display unit is the first storage area or the second storage area.   The pass / fail judgment result input unit includes a first operation button for associating a judgment result with a non-defective product with an image displayed on the image display unit, and a second operation button for associating a judgment result with a defective product. The quality determination apparatus as described in any one of Claims 1-3.   The pass / fail determination apparatus according to any one of claims 1 to 4, wherein the operation unit further includes a determination reference generation instruction unit that starts generation of a pass / fail determination reference.   The operation means includes a verification instruction unit that causes the quality determination means to determine the quality of the image stored in the storage means, and a determination result display that displays the quality determination result determined by the quality determination means. The quality determination apparatus according to claim 1, further comprising a unit.

Images