WO2016168823A1

WO2016168823A1 - Defect detection using a first derivative of an image

Info

Publication number: WO2016168823A1
Application number: PCT/US2016/028114
Authority: WO
Inventors: Mark Dickson; Nicholas TOKOTCH
Original assignee: Jabil Circuit, Inc.
Priority date: 2015-04-16
Filing date: 2016-04-18
Publication date: 2016-10-20

Abstract

The disclosure describes a polar transformed image to unwrap and flatten a region of interest of a plurality of similar objects under inspection. Gray scale values of the flattened images are analyzed and plotted on a curve in the direction of polar rotation, and smoothing may be applied. The first derivative of the gray scale value curve is plotted as a first derivative curve, that is, the slope of the tangent line along the curve of gray scale values. The slope of the tangent line represents a change in the appearance of the flattened image. A plurality of such curves for non-defective objects provides a range of expected values along the curves.

Description

DEFECT DETECTION USING A FIRST DERIVATIVE OF AN IMAGE

RELATED APPLICATIONS

[0001] The present disclosure claims priority to U.S. Provisional Application No.

62/148,524, entitled "DEFECT DETECTION USING A FIRST DERIVATIVE OF AN IMAGE," filed April 16, 2015, the contents of which is incorporated by reference in its entirety herein.

FIELD OF THE DISCLOSURE

[0002] The disclosure relates to detection of defects in a manufactured object, and, more particularly, relates to an apparatus, system and method for defect detection using a first derivative of an image.

BACKGROUND

[0003] Detecting anomalies using machine vision for the purpose of automated cosmetic inspection is well known, but still presents ongoing challenges. The detection of defects can be problematic, for example, where a plurality of objects under inspection contain a circular feature, or where the difference in appearance between normal and defective objects is subtle. Defects in a feature can cause such subtle changes to the appearance of its surface structure which can be difficult to detect. Prior art approaches, including feature matching, normalized correlation, or other methods have been used to detect defects, but often their detection capability suffers because the inherent range of normal variations of features can make it difficult to distinguish between a feature within the normal range of variability and a similar feature that contains a defect.

[0004] Therefore, the need exists for an apparatus, system and method that provide an improved detection of anomalies indicative of defects in an inspected object. SUMMARY OF THE DISCLOSURE

[0005] Detection of defects in objects being subjected to automated inspection is enhanced by examining the expected gray scale slope of a feature of a plurality of objects, given a controlled lighting and image capturing arrangement. In an exemplary embodiment, the controlled lighting and images create a light intensity signature of non-defective normal objects based on the appearance of features on the surface of the objects.

[0006] The disclosure describes a polar transformed image to unwrap and flatten a circular feature of a plurality of similar objects under inspection. Gray scale values of the flattened images are analyzed and plotted on a curve in the direction of polar rotation and smoothing may be applied. The first derivative of the gray scale value curve is plotted as a first derivative curve, that is, the slope of the tangent line along the curve of gray scale values. The slope of the tangent line represents a change in the appearance of the flattened image. A plurality of such curves for non-defective objects provides a range of expected values along the curves.

[0007] A defective feature of an object typically appears different than normal features; however the difference can be subtle. The first derivative curve has been determined to be sensitive in detecting defects, and robust with respect to differentiating between a defective feature and a normal feature within the expected range of variation.

[0008] Thus, the disclosure provides an apparatus, system and method that provide an improved detection of anomalies indicative of defects in an inspected object. It is to be understood that both the foregoing summary and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate disclosed embodiments and/or aspects and, together with the description, serve to explain the principles of the invention.

[0010] In the drawings:

[0011] FIGs. 1A and IB show images of two objects, one of which contains a defect;

[0012] FIGs. 2A and 2B show a circular region of interest extracted from each image of FIGs. 1A and IB, respectively;

[0013] FIGs. 3A and 3B illustrate polar transformation of the circular regions of FIOGs. 2A and 2B, respectively, to form flattened images;

[0014] FIGs. 4A and 4B illustrate determining an average grayscale value along the flattened image, graphing such grayscale values for a plurality of objects (including the defective one), and graphing the derivative (slope) of the average values;

[0015] FIG. 5 shows the curves of the derivative values of the curves of Figs. 4A and 4B;

[0016] FIG. 6 shows an average of the curves of a plurality of derivatives for non-defective objects;

[0017] FIG. 7 shows a curve of the standard deviation of the first derivative curves from the average;

[0018] FIG. 8 shows an envelope of an expected normal range of non-defective objects' derivative curves as the average curve +/- the standard deviation; and

[0019] FIG. 9 shows the derivative curve of an object extending beyond the defect-free envelope for expected values, effectively identifying a defect in the object. DETAILED DESCRIPTION

[0020] It is to be understood that the figures and descriptions provided herein may have been simplified to illustrate aspects that are relevant for a clear understanding of the herein described processes, machines, manufactures, and/or compositions of matter, while eliminating, for the purpose of clarity, other aspects that may be found in typical devices, systems, and methods. Those of ordinary skill in the pertinent art may recognize that other elements and/or steps may be desirable and/or necessary to implement the devices, systems, and methods described herein. Because such elements and steps are well known in the art, and because they do not facilitate a better understanding of the present disclosure, a discussion of such elements and steps may not be provided herein. However, the present disclosure is deemed to inherently include all such elements, variations, and modifications to the described aspects that would be known to those of ordinary skill in the pertinent art.

[0021] Exemplary embodiments are provided throughout so that this disclosure is sufficiently thorough to fully convey the scope of the disclosed embodiments to those who are skilled in the art. Numerous specific details are set forth, such as examples of specific components, devices, and methods, to provide this thorough understanding of embodiments of the present disclosure. Nevertheless, it will be apparent to those skilled in the art that specific disclosed details need not be employed, and that exemplary embodiments may be embodied in different forms. As such, the exemplary embodiments should not be construed to limit the scope of the disclosure. In some exemplary embodiments, well-known processes, well-known device structures, and well-known technologies may not be described in detail.

[0022] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" may be intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises," "comprising," "including," and "having," are inclusive and therefore specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The steps, processes, and operations described herein are not to be construed as necessarily requiring their respective performance in the particular order discussed or illustrated, unless specifically identified as a preferred order of performance. It is also to be understood that additional or alternative steps may be employed.

[0023] When an element or layer is referred to as being "on", "engaged to", "connected to" or "coupled to" another element or layer, it may be directly on, engaged, connected or coupled to the other element or layer, or intervening elements or layers may be present. In contrast, when an element is referred to as being "directly on," "directly engaged to", "directly connected to" or "directly coupled to" another element or layer, there may be no intervening elements or layers present. Other words used to describe the relationship between elements should be interpreted in a like fashion (e.g., "between" versus "directly between," "adjacent" versus "directly adjacent," etc.). As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

[0024] Although the terms first, second, third, etc. may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms may be only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Terms such as "first," "second," and other numerical terms when used herein do not imply a sequence or order unless clearly indicated by the context. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the exemplary embodiments. [0025] The disclosure describes using a polar transformed image to unwrap a region or feature of interest, such as a circular feature. With a flattened image, the gray scale values may be linearly analyzed along the direction of polar rotation. The mean gray scale values of the transformed image may be calculated in a perpendicular direction with respect to polar rotation. The mean gray scale values may be optionally smoothed. To improve sensitivity and minimize measurement bias that can be introduced by lighting and other factors, the local gray scale slope may be used as a quality metric.

[0026] The first derivative of the gray scale values may be calculated along the direction of polar rotation to obtain the slope of the tangent line relative to each of the optionally smoothed gray scale values in the direction perpendicular to polar rotation. The series of tangent line slopes that are created represents the change in gray scale intensity with respect to the polar transformation. This series of values can be used to compare intensity change with respect to polar rotation. The intensity pattern created by the lighting strategy may allow for a distinct series of gray scale values that correlate with the defect of interest to be obtained. Expected first derivative series values may be determined by averaging a set of features without defects.

[0027] In addition, the standard deviation of the first derivative series values may also be calculated from the same set of features without defects. Comparison ranges may be made by calculating the product of a number used as a tolerance multiplier (sigma value) and each of the standard deviation series values, and by then adding and subtracting each of those values from the corresponding first derivative series values to form two series of values that form an envelope representing the expected range of features without defects. The length of the calculated series is related to the granularity of the gray scale values created by the polar transformation and is also linked to sensitivity and any filtering applied to the calculated series.

[0028] Selecting the proper feature regions for polar transformation may also be an aspect of the disclosed embodiments. The selected feature polar region should contain pixel gray scale values that are expected to change based on the defect of interest, given the inspection conditions. Therefore, proper feature referencing and segmentation may be of significant importance.

[0029] FIGs. 1A and IB are exemplary illustrations of two features, and more specifically two circular features in the form of two RCA connectors, that are subject to an automated inspection to produce the machine views shown. The provided illustrative views may comprise grey-scale conversions of color images taken as each respective machine view, or the machine images may be obtained initially as grey-scale.

[0030] In the illustration, FIG. 1A shows an RCA connector that does not include significant defects. Particular attention is directed to the lower left quadrant of the object in FIG. 1A, labelled as portion 110A. FIG. IB illustrates an RCA connector that has a defect, which appears as a variation in the lower left quadrant of the RCA connector in the illustration of FIG. IB. The defect is shown as portion HOB.

[0031] FIGs. 2A and 2B illustrate the extraction of a region of interest from each respective machine view. In the illustration, it is a particular ring of the circular element of each RCA connector that is of most significant interest with regard to likely production of defects. Those skilled in the pertinent arts will appreciate, in light of the disclosure herein, that the regions of interest 210A, 210B respectively shown in FIGs. 2A and 2B are provided by way of example only, and particular regions of interest will likely vary based on the product under manufacture.

[0032] The regions of interest 210A, 210B of FIGs. 2A and 2B may then be subjected to a polar transformation to unwrap the region of interest. With a flattened image, the gray scale values may be analyzed, such as linearly analyzed, by indexing the image along the direction of polar rotation.

[0033] FIG. 3A illustrates the unwrapping of the circular region of interest 210A shown in

FIG. 2A. By way of non- limiting example, the region of interest is unwrapped by way of a polar transformation, in which the region of interest is subjected to a polar rotation and flattening of the image. Image 210A shows an unwrapped circular region of interest, in which the defect free lower left quadrant of FIG. 2A is at image portion 11 OA. Of course, other unwrapping, image transformation, and/or image flattening techniques aside from or in addition to those illustrated may be employed.

[0034] FIG. 3B illustrates the unwrapping of the circular region of interest shown in FIG. 2B. By way of non-limiting example, the region of interest is unwrapped in substantially the same manner as was unwrapped the image of FIG. 3 A. Image 21 OB shows an unwrapped circular region of interest, in which the defect in the lower left quadrant of FIG. 2B is at image portion HOB. Again, other unwrapping, image transformation, and/or image flattening techniques may be employed.

[0035] The mean gray scale values of the transformed image may then be calculated, such as in a perpendicular direction with respect to polar rotation. The mean gray scale values may also be optionally smoothed. To improve sensitivity and minimize measurement bias that can be introduced by lighting and other factors, the local gray scale slope may be also be used as a quality metric.

[0036] More particularly, following the polar transformation processing of the image of the region of interest, mean values of the various regions of interest may be obtained, and the mean values curve may be optionally smoothed, as shown in FIGs. 4A and 4B. More particularly, mean value curves of image characteristics for defect-free portions, including portion 110A, of the manufactured product may be obtained, as shown in FIG. 4A. FIGs. 4A and 4B illustrate mean value grey scale curves for the transformed images of FIGs. 3A and 3B, respectively.

[0037] Of note, the quadrants of the curved region of interest flattened linearly along the linear indexing axis of FIGs. 4 A and 4B may be aligned with the quadrants of the flattened images of FIGs. 3A and 3B, as shown. Alignment of the flattened images shows that the defect-free flattened image of the region of interest of FIG. 3A aligns substantially with the light grey graph of the grey-scale means values (i.e., undamaged) curves at portion 110A of the illustration of FIGs. 4A and 4B, but the flattened image of FIG. 3B contributes to the dark grey graph of FIGs. 4A and 4B, which is indicative of a possible defect HOB in the lower left quadrant shown in FIG. 2B, i.e., portion HOB of the graph of FIG. 4B. Thus, the defect HOB may contributes to a graphically outlying curve, and this outlier outside of the damage free image range for the transformed images may be a graphical indication of a defect if proven to be a sufficiently substantial deviation from the defect-free means curves.

[0038] The first derivative of the gray scale means values may be calculated, such as along the direction of polar rotation, to obtain the slope of the tangent line relative to each of the optionally smoothed gray scale values in the direction perpendicular to polar rotation. The series of tangent line slopes that are created represent the change in gray scale intensity with respect to the polar transformation. This may provide a distinct series of gray scale values that correlate with a known or suspected defect of interest. Expected first derivative series values may be determined by averaging a set of features without defects.

[0039] FIG. 5 is a graphical illustration 400 of the taking of the first derivative of the means curves of FIGs. 4A and 4B. Again, the defect along the linear axis may be evident as the outlying portion HOB of the dark grey curve as compared to the acceptable ranges, such as defect-free curve portion 110A, indicated by the light grey curves. Further, the means curves of the first derivatives 410 may be calculated for all defect free regions of interest, as is illustrated in the example of FIG. 6.

[0040] Moreover, the standard deviations of the first derivatives may also be calculated for all defect free regions of interest, as is indicated by the non-limiting example of the curve 420 shown in FIG. 7. A comparison range envelope may be made by calculating the product of a number used as a tolerance multiplier (sigma value) with each of the standard deviation series values, and by then adding and subtracting each of those values from the corresponding first derivative series values to form two series of values that form an envelope representing the expected range of features without defects.

[0041] Calculating the expected envelope of defect free first derivative values along the grey means curve may be performed. As is illustrated in FIG. 8, the envelope 430 may be provided by calculating the means of the first derivative values, plus/minus the standard deviation of the first derivative values multiplied by the sigma multiplier.

[0042] FIG. 9 illustrates the overlay of the derivative curves of FIG. 5 on the defect-free envelope curve of FIG. 8. The exemplary illustration of FIG. 9 clearly indicates that the defect HOB in the lower left quadrant of FIG. 2B provides a first derivative curve that lies outside of the envelope of acceptable portions of the region of interest. That is, FIG. 9 clearly shows the defect 110B in the lower left quadrant of the product as it is shown in FIG. IB.

[0043] The disclosed embodiments and descriptions are provided to enable any person skilled in the art to make or use the invention, the scope of which is defined by the claims. Various modifications to the described subject matter will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to such variations without departing from the spirit or scope of the invention. Thus, the invention is not intended to be limited to the specific

embodiments described herein, but rather is deemed to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

CLAIMS What is claimed is:

1. A method of assessing defects in an image of a first one of a region of interest of a manufactured product, comprising: polar transforming the image to unwrap the first region of interest to form a flattened image; linearly analyzing the mean gray scale values of at least two images, including the flattened image, along the direction of polar rotation; taking the first derivative of the mean gray scale values; calculating the standard deviations of the first derivative mean gray scale values of ones of the region of interest without defects; forming an envelope representing an expected range of ones of the regions of interest without defects by creating two curves that each comprise, respectively, the first derivative mean gray scale values plus the standard deviations multiplied by a predetermined tolerance factor, and the first derivative mean gray scale values minus the standard deviations multiplied by the predetermined tolerance factor; and comparing the envelope with the first derivative curve of the flattened image to assess a presence of the defects in the manufactured product.

2. The method of claim 1, wherein the linearly analyzing comprises curve smoothing.

3. The method of claim 1, wherein the linearly analyzing comprises a quality measure to assess measurement bias.

4. The method of claim 1, wherein the taking further comprises providing expected first derivative values by determining the average derivative curves of known regions of interest without defects.