KR101945927B1 - System and method for detecting defects on a specular surface with a vision system - Google Patents

Abstract

The present invention relates to a system and method for detecting defects on a specular surface,
The optical path utilizes a knife-edge technique in which a camera capture or external device is formed into a physical knife edge within an optical path, the optical path effectively shields light reflected from the irradiated specular reflection surface at a predetermined angle of slope value, To refract light in other slopes reaching the slope. Light reflected from the flat portion of the surface is blocked by the knife-edge. The light reflected from the slope portion of the oyster is reflected to an entrance margin. The radiation beam is angled with respect to the optical axis of the camera to provide a suitable angle of inclination to the surface during inspection. The surface may be motion-related or stopped relative to the camera.

Description

[0001] SYSTEM AND METHOD FOR DETECTING DEFECTS IN A REGULARLY SURFACE SURFACE OF A VISION SYSTEM [0002]

Related application

This application claims the benefit of US Provisional Application No. 62 / 255,360, filed November 13, 2015, entitled " SYSTEM AND METHOD FOR DETECTION DEFECTS ON SPECULAR SURFACE WITH A VISION SYSTEM & US Provisional Application Serial No. 62 / 274,094 entitled " SYSTEM AND METHOD FOR DETECTING DEFECTS ON A SPECULAR SURFACE WITH A VISION SYSTEM ", filed October 5, 2016, entitled " SYSTEM AND US Provisional Serial No. 62 / 404,431 entitled " METHOD FOR DETECTING DEFECTS ON A SPECULAR SURFACE WITH A VISION SYSTEM ", each of which is expressly incorporated herein by reference in its entirety.

Technical field

The present invention relates to a machine vision system for inspecting the surface of an object, and more particularly to a vision system for inspecting a specular surface.

Herein, a machine vision system, also referred to as a " vision system " is used to perform various tasks in a manufacturing environment. Generally, a vision system consists of one or more cameras having an image sensor (or " imager ") that acquires a grayscale or color image of a scene that contains the object being manufactured. An image of the object may be analyzed to provide data / information to the user and associated manufacturing process. The data generated by the image is typically generated by a vision system in one or more vision system processors, which may be a special purpose application, or by a visualization system instantiated in a general purpose computer (e.g., a PC, laptop, tablet or smart phone) instantiated) by one or more software application (s). Some types of work performed by the vision system may include inspecting a fixed surface or moving surface, e.g., a surface of a conveyor or a moving stage (moving means) and an object.

Performing surface inspection on objects with specular surface finishes can pose problems for vision systems. In general, the reflection from such a surface is small due to the small difference in slope over a small area relative to the surrounding surface, which is lost due to the large volume of reflected light incident on the camera, resulting in defects and surface imperfections (e.g., (E.g., small craters / valleys and / or bumps / hills). One technique that attempts to detect surface imperfection on a specular surface is to use a dark illumination area where illumination light projected into the object is not collected at the objective lens. This serves to highlight any surface imperfections that scatter light. However, this technique is limited to being installed and used in, for example, an environment in which there is relative movement between the object and the camera assembly.

The present invention provides a physical knife-edge structure that effectively blocks light reflected at a predetermined degree of inclination from an illuminated specular surface and allows differently obliquely deflected rays to reach the vision system camera sensor SUMMARY OF THE INVENTION It is an object of the present invention to provide a system and method for detecting and imaging specular surface defects on a specular surface using a knife-edge technique for setting a camera aperture or an external device to form within an optical path. Overcome. In one embodiment, the illumination is condensed by the optics to exit the illuminator and converge into the area of the surface in areas exceeding the area of the surface being examined. The light is reflected by the (specular) zone and continues to converge to a spot near the entrance aperture of the camera or to a spot on the aperture stop (e.g., adjustable iris) in the camera . At any position, light reflected from the flat portion of the surface is mostly blocked by the knife-edge or aperture stop. The light reflected from the inclined portion of the defect is, conversely, mostly reflected to the entrance opening. The illumination beam is angled with respect to the optical axis of the camera, providing an appropriate angle of incidence for the surface to be inspected. Illustratively, the illuminator may include a linear polarizer that transmits polarized light to the surface of the object. The object may be multi-layered and may comprise (e.g., a polarizing layer). The polarized light is reflected from the surface to an intersected polarizer in the camera sensor / camera optics. Illustratively, the surface is stationary and may be acquired by a 2D sensor array, or the surface may be movable relative to the camera, which may include a line scan sensor of a line scan camera.

In an exemplary embodiment, a system and method for imaging defects on the specular surface of an object are provided. The surface is imaged by a vision system camera having an image sensor and optical equipment and having an optical axis. The fixture assembly projects the structured light beam onto the surface at a predetermined angle that is not parallel to the optical axis. The knife-edge element is associated with the optical device and variably occludes a portion of the maximum field of view of the optical device. Wherein the knife-edge element and the predetermined angle are such that the light reflected by the sensor through the optics substantially deviates from a sloped floor and bony or ripple and waviness of the defect feature of the surface. And the reflected light surrounding the inclined defect feature is blocked by the knife-edge element, respectively. Illustratively, the knife-edge element comprises a variable aperture in the optics, and the predetermined angle is associated with a gradient of surface deformity from a flat surface. In an embodiment, the sensor is a 2D sensor, and the object is fixed relative to the camera. Alternatively, the sensor has a line scan camera array, the object moves relative to the camera, and the illuminator assembly projects an illumination line onto the surface. Using a line illuminator, it is possible to inspect the moving part and inspect a portion far beyond the area covered by the single image from the 2D sensor. In an embodiment, the illumination in addition to visible light has a substantially IR (infrared) or near-IR (infrared) wavelength range, and the object may comprise a layer comprising an anti-reflective coating and / In this case the illumination may be polarized, and the optics comprise a polarized filter. As a non-limiting example, the object may be an AMOLED display, the polarizing layer is a 1/4? Retarder, and the polarized filter has crossed polarized filters. The illuminator may include a polarizer that polarizes the illumination, and the optical device includes a polarizing filter. The illumination source may form a focused beam that converges towards a point near the knife-edge structure. The knife-edge structure may have an external structure located in the front optical path of the optical device (between the optical device and the object). Illustratively, the illuminator assembly projects light through a beam splitter present in the optical axis of the vision system camera such that off-axis illumination from the illuminator assembly coincides with the optical axis To be projected onto the surface of the object. In another embodiment, the fixture assembly includes a plurality of illumination sources, each of which projects light with a respective beam splitter, each beam present in the optical axis of the vision system camera, so that off- And projected onto the surface of the object coincident with the optical axis by the beam splitter.

Illustratively, the knife edge element may have an occulting structure in the optic positioned on the optical axis. The cover structure is present in a mask member provided adjacent to the front of the optical apparatus. The cover structure may be arranged to selectively enhance or suppress scattered light associated with the feature. The cover structure may comprise a line extending across the optic in an elongation direction and having a width in the stretching direction and a width transverse to the size of the focused spot of light in the optic . The stretching direction may be defined by the orientation of the feature. The mask member may include opaque peripheral zones in opposing sides of the line having a linear opening between the line and the opaque zone. The cover structure may include a circular disc that is approximately centered on the optical axis, having a diameter for the size of one or more of the features. An annular zone surrounds the disc and may have an annular opening between the inner periphery of the annular zone and the disc. The annular zone may be arranged to suppress scattered light. Illustratively, the mask member comprises at least one of a snap-on or screw-on lens cover, an applique disposed in front of the optics, and a variable-pattern electro-optic mechanism located in the optics . In an embodiment, the arrangement may comprise a first polarizer positioned with the optics, and a second polarizer positioned with the fixture assembly.

The following detailed description of the invention refers to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 shows an illuminator and an aperture controller, arranged to analyze a surface defective feature on a specular surface of an object, comprising a camera with a 2D pixel array for capturing an image of an exemplary fixed object surface with a defect ≪ RTI ID = 0.0 > a < / RTI >
FIG. 2 is a block diagram of an embodiment of an illumination system including a camera with a line-scan (1D) pixel array for capturing an image of an exemplary moving object surface with defects, And an aperture control; FIG.
FIG. 3 illustrates application of a knife-edge effect to analyze surface features on a surface in accordance with the arrangement of FIG. 1 or FIG. 2; FIG.
Figure 4 shows a vision system arrangement of Figure 2 that is configured to include a plurality of layers including an exemplary polarizing layer in an AMOLED display manner and that scans an exemplary object that includes floor and bone defect features in at least one layer, A drawing;
5 is a side view of the vision system arrangement of FIG. 4 and an exemplary scanned object, showing the optical path of the illumination light and the optical path of the light reflected and acquired by the floor defect feature;
Figure 6 is a side view of the vision system arrangement of Figure 4 and an exemplary scanned object, showing the optical path of the illumination light and the light reflected and acquired by the bone defect feature;
FIG. 7 is a side view of a vision system arrangement showing the use of an external knife-edge structure in connection with a camera and an optical apparatus, in accordance with an exemplary embodiment; FIG.
8 is a side view of an exemplary scanning object and a vision system arrangement showing a plurality of illumination sources and associated knife-edge assemblies in accordance with a further embodiment;
Figures 9 and 10 illustrate a vision system operating in accordance with the description of Figure 1, including a beam splitter, using one illuminator and two illuminators, respectively, to provide off-axis illumination consistent with the camera ' A drawing;
11 illustrates an image of an exemplary object surface having visible defect features each displayed in shadow graph form on a peripheral surface using a vision system in accordance with an embodiment of the present disclosure;
12 is a flow diagram of a process for determining a wavy shape on an specular surface of an object using off-axis illumination and a knife-edge arrangement in accordance with one embodiment;
Figs. 13-15 illustrate exemplary histograms of image intensities, each showing a response from a combination of a smooth surface feature and a wavy surface feature, a wavy surface feature, and a smooth surface feature;
16 is a diagram illustrating an exemplary optical arrangement of an image sensor and an illuminator that illuminates and scans a line of a surface in accordance with one embodiment;
FIG. 17 is a perspective view of the view of FIG. 16; FIG.
18 illustrates an exemplary vision according to an exemplary embodiment using a mask-shaped knife-edge element located in front of, or within, the optical instrument assembly of the camera or comprising a fixed or variable filter element positioned within the optical instrument assembly. A system camera and a fixture;
19 is a front view of a mask used in the exemplary camera of Fig. 18 having an exemplary opaque centerline within a linear aperture; Fig.
Figure 20 is a front view of a mask having an exemplary circular disk element in the center of the optics assembly;
Figs. 21-25 are front elevational views, respectively, of an exemplary opaque disk element and an outer annular element, wherein these elements are separated by an annular opening and each element has a predetermined diameter; Fig.
26 illustrates an image of an exemplary object (e.g., a touch screen surface) imaged by the vision system camera and the arrangement of the illuminator of FIG. 18 having a mask, showing the wavy surface detail on the object;
Figure 27 illustrates an image of the exemplary object of Figure 26 imaged by the vision system camera and the illuminator arrangement (e.g., sensor array) of Figure 18, showing additional fine detail on the touch screen surface; And
28 illustrates an image of the exemplary object of FIG. 26 imaged by the vision system camera and illuminator arrangement of FIG. 18, showing very precise details of the sensor array shown in FIG. 27;

I. System Overview

FIG. 1 illustrates an exemplary vision system arrangement 100 in accordance with an exemplary embodiment in which a scene includes a fixed specular object 110 positioned relative to a fixed vision system camera 120. In this embodiment, the vision system camera 120 includes a two-dimensional (2D) image sensor S that includes an N X M pixel array in a rectangular array (e.g.). The camera includes an optics package (OC) that can include any acceptable lens assembly (e.g., a C-mount, F-mount, or a lens with an M12 base). In this embodiment, the lens includes a manual or automated aperture control (e. G., A variable aperture-aperture) that allows the user or other external controller to enter an appropriate aperture setting 124 (as described further below) . The sensor S and the optical device OC collectively have an optical axis OA which is generally normal to the generalized surface plane of the object 110. [ The array 100 includes an illuminator 130 that projects a collimated light beam 132 onto the surface 110 (e.g., by optical instrument OI). Beam 132 is adapted to avoid being largely retained on the object and extending to the rest of the scene in one embodiment. The beam 132 is oriented at an angle A relative to the generalized plane of the specular surface of the object 110. This angle is not perpendicular to the plane normal N (generally acute). This normal line N is generally parallel to the camera optical axis OA.

As shown, the surface of the object 110 may include a downwardly projecting bone (also referred to as a " hole ") or an upwardly projecting floor (downwardly) (Also referred to as a defect feature 140). The illuminated scene and image data 140 from object 110 are passed to vision system processor 150 in an exemplary embodiment. Processor 150 may be directly integrated into one or more of the camera assemblies or may be coupled to one or more of the camera assemblies either directly or indirectly via a suitable user interface (e.g., mouse 162, keyboard 164) (E.g., a computing device 166). The computing device 160 may be a server, a personal computer, a laptop, a tablet, a smartphone or a particularly special purpose processing device with associated memory, a networking device, Data storage media, and the like.

 The vision system process (processor) 150 may include various functional software processes and modules. The process / module may include a controller 152 that controls the camera / sensor and various parameters of the illuminator 130 (via the illumination control information 170). The vision system process (processor) 150 further includes a number of vision tools 152, such as feature detectors (e.g., edge detectors, corner detectors, blob tools, etc.). These tools are used to analyze the surface features of the image and to locate the defect feature 140 (e.g.) under the illumination and optical conditions described below. The vision system processor further includes a defect finder / discovery module 156 that uses various tools 154 to locate and identify defects on the surface. The defects may be quantified and the appropriate information 172 may be passed to the process (e. G., Partial removal and alarm process) 174.

As described further below, the camera 120 may include a polarization filter (P) in various embodiments (within the optical path). Other filters (PI) may be provided in the illuminator to transmit the polarized light beam to the surface.

Referring to FIG. 2, a vision system arrangement 200 is shown, wherein an exemplary object 200 having a specular surface is oriented along the direction of movement (arrow M) through the imaged scene. Illustratively, the object is moved by a moving means 212 which may include a moving stage or a conveyor. The camera 220 includes the sensor S1 and the optical device OC1 in this embodiment. Illustratively, the sensor S1 is arranged in a 1D pixel array (or a 2D array in which one pixel row is addressed) in this embodiment, and has a line scan camera. In one embodiment, the camera is operable to read one or more lines of the 2D pixel array. In such an arrangement, time-domain techniques, which can be readily apparent to those skilled in the art, are used to combine pixel information from multiple lines into a single image. Thus, an object can be scanned mechanically while sequentially reading the imager, forming an image that is progressively scanned over this portion. The size of the image may thus far exceed the high contrast area of the imaging system or detector. It is understood that in an alternative embodiment the imaging / illumination system may be scanned into one unit while this portion remains fixed. Thus, line-scanning can often provide high contrast over unlimited sized areas, as opposed to the fixed arrangement of FIG. 1 where high-contrast is focused on a spot or area. However, moving object arrangements and fixed object arrangements provide advantages in certain applications.

The optical device OC1 includes an aperture control section 224 as described above. The scene is illuminated by an illuminator 230 that illustratively projects a line of light 232 to the scene and surface of the object 210. In particular, the line is parallel to the extending direction of the sensor pixel array and extends orthogonally to the moving direction M. The optical axis OA1 of the camera sensor S1 and the optical device OC1 is generally perpendicular / normal to the generalized surface plane of the object and the projected " fan " of light is perpendicular to the surface plane normal N1 (Acute angle) A1. The camera passes the image data 240 to the vision system process (processor) 250 in the form of a sequence of scan lines. The process (processor) operates similarly to the process (processor) 150 (FIG. 1) described above. In this embodiment, the moving means also conveys movement information 242 (e.g., in the form of an encoder click or pulse) to a process (processor) 250. This information is used to register each scan line with respect to the physical coordinate space of the object (e.g., based on a predetermined physical movement increment in the direction of movement M associated with each pulse). Thereby allowing the process (processor) 250 to construct a 2D image of the object from a series of 1D pixel lines. Process (processor) 250 may be part of a suitable processing device (not shown, but may be similar to device 160). The process (processor) 250 may also provide illumination control 270 and may control the illumination of the object surface based on the operation of the imager and illumination control 252, vision system tool 254, And transmits defect information. These elements operate similar to the operation of the above-described process (processor) 150 (FIG. 1). The camera optics may include a polarizer P1, and the illuminator 230 may also include a polarizer, the function of which will be described further below.

In one embodiment, the illuminator may comprise an LED-driven, fiber optic illuminator or any other acceptable illuminator. In either arrangement, the light may be provided in particular as visible light, IR, or near IR. In various embodiments, it is understood that relative movement between an object and a camera may be achieved by moving the object, by moving the object, or by moving both the object and the camera. The movement may be linear or arcuate.

II. Optical relationship

Using the two illustrated exemplary arrays 100 and 200 to obtain an image of an object having a specular surface, including surface defects such as hole defects and floor defects, the operation of the system associated with the various exemplary surfaces is now Will be described in more detail. The following description relates to two arrays. 3, an exemplary surface 310 comprising a floor 312 and a valley 314 is illuminated by a light source oriented at a non-vertical angle, do. The illuminator 320 forms an area (denoted by width IA) that generally exceeds the area (indicated by width IS) by the spot illuminated on surface 310. [ This illuminated spot or surface is the area to be inspected which may include floor and bone. Light 322 and reflected light 324 illuminated by converging to the spot based on a suitable condensing optics (which may be conventional) in illuminator 320 may be directed to spot 326 near the entrance of the camera, Converge to the spot 326 on the aperture stop in the camera. At any location, the light reflected from the surface is essentially blocked by the knife-edge structure 330 and / or the aperture stop (e.g., a variable lens aperture). Based on the fact that the camera optical axis 342 and the illumination beam are tilted relative to each other and to the surface 310, the light reflected from the inclined portion of the floor defect and bone defect (light ray 344) Passes through the knife-edge structure 330, is largely reflected by the entrance aperture of the optical device 340 and reaches the sensor 350. The light at the opposing slopes of each defect is completely reflected from the entrance aperture / optics 340 and / or the knife-edge structure 330 (ray 360).

The image 370 generated in the area of the surface 310 is bright in the one half of the imaged floor 372 (based on the ray 344 incident from the facing sloped surface), the ray 360 blocked from the opposite sloped surface, Whereas the other half has a dark, shadow graph shape; The imaged bone is dark in one half (based on blocked ray 360) and bright in the other half (based on ray 344 from the facing slope). The system can distinguish a floor from a bone based on what is half dark and half bright - as shown, the left half represents the floor while the right half represents the floor. The area surrounding the floor and corners can be an area of darker or smaller intensity than the area reflected on the slopes. With this effect, the inclined surface facing the camera's optical axis tends to focus the reflected light there, and the deviation in this tilt (primary differential) results in a high contrast intensity variation in the defect, Light is effectively damped by a combination of knife-edge tilting and blocking effects (intensity is reduced by a few orders of magnitude).

For those of ordinary skill in the art, it will be appreciated that arranging the array 300 involves adjusting the spacing and tilting of the illumination beam by adjusting the tilting and spacing of the camera relative to the surface . Setting the knife-edge by locating the external structure or moving the adjustable iris is used to derive the desired light interruption level required to improve the defect in the imaged area.

III. Additional applications

The vision system arrangement can operate on various objects and surfaces. A line-scan version of the array 400 is shown in FIG. 4, wherein the line scan vision system camera 220 (described above in FIG. 2) includes a moving object 420 passing under the line- (Moving direction M). In an exemplary embodiment, the illuminator 230 may comprise a linear polarizer PI1 and the camera optics OC1 may comprise an intersected polarizing filter P1. By way of example, an object may comprise a specular layered surface, for example, an AMOLED display. This example includes an upper antireflective coating 422 on a glass or sapphire top layer 424. This covers the polarizer and / or other filters and coating 426 present on the active display layer 428. The active layer 428 includes exemplary floor defects 430 present over this layer and bone defects 440 present below this layer.

5 and 6 illustrate an illuminator light beam 510 tilted (AP1) with respect to the camera optical axis OA1 and an exemplary (active) AMOLED layer 428 is depicted at 1 / 4 < / RTI > < RTI ID = 0.0 > retarder. ≪ / RTI > Thus, by transmitting the polarized illumination beam 510, the array 400 can utilize the inherent characteristics of the object. The upper surface generally reflects some illumination light 510 through Fresnel reflection. This light is mostly blocked by the edge of the entrance aperture of the camera optics OC1. The remaining light that can be incident on this aperture is blocked by the polarizer P1 crossed at the entrance aperture oriented at 90 degrees to the illumination polarizer PI1. The illumination light passing through the upper layers 422 and 424 passes through the 1/4? Retarder and then is reflected by the active surface 428 and then passes through the 1/4? Is converted into circularly polarized light, and then rotated 90 degrees again in the second pass and is converted into linearly polarized light again. This reflected light beam 520 exits the surface and passes through the polarizer P1 to the entrance aperture of the camera optics OC1. In this manner, only light reaching the layer containing the defect (floor 430 in FIG. 5 and the valley 440 in FIG. 6) is received by the image sensor and the received (filtered) - edge to distinguish inclined defect features.

Due to the presence of various films and coating layers (e.g., antireflective coating layer 422) on the surface of the object, it is desirable to provide an illumination beam 510 in the IR or near-IR range / wavelength band . Most coatings and films (e.g., AMOLED displays, etc.) on the specular surface are used to filter light in the visible spectrum. Thus, the use of an IR or near-IR illuminator can overcome the filtering effect of this coating or film layer due to the longer wavelength of the illuminating light being transmitted. It is understood that the knife-edge structure KE1 of any permissible arrangement is provided with the camera optics OC1. In one embodiment, this may be located between the lens and the polarizer P1. In an embodiment, the knife edge may be integrated with the polarizer described further below.

Referring to Figure 7, it is contemplated that the knife-edge structure may be applied in various ways to the optical path of the camera and the optical device. The basic knife-edge structure 710 with the blade 712 and the bracket 714 is shown mounted on the front of the lens optics 720 of the vision system camera 730. The knife-edge structure occludes a portion of the entire aperture AC such that the tilted illumination beam 740 of the illuminator 750 interacts with the light beam as it is reflected at the surface 760. [

Figure 8 illustrates another embodiment of an arrangement 800 in which a pair of illumination assemblies 810 and 812 project a respective light beam 820 and 822 onto a defects-specific specular surface 830. [ Each beam 820,822 is tilted in different orientations (angles 840 and 842) with respect to the optical axis of the optical device 852 and the sensor 854 of the vision system camera 850. Thus, this light is reflected differently (potentially by the inverse of the slope of the floor and the valley). A pair of corresponding knife-edge structures 860 and 862 are positioned in front of the optics inlet to occlude each of the reflected beams 820 and 822, respectively. Alternatively, the knife-edge may be provided to the two beams by an adjustable (double arrow 870) diaphragm 872 of the optics (lens) assembly 852. It is understood that additional illuminators (greater than two) may be used to illuminate the surface at different tilting angles, and that appropriate knife-edge structures may be used.

In general, adjusting the lens opening can be accomplished in a number of ways. If an adjustment ring is provided on the lens body, the user can rotate the adjustment ring while observing the display of an exemplary object until a suitable high-contrast image of the defect is achieved. This process can be performed automatically when the lens and / or camera assembly includes a diaphragm that is electromechanically (or otherwise) driven. The vision system processor may be used to optimize the aperture setting by determining a setting that provides the highest contrast difference in the image acquired by the defect.

Referring now to Figures 9 and 10, Figures 9 and 10 illustrate a vision system camera 910 with a 2D pixel array that acquires a 2D image of an exemplary fixed object 920 having a specular surface, An exemplary vision system 900 and 1000 that includes a single illuminator 912 and a single illuminator 930 (Figure 9) or a plurality (e.g., two) of illuminators 1030 and 1032 (Figure 10) (Each), each of which provides off-axis illumination to illuminate the floor and bone defect features described above or below the specular surface of the object 920. The aperture stop or other structure associated with optical device 912 provides the knife-edge described above (generally indicated by element KE2). The illuminator (s) 930, 1030, and 1032 may each include an LED illuminator (shown as exemplary off-axis LED 940 and optics 942), a fiber bundle illuminator, May be a front light illuminator. The beam splitter 950, 1050 and 1052 of conventional design is positioned along each optical axis 960 and 1060 and the illuminator projects the beam 90 degrees to the optical axes 960 and 1060, A cube, a prism, or any other device capable of dividing a beam of light into two or more beams, wherein the two or more beams may or may not have the same optical magnification, May be oriented at an angle or may not be oriented. In this way, the off-axis illumination coincides with the optical axis of the imager. This allows a more compact design, potentially allowing the fixture to be integrated with the camera optics. Although one illuminator 930 is used in FIG. 9, using two illuminators 1030 and 1032 (FIG. 10) to provide illumination from (for example) opposite sides, And more uniformly generated. It is understood that the beam splitter may comprise several polarization filters and other light-regulating components, including lenses and the like. For example, the camera may include a polarizer P2 with an optical device 912. [ The illuminators 930 and 1030 may include a corresponding polarizer PI2 in the optical path and the illuminator 1032 may include a corresponding polarizer PI3 in the optical path. Polarizers are arranged and function as described above (see Figure 5).

IV. result

11 is a graphical representation of a display image 1100 generated by a vision system in accordance with one embodiment of the present disclosure. The image details the object surface on which a plurality of surface (or sub-surface) defects 1110 1120, 1130, 1140, 1150 and 1160 are identified. This exemplary defect is each trough 1110, 1120, 1130, and 1140 or floors 1150 and 1160, with bright halves and dark halves oriented in different directions, depending on whether the floor is illuminated or the trough is illuminated . However, each floor and each bone shows a bright half and a dark half having a common orientation regardless of size / shape as a result of the illumination being tilted. The additional vision system process may use image data associated with the defect to determine whether the defect represents a potentially unacceptable size.

V. Detection and evaluation of wave feature surface features

The systems and methods described above can be used to determine imperfections / defects in undulating, ripple-like, or wavy surface features in a specular object. By way of example, a flat panel screen may have a zone of features in a wavelike shape (wavy) rather than a floor or a dimple (dimple). Although some wavy shapes may be acceptable, it is contemplated that an object may be considered defective if the size exceeding an area or size (amplitude) of the ripple shape exceeds an acceptance threshold.

Figure 12 details the process 1200 for determining and evaluating the wave shape of the specular surface. At step 1210 of process 1200, the image system generally acquires an image of the wavy surface of a specular object, which may be using off-axis illumination and a knife-edge structure, as described above. This image can be acquired for the entire object at the same time using the appropriate lens, or acquired in a line-scan manner (described further below). With the illumination and knife-edge arrangement, most of the light projected to the surface is reflected from the image sensor optics or into the knife-edge structure, and a portion of the reflected light is directed to the image sensor based on a wavy or ripple- . This results in a bright ripple shape (e.g., line) surrounded by darker areas. In the acquired image, this series of ripple shapes is formed to look like a bright line.

In an exemplary embodiment, the acquired image data may be subjected to various image processing processes, such as a Gaussian smoothing process.

In step 1220 of the process 1200, the entire image intensity map of a pixel in the acquired image may be subjected to statistical analysis - for example, a pixel (gray scale) intensity versus a histogram of the pixel frequency . Referring to Fig. 13, there is shown a histogram illustrating an image having both smooth surface features and wavy surface features. Generally, a smooth zone represents a tightly packed intensity distribution at high frequencies. Conversely, the wavy-shaped area represents the intensity (histogram regions 1310 and 1320) that is more widely spread at low frequencies. Thus, the wavy region can be represented by a relatively broad histogram 1400 in FIG. 14, while a smooth region can be represented by a relatively narrow histogram 1500. This is understood to be one of a number of statistical techniques for analyzing a smooth zone versus wave shape region in an acquired image, usually accompanied by the degree to which a particular pixel intensity value occurs.

Referring again to process 1200 of FIG. 12, for example, the distribution of intensity values in the histogram (s) is evaluated (step 1230). This evaluation may include histogram analysis that computes the gray scale level distribution of the pixel values (e.g.) and generates a histogram tail. The process 1200 determines whether a wave shape or other defect is present (e.g., by determining whether the histogram tail is within an acceptable range of average values) (decision 1250). If a wavy shape / defect is present (e.g., the histogram tail is outside the average value range), the process 1200 branches to step 1260. As an example, an image for each histogram having a tail outside the range may be subject to a threshold. This threshold may be set by the user or automatically determined. The size and position of all defects are measured in the image (s) to which the threshold is applied. If the measured value of any defect (or set of defects) results in a result exceeding a predetermined metric (which may be user-defined or automatically set), the process 1200 may include determining whether a particular defect And / or the location of this defect. The process may take further action (step 1270) such as partial removal or generation of an alarm signal. Conversely, if the wavy shape and / or defects are not exhibited by the histogram tail, the object is considered acceptable and generally considered to be free from significant defects. This is indicated and / or no action is taken (step 1280).

The above-described process 1200 for evaluating the wavy surface features on the specular object may be performed in the manner of a line-scan process. Figures 16 and 17 show that an object 1610 having an exemplary wavy surface feature 1720 (Figure 17) is visible through the view (line 1630) of the image sensor LS (e.g., via an encoder ) Direction (double arrow M), respectively. It is understood that the scan movement MO can be either one of the opposite directions or suitably both opposite directions. The image sensor LS is present in the camera LSC and is comprised of a line-scan sensor, where one row of pixels is configured to acquire a reflected image from an object surface. The camera field of view 1630 is illuminated by the off-axis illumination provided by the line illumination source LI in the housing LIH. This illumination source LI may be any permissible optical arrangement-for example, a line of LEDs. This light is focused by a cylindrical lens 1650 of conventional design and shape and the cylindrical lens has a desired width WI and an indefinite ) Length of illumination area. In an exemplary embodiment, it is understood that the illumination line width WI may be less than a few millimeters, but may be narrower or wider depending on the scan resolution. The length is determined by the corresponding length of the light source (LI) and the cylindrical lens 1650. The cylindrical lens is located at a distance as far as the lens holder 1640 from the illumination source LI and provides a desired focal distance between the surface 1610 of the object and the source LI. In an exemplary embodiment, the cylindrical lens 1650 may be formed as a half-cylinder that is spaced apart by a lens holder 1640 at a distance that focuses the line on the surface. By projecting light off-axis as shown, most of the emitted light 1652 (projected in the plane or fan shown in Figure 17) is transmitted to the image sensor optics (e.g., lens opening) LA) and / or out any external knife-edge structure (to line 1654). Light 1656 reflected by the inclined surface is received by a line scan camera (LSC) as shown. In an exemplary embodiment, another cylindrical lens 1660 located at the end of the enclosed lens holder 1670 focuses the received light onto the camera optics (knife-edge structure LA) and the line scan sensor LS do. Various camera optic arrangements other than the cylindrical lenses shown will be apparent to those of ordinary skill in the art. As shown in Fig. 16, a polarizer PI4 may be provided in the optical path of the illumination source LI (at a variable position along the optical path). In addition, the polarizer P3 may be provided in the received light path of the sensor LS. These elements are also provided in the arrangement 1700 of FIG. 17, but are not shown in the arrangement 1700 of FIG. 17 for clarity.

Although a cylindrical lens shape is used, it is understood that multiple cross-sectional shapes-for example, a paraboloid- can be used in alternative arrangements. In addition, a mirror may be used instead of or in addition to the lens for focusing illumination light. Advantageously, the illumination arrangement ensures that the entire surface achieves consistently high illumination and that each scanned line fully represents the local inclination of the surface. This arrangement also allows advantageously any size surface to be imaged and analyzed for recessed features, flooring, and wavy shapes. For example, a tablet or laptop screen, or a larger flat panel television, can be analyzed by providing a sufficiently long line-lighting assembly and one or more line scan cameras across the object surface. Each camera can image some of the entire object and evaluate these surfaces either individually or in combination.

It is also explicitly contemplated in an alternative embodiment that a larger area of an object may be imaged using a fixture, for example with a Fresnel lens or other optical arrangement.

VI. Line, disk and ring optics mask

18 illustrates an exemplary embodiment of a generalized vision system arrangement 1800 that includes a vision system camera assembly 1810 having an image sensor 1820 and an optics assembly 1830. The sensor 1820 is interconnected (as shown) to the vision system process (processor) in the manner generally described above and performs the vision system tasks appropriate to the image acquired by the sensor 1820. The optics assembly 1830 may include any permissible variable or fixed focus and / or variable or fixed aperture lens unit, or lens units-such as a conventional M12 base, F-mount or C-mount Lens combination.

According to an exemplary embodiment, the front 1832 of the optics / lens assembly 1830 may be covered with a fixed or movable mask assembly 1840. The mask assembly 1840 may be a threaded or snap-coupled type, or may be mounted through a bracket (not shown) in front of the lens assembly 1830. The mask assembly 1840 can also be applied directly to the front (or other) surface of the lens assembly with an adhesive applicator or coating. In the case of a threaded attachment, the mask assembly 1840 may operate similarly to other conventional filters for use with various lens arrangements and may be adapted to be threaded to the end of a conventional lens filter mount.

Alternatively, the mask assembly 1840 can be manually or automatically moved into or out of the optical path of the lens as desired (e.g., via a solenoid, servo, stepper, etc.). The mask assembly may further include (for example) an electro-optic mechanism that may vary between a completely transparent pattern of a desired size and shape and a partially opaque pattern through an optional control unit 1850. As a non-limiting example, the mask assembly 1840 may include a window (generally circular) that includes an LCD shutter or other type of configurable window.

Arrangement 1800 includes illuminator 1860 described above that is oriented to project light at an angle that is not perpendicular to the entire plane of surface 1870 (as shown). In this example, surface 1870 has a wavy configuration comprised of a series of ridges 1872 and intercalations 1874 along at least one direction. Angled light reaches the floor and the valleys, and some of the light incident on the camera optics assembly 1830 is scattered by the floor and the valleys. Various types of mask assemblies 1840 include a knife-edge element that significantly attenuates scattered light and directs only a given limited angular range of light to sensor 1820. In this embodiment, the mask cover / coating is represented by a dashed line 1880 that includes a center cover area 1882 and an outer cover area 1884, wherein the center cover area 1882 and the outer cover area 1884 , An open aperture is formed through which the reflected light 1890 passes from the surface 1870. In various embodiments, a polarizer PI5 may be provided with the illuminator 1860 and a corresponding polarizer P4 may be provided with the optics / lens assembly. These polarizers can generally be arranged and function as described above (see, e.g., Fig. 5).

19 shows a more detailed example of a vision system arrangement 1900 according to an exemplary embodiment. This embodiment includes a beam splitter and a polarizer arrangement similar to those shown and described in FIG. In particular, the arrangement 1900 includes a camera assembly 1910 and a lens / optical device 1920. Lens / optics 1920 includes a mask assembly 1930 according to embodiments of the present disclosure. In front of the mask assembly 1940, there is a polarizer P5 which operates in accordance with the above-described principle. A beam splitter 1950 is provided and the light reflected from the object 1960 to be inspected through this beam splitter 1950 is transmitted to the camera 1910. An illumination assembly 1970 is provided. The illumination assembly 1970 includes an illumination source 1972 and a condenser lens 1974. [ A polarizer PI6 is positioned in front of the concentrator lens. It is understood that the polarizer P5 may include a mask pattern on one side thereof and the assembly may be provided as a screw or snap fit attachment in front of the lens 1920. [

Another vision system arrangement 2000 is provided in accordance with an exemplary embodiment. The arrangement 2000 includes a camera assembly 2010 and a lens / optical device 2020. Lens / optic device 2020 includes a mask assembly 2030 according to embodiments of the present disclosure. In front of the mask assembly 2040, there is a polarizer P6 that operates in accordance with the above-described principle. A beam splitter 2050 is provided and the light reflected from the object 2060 to be inspected is transmitted to the camera 2010 through the beam splitter 2050. In this embodiment, a condenser lens 2070 is disposed between the beam splitter 2050 and the object 2060. The concentrator operates with an illumination assembly 2080 that includes an illumination source 2082, a focusing lens 2084, and a polarizer PI7. It is understood that the focusing lens 2084, the focusing lens 2070, and other optical components can be sized and arranged according to known optics principles known to those skilled in the art.

The center cover region and the outer cover region of the various mask assemblies described above may have different geometric shapes, sizes, and relationships. Choosing an appropriate mask may be done empirically or by trial and error to achieve the best image for a given surface being inspected. This is illustrated in more detail in Figures 21-27 which provide various types / sizes of mask patterns.

21, one type of mask 2100 that creates a knife-edge element has an opaque centerline 2110 centered in a transparent linear opening 2120. The centerline 2110 is shown in Fig. The remaining outer region 2130 of the mask surrounding the transparent opening 2120 is also opaque. The line (s) 2110 and 2120 are generally oriented parallel to the stretching direction of the surface (if any) characteristic wave shape, and this form of mask is most effective in this condition. More generally, the stretching direction may be selected (e.g., by rotating the mask) to enhance or suppress the surface features as desired. By way of non-limiting example, to better understand the functioning of the array, the width of the opening WLA is variable - for example from 5 millimeters to 10 millimeters, and the opaque center line WL is from 50 millimeters to 55 millimeters Lt; RTI ID = 0.0 > D0 < / RTI > In general, the width WL of the line is a size that matches the width of the spot focused from the illumination. It is understood that each of the following mask arrangements (FIGS. 22-27) takes a similar lens diameter D0. The overall size may vary in proportion to the lens of larger or smaller diameter.

22 shows a mask 2200 comprised of an opaque central circular (cover) disk 2210 of diameter (DD) to (5 millimeters to 10 millimeters). This disk provides the desired knife edge element to the array. In general, the size of the disc is selected to match the size of the surface features (e.g., defects) to be improved or suppressed. In this exemplary mask arrangement 2200, there is no opaque outer zone at the edge of the lens (dashed circle 2230), and the mask arrangement is understood to be transparent. This basic knife edge element can receive light within a given angular range from the floor and the valley, which can be oriented in various directions at the surface.

23 shows an opaque center disc 2310 with a diameter DD1 of about 9 millimeters and an opaque circular outer zone 2330 with an inner diameter DA of about 14 millimeters Gt; 2300 < / RTI > The difference between the disc diameter DD1 and the outer zone 2330 creates a transparent annular window 2320 through which light reflected from the surface can pass. In particular, the diameter of the centering disc limits the attenuation of the light in the manner of the knife edge element, while the diameter of the annular outer zone increases the clarity by defining a confocal effect in the optics system.

Some additional examples of mask configurations 2400, 2500, 2600, and 2700 having a central occlusion disk and an outer annular zone and having an annular opening therebetween are illustrated in Figures 24, 25, 26, and 27 . As a non-limiting example, the disk diameter DD2 of the mask 2400 is approximately 5 millimeters to 6 millimeters and the inner diameter DA1 of the outer annular zone is approximately 8 millimeters to 9 millimeters. The disk diameter DD3 of the mask 2500 is approximately 3 millimeters to 4 millimeters and the inner diameter DA2 of the outer annular zone is approximately 5 millimeters to 6 millimeters. The disk diameter DD4 of the mask 2600 is approximately 3 millimeters to 4 millimeters and the inner diameter DA3 of the outer annular zone is approximately 8 millimeters to 9 millimeters. In addition, the disk diameter DD5 of the mask 2700 is approximately 5 millimeters to 6 millimeters, and the inner diameter DA4 of the outer annular zone is approximately 10 millimeters to 12 millimeters. This size is merely illustrative of the broadest possible size range that can be tuned to the individual characteristics of the surface depending on the angle of inspection, illumination intensity and / or wavelength (s) in the vision system arrangement.

Generally, as described above, the mask can be patterned using a variety of techniques (e.g., screen-printing, photolithography, applying a transparent film in a printed or molded pattern, etc.) Lt; RTI ID = 0.0 > a < / RTI > It will be apparent to those skilled in the art that various techniques can be used to apply fixed mask patterns to camera optics. Also, as discussed above, the mask may comprise active components, including, for example, a pixilated surface. The controller, which is separate or part of the vision system processor, optionally addresses the individual pixels of the active mask to create a mask pattern of the desired shape and size. In particular, the controller can be adapted to step-by-step through various configurations until the user or an automated vision system process (e.g., based on contrast) determines the best pattern setting. The pattern may have a shape similar to that described in Figs. 21-27, or may have a more complex shape that better conforms to unique surface characteristics and / or wavy pattern.

It is understood that in certain embodiments, a plurality of cameras interconnected to one or more vision system processors may be used. Each camera can acquire an image of an object surface at a different size and / or configuration (e.g., a different sized cover disc) of the mask from similar or different angles, and multiple images of the surface can have different sizes, / ≪ / RTI > or wavy-shaped features of the orientation are properly imaged. Similarly, if the mask is variable (by placing a different mask in front of the optical device or by varying the pattern of the mask), multiple images can be acquired and analyzed.

Referring to image 2800 of FIG. 28, it is shown that a conventional touch screen of a handheld device is imaged using a mask according to the above-described embodiment. In this image, the surface wave form of a relatively flat surface without features can be clearly distinguished, even though it is viewed in me or in a more conventional vision system arrangement. In Fig. 29, the image 2900 further shows the details not normally seen - in this example, the sensor matrix / array 2910 of the touch screen. The level of detail that may be achieved using the mask and imaging techniques described herein is further illustrated, for example, in the image of FIG. 30 by an enlarged view of the area of the touch screen where the individual wires of the array 2910 of FIG. 29 Lt; RTI ID = 0.0 > 3010 < / RTI >

VII. conclusion

It is apparent that the above-described systems and methods provide an effective technique for identifying multi-layered specular surfaces and inclined defects, including floor defects and bone defects and ripple shape / wavy defects, on a layerless specular surface. By suitably applying the wavelength of the illumination light and a filter (e.g., several polarizers), the present system and method can effectively image a surface with multiple coatings and layers. Preferably, the exemplary knife-edge arrangement has a slope of the defect that causes the light reflected from the floor or bone to appear brighter or darker than the background, depending on whether the floor and the bone are present on either side Differential) can be identified. The size of the defect is potentially combined proportionally to the slope of the defect. Small defects can have small inclination and can deflect the illumination light by a small amount from the background. By having a small spatial extent of the light source, the light source can produce a small focus after being reflected from the surface under test, thereby blocking the background better without blocking the defective light. However, extending the source further can reduce the negative impact of random test surface tilting that may occur in a manufacturing environment, in addition to reducing defect contrast. Thus, the knife-edge preferably improves the contrast by reducing the background through blocking background rays. Additionally, the inclination, shape, and polarization detection may be combined in an exemplary fashion to filter out most of the background light and filter out the aperture of the camera while focusing light from the tilted defects in the camera at a high-contrast. Further, with the exemplary arrangement, the variation of the size of the specular surface can be broadened by using the line-scan camera and the focused illumination line in general. Embodiments herein also provide a mask that includes a knife-edge element and other elements (e.g., the same focus element) that provide a highly improved view of a particular type of surface wave shape.

The foregoing is a detailed description of exemplary embodiments of the invention. Various modifications and additions may be made to the present invention without departing from the spirit and scope of the invention. The features of each of the various embodiments described above may, where appropriate, be combined with the features of the other described embodiments to provide multiple feature combinations in the associated new embodiment. Further, while the foregoing describes a number of separate embodiments of the apparatus and method of the present invention, the disclosure herein is merely illustrative of the application of the principles of the present invention. For example, and as used herein, the term " process " and / or " processor " should be broadly interpreted as including multiple electronic hardware and / or software- Quot; or " element "). Further, the depicted process or processor may be combined with other processes and / or processors, or may be subdivided into multiple sub-processes or processors. Such sub-processes and / or sub-processors may be combined in various manners in accordance with the embodiments disclosed herein. Furthermore, any function, process, and / or processor may be embodied as an electronic hardware, software comprised of non-transient computer-readable media of program instructions, or a combination of hardware and software, . Additionally, the terms "vertical", "horizontal", "up", "down", "lower", "upper", "side", "forward", "rearward", " "Right", etc., are used only as a relative convention and not as an absolute direction / placement for a fixed coordinate space, such as the direction of gravity. Additionally, the term " substantially " or " approximately " when used in a given measure, value, or characteristic refers to an amount that falls within the normal operating range to achieve the desired result, And some variation due to errors in tolerances (e.g., 1-5 percent). Accordingly, the description is to be construed as illustrative only and is not intended to limit the scope of the invention.

Claims (26)

A system for imaging a defect in a specular surface of an object,
A vision system camera having an image sensor and an optical instrument and having an optical axis oriented to image the surface;
A fixture assembly configured to project a structured light beam onto the surface at a predetermined angle that is not parallel to the optical axis; And
And a knife-edge element associated with the optical device, the knife-edge element variably occluding a portion of a maximum field of view of the optical device,
Wherein said knife-edge element and said predetermined angle are such that light reflected by said sensor through said optics is substantially transmitted from a sloped hill and valley of a feature on said surface, And the reflected light surrounding the bone is occluded by the knife-edge element, respectively. 2. The system of claim 1, wherein the knife-edge element comprises a variable aperture in the optics. 2. The system of claim 1, wherein the predetermined angle is associated with the tilt of the floor and the trough. 2. The system of claim 1, wherein the sensor is a 2D sensor and the object is fixed relative to the camera. 2. The system of claim 1, wherein said sensor comprises a line scan camera arrangement, said object moving relative to said camera. 6. The system of claim 5, wherein the fixture assembly projects an illumination line to the surface. 7. The system of claim 6, wherein the illumination has a substantially IR (IR) or near-IR wavelength range. 8. The system of claim 7, wherein the object comprises a layer comprising an anti-reflective coating. 9. The system of claim 8, wherein the layer comprises a polarizing layer, the illumination is polarized, and the optics comprises a polarized filter. 10. The system of claim 9, wherein the object is an AMOLED display, the polarizing layer is a 1/4 lambda retarder, and the polarized filter comprises an intersected polarized filter. 7. The system of claim 6, wherein the illuminator comprises a polarizer for polarizing illumination, the optics comprising a polarizing filter. 2. The system of claim 1, wherein the fixture assembly includes at least one illumination source, the illumination source forming a focused beam that converges towards a point near the knife-edge element. 2. The system of claim 1, wherein the knife-edge element has an outer structure located in a front optical path of the optical device. 2. The illumination system of claim 1, wherein the fixture assembly projects light through a beam splitter present in the optical axis of the vision system camera, such that off-axis illumination from the fixture assembly is reflected by the beam splitter To be projected onto the surface of the object corresponding to the surface of the object. 2. The illumination system of claim 1, wherein the fixture assembly comprises a plurality of illumination sources, each of the illumination sources projecting light to each beam splitter present in the optical axis of the vision system camera, Is projected onto the object surface coincident with the optical axis by the beam splitter, respectively. 6. The system of claim 5, wherein the optics comprises at least one imaging lens, wherein the fixture assembly focuses onto a line that is projected onto the surface and then, after reflection, is outside the entrance aperture of the imaging lens. . 17. The system of claim 16, wherein the fixture assembly includes a cylindrical lens for focusing the line. 2. The system of claim 1, wherein the feature comprises waviness in a region of the surface, and wherein the analysis includes determining a distribution of pixel intensity values in an image acquired by the image sensor and comparing the distribution to a threshold value, RTI ID = 0.0 > 1, < / RTI > further comprising an evaluation process. 19. The system of claim 18, wherein the distribution is formed by at least one histogram of pixel intensity values versus frequency in the image. The optical device according to claim 1, wherein the knife edge element has an occulting structure in the optical device positioned on the optical axis, and the cover structure is present on a mask member provided adjacent to the front of the optical device, Wherein the cover structure is arranged to selectively enhance or suppress scattered light associated with the feature. 21. The optical device according to claim 20, wherein the cover structure has a line extending across the optical device in the direction of extension, and has a width in the transverse direction with respect to the size of the illumination spot focused on the optical device , The extension direction being defined by the orientation of the feature. 22. The system of claim 21, wherein the mask member comprises an opaque peripheral zone on each of the opposite sides of the line, and wherein there is a linear aperture between the line and the opaque peripheral zone. 21. The system of claim 20, wherein the containment structure comprises a circular disc centered on the optical axis, the disc having a diameter for the size of one or more features of the features. 24. The system of claim 23, further comprising an annular zone surrounding the disc and having an annular opening therebetween, the annular zone being arranged to suppress scattered light. 21. The apparatus of claim 20, wherein the mask member comprises at least one of a snap-on or screw-on lens cover, an applique disposed in front of the optical instrument, and a variable pattern of electro- Wherein the system comprises: 21. The system of claim 20, further comprising: a first polarizer positioned with the optics; and a second polarizer positioned with the fixture assembly.

Images