JP6568672B2 - System and method for detecting defects on specular surfaces in a vision system - Google Patents

Description

RELATED APPLICATIONS This application is a co-pending US Provisional Application No. 62 / 255,360, “System and Method for Detecting Specular Defects in a Vision System,” filed November 13, 2015, December 31, 2015. US Provisional Application No. 62 / 274,944 entitled “System and Method for Detecting Specular Defects with a Vision System”, and US Provisional Patent Application No. 62/404431, filed Oct. 5, 2016. No. "claims the benefits of a system and method for detecting specular defects in a vision system, the teachings of which are incorporated herein by reference.
Technical field

  The present invention relates to a machine vision system for inspecting the surface of an object, and more particularly to a vision system for inspecting a mirror surface.

  Machine vision systems, also referred to herein as “vision systems”, are used to perform a variety of tasks in a manufacturing environment. In general, a vision system consists of one or more cameras equipped with an image sensor (or “imager”) that captures a grayscale or color image of a scene containing the object being manufactured. The image of the object can be analyzed to provide data / information to the user and associated manufacturing process. The data produced by the images is usually analyzed and processed by a vision system on one or more vision system processors, which are either manufactured for it or are general purpose computers (eg PCs, laptops, tablets or smartphones). ) Can be part of one or more software applications created within. Some types of tasks performed by the vision system may include inspection of objects and surfaces on stationary or moving surfaces (conveyance), such as conveyors or motion stages.

  Performing surface inspection on mirror-finished objects can be a challenge for vision systems. In general, reflections from the surface can cause defects and surface defects (e.g. small depressions / valleys and / or protrusions / hills), which represent a small difference in slope in a small area relative to the surrounding surface, It seems to be erased by a large amount of reflected light entering the camera. One technique for attempting to detect surface imperfections on specular surfaces uses dark field illumination, and the illumination light projected onto the object is not collected by the objective lens. This helps to highlight any surface defects that scatter light. However, this technique is limited in terms of setup and use in environments that involve relative movement between the object and the camera assembly.

  The present invention overcomes the disadvantages of the prior art by providing a system and method for detecting and imaging specular defects on a specular surface using knife edge technology. Here the camera aperture or external device is set up to form a physical knife edge structure in the optical path, which effectively blocks the reflected light from the illuminated mirror surface of a given tilt and has a different tilt. Allows the light beam deflected in to reach the vision system camera sensor. In one embodiment, the illumination is collected by an optical system and advances from the illuminator with a larger area than the area of the surface under inspection and converges to the area of the surface. The light is reflected by the (specular) area and continues to converge to a spot near the entrance aperture of the camera or on an aperture stop (eg, an adjustable iris) inside the camera. At either position, most of the light reflected from the flat portion of the light surface is blocked by the knife edge or aperture stop. On the contrary, most of the light reflected from the inclined portion of the defect is reflected in the incident aperture. The illumination beam is tilted with respect to the optical axis of the camera to provide an appropriate angle of incidence with respect to the surface under inspection. Illustratively, the illuminator may include a linear polarizer that transmits polarized light to the surface of the object. The object may be multi-layered and may include (eg, a polarizing layer). Polarized light is reflected from the surface by the camera sensor / camera optics into the crossed polarizer. Illustratively, the surface may be acquired stationary by a 2D sensor array, or the surface may move relative to the camera, in which case the camera may be defined as a line scan camera or a line scan sensor.

In an exemplary embodiment, a system and method for imaging a specular defect on an object is provided. The surface is imaged by a vision system camera having an image sensor and an optical system and defining an optical axis. The illuminator assembly projects structured light at a predetermined angle onto a surface that is not parallel to the optical axis. The knife edge member is associated with an optical system that variably blocks a part of the maximum field of view of the optical system. The knife edge member and the predetermined angle, respectively, cause the light reflected by the sensor through the optical system to be transmitted substantially from the sloped hills and valleys or ripples and ripples of the defect feature on the surface, and the reflected light is the knife edge. It is set to surround the tilted defect feature blocked by the member. Illustratively, the knife edge member includes a variable aperture in the optical system, and the predetermined angle is associated with a slope of the surface deformation from a flat surface. In an embodiment, the sensor is a 2D sensor and the object is stationary with respect to the camera. Alternatively, the sensor is defined as a line scan camera configuration, the object is moving relative to the camera, and the illuminator assembly projects a line of illumination onto the surface. Using a line illuminator allows inspection of moving parts and inspection of parts much larger than the field covered by a single image from a 2D sensor. In embodiments, illumination can be defined as a substantially infrared or near infrared wavelength range in addition to visible light, and an object can be defined as a layer that includes an anti-reflective coating and / or a polarizing layer. In this case, the illumination can be polarized and the illuminator optics includes a polarizing filter. As a non-limiting example, the object can be an AMOLED display, the polarizing layer is a 1 / 4λ retarder, and the polarizing filter is defined as a crossed polarizing filter. The illuminator may include a polarizer for polarized illumination, and the illuminator optics includes a polarizing filter. The illumination source can be defined as a focused beam that converges toward a point in the vicinity of the knife edge structure. The knife edge structure can be an external structure positioned on the optical path in front of the optical system (between the optical system and the object). Illustratively, the illuminator assembly projects light through a beam splitter that is on the optical axis of the vision system camera, and off-axis illumination from the illuminator assembly is projected by the beam splitter toward the object surface that coincides with the optical axis. . In another embodiment, the illuminator assembly is defined as a plurality of illumination sources, each projecting light into a respective beam splitter, each beam splitter being on the optical axis of the vision system camera and exiting from each illumination source Off-axis illumination is projected by the beam splitter toward the object surface that coincides with the optical axis.

  Illustratively, the knife edge member can be a concealment structure in an optical system placed on the optical axis. The masking structure is on a mask element provided adjacent to the front surface of the optical system. The obscuring structure may be arranged to selectively enhance or suppress scattered light associated with the feature. A concealment structure is defined as a line extending across the extension direction of the optical system and can have a width transverse to the direction of extension relative to the size of the focused illumination spot on the optical system. The direction of extension can be defined by the orientation of the feature. The mask element includes a surrounding opaque area and a linear aperture between the line and the opaque area on opposite sides of the surrounding line. The concealment structure includes a disk centered about the optical axis and has a diameter that is relative to the size of one or more features. The annular area can surround the disk and can define an annular aperture between the disk and the inner periphery of the annular area. The annular area can be arranged to suppress scattered light. Illustratively, the mask element can be at least one of a snap-on or screw-type lens cover, the applique is placed on the front surface of the optical system, and the variable pattern electro-optic mechanism is placed on the optical system. . In an embodiment, the arrangement may include a first polarizer placed in association with the optical system and a second polarizer placed in association with the illuminator.

  The present invention will be described below with reference to the accompanying drawings.

Illuminator and aperture control arranged to resolve surface deformation defect features on a mirror surface of an object with a camera having a 2D pixel array to acquire an image of an exemplary stationary object's defective surface 1 is a block diagram of an exemplary vision system that includes.

Illumination with a camera having a line scan (1D) pixel array that captures an image of a defective surface of an exemplary moving object and is arranged to resolve surface deformation defect features on the specular surface of the object 1 is a block diagram of an exemplary vision system that includes instrument and aperture control.

FIG. 3 is a block diagram applying a knife edge effect to resolve deformed surface features on the surface according to the arrangement of FIG. 1 or FIG.

FIG. 3 is a block diagram of the vision system arrangement of FIG. 2 scanning an example object configured to include multiple layers including an example polarizing layer in accordance with an AMOLED display scheme, the object being at least one hill in the layer Includes defect and valley defect features.

FIG. 5 is a side view of the vision system arrangement and the example scanned object of FIG. 4 showing reflected light acquired for illumination light paths and hill defect features.

FIG. 5 is a side view of the vision system arrangement and the example scanned object of FIG. 4 showing reflected light acquired for illumination light path and valley defect features.

2 is a side view of a vision system arrangement showing the use of an external knife edge structure in combination with a camera and optics according to an exemplary embodiment. FIG.

FIG. 5 is a side view of a vision system arrangement and an exemplary scanned object showing multiple illumination sources and associated knife edge assemblies according to another embodiment.

2 is a block diagram of a vision system that operates in accordance with the description of FIG. 1 using a single illuminator and including a beam splitter that provides off-axis illumination that coincides with the optical axis of the camera. FIG. 2 is a block diagram of a vision system that operates in accordance with the description of FIG. 1 using two illuminators and including a beam splitter that provides off-axis illumination that coincides with the optical axis of the camera.

2 is a representation of an image of an exemplary object surface having visible defect features, each shown in the form of a shadow graph on a surrounding surface, using a vision system according to embodiments herein.

6 is a flowchart of a procedure for determining a waveform on an object's specular surface using off-axis illumination and a knife edge configuration according to one embodiment.

FIG. 4 is an exemplary histogram of image brightness showing a combination of smooth and wavy surface features, wavy surface features and responses from smooth surface features. FIG. FIG. 4 is an exemplary histogram of image brightness showing a combination of smooth and wavy surface features, wavy surface features and responses from smooth surface features. FIG. FIG. 4 is an exemplary histogram of image brightness showing a combination of smooth and wavy surface features, wavy surface features and responses from smooth surface features. FIG.

2 is an exemplary optical arrangement of an illuminator and an image sensor for illuminating and scanning a surface line according to one embodiment.

FIG. 17 is a perspective view of FIG. 16.

FIG. 5 is an illustration of an exemplary vision system camera and illuminator according to an exemplary embodiment that is placed in front of or within a camera optics assembly and uses a knife edge member in the form of a mask that includes a fixed or variable filter member.

FIG. 19 is a front view of a mask for use with the example camera of FIG. 18 that defines an opaque centerline within the example linear aperture.

FIG. 5 is a front view of a mask defining an exemplary circular disc member positioned in the center of the optics assembly.

1 is a front view of an exemplary opaque disk member and an outer annular member separated by an annular aperture, each member defining a predetermined diameter. FIG. 1 is a front view of an exemplary opaque disk member and an outer annular member separated by an annular aperture, each member defining a predetermined diameter. FIG. 1 is a front view of an exemplary opaque disk member and an outer annular member separated by an annular aperture, each member defining a predetermined diameter. FIG. 1 is a front view of an exemplary opaque disk member and an outer annular member separated by an annular aperture, each member defining a predetermined diameter. FIG. 1 is a front view of an exemplary opaque disk member and an outer annular member separated by an annular aperture, each member defining a predetermined diameter. FIG.

FIG. 19 is an image of an exemplary object (eg, a touch screen surface) imaged by the vision system camera and illuminator arrangement of FIG. 18 with wavy surface details shown thereon.

FIG. 27 is an image of the exemplary object of FIG. 26 taken with the vision system camera and illuminator arrangement of FIG. 18, showing finer details on the touch screen surface (eg, sensor array).

FIG. 27 is an image of the exemplary object of FIG. 26 taken with the vision system camera and illuminator arrangement of FIG. 18, showing very precise details of the sensor array shown in FIG. It is explanatory drawing about the detail of the sensor matrix / array of a touch screen. It is explanatory drawing about the example of the detail level which can be achieved using a mask and an imaging technique.

        I. System overview

  FIG. 1 is a block diagram of an exemplary vision system arrangement 100 according to an exemplary embodiment, where a scene includes a stationary specular object 110 positioned with respect to a stationary vision system camera 120. In this embodiment, the vision system camera 120 includes a two-dimensional (2D) image sensor S having an NXM array of pixels in a (for example) square arrangement. The camera has an optics package that may include any acceptable lens assembly (eg, C mount, F mount, or M12 base lens). In this embodiment, the lens includes manual or automatic aperture control—for example, a variable iris—and a user or other external controller can input the appropriate aperture setting 124 (described below). Both sensor S and optical system OC define an optical axis OA that is generally perpendicular to the generalized surface of object 110. In one embodiment, the arrangement 100 includes an illuminator 130 that projects a collimated beam 132 of light (eg, by the optical system OI) onto the surface 110. Beam 132 is adapted to reach most objects and not expand into the rest of the scene. The beam 132 is directed at an angle A with respect to the generalized plane of the mirror surface of the object 110. This angle A is not perpendicular to the vertical plane N (typically an acute angle). The vertical plane N is generally parallel to the camera optical axis OA.

  As illustrated, the surface of the object 110 can be a defect feature that can be a downwardly swirling valley (also referred to as a “concave”) or a hill protruding upward (also referred to as a “convex”). 140, which are effectively imaged using the arrangements and techniques described below. Image data 140 from the illuminated scene and object 110 in the exemplary embodiment is communicated to the vision system processor 150. The processor 150 can be integrated directly into one or more camera assemblies or, as shown, a separate user interface (eg, mouse 162, keyboard 164) and a display function (screen and / or touch screen 166). Within the computing device 160. The computing device 160 may include servers, PCs, laptops, tablets, smartphones or specially manufactured processing devices, as well as other types of processors and associated storage, networking configurations, data storage, etc. However, this will be obvious to those skilled in the art.

  The vision system process (processor) 150 may include a variety of functional software processes and modules. The process / module may include a controller 152 (via illumination control information 170) for various parameters of the camera / sensor and illuminator 130. The vision system process (processor) 150 also includes various vision tools 152, such as feature detectors (eg, edge detectors, corner detectors, blob tools, etc.). These tools are used to analyze the surface features of the image and look for defect features 140 (for example) under illumination and optical conditions described below. The vision system processor also includes a defect detector / detection module 156 that uses various tools 154 to locate and identify defects on the surface. Defects can be quantified and corresponding information 172 can be communicated to a processing process (eg, part rejection and alert process) 174.

  As described below, in various embodiments, the camera 120 may include a polarizing filter P (in the optical path). Another filter PI can be provided on the illuminator to transmit the polarized light beam to the surface.

  Referring to FIG. 2, a vision system arrangement 200 is shown, with an exemplary object 200 having a specular surface oriented along the direction of motion (arrow M) over the imaged scene. Illustratively, the object is moved by a transport device 212 that may include a motion stage or conveyor. In this embodiment, the camera 220 includes a sensor S1 and an optical system OC1. In this embodiment, sensor S1 is exemplarily arranged as a 1D array of pixels (or a 2D array in which a column of pixels is addressed), thereby defining a line scan camera. In one embodiment, the camera is operable to allow reading one or more lines of a 2D pixel array. In such an arrangement, the pixel information from the line can be combined into a single image using time domain methods that should be obvious to those skilled in the art. In this way, the object can be mechanically scanned, while the imager progressively scans the images of the parts that have been scanned progressively. Thus, the dimensions of the image may be much larger than the high contrast area of the imaging system or detector. It should be noted that in an alternative embodiment, the imaging / illumination system can be scanned as one piece with the parts stationary. Thus, line scanning can often provide high contrast over an area of unlimited size, as opposed to the stationary arrangement of FIG. 1 where high contrast is concentrated in spots or areas. However, both moving and stationary object configurations have advantages for specific applications.

  The optical system OC1 includes the aperture control 224 as described above. The scene is illuminated by an illuminator 230 that projects an exemplary line of light 232 onto the scene and surface of the object 210. In particular, the lines extend parallel to the direction of extension of the sensor pixel array and perpendicular to the direction of movement M. The optical axis OA1 of the camera sensor S1 and the optical system OC1 is generally perpendicular / perpendicular to the generalized surface of the object, and the “fan” of the projected light is not perpendicular to the normal plane N1 relative to the surface. It is oriented at an angle (acute angle) 1. The camera communicates image data 240 to a vision system process (processor) 250 in the form of a series of scan lines. The process (processor) operates similarly to the process (processor) 150 (FIG. 1) described above. In this embodiment, the conveyor also communicates motion information (eg, in the form of encoder clicks or pulses) 242 to the process (processor) 250. This information is used to register each scan line with respect to the physical coordinate space of the object (eg, based on a predetermined physical motion increment in the motion direction M associated with each pulse). This allows the process (processor) 250 to construct a 2D image of the object from a series of 1D pixel lines. The process (processor) 250 may be part of a suitable processing device (not shown but similar to device 160 described above). Process (processor) 250 also provides illumination control 270 to provide appropriate defect information for the object surface based on the operation of imager and illumination control 252, vision system tool 254 and defect detector 256. These operate in the same manner as in the process (processor) 150 (FIG. 1) described above. The camera optics can include a polarizer P1, and the illuminator 230 can also include a polarizer, the function of which will be described below.

  In one embodiment, the illuminator can be an LED driven, fiber optic illuminator or other acceptable illuminator. In either arrangement, light may be provided at visible light, infrared light or near infrared light, and other wavelengths. Note that in various embodiments, the relative motion between the object and the camera can be achieved by moving the object, by moving the camera, or by moving both the object and the camera. The movement may be linear or arcuate.

        II. Optical relationship

  Having described two exemplary arrangements 100 and 200 for acquiring images of objects having specular surfaces including surface deformations such as recesses and hill defects, the operation of the system in connection with various exemplary surfaces will now be described. Will be described in detail. The following description applies to both configurations. As illustrated in the exemplary (schematic) arrangement 300 of FIG. 3, an exemplary surface 310 that includes hills 312 and valleys 314 is illuminated by a light source directed at a non-normal angle. . The illuminator 320 defines an area (represented by the width IA) that is generally larger than the area (represented by the width IS) due to the spot illuminated on the surface 310. This illuminated spot or surface is the area to be examined and may include hills and valleys. The illumination beam 322 and the reflected beam 324 focused on the spot based on suitable collection optics (which may be conventional) in the illuminator 320 are either near the incident aperture of either camera, Or it continues to converge to the spot 326 on the aperture stop inside the camera. At either location, the light reflected from the surface is blocked primarily by the knife edge structure 330 and / or the aperture stop (eg, variable lens iris). Based on the illumination beam and camera optical axis 342 being tilted relative to each other and the surface 310, most of the light reflected from the slopes of the hill and valley defects (ray 344) is the knife edge structure 330. Near the incident aperture of the optical system 340 and reaches the sensor 350. All light striking the slope on the opposite side of each defect is separated from the firing aperture / optical system 340 and / or reflected into the knife edge structure 330 (ray 360).

  The resulting image 370 of the area on the surface 310 has the form of a shadow graph, with the imaged hill 372 having one half bright (based on rays 344 entering from the opposite slope) and the other Are dark (based on the blocked ray 360 from the opposite slope). In contrast, the imaged trough is dark on one half (based on blocked light beam 360) and bright on the other half (based on light beam 344 from the opposite slope). The system can distinguish hills from valleys based on which half is dark and which half is bright. That is, the left half light shown represents a hill, and the right half light represents a valley. The area surrounding the hills and valleys may be a dark field or a field with a lower brightness than the slope reflection area. As a result of this effect, the slope opposite the camera optical axis concentrates the reflected light on the optical axis, and deviations in this slope (first derivative) tend to produce a contrast change with high brightness for the defect. In contrast, light from the area surrounding the defect is effectively attenuated (lightness is several orders of magnitude smaller) by combining the knife edge tilt and the blocking effect.

  Those skilled in the art will appreciate that the setup of the arrangement 300 involves the proper spacing and tilt of the illumination beam relative to the tilt and spacing of the camera relative to the surface. Knife edge settings by positioning the external structure or moving the adjustable iris are then used to derive the desired level of light blocking necessary to enhance defects in the imaged field.

        III. Other applications

  The vision system arrangement described above can work with a variety of objects and surfaces. A line scan version of the arrangement 400 is shown in FIG. 4, where the line scan vision system camera 220 (described above in FIG. 2) moves as described above and passes through the object 420 (motion) passing under the line illuminator 230. M). In the exemplary embodiment, the illuminator 230 may include a linear polarizer PI1, and the camera optics OC1 may include a cross polarization filter P1. By way of example, the object can be a layered mirror, such as an AMOLED display. This example includes an anti-reflective coating 422 applied over a glass or sapphire top layer 424. This covers the coating 426 overlying the polarizer and / or other filters and active display layer 428. The active layer 428 includes an exemplary hill defect 430 above the layer and a valley defect 440 below the layer.

  Referring also to FIGS. 5 and 6 which show the illuminator beam 510 at an inclination AP1 with respect to the camera optical axis OA1, the exemplary (active) MOLED layer 428 is a conventional polarization rotation layer, eg, a 1 / 4λ retarder. be able to. Thus, the arrangement 400 can take advantage of object-specific properties by transmitting a polarized illumination beam 510. The top surface typically reflects through part of the illumination light 510 by Fresnel reflection. Most of this light is blocked by the edge of the incident aperture of the camera optical system OC1. The remaining light that can enter this aperture is blocked by the crossed polarizer P1 that is directed at 90 degrees to the illumination polarizer PI1 at the incident aperture. Illumination light transmitted through the top layers 422, 424 passes through the 1 / 4λ retarder, reflects off the active surface 428, then passes through the 1 / 4λ retarder for the second time, and is linearly polarized after the first pass. To circularly polarized light and then rotated 90 degrees on the second pass back to linearly polarized light. The reflected light beam 520 exits from the surface, passes through the polarizer P1, and enters the incident aperture of the camera optical system OC1. In this way, only the light reaching the layer containing the defects (hill 430 shown in FIG. 5 and valley 440 shown in FIG. 6) is received by the image sensor, and this received (filtered) light is The defect features that are resolved and tilted by the knife edge are then identified.

  Since there are different coatings and coating layers (eg anti-reflection coating layer 422) on the surface of the object, it is desirable to provide an illumination beam 510 in the wavelength range / wavelength band of infrared light or near infrared light. There is a case. Most coatings and coatings on mirror surfaces (such as MOLED displays) are used to filter light in the visible light spectrum. Therefore, the use of infrared or near infrared illuminators overcomes the filtering effect of these coatings or coating layers due to the longer wavelength of transmitted illumination light. Note that the knife edge structure KE1 of any acceptable arrangement is provided in conjunction with the camera optics OC1. In one embodiment, this can be placed between the lens and the polarizer P1. As described below, in embodiments, the knife edge may be integrated with a polarizer.

  Referring to FIG. 7, it is envisaged that the knife edge structure can be applied to the optical path of cameras and optics in various ways. A basic knife edge structure 710 having a blade 712 and a bracket 714 is shown assembled to the front of the lens optics 720 of the vision system camera 730. The knife edge structure occludes a portion of the overall aperture AC, thereby interacting when the tilted illumination beam 740 of the illuminator 750 reflects off the surface 760.

  FIG. 8 illustrates another embodiment of an arrangement 800 in which a pair of illumination assemblies 810 and 812 project respective beams of light 820 and 822 onto a specular surface 830 that includes a defect. Each beam 820, 822 is tilted at different orientations (respective angles 840 and 842) with respect to the optical system 852 of the vision system camera 850 and the optical axis of the sensor 854. Thus, light is reflected by different slopes (potentially slopes on opposite sides of hills and valleys). A pair of corresponding knife edge structures 860 and 862 are placed at the front entrance of the optical system to block the reflected beams 820 and 822, respectively. Alternatively, knife edges can be provided for both beams by an adjustable (double arrow 870) iris 872 of the optics (lens) assembly 852. Note that additional (two or more) illuminators can be used to illuminate the surface at other tilt angles and a suitable knife edge structure can be employed.

  In general, adjustment of the lens aperture can be accomplished in a variety of ways. If the lens body is provided with an adjustment ring, the user can rotate it and observe the display of the exemplary object until a suitable high contrast image of the defect is achieved. This process can be performed automatically if the lens and / or camera assembly includes an iris that is driven electromechanically (or otherwise). A vision system processor can be used to optimize aperture settings by determining which settings allow the defect that gives the highest contrast difference in the acquired image.

  9 and 10, exemplary vision systems 900 and 1000 with a vision system camera 910 and an optical system 912 having a 2D pixel array (respectively) are shown. These vision systems acquire 2D images of a stationary object 920 having an exemplary specular surface and include one illuminator 930 (FIG. 9) or multiple (eg, two) illuminators 1030 and 1032 (FIG. 10). Each can provide off-axis illumination to illuminate hill and valley defect features above or below the specular object 920 described above. An aperture iris or other structure associated with optical system 912 provides the knife edge described above (generally represented by member KE2). Illuminators 930, 1030, and 1032 may each be an LED illuminator (shown by exemplary off-axis LED 940 and optics 942), a fiber bundle illuminator, or other acceptable front illuminator. Conventionally designed beam splitters 950, 1050 and 1052 are placed in alignment with each optical axis 960, 1060, and the illuminator projects the beam at an angle of 90 degrees with respect to the axes 960, 1060 and emits incident light. It can be a plate, cube, prism or other device that can be split into two or more beams and may or may not have the same refractive power and may or may not be oriented at an angle of 90 degrees . In this way, off-axis illumination coincides with the optical axis of the imager. This allows for a more compact design and potentially allows the illuminator to be integrated with the camera optics. In FIG. 9, a single illuminator 930 is used, whereas using two illuminators 1030 and 1032 (FIG. 10) that illuminate from opposite sides (for example), more uniform defects on the surface. A typical image. It should be noted that the beam splitter may include various polarizing filters and other light conditioning components including lenses. For example, the polarizer P2 may be included in cooperation with the camera optical system 912. The illuminators 930, 1030 can include a corresponding polarizer PI2 in the optical path, and the illuminator 1032 includes a corresponding polarizer PI3 in the optical path. The polarizer is arranged and functions as described above (see FIG. 5).

        IV. result

  FIG. 11 shows a schematic representation of a display image 1100 produced by a vision system according to one embodiment. The image represents the object surface where a plurality of surface (or subsurface) defects 1110, 1120, 1130, 1140, 1150 and 1160 are identified. These exemplary defects are valleys (1110, 1120, 1130, and 1140) or hills (1150 and 1160), respectively, and their bright halves and darks depend on whether the hills or valleys are illuminated. Half are directed in different directions. However, each hill and trough, regardless of size / shape, shows a bright half and a dark half in a common orientation as a result of the illumination tilt. Furthermore, the vision system process can use the image data related to the defects to determine if they represent a potentially acceptable size.

        V. Detection and evaluation of wavy surface features

  Using the systems and methods described above, defects / defects in the form of undulating, wavy or wavy surface features on specular objects can be determined. By way of example, a flat panel screen may have areas of undulating (waving) features (somewhat continuous) rather than hills or depressions. Some waveforms are acceptable, but it is assumed that if such features are excessive in terms of ripple area or magnitude (amplitude), the object is considered defective beyond the acceptable threshold. Yes.

  FIG. 12 shows a procedure 1200 in which the specular waveform is determined and evaluated. At step 1210 of the procedure 1200, the imaging system generally acquires an image of a possibly wavy specular object surface using off-axis illumination and a knife edge structure as described above. This image can be acquired for all objects at the same time using an appropriate lens, or it can be acquired in a line scanning manner (described below). The illumination and knife edge arrangement reflects most of the light projected onto the surface from the image sensor optics or into the knife edge structure, and some of the reflected light is reflected in the image sensor based on the wave or ripple slope. Directed. The result is a bright ripple (eg, a line) surrounded by a darker field. A series of such ripples appearing as bright lines is defined in the acquired image.

  In an exemplary embodiment, various image processing procedures, such as a Gaussian smoothing process, can be applied to the acquired image data.

  At step 1220 of the procedure 1200, a statistical analysis may be performed on the overall image brightness map of pixels in the acquired image to create a histogram that compares, for example, pixel (grayscale) brightness in the image and pixel frequency. Referring to FIG. 13, a histogram of an image having both smooth and wavy surface features is shown. In general, smooth areas show a high density of lightness distribution. Conversely, the wavy area shows that the lightness is widely diffused with low frequency (histogram areas 1310 and 1320). Thus, the wavy area can be represented by the relatively wide histogram 1400 of FIG. 14 while the smooth area can be represented by the relatively narrow histogram 1500. Note that this is one of a wide variety of statistical techniques for analyzing the smoothed and wavy areas of the acquired image, typically using the degree of occurrence of a certain pixel brightness value.

  Referring again to procedure 1200 of FIG. 12, for example, the distribution of brightness values in the histogram is evaluated (step 1230). Evaluation can include histogram analysis, where (for example) a grayscale level distribution of pixel values is calculated to generate a histogram tail. Procedure 1200 then determines (for example) whether a wavy defect or other defect is present by calculating whether the histogram tail is within an acceptable range (decision step 1250). If a waveform / defect exists (eg, the histogram tail is outside the average range), the procedure 1200 branches to step 1260. For example, a threshold can be applied to an image for each histogram whose tail is out of range. This threshold is set by the user or can be determined automatically. The size and position of all defects in the threshold image are then measured. If the measurement of any defect (or set of defects) exceeds a predetermined metric (defined by the user or set automatically), then the procedure 1200 may identify a specific defect on the object surface and / or Indicate the location of the defect. The procedure can also take other actions, such as part rejection or alarm generation (step 1270). Conversely, if the waveform tail and / or defects are not indicated by the histogram tail, the object is considered acceptable and generally has no substantial defects. This is indicated and / or no action is taken (step 1280).

  The procedure (1200) described above for evaluating wavy surface features on a specular object can be performed in a line scanning process manner. FIGS. 16 and 17 show arrangements 1600 and 1700, respectively, where an object 1610 having an exemplary wavy surface feature 1720 (FIG. 17) is known over the field of view (line 1630) of the image sensor LS (eg, an encoder). Through) the movement (both arrows M). It should be noted that the scanning motion MO applies to either one of the opposite directions or both directions. The image sensor LS is in the camera LSC and is configured as a line scanning sensor so that an image reflected from the surface of the object can be acquired by one column of pixels. The area of the camera field 1630 is illuminated by off-axis illumination provided by a line illumination source LI in the housing LIH. This illumination source (LI) may be any acceptable light configuration, for example a line of LEDs. The light is focused by a cylindrical lens 1650 of conventional design and shape that provides a desired width WI and an indefinite length illumination zone (over the object surface transverse to the direction of motion MO). It should be noted that in the exemplary embodiment, the width WI of the illumination line can be several millimeters or less, but can be narrower or wider depending on the resolution of the scan. The length is determined by the corresponding length of the light source LI and the cylindrical lens 1650. The cylindrical lens is placed away from the illumination source LI by an enclosed lens holder 1640 to provide the desired focal length between the illumination source LI and the surface 1610 of the object. In the exemplary embodiment, cylindrical lens 1650 may be defined as a semi-cylindrical lens separated by a lens holder 1640 at a distance that focuses on a line in the surface. As shown, the off-axis projection of the light is a large fraction of the emitted light 1652 (projected as a surface or fan illustrated in FIG. 17) image sensor optics (eg, lens aperture) LA and / or some external Reflect outside the knife edge structure (line 1654). Light 1656 reflected and received by the inclined surface is received by the line scanning camera LSC as shown. In the exemplary embodiment, another cylindrical lens 1660 placed at the end of the enclosed lens holder 1670 focuses the received light onto the camera optics (knife edge structure LA) and the line scan sensor LS. Various camera optical system arrangements will be apparent to those skilled in the art in addition to the illustrated cylindrical lens. As shown in FIG. 16, a polarizer PI4 can be provided in the optical path of the illumination source LI (at various positions along the optical path). Similarly, a polarizer P3 can be provided in the optical path received by the sensor LS. These members are also provided, but are not shown in the arrangement 1700 of FIG. 17 for ease of viewing.

  Note that although cylindrical lens shapes are used, various cross-sectional shapes, such as paraboloids, can be employed in alternative arrangements. Similarly, a mirror can be used instead of or in addition to the lens to focus the illumination light. Advantageously, the illumination arrangement ensures that the entire surface consistently achieves high brightness and that each scanned line fully represents the local slope of the surface. This arrangement also advantageously allows any size surface to be imaged and analyzed for indentations, hills and waveforms. For example, a tablet or laptop screen, or a large thin flat panel television can be analyzed by providing a sufficiently long line lighting assembly and one or more line scanning cameras across the object surface. Each camera can image a portion of the entire object and provide a separate or stitched evaluation of the surface.

  It was also noted that it is clearly envisioned that a large area of the object can be imaged in an alternative embodiment using an illuminator in conjunction with, for example, a Fresnel lens or other optical arrangement.

        VI. Line, disc and ring optics mask

  FIG. 18 shows a schematic diagram of an exemplary embodiment of a generalized vision system arrangement 1800 including a vision system camera assembly 1810 with an image sensor 1820 and an optics assembly 1830. Sensor 1820 is interconnected with a vision system process (processor) in a manner generally described above (as shown) to perform the appropriate vision system tasks on the images acquired by sensor 1820. The optics assembly 1830 may be any acceptable variable or fixed focus and / or variable or fixed aperture lens unit, or a combination of lens units, such as a conventional M12 base, F mount or C mount lens.

  According to an exemplary embodiment, the front surface 1832 of the optics / lens assembly 1830 can be covered with a fixed or movable mask assembly 1840. Mask assembly 1840 may be screwed or snapped, or may be assembled via a bracket (not shown) in front of lens assembly 1830. The mask assembly 1840 can also be applied directly to the front surface (or other surface) of the lens assembly as an adhesive applique or coating. In the case of a threaded attachment, the mask assembly 1840 can operate like other conventional filters for use with various lens configurations and can be adapted to attach to a conventional lens filter mount.

  Optionally, mask assembly 1840 includes: It can be provided manually or automatically (eg, via a solenoid, servo, stepper, etc.) in or out of the optical path of the lens as desired. The mask assembly may be an electro-optic mechanism (for example) and can be varied between a fully transparent pattern and a partially opaque pattern of desired size and shape via an optional control circuit 1850. As a non-limiting example, the mask assembly 1840 can have a window (typically circular), which includes an LCD shutter, or other type of configurable window.

  Arrangement 1800 includes an illuminator 1860 as described above, which is directed to project light at an angle (not shown) that is not perpendicular to the entire surface 1870. In this example, the surface 1870 is corrugated comprising a series of hills 1872 along at least one direction and a recess 1874 interposed therebetween. The oblique light hits the hill and the recess and is scattered, and a part of the light enters the camera optical system assembly 1830. Mask assembly 1840 is defined as a knife edge member in its various forms and attenuates much of the scattered light to direct only light in a predetermined limited angular range to sensor 1820. In this embodiment, the mask cover / coating is represented by dashed line 1880, which includes a central covered area 1882 and an outer covered area 1884, with the central covered area 1882 and the outer covered area. Between areas 1884 is an open aperture through which light rays 1890 reflected from surface 1870 pass. In various embodiments, the polarizer PI5 is provided in conjunction with the illuminator 1860, and the corresponding polarizer P4 can be provided in conjunction with the optics / lens assembly. The polarizer can be arranged and function as described above (see, for example, FIG. 5).

  FIG. 19 shows a more detailed example of a vision system arrangement 1900 according to an exemplary embodiment. This embodiment includes a beam splitter and polarizer configuration similar to that shown and described above in FIG. Specifically, the arrangement 1900 includes a camera assembly 1910 and a lens / optical system 1920. The lens / optical system 1920 includes a mask assembly 1930 according to embodiments herein. In front of the mask assembly 1940 is a polarizer P5, which operates according to the principles described above. A beam splitter 1950 is provided through which light reflected from the object 1960 under inspection is transmitted to the camera 1910. A lighting assembly 1970 is provided. The illumination assembly 1970 includes an illumination source 1972 and a condenser lens 1974. The polarizer PI6 is placed in front of the condenser lens. Note that the polarizer P5 can include a mask pattern on its surface, and the assembly can be provided as a screw-type or snap-on attachment to the front surface of the lens 1920.

  Another vision system arrangement 2000 according to an exemplary embodiment. The arrangement 2000 includes a camera assembly 2010 and a lens / optical system 2020. The lens / optical system 2020 includes a mask assembly 2030 according to embodiments herein. In front of the mask assembly 2040 is a polarizer P6 which operates according to the principles described above. A beam splitter 2050 is provided through which light reflected from the object 2060 under inspection is transmitted to the camera 2010. In this embodiment, the condenser lens 2070 is placed between the beam splitter 2050 and the object 2060. The concentrator operates in conjunction with the illumination assembly 2080, which includes an illumination source 2082, a focusing lens 2084, and a polarizer PI7. It should be noted that the focusing lens 2084, the condenser lens 2070, and other optical components can be dimensioned and arranged according to known optical principles that will be apparent to those skilled in the art.

  The central cover area and the outer cover area of the various mask assemblies described above can be a variety of geometric shapes, sizes and relationships. Selection of an appropriate mask can be based on experience or by trial and error to obtain the best image for a given surface under inspection. This is illustrated in more detail in FIGS. 21-27, which present various types / sizes of mask patterns.

  Referring to FIG. 21, one type of mask 2100 that produces knife edge members defines a central opaque line 2110 that is centrally located within a transparent linear aperture 2120. The remaining outer area 2130 of the mask surrounding the transparent aperture 2120 is opaque. Lines 2110, 2120 are generally oriented parallel to the characteristic corrugated extension direction (if any) of the surface, and this type of mask is most effective in such conditions. More generally, the direction of extension is chosen (eg, by rotating the mask) to enhance or suppress the desired surface features. To better understand the function of the arrangement, to give a non-limiting example, the width of the aperture WLA is variable, for example 5-10 mm for a lens with a diameter D0 of 50-55 mm, The opaque line WL is 1 to 5 millimeters. In general, the line width WL is dimensioned to match the width of the spot focused from the illumination. Note that the following mask configurations (FIGS. 22-27) assume a similar lens diameter D0. The overall dimensions can vary in proportion to the size of the lens diameter.

  FIG. 22 shows a mask 2200 consisting of a central opaque circular (hidden) disk 2210 of diameter DD (5-10 millimeters). This disc provides the desired knife edge member for the arrangement. In general, the size of the disc is chosen to match the size of the surface features (eg, defects) to be enhanced or suppressed. Note that this exemplary mask arrangement 2200 is transparent with no outer opaque area at the lens edge (dashed circle 2230). This basic knife edge member allows light to be received within a predetermined angular range from hills and depressions that may be oriented in various directions on the surface.

  FIG. 23 shows a mask 2300 defining a central opaque (hiding) disk 2310 having a diameter DD1 (about 9 millimeters) and an outer opaque annular area 2330 having an inner diameter DA (about 14 millimeters). The difference between the disc diameter DD1 and the outer area 2330 creates a transparent annular window 2320 through which light reflected from the surface can pass. In particular, the diameter of the central concealed disk defines the attenuation of light in the manner of a knife edge member, and the diameter of the outer annular area defines the confocal effect of the optical system to increase the resolution.

  Further examples of mask arrangements 2400, 2500, 2600 and 2700 having a central concealment disk and an outer annular area and defining an annular aperture therebetween are shown in FIGS. 24, 25, 26 and 2, respectively. 27. As a non-limiting example, the disk 2400 disk diameter DD2 of the mask is about 5-6 millimeters and the inner diameter DA1 of the outer annular section is about 8-9 millimeters. The mask 2500 has a disk diameter DD3 of about 3-4 millimeters and an outer annular section inner diameter DA2 of about 5-6 millimeters. The mask 2600 has a disk diameter DD4 of about 3-4 millimeters, and an outer annular section inner diameter DA3 of about 8-9 millimeters. In addition, the disk diameter DD5 of the mask 2700 is about 5-6 millimeters and the inner diameter DA4 of the outer annular section is about 10-12 millimeters. These dimensions are merely examples of the widest possible dimensions and can be adjusted depending on the individual characteristics of the surface placed under the inspection angle and the brightness and / or wavelength of the illumination in the vision system arrangement.

  In general, as described above, the mask is formed into a filter by applying a coating having an appropriate pattern using various techniques (for example, screen printing, photolithography, application of a transparent coating on which a pattern is printed or formed). The glass surface can be configured. Those skilled in the art will appreciate that a variety of techniques can be used to apply the fixed mask pattern to the camera optics. Also as described above, the mask can be an active component that includes a surface, eg, represented by pixels. To generate the desired shape and size of the mask pattern, a controller that is separate from or part of the vision system processor addresses the individual pixels of the active mask. In particular, the controller can be adapted to go through various configurations until a user or automated vision system process determines the best pattern settings (eg, based on contrast). The pattern can have a more complex shape that is similar in shape to that shown in FIGS. 21-27 or that matches a unique surface feature and / or corrugated pattern.

  Note that in some embodiments, multiple cameras interconnected with one or more vision system processors can be used. Each camera can acquire images of the object surface from similar or different angles using different sized and / or configured masks (eg, different sized concealment disks), can analyze multiple images of the surface, It can be ensured that waveform features of shape and / or orientation are properly imaged. Similarly, when the mask is variable (a different mask is placed on the front surface of the optical system or the mask pattern is changed), a plurality of images can be acquired and analyzed.

  Referring to image 2800 of FIG. 28, a conventional touch screen of a handheld device is shown imaged using a mask according to the embodiments described above. Although this image can clearly identify the waveform of the surface, it appears as a relatively flat featureless surface to the naked eye or, more commonly, to a vision system arrangement. In FIG. 29, the image 2900 is not more generally visible, and in this example details of the touchscreen sensor matrix / array 2910 are shown. Further, the level of detail that can be achieved using the mask and imaging techniques described in this document is shown in the example image of FIG. 30, where the individual wires 3010 of the array 2910 of FIG. Can be clearly identified.

        VII. Conclusion

  It will be apparent that the systems and methods described above provide an effective technique for identifying slope defects, including hill and valley defects and ripple / waveform defects on a variety of multilayer and single-layer mirrors. By applying appropriate wavelengths of illumination light and filters (eg, various polarizers), the system and method can effectively image surfaces with various coatings and layers. Desirably, the exemplary knife edge configuration identifies the slope of the defect (first derivative) and makes the light reflected from the hills or valleys appear brighter or darker than the background depending on which side of the defect it is. it can. The size of the defect is potentially proportional to the slope of the defect. A small defect will have a small slope, which will deflect a small amount of illumination light from the background. If the spatial extent of the light source is small, it is possible to bring it to a small focal point after reflection from the surface being inspected, and it becomes easy to block the background without blocking defect light. However, increasing the light source reduces the negative effects of surface tilt random testing that may occur in the production environment, but at the cost of reduced defect contrast. Thus, the knife edge desirably enhances contrast by blocking background light. In addition, by using an exemplary combination of slope, shape and polarization detection, most of the background light is reflected and removed from the camera aperture, while light from tilted defects is in high contrast in the camera. Focused. Furthermore, the exemplary arrangement allows the size of the mirror surface to be widely varied, typically using a line scan camera and a focused illumination line. The embodiments described herein also provide a mask that includes a knife edge member and other members (e.g., confocal members) that allow for very fine display of a particular surface waveform.

  The exemplary embodiments of the present invention have been described in detail above. Various modifications and additions can be made without departing from the spirit and scope of the invention. The features of each of the various embodiments described above may be combined with the features of other described embodiments as long as they are suitable to provide a combination of multiple features in the related new embodiment. Furthermore, while a number of separate embodiments of the apparatus and method of the present invention have been described above, what has been described is merely illustrative of the application of the principles of the present invention. For example, as used herein, the term “process” and / or “processor” refers to a wide variety of functions and components based on electronic hardware and / or software (or functional “modules” or “elements”). Should be construed as including). Further, the illustrated process or processor may be combined with other processes and / or processors or divided into various sub-processes or sub-processors. Such sub-processes and / or sub-processors can be variously combined according to the embodiments described herein. Similarly, any functions, processes, and / or processors herein may be implemented using electronic hardware, software, or a combination of hardware and software comprising non-transitory computer readable media of program instructions. Assumed. Further, terms used in the present specification to represent various directions and / or orientations, such as “vertical”, “horizontal”, “top”, “bottom”, “bottom”, “top”, “side” , "Front", "Rear", "Left", "Right" and the like are only used as relative expressions and are based on a fixed coordinate system such as the direction of gravity action. Does not represent an absolute orientation. In addition, when the term “substantially” or “approximately” is used with respect to a given measurement, value or characteristic, it is an amount that is within the normal operating range to achieve the desired result. , But includes some variation due to inherent inaccuracies and errors within the tolerance of the system (eg, 1-5 percent). Accordingly, this description is by way of example only and is not meant to otherwise limit the scope of the present invention.

The claims are described below.

Claims (26)

A vision system camera, having an image sensor and an optical system, defining an optical axis and oriented to image a surface;
An illuminator assembly that projects structured light onto the surface at a predetermined angle non-parallel to the optical axis;
A knife edge member associated with the optical system that variably blocks a portion of the maximum field of view of the optical system;
Wherein each knife edge member and the predetermined angle described above, the light reflected to the image sensor is transferred from the inclined lands feature on substantially the surface valleys through the optical system, the inclined lands The reflected light from the area surrounding the valley is set to be blocked by the knife edge member ,
A system for imaging defects on an object's specular surface. The knife edge member comprises a variable aperture in the optical system, the system according to claim 1.   The system of claim 1, wherein the predetermined angle is associated with an inclination of the hill and valley. Wherein the image sensor is a 2D sensor, the object is stationary relative to the camera system of claim 1. Wherein the image sensor is defined as a line scan camera arrangement, the object is moving relative to the camera system of claim 1. The system of claim 5, wherein the illuminator assembly projects a line of illumination onto a surface. The system of claim 6, wherein the illumination is defined as a wavelength range of substantially infrared light or near infrared light. The system of claim 7, wherein the object is defined as a layer comprising an anti-reflective coating. It said layers comprising a polarizing layer, the illumination is polarized illuminator optical system includes a polarizing filter system of claim 8,. The object is AMOLED display, the polarizing layer is 1 / 4.lamda retarder, the polarization filter is defined as cross polarized filters, according to claim 9 system. The system of claim 6, wherein the illuminator includes a polarizer for polarized illumination, and the illuminator optics includes a polarizing filter. The system of claim 1, wherein the illumination source is defined as a focused beam that converges toward a point in the vicinity of the knife edge member . The knife edge member is defined as an external structure that is positioned in front of the optical path of the optical system, according to claim 1 system. The illuminator assembly projects a light through a beam splitter located on the optical axis of the vision system camera, axis illumination from the illuminator assembly is projected toward the object surface which is coincident with the optical axis by the beam splitter, The system of claim 1. The illuminator assembly is defined as a plurality of illumination sources, each projecting light into a respective beam splitter, each beam splitter being on the optical axis of the vision system camera, and off-axis illumination coming from each illumination source is a beam is projected toward the object surface, respectively aligned with the optical axis by the splitter system according to claim 1. 6. The system of claim 5, wherein an imaging lens is defined as a lens that images a line, and the illuminator assembly projects onto a surface and then focuses on a line that does not enter the imaging lens's incident aperture after reflection. The system of claim 16, wherein the illuminator assembly includes a cylindrical lens for focusing on a line. The feature is defined as a waveform in a region of a surface, and further comprises an analysis and evaluation process that determines a distribution of pixel brightness values in an image acquired by the image sensor and compares the distribution to a threshold value. System.   The system of claim 18, wherein the distribution is defined by at least one histogram contrasting pixel brightness values and frequency in the image. The knife edge member is defined as hidden structure placed on the optical axis in the optical system, hidden structure is on the mask element provided adjacent to the front surface of the optical system, hidden structure The system of claim 1, wherein the system is arranged to selectively enhance or suppress scattered light associated with the feature. The concealed structure is defined as a line extending across the extension direction of the optical system, has a width transverse to the direction of extension focus is the relative size of the illumination spot combined on the optical system, 21. The system of claim 20, wherein the direction of extension is defined by the feature orientation. The system of claim 21, wherein the mask element includes a surrounding opaque area on each opposite side of the surrounding line and a linear aperture between the line and the opaque area. 21. The system of claim 20, wherein the concealment structure includes a circular disk centered approximately on the optical axis and having a size and a relative diameter of one or more features. 24. The system of claim 23, further comprising an annular area surrounding the circular disk , defining an annular aperture therebetween, the annular area being arranged to suppress scattered light. The mask element is defined as a lens cover of the snap-on or screw-on applique is placed on the front surface of the optical system, variable pattern electrooptic mechanism is placed on the optical system, according to claim 20 system. 21. The system of claim 20, further comprising a first polarizer placed in association with the optical system and a second polarizer placed in association with an illuminator.

Images