DE102014107143B4 - System and method for measuring the displacement of an object surface - Google Patents

Abstract

A system for measuring the displacement of an object surface (144) comprising:
- a displacement sensor (110) projecting a line (156) onto the object surface (144) and receiving light from the projected line (156) on an imager (164) to define laser line position data (1210C) and laser line intensity data (1220C) ; and
- a vision system processor (180) constructed and arranged to generate a height image (1010) from the laser line position data (1210C), a gray scale image (1110) from the laser line position data (1210C) and the laser line intensity data (1220C), and an estimate of a vertical grayscale image gradients, the grayscale image gradient being used to calculate a height correction factor to be applied to each pixel of the height image (1010).

Description

FIELD OF THE INVENTION

This invention relates to machine vision systems used to determine the displacement of an imaged object surface in three dimensions (3D).

BACKGROUND OF THE INVENTION

In manufacturing and assembly processes, it is often desirable to measure some or all of the object surfaces with a high degree of accuracy and map the overall displacement or "profile" (e.g., height in the physical Z coordinate direction) with respect to various locations to create the object surface. This profile can be defined with the help of a machine vision system (also called “vision system” here) in the form of a laser displacement sensor (also called laser beam “profiler”). A laser displacement sensor detects and defines the (three-dimensional) profile of a scanned object surface with the help of a planar curtain, which is formed by the optical propagation of a laser beam in a "fan" across the propagation path. In a conventional arrangement, the vision system camera device is oriented in such a way that it sees the beam plane from outside the plane. This arrangement captures the profile of the projected line (which extends e.g. along the physical X-axis) on the object surface, corresponding to the baseline (i.e. the relative distance along the Y-axis) between the rays (fan) plane and the camera causes the imaged line to appear as a variation in the direction of the image Y-axis as a function of the Z-axis physical height of the imaged point (along the image X-axis). In a typical arrangement, the optical axis of the camera intersects the laser plane at an acute angle and the well-known Scheimpflug configuration of the laser plane, camera lens and camera lens sensor can be used to form an image in which the laser beam deviation travels through the area of interest in the The focus is in spite of the changing distance from the camera. This deviation represents the elevation profile of the surface. Laser displacement sensors are useful in a wide range of inspection and manufacturing operations where the user wishes to triangulate and characterize surface details of a scanned object. One embodiment of the laser displacement sensor uses a vision system camera with a lens assembly and an imaging sensor (or "imager") that may be based on an SSD or CMOS design. The imager defines a predetermined field of grayscale or perceived color pixels on an image plane that receives focused light from an imaged scene through a lens.

In a typical arrangement, the displacement sensor and / or the object are in relative motion (usually in the direction of the physical Y coordinate) so that the object surface is scanned by the camera and a sequence of images of the laser line is captured at desired spatial intervals - typically in conjunction with an encoder or other motion measuring device (or alternatively at time-based intervals). Each of these individual profile lines is typically derived from a single captured image of a larger field of view known to include the projected line. These lines together describe the surface of the object shown. The field of view is characterized by a working distance - that is, the surface of the object should be between a maximum and a minimum height in order to approximately capture the profile information. Within this working distance, the size and shape of the line can change due to a variety of factors including the orientation and reflectivity of the surface to be scanned, the changing thickness of the laser plane (the line is typically at a "waist" at a about mean depth from the sensor narrowest and wider both at and further away from the sensor) and the changing amount of magnification and perspective foreshortening in the camera system as a function of the height. This change in line size / geometry presents one of several challenges in obtaining an accurate measurement of a surface at a desired scan speed.

When measuring an object surface profile, it is sometimes desirable to simultaneously generate a grayscale image of the object. While such grayscale images can provide a view of the object, grayscale images can have different characteristics that can prevent a useful comparison between this and the derived object surface profile. For example, the grayscale image and other generated images can have different pixel gradations. In this regard, if an object in physical space is further away from the camera, e.g. B. at a lower height, then a pixel can appear wider and cover more physical space than an object that is closer to the camera. This effect can cause a distortion in the resulting grayscale image and a comparison between the grayscale image and other images taken during the process prevent or make them difficult. It is therefore desirable that the grayscale images have uniform pixel gradations, both within a single grayscale image and in separate grayscale images.

Machine vision applications often require the use of imaging components of the highest quality. Recently, the cost of imaging devices such as cameras has increased. As a result, market participants will either be forced to bear the ever increasing costs for components of the highest quality or be satisfied with lower quality imaging devices at lower costs. Such lower quality devices often include defects or imperfections that make them less desirable for use than higher quality devices. The use of such lower quality devices can potentially degrade the results of machine vision applications. In particular, sensors can often contain one or more bad measuring elements. Bad measuring elements can “jam” because they do not react visibly at all to the lighting, but rather display an approximately constant value. Other bad measuring elements can react to lighting, but one or more reaction parameters such as offset, gain, non-linearity or noise level differ significantly from the target reaction of the "good" elements.

Similar devices and methods for measuring a three-dimensional object surface by means of line triangulation are shown in FIG DE 10 2006 004 060 A1 , DE 10 2006 036 586 A1 , US 2004/0246473 A1 , DE 20 2012 104 890 U1 , US 2011/0310399 A1 , DE 10 2005 051 318 A1 and DE 10 2005 052 044 A1 described.

SUMMARY OF THE INVENTION

The problems described above are solved with a system according to the invention according to claim 1 and a method according to the invention according to claim 11. This invention overcomes the disadvantages of the prior art in an embodiment in which a laser displacement sensor system and corresponding methods for scanning the height profile of an object surface are provided, which improve the scanning pass, processing speed, efficiency and versatility of the system. In an embodiment not covered by the claims, a displacement sensor projects a line on the object surface and receives light from the projected line on an imager so that a plurality of displacement values are defined in a height direction. A vision system processor works on rows of imager pixels to define a laser line center, or more generally, the position of the laser line and / or specific parts of the line (e.g. the stop and start points of the line thickness) in columns of imager pixels in each of one Variety of areas of interest. Rows in zones outside of these areas of interest are typically not captured and / or processed, so that the overall speed of image capture and processing of the line is increased. Each region of interest defines a plurality of rows that correspond to the expected locations of the projected line on the object surface. A height image is generated from the positions of the projected laser line for each scanned image when the object moves relative to the imager's field of view. A GUI can be used to define the areas in which the user enters numerical (physical unit) or graphical boundaries for each area. In other embodiments, the system uses line width, intensity, and location to generate grayscale information (e.g., along each laser line of the scan) or full grayscale images with the imager and laser lighting. The grayscale image data and height image data can be used in conjunction with one another to generate correction factors that can compensate for incorrect height measurements caused by contrast. For example, imager pixels can also be transmitted to the processor in a “binary” manner by comparing each pixel intensity (“intensity” is defined as including, but not limited to a voltage value) in each column of the imager directly to a reference intensity. The pixels that match or exceed the reference intensity are given a logical value 1 (or another) and the pixels that fall below the threshold are given a logic value 0 (or another). The line “center” in each column is determined from a probable grouping of “1” pixels, and this position is passed on to the processor in order to form part of the height image. For example, the center can be a physical center of the line or any other suitable special location associated with the line, such as start and end points.

In an embodiment not covered by the claims, a system and method for measuring the displacement of an object surface includes a displacement sensor that projects a line onto the object surface and receives light from the projected line on an imager such that a plurality of displacement values in a height direction is defined. A vision system processor operates to process rows of imager pixels to determine a position of the projected line in columns of the imager pixels to be determined exclusively in each of a large number of areas of interest. Each of the regions of interest defines a plurality of rows of imager pixels, which correspond to the expected positions of the projected line on the object surface. A position of the position of the projected line in each of the columns corresponds to a physical displacement value on the object surface. For example, the object and the displacement sensor are in relative motion and a large number of the captured images are captured on the projected line. The captured images define a total height of the object surface. A graphical user interface (GUI) is operatively connected to the vision system processor. It enables a user to enter the boundaries of any of the areas of interest. The GUI provides inputs for numerical, physical units of the limits. For example, the GUI can show a captured image of the object with the projected line contained therein. This GUI is designed and arranged to allow the user to graphically position the boundaries. The captured image can be moved in the GUI in order to determine a range of displacements of the projected line during a scanning process. For example, the position of the projected line is approximately halfway along the column direction. In some embodiments, a measurement process analyzes captured image data at each of the plurality of areas of interest to generate information for use in a downstream processing task. The information that is used in the downstream processing task is generated from a combination of analyzed recorded data from a large number of areas of interest. The downstream processing task can use information from at least a single captured image of the object (a slice), a plurality of captured images of the object (a group of slices) and all captured images that correspond to the object (an overall height image). The downstream processing task may include a vision system task or tool with a bespoke or conventional design. The vision system task or tool can include at least one of the following: registration, review, alignment, and pattern matching. Other tasks / tools can include smoothing, averaging, other image processing operations, etc. that can be performed on a group of slices, for example. For example, the downstream processing task includes a decision making task. This decision-making task can be structured and arranged to perform at least one of the following processes: object rejection, line control, or alarm. The measurement process can use a single scan of the object or a predetermined number of scans in a group (such as smoothing, averaging, or other image processing processes).

In another embodiment not covered by the claims, a system and method for measuring displacement of an object surface includes a displacement sensor that projects a line onto an object surface and receives light from the projected line on an imager so that a plurality of displacement values in a Height direction is defined. A vision system processor operates to process rows of imager pixels to determine a location of the projected line in columns of the imager pixels. The vision system processor is constructed and arranged in such a way that it generates a height image of the object surface from the position of the projected line. The vision system processor is also constructed and arranged in such a way that it generates a grayscale image of the object surface from the intensity values of the projected line. The vision system processor is constructed and arranged to use measurements in the grayscale image to calculate correction factors and apply them to the height image so that errors in the height image are compensated for. The errors may include incorrect height readings in the height image, such as those that occur with a contrast change on the surface - for example where a printed character or symbol is provided in a contrasting level or color. For example, a measurement process uses (a) measurements of data from the height image to shape measurements of the grayscale image, or (b) measurements of the grayscale image to shape measurements of the height image. A decision-making process can also be provided that generates decisions based on the measurement process.

In a further embodiment not covered by the claims, a system for measuring the displacement of an object surface comprises a displacement sensor that projects a line onto the object surface and receives light from the projected line on an imager so that a plurality of displacement values are defined in a height direction . A threshold circuit receives intensity values from pixels of the imager and carries out a comparison of the respective intensity values with a reference intensity. Here, a binary value is assigned to each of the pixels on the basis of the comparison. A vision system processor operates to process rows of pixels to determine a location of the projected line in columns of the pixels having a binary value indicating the presence of the projected line, the vision system processor being constructed and arranged to provide a height image generated from the position of the projected line in the columns of the pixels. For example, the intensity is defined either as a voltage or as a current strength.

In a further embodiment not covered by the claims, a system for correcting a suspect pixel in a grayscale image comprises a processor and a memory including the instructions thereon which, when executed by the processor, cause the processor to generate laser line position data and corresponding laser line intensity data which are based in part on a captured image of an object, the laser line position data including at least one laser line position information. The processor can identify at least one laser line position information and replace it with the aid of the processor, an original value of the laser line intensity information corresponding to the identified laser line position information with a substitute value for the laser line intensity information. The processor can generate the grayscale image from the laser line position data and the corresponding replaced laser line intensity data.

In a further embodiment not covered by the claims, a system for generating a uniformly graded grayscale image corresponding to a height image comprises a processor and a memory including the instructions contained therein which, when executed by the processor, cause the processor to read laser line position data and Generate laser line intensity data, each pixel of the laser line intensity data including an index corresponding to a measured intensity. The processor may map each pixel of the laser line position data in a coordinate system representing physical space, with at least two of the mapped pixels having a connection function interposed therebetween, the connection function including indexing corresponding to the measured intensity. The processor can use the processor to calculate a representative position value and a representative indexing value for a plurality of bins, each of the plurality of bins corresponding to one or more of the mapped pixels and part of the connection function. The processor can output the uniformly graded grayscale image with the aid of the representative indexing values and the output of a height image with the aid of representative position values.

According to the invention, the system for measuring the displacement of an object surface comprises a displacement sensor which projects a line onto the object surface and receives light from the projected line on an imager so that the laser line position data and the laser line intensity data are defined. A vision system processor can be constructed and arranged to generate a height image from the laser line position data, a grayscale image from the laser line position data and the laser line intensity data, and an estimate of the grayscale image gradient, the grayscale image gradient being used to calculate a height correction factor that is applied to each pixel of the height image is applied.

Figure list

• 1 eine schematische perspektivische Darstellung eines Laserverschiebungssensorsystems ist, das ein Bild eines Objektes erfasst, wenn relative Bewegung dazwischen in einer Scanningrichtung auftritt, gemäß eines Ausführungsbeispiels;
• 2 ein schematisches Diagramm eines Teils einer Imagerpixelanordnung ist und Bereiche einschließt, die Teile einer beispielhaften abgebildeten Laserlinie enthalten, gemäß der Ausführungsform von 1;
• 3 ein schematisches Diagramm des Imager der 1 ist, die die Erzeugung eines Höhenbildes aus einem erfassten Bild einer Laserlinie innerhalb des Sichtfeldes des Verschiebungssensors zeigt;
• 4 ein Flussdiagramm eines Prozesses zur Verarbeitung von Höhenbilddaten innerhalb des interessierenden Bereichs des gesamten erfassten Bildes ist, gemäß eines Ausführungsbeispiels;
• 5 ein Diagramm eines Bildes einer Laserlinie ist, in der zwei interessierende Bereiche nebeneinander ohne Spalt dazwischen liegen;
• 6 eine Bildschirmanzeige eines grafischen Benutzerinterface (GUI) zum Einsatz beim Verschiebungssensorsystem der 1 ist, bei dem die interessierenden Bereiche zur Erkennung auf dem Gesamtbild des gescannten Objekts in Bezug zu physikalischen Einheiten gesetzt werden können;
• 7 ein Diagramm einer Bildschirmanzeige einschließlich eines GUI ist, bei dem der Anwender die interessierenden Bereiche und Ausschluß auf ein Bild des Objektes manipuliert, das vom Verschiebungssensor erfasst ist;
• 8 ein Diagramm von Teilen eines Imagers an zwei interessierenden Bereichen ist, das Pixelreihen zeigt, die selektiv an- oder abgeschaltet werden, um die Auflösung des Bildes in dem Bereich zu verändern;
• 8A ein Diagramm von Teilen eines Imager an zwei interessierenden Bereichen ist, bei dem der erste Bereich bei voller Auflösung ist, wobei alle Pixelreihen angeschaltet sind, während die zweite Reihe bei halber Auflösung ist, wobei jede zweite Pixelreihe geschaltet ist;
• 9 ein Diagramm eines Bildes eines beispielhaften Objektes ist, das sowohl höhenver-ändernde Funktionen als auch Änderungen im Kontrast enthält, von dem ein zeitgleiches Höhenbild und Graustufenbild mithilfe des Verschiebungssensorsystems der 1 erfasst werden kann;
• 10 ein Diagramm eines Höhenbildes des beispielhaften Objektes der 9 ist, das Änderung in der Höhe verschiedener Funktionen als Intensitätsmessung zeigt und auch kontrastbasierte, falsche Höhenmessungen;
• 11 ein Diagramm eines Graustufenbildes des beispielhaften Objekts der 9 ist, das Änderung im Kontrast von gewissen Funktionen als Änderung in der Graustufenintensität zeigt und auch Schattierungen von gewissen erhabenen Funktionen an deren Höhenübergängen zeigt;
• 12 ein Flussdiagramm eines Verfahrens zur Erzeugung eines Höhenbildes und eines Graustufenbildes aus Bilddaten ist, die mithilfe eines gescannten Bildes des Verschiebungssensors der 1 erfasst wurde;
• 12A ein Flussdiagramm eines Verfahrens zur Korrektur für null oder mehrere verdächtige Pixel in einer Reihe des Graustufenbildes ist, das möglicherweise verdächtige Pixel enthält;
• 12B ein Flussdiagramm eines Verfahrens zum Resampling der erfassten Bilder oder der Laserlinienpositionsdaten und Laserlinienintensitätsdaten ist, um ein Graustufenbild mit einer einheitlichen Pixelabstufung zu erzeugen;
• 12C ein Diagramm von Laserlinienpositionsdaten und Laserlinienintensitätsdaten ist;
• 12D ein Diagramm von abgebildeten Pixelwerten ist, die den Laserlinienpositionsdaten entsprechen;
• 12E ein Diagramm von abgebildeten Pixelwerten ist, die die Verbindungsfunktionen einschließen;
• 12F ein Diagramm von abgebildeten Pixelwerten ist, die repräsentative Positionswerte und repräsentative Indizierungswerte einschliessen;
• 12G ein Diagramm eines Teils des kalibrierten Höhenbildes und des einheitlich abgestuften Graustufenbildes ist;
• 13 ein Flussdiagramm ist, das die Kompensation für die kontrastbasierten, falschen Höhenauslesungen im Höhenbild mithilfe eines Vergleichs zwischen den Funktionen des Höhenbildes gegen entsprechende Funktionen im Graustufenbild zeigt; und
• 14 ein Blockdiagramm einer binären Technik zur Festlegung einer Mitte der Laserlinie in jeder Spalte durch Vergleich einer Intensität jedes Spaltenpixel mit einer Referenzintensität ist.

The description of the invention below refers to the accompanying drawings, wherein:

• 1 Figure 12 is a schematic perspective illustration of a laser displacement sensor system that captures an image of an object when there is relative movement therebetween in a scanning direction, according to an embodiment;
• 2 FIG. 13 is a schematic diagram of a portion of an imager pixel array including areas containing portions of an exemplary imaged laser line, in accordance with the embodiment of FIG 1 ;
• 3 a schematic diagram of the imager of the 1 Figure 13 shows the generation of a height image from a captured image of a laser line within the field of view of the displacement sensor;
• 4th Figure 12 is a flow diagram of a process for processing height image data within the region of interest of the entire captured image, according to one embodiment;
• 5 Figure 12 is a diagram of an image of a laser line in which two regions of interest are adjacent with no gap therebetween;
• 6th a screen display of a graphical user interface (GUI) for use with the displacement sensor system of 1 is, in which the areas of interest for recognition on the overall image of the scanned object can be related to physical units;
• 7th Figure 13 is a diagram of a screen display including a GUI in which the user manipulates the areas of interest and exclusions on an image of the object detected by the displacement sensor;
• 8th Figure 13 is a diagram of parts of an imager at two areas of interest showing rows of pixels being selectively turned on or off to change the resolution of the image in the area;
• 8A Figure 13 is a diagram of parts of an imager at two areas of interest with the first area at full resolution with all rows of pixels turned on while the second row is at half resolution with every other row of pixels turned on;
• 9 FIG. 13 is a diagram of an image of an exemplary object containing both height-changing functions and changes in contrast, of which a simultaneous height image and grayscale image using the displacement sensor system of FIG 1 can be captured;
• 10 a diagram of a height image of the exemplary object of FIG 9 which shows change in height of various functions as an intensity measurement and also contrast-based, incorrect height measurements;
• 11 FIG. 3 is a diagram of a grayscale image of the exemplary object of FIG 9 is showing change in contrast of certain functions as a change in grayscale intensity and also showing shades of certain raised functions at their height transitions;
• 12th FIG. 12 is a flow diagram of a method for generating a height image and a grayscale image from image data obtained using a scanned image of the displacement sensor of FIG 1 was recorded;
• 12A Figure 3 is a flow diagram of a method of correcting for zero or more suspect pixels in a row of the grayscale image that may contain suspect pixels;
• 12B Figure 3 is a flow diagram of a method for resampling the captured images or laser line position data and laser line intensity data to produce a grayscale image having a uniform pixel gradation;
• 12C Fig. 13 is a diagram of laser line position data and laser line intensity data;
• 12D Figure 12 is a diagram of mapped pixel values corresponding to laser line position data;
• 12E Figure 3 is a diagram of mapped pixel values including the link functions;
• 12F Figure 3 is a diagram of mapped pixel values including representative position values and representative indexing values;
• 12G Figure 13 is a diagram of a portion of the calibrated height image and the uniformly graded grayscale image;
• 13th Figure 12 is a flow chart showing compensation for the contrast-based false height readings in the height image by comparing the functions of the height image against corresponding functions in the grayscale image; and
• 14th Figure 13 is a block diagram of a binary technique for determining a center of the laser line in each column by comparing an intensity of each column pixel with a reference intensity.

PRECISE DESCRIPTION

System overview

1 Figure 3 shows a vision system arrangement 100 including a laser displacement sensor assembly 110 that is oriented so that it is an object 120 (also sometimes referred to as “part”). The displacement sensor assembly (or simply "displacement sensor") can be in a single housing 112 that is mounted at a suitable location in relation to the depicted scene. In other embodiments, the displacement sensor can comprise individual, separate sub-elements. The object 120 and the displacement sensor 110 are in relative movement (double arrow My), with either the displacement sensor 110 , the object 120 or move both along at least one axis (the direction of scan motion). The direction of movement is often based on the relative (e.g. global) coordinate system shown 124 aligned (in this example the physical y-axis direction). In a typical arrangement, the object is in place 120 on a moving surface 130 (shown in phantom) such as a conveyor belt or moving platform that provides motion data from a decoder or similar device that is operative with the drive system 132 the moving surface in a manner that is clear to those skilled in the art. The object 120 is for example a structure with a surrounding surface 140 that is relatively planar, with a height H1 , and it extends to an outer perimeter 142 . Within the exemplary outer perimeter 142 there is a surface 144 which when shifted along the physical z-axis direction between a height H2 and a height H3 changes. It is noted that any height H1 , H2 and H3 in physical quantities, such as millimeters from a ground plane, which is the plane of the moving surface 130 or another reference plane can be described. It must also be noted that this object shown 120 represents only one example of a wide range of possible surface shapes, and that this is described as an application example of the principles of the exemplary embodiment of the system and method. In other examples, the outer, surrounding surface can define a more complex perimeter with additional functions, such as mounting holes, and the internal structure can have a more variable shape. An outer, surrounding surface can also be completely or partially omitted in the case of an exemplary object.

The displacement sensor assembly 110 includes a laser assembly 150 one that is a lens or other optical arrangement 152 includes (such as a Powell lens or a cylindrical lens) that has a curtain or fan 154 of laser light so on the object 120 that projected a line 156 forms along the physical x-axis (by and large transverse to the scanning movement direction My). The subjects 154 is generally in a plane which, in this embodiment, is approximately (or substantially) perpendicular to a reference plane, such as the physical xy plane of the movable surface 130 and / or a plane of the surrounding surface 140 of the property. The displacement sensor 110 also includes a camera assembly (dashed box 160 ), which generally consists of a lens / optic assembly, for example in a well-known Scheimpflug design 162 and an imager assembly ("Imager") 164. The imager 164 includes an array of photosensitive pixels having a predetermined resolution and the camera array generally defines an optical axis OA falling within the field of view where the line is 156 is visible on the object at different surface heights. The optical axis OA defines a non-parallel (e.g. acute) angle □ with respect to the plane (and the line connected to it 170 ) of the laser fan 154 . So any change / shift in the height of the physical z-axis in the line along the physical x-axis is made by the camera arrangement 160 as a change in position (along the image axis YI) in the resulting line 171 of a displayed two-dimensional (2D) image 172 , as generally in the example (and optional) ad 174 shown, which can be part of a computer (e.g. hand-held device, laptop, tablet or PC). As described below, the computer can interact with hardware and / or software, which is a graphical user interface process 176 and includes a device setup process, each of which enables the user with the vision system processor 180 to interact with the various functions and parameters of the displacement sensor 110 set up, monitor and control.

It must be noted that the illustrated arrangement of the axes (x, y, z) 124 is a convention and other representations of the orientation to one another (e.g. polar coordinates) are expressly contemplated. As shown, rotation about a given axis is also represented, e.g. B. by the curved double arrow Rx, which shows the rotation around the physical X-axis.

The imager 164 is effective with a vision system processor 180 connected, which is in the housing 112 of the displacement sensor 110 in this embodiment is located. In alternative exemplary embodiments, some or all of the components of the vision system processors and / or processes can be located outside the housing (e.g. on a connected PC). The vision system processor 180 conducts a displacement sensor process 182 in accordance with the illustration example, which is described in more detail below. The processor 180 also receives relative motion signals from the moving surface 130 or another device that causes movement (e.g. a portal or a manipulator that carries the displacement sensor and / or the object). Captured images are captured in conjunction with such motion signals so that the line 156 is imaged at a selected object positioning in the scene. As used herein, the term captured images refers to a collection of pixel values on the imager. As also used herein and described in more detail below, the term height image (or area image) refers to the collection of height pixel values generated from one or more captured images, and the term grayscale image refers to the collection of grayscale pixel values that is generated from one or more captured images.

As will also be described further below, the image data (i.e. a 2D height image - often also referred to as an “area image”) for the objects are transferred to downstream data processing devices and processes (190 in 1 ) in order to use them in various production processes, such as quality control, object manipulation, assembly, etc. Users typically require measurements that are in physical units are expressed (e.g. millimeters or micrometers in x and y and z) rather than image units (i.e. pixels / pels in x, motion-based triggers from encoders in y, and row shifts in z). For this reason, displacement sensors are typically calibrated to convert image pixel outputs into engineering units so that the output of the calibrated displacement sensor appears in engineering units. Calibration of the displacement sensor can be done by the manufacturer so that the displacement sensor provides output in engineering units (such as the model DS1100 available from Cognex Corporation of Natick, Massachusetts, USA). Alternatively, sensors can provide outputs in uncalibrated imaging units. If the output appears in uncalibrated image units, the user or the application is requested to process input data in image-pixel units. The user may also designate the desired height units in terms of height image pixel level (in engineering units) and height image pixel level level (in terms of how many engineering units each pixel level represents) when the height image uses integral pixel values. In another example, the height image can be displayed as a 2D arrangement of the flowing point values.

It should also be noted that the terms "process" and / or "processor" as used herein should be interpreted broadly to encompass a range of functions and components that rely on electronic hardware and / or software, lock in. In addition, a process or processor shown can be combined with other processes and / or processors or divided into different sub-processes or processors. Such sub-processes and / or sub-processors can be combined in various ways according to the exemplary embodiments here. It is also expressly contemplated that every function, every process and / or processor that is mentioned here, implemented here with the help of electronic hardware, software consisting of a non-transitory, computer-readable medium of the program instructions, or a combination of hardware and software can be.

The sensor process 182 includes calibration values introduced as a means of mapping the location of the line within the field of view of the imager to a physical elevation (e.g., H1, H2, H3). The calibration process can employ a calibration object that moves along the z-axis via a motion platform or other means of transport to precise, known heights, and the relative y-position (i.e. Y1 in the 2D representation) of the imaged line at this physical height is used to create a curve or table of values relating each y-location to a given physical height. Each y-position in each column (as noted below, there can be many y-values per column) is mapped to a physical height, and the same y-position in different columns can map to a different physical height based on calibration parameters and the like is based. Alternatively, values can be generated in engineering units using a parametric model of the relationship between pixel locations and the x, z locations. For example, the sensor process determines 182 the position of the line (e.g. the center or another selected position / positions, such as the start and end position of the recognized line thickness) and then uses this value to determine the physical height of this part of the object surface. More precisely and with reference to the diagram of 2 becomes part of the imager 164 shown. The imager consists of an M row X N column array of photosensitive elements, commonly referred to as "pixels" 210. These pixels each generate a charge based on exposure to incident light. This charge is sent through an analog-to-digital converter (not shown) where it is converted into a digital, numerical value related to the charge level. This value is used as image pixel data for use by the processor 180 saved. In one example that is not limiting, a pixel array of 768 rows X 1025 columns can be used. As another non-limiting example, a field of view of approximately 175 mm is provided with a vertically operating area (WR) extending (along 170 from the laser 150 ) varies between about 140 mm (WRmin) and 315 mm (WRmax). These dimensions are only one example of a wide range of effective parameters and are provided through further descriptions of the functionality here.

The illustrated part of the pixel arrangement contains segments 220 , 230 the laser line shown, where the height merges between the lower and higher surfaces 140 and 144 . That is, the pixels in the array that are exposed to the light of the line will be within the dashed lines of the segments 220 , 230 shown. It is noted that some rows fall within each line segment. Typically the vision sensor process calculates 182 a line center in each column. This line center can be resolved into a sub-pixel plane based on a number of factors and calculations known to those skilled in the art. For example, a resolution of 1/16 of a pixel can be obtained with appropriate algorithms and the data from one or more scanning processes.

Each scan creates a line of a height or area image. As in 3 the captured image is shown 310 of the field of view, which is a laser line 312 contains, processed to an N-wide field 320 of values 322 to generate a physical height (z-height, e.g. in mm) for each pixel column ( 0 .... N-1) of the captured image. While in some embodiments, at this point in time the physical height for each value 322 can be generated, the present embodiment postpones the generation of physical height to another, downstream process step. In the present embodiment, the position of the laser line is generated in sub-pixel resolution in pixel units and in a subsequent step calibration parameters are applied to the pixel units in order to generate a set of x, z values (i.e. the scan line for physical units). It is noted that there may be more or fewer (x, z) pairs in the physical unit scan line than in the line position vector (in pixels) generated by the imager because the scan line can again be mapped in physical units, such as desired to include a finer / coarser sample. The number of values ( 322 ) per column along the X direction so that more than one value per column is generated. For example, the start and end positions of the laser line can be provided in the x direction, as well as the center or other dimensions, such as a standard deviation. In general, the field represents 320 represents a height image consisting of height values for each x-coordinate (column) over the scanned object surface. When the object is scanned, a series of successive fields can be combined (with knowledge of the y-directional movement between scans based on the encoder) to derive an overall profile of the object surface.

When generating each height image ( 320 ) the process reduces noise and improves the accuracy in finding the line center position (and thus the height value) by applying a filter algorithm to a perimeter of rows. A variety of conventional and / or new filters can be used. In one embodiment, a line search algorithm runs on the result of applying a symmetrical 1-2-4-2-1 vertical filter kernel to the captured image for approximately every line for which a measurement was made, that is, for each row in the captured image, pixel values are present Row multiplied by 1, the pixel values of the previous row are multiplied by 2, the row before it is multiplied by 4, the row before it is multiplied by 2, and the row before it is multiplied by 1. The filter output is the sum of the five products above. The choice of the 1-2-4-2-1 filter is odd so that the midpoint of the application of the filter as a result of the application of the filter may correspond to the same location. Other filters can be used depending on the user's preference, project parameters, or other circumstances. For example, such other filters may include different widths and / or different coefficients, such as 1-2-4-6-4-2-1 or 1-3-3-1, etc. The filter values in the rows above, at and below the maximum filter response position are used to determine line position with sub-pixel accuracy using quadratic interpolation (or other known techniques). If there is a plateau in the filter response then the center of the plateau is used instead of the peak position.

Particular attention should be paid to the rows that are near the beginning and end of the picture. The system does not generate a filter measurement for the first and last lines. For the second and penultimate lines, the system approximates the desired 1-2-4-2-1 kernel with a 3-4-2-1 or a 1-2-4-3 kernel. In one embodiment, the line search process can specifically employ the following filters relative to different locations in the area of interest (also described below) of the image: Row number (in the analyzed area) Applied filter 0 no filter measurement carried out 1 3-4-2-1 2, 3, ... 1-2-4-2-1 penultimate row 1-2-4-3 last row no filter measurement carried out

Again, it is noted that the filter response for a given row cannot actually be calculated until the next two rows have been acquired - unless only a single consecutive row is used when calculating the response for the penultimate row in a range. If, in addition, the user specifies that the height measurement should cover a range Hmin - Hmax in physical units, we have to convert this into a range from rows Rmin to Rmax and then add at least one row at the top and one below so that we can get filter outputs that correspond to measurements in full from Hmin to Hmax.

Interesting sub-areas

Regarding 2 It is considered that different surface profiles can contain interesting functions that are separated by a considerable distance within the working area and functions outside the areas that contain interesting functions are missing or are unimportant for the analysis of the object. In the example of the object 120 is the base area 140 and the raised surface 114 interesting and there are no significant functions in the height range between these surfaces. So there is an area of interest around height H1 and between the heights H2 and H3 . All other heights are not used for the analysis. However, important processing time and resources can be consumed in analyzing the entire field of view for line information. In many applications, the general location of a function of interest within a field of view can be predictably determined within a relatively small height range. According to an exemplary embodiment and further with reference to the method 400 of the 4th the sensor process defines a large number of areas of interest RO11 and RO12 ( 2 ) in the field of view. Each area includes a plurality of rows in the imager. Each area of interest is sufficiently large in the column unit of the imager to accommodate the expected location of a laser line based on the height of the object functions and changes in height according to the movement of the object relative to the field of view or according to other process requirements. In one embodiment, each row area of interest includes pixels from all of the adjacent columns. According to the procedure 400 the user forms areas of interest based on an estimate of the physical height of the object functions of interest, their changes in height, since these functions move through the scanning process and process changes, areas of interest that the process with pixel positions in the imager (step 410 ) coordinated. With the corresponding encoder value (or another value based on movement), the imager captures a captured image of the object (step 420 ). The process of defining areas of interest causes the imager or other downstream component to acquire image pixel data only within the rows of pixels that are within the defined areas of interest. The processor can concentrate on resources in image data only from each area of interest and thus reduce the image read-out time (since image pixels outside the areas of interest are not transmitted) and accelerate the calculation of the surface height in each image. It is noted that the area of interest is for the line segment 230 is sufficiently large to accommodate the change in surface height of the surface 144 as the object moves along the y-axis as a result of the scanning process, as well as other process requirements.

In one embodiment, the imager includes the ability to output the selected rows through the processor 180 on or off. In other exemplary embodiments, the imager can define ranges of rows to be switched on (reading out of the selected rows is possible) or off (reading out of the deselected rows is prevented). Other arrangements that block the processing of predetermined series of image data can be implemented in other embodiments.

In step 430 uses the procedure 400 the line search algorithm described above to search for the location of the line with sub-pixel accuracy. Laser line intensity data and laser line position data can also be calculated by peak detection. For each column the peak value / position of the filter response is found, e.g. B. the captured pixel values are subjected to the filter process mentioned above. The laser line position data, which includes one or more laser line position indicators, correspond to the position of the filter peak value (in each column). The laser line intensity data, which includes one or more laser line intensity values, correspond to this filter peak value. The term indication is used with reference to a measured value, such as a pixel value; the line intensity indication relates to the laser line intensity pixel value, and the line position indication relates to the laser line position. The term data is sometimes used with reference to a set of individual pieces of information and sometimes with reference to a single piece of information. The location of the line in each column is shown in the step 440 used to form a slice of an overall elevation image that represents the profile of that slice of the object. In the optional decision step 450 For example, measurements of a disc can be analyzed to determine if a condition exists - such as an object that is out of specification. If this is the case, the system shows the condition (e.g. an error) in the step 452 and takes other appropriate action (e.g. ejecting the faulty object, ringing alarms, stopping the tape, etc.) and can loop to the next object in the step 410 to consider. However, it is expressly contemplated that various applications can continue to scan an object even after a faulty state has been detected (via the dashed branch 454 leading to the scan acquisition step 420 returns). It is also noted that the step 450 can also be used to count single or multiple scans (slices) of the image. These slices can represent a region of interest or a combination of regions of interest in the field of view. That is, while some System processes use a scanning process of the entire object surface (for example to ensure that alignment, measurements and decision-making are based on the full 3D representation of the object), some system processes can, if desired, use individual scanning processes (slices) of the object surface (i.e. partial cross-sections) and Align example, measure, collect data and / or make decisions based on each individual scan process / pane. So the disc will step over 456 forwarded to one or more downstream processes. In addition, some system processes use groups of scans (e.g. ten cross sections at a time) and combine this data using an appropriate technique (such as smoothing, averaging, other image processing, etc.). This combined data is then used to align, measure, collect data and / or make decisions. Scanning can continue as this group is on the branch 454 is processed as slices to the downstream system process / s in step 456 be sent. In addition, some system processes use a combination of the above to provide quick measurements / decisions, etc. based on individual or groups of scans, and final measurements, etc. based on the entire (or greater part of) scanned object.

In the absence of a signaled state (error, single target, large number of targets) in step 450 , the scanning process can continue (via the decision step 460 ) until images of the line that is located over the entire object surface (or a desired part of the surface) are captured. The individual height images are then used to form a 3D profile of the object surface in order to use it in downstream data processing processes ( 190 in 1 ) such as quality control, error detection, partial ejection, etc. Downstream measurements or similar information that includes physical data generated by analyzing more than one area of interest may be included in one or more decision making processes (192 in 1 ) and / or vision system task (s) (194 in 1 ) can be used. It is also contemplated that the downstream measurements or similar information may insert physical data between more than one ROI generated by the same scan. More specifically, various image processing operations performed in the areas of interest can be applied over (and using data from) many areas of interest throughout the elevation image to infer specific information about the object. Such processes can include, but are not limited to, pattern matching, testing, picking and dropping (and other robot manipulations), tolerance operations, and / or any other vision system function that can employ image data derived from a height image. The process continues (via decision step 480 ) for each object to the end (step 490 ).

Regarding 5 With certain implementations, the functions of interest can be in closer proximity to the height, so that the field of view 510 contains a line that is analyzed using two adjacent or in closer proximity (and non-overlapping) regions of interest RO11a and RO12a. In this example of an arrangement, the line search and filtering process described above ( 530 ) adjusted to accommodate the missing gap between areas. In this particular case, the beginning part of the table described above is adapted to accommodate the second part. It is noted that areas can generally be arranged such that the first row in the second area is the penultimate row in the first image, that is to say there is no gap between them. The filters can be applied to each neighboring area (RO11a and RO12a) in the following way:

First area:

Row number Filter used 0 No filter measurement carried out 1 3-4-2-1 2, 3, up to the penultimate row 1-2-4-2-1 Last row No filter measurement Since the sensor only detects every second row in the second area, the narrower filter that is used in the first area cannot run in the second area. However, the wider filter inserted through the second area can also be used in the last few rows of the first area. The results of the wider filter in this area overlap the results of the narrow filter and can be used to “blend” the data between the areas.

Overlap zone of the first area:

Row number Installed filter Last - 3 rows 1-2-4-2-1 Last - 1 row 1-2-4-2-1

Second area:

Row number Installed filter 0, 1, next 1-2-4-2-1 Last - 1 row 1-2-4-3 Last row No filter measurement

A GUI process ( 176 in 1 ) that is in the processor 180 , the connected computing device, or both, enable the user to (typically) select areas of interest during setup. 6th shows a GUI display 610 of the setup interface for the connected displacement sensor 110 . The ad 610 includes a variety of tabs 620 one that has access or that shows what the sensor and the various parameters used are being trained with. The selected tab 622 shows the "Profile" screen 630 , which includes input boxes that allow the user to select "Both" (632) to define many (in this example two) perceived areas of interest throughout the recognition base. For example, choosing “Both” returns two columns 634 and 636 associated with each respective area of interest to be formed in the field of view. Each area contains a setting for the physical position (in mm) of the Z recognition base 640 (the start of the detection area from the z origin of the work area), the Z detection height 642 (the z-distance of the particular area of interest) and a z-recognition sample value. In this example, the Z detection base of the first area starts at 30 mm above the origin and extends to a height of 10 mm. The Z detection base of the second area starts at 79mm and extends 14mm above it. Both areas are sampled at a value of 1. All areas in the field of vision outside these areas are excluded from processing. It should be noted that in the case of exemplary embodiments in which more than two areas of interest are to be formed, the selection “Both” can be replaced by another dialog box that forms the number of areas and associated columns can be used. For a better understanding of the user, these references (for each area of interest) are shown individually 650 on the side of the input area of the display 630 shown. This interface can enable the creation of a variety of areas of interest.

7th shows an alternative implementation of a GUI for setting up detection and exclusion areas. In this embodiment the display closes 710 a displayed image 720 of the entire field of view of the object 730 on. A laser line 740 is shown over the object 730 is projected. The user can move the object (via the conveyor, moving platform, etc.) to determine the boundaries of movement of each line in each area, and the boundaries of the associated areas of interest RO11b and RO12b are selected by the user. One or more corresponding Button / s 760 can be used to indicate areas of interest in connection with the cursor movements ( 750 ) or similar interface functions known to those skilled in the art. With the areas of interest set graphically, the setup process translates these locations into physical measurements and locations along columns of pixels in the imager.

Adjustment of the resolution

In further exemplary embodiments as described below, some or all of the rows can be defined at a reduced resolution - for example by samples in every other row. Additionally and as in the example of the 2 the line segment further away appears 220 with a lower resolution and defines a width LW1, crosses fewer rows of pixels than the closer line segment 230 , which defines a width LW2 that crosses more rows of pixels than the line segment 220 . The closer line segment is subjected to a denser scanning process and a higher resolution than the more distant line segment. Generally speaking, the closer the distance of the imaged light is within the work area, the higher the resolution of the image. Typically, the resolution of the further line segment is im Generally sufficient. For example, if the work area is shorter, the resolution of the areas of interest can be reduced by selectively switching off the output and / or processing of rows. For example, one row can be switched off alternately in the area RO12 of interest. To further reduce tripping, every third or fourth row can be turned on while intervening rows are turned off. The resolution of an area (RO11) with a longer working area can, if necessary, also define a reduced resolution according to this technology. When the resolution of both areas is reduced, a larger reduction is typically applied to the area with a shorter working area to compensate for its initially high resolution. It is noted that the ability to change the resolution can also be used to address the presence of a narrowed “waist” in the central area of the imaged laser line and widened ends on either side of the waist.

In another example shows 8th an exemplary part of an imager 800 including the rows of pixels 810 associated with the two regions of interest RO11c and RO12c. As shown, there are output statements 810 to the imager control circuit 830 and / or the vision system processor (e.g. due to an automated arrangement or a GUI facility) to clear signals (readings) from each alternating row (turned-off rows are denoted by an “X”) in the first area RO11c. Likewise, in the second area RO12c, every third row is switched on, while the two rows in between are switched off. In general, lowering the resolution will result in an acceptable profile while reducing the load on the processor and increasing the throughput speed for the scan. More specifically, the embodiment in FIG 8A an imager 850 , which transmits full resolution in the first area RO11d (switched on, read out every row) and half resolution in every second area RO12d (switched on, read out every second row). Control functions and instructions are similar to those related to 8th described above.

Generation of grayscale images

In many production applications, among other processes, it is desirable to check both the height profile of an object and certain 2D elements, such as a print. For example, all the keys on a keyboard should be at a predetermined height and they should all have the appropriate printed indexing (alphanumeric characters, symbols, etc.).

For example poses 9 a picture 910 of an ordinary object 920 (of a license plate), which is both flat (compared to the surrounding shield base 930 ) printed items 940 as well as raised elements. While each set of elements can be supplied in a contrasting color (e.g. blue and red), it is contemplated that the laser color can be selected to suitably resolve both colors in a captured image. On the basis of the captured image data, the system can measure the maximum filter response of the captured image intensities after each column has been subjected to the filtering processes described above, such as a 1-2-4-2-1 filter, in order to generate a grayscale image that is as insensitive as possible against changes in height and that shows changes in contrast like a pressure on the surface. Alternatively, the system can use the same quadratic or other interpolation function to generate an estimate of the response at the sub-pixel position of the measured lines and use this to generate the gray-scale image. The system can either generate a height image, a grayscale image or both at the same time (overlapped images). In some examples, the grayscale image can be generated from the filtered laser line intensity data, and the filtered laser line intensity images can be greater than an 8 bit value. This is because if the 1-2-4-2-1 filter is applied to an 8 bit value, the result can be greater than 8 bits. Since some machine vision operations expect 8 bit grayscale image input, the system can 100 Map laser line intensity with values from 16 bit to 8 bit grayscale image values. The user can use a pixel map to map a grayscale image with up to 8 bits from the filtered laser line intensity image, which can have a bit value greater than 8. The grayscale image data can be processed afterwards according to the corresponding height information, since laser lines can be thinner than the laser return and the laser return can be darker the further the laser lines are positioned from the sensor. The bit mapping described above can be an application specification and / or partially specific.

10 shows a corresponding elevation image in detail 1010 of the object 920 . It is noted that the height image 1010 is shown at different intensities due to altitude (for clarity), being black 1030 represents the height of the baseline of the work area, gray 1040 the raised height (from the baseline) of the plate base 930 and white 1050 the height of the raised elements 950 . The flat printing elements 940 are also generally gray, so they are the same color as the height of the base plate 930 have, but, as described below, "wrong" edges 1060 that appear as slight changes in altitude.

11 shows a grayscale image 1110 of the object 920 that was captured using the image data from the scanning process of the object under illumination of the laser line. As shown, the contrasting colors of the flat elements 940 clear compared to the contrasting background of the shield base 930 pictured. The sublime elements 950 are also mapped (although the color may be washed out if a similar laser color is used - choosing a suitable color and / or optical filter can ensure that the full grayscale image of the object is captured). The raised elements can also have fine features 1150 included, according to the shading of the angled outer contours of the raised elements. This effect can be significantly reduced by applying the corrections due to the gray-scale image. For example, the vertical gradient can be calculated at each pixel of the grayscale image and the correction based on this gradient can be applied to each height measurement in the height image.

As in the diagram of the 12th As shown, the scanning process of the laser line creates a captured image 1210 . This captured image is used by the line search processes / algorithms 1220 as generally described above. These processes / algorithms 1220 can act on an area of interest or, if necessary, on the entire image. The image data 1230 , which consist of pixel gray level intensity values in each column, and the location of the peak intensity (and intensity value) are passed through the line search process for use in forming a slice of the height image 1240 and also a slice of the grayscale image 1250 can be used. In another example, the image data 1230 Include grayscale intensity values generated by subjecting the laser line intensity data to one or more of the filters described above. The height image uses the top position in each column to generate a physical height value, while the intensity at the top is used as a gray level value for this xy position in the image.

If the image is scanned and a variety of scans ( 1210 ) is recorded, the process creates additional height and gray scale slices ( 1240 , 1250 ) leading to overall elevation and / or grayscale images 1260 , 1270 of the property. As described above, the user can determine (e.g. via the GUI) that one of the types of image or both types of images are generated. In one embodiment, one group of rows of pixels can be used to generate height images and another group of rows of pixels (e.g., alternating rows of pixels, etc., as generally in FIG 8th shown) can be used to create grayscale images. In this way, the resolution of each image can be reduced by using fewer rows of pixels, but individual (different) rows of pixels in the imager can be used simultaneously to create side-by-side height and grayscale images.

In some examples, the generated grayscale image data may include peak intensity data values that are a result of a bad pixel location in the captured image. As described above, the appearance of suspicious or bad pixels in the grayscale image may be a manifestation of a bug or anomaly in the hardware or software used to create the grayscale image. In particular, the use of the imaging components with poor perceptual elements can cause such suspicious or poor pixels to appear. This can also lead to incorrect results in the height images and the grayscale images. Therefore, it is desirable to ignore the data corresponding to poor perceptual elements by identifying suspicious peak data values and substituting interpolated grayscale values from nearby data in place of the suspicious peak data values. In another example, abnormal height values may mean that the associated gray level information is suspicious, e.g. B. suspicious grayscale image values can be identified by analyzing the associated height information, either in the laser line information or in the height / grayscale information.

12a Figure 3 is a flow diagram of a method 1200A for correcting zero or more suspect pixels in a row of the grayscale image that may contain suspect pixel data values as described above. As an example, the procedure 1200A instantiated at every component within the system, such as the vision system processor 180 , the sensor process 182 or the data processing devices / processes 190 . The blocks of the procedure 1200A Although shown below in some order, may appear or be performed by one or more processors in any particular order.

At the block 1210A the laser line tip position data and the corresponding laser line tip intensity data are received or generated. The laser line tip position data and the corresponding laser line tip intensity data can be output as the block 430 as described above. The laser line tip position data and the laser line tip intensity data may each include one or more individual laser line position information and laser line intensity information, each having a value. At block 1220A For example, the peak intensity location can be identified in the laser line peak intensity data as described above.

At block 1230A each data value of the laser line position and intensity data is compared to a suspect pixel map to identify one or more suspicious or poor data values in the laser line intensity data that correspond to the laser line position data. The location of suspicious or bad pixels can be known prior to the imaging process. In this regard, the suspicious pixel map at Block 1235A be developed for the imager during the calibration process. The suspicious pixel map can identify predetermined pixels known to be suspicious or bad to an imaging device. The suspicious pixel image can be generated at any time during the process, and in particular it can be generated during calibration of the imager. The suspicious pixel map can be generated by capturing one or more images with uniformly light or dark scenes during a calibration mapping process and identifying pixels that measure values that are significantly different from nearby perceptual elements. Alternatively or additionally ....... can be generated with the help of known suspicious pixel information supplied by the manufacturer.

At block 1240A a comparison is made between the peak intensity position at Block 1220A was identified, and the location of all suspicious pixel locations performed. Once the peak intensity pixel location is identified that corresponds to a suspicious pixel location, overlaps, or is within a predetermined proximity (e.g., within a few pixels), the system can determine that a correction should be made to include the suspicious data value in the Correct laser line intensity data. Such a correction can e.g. B. generating a replacement intensity pixel value at block 1250A and the replacement of the original intensity value, which corresponds to a suspicious pixel location in the imager, with the replacement intensity pixel value in the laser line intensity data at block 1260A lock in. The term original value refers to the intensity value returned by the sensor or processing steps prior to the execution of the blocks 1250A and or 1260A , and the term replacement value refers to a new intensity value given by Block 1250A is delivered. The replacement pixel value can be calculated according to any number of methods. In one example, the replacement pixel value can be calculated by taking averages of the intensity of neighboring gray level pixels, such as a gray level pixel on the left and a gray level pixel on the right of the bad pixel. In another example, the replacement pixel can be a predetermined, constant pixel value. In another example, a neighborhood-based operation can be performed on the pixel values in the area to determine the replacement pixel value. Such neighborhood-based operations can include any type of calculation performed on pixel values within a predetermined range of bad pixels. As stated above, when the replacement pixel value is fixed, the replacement pixel value can replace the value of the original data value associated with the bad pixel location on the imager. A corrected gray line pixel for the laser line peak intensity data can, if replaced, at block 1270A are issued.

If the location of the peak intensity pixel in the grayscale image does not correspond to a data value that is assigned to a suspicious pixel location, overlaps with it or is within a predetermined proximity (e.g. within a few pixels), then the system can the grayscale pixel without correction to the block 1245A output. In another example, the system can alternatively make corrections as in the blocks 1250A - 1270A although there is no overlap or correspondence within the peak intensity pixel location and the suspect pixel location.

In another embodiment, suspicious pixel values can be identified from the corresponding height image. If, for example, a certain height value differs from its expected height value in the height image by a previously determined threshold, then the corresponding pixel in the grayscale image can be marked as a suspicious pixel. The predetermined threshold of the difference between the expected and the observed altitude values can be defined by the user. In another example, the predetermined threshold can be calculated as the difference between neighboring pixels.

In another embodiment, suspicious pixel values from the corresponding height image can be identified by pixels for which the height value is missing. If, for example, a certain height value is missing from the height image due to laser speckle or clogging or some other reason, then the corresponding pixel in the grayscale image can be marked as a suspect pixel.

The grayscale image data can be used in a variety of vision system processes such as inspection, registration, print quality analysis, and any other process in which the presence, absence, location and / or quality of contrasting elements is analyzed. The system of the example uses the same imager and simultaneously acquired data from the imager to derive both the height image and the grayscale image, so that these images essentially represent a perfect registration - that is, each image pixel of the height image corresponds to the same pixel in the grayscale image . This eliminates the need for a separate camera assembly to capture each type of image and the dependent need for an alignment step to match the functions of the height image and the grayscale image. The presence of two physically aligned images with different data types (height and gray level) makes it possible to use a number of vision system tools with both data types together. This increases the usefulness of the entire image data set, whereby such simultaneous operations as registration, pattern comparison, etc. are possible with the aid of the gray-scale image and the measurement, inspection, etc. are possible using the height image. Certain similar vision system processes (e.g. testing) can also be applied to both images to provide a higher level of confidence in the outcome. For example, a hole is located in the grayscale image and the position of the corresponding change in height in the aligned height is used to verify its position. More generally speaking, the system can provide a measurement process that uses (a) measurements of data from the height image to form measurements of the gray scale imager, or (b) uses measurements of the gray scale image to form measurements of the height image. A decision making process ( 192 in 1 ) then generates decisions based on the measurement process. A number of decisions can be made in this and other embodiments herein, including, but not limited to, object rejection, tape pausing, alarms and warnings.

Resampling of a grayscale image to match the pixel scale

In some examples, the grayscale image and the height image may have different characteristics that prevent the two from being usefully compared or prevent them from being used simultaneously in a machine vision task. As already stated above, the grayscale image can be distorted or have a different pixel scale than the height image. To correct for such differences, the vision system can resample the underlying captured images or the laser line position data, laser line intensity data, or intensity data from corrected bad pixels (as described above) to generate a grayscale image with a uniform pixel scale with respect to the height image.

12B Figure 3 is a flow diagram of a method 1200B resampling captured images or the laser line position data and the laser line intensity data to produce a grayscale image with a uniform pixel scale. The resampling process 1200B can be used as a pre-process step at the vision system processor 180 executed, or more precisely, the sensor process 182 . The blocks of the procedure 1200B Although presented below in a particular order, may occur or be carried out by one or more processors and in any particular order.

In block 1210B become laser line position data 1210C and laser line intensity data 1220C through the system 100 generated from captured images. In particular, the laser line position data 1210C and the laser line intensity data 1220C from the images captured on the vision system processor 180 or the sensor process 182 can be generated and, as described above, the output of Block 430 which outputs the laser line intensity data corresponding to the laser line position data. As in 12C shown, both the laser line position data 1210C as well as the laser line intensity data 1220C can be arranged as rows of pixels, but any configuration or orientation of the pixels is contemplated. The array of laser line position data 1210C may include a plurality of pixels arranged in a row that is a row of pixels from the imager 164 correspond to captured images and which are provided with peak intensity values. The laser line intensity data may include a plurality of arrayed pixels corresponding to height image data that has been subjected to one or more filters as described above. In one example, the laser line intensity data may be the output of the method 1200A correspond to what was also described above. The laser line intensity data 1220C can also do an indexing 1230C that indicate albedo values measured at each pixel. For example, an albedo value that corresponds to the color blue can be used to display a darker gray level value measured in the laser line intensity data generated from the captured images. Similarly, an albedo value corresponding to the color red can be used to indicate a lighter gray level value measured in the laser line intensity data generated from the captured images. In other examples, different albedo values or colors can be used to indicate particular gray level values in the laser line intensity data.

At block 1220B becomes each pixel of the series of laser line position data 1210C mapped to a corresponding coordinate (x, z) in physical space. In an example, as in 12D shown, the figure can be the graphical representation 1240C include what is displayed pixel 1250C includes. Each of the displayed pixels 1250C includes a z coordinate that is the value of the laser line position data 1210C corresponds to. In particular, the pixels correspond to the laser line position data 1210C with a darker shade of gray a larger z-value. More precisely, the calibration processes described above provide a mapping from imager coordinates to physical coordinates. The calibration is used to map (x, y) imager pixel positions (x = column and y = row) to physical positions (x, z in mm). The laser line position data characterize the found y position (row) of the laser line tip imager for each imager x position (column). The calibration is used to apply the values in 1210C to the displayed pixels 1250C map. In this example, the pixel corresponds to the laser line position data 1210C furthest to the left, the pixel 1250C , which is shown furthest to the left, as well as the remaining pixels, which continue to the right side of the displayed pixels. In other examples, there may be a pixel shift in the x-direction.

At block 1230B as in 12E shown connection functions 1260C formed around the neighboring, displayed pixels 1250C connect to. In one example, the connectivity functions 1260C Line segments and similar indexes 1270C include those of indexing 1230C corresponding to those with respect to the laser line intensity data 1220C have been described. Indexing can be done in a similar way 1270C have an albedo value that corresponds to the albedo value used in the laser line intensity data 1220C has been described. The corresponding albedo values enable the appropriate color to be drawn into the uniform grayscale image of the graded pixels at the corresponding location.

As in 12F are shown at block 1240B for every bin 1280C with a width parallel to the x-axis, representative position values or area values are calculated from the z-coordinate values that fall into the bin. In one example, the representative location or area values may include an average. The width of the bin can be determined by any number of factors, including the desired width and resolution of the resulting uniform grayscale image of the graded pixels. In this regard, any of the bins can 1280C zero or more values from the laser line position data 1210C correspond. From each of these values within each respective bin 1280C fall, an average is taken to an average position value or range value for each bin 1280C to calculate. In another example, the average computer position value can be derived from the average range values of the connection functions 1260C within each bin.

For every bin 1280C a corresponding indexing or an albedo value is also calculated. In one example, the representative indexing or albedo value may include an average. The connection functions are discussed as above 1260C labeled according to the indexing that corresponds to certain albedo values. For every bin 1280C the albedo values that fall within the bin are also averaged to obtain an average albedo value for each bin. In another example, the calculated average albedo value can be calculated from the average albedo values of the connection functions within each bin.

As in 12G shown corresponds to at block 1250B the edition 1290C the calibrated height image in block 440 what the output is in uniform pixel scale. The calibrated height image corresponds to the average position value or the average area value calculated above. A uniform grayscale image of the graded pixels 1295C is also issued. The uniform grayscale image of the graded pixels 1295C closes indexing 1297C according to the average index or the albedo values calculated beforehand. For example, the indexing 1297C Include the color purple, with one of the bins having a connection function 1260C which is partly red and partly blue, where purple is the average of purple (?) (must mean “red” here, translator's note) and blue. Expenditure 1290C and 1295C are now re-sampled to have the same pixel scale, which allows pixel-to-pixel comparison to be made within the outputs. This can have many advantages and allows various machine vision tasks to be performed on the images become. In one embodiment, the resampled grayscale image can be generated at different (x, y) scales than the height image. This is achieved by performing the various blocks that are in the process 1200B are set out twice to provide a different x-measure. The grayscale image can be further resampled in y to provide a different y-scale.

Compensation for wrong height transitions

As in the height image 1010 ( 10 ) shown above, changes in contrast appear as slight, false contours (i.e. elements 1060 ) in the height image. The height measurement is not significantly affected by the absolute albedo of the surface in a zone of constant albedo, and so the middle area of each element shows 940 about the same height reading as the surrounding shield base 930 . However, on the verge of function 940 an error introduced in the height measurement when the transition from light to dark or dark to light passes the measuring zone.

13th shows a generalized procedure 1300 that compensates for such false height transitions. Height image data 1310 and grayscale image data 1320 are registered in each case (step 1330 ) so that height correction factors are stepped at each point in the image 1340 can be generated by the processor. For example, the method works 1300 continuously using an estimate of the grayscale image gradient in the direction of the scan to determine the height correction factor (s) applied at each point of the height image (step 1350 ) is / are used to calculate. For example, the correction factor can provide a numerical value - where, for example, zero (0) is no correction at a point in the height image. A correction factor of zero or near zero is applied to areas of the height image where the gray level contrast does not change. The correction factor is, for example, minimal or zero at points that are remote from (and inside or outside) the edges 1060 of the 10 while a more important correction factor is applied to points that are on the edges with respect to the important contrast gradient. When such correction factors are applied, a corrected / balanced height image is obtained in step 1360 generated. The elevation image eliminates most or all of the effects of contrast-based false elevation readings. A variety of techniques can be used in other embodiments for false heights and other contrast-based effects.

In determining whether there are incorrect height and / or grayscale functions, setting up and training the system may include user identifications or incorrect candidate functions and / or areas that contain candidate functions. This can speed up the analysis of such functions and improve the throughput of the displacement sensor.

It is expressly contemplated that a run time (or setup) display ( 6th ) can be provided to a user showing the above-described height image, grayscale image and corrected / balanced image individually or in combination (s). With the help of tailor-made and conventional GUI techniques, the user can be equipped with various functions that enable the various image parameters and / or characteristics to be adapted. Buttons and facility entries can be provided on a GUI, for example, to enable the user to set up thresholds and other image parameters (e.g. sensitivity, display, contrast, etc.) based on the displayed height image, grayscale image and / or corrected / balanced image and / or adapt.

Line search from threshold pixel voltages of columns

It is contemplated that the processing speed of various imager arrangements can be increased by finding the line center value for each column of pixels directly from "binary" pixels that either exceed or fall below the threshold. In this way, each pixel is delivered to a downstream process / processor, either as a logical value 1 (which e.g. indicates the presence of the laser line) or as a value 0 indicating the absence of the line at that location in this imager pixel column. 14th shows an arrangement 1400 where the imager 1410 Intensity values 1412 outputs from the pixels to a circuit (e.g. a comparison circuit) which compares the intensity of each pixel in a column with a reference intensity 1422 compares (1420) and determines which pixels exceed the threshold value established by the reference intensity. The column locations of the first and last pixels in each column that exceed the threshold (and have a value of 1) and generally tend to form a laser line are in block 1430 set. It should be noted that other pixels in the column spaced from the laser line may return a value of 1 due to noise, shine, etc., but the process does not create a group of such pixels that are large and close enough (although they may contains small gaps of pixels with 0 values) in order to define a laser line, should be filtered out. It is noted that the term “intensity” can be defined as a voltage value in various embodiments. That is, while a number of commercially available CMOS imagers convert the read pixel signal of each row from integrated irradiance to charge and then apply voltage to the readout process, it is contemplated that other types of imagers may be able to read directly directly in the charge area or as a current value instead of a voltage value. The term intensity means the definition of such a signal.

Based on the first and last pixel position in the column, the process sets the center value in the block 1440 firmly. This can be achieved by a specialized hardware circuit or by transferring the pixel value positions to the vision system processor. It is contemplated that while a thresholding process is employed to locate line pixels in each row to build the height image, the processor may also receive the intensity value of the pixels (e.g., in an area of interest) and use these values to calculate to form a slice of a grayscale image of the object surface.

It is noted that in generating a "mean" value, as used herein, a variety of techniques can be employed that allow an increase in processing speed versus accuracy - for example, the first value in a column that exceeds the threshold can be in a cluster of pixels above the threshold are identified as the “center.” For example, the term “center” should be used broadly to include locations along the column within the laser line image that send back the predictable result For example, knowledge of the laser's waist size (width) for a given column (x-position) in conjunction with the row position of the first pixel above the threshold can be used to calculate the center the following simplified calculation can be used: $$\text{Middle layer = first pixel layer + waist size / 2nd}$$

conclusion

It should be understood that the various embodiments described herein increase the versatility, processing efficiency, and throughput speed of laser displacement sensor systems without sacrificing the required accuracy. The examples of functions described here can be implemented easily and uncomplicated by a user during the setup, if appropriate interfaces are used. These functions enable the correction of certain inherent errors in height images, such as contrast-based incorrect height readings. Systems and methods are intended to produce registered gray-scale images of an object during the scanning process, free of the use of a dedicated camera system and assigned processors.

The above is a detailed description of the embodiments of the invention. Functions of the various embodiments described above may optionally be combined with functions of other described embodiments to provide a variety of combinations of functions in newly connected embodiments. Furthermore, while the foregoing describes a number of individual embodiments of the apparatus and method of the present invention, what has been described herein is only one example of application of the principles of the present invention. Terms of direction and orientation that were used here, for example, such as "vertical", "horizontal", "on", "below", "below", "above", "sideways", "front", "rear", " left ”,“ right ”and the like are only used as relative agreements and not as absolute orientation in relation to a fixed coordinate system, such as gravity. While areas of interest therefore have constant size and position in the field of view / imager through the scanning process of each object, it is taken into account that the size and / or positions of areas of interest may change during the scanning process in order to achieve the expected change in the positions of the function or change in the functions of interest along the y-axis. A change in the areas of interest can occur in response to the encoder position or some other indication of the relative position of the object with respect to the field of view. In addition, while the terms “rows” and “columns” are used to describe a specific arrangement of pixels in the imager, it is expressly contemplated that these terms can be interchanged and similar results can be achieved according to the exemplary embodiments here. Even while a laser is used to generate the lighting to capture both height information / image and grayscale information / image, additional lighting can be used to capture the grayscale image - for example, for alternating acquisition of the height image and the grayscale image and stroboscopically the additional lighting with each gray level detection. While embodiments here also define areas of interest in which the processing of image pixels takes place, and areas in between that are free from image pixel processing, such areas in between can be functionally equivalent to one or more "exclusion zones" in which the user defines the non-processing areas, and the system will otherwise fail in terms of processing power outside of this “exclusion zone (s). More generally speaking, in embodiments, the system and method define one or more areas of interest and associated exclusion zone (s), which enables selective detection or non-detection (or) of a laser line within the entire imager field of view.

Claims (20)

A system for measuring the displacement of an object surface (144) comprising: - a displacement sensor (110) projecting a line (156) onto the object surface (144) and receiving light from the projected line (156) on an imager (164) to define laser line position data (1210C) and laser line intensity data (1220C) ; and - a vision system processor (180) constructed and arranged to generate a height image (1010) from the laser line position data (1210C), a gray scale image (1110) from the laser line position data (1210C) and the laser line intensity data (1220C), and an estimate of a vertical grayscale image gradients, the grayscale image gradient being used to calculate a height correction factor to be applied to each pixel of the height image (1010). System according to Claim 1 wherein height image data (1310) of the height image (1010) and grayscale image data (1320) of the grayscale image (1110) are registered, respectively. A system according to any preceding claim, wherein the altitude correction factor provides a numerical value. System according to Claim 3 , where the numeric value is zero or non-zero. The system of any preceding claim, wherein applying the height correction factor to each pixel of the height image (1010) produces a corrected height image. System according to Claim 1 wherein elevation slices (1240) and grayscale slices (1250) are added to overall elevation and / or grayscale images (1260, 1270). System according to Claim 1 wherein the system is configured to move either the displacement sensor (110), the object (120), or both along at least one axis to detect a plurality of scans (1210) from the displacement sensor (110). System according to Claim 7 wherein one group of rows of pixels is used to create the height image (1010) and another group of rows of pixels is used to create the grayscale image (1110). System according to one of the Claims 7 or 8th wherein a method (1300) executable on the system operates continuously by estimating the grayscale image gradient in the direction of the scan to calculate the height correction factor applied to each point of the height image (1010). System according to Claim 1 wherein the generated grayscale image (1110) includes peak intensity data values that are a result of a bad pixel location. A method for measuring the displacement of an object surface (144) comprising: - by means of a displacement sensor (110), projecting a line (156) onto the object surface (144) and receiving light from the projected line (156) on an imager (164) so that laser line position data (1210C) and laser line intensity data (1220C) are defined become; and - by means of a vision system processor (180), generating a height image (1010) from the laser line position data (1210C), a grayscale image (1110) from the laser line position data (1210C) and the laser line intensity data (1220C), and generating an estimate of a vertical grayscale image gradient, the grayscale image gradient is used to calculate a height correction factor to be applied to each pixel of the height image (1010). Procedure according to Claim 11 wherein height image data (1310) of the height image (1010) and grayscale image data (1320) of the grayscale image (1110) are registered, respectively. Method according to one of the Claims 11 or 12th , where the altitude correction factor provides a numerical value. Procedure according to Claim 13 , where the numeric value is zero or non-zero. The method of any preceding claim, wherein applying the height correction factor to each pixel of the height image (1010) produces a corrected height image. Procedure according to Claim 11 wherein elevation slices (1240) and grayscale slices (1250) are added to overall elevation and / or grayscale images (1260, 1270). Procedure according to Claim 11 wherein either the displacement sensor (110), the object (120), or both are moved along at least one axis to detect a plurality of scans (1210) from the displacement sensor (110). Procedure according to Claim 17 wherein one group of rows of pixels is used to create the height image (1010) and another group of rows of pixels is used to create the grayscale image (1110). Method according to one of the Claims 17 or 18th wherein the method (1300) operates continuously by estimating the grayscale image gradient in the direction of scanning to calculate the height correction factor applied to each point of the height image (1010). Procedure according to Claim 11 wherein the generated grayscale image (1110) includes peak intensity data values that are a result of a bad pixel location.

Images