KR102153962B1 - System and method for finding lines in an image with a vision system - Google Patents

Abstract

The present invention relates to a method and system for finding multiline features in an image.
Two related steps are used to identify line features. The first process calculates the x and y components of the gradient field at each image location, projects the gradient field over multiple sub-regions, and calculates multiple edge points with locations and slopes. It detects a plurality of gradient extrema. The second process selects two edge points, fits a model line to them, and if the edge point slope is consistent with the model, then the entire set of inner left-floor points whose position and slope match that model ( of the full set of inlier points).
The candidate line with the maximum left-most count is maintained and a set is derived leaving outlier points. The process repeatedly applies the line fitting operation and a subsequent set of outer left layers to find a plurality of line results. This process can be done on a thorough RANSAC basis.

Description

A system and method for finding lines in an image with a vision system {SYSTEM AND METHOD FOR FINDING LINES IN AN IMAGE WITH A VISION SYSTEM}

Related application

This application is a co-pending US provisional application filed on November 2, 2015, the entire contents of which are incorporated herein by reference, entitled "SYSTEM AND METHOD FOR FINDING LINES IN AN IMAGE WITH A VISION SYSTEM" Claims the interest of serial number 62/249,918.

Technical field

FIELD OF THE INVENTION [0001] The present invention relates to a machine vision system, and more particularly, to a vision system tool for finding line features in an acquired image.

Machine vision systems (also referred to herein simply as “vision systems”) are used in manufacturing, logistics, and many operations in industry. Such operations may include inspection of surfaces and parts, alignment of objects during assembly, reading patterns and ID codes, and any other operations in which visual data is acquired and interpreted for use in further processes. Vision systems generally use one or more cameras to acquire an object of interest or a scene image that includes an object of interest. This object/object can be either stationary or relatively movable. Movement may also be controlled by information guided by the vision system, such as when manipulating parts by a robot.

A common task of vision systems is to find and characterize line features in images. Several tools are used to identify and analyze these line features. In general, this tool relies on sharp contrast differences occurring in parts of the image. Analyze this contrast difference using, for example, a caliper tool to determine whether individual points in the image with contrast differences can be assembled into line-like features. If it can be assembled, lines are identified in the image. In particular, a tool that detects edge points and a tool that attempts to fit a line to the points are independent of each other. This increases processing overhead and reduces reliability. If the image contains multiple lines, these tools may have limited ability to accurately identify these lines. Furthermore, traditional line-finding tools designed to find a single line in an image can be problematic to use when the image contains multiple closely spaced lines of similar orientation and polarity.

The present invention overcomes the drawbacks of the prior art by providing a system and method for finding line features in an image that can efficiently and accurately identify and characterize multiple lines. First, the process calculates the x- and y-components of the gradient field at each position of the image, projects the gradient field over a plurality of image subregions, and each subregion By detecting a plurality of gradient extremas within, a plurality of edge points having associated positions and gradients are calculated. Next, the process iteratively selects two edge points, fits the model line to these edge points, and if the gradients of these edge points are consistent with the model line, the position coincides with the model line. Compute the entire set of inlier points with gradients and. The candidate line with the maximum normal count is retained as the line result, and a set of remaining outlier points is derived. The process finds a plurality of line results by repeatedly applying a line fitting operation to this set of outliers and subsequent sets of outliers. The line-to-fit process may be performed entirely or may be based on random sample consensus (RANSAC) techniques.

In an exemplary embodiment, a system for finding line features in an acquired image is provided. The vision system processor receives image data of the scene including line features. An edge point extractor generates intensity gradient images from the image data and finds edge points based on the intensity gradient images. A line-finder fits the edge points to one or more lines based on an intensity gradient at the edge points. Illustratively, the line finder operates a RANSAC-based process of fitting normal edge points to new lines, including repetitively defining lines from outlier edge points for a previously defined line. The edge point extractor performs gradient field projection of regions including line-feature portions of the intensity gradient images. Illustratively, the gradient field projection is oriented along a direction set in response to one or more or expected orientations of the line features, and the gradient field projection will define a granularity based on a Gaussian kernel. I can. For example, the image data may include data from a plurality of images acquired from a plurality of cameras and converted into a common coordinate space. The image data may also be smoothed using a 1D Gaussian kernel or a smoothing (weighting) kernel, which may include other weighting functions. The edge points may be selected based on a threshold value defined by an absolute contrast and a normalized contrast based on an average intensity of the image data. Exemplarily, the line finder exchanges edge points representing portions of parallel lines or intersecting lines to correct an erroneous orientation, and/or a line based on the gradient values of the edge points. Constructed and arranged to identify lines with polarity variation, including mixed polarities of the features. Also, by way of example, the edge point extractor is arranged to find a plurality of gradient magnitude maxima in each of the gradient projection sub-zones. Each of these gradient magnitude maximum values can be identified as some of the plurality of edge points, and can be described by a position vector and a gradient vector. Additionally, the line finder calculates a consistency between at least one edge point among the plurality of extracted edge points and at least one candidate line among the plurality of found lines by calculating a metric. Can be arranged to determine. This metric may be based on a distance of the at least one edge point from the candidate line, and an angle difference between a gradient direction of the edge point and a normal direction of the candidate line.

The following description of the present invention refers to the accompanying drawings.
1 is a diagram illustrating an exemplary vision system arrangement for acquiring an image of an object, including a vision system processor and a plurality of edge features including an edge-discovery tool/module in accordance with an exemplary embodiment;
Fig. 2 is a diagram showing an overview of a system and method for extracting an edge point and finding a line from an acquired image, according to an exemplary embodiment;
3 is a flowchart of a procedure for extracting an edge point according to the system and method of FIG. 2;
FIG. 4 is a diagram showing a gradient field projection in a region of an image including edge features that are part of the procedure for extracting an edge point of FIG. 3;
FIG. 5 is a diagram illustrating applying a Gaussian kernel to an image to smooth the image for use in the procedure of extracting the edge point of FIG. 3;
6 is a diagram illustrating a gradient field projection including applying a Gaussian kernel to smooth the projection, for use in the procedure of extracting the edge point of FIG. 3;
Fig. 7 is a graph showing an overview of the procedure for extracting the edge point of Fig. 3, including applying Gaussian kernels and calculating an absolute contrast threshold and a normalized contrast threshold for the edge points;
8 is a graph showing a region of qualified contrasts for edge points with a sufficient absolute contrast threshold and a normalized contrast threshold;
9 is a flow diagram of a procedure for finding a line based on edge points found in FIG. 3 using an exemplary RANSAC process according to an exemplary embodiment;
Figures 10 and 11 show edge points erroneously and correctly aligned for closely spaced parallel line features, respectively;
12 and 13 are diagrams showing correct and erroneous alignment of edge points, respectively, for cross-line features that may be analyzed according to the line-finder of an exemplary system and method;
14-17 show light-to-dark polarity, dark-to-brightness, which can be analyzed according to the line-finder of exemplary systems and methods. light polarity), light to dark polarity or dark to light polarity; And
18 is a diagram showing a change in coverage score for a line found taking into account a user-defined mask.

An exemplary vision system arrangement 100 that may be used in accordance with an exemplary embodiment is shown in FIG. 1. System 100 includes at least one vision system camera 110 and may include one or more additional optional cameras 112 (shown in dotted lines). Exemplary camera(s) 110, 112 include an image sensor (or imager) S and associated electronics that acquire an image frame and transmit the image frame to a vision system process (processor) 130. ), and this image frame may be instantiated in an independent processor and/or computing device 140. The camera 110 (and 112) includes a suitable lens/optical device 116 that focuses on the scene containing the object 150 being inspected. Cameras 110 (and 112) may include internal and/or external illuminators (not shown) that operate according to the image acquisition process. Computing device 140 may be any acceptable processor-based system capable of storing and manipulating image data in accordance with an exemplary embodiment. For example, computing device 140 may include a PC, server, laptop, tablet, smartphone, or other similar device (shown). Computing device 140 may include suitable peripherals such as a bus-based image capture card interconnected to the camera. In an alternative embodiment, the vision processor may be partially or completely contained within the camera body itself, and may be networked with other PCs, servers and/or camera-based processors that share and process image data. The computing device 140 selectively selects a suitable display 142 capable of supporting a suitable graphical user interface (GUI) capable of operating in accordance with the vision system tools and processor 132 provided to the vision system process (processor) 130. Can be included as. It is understood that the display may be omitted in various embodiments and/or provided only for setup and service functions. The vision system tool is any acceptable software and/or acceptable for use in inspecting objects, such as those commercially available from Cognex Corporation of Natick, Massachusetts (MA). May be part of a hardware package. The computing device may include, for example, a keyboard 144 and a mouse 146, and associated user interface (UI) components including a touchscreen in the display 142.

Camera(s) 110 (and 112) images some or all of the object 150 positioned within the scene. Each camera defines an optical axis OA, and a field of view is established around the optical axis based on the optics 116, focal length, and the like. The object 150 includes a plurality of edges 152, 154, and 156, each arranged in various directions. For example, the object edges may include the edge of the cover glass mounted within the smartphone body. Illustratively, the camera(s) may image an entire object or a specific location (eg, a corner where the glass meets the body). A (common) coordinate space may be established for an object, a camera of one of the cameras, or another reference point (eg, a moving stage supporting the object 150). As shown, the coordinate space is represented by axes 158. These axes exemplarily define an orthogonal x-axis, y-axis, and z-axis, and a rotation θ _{z about the} z-axis in the xy plane.

According to an exemplary embodiment, the vision system process 130 is interoperable with one or more applications/processes (running on the computing device 140) collectively comprising a set of vision system tools/processes 132. It works. These tools can include several conventional specialized applications used to analyze image data-for example, using several calibration tools and affine transform tools to convert the acquired image data. It can be converted to a predetermined (eg common) coordinate system. Tools for converting image grayscale intensity data into a binary image based on a predetermined threshold may be included. In addition, tools for analyzing the gradient of intensity (contrast) between adjacent image pixels (and subpixels) may be provided.

The vision system process (processor) 130 includes a line-discovery process, tool, or module 134 that finds multiple lines in an acquired image according to an exemplary embodiment. Referring to Fig. 2, Fig. 2 graphically shows an overview of a line-discovery procedure 200 according to an exemplary embodiment. This procedure 200 consists of two main parts. The input image 210 is provided to the processor. As shown, this image includes a pair of intersecting edges 212 and 214. These intersecting edges may represent the corner regions of the object 150 described above. The edge point extractor 220 processes the input image 210 to obtain a set of candidate edge points 230 including edge points 232 and 234 that exist along edges 212 and 214, respectively. Edge points 232 and 234 and their associated data (e.g., intensity gradient information described below) are provided to a recursive line finder 240, which line finder at the selected edge points. It performs a series of iterative processes. The goal of the iterative process is to try to fit other found edge points to candidate line features. As a result of performing the line-discovery process 240, lines 252 and 254 are found as shown. These results may be provided to other downstream processes 260 using the information-for example alignment processes, robotic manipulation, inspection, ID reading, partial/surface inspection, and the like.

Referring to FIG. 3, FIG. 3 describes a procedure of extracting edge points according to an embodiment. One or more images comprising an object or surface having edge features to be found in the scene are acquired (step 310). These image(s) can be extracted by a single camera or multiple cameras. In either case, the image pixels may (optionally) be transformed into a new coordinate space and/or a common coordinate space in step 320 with appropriate calibration parameters. This step may further include smoothing the image as described below. In certain embodiments, when multiple cameras are imaging discrete areas of the scene-e.g. focusing on a corner area of a larger object-the common coordinate space is an empty region between the camera's fields of view. Can be considered. As described below, lines extending between these fields of view (eg, an object edge connecting two found corner regions) can be extrapolated by the system and method of the exemplary embodiment. The edge points required to find the lines are extracted from the image(s) in the appropriate coordinate space by the edge point extractor in step 330 using gradient field projection. First, the gradient values are calculated for each pixel, resulting in two images for the x and y gradient components. This image(s) is further processed by projecting the gradient field over many caliper-like areas. Unlike conventional caliper tools that project intensity values, by projecting the gradient field according to this embodiment, the gradient orientation can be preserved, which facilitates the process of finding subsequent lines as described below. do.

Referring also to the diagram of FIG. 4 at step 340, the portion of the image (caliper-like area) 400 containing the candidate edge features is the gradient field (represented by multiple projections 410, 420, 430). Projected and searched over the (approximately) expected orientation of the edges in the search direction (arrow SD), which projection is repeated over the region 400 in the orthogonal projection direction (arrow PD). The edges in each projection (eg projection 420) appear to be local maximums in the gradient field 440 associated with this projection. In general, a series of edge points in a projection associated with an edge represent an intensity gradient (vectors 552, 554) orthogonal to the direction of extension of this edge. As described below, the user can define the projection direction based on the expected line orientation. Alternatively, this could be provided by default or by another mechanism-for example by analyzing features in the image.

Two granularity parameters are involved in the gradient projection step described above. Before calculating the gradient field, the user can choose to smooth the image using an isotropic Gaussian kernel. The first particle size determines the size of this Gaussian smoothing kernel. As shown in the diagram 500 of FIG. 5, the image 210 is smoothed by applying an appropriately sized Gaussian kernel (eg, large 512, medium 514, small 516). Thus, the first particle size parameter determines the size of the isotropic Gaussian smoothing kernel before calculating the gradient field.

After calculating the gradient field, a Gaussian-weighted projection rather than the uniform weighting performed in a conventional caliper tool is performed by this process. Thus, the second granularity parameter is the size of the one-dimensional (1D) Gaussian kernel used during gradient field projection, as shown in FIG. 6 in which the region 600 received Gaussian-smoothed kernels 610, 620, 630 Decide. During normal operation, the user verifies (using the GUI) all the extracted edges overlaid on the image, and then the number of edges extracted along the line to be found is reduced, avoiding an excessive number of edges due to background noise in the image. Adjust the granularity and contrast thresholds until they look satisfactory. In other words, this step allows the signal-to-noise ratio to be optimized for image characteristics. This adjustment may be performed automatically by the system using default values in various embodiments. Using a Gaussian weighting function is understood to be one of several approaches to weighting projections, including (for example) uniform weighting.

The overall flow of extracting and projecting the gradient field is shown graphically in the diagram 700 of FIG. 7. Two granularity parameters, i.e., isotropic Gaussian kernel 710 and 1D Gaussian kernel 720 are shown in each half of the whole diagram 700, respectively. As shown, each acquired image 210 is subjected to smoothing and decimation (730). The resulting image 740 is subjected to a gradient field calculation 750 as described above, and two gradient images 752 and 754 are generated. These gradient images are also represented by g _x and g _y representing the two orthogonal axes in a common coordinate space. In addition to the two gradient images, the processed intensity information is also used to calculate the normalized contrast according to one embodiment-the embodiment described below, so the intensity image 756 is also generally (1D Gaussian kernel 720 ) Based on a Gaussian-weighted projection 770), which is understood to be subjected to a smoothing, decimation, and projection process 760. As a result, a projection profile of the gradient image 772 (g _x ), image 774 (g _y ), and intensity image 776 is generated.

Also referring to step 350 in procedure 300 (FIG. 3), qualified edge points are extracted by combining the 1D projection profiles of x and y gradient images. This is achieved using calculating raw contrast based on the intensity image 780 and calculating normalized contrast 790. More specifically, any localized peak with the original projected gradient size and normalized projected gradient size exceeding each threshold value is considered as a candidate edge point for finding a subsequent line according to the following exemplary equation:

(g _x ² + g _y ² ) ^1/2 > T _ABS

(g _x ² + g _y ² ) ^1/2 /I> T _NORM

Where g _x and g _y are, respectively, the values of the x- and y-gradient projections at the pixel position, I is the intensity, T _ABS is the absolute contrast threshold for the original projected gradient size, and T _NORM is the intensity -Is the normalized contrast threshold for the normalized projected gradient size.

In particular, when the absolute contrast and the normalized contrast of a point exceed both of each threshold, only this point is considered as a candidate edge point. This is shown as an upper right quadrant 810 in an exemplary graph 800 of a normalized contrast threshold T _NORM versus an absolute contrast threshold T _ABS . The use of dual thresholds (absolute and normalized thresholds) is generally different from conventional approaches that use absolute contrast thresholds. The benefit of the dual contrast threshold is obvious when, illustratively, the image contains both a dark intensity region and a bright intensity region including the edges of interest. In order to detect edges in dark areas of the image, it is desirable to set a low contrast threshold. However, such a low contrast setting can detect false edges in bright areas of the image. Conversely, in order to avoid detecting false edges in bright areas of the image, it is desirable to set a high contrast threshold. However, at high contrast settings, the system may not properly detect edges in dark areas of the image. By using the second normalized contrast threshold in addition to the traditional absolute contrast threshold, the system can properly detect edges in both dark and bright areas, and avoids detecting false edges in bright areas of the image. I can. Thus, through the use of a dual contrast threshold, the speed and robustness of the step of finding the next line in the whole process is maximized by being able to detect the relevant edge while avoiding the spurious edge. Functions to do.

With further reference to procedure step 350 (Fig. 3), once all edge points are extracted, these edge points are represented and stored in a data structure convenient for subsequent line-finders to operate. For example, the following tuple:

p = (x,y,gx,gy,gm,go,I,gm/I,m,n)

Where (x,y) is the position of the edge point, (g _x ,g _y ) is the value of each x-gradient and y-gradient projection, and (g _m ,g _o ) is from (g _x ,g _y ) Is the calculated gradient size and orientation, I is the intensity at the edge point position, g _m /I is the intensity-normalized contrast obtained by dividing the gradient size (g _m ) by the intensity (I), and m is Is the image index, and n is the projected area index. As in a standard caliper tool, the position of the edge point can be interpolated to improve accuracy.

It is generally understood that the process of extracting an edge point operates to perform a gradient field projection in a single direction that substantially matches the expected line angle. Thus, the tool is most sensitive to the edge of this angle, and its sensitivity gradually drops at the edge of the other angle, and the rate at which it falls depends on the grain size setting, which indirectly determines the gradient field projection length. As a result, this process is limited to finding lines with an angle of "near" the expected line angle, according to the angle range specified by the user. While this process is adapted to find lines that are not orthogonal, this process in several embodiments finds any angular line over 360 degrees by performing projections in multiple directions, including orthogonal directions. It is considered that it can be generalized to find).

Referring now to step 360 in procedure 300 (FIG. 3), edge point candidates above the threshold are provided to the line-finder according to an exemplary embodiment. For example, line-finders operate recursively and use (for example) RANSAC (RANdom SAmple Concensus)-based technology. See also line-discovery procedure 900 in FIG. 9. In step 910, the user is given an expected line in the image along with (e.g.) an expected angle, angular tolerance, distance tolerance, and (exemplarily) a minimum coverage score (generally defined below) via the GUI. Specifies the maximum number of. This parameter is then used by the line-finder to run the process. This line is found for each sub-region of the image by recursively executing the RANSAC line finder, and edge outliers from one step become the input points for the next step. Thus, in step 920, procedure 900 selects a pair of edge points that are part of a group of edge points identified as extreme values in the edge-discovery process. Procedure 900 attempts to fit the model line to selected edge points based on matching gradient values (within the selected tolerance range) that match the model line. In step 924, one or more line candidate(s) from step 922 are returned. Each line-discovery step returns a candidate line, its normal and outliers. The returned line(s) are computed on the normal edge points with the position and gradient matching the line candidate (step 926). In step 928, a candidate line with the maximum normal count is identified. The line-discovery steps described above (steps 920-928) end when the maximum number of allowed RANSAC iterations is reached (decision step 930). The maximum number of iterations in each line-discovery step is calculated automatically using the assurance level specified by the user and the internally calculated worst case ratio of outliers. Each line discovery step returns a line with the maximum number of captured edge points out of all iterations, according to user-specified fit tolerances, geometric constraints and polarity. Each edge point can only be assigned to a single line's normal list, and each line is allowed to contain at most one edge point from each projected area. Along with its position, the gradient orientation of the edge point is used to determine whether the edge point should be included in the normal value list of the candidate line. In particular, the edge points must have a gradient orientation that matches the angle of the candidate line.

If decision step 930 determines that more iterations are allowed, outliers from the best normal candidates are returned to the RANSAC process (step 920) for use in finding line candidates (step 940).

During each RANSAC iteration, two edge points belonging to different projection zones are randomly selected, and there may be a line that fits these two points. The resulting candidate line is subject to additional consideration only when the angle of the candidate line coincides with the draft angle of the two edges at a pair of points, and when the angle of the line coincides with the uncertainty range specified by the user. In general, the draft direction of the edge point is nominally orthogonal, but different by a user-configured angular tolerance is allowed. If the candidate line passes these initial tests, the number of normal edge points is evaluated, and if not, a new RANSAC iteration is initiated. The edge point is considered as a normal value of the candidate line only when the gradient direction and position coincide with the line based on the gradient angle and distance tolerance specified by the user.

When the RANSAC iteration reaches its maximum value (decision step 930), the best found line candidate's normal values (e.g.) use least squares regression or other acceptable approximation techniques for improved line fit. And, until the number of normal values further increases or does not decrease, this step is repeated at most N times (eg, 3 or more times) to re-evaluate the set of normal value edge points (step 960). This is the line indicated in step 970 as found.

Decision step 980 determines whether more lines are to be found (based on searching for additional sub-zones or other criteria (for example)), and if found, the process returns to step 920 and the new Operates on the edge points of the set (step 982). If the points have been performed all over or the maximum iteration count has been reached, procedure 900 returns a set of lines (ie, multiple) found in the image at step 990.

The multi-line finder is adapted to finally adjust the existing results in case the two line results intersect each other in the inspection area. In general, for closely spaced parallel lines 1010 and 1020, as shown in FIGS. 10 and 11, often an erroneous line result (i.e. FIG. 10) may be obtained due to the statistical nature of the RANSAC procedure. . However, when this error occurs, exchanging a group of points of normal value (arrow 1120 in group 1110 in FIG. 11) often results in an accurate line with an increased coverage score and a reduced-fit residual. Can be found. Swapping points may be most effective when the image contains parallel lines spaced close to each other as shown. Conversely, when the image includes a line 1210 and a line 1220 that actually intersect each other as shown in Figs. 12 and 13, the coverage score exchanges points (arrows in the group 1240 in Fig. 12). 1230), the original result obtained before the exchange is maintained in order to successfully detect the lines intersecting by the process.

It is understood that the RANSAC procedure is only one of several techniques by which the line-finder can fit points into lines. In an alternative embodiment, candidate points may be selected according to the displacement established between these points, or the image may be processed using (eg) an exhaustive search technique. Thus, reference to RANSAC technology as used herein is to be broadly interpreted as including several similar point-to-measure techniques.

Additional functionality of this system and method may be provided. These features support mixed-polarity, automatically compute projected area widths, support multi-view line discovery, and eliminate optical distortion by removing pre-warpage from the input image. Includes doing. These functions are further described below.

With further reference to the example of FIGS. 14-16, the line-discovery system and method of the exemplary embodiment includes a standard light-to-dark polarity, a dark-to-bright polarity for contrast between the found edges, and Either polarity setting is generally supported (respectively). Furthermore, this system and method can further support a mixed-polarity setup (FIG. 17) where the light-to-dark polarity and dark-to-bright polarity characteristics are both visible on the same line. The line-discovery results of all four settings are shown in the following figure. In an exemplary embodiment, this system and method may include a mixed-polarity setting that can find a single line containing edge points of opposite polarity. This is different from the conventional setup where all the edge points of a single line are of either polarity-but only one polarity, "any one" polarity. A mixed-polarity setting can be advantageous when used to analyze the bright and dark checkerboards of the calibration plate (eg) among other applications.

The user can select an improved shift invariance to find the line. In this case, the edge point extractor substantially improves the resulting stability by using overlapping projection areas. If the zones do not overlap, the pixels under consideration may potentially move out of the projected zone when the image is shifted, resulting in poor shift invariability in the line-discovery result. The overlapped projection zones ensure that the pixels under consideration are continuously covered by the projection zones. If overlapping projected areas are used, incremental operations can be performed to maintain speed with as low a level of optimization as possible.

The user may provide a mask that omits certain portions of the imaged surface and/or images obtained from analysis of the line features. This may be desirable if the surface contains known line features of no interest (eg, barcodes that are analyzed by other mechanisms, text, and any other structure not related to the operation in which the lines are found). Thus, the edge point extractor can support image masking in which “don't care” areas in the image can be masked out, and “interesting” areas are masked in. When this masking occurs, the coverage scores of the found lines are exemplarily reweighted according to the number of edge points in the mask.

Referring to the exemplary image region 1800 of FIG. 18, FIG. 18 shows coverage scores when image masks are present and the effect of image masking on these coverage scores. The edge point extractor supports image masking in which "areas of interest" can be masked out in the image. As shown, the found line 1810 is characterized with edge points of interest (based on the “interesting” mask region 1820). These edge points of interest are composed of an edge point of interest normal value 1830 to line 1810 and an edge point of interest outlier 1840 to line 1810. Edge points 1850 of no interest in the line 1810 exist between the regions of interest 1820 of the mask, as shown in this example, and the coverage score is calculated even when these uninteresting edge points exist in the line with normal values. Not included in Potential locations 1860 of edge points along line 1810 are also determined as shown. This potential location is located between known points with predictable spacing based on the spacing of the found points. Illustratively, the coverage scores of the found lines are reweighted according to the number of edge points within the mask. Thus, the coverage score changes as follows:

Coverage score = number of edge point normals of interest for line/(number of edge point of interest normals for line + number of edge point of interest outliers for line + number of potential positions of interest of edge points).

After executing the line-discovery process according to the systems and methods herein, the found lines can be classified in several ways based on the classification criteria specified by the user (for example, through a GUI). The user may be selected from intrinsic classification measures such as normal coverage score, intensity or contrast. Users can also choose from extrinsic classification measures such as signed distance or relative angle. When using the extrinsic classification action, the user can specify a reference line segment to compute the extrinsic action of the found lines.

As generally described above, this system and method may include a Multi-Field-of-View (MFOV) overload that can transfer image vectors from different fields of view to the process. Images should all be in a common client coordinate space based on calibration. As mentioned above, this function can be extremely advantageous in application scenarios where multiple cameras are used to capture partial regions of a single part. Because the edge points maintain the gradient information, the projected line features between the gaps in the field of view (when the gradients in both FOVs match a given line orientation and alignment in each FOV) can still be analyzed.

In particular, this system and method does not require removing distortion (in the image) to remove nonlinear distortion (i.e., it does not require that the image is not distorted), assuming that the distortion is not severe. If the image is not distorted, the system and method can still detect candidate edge points and map the position of the point and the gradient vector through nonlinear transformation.

It is evident that the line-finder provided according to this system and method and several alternative embodiments/improvements is an effective and powerful tool for determining multiple line features under different conditions. In general, these systems and methods, when used to find line features, do not place any particular limitation on the maximum number of lines found in an image. Only memory and computation time impose a practical limit on the number of lines that can be found.

The above is a detailed description of an exemplary embodiment of the present invention. Various modifications and additions may be made without departing from the spirit and scope of the present invention. Features of each of the various embodiments described above may be combined with features of other described embodiments, where appropriate, to provide multiple feature combinations in associated new embodiments. Further, while the above describes a number of distinct embodiments of the apparatus and method of the present invention, what has been described herein is merely illustrative of the application of the principles of the present invention. For example, as used herein, the terms "process" and/or "processor" should be broadly interpreted as including various electronic hardware and/or software-based functions and components (and alternatively functional "modules"). May be referred to as "or "element"). Furthermore, the illustrated process or processor may be combined with other processes and/or processors or may be divided into several sub-processes or processors. These sub-processes and/or sub-processors can be combined in several ways according to the embodiments disclosed herein. In addition, in the present specification, any function, process, and/or processor is expressly stated that it can be implemented using electronic hardware, software composed of a non-transitory computer-readable medium of program instructions, or a combination of hardware and software. Is considered. Additionally, as used herein, "vertical", "horizontal", "upward", "down", "lower", "upper", "side", "front", "rear", "left", Several orientation and arrangement terms such as "right" are used only as a relative convention, and are not used as absolute orientations/arrangements for a fixed coordinate space such as the direction of action of gravity. Additionally, the terms "substantially" or "approximately" when used for a given measurement, value, or characteristic, this term refers to an amount within the normal operating range that achieves the desired result, but this is the internal inaccuracies of the system Includes some variation due to errors in tolerance (eg, 1-5 percent). Accordingly, this description is to be construed as illustrative only, and is not intended to limit the scope of the invention.

Claims (23)

As a system for finding line features in an acquired image,
A vision system processor that receives image data of a scene including line features, comprising:
(a) calculating a gradient vector field from the image data,
(b) an operation of projecting the gradient vector field over a plurality of gradient projection sub-regions, and
(c) a vision system processor having an edge point extractor that performs an operation of finding a plurality of edge points in each of the gradient projection sub-zones based on the projected gradient data; And
And a line-finder for generating a plurality of lines consistent with edge points extracted from the image. The method of claim 1, wherein the line finder includes repetitively defining a line from outlier edge points with respect to previously defined lines, including new lines of inlier edge points. A system, characterized in that it operates a RANSAC-based process that fits. The system of claim 1, wherein the gradient vector field projection is oriented along a direction established in response to one or more or an expected orientation of the line features. The system of claim 1, wherein the gradient vector field projection defines a granularity based on a Gaussian kernel. The method of claim 1, wherein the edge point extractor is arranged to find a plurality of gradient size maxima in each of the gradient projected sub-zones, wherein the gradient size maxima is the plurality described by a position vector and a gradient vector. And each identified as part of an edge point of. The method of claim 1, wherein the line finder comprises, by calculating a metric, between at least one edge point among the extracted plurality of edge points and at least one candidate line among the found plurality of lines. Arranged to determine consistency, the metric is a distance of the at least one edge point from the at least one candidate line, a gradient direction of the at least one edge point, and a normal direction of the at least one candidate line System, characterized in that based on the difference in angle between. The system according to claim 1, wherein the image data includes data from a plurality of images acquired from a plurality of cameras and converted into a common coordinate space. The system of claim 1, further comprising a smoothing kernel to smooth the image data. 9. The system of claim 8, wherein the smoothing kernel comprises a Gaussian kernel. The system of claim 1, wherein the edge points are selected based on a threshold value defined by an absolute contrast and a normalized contrast based on an average intensity of the image data. The method of claim 1, wherein the line finder is configured to exchange at least some edge points from a plurality of extracted edge points representing portions of parallel lines or intersecting lines from the found plurality of lines to correct an erroneous orientation. System, characterized in that arranged. The system of claim 1, wherein the line finder is arranged to identify a line from a plurality of found lines having a polarity variation. 13. The system of claim 12, wherein the identified lines are lines defined by mixed polarities of line features of the found plurality of lines based on gradient values of the extracted plurality of edge points. As a method of finding line features in an acquired image,
Receiving image data of a scene with a vision system processor, the data including line features;
Calculating a gradient vector field from the image data;
Projecting the gradient vector field over a plurality of gradient projection sub-regions;
Finding a plurality of edge points in each of the gradient projection sub-zones based on projected gradient data; And
And generating a plurality of lines coincident with the edge points extracted from the image. 15. The method of claim 14, further comprising applying a RANSAC-based process of fitting normal edge points to new lines, including repetitively defining the line from outlier edge points for previously defined lines. How to do it. 15. The method of claim 14, wherein the gradient vector field projection is oriented according to a direction established in response to one or more or an expected orientation of the line features. 15. The method of claim 14, wherein the gradient vector field projection defines a particle size based on a 1D Gaussian kernel. The method of claim 14, further comprising finding a plurality of gradient magnitude maximums in each of the gradient projection sub-regions, wherein the gradient magnitude maximum is the plurality of edge points described by a position vector and a gradient vector. Method, characterized in that each identified as part of. The method of claim 14, further comprising determining a degree of correspondence between at least one edge point of the extracted plurality of edge points and at least one candidate line of the found plurality of lines by calculating a metric, , The metric is based on a distance of the at least one edge point from the at least one candidate line, and an angle difference between a gradient direction of the at least one edge point and a normal direction of the at least one candidate line How to do it 15. The method of claim 14, wherein the image data comprises data from a plurality of images acquired from a plurality of cameras and converted into a common coordinate space. 15. The method of claim 14, further comprising smoothing the image data using a smoothing kernel. 15. The method of claim 14, further comprising selecting the edge points based on a threshold value defined by an absolute contrast and a normalized contrast based on an average intensity of the image data. The method of claim 14, further comprising: (a) exchanging edge points representing parallel lines or portions of intersecting lines to correct an erroneous orientation, and (b) blending of lines based on a gradient value of the edge points. And identifying a line having a polarity variation of the line features comprising the polarity of the line.

Images