WO2000034918A1

WO2000034918A1 - Interactive edge detection markup process

Info

Publication number: WO2000034918A1
Application number: PCT/US1999/028778
Authority: WO
Inventors: Jean-Pierre Schott
Original assignee: Synapix, Inc.
Priority date: 1998-12-11
Filing date: 1999-12-06
Publication date: 2000-06-15
Also published as: AU2041600A; WO2000034918A9

Abstract

A technique is disclosed for iteratively and interactively identifying edges in a graphic image editing system. The system uses a variable contrast edge detection threshold to first find only strong sharp edges, then uses a lower threshold to examine areas indicated by the user for weaker edges missed during the first pass. The system can also fill in gaps in identified edges using subpixel resolution with parabolic interpolation and line direction and end point separation. With such a system the user time and the computer time required to identify and capture an image on a computer vision system may be reduced, and the accuracy of the captured image may be increased to subpixel levels.

Description

INTERACTIVE EDGE DETECTION MARKUP PROCESS

BACKGROUND OF THE INVENTION

The television, movie, video game, computer graphics, industrial design and architecture fields all have a need to analysis and manipulate images. Special effects may add a new dimension to movies and television shows, and the ability to effectively rotate a flat architectural drawing may help observers to better visualize what the finished three dimensional structure will look like.

To make such image analysis and manipulation techniques economically viable, it is beneficial to have an automated system identify and mark the edges of objects in the image field. Automated image analysis techniques in the current art use a digitized stream of image data points. These image data points are typically in the form of what are known in the art as pixels. Each pixel contains the data indicating a gray level, in the case of a black and white image, or color intensity levels, in the case of color images. The automated image system analyzes and manipulates the image by grouping the pixels together in predefined ways. For example, the automated image system may create an abstraction of the image in the form of a wire frame or set of edge boundaries. The abstraction of the image may then be more easily mathematically manipulated due to its greater simplicity.

There exists a problem in the automated acquisition of images due to the large number of potential edges found in images having non uniform lighting, resulting in the misidentification of non edges. If the user sets the contrast edge detection threshold to a low level, many weak edges maybe found in addition to strong sharp edges. Some of the edges found in the low contrast threshold case will not be actual edges of objects, but will be due to shadows, lighting irregularities and variations in texture and color of the object. The large number of edges found in this case may require a great deal of manual user time to eliminate the unwanted spurious edges.

There is also exists a problem of missing correct edges because of low edge contrast and because of edges being partly obscured by intervening objects. If the user sets the contrast edge detection threshold to a high level, only sharp strong edges will be found, and some real object edges may be missed. This situation also results in the expenditure of a great deal of manual user time to add in the missing edges. There is another problem in the art of discontinuous object edges. An obstruction or a shadow may cause the edge detector to find two disconnected edges rather than a single continuous edge. Some of these discontinuities may be so small that the user may not notice, but the discontinuities may cause problems during image processing. For example, if a user wishes to shade a particular object in the image, typically they would select the desired object in the image field and initiate an automatic fill operation. If the object has a discontinuity somewhere in what appears to be the edge, even an effectively invisible gap, then the shading will spill over into an area that is not within the desired object.

SUMMARY OF THE INVENTION Thus the image processing art has a problem with the large number of supposed edges found when using a low contrast threshold value. There is a problem with missing many real edges when using a high contrast threshold value. In both cases there is a problem with discontinuities in the edges found. It is not possible to simply find the correct contrast threshold since each image is different, and within each image there are areas that will require different contrast thresholds for optimum edge detection. Thus there exists a need in the art for a rapid and accurate method to analyze an image, whether real or synthetic, and correctly identify the edges of the objects in the image.

In a preferred embodiment of the invention, a system is described for an iterative and interactive precision edge detection process for a computer image processing system that has a variable contrast edge detection threshold. The system first scans the image with a high contrast threshold, producing only a few strong edges. The initial edge detection may be flawed due to shadows in the image that locally weaken the edge contrast, or due to part of the edge being obscured by an intervening object. The user then identifies either the correct ones of the detected strong edges, or indicates the correct region for the computer to reexamine, using either roto-splines or free hand scribbles. The oriented edge detector estimates the direction of the edge and the precise subpixel edge location by computing a parabolic inteφolation of the edge gradient magnitude value in the direction peφendicular to the edge tangent. The parabolic inteφolation uses the gradient magnitude of the current edge pixel and the two neighboring pixels on either side of the edge in the peφendicular direction. If the direction is not a multiple of 45 degrees, the value of the neighboring pixel magnitude can be obtained by bilinear inteφolation of neighboring pixel magnitude values. The edge detector displays the results of the next estimation done with a lower contrast threshold, allowing weaker edges to be found. The user again indicates which of the detected edges is the correct edge or again identifies the region in which to search further. The edge detector repeats the process in the newly defined region with a lowered contrast threshold. This iterative and interactive process continues until all of the correct edges are identified.

In a further embodiment of the invention, gaps between identified edges are automatically filled in with a best guess curve fit by examining all the edges in the indicated region, and matching the two edges that have the best combination of the longest segment length, the closest endpoints and the closest slope. With such a process the large number of computer detected edges found when using a low enough threshold to assure detection of the desired edge, may be greatly reduced, and the time required to identify the correct edges may be greatly reduced.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. lis a representation of pixel intensity.

Fig. 2 is a bar graph of pixel intensity at an edge.

Fig. 3 is a drawing of a scribble.

Fig. 4 is a drawing showing discontinuities.

Fig. 5 is a flow chart showing the interactive method according to the invention. The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Referring to Fig. 1, an image field has been analyzed into an 1 1 by 13 array 10 of pixels, with each of the 130 pixels having a typical luminosity value on the scale of 1 to 256. The image field shown in pixel array 10 contains an edge 12, which divides the image field into two parts in this illustrative example, a generally brighter part 14 having a typical luminosity value of 180 out of 256, and a generally duller part 16 having a typical luminosity value of 100. Edges such as 12 have a different luminosity than the surrounding regions 14 and 16. In the illustrative example shown, the edge 12 has lower luminosity than either of the two surrounding regions 14 or 16. In other cases the edge 12 might have a higher luminosity than the surrounding regions, the direction of the ambient light having a major effect on the direction of edge luminosity.

It is apparent in Fig. 1 that the actual edge 12 does not equally affect all the pixels that it crosses, since pixels containing a long segment of edge 12, such as the pixel labeled 18, will have a very low luminosity in this illustrative example, whereas pixels such as 20, which have only a short segment of edge 12 will have luminosity similar to the adjacent pixels having no segment of edge 12. Thus it is clear that using the low valued pixels to determine the location of edge 12 would result in a non smooth and discontinuous line because pixels such as 20 in this example would not be low enough luminosity to be considered as part of the edge 12. It would therefore be beneficial to have some measure of where an edge such as 12 crosses a particular pixel. It would also be beneficial to have a method to connect line segments together whenever the identification of an edge is interrupted by a bright pixel such as 20. Referring now to Fig. 2, a series of adjacent pixels 30 to 42 are shown, each having a gradient magnitude indicated by the height of the bar. In this illustrative example the edge is brighter than the surrounding regions and the edge is somewhere in pixel 34. Using an inteφolation, for example a parabolic function, the true position of the peak illumination, and therefore in this example the location of the peak, may be determined with subpixel resolution. In the illustrative example the location of the true edge is about 40% of the way from the center of pixel 36 toward pixel 38. This peak location provides an estimate of the edge location in units of measure which are smaller than a single pixel, and is stored in memory for use in future edge calculations.

Referring now to Fig. 3, an object 50 in an image field is shown. In this illustrative example, the edges of object 50 are presumed to have had too low a contrast for the edge detector to have found an edge. In other words, object 50 was not seen by the vision system. The user draws a free hand line, known as a scribble, such as dashed line 52 around the area where the user desires the edge detector to look again for the object 50, but with a lower edge contrast threshold detection level. The edge detector asymmetrically fattens up the user drawn line and creates an inner line 54, typically 5 pixels inside of line 52, and an outer line 56, typically 6 pixels outside of line 52. This creates a toroid shape, and the edge detector looks for edges within the toroid with greater sensitivity, thereby improving the chances of finding the edges of object 50.

Referring now to Fig. 4, an object 60 is shown which has, in this illustrative embodiment, had a discontinuity in the real edge of the object. Thus the right hand edge of object 60 as detected consists of line 62 and line 64. The edge detector in this example has also found two spurious edges, lines 66 and 68. The problem is to connect the correct two lines, namely 62 and 64 together. This is done by having the user indicate the area to be reexamined by means of a scribble as was discussed above, or by means of a formula for a known curve, known as a roto- spline. The edge detector then looks at all detected edges within the toroid area, as was done above with reference to Fig. 3, and determines the endpoint locations 72- 78. The edge detector then measures the average slopes of the lines, and measures the length of each of the lines. The edge detector connects the two endpoints with are a combination of the closest together, have the closest slopes and connect the two longest lines.

Referring now to Fig. 5, a flow chart of the steps of the interactive and iterative edge detection process which may be performed in an image processing system such as Silicon Graphics Octane Workstation, or N.T. workstation. After acquiring the image in step 80, typically a data stream of pixels, the image is displayed on some form of user output device, typically a computer screen. The graphics workstation provides the user with a tool that permits the user to mark the image with either a free form curve such as a scribble, or with a calculated roto- spline. The user thus may either manually mark the image or let the system attempt to find the edges of the objects in the image automatically without user input. The edges are preferably specified using the pixel peak location technique discussed in connection with Fig. 2. In step 84, the system proceeds to create an outline around any scribbles that the user may have made with a toroid shape of a thickness controlled by the user, typically 5 to 6 pixels in width. The toroid thickness is determined by the amount that will capture the desired object edge without including overly much of the surrounding image.

The toroid and image then go through edge detection process 86, using an initial edge contrast threshold value predetermined by the user, typically a high value such as 10 gray scale levels change per pixel. The edge detector 86 highlights the found edges and sends the data to the user screen in state 88, where the user decides if the image has been correctly processed. If the image edge detection is not good enough, the user lowers the edge contrast detection threshold in state 92, typically to one gray scale change per pixel, and goes back to the free form curve tool in state 82 to mark missing edges and delete extraneous edges. This process of interaction between the edge detection system and the user continues iteratively until the user accepts the image edged detection and ends the process in state 94. It should be understood that the process flow chart could also be implemented with hardware designed to perform the tasks described and therefore the invention encompasses apparatus and should not be limited to only the disclosed process.

While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

CLAIMSWhat is claimed is:

1. A method for interactive edge detection in an automated image analysis system comprising the steps of: defining an edge contrast threshold level, examining said image using said threshold level and marking all image edges having a greater contrast level than said threshold level; identifying and marking incorrect edges for removal; identifying a region for reexamination by drawing an annulus around said region; reducing said edge contrast threshold level by a predetermined amount and reexamining said image in said annulus at said reduced threshold level and marking all image edges having a greater contrast level than said reduced threshold level; and repeating steps b to d, until correct edges have been identified.

2. The method of claim 1 wherein said marking of edges further comprises the steps of: marking the location of the edge at a subpixel resolution; selecting the pixel having a local maximum gradient magnitude, examining the gradient magnitude of a predetermined number of adjacent pixels on both sides of said local maximum; and determining the subpixel location of a peak of the local maximum by inteφolation of the gradient magnitude of the adjacent pixels.

3. The method of claim 2 wherein said inteφolation uses a parabolic method.

4. The method of claim 1 wherein said step of identifying a region for reexamination by drawing an annulus around said region further comprises using a roto-spline.

5. The method of claim 1 wherein said step of identifying a region for reexamination by drawing an annulus around said region further comprises using a free form curve.

6. The method of claim 1 wherein said annulus has a predetermined number of pixels of line width on the inside and outside of said annulus.

7. The method of claim 1 wherein said step of defining an edge contrast threshold level further comprises a threshold of 10 gray levels per pixel.

8. The method of claim 1 wherein said step of reducing said edge contrast threshold level further comprises a threshold of 1 gray level per pixel.

9. The method of claim 1 wherein said step of reexamining said image in said annulus includes filling gaps between lines marked as edges, comprising the steps of: examining said annulus for the longest image edge segments; determining the difference in average direction of said longest edge segments; determining the distance between endpoints of said longest edge segments; and connecting the two image edge segments having the longest edges and the least difference in average direction and endpoint difference.

10. An apparatus for interactive edge detection in an automated image analysis system comprising: means for defining an edge contrast threshold level; means for examining said image using said threshold level; means for marking all image edges having a greater contrast level than said threshold level; means for identifying and marking incorrect edges for removal; means for identifying a region for reexamination by drawing an annulus around said region; means for reducing said edge contrast threshold level by a predetermined amount; means for reexamining said image in said annulus at said reduced threshold level; and means for marking all image edges having a greater contrast level than said reduced threshold level.

11. The apparatus of claim 10 wherein said means for marking of edges further comprises: means for marking the location of the edge at a subpixel resolution; means for selecting the pixel having a local maximum gradient magnitude; means for examining the gradient magnitude of a predetermined number of adjacent pixels on both sides of said local maximum; and means for determining the subpixel location of a peak of the local maximum by inteφolation of the gradient magnitude of the adjacent pixels.

12. The apparatus of claim 11 wherein said inteφolation uses a parabolic method.

13. The apparatus of claim 10 wherein said means for identifying a region for reexamination by drawing an annulus around said region further comprises using a roto-spline.

14. The apparatus of claim 10 wherein said means for defining an edge contrast threshold level further comprises a threshold of 10 gray levels per pixel.

15. The apparatus of claim 10 wherein said means for reducing said edge contrast threshold level further comprises a threshold of 1 gray level per pixel.

16. The apparatus of claim 10 wherein said means for reexamining said image in said annulus includes means for filling gaps between lines marked as edges, comprising: means for examining said annulus for the longest image edge segments; means for determining the difference in average direction of said longest edge segments; means for determining the distance between endpoints of said longest edge segments; and means for connecting the two image edge segments having the longest edges and the least difference in average direction and endpoint difference.