Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/12—Edge-based segmentation

Definitions

• the invention pertains to machine vision and, more particularly, to methods and apparatus for histogram-based analysis of images.
• Image segmentation is a technique for determimng the boundary of the object with respect to other features (e.g. , the object's background) in an image.
• One traditional method for image segmentation involves determining a pixel intensity threshold by constructing a histogram of the intensity values of each pixel in the image.
• the histogram which tallies the number of pixels at each possible intensity value, has peaks at various predominant intensities in the image, e.g. , the predominant intensities of the object and the background. From this, an intermediate intensity of the border or edge separating the object from the background can be inferred.
• the histogram of a back-lit object has a peak at the dark end of the intensity spectrum which represents the object or, more particularly, portions of the image where the object prevents the back-lighting from reaching the camera.
• the image intensity at the edges bounding the object is typically inferred to lie approximately half-way between the light and dark peaks in the histogram.
• a problem with this traditional image segmentation technique is that its effectiveness is principally limited to use in analysis of images of back-lit objects, or other "bimodal" images (i.e. , images with only two intensity values, such as dark and light).
• images of back-lit objects or other "bimodal" images (i.e. , images with only two intensity values, such as dark and light).
• the resulting images are not bimodal but, rather, show a potentially complex pattern of several image intensities.
• Performing image segmentation by inferring image intensities along the edge of an object of interest from the multitude of resulting histogram peaks is problematic.
• image characteristics along object edges are used by other machine vision tools to determine an object's location or orientation.
• Two such tools are the contour angle finder and the "blob" analyzer.
• the contour angle finder tracks the edge of an object to determine its angular orientation.
• the blob analyzer also uses an image segmentation thereshold to determine both the location and orientation of the object. Both tools are discussed, by way of example, in United States Patent No. 5,371,060, which is assigned to die assignee hereof, Cognex Corporation.
• An object of this invention is to provide improved methods and apparatus for machine vision analysis and, particularly, improved methods for determining characteristics of edges and edge regions in an image.
• a further object of the invention to provide improved methods and apparatus for determining such characteristics as predominant intensity and predominant edge direction.
• Still another object is to provide such methods and apparatus that can determine edge characteristics from a wide variety of images, regardless of whether the images represent front-lit or back-lit objects.
• Yet another object of the invention is to provide such methods and apparatus as can be readily adapted for use in a range of automated vision applications, such as automated inspection and assembly, as well as in other machine vision applications involving object location.
• Yet still another object of the invention is to provide such methods and apparatus that can execute quickly, and without consumption of excessive resources, on a wide range of machine vision analysis equipment.
• Still yet another object of the invention is to provide articles of manufacture comprising a computer readable medium embodying program code for carrying out such improved metiiods.
• the invention provides an apparatus for identifying a predominant edge characteristic of an input image that is made up of a plurality of image values (i.e. , "pixels") that contain numerical values representing intensity (e.g. , color or brightness).
• the apparatus includes an edge detector that generates a plurality of edge magnitude values based on the input image values.
• the edge detector can be a Sobel operator that generates the magnimde values by taking the derivative of the input image values, i.e. , the rate of change of intensity over a plurality of image pixels.
• a mask generator creates a mask based upon the values output by the edge detector.
• the mask generator can create an array of masking values (e.g. Os) and non- masking values (e.g., Is), each of which depends upon the value of the corresponding edge magnitude values. For example, if an edge magnimde value exceeds a tiireshold, the corresponding mask value will contain a 1, otherwise it contains a 0.
• the mask is utilized for masking input image values that are not in a region for which there is a sufficiently high edge magnitude.
• the mask is utilized for masking edge direction values tiiat are not in such a region.
• a mask applicator applies the pixel mask array to a selected image to generate a masked image.
• That selected image can be an input image or an image generated tiierefrom (e.g. , an edge direction image of the type resulting from application of a Sobel operator to the input image).
• a histogram generator generates a histogram of the pixel values in the masked image, e.g. , a count of the number of image pixels that pass through the mask at each intensity value or edge direction and that correspond with non-masking values of the pixel mask.
• a peak detector identifies peaks in the histogram representing, in an image segmentation thresholding aspect of the invention, the predominant image intensity value associated with the edge detected by the edge detector - and, in an object orientation determination aspect of the invention, the predominant edge direction(s) associated with the edge detected by the edge detector.
• Still further aspects of the invention provide methods for identifying an image segmentation threshold and for object orientation determination paralleling the operations of the apparatus described above.
• Still other aspects of the invention is an article of manufacture comprising a computer usable medium embodying program code for causing a digital data processor to carry out the above methods of edge-based image histogram analysis.
• the invention has wide application in industry and research applications.
• Those aspects of the invention for determining a predominant edge intensity threshold provide, among other things, an image segmentation threshold for use in other machine vision operations, e.g. , contour angle finding and blob analysis, for determining the location and orientation of an object.
• Those aspects of the invention for determining a predominant edge direction also have wide application in the industry, e.g., for facilitating determination of object orientation, without requiring the use of additional machine vision operations.
• Figure 1 depicts a digital data processor including apparams for edge-based image histogram analysis according to the invention
• Figure 2A illustrates the steps in a method for edge-based image histogram analysis according to the invention for use in identifying an image segmentation threshold
• Figure 2B illustrates the steps in a method for edge-based image histogram analysis according to the invention for use in determining a predominant edge direction in an image
• Figures 3 A - 3F graphically illustrate, in one dimension (e.g. , a "scan line"), the sequence of images generated by a metiiod and apparatus according to the invention
• Figures 4A - 4F graphically illustrate, in two dimensions, the sequence of images generated by a method and apparams according to the invention
• Figure 5 shows pixel values of a sample input image to be processed by a method and apparatus according to the invention
• Figures 6 - 8 show pixel values of intermediate images and of an edge magnimde image generated by application of Sobel operator during a method and apparams according to the invention
• Figure 9 shows pixel values of a sharpened edge magnitude image generated by a method and apparatus according to the invention
• Figure 10 shows a pixel mask array generated by a method and apparams according to the invention
• Figure 11 shows pixel values of a masked input image generated by a method and apparams according to the invention.
• Figure 12 illustrates a histogram created from the masked image values of Fig. 11.
• Figure 1 illustrates a system for edge-based image histogram analysis.
• a capturing device 10 such as a conventional video camera or scanner, generates an image of a scene including object 1.
• Digital image data (or pixels) generated by die capturing device 10 represent, in the conventional manner, the image intensity (e.g. , color or brightness) of each point in the scene at the resolution of the capturing device.
• That digital image data is transmitted from capturing device 10 via a communications path 11 to an image analysis system 12.
• This can be a conventional digital data processor, or a vision processing system of the type commercially available from the assignee hereof, Cognex Corporation, as programmed in accord with the teachings hereof to perform edge-based image histogram analysis.
• the image analysis system 12 may have one or more central processing units 13, main memory 14, input- output system 15, and disk drive (or other static mass storage device) 16, all of the conventional type.
• the system 12 and, more particularly, central processing unit 13, is configured by programming instructions according to teachings hereof for operation as an edge detector 2, mask generator 3, mask applicator 4, histogram generator 5 and peak detector 6, as described in further detail below.
• edge detector 2 mask generator 3
• mask applicator 4 histogram generator 5
• peak detector 6 peak detector 6
• Figure 2A illustrates the steps in a method for edge-based image histogram analysis according to the invention for use in identifying an image segmentation threshold, that is, for use in determining a predominant intensity of an edge in an image.
• Figures 3 A - 3F and 4A - 4F graphically illustrate the sequence of images generated by the method of Figure 2 A.
• the depiction in Figures 4A - 4F is in two dimensions; that of Figures 3A - 3F is in "one dimension," e.g., in the form of a scan line.
• Figures 5 - 12 illustrate in numerical format the values of pixels of a similar sequence of images, as well as intermediate arrays generated by the method of Figure 2 A.
• a scene including an object of interest is captured by image capture device 10 (of Figure 1).
• image capture device 10 of Figure 1.
• a digital image representing the scene is generated by the capturing device 10 in the conventional manner, with pixels representing the image intensity (e.g. , color or brightness) of each point in the scene per the resolution of the capturing device 10.
• the captured image is referred to herein as the "input" or "original” image.
• the input image is assumed to show a dark rectangle against a white background. This is depicted graphically in Figures 3A and 4A. Likewise, it is depicted numerically in the Figure 5, where the dark rectangle is represented by a grid of image intensity 10 against a background of image intensity 0.
• step 22 the illustrated metiiod operates as an edge detector, generating an edge magnitude image with pixels that correspond to input image pixels, but which reflect the rate of change (or derivative) of image intensities represented by the input image pixels.
• edge detector step 22 utilizes a Sobel operator to determine the magnitudes of those rates of change and, more particularly, to determine the rates of change along each of the x-axis and y-axis directions of the input image. Use of the Sobel operator to determine rates of change of image intensities is well known in the art.
• the rate of change of the pixels of the input image along the x-axis is preferably determined by convolving that image with the matrix:
• Figure 6 illustrates the intermediate image resulting from convolution of the input image of Figure 5 with the foregoing matrix.
• the rate of change of the pixels of the input image along the y-axis is preferably determined by convolving that image with the matrix:
• Figure 7 illustrates the intermediate image resulting from convolution of the input image of Figure 5 with the foregoing matrix.
• the pixels of the edge magnimde image e.g. , of Figure 8 are determined as the square-root of the sum of the squares of the corresponding pixels of the intermediate images, e.g. , of Figures 6 and 7.
• the magnitude represented by the pixel in row 1 /column 1 of the edge magnimde image of Figure 8 is the square root of the sum of (1) the square of the value in row 1/column 1 of the intermediate image of Figure 6, and (2) the square of the value in row 1/column 1 of the intermediate image of Figure 7.
• step 23 the illustrated method operates as peak sharpener, generating a sharpened edge magnimde image that duplicates the edge magnimde image, but with sha ⁇ er peaks.
• the peak sha ⁇ ening step 23 strips all but the largest edge magnimde values from any peaks in the edge magnimde image.
• sha ⁇ ened edge magnimde images are shown in Figure 3C (graphic, one dimension), Figure 4C (graphic, two dimensions) and Figure 9 (numerical, by pixel).
• the sha ⁇ ened edge magnimde image is generated by applying the cross-shaped neihborhood operator shown below to the edge magnimde image. Only those edge magnimde values in the center of each neighborhood that are the largest values for the entire neighborhood are assigned to the sha ⁇ ened edge magnimde image, thereby, narrowing any edge magnimde value peaks.
• the illustrated method operates as a mask generator for generating a pixel mask array from the sha ⁇ ened edge magnimde image.
• the mask generator step 24 generates the array with masking values (e.g. Os) and non-masking values (e.g. , Is), each of which depends upon the value of a corresponding pixel in the sha ⁇ ened edge magnimde image.
• masking values e.g. Os
• non-masking values e.g. , Is
• the mask generator step 24 is tantamount to binarization of the sha ⁇ ened edge magnimde image.
• the examples shown in the drawings are labelled accordingly. See, the mask or binarized sha ⁇ ened edge magnimde image shown in Figures 3D, 4D and 10.
• the threshold value is 42.
• the peak sha ⁇ ening step 23 is optional and that the mask can be generated by directly “binarizing" the edge magnimde image.
• the illustrated method operates as a mask applicator, generating a masked image by applying the pixel mask array to the input image to include in the masked image only those pixels from the input image that correspond to non-masking values in the pixel array.
• the pixel mask array has the effect of passing through to the masked image only those pixels of the input image that are in a region for which there is a sufficiently high edge magnimde.
• a masked input image is shown in Figure 3E (graphic, one dimension), Figure 4E (graphic, two dimensions) and Figure 11 (numerical, by pixel).
• Figure 11 particularly, reveals that the masked image contains a portion of the dark square (the value 10) and a portion of the background (the value 0) of the input image.
• the illustrated method operates as a histogram generator, generating a histogram of values in the masked image, i.e. , a tally of die number of non ⁇ zero pixels at each possible intensity value that passed from the input image through the pixel mask array.
• a histogram is shown in Figures 3F, 4F and 12.
• the illustrated metiiod operates as a peak detector, generating an output signal representing peaks in the histogram.
• those peaks represent various predominant edge intensities in the masked input image.
• This information may be utilized in connection with other machine vision tools, e.g. , contour angle finders and blob analyzers, to determine the location and/or orientation of an object.
• a software listing for a preferred embodiment for identifying an image segmentation threshold according to the invention is reprinted below.
• the program is implemented in the C programming language on the UNIX operating system.
• Figure 2B illustrates the steps in a method for edge-based image histogram analysis according to the invention for use in determining object orientation by determining one or more predominant directions of an edge in an image. Where indicated by like reference numbers, the steps of the method shown in Figure 2B are identical to tiiose of Figure 2 A.
• the method of Figure 2B includes the additional step of generating an edge direction image with pixels that correspond to input image pixels, but which reflect the direction of the rate of change (or derivative) of image intensities represented by the input image pixels.
• step 28 utilizes a Sobel operator of the type described above to determine the direction of those rates of change. As before, use of the Sobel operator in this manner is well known in the art.
• mask applicator step 25 of Figure 2 A generates a masked image by applying the pixel mask array to the input image
• the mask applicator step 25' of Figure 2B generates the masked image by applying the pixel mask array to the edge direction image.
• the output of mask applicator step 25' is a masked edge direction image (as opposed to a masked input image). Consequently, the histogram generating step 26 and the peak finding step 27 of Figure 2B have the effect of calling out one or more predominant edge directions (as opposed to the predominant image intensities) of the input image.
• the orientation of the object bounded by the edge can be inferred.
• the values of the predominant edge directions can be inte ⁇ reted modulo 90-degrees to determine angular rotation.
• the values of the predominant edge directions can also be readily determined and, in turn, used to determine the orientation of the object.
• the orientation of still other regularly shaped objects can be inferred from the predominant edge direction information supplied by the method of Figure 2B.
• a further embodiment of the invention provides an article of manufacmre, to wit, a magnetic diskette (not illustrated), composed of a computer readable media, to wit, a magnetic disk, embodying a computer program that causes image analysis system 12 ( Figure 1) to be configured as, and operate in accord with, the edge-based histogram analysis apparams and method described herein.
• a diskette is of conventional construction and has the computer program stored on the magnetic media therein in a conventional manner readable, e.g. , via a read/write head contained in a diskette drive of image analyzer 12.
• # ifndef c_ansiprot_h include ⁇ ansiprot.h> #endif
• #ifndef c_cip_peak_h .. include ⁇ edgedet/cip_peak.h> /* cip_peak_params */ #endif
• the Sobel Threshold is such an intensity that
• cip_edge_params * edge_params_p /* input to cip_sobel_init() */ cip_peak_params * peak_params_p; /* a part of cip_edge_params */ cip_edge_data * edge_data_p; /* output from cip_sobel_init() +/ /* input to cip_sobel(). Need not */ /* be allocated by a user */ cip_edge_results * edge_results_p; /* output from cip_sobel() */ int internal; /* internal use */ ⁇ ceet data;
• threshold e.g. the Sobel
• VC2 is used for Sobel operations and VCl (or VC3) is
• ceet_create() can be modified. (See cip_edge.h and cip_peak.h for
• ceet_delete PROTO (ceet_data * thresh_data)
• ceet_data * thresh_data; /* ceet_data pointer */ thresh_data ceet_create(NULL); /* when ceet_create() is called */
• thresh_data->peak_params_p is NULL
• # include ⁇ cvc.h> #endif
• # include ⁇ cvc_def.h > #endif
• #ifndef c_cct_h ⁇ include ⁇ cct.h > /* CGEN ERR BADARG - via cgen er.h */ #endif #ifndef c_cgr_zoom_h #include ⁇ cgr_zoom.h> /* scroll/zoom graphics */
• cip_buffer *cip_minmax PROTO ((cip_buffer*,cip_buffer*,cip_buffer*, cip_buffer*, unsigned char *, cip_minmax_mode, int *, int *)); /* * If VC2 is not available, cip_minmax() is not called because of its * incompatible results. Instead, cip_min() is used. */
• peak_params_p (cip_peak_params *)cvc_alloc(sizeof(cip_peak_params)) ;
• edge_results_p edge_results */
• edge_params_p 159 if(!td- > edge_params_p) /* edge_params */
• edge_params_p (cip_edge_params *) cvc_alloc(sizeof(cip_edge_params));
• edge_params_p edge_params_p; 167
• edge_params_p- > num_bits_xy * * 7 ;
• edge_params_p- > xy_compress_type CIP EDGE MAP LINEAR;
• edge_params_p- > mag_compress_type CIP EDGE MAP LINEAR; 174
• edge_params_p- > output_type CIP_EDGE_OUTPUT_MAG
• edge _p arams_p- > xy_compress_map_p NULL;
• mapping_p NULL
• edge_params_p- > peak_params_p td- > peak_paramsj_;
• edge_data_p cip_sobel_init(td- > edge_params_p);
• ttable is a threshold map table ⁇ 0,0,0,...,0,255,...,255.255 ⁇
• map is a subset in ttable such that
• 331 tablet tp->diag_context->tablet
• mag_image td->edge_results_p->mag_img_p;
• 390 limit tp->sobel_top * (mag_image->width * mag_image->height - hist[0]); 391
• 396 map &ttable[256-threshold]; /* starting address of a map array */ 397
• bin_image cip_create(mag_image->width,mag_image->height,8);
• zoom cgrz_get_zoom (tablet);
• zoom cgrz_get_zoom (tablet); 554
• ceet_test_bIob PROTO ( ceet_params*, ceet_demo.c
• win_src ceet_get_image(&tablet, ctb3, &x_offset, &y_offset);
• bin_image cip_create(win_original- > width, win original- > height, 8) ;
• static void place_box PROTO ((cgrz_tablet * t, box * w,ctb3_obj * ctb3_p)); static void draw_box PROTO ((cgrz_tablet * t, box * w, int color));
• do_pe 1 values 1 ;
• img_copy cip_create(img- > width, img- > height, 8);
• CDB_ERR_BADFD printf("Bad file name (%s)",infile); break;
• CDB_ERR_OPEN printf("Bad drive name (%s)",infile); break;
• CDB_ERR_EOF printf(" Passed end of file (rec # %d)",rec); break;

Landscapes

• Engineering & Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Image Analysis (AREA)
• Image Processing (AREA)
• Length Measuring Devices By Optical Means (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=24327330

Family Applications (1)

Country Status (3)

Cited By (10)

Families Citing this family (42)

Citations (3)

Family Cites Families (30)

• 1996
  • 1996-01-02 US US08/581,975 patent/US5845007A/en not_active Expired - Lifetime
  • 1996-12-31 WO PCT/US1996/020900 patent/WO1997024693A1/en not_active Ceased
  • 1996-12-31 JP JP52461897A patent/JP4213209B2/ja not_active Expired - Lifetime

Patent Citations (3)

Also Published As

Similar Documents

Legal Events