JP2006268339A - Method for generating list of connected elements of image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an efficient method for extracting connected elements corresponding to an object in an image and arranging the image of the connected elements according to a feature quantity of the connected elements. <P>SOLUTION: The method for generating a list of connected elements comprises a step for converting a grayscale image into a binary image, a labeling step for generating a label image with a certain number for each of the connected elements assigned to pixels of the connected elements on the binary image, a window framing step for setting up a minimum rectangle enclosing each of the connected elements with reference to the label image, a feature extracting step for calculating the feature quantity of each of the connected elements, a sorting step for sorting the connected elements according to the feature quantity, and a drawing step for generating a list image of the connected elements to display a list of images of the connected elements in the sorted order. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing technique, and more particularly to a technique for creating a list of connected components of an image corresponding to an object in an image in a method for creating a list of image connected components using a computer.

  In general, it is often difficult to manually analyze, count, and classify an image having a large number of objects. In order to solve this problem, it is conceivable to let the computer support the work.

  In the case where the shape and size of an object is strongly limited, such as a medicine tablet, a method of automatically counting by a computer is known (for example, see Patent Document 1 and Patent Document 2).

JP 09-231342 A Japanese Unexamined Patent Publication No. 07-282219

  However, processing for objects with complex shapes has not been developed yet.

  For example, factories and dealers that handle food constantly investigate pests for hygiene management. Conventionally, paper coated with an adhesive, so-called insect trap paper, is placed in a predetermined place for a certain period, and insects caught by the trap insect paper are classified manually. Such an operation for classifying insects takes time, and since there are large individual differences in experience and ability, there has been a problem that the results vary. Therefore, it has been desired to semi-automate the classification work using a computer.

  There is a similar problem in traffic volume observation using satellite images. The traffic volume observation work is an effective means for reducing traffic congestion by grasping traffic volume and giving information on traffic congestion status to each vehicle to avoid traffic congestion. Traffic volume observation using satellite images has the advantage of lower costs compared to the method of observing traffic volume by installing sensors on the road, and its realization is expected from various directions. However, it is impossible with the current technology to detect a vehicle from a satellite image fully automatically. Therefore, it is necessary to manually analyze the satellite image. However, since it is difficult to directly analyze the image, it is desired to support the image analysis by some method.

  An object of the present invention is to obtain an efficient method for extracting a connected component corresponding to an object in an image even for an object having a complicated shape and arranging an image of the connected component according to a feature amount of the connected component.

  If there are many objects in the image, the operator can identify the objects by extracting individual objects from the image and displaying them side by side according to appropriate criteria, such as to avoid background effects. Is effective. For example, insect trapping paper is photographed with a camera, the image is processed with a computer, and the insects are arranged in order of size, so that the insect classification work efficiency is improved.

  FIG. 1 is an image obtained by photographing insect trapping paper. Many insects of various sizes are captured on the trapping paper. FIG. 3 is a diagram in which the insects on the trapping paper are cut out and arranged in order of increasing area occupied by the insects. As you go in the direction of the arrow, the insect area decreases. Rather than using FIG. 1 or binarized FIG. 2, using FIG. 3 can more efficiently classify insects.

  In addition, the same processing can be performed on the satellite image to facilitate traffic volume observation. FIG. 4A shows a satellite image to be processed, and the number of vehicles in this image is counted. FIG. 4 (c) is an image in which objects in a satellite image are arranged in order of increasing area. More specifically, the area of the object becomes narrower in the direction indicated by the arrow. This image contains objects such as traffic signs and lane fragments in addition to the vehicle. If the operator uses FIG. 4 (c), FIG. 4 (a) or binarized FIG. 4 (b) ) Can be selected and counted from the object shape more easily.

  More specifically, the present invention generates a label image in which a grayscale image is converted into a binary image, and a label image in which a number determined for each connected component is assigned to pixels of the connected component on the binary image. A labeling step, a window frame step for setting a minimum rectangle surrounding each connected component with reference to the label image, a feature extracting step for calculating a feature amount of each connected component, and a sort for sorting the connected components according to the feature amount A connected component list creating method comprising: a step; and a drawing step for generating a connected component list image that displays a list of connected component images in sorted order.

In the window frame step, first, for each connected component i, a sufficiently large initial value is set for the variable x ⁱ _min , a sufficiently small initial value is set for the variable x ⁱ _max, and a sufficiently large initial value is set for the variable y ⁱ _min. If the variable y ⁱ _max is set to a sufficiently small initial value, and then the label image is raster scanned and the current position (x, y) is on the connected component i, x is smaller than x ⁱ _min ^when, assigns the value of x to _{x i min,} when x is greater than ^x _{i ^max,} assigns the value of x to _{x i max,} when y is smaller than ^y _{i ^min,} the value of y to _{y i min} assignment, and when y is greater than y ⁱ _max, by connected component list generation method characterized by setting the minimum rectangle surrounding each connected component by substituting the value of y to y ⁱ _max.

  FIG. 5A is a diagram showing the image to be processed by shading. The numbers shown in FIG. 5A indicate the pixel density. FIG. 5B shows a binary image obtained by binarizing a grayscale image with a threshold value of 5. That is, pixels with a density of 5 or more in the grayscale image are converted to regions, and pixels less than 5 are converted to the background. The number 1 in the binary image is a region pixel, and the number 0 means a background pixel. In the case where the object corresponds to an area where the density of the grayscale image is low as shown in FIG. 1, binarization is performed after the grayscale image is converted into an image with inverted density. There are three connected components in the binary image of FIG. For the connectivity of the pixels, what is defined by so-called 8-connection that takes into account the diagonal direction as well as the top, bottom, left, and right of the pixel of interest is adopted. FIG. 5C shows a label image obtained as a result of labeling processing performed on a binary image (see Non-Patent Document 1).

Nobuyuki Yagi and 7 others: Practical image processing learned in C language, Chapter 7, pp.95-108: Ohmsha (1992)

  Labeling is a process of assigning the same number to connected pixels and assigning different numbers to separated pixels. The value of the background pixel in the label image is set to 0, and the pixels of the three connected components are numbered for each connected component with numbers from 1 to 3. FIG. 5D is a diagram showing a state in which a window frame is set for each connected component in the label image. First, “window frame” means the smallest rectangle surrounding the connected component, and the position of the window frame is defined by the coordinates of the upper left pixel and the lower right pixel of this rectangle. The window frame 501 is a window frame related to a connected component with a number (number) of “1”, window frame 502 “2”, and window frame 503 “3”. Next, the feature amount of the connected component is extracted. The feature amount includes various feature amounts such as an area, a perimeter, and a circularity, but here, the area is targeted as the feature amount.

  The areas of the connected component 1 (window frame 1), the connected component 2 (window frame 2), and the connected component 3 (window frame 3) are 8 pixels, 6 pixels, and 9 pixels, respectively. FIG. 5 (e) is an image in which connected components are arranged in order of area from left to right and from top to bottom. That is, the connected component 3 is arranged at the upper left in the image, the connected component 1 is arranged at the right, and the connected component 2 is arranged at the lower left. The value of each connected component pixel represents the density (luminance) in the original grayscale image.

  According to the method of extracting connected components corresponding to objects in an image of the present invention and arranging images of the connected components according to the feature quantities of the connected components, the operator can easily count or classify the objects. become able to. Further, by using a window frame, it is possible to extract information on connected components at high speed.

  Hereinafter, a connected component list creation method according to an embodiment of the present invention will be described. FIG. 6 is a functional block diagram showing an example of an image processing system for creating a connected component list according to an embodiment of the present invention. The image processing system according to the present embodiment includes an input device, an output device, a processing device, and a storage device. The input device is, for example, a keyboard 601 or a mouse. The keyboard 601 and the mouse are used for parameter input and command activation. The output device is provided with an LCD display 615, for example. The display 615 is used for displaying various images and confirming data processing. The processing device 602 includes a binarization module 603, a labeling module 604, a window frame module 605, an area calculation module 606, a sorting module 607, and a drawing module 608, for example, six modules.

  The storage device 609 is a grayscale image 610 that is an input of the binarization module 603 and the drawing module 608, a binary image 611 that is an input of the labeling module 604 at the output of the binarization module 603, and an output of the labeling module 604. Each storage area stores a label image 612 that is an input of the window frame module 605, a work image 613 that is an output of the drawing module 608, and an area array 614 that is an output of the window frame module 605.

  FIG. 7 is a flowchart showing a detailed flow of processing in the window frame module 605 of FIG. This flowchart includes an array r called a region array. Element r [i] of region array r is a structure having information about connected component i, and includes six members of id, xmin, xmax, ymin, ymax, and s, as shown in FIG. 9B. . id is an identifier of the connected component i, and xmin, xmax, ymin, and ymax specify a window frame surrounding the connected component i. s represents the area of the connected component i.

  As shown in FIG. 7, the processing in the window frame module 605 according to the present embodiment is 1) processing from step 701 to step 708, and a processing part for setting an initial value in the region array r, and 2) step 709. And the process from step 724 to step 724, where the window frame information is set in the area array r. In the first half of the flowchart (1) above, if the number of connected components in the label image is nRegion, the variable i is changed from 1 to nRegion, and in step 704, the member id of the element r [i] of the region array is changed. i, an appropriate large number bigN for member xmin in step 705, an appropriately small number smallN for member xmax in step 706, an appropriately large number bigN for member ymin in step 707, and an appropriate number yN for member ymax in step 708 Substitute small numbers smallN respectively.

  The x-axis direction of the image is from left to right on the paper surface, and the y-axis direction is from top to bottom on the paper surface. The size of the image in the x-axis direction is represented by W, and the size in the y-axis direction is represented by H. In the latter half of the flowchart, in step 709 to step 714, the label image is scanned from left to right and from top to bottom. In step 715, the value of the pixel of the label image at coordinates (x, y) is substituted into variable i. In order to check whether or not this pixel is a connected component pixel, it is determined in step 716 whether or not the value of the variable i is zero. If the value of the variable i is 0, since it is not a connected component pixel, the process proceeds to step 714 to repeat scanning. If the value of the variable i is not 0, the pixel is a connected component, and the process proceeds to step 717. Therefore, it is determined whether or not the member xmin of the element r [i] of the region array is larger than x. If it is larger (YES), x is substituted into the member xmin in Step 718. In step 719, it is determined whether or not the member xmax is smaller than x. If it is smaller (YES), x is substituted into the member xmax in step 720. A similar process is performed in steps 721 to 724 for the member ymin and the member ymax. At the end of scanning, the member xmin of the element r [i] of the region array is the value of the left end x coordinate of the connected component i, the member xmax is the value of the right end x coordinate, and the member ymin is the value of the upper end y coordinate. , Member ymax has the value of the y-coordinate at the lower end. The use of the algorithm represented by the flowchart according to the present embodiment has an advantage that the processing can be performed only by scanning the label image once, so that it is extremely efficient.

  FIG. 8 is a flowchart showing details of processing in the area calculation module 606 of FIG. This flowchart includes a loop for variable i from step 801 to step 803, a loop for variable y from step 805 to step 807, and a loop for variable x from step 808 to step 810. In the first loop, the variable i represents the connected component and varies from 1 to nRegion. In the latter two loops, the variable y changes from r [i] .ymin to r [i] .ymax, and the variable x changes from r [i] .xmin to r [i] .xmax. That is, in these loops, the window frame surrounding the connected component i is scanned from left to right and from top to bottom. In FIG. 8, the element r [i] .s of the region array element is initialized to 0 for all connected components i (step 804), and the pixels of the label image in the window frame surrounding the connected components i are connected components i. (Step 811), the value of the member r [i] .s is incremented by one (step 812), and the area of the connected component i is calculated. In the area calculation algorithm using the window frame, it is only necessary to access the minimum number of pixels, so that there is an advantage that the calculation process is efficient.

  Next, the sorting module 607 in FIG. 6 sorts the connected components in descending order of area. FIG. 9 (a) shows the structure of the elements of the array order used for sorting. As shown in FIG. 9A, the element is a structure composed of a member s and a member id. The initial values of order [i] .s and order [i] .id are r [i] .s and r [i] .id, respectively. FIG. 9C is a diagram showing the relationship between the label image with a window frame, the region array r, and the array order in FIG. The array order on the left side of FIG. 9C shows the state before sorting, and the array order on the right side shows the state after sorting. As a result of the sorting, the elements of the array order can be arranged in the order of connected component 3, connected component 1, and connected component 2.

  FIG. 10 is a flowchart showing details of processing of the drawing module 608 of FIG. The drawing module 608 draws the connected components by arranging them from the left to the right of the output image in descending order of area. If it cannot fit on one line, a line feed is performed. The variables x and y mean the x and y coordinates of the upper left point of the window frame surrounding the connected component when the connected component is arranged in the output image. The variable dy is a variable for obtaining the largest size in the y direction in one row of connected components.

  As shown in FIG. 10, first, in step 1001 to step 1003, initial values 0 are set to the variable x, the variable y, and the variable dy. Next, in step 1004, 1 is substituted into the variable i. In step 1005, it is determined whether i is less than or equal to the number n of connected components. If it is not less than nRegion (NO), the process is terminated (return). If nRegion or less (YES), the value of order [i] .id is substituted into the variable id to obtain the identifier of the connected component with the i-th largest area in Step 1006. In step 1007, the right end x-coordinate xRight when the connected component id is arranged in the output image is calculated. In step 1008, it is determined whether xRight is larger than the width W of the output image. If not larger (NO), in step 1013, the y coordinate yLow of the lower end when the connected component id is arranged in the output image is calculated. In step 1014, it is determined whether yLow is greater than the height H of the output image. If it is larger (YES), the process is terminated (return). If not larger (NO), the procedure draw (id, x, y) is called in step 1015 to draw the connected component id. Next, in step 1016 and step 1017, dy and the height of the connected component id are compared. If dy is smaller, the height of the connected component id is substituted into dy.

  Next, in step 1018, the variable x is set to a value obtained by adding xRight and the interval SPACE between connected components. In step 1019, the value of the variable i is incremented by 1, and the process returns to step 1005 to loop. If it is larger in step 1008 (YES), a line feed process is performed. That is, in step 1009, the value of the variable x is set to 0, and in step 1010, the value of the variable y is set to y + dy + SPACE. In step 1011, the value of the variable dy is set to zero. In step 1012, it is determined whether the variable y is greater than the height H of the output image. If the variable y is larger than the height H of the output image (YES), the process is terminated. If the variable y is not greater than the height H of the output image (NO), the process returns to step 1005 and loops.

  FIG. 11 is a flowchart showing details of the procedure draw (id, x, y) called in step 1015 of FIG. The flowchart shown in FIG. 11 includes a loop for variable y1 from step 1101 to step 1103 and a loop for variable x1 from step 1104 to step 1106. In the two loops, the variable y1 changes from r [id] .ymin to r [id] .ymax, and the variable x1 changes from r [id] .xmin to r [id] .xmax. That is, in these loops, the window frame surrounding the connected component id is scanned from left to right and from top to bottom. In step 1107, it is determined whether the pixel of the label image at the coordinates (x1, y1) is a pixel of the connected component id. If the pixel of the label image at the coordinates (x1, y1) is not a pixel of the connected component id (NO), the process proceeds directly to step 1106. If the pixel of the label image at the coordinates (x1, y1) is a pixel of the connected component id (YES), in step 1108 and step 1109, the upper left point (r [id] .xmin, The point (x1, y1) is converted to the point (x2, y2) by the parallel movement that corresponds r [id] .ymin) to the point (x, y). In step 1110, the value of the pixel at the coordinates (x1, y1) of the input image imgIn is substituted into the pixel at the coordinates (x2, y2) of the output image imgOut. Next, the process proceeds to step 1106. In step 1106, the value of the variable x1 is incremented by one, and the process returns to step 1105 to loop.

  This window frame algorithm is efficient because it accesses the minimum number of pixels. In the conventional method (Non-patent Document 1 p.108), since it is necessary to scan the entire image every time one connected component is drawn, the entire image is scanned by the number of connected components as a whole. On the other hand, in the algorithm using the window frame according to the present embodiment, since it is only necessary to scan the inside of the window frame of the connected components, it is only necessary to scan the entire image at most once to draw all the connected components.

  In the above example, image analysis such as insects and resident is described as an example, but it goes without saying that it can also be used for analysis of astronomical photographs, star brightness, and the like. A program for causing a computer to execute each of the above processes also falls within the scope of the present invention.

  As described above, in the connected component list creation method in the image processing technique according to the present embodiment, the operator extracts the connected component corresponding to the object in the image and arranges the image of the connected component according to the feature amount of the connected component. You can now easily count and classify objects. Moreover, there is an advantage that the information of the connected component can be extracted at high speed by using the window frame.

  The present invention is applicable to image processing technology.

It is a figure which shows the example of the light and shade image of insect trap paper. It is a figure which shows the example of the binary image of insect trap paper. It is a figure which shows the connection component list of insect trap paper. It is a figure which shows the example of a process of the satellite image of a road. It is a figure which shows the example of a process of the grayscale image by this Embodiment. It is a figure which shows the block diagram of the image processing hardware system by this Embodiment. It is a flowchart figure which shows the detail of a process of the window frame module 605. FIG. FIG. 10 is a flowchart showing details of processing of an area calculation module 606. It is a figure which shows the relationship between the label image with a window frame, area | region arrangement | sequence r, and arrangement | sequence order. 5 is a flowchart showing details of a drawing module 608. FIG. It is a flowchart figure which shows the detail of procedure draw (id, x, y).

Explanation of symbols

601 ... Keyboard, 602 ... Processing device, 603 ... Binarization module, 604 ... Labeling module, 605 ... Window frame module, 606 ... Area calculation module, 607 ... Sort Module, 608 ... Drawing module, 609 ... Storage device, 610 ... Gray image, 611 ... Binary image, 612 ... Label image, 613 ... Work image, 614 ... Area arrangement, 615... Display.

Claims (3)

In a method of creating a list of image connected components using a computer,
A binarization step for converting the image into a binary image relating to shading;
A labeling step for generating a label image in which a number determined for each connected component is assigned to pixels of the connected component on the binary image;
With reference to the label image, a window frame setting step for setting a window frame consisting of a minimum rectangle surrounding each connected component;
A feature amount extracting step of calculating a feature amount of each connected component within the window frame;
A sorting step of sorting each connected component based on the feature amount;
Generating a connected component list for displaying a list of images of the respective connected components in the sorted order. In the window frame step,
First, for each connected component i, a sufficiently large initial value is set for the variable x ⁱ _min , a sufficiently small initial value is set for the variable x ⁱ _max , a sufficiently large initial value is set for the variable y ⁱ _min , and the variable y ⁱ _max Set a sufficiently small initial value to
Next, the label image is raster scanned, if the current position (x, y) is connected component i, at x is less than x ⁱ _min, and assigns the value of x to x ⁱ _min, x is when greater than x ^{i _max,} assigns the value of x to _{x i max,} when y is smaller than ^y _{i ^min,} and assigns the value of y to _{y i min,} when y is greater than ^y _{i ^max, y i max} The image connected component list creation method according to claim 1, wherein a value of y is substituted into and a minimum rectangle surrounding each connected component is set. A binarization module that binarizes the image based on the magnitude of the pixel density;
A labeling module for attaching the same label to connected pixels;
A window frame module for setting a window frame consisting of a minimum rectangle surrounding each connected component with the same label;
A feature amount extraction module for calculating a feature amount of each connected component in the window frame;
A sorting module for sorting the connected components based on the feature amount;
An image processing apparatus comprising: a connected component list generation module that generates a connected component list that displays a list of images of the respective connected components in the sorted order.

Images