JP2004086793A - Method and program for extracting object image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To exactly calculate contours even when other object exists in the vicinity and when an object is thin. <P>SOLUTION: This object image extraction program is constituted of a first step for arranging a plurality of search reference blocks at contour parts of an alpha mask by shifting mutual positions on receiving image data and input of the alpha mask of the approximate shape of an extraction object, a second step for searching similar blocks which are larger than the search reference block and having the similar patterns from the image data by every search reference block and a third step for replacing the alpha masks in each search reference block with the ones formed by reducing the alpha masks of each similar block. In an object image extraction method for making the contours of the alpha masks coincident with the contours of the object by recursively repeating the third step for prescribed frequencies, label data for distinguishing an object defined area, a background defined area and an undefined area is separately inputted and the similar blocks are searched by referring to the label data in the second step. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an object image extraction method and an object image extraction program for finding an object region in an image.
[0002]
[Prior art]
A technique for extracting a region of an arbitrary object from an image is useful, for example, when performing image processing such as replacing the background with another image. For that purpose, it is necessary to correctly determine the region of the object along its contour.
[0003]
The region of the object is specifically represented by an image called an alpha mask (or shape data). The alpha mask is an image in which different pixel values are set in a region where an object is present (object region) and other regions. An example of the original image and its alpha mask are shown in FIGS. 3 and 4, respectively. FIG. 4 shows the alpha mask of the ship 031 on the right shown in the original image of FIG.
[0004]
In other words, extracting an object is generating an alpha mask.
Once the alpha mask is obtained, the background area other than the object area can be easily replaced with another image by referring to the alpha mask.
[0005]
In the case of such an object extraction, when a rough shape of an object is temporarily obtained in advance by some method, a technique for obtaining an accurate shape of the object based on the rough shape is required. As one of the methods, an object extraction method (a fractal method) using a self-similar mapping has been proposed (for example, see Patent Literature 1, Patent Literature 2, Non-Patent Literature 1). The outline of this technique will be described with reference to FIGS. 5, 6, 7, and 8. FIG.
[0006]
FIG. 5 shows a state in which a schematic shape 051 is given to a ship 031. FIG. 6 shows an alpha mask thereof. At this stage, the contour of the alpha mask is shifted from the contour of the object. In the fractal method, a plurality of square blocks are first set along the approximate shape of the object using the alpha mask with respect to such input of the original image data and the alpha mask. These blocks are called child blocks. Then, a child block is set at the same position in the original image data as in the case of the alpha mask. FIG. 7 shows a state where child blocks are set in the original image data. The schematic shape is also shown in the original image data for easy understanding.
[0007]
Next, a parent block corresponding to each child block is obtained using the original image data. The size of the parent block is twice as long and wide as the child block, and the image pattern (pattern) in the parent block is the same as the image pattern in the child block.
[0008]
In order to find such a parent block similar to the design of the child block, a block that is twice as long and wide as the candidate block is temporarily set near the child block on the original image data. Then, an error between a reduced block obtained by reducing the candidate block in half vertically and horizontally and a child block is obtained.
[0009]
In the case of a grayscale image, the error is, for example, an absolute value error (the absolute value of the difference between the pixels is added for all the pixels in the block). Such an error evaluation is performed on some candidate blocks, and the candidate block having the smallest error from the child block is determined as the parent block. Such parent blocks are obtained for all child blocks shown in FIG.
[0010]
FIG. 8 is an example of the parent block 082 obtained for the child block 081 set at the tip of the ship 031. It can be seen that the symbols in each block are similar. FIG. 8 shows a binary image, but the same applies to a grayscale image in which pixels are given pixel values of 256 levels.
[0011]
By the above processing, data of a set of a child block and a parent block having a pattern similar to this can be obtained. Next, the obtained parent block is set at the same position of the alpha mask in the same manner as the original image data. I do.
[0012]
The extraction of the object is performed by correcting the alpha mask using the child block and the parent block. More specifically, a process may be performed in which the alpha mask data of each child block is replaced with data obtained by reducing the alpha mask data of the corresponding parent block by half in length and width.
[0013]
When this replacement transformation (self-similar mapping) is performed once, the approximate shape slightly approaches the correct shape. The same replacement conversion is performed again on the approached alpha mask. As described above, when the replacement conversion is repeated recursively to the alpha mask, the shape converges to a substantially correct shape. An alpha mask that has undergone a certain number of replacement transformations is defined as an object extraction result.
[0014]
[Patent Document 1]
JP 2000-82145 A
[0015]
[Patent Document 2]
JP 2001-188910 A
[0016]
[Non-patent document 1]
Ida, Mihonsugi, "Contour fitting by self-similar mapping", Proc. Of the 5th Image Sensing Symposium, C-15, pp. 115-120, June 1999.
[0017]
[Problems to be solved by the invention]
However, in the conventional fractal method, the contour of another object may be extracted when another object is nearby. In addition, when the object shape is thin, the object shape may not always be accurately extracted, for example, the object shape may disappear.
[0018]
Therefore, an object of the present invention is to provide a method and a program that can accurately extract the shape of an object even when another object is nearby or the object shape is small.
[0019]
[Means for Solving the Problems]
In order to solve the above problem, an object image extracting method according to the present invention includes a first step of receiving image data including a target object to be extracted and shape data representing a region of a schematic shape of the target object in the image data. A second step of setting a plurality of search reference blocks at different positions along the outline of the schematic shape represented by the shape data; and for each of the search reference blocks, a size larger than the search reference block. A third step of searching for a similar block which is a block and has a pattern in an area corresponding to the block on the image data most similar to a pattern in an area corresponding to the search reference block on the image data; The shape data of the area corresponding to each search reference block is converted to the shape data of the area corresponding to each similar block. And a fourth step of repeatedly performing a process of replacing the object with a reduced one until a predetermined condition is satisfied. In the first step, the probability that the target object exists in each pixel in the image data is expressed. Label data is also received, and in the third step, a weighted sum of errors of the entire block is calculated by referring to the label data, ignoring or neglecting an error between pixels having a low possibility that the target object is present, A search is performed for a similar block in which the weighted sum is minimized.
[0020]
Further, the shape data represents a region of a schematic shape of the target object, each pixel of which corresponds to each pixel of the image data on a one-to-one basis, and the label data indicates that the image data corresponds to the target object. The image is divided into at least two types of regions out of three types of a determined region, a non-target object determined region, and an undetermined region, and each of the pixels is one-to-one corresponding to each pixel of the image data. In the third step, for each of the plurality of candidate blocks that are candidates for the similar block, data of an area corresponding to the search reference block in the image data and data obtained by reducing an area corresponding to the candidate block in the image data are obtained. Of each corresponding point is calculated, and the weighted sum is stored in an area where both of the corresponding points are divided into target object defined areas by the label data. If, or if both are present in a region that is partitioned into non-target object determined regions, ignore or neglect the error for the corresponding point, find the candidate block with the minimum weighted sum of the error. It may be characterized in that it is determined as a similar block.
[0021]
In the third step, for each of the plurality of candidate blocks that are candidates for the similar block, data of an area corresponding to the search reference block in the image data and an area corresponding to the candidate block in the image data are determined. The error of each corresponding point with the reduced data is calculated, and the weighted sum is calculated when the corresponding points are both present in the area divided into the target object defined area by the label data, or both are non-target objects. If it exists in the area divided into the definite area, the weight coefficient multiplied by the error for the corresponding point is determined to be 0 or a small value, and the candidate block having the minimum weighted sum of the errors is determined as the similar block. It may be characterized by doing.
[0022]
The object image extracting method according to the present invention includes: a first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data; A plurality of search reference blocks are set to be shifted from each other along the outline of the approximate shape to be set, and among these, a pattern of an area corresponding to the search reference block on the shape data is a predetermined pattern. Is a second step of canceling the setting, and for each of the search reference blocks, a symbol of a region larger than the search reference block and corresponding to the block on the image data is corresponding to the search reference block. A third step of searching for a similar block most similar to the above, and searching for each of the search reference blocks in the shape data. The process of replacing the data of the corresponding area obtained by reducing the data of the region corresponding to each of the similar block in, and having a fourth step of performing repeated until a predetermined condition is satisfied.
[0023]
In the second step, a plurality of search reference blocks are set to be shifted from each other along the outline of the schematic shape represented by the shape data, and the inspection is performed so that the center position is the same as each search reference block. For the block, a pattern of an area corresponding to the inspection block on the shape data is examined, and if the inspection block pattern is a predetermined pattern, the setting of the search reference block may be canceled.
[0024]
The object image extracting method according to the present invention includes: a first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data; A second step of setting a plurality of search reference blocks at different positions along the outline of the approximate shape to be set, and for each of the search reference blocks, a size larger than the search reference block and A similar block whose pattern in the region corresponding to the block is most similar to the symbol in the region corresponding to the search reference block is searched, and a similar block in which the pattern of the corresponding region on the shape data is a predetermined pattern. A third step of canceling the setting of the search reference block corresponding to the search data; The process of replacing the data in the area corresponding to the reference block obtained by reducing the data of the region corresponding to each of the similar block, and having a fourth step of performing repeated until a predetermined condition is satisfied.
[0025]
The object image extraction method of the present invention receives image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data, and receives both the image data and the shape data. A first step of extending at least one of the upper, lower, left, and right sides by a predetermined width, and a second step of setting a plurality of search reference blocks at different positions along the outline of the schematic shape represented by the shape data. Searching, for each search reference block, a similar block that is larger than the search reference block and whose pattern in the area corresponding to the block on the image data is most similar to the pattern in the area corresponding to the search reference block A third step, and in the shape data, data of an area corresponding to each of the search reference blocks is Processing for replacing in those reduced data in the area corresponding to the similar block, and having a fourth step of performing repeated until a predetermined condition is satisfied.
[0026]
The object image extracting method according to the present invention includes: a first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data; A second step of setting a plurality of search reference blocks to be shifted from each other along the outline of the approximate shape to be set, and for each of the search reference blocks, the search reference blocks are smaller than the search reference blocks and are set on the image data. A third step of searching for a similar block in which the design of the region corresponding to the block is the most similar to the design of the region corresponding to the search reference block; and converting the data of the region corresponding to each of the similar blocks in the shape data. The process of replacing the data in the area corresponding to the search reference block with the reduced data is performed until a predetermined condition is satisfied. Characterized by a fourth step of performing repeatedly.
[0027]
The object image extracting method according to the present invention receives image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data, and enlarges these to obtain enlarged image data and enlarged image data. A first step of generating shape data, a second step of setting a plurality of search reference blocks to be shifted from each other along a contour of a schematic shape represented by the enlarged shape data, and the search For each reference block, a third similar block is searched for a similar block that is larger than the search reference block and whose symbol in the area corresponding to the block on the enlarged image data is most similar to the symbol in the area corresponding to the search reference block. And, in the enlarged shape data, data of an area corresponding to each of the similar blocks is set as a respective search reference block. A fourth step of repeatedly performing a process of replacing the data in the corresponding area with the reduced data until a predetermined condition is satisfied, and a fifth step of reducing the enlarged shape data obtained in the fourth step to the original size. And a step.
[0028]
The object image extracting method according to the present invention includes a first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data; A second step of generating shape data in which the shape of the target object is extracted using the image data, and reducing the extracted shape data to a predetermined size, and extracting a pixel value of one pixel from the extracted shape data. And a third step of generating reduced-shape data obtained by using pixel values of a plurality of corresponding pixels.
[0029]
The object image extracting method of the present invention receives image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data, and expands these to obtain enlarged image data and enlarged image data. A first step of generating shape data, a second step of generating enlarged shape data obtained by extracting the shape of the target object using the enlarged shape data and the enlarged image data, and generating the extracted enlarged shape data. Generating the shape data obtained by reducing the pixel value of one pixel to the original size and using the pixel values of a plurality of corresponding pixels of the extracted shape data. .
[0030]
The object image extracting program according to the present invention includes: a first step of receiving, by a computer, image data including a target object to be extracted, and shape data representing a general shape region of the target object in the image data; A second step of setting a plurality of search reference blocks so as to be shifted from each other along the outline of the general shape represented by the data, and for each of the search reference blocks, a block larger than the search reference block. A third step of searching for a similar block in which the design of the region corresponding to this block on the image data is most similar to the design of the region corresponding to the search reference block; and a region corresponding to each of the search reference blocks. Replace the shape data of the area with the reduced shape data of the area corresponding to each similar block And a fourth step of repeating the process until a predetermined condition is satisfied. In the first step, label data representing the certainty of the existence of the target object at each pixel in the image data is also received. In the third step, a weighted sum of errors of the entire block is calculated by referring to the label data and ignoring or neglecting an error between pixels having a low possibility that the target object exists, and calculating the weighted sum of the errors. A search for similar blocks is performed.
[0031]
Further, the shape data represents a schematic shape of the target object, and each pixel thereof corresponds to each pixel of the image data on a one-to-one basis, and the label data represents the image data representing the target object determination area. , A non-target object determined area and an undetermined area are divided into at least two types of areas, and each pixel is made to correspond to each pixel of the image data on a one-to-one basis. In the third step, For each of the plurality of candidate blocks that are candidates for the similar block, corresponding points of data of an area corresponding to the search reference block in the image data and data obtained by reducing an area corresponding to the candidate block in the image data Is calculated, and the weighted sum thereof is calculated assuming that the corresponding points are both present in an area divided into the target object defined area by the label data, Alternatively, when both are present in an area partitioned into a non-target object confirmed area, the error for the corresponding point is determined by ignoring or neglecting, and the candidate block in which the weighted sum of the error is minimized is regarded as the similar block. The determination may be a feature.
[0032]
The object image extracting program according to the present invention includes: a first step of receiving, by a computer, image data including a target object to be extracted, and shape data representing a general shape region of the target object in the image data; A plurality of search reference blocks are set to be shifted from each other along the outline of the schematic shape represented by the data, and among these, a pattern of an area corresponding to the search reference block on the shape data is a predetermined pattern. Is a second step of canceling the setting, and for each of the search reference blocks, a symbol in a region larger than the search reference block and corresponding to the block on the image data corresponds to the search reference block. A third step of searching for a similar block most similar to the pattern in the area; Processing for replacing in those reduced data in the area corresponding data in the area corresponding to the each search reference block in each of the similar block, to execute a fourth step of performing repeated until a predetermined condition is satisfied.
[0033]
In the second step, a plurality of search reference blocks are set to be shifted from each other along the outline of the schematic shape represented by the shape data, and the inspection is performed so that the center position is the same as each search reference block. For the block, a pattern of an area corresponding to the inspection block on the shape data is examined, and if the inspection block pattern is a predetermined pattern, the setting of the search reference block may be canceled.
[0034]
The object image extracting program according to the present invention includes: a first step of receiving, by a computer, image data including a target object to be extracted, and shape data representing a general shape region of the target object in the image data; A second step of setting a plurality of search reference blocks to be shifted from each other along the outline of the schematic shape represented by the data; A similar block in which the pattern of the region corresponding to the block on the data is most similar to the symbol of the region corresponding to the search reference block is searched, and the pattern of the corresponding region on the shape data is a predetermined pattern. A third step of releasing the setting of the block and the corresponding search reference block; And replacing the data of the area corresponding to each search reference block with the reduced data of the area corresponding to each similar block in the data until a predetermined condition is satisfied. .
[0035]
An object image extraction program according to the present invention is configured such that a computer receives image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data, and receives the image data and the shape data. A first step of extending at least one of the upper, lower, left, and right sides by a predetermined width, and setting a plurality of search reference blocks to be shifted from each other along the outline of the schematic shape represented by the shape data A second step, wherein, for each of the search reference blocks, the similarity of the pattern in the area corresponding to the block larger than the search reference block and corresponding to the block on the image data is most similar to the design of the area corresponding to the search reference block. A third step of searching for a block; and a step in the shape data corresponding to each of the search reference blocks. The process of replacing the data with those obtained by reducing the data of the region corresponding to each of the similar block, to execute a fourth step of performing repeated until a predetermined condition is satisfied.
[0036]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a plurality of embodiments of the present invention will be described with reference to the drawings.
[0037]
(Matters Common to Each Embodiment) FIG. 40 is a diagram of a PC (personal computer), WS (workstation), or the like used in each embodiment of the present invention. The PC and WS used in each embodiment include an input device 4007 such as a pointing device such as a mouse, a keyboard, and other external devices (for example, a digital camera), an input receiving unit 4004 that receives input from the input device 4007, a CRT and an LCD. And the like, and an image output unit 4006 for outputting an image signal to the image display device 4008.
[0038]
The PC and WS used in each embodiment include a magnetic disk drive 4002 such as a hard disk, a memory 4005, a central processing unit 4001, and an optical disk drive 4003.
[0039]
Each of the embodiments of the present invention is an object image extraction program that operates on a PC (personal computer) or WS (workstation). The user reads the program stored in the magnetic disk drive 4002 in advance into the memory 4005 and causes the PC or WS to execute the program.
[0040]
Before executing the object image extraction program, the user acquires image data to be subjected to the object shape extraction processing and stores it in the magnetic disk drive 4002 in advance. When the object image extraction program is executed, the image data is read from the magnetic disk drive 4002 and stored in the memory 4005.
[0041]
Then, the image data is displayed on the display device 4008, and the user inputs the data of the alpha mask of the schematic shape specifying the schematic shape of the object using the mouse. The input alpha mask data is also stored in the memory 4005.
[0042]
Further, the object image extraction program extracts an object shape based on the image data and the data of the alpha mask of the approximate shape. An alpha mask having an accurate shape is generated by extracting the object shape, and this is output as an extraction result.
[0043]
The alpha mask resulting from the contour extraction is stored in the magnetic disk drive 4002 or the memory 4005 of the PC or WS. Then, for example, it is used in another program or apparatus that cuts out an image of an object from an image.
[0044]
Note that a camera may be connected to a PC or WS so that a user can acquire an image at an arbitrary timing. Further, the data of the alpha mask of the approximate shape may be automatically generated using a background subtraction method or the like.
[0045]
In the following description, the search reference block is called a child block, and a similar block that is larger than the search reference block (for example, twice vertically and horizontally) and has a similar design is called a parent block.
[0046]
(First Embodiment) This embodiment is intended to improve the case where the shape of an object cannot be accurately extracted when there is an edge such as another object contour near the contour extraction target object. .
[0047]
Hereinafter, the processing of this embodiment will be described with reference to the flowchart of FIG.
[0048]
(Step 010) Image data, a roughly shaped alpha mask, and label data are prepared in a memory.
[0049]
As the image data, for example, the user reads out image data which has been shot by a digital camera or the like in advance and stored in a hard disk.
[0050]
The alpha mask is, for example, data representing the shape of an object specified and created by a mouse while viewing image data displayed on an image display device by a user. The alpha mask is data having the same number of pixels as the image data. A pixel where the object exists is 1 and a pixel where the object does not exist is 0.
[0051]
The label data is data representing information related to an object area and a background area different from the alpha mask, and is created by a user designating the image data using a mouse while viewing the image data. For each pixel, for example, a region that is known in advance to be an object other than the background or the target object is defined as a non-target object defined region (background defined region) with the pixel value of each pixel set to 0, and a region that is known in advance to be an object Sets the pixel value of each pixel to 1 as a target object fixed area (object fixed area), and sets the pixel value of each pixel to 2 as an undetermined area for a background or object undetermined area. The generation of the label data may be performed automatically with reference to, for example, an alpha mask.
[0052]
(Step 011) A child block C is set at the outline of the alpha mask. In the present embodiment, the child blocks C are set so as to overlap each other as shown in FIG. The position information (for example, the coordinates of the upper left corner and the coordinates of the lower right corner) of the set child block C is stored in the memory.
[0053]
(Step 012) A parent block P similar to each child block is found with reference to the label data and the image data.
[0054]
First, a candidate block that is a parent block candidate is set near the child block. Then, an error is obtained by referring to the label data of the pattern of the child block area and the pattern of the candidate block area on the image data.
An error is calculated for a plurality of candidate blocks, and the one with the smallest error is defined as a parent block. Since the candidate block is a block having the same number of pixels as the parent block, when calculating an error between the child block and the candidate block, the candidate block is reduced to have the same number of pixels as the child block.
[0055]
The error calculation with reference to the label data is performed as follows. For example, when the user designates an area inside the area 151 of FIG. 15 as not to extract an object, the label data has a pixel value of 0 (background defined area) inside, and a pixel value of 2 otherwise (undefined area). ). Then, a pixel in the child block is represented by X _c , And the pixel of the corresponding reduced block is X _p X _c And X _p Is defined as follows. (1) X _c And X _p When both are within the background fixed area, "X _c And X _p Is ignored and "0" is ignored.
(2) X _c And X _p When both are within the object confirmation area, “X _c And X _p Is ignored and "0" is ignored.
(3) If neither (1) nor (2) above, "X _c And X _p Error ”=“ X _c And X _p The absolute value of the pixel value difference ”.
[0056]
X obtained in this way _c And X _p Is added in the block to obtain an error between the candidate block and the child block. Then, the candidate block with the smallest error is determined as a parent block. Then, the determined position information of the parent block is stored in the memory in association with the child block.
[0057]
(Step 013) Step 012 is repeated until a parent block for all child blocks is determined.
(Step 014) On the alpha mask, the data of the area corresponding to each child block is replaced with the data of the corresponding parent block reduced to the same number of pixels as the child block.
(Step 015) Step 014 is performed for all child blocks. This means that all child blocks of the alpha mask have been replaced once.
(Step 016) Steps 014 and 015 are recursively repeated a predetermined number of times. Thereby, the contour of the alpha mask approaches the contour of the object.
[0058]
In the calculation of the error in step 12, in the cases of (1) and (2), the error is ignored and set to 0. However, the error is neglected, that is, even if the error is multiplied by a predetermined coefficient larger than 0 and smaller than 1. good. Alternatively, in the cases (1) and (2), the calculation itself may be omitted ignoring the error.
[0059]
Hereinafter, an example of extracting an object shape by the conventional fractal method will be compared with an example of extracting an object shape according to the present embodiment.
[0060]
An example in which an object shape is extracted using a conventional fractal method when another object is nearby will be described. The child block 091 shown in FIG. 9 includes the region of the ship 032 at the upper left in addition to the region of the ship 031 at the lower right. In this case, there is a high possibility that a candidate block completely similar to the child block 091, that is, a candidate having an absolute value error of 0 is not found, but a candidate block with the smallest error is selected as a parent block.
[0061]
In order to extract the ship 031, it is convenient to select a block 093 similar to the parent block with respect to the region of the ship 031. In this case, the contour of the alpha mask converges to the contour of the boat 031 by the replacement transformation. However, block 093 is not always selected.
[0062]
FIG. 11A is an enlarged view of the periphery of the child block 091, and shows a parent block candidate block 093, a contour 114 of the ship 032, and a contour 115 of the ship 031. In FIG. 11A, the parent block candidate block 093 is set to be similar to the outline 115 of the ship 031.
[0063]
FIG. 11B shows an image pattern 111 of the child block 091, an image pattern 112 obtained by reducing the block 093, and a difference pattern 113 between the image pattern 111 and the image pattern 112.
[0064]
Here, it is assumed that the pixel values of the region of the ship 032 and the region of the ship 031 painted with the horizontal line are the same value, and the other regions are constant with different pixel values. In the difference pattern 113, a portion where a pixel value error occurs between the image pattern 111 and the image pattern 112 is painted black. The absolute value error is proportional to the area of this black region.
[0065]
On the other hand, FIGS. 12A and 12B show a case in which the block 121 which is a parent block candidate is set to be similar with respect to the contour 114. In this case, the area of the black region of the difference pattern 122 is smaller than the difference pattern 113 of FIG. 11B. As a result, as shown in FIG. 10, the block 121 is selected as a parent block, and the outline of the alpha mask of the ship 031 approaches the ship 032 by the replacement conversion for this portion, so that a correct outline cannot be obtained.
[0066]
On the other hand, by performing processing using the label data as in step 012, a parent block convenient for object extraction is selected. This will be described with reference to a case where an alpha mask as shown in FIG. 4 is generated from the original image of FIG.
[0067]
FIG. 5 shows a state in which the general shape of the object is given, and FIG. 6 shows an alpha mask of this general shape. FIG. 7 shows the result of the arrangement of the child blocks.
[0068]
FIG. 13A is an enlarged view of the periphery of the child block 091. The outline 114 is the outline of the ship 032, and the outline 115 is the outline of the ship 031. The parent block 093 is arranged so as to be similar to the ship 031 outline 115. The upper left side of the background determined area line 131 which is the boundary line of the background determined area is the background determined area 151 shown in FIG.
[0069]
FIG. 13B shows an image pattern 132 of the child block 091, an image pattern 133 of the parent block 093 (reduced to the same size as the child block), and a difference pattern 134 between the image pattern 132 and the image pattern 133. FIG.
[0070]
In FIG. 13B, the image pattern 132 and the image pattern 133 also show a background defined area line 131, and the difference pattern 134 is a common area of the background defined area in the image pattern 132 and the background defined area in the image pattern 133. Is also shown.
[0071]
As described with reference to FIGS. 11A and 11B, conventionally, in such an arrangement of the parent blocks, many errors have occurred on the upper left side. However, in the present embodiment, since the error of the pixel which is the background defined region in both the image pattern 132 and the image pattern 133 is zero, no error appears in the region on the upper left side of the boundary line 135 of the difference pattern 134. . For this reason, the error in the entire block like the difference pattern 134 is smaller than in the related art.
[0072]
FIGS. 14A and 14B are diagrams showing a case where the parent blocks are arranged so as to be similar to the contour 114 of the ship 032, similarly to FIGS. 12A and 12B. In this case as well, the error in the upper left of the line 135 is canceled as in the case of FIG. 13, but since there is no error in the upper left of the line 135, the lower right of the line 135 as a whole is the same as in the case of FIG. An error occurs.
[0073]
Comparing FIG. 13 with FIG. 14, in the present embodiment, the error of the case of FIG. 13 is smaller than the case of FIG. 14, and the parent block is self-similar with respect to the case of FIG. Is selected, the contour of the ship 031 is correctly extracted.
[0074]
In the present embodiment, the label data is specified in advance by the user using the mouse, but the label data may be obtained by using a method for automatically obtaining the label data described below.
[0075]
In the present embodiment, steps 014 and 015 are repeated a predetermined number of times, but the condition of repetition is not limited to “number of times”. For example, when the replacement is performed in step 015, the sum of the absolute difference between the alpha mask of the child block and the reduced alpha mask of the parent block is calculated, and the calculation is repeated until the sum of the absolute difference becomes equal to or less than a predetermined threshold. You may do it.
[0076]
(First Label Data Generation Method) For example, when the color of an object or the background color is known in advance, there is also a method of setting the color regions as the object defined region and the background defined region, respectively.
[0077]
This method will be described by taking the case where the schematic shape 051 is given as described above as an example.
[0078]
(1) As shown in FIG. 37, the general shape 051 is expanded with a constant width W to obtain the shape 371. In the same manner, the shape 372 is obtained by expanding at a width of 2W.
[0079]
Here, the dilation is a kind of processing by a morphological filter. When a pixel other than a processing target area (here, an object area) is scanned and a processing target area is within a predetermined distance from a target pixel, the pixel is expanded. Is added to the processing target area.
[0080]
(2) The approximate shape 051 is contracted by the widths of W and 2W to obtain the shapes 373 and 374, respectively.
[0081]
Here, shrinkage is also a type of processing by a morphological filter. When scanning a pixel in a processing target area, if there is a non-processing target area within a certain distance from a target pixel, the pixel is excluded from the processing target area. It is.
[0082]
(3) An area 381 (an area shaded in FIG. 38) sandwiched between the shapes 371 and 372 is set as a background color sampling area. An area 382 (an area indicated by a horizontal line in FIG. 38) sandwiched between the shapes 373 and 374 is set as a foreground color sampling area. Here, the reason why the color is sampled from a portion at a certain distance from the general shape 051 is that the general shape is deviated from the correct shape.
[0083]
(4) A background color table is created by extracting all the colors of the pixels in the background color sampling area in the original image. A foreground color table is created in the same manner.
[0084]
Instead of extracting all colors, if a certain number of pixels have the same color, that color may be registered.
[0085]
(5) Compare the background color table with the foreground color table, and if the same color is registered, remove that color from both tables.
[0086]
(6) Each pixel of the original image is scanned using the two color tables thus obtained. Then, the pixel of the color in the background color table is defined as the background defined area, the pixel of the color in the foreground color table is defined as the object defined area, and the pixel of the color not in any table is defined as the undetermined area to create the label data.
[0087]
By performing the above (1) to (6), regions that can be estimated as a background or an object from the viewpoint of color can be set as respective defined regions.
[0088]
It should be noted that if the similar color is not individually registered, but the color level of the original image is quantized in advance and then the color table is created and the label data is created, the number of colors to be handled is reduced, so that the processing amount can be reduced. , Which contributes to faster processing.
[0089]
(Second Label Data Generation Method) If a color table is locally created in a screen and label data is created while switching the respective tables, locally optimized information can be obtained.
[0090]
As an example of locally creating a color table, the following method can be considered. For example, as shown in FIG. 39, blocks 391 and 392 are arranged along the schematic shape 051, and the above-described color table is created only inside each block. Then, when creating the label data, one of the background fixed, the object fixed, and the undetermined is determined using the color table of the block to which each pixel belongs. At this time, in the portion where the blocks 391 and 392 overlap, first, a label is determined in the color table of one of the blocks, and if the label is undecided, the label is relabeled in the other color table.
[0091]
(Third Label Data Generation Method) In generating a background color table and an object color table, instead of using a morphological filter, a method of registering and generating a color of a pixel for which a user has designated a screen as a background or an object may be used. is there.
[0092]
(Third Label Data Generation Method) When a background image in which the object whose contour is to be extracted is not obtained in advance, the difference between the background image and the current image is obtained, and pixels having a difference equal to or greater than a certain value There is also a method of confirming.
[0093]
(Fourth Label Data Generation Method) If it can be assumed that there is a correct outline within a certain range from the outline of the outline shape, the result of contracting the background region of the outline shape is used as the background defined region, or the object of the outline shape is determined. The result of contracting the region may be used as the object confirmation region. With this method, the periphery of the processing target area is reduced.
[0094]
(Fifth Label Data Generation Method) If it is known that the roughly-shaped object region always includes the entire object, the background region of the roughly-shaped background is left as it is, or a background image obtained by contracting one to two pixels is fixed. Area. On the other hand, a roughly contracted object region is reduced by several pixels, for example, a region contracted by about の of the number of pixels on one side of the child block is defined as an object defined region.
[0095]
Further, the object determined area and the background determined area determined in this way may be expanded by using the area growing method. Conversely, the same applies to a case where the whole of the roughly shaped object region is always included in the object.
[0096]
Here, the area growth method compares the pixel values inside and outside the area at the boundary of the area to be grown, and incorporates the external pixels into the area if the difference between the pixel values is equal to or less than a certain value. It goes out. By repeating this process, a portion having a similar pixel value is captured in the area.
[0097]
(Sixth Label Data Generation Method) When one point on the object is input by the user, an object area may be obtained by the area growing method with the point as a starting point, and may be used as the object confirmed area. Similarly, the background defined area can be set for the background by using the area growing method.
[0098]
(Number of Levels of Label Data) In the label data generation method up to this point, an example has been described in which data of three levels of background determination, object determination, and undetermined are generated, but the number of levels may be increased. For example, it is conceivable that the pixel value of the label data is made closer to 0 as the probability of being the background is higher, the pixel value is made closer to 255 as the probability of being the object is higher, and the undetermined region is made to correspond to the pixel value of 128.
[0099]
The colors of the background and the object are checked by some method (for example, by the method of sampling the background color and the object color in the first label data generation method described above). Then, the label data is generated by determining the pixel value of the label data depending on whether each pixel is closer to the background or the color of the object.
[0100]
In the parent block search using the label data generated in this manner, the pixel value of a certain pixel in the child block is calculated as X _c , The pixel value of the pixel of the reduced block of the parent block candidate is X _p , "X _c And X _p The absolute value of the difference between the pixel values of E is E, and the weighting coefficient for error calculation is W.
"X _c And X _p Error = W × E
Is calculated for each pixel to obtain a sum, and “X _c And X _p Is used as a parent block.
[0101]
Here, the weight coefficient W is
(A) X _c And X _p When the pixel values of both label data are not close to 0 or 255, they are always kept at 1.
(B) X _c Or X _p When the pixel value of any one of the label data is close to 0, the weight coefficient W is set to be smaller than 1 as the other label is smaller.
(C) X _c Or X _p If the pixel value of any one of the label data is close to 255, the weight coefficient W is made smaller than 1 as the other label becomes larger.
[0102]
By controlling W in this manner, the influence of an error between pixels having a high probability of being an object and pixels having a high probability of being a background can be reduced in multiple stages.
[0103]
As described above, in the present embodiment, the outline can be accurately extracted even when a plurality of objects are close to each other by using the masking information called the label data when extracting the outline of the object.
[0104]
(Second Embodiment) Hereinafter, a second embodiment of the present invention will be described with reference to FIGS. This embodiment is effective when a plurality of objects are individually and sequentially extracted. Note that, here, the description will be focused on parts different from the first embodiment.
[0105]
A ship 032 and a ship 031 are present in the image shown in FIG. 3, and in the present embodiment, these contours are sequentially extracted. Here, first, the ship 032 is extracted by the method described in the first embodiment, and then the ship 031 is extracted.
[0106]
FIG. 26 is a diagram in which the object region 261 of the ship 032 extracted by the method described in the first embodiment is painted black. In this situation, as in the first embodiment, a schematic shape is set as shown in FIG. 5, and the child blocks are arranged as shown in FIG.
[0107]
Then, the pixel value of a certain pixel in the child block is represented by X _c , The pixel value of the pixel of the corresponding reduced block is X _p As X _c And X _p Equation for calculating the error
(1) X _c Or X _p Is within the area 261, “X _c And X _p Error = “0”
(2) When the above does not apply to (1), "X _c And X _p Error = “X _c And X _p The absolute value of the difference between the pixel value and
Is defined as
[0108]
With this definition, the error is not counted in the upper left horizontal line area of the image pattern 112 in FIG. 11, and the error of the block 093 becomes zero, which is selected as a parent block. In other words, the calculation formula of the error is such that a parent block 093 similar to the child block 091 in FIG. 9 with respect to the contour of the ship 031 is obtained.
[0109]
As described above, according to the present embodiment, when sequentially extracting the contours of a plurality of objects, the detected objects are masked using the information of the contours of the already detected objects. However, the contour of the object can be accurately extracted.
[0110]
Third Embodiment A third embodiment of the present invention will be described with reference to the drawings. In the present embodiment, conventionally, there has been a case where it is difficult to detect the contour when the object region is thin, but this is to solve this. Note that, here, the description will be focused on parts different from the first embodiment.
[0111]
With the conventional fractal method, it is difficult to extract an object when the object shape is small, such as around the child block 161 in FIG.
[0112]
There is no completely similar parent block for the child block 161 arranged in the narrow portion. For the child block 161, for example, the block 162 is selected as the parent block.
[0113]
When the parent block 162 is replaced with the child block 161 by using the child block 161 and the parent block 162, the width of the object area in this portion is reduced by half each time. Then, if the replacement conversion is repeatedly performed, the object region will eventually disappear.
[0114]
This embodiment can address the above-mentioned problem. Hereinafter, the flow of processing in the present embodiment will be described with reference to the flowchart of FIG.
[0115]
(Step 020) The same processing as step 010 of the first embodiment is performed.
[0116]
(Step 21) A child block is arranged as shown in FIG. 7 along the outline portion of the roughly-shaped alpha mask.
(Step 022) Of the child blocks arranged in step 211, those that are determined to be arranged in the narrow part of the object are removed. A method for determining whether or not the portion is a thin portion will be described later.
[0117]
(Step 023) A parent block similar to the child block not canceled in step 022 is found with reference to the image data. In the present embodiment, similar blocks are obtained with reference to label data as in the first embodiment.
(Step 024) Step 023 is repeated until the parent blocks for all the child blocks are determined.
[0118]
(Step 025) On the alpha mask, the data of the area corresponding to each child block is replaced with the data of the area of the corresponding parent block reduced to the same number of pixels as the child block.
(Step 026) Step 025 is performed for all child blocks. This means that the inside of all the child blocks of the alpha mask has been replaced once.
(Step 027) Steps 025 and 026 are recursively repeated a predetermined number of times. Thereby, the contour of the alpha mask approaches the contour of the object.
[0119]
In this embodiment, label data is used, but this is not essential. The label data need not be used as in the related art.
[0120]
By performing such processing, since the child block is not arranged in the thin object portion, it is possible to avoid the problem that the portion disappears in many cases.
[0121]
The determination as to whether or not the object is a thin portion in step 022 is performed using the pixel value of the child block to be determined in the alpha mask. Hereinafter, a specific processing method will be described with reference to FIGS. 18, 19, 20, 21, and 22. FIG.
[0122]
FIG. 18 is an enlarged view of the alpha mask of a child block of 16 × 16 pixels. White pixels represent the object and black pixels represent the background.
[0123]
With respect to the alpha mask of the child block, the pixel is sequentially scanned one pixel at a time from the upper left pixel to the upper right pixel, and the number of pixel value changes is counted. Further, the four pixels are scanned from the upper right pixel to the lower right pixel, the lower right pixel to the lower left pixel, and the lower left pixel to the upper left pixel, and the number of changes is counted.
[0124]
The number of changes obtained in this way is always an even number. If the number of changes is four or more, it is determined that the change is a thin part of the object.
[0125]
In the case of FIG. 18, the scanning from the upper left pixel to the upper right pixel and the scanning from the lower right pixel to the lower left pixel change twice from black to white and from white to black, respectively. Since the total is four times, it is determined to be a thin portion.
[0126]
Since the child block is arranged along the outline of the general shape, scanning around the block in this way usually results in two changes as shown in FIG. 21 unless it is a narrow part. When the number of changes is two, it can be extracted by the fractal method.
[0127]
If the number of changes is more than two, it is estimated that the part has a complicated shape or is a thin part. In the case of complex shapes, the fractal method often cannot cope.
[0128]
FIG. 19 shows an example in which the number of changes is six, and FIG. 20 shows an example in which the number of changes is four. In such a case, since there is no self-similarity, it is considered better to avoid the disappearance of the object region by canceling the child block and leaving the schematic shape as it is without forcibly applying the fractal method.
[0129]
In the case shown in FIG. 22, the number of changes is 0, and in this case, the white object region disappears by the replacement conversion. If this small object is noise, etc. and it is OK to disappear, the child block can be left as it is, but if it is known that this is a small object, the user specifies with a mouse etc. The child block is canceled by giving information by a technique.
[0130]
In the above example, in step 022, four sides of the child block were scanned. Although this method is simple and convenient, it does not completely avoid the disappearance of thin object parts.
[0131]
For example, even if the pattern of the child block is as shown in FIG. 21 (a), if the parent block is as shown in FIG. 20 (a), the child block is overwritten by one replacement conversion and the replacement conversion is performed. By repeating this, the object area disappears.
[0132]
In order to avoid such a situation, a parent block is first obtained, and four sides of the parent block are scanned to count the number of changes in the pixel value of the alpha mask. If the number of changes is greater than 2, the parent block and the child block are cancelled.
[0133]
FIG. 23 shows a flowchart in this case. Except for the step 234 for removing the child block and the parent block, it is the same as FIG. 2, and the description of the other steps is omitted.
[0134]
FIG. 17 is a diagram illustrating an example of a positional relationship between the child block 161 and the parent block 162. The example in which the four sides of the child block 161 are scanned and the example in which the parent block 162 is scanned have been described above.
[0135]
In addition, the four sides of the block 163 that shares the center with the child block and has the same size as the parent block 162 may be scanned, and whether or not the part is a thin part may be similarly determined based on the number of changes. In the case of the block 163, since the determination can be made before the search for the parent block, the processing amount can be reduced. Further, a result closer to the determination using the parent block 162 is obtained than using the child block 161.
[0136]
As described above, in the present embodiment, the pixels on the four sides of the parent block are scanned, the shape of the object included in the parent block is estimated, and it is determined whether or not to perform the replacement process. It is possible to leave a contour of a thin portion or a portion having a complicated shape.
[0137]
(Fourth Embodiment) An object region that disappears due to repeated replacement conversion is an object region having a shape smaller than the width of a parent block. Therefore, in order to avoid the disappearance of the object region, it is considered that the parent block and the child block should be made small, although the allowable error of the general shape becomes small.
[0138]
However, originally, when the child block is a small block such as 4 × 4 pixels, if the size is further reduced, the resolution becomes coarse, and an image pattern sufficient for utilizing the self-similarity cannot be obtained.
[0139]
Therefore, in the present embodiment, the original image and the general shape are enlarged without changing the size of the parent block, and the disappearance is avoided by making the object region thicker than the width of the parent block. Then, if necessary, the alpha mask resulting from the extraction is reduced to the original size.
[0140]
FIG. 27 shows a flowchart of this embodiment. The difference from the previous embodiments is that step 271 for enlarging the original image and the schematic shape is performed at the very beginning of the processing. The enlargement of the original image in step 271 is performed using, for example, zero-order interpolation, linear (first-order) interpolation, or a higher-order pixel interpolation method. For example, zero-order interpolation is used for enlarging the approximate shape that is an alpha mask. Subsequent steps are the same as those in the previous embodiments, and a description thereof will be omitted.
[0141]
As described above, in the present embodiment, since the contour is extracted by enlarging the original image, it is possible to accurately extract even the contour of a thin part of the object, which is highly likely to lose its shape by the conventional contour extraction processing method. Become like
[0142]
(Fifth Embodiment) This embodiment is an embodiment relating to a contour extraction process suitable for extracting a contour of an object at an end of an image.
[0143]
In the fractal method, higher extraction accuracy may be obtained by padding a fixed width outside the original image. For example, as shown in FIG. 24, it is assumed that an object 242 is present in a portion indicated by oblique lines in the original image 241, and a rough shape 243 slightly deviated from the contour of the object 242 is given.
[0144]
In the case of the child block 244 arranged as shown in FIG. 24, in order to bring the schematic shape 243 closer to the contour of the object 242, the parent block is positioned as shown in FIG. Is desirably arranged in the block 245 of FIG.
[0145]
Since the search for the parent block is performed within a certain range centered on the child block, the block 245 only needs to be within that range. However, in the case of FIG. 24, the right side of the block 245 protrudes from the image. Therefore, even if the block 245 is within the search range, the error evaluation is not performed and the block 245 is excluded from the parent block candidates. Therefore, in the child block 244, a block different from the illustrated one is selected as the parent block, and the contour cannot be obtained accurately.
[0146]
Therefore, in the present embodiment, the image is expanded outward in advance by an amount corresponding to the possibility of the search range, as in a region 253 indicated by a horizontal line in FIG. This is called padding.
[0147]
In the present embodiment, the pixel value of padding is extrapolated with the outermost pixel value of the image, and the position and width of padding are determined as follows.
[0148]
If it is known in advance that a child block is arranged on the outline of the general shape, a rectangle 251 circumscribing the general shape 243 is first obtained before extracting the object. When the rectangle 251 is obtained, the outer frame 252 of a portion that can be a search range of the parent block when the child block is arranged around the inside is determined.
[0149]
The error inside the outer frame 252 may be calculated as a candidate block when searching for a parent block. Therefore, padding is performed by the width of the outer frame 252 protruding from the image 241. At this time, padding of the portion where the outer frame 252 does not protrude such as the upper left and lower right sides of the image 241 is not necessary. However, when there is no particular limitation on the capacity of the memory for storing the image data, A sufficient width of padding may be applied to all the left and right sides. By padding the outside of the image in this way, it is possible to avoid the problem that a contour line near the edge of the image cannot be accurately extracted.
[0150]
FIG. 28 is a flowchart of the object extraction processing according to the present embodiment. At the beginning of the object extraction by the so-called fractal method, padding is performed by executing step 281. Other steps are the same as in the previous embodiments.
[0151]
As described above, in the present embodiment, by providing padding with a predetermined width outside the original image, the contour existing at the end of the image can be accurately extracted.
[0152]
(Sixth Embodiment) This embodiment is an embodiment relating to the contour extraction processing when the deviation between the approximate shape and the correct shape is large.
[0153]
FIG. 30 shows an example of a case where an object contour cannot be correctly extracted by the conventional fractal method. FIG. 30 is a diagram showing a contour 301 of an object, a schematic shape 302 thereof, and a child block 303 which is one of child blocks arranged so that the center is located on the outline of the schematic shape 302.
[0154]
Since the child block 303 does not include the object outline 301, the block 311 shown in FIG. 31, for example, is selected as a block similar to the child block 303 in the search for the parent block. As described above, when the difference between the approximate shape and the correct shape is large, the self-similar structure of the object cannot be extracted, and as a result, the contour of the object may not be correctly extracted.
[0155]
Therefore, in the present embodiment, the parent block is arranged first. This is shown in FIGS. 32 and 33. First, the parent block is arranged such that the center is located on the outline of the schematic shape 302 as in a block 321 in FIG. Then, a block having a length and width that are similar to the parent block is searched for.
[0156]
In this case, since the object outline 301 is included at the lower left of the block 321, if a block similar to this block 321 is searched for a half-width and half-length block, the block 331 is selected as a child block as shown in FIG.
[0157]
FIG. 29 is a flowchart of the contour extraction processing according to the present embodiment. Steps 291 to 293 are processing for arranging parent blocks first and searching for similar child blocks. Subsequent replacement processing is the same as that of the previous embodiment.
[0158]
(Step 291) Parent block P is arranged at the outline of the alpha mask. Similar to FIG. 7 showing an example of arranging child blocks, the outline of the outline shape passes through the center of each parent block P, and adjacent parent blocks are arranged so as to overlap.
(Step 292) Find a child block C similar to the parent block P.
(Step 293) The above step 292 is performed for all parent blocks, and child blocks are determined respectively.
Since the following is the same replacement processing as before, the description will be omitted.
[0159]
As described above, in the present embodiment, the parent block is arranged on the outline of the schematic shape first, and the child block is searched later. Becomes larger, so that it becomes easier to extract the self-similar structure of the object than before. As a result, the object can be easily extracted correctly.
[0160]
(Seventh Embodiment) The present embodiment relates to a contour extraction process suitable for obtaining an alpha mask that also represents an intermediate state between an object region and a background region. Hereinafter, the present embodiment will be described.
[0161]
FIG. 36 is a flowchart of the contour extraction processing according to the present embodiment.
(Step 361) As shown in FIG. 34, the image data 342 and the roughly shaped alpha mask 341 are enlarged at a predetermined magnification to generate enlarged image data 344 and an enlarged alpha mask 343. In the present embodiment, the enlargement ratio is twice as high in the horizontal and vertical directions.
(Step 362) The above-mentioned fractal processing is performed on the enlarged image data 344 and the enlarged alpha mask 343 to extract an accurate alpha mask. Here, the pixel value of each pixel of the alpha mask is 0 for the background and 255 for the object.
(Step 363) An alpha mask for the image data 342 is obtained by reducing the alpha mask obtained by the fractal method to the original size.
[0162]
FIG. 35 is a view for explaining the process of reducing the enlarged alpha mask to the original size in step 363. In the present embodiment, since the original size image is enlarged by two times vertically and horizontally to perform the fractal process, the four pixels 353 of the enlarged alpha mask correspond to the pixels 352 of the alpha mask obtained by reduction. In the present embodiment, the average value of the pixel values of the pixels 353 is used as the pixel values of the pixels 352 of the alpha mask obtained by reduction. Therefore, the pixel value of each pixel of the alpha mask obtained by reduction is an average value of the four pixels.
[0163]
In the present embodiment, each pixel of the enlarged alpha mask is given a pixel value of 0 in the case of a background, and a pixel value of 255 in the case of an object. Therefore, if the number of object pixels is 0, 1, 2, 3, or 4 among the four enlarged alpha mask pixels corresponding to each pixel of the reduced alpha mask, the average value is respectively , 0, 64, 128, 192, 255.
[0164]
In FIG. 35, since the pixel 352 includes two object pixels, the pixel value of the pixel 352 is 128. If the values of the corresponding pixels are all the same as in the pixel 354, the corresponding pixel 351 has the same value.
[0165]
By using the pixel value of each pixel of the alpha mask obtained in this way as a pixel mixture ratio when synthesizing the extracted object with another image, the jagged feeling given by the outline is improved, and a more natural synthesis Images can be obtained.
[0166]
The formula for calculating the pixel value of the composite image is as follows: X is the pixel value of the pixel of the image in which the extracted object is captured, A is the value of the alpha mask corresponding to the pixel, and A is the value of the pixel at the same position in another image to be combined. If the pixel value is Y,
“Pixel value of composite image” = (A × X + (255−A) × Y) / 255
It becomes like. As described above, the pixel value of each pixel of the alpha mask can be regarded as representing transparency in which 0 is completely transparent and 255 is completely opaque.
[0167]
In the above-described enlargement processing, pixel interpolation is performed on the image data by, for example, primary interpolation, and on the alpha mask by, for example, zero-order interpolation. Also, for example, when an alpha mask for a reduced image called a thumbnail image is generated, there is no need to enlarge the original image, and after generating an alpha mask having values of 0 and 255 in the original size, What is necessary is just to reduce in the manner described.
Further, if an image corresponding to transparency, such as the image compression standard PNG or GIF, is generated and stored using the image data and the generated alpha mask, the image becomes a general-purpose object image and can be reused.
[0168]
As described above, in the present embodiment, when generating an alpha mask by reducing an enlarged alpha mask obtained by enlarging an original image and performing contour extraction, a multi-step calculation is performed with reference to a plurality of pixel values of the enlarged alpha mask. Is used as each pixel value of the alpha mask. Therefore, by using the pixel value of the alpha mask as transparency when extracting an object, it is possible to obtain a smooth outline of the object.
[0169]
【The invention's effect】
As described above, according to the present invention, in the fractal method, the contour of an object can be accurately obtained even when an edge such as another object contour is close or the object region is narrow.
[Brief description of the drawings]
FIG. 1 is a flowchart of a contour extraction process according to a first embodiment of the present invention.
FIG. 2 is a flowchart of a contour extraction process according to a third embodiment of the present invention.
FIG. 3 is an example of an extraction target image.
FIG. 4 is an example of an alpha mask for the image of FIG. 3;
FIG. 5 shows a state in which a schematic shape is set.
FIG. 6 is a schematic shape of an alpha mask.
FIG. 7 shows a state where child blocks are arranged.
FIG. 8 shows an example of a certain child block and its parent block.
FIG. 9 is an example of another child block and its parent block.
FIG. 10 is an example in which another parent block is selected in FIG. 9;
FIG. 11 shows how an error is calculated in FIG.
FIG. 12 shows how an error is calculated in FIG.
FIG. 13 is an example in which an error is calculated according to the present invention.
FIG. 14 is another example in which an error is calculated according to the present invention.
FIG. 15 shows an example in which label data is input.
FIG. 16 is an example in which a child block is arranged in a thin portion.
FIG. 17 is an example of a child block arranged in a thin portion and a parent block around the child block.
FIG. 18 is an alpha mask in a child block.
FIG. 19 shows an example of an alpha mask pattern: the number of changes is 6.
FIG. 20 shows an example of an alpha mask pattern: the number of changes is four.
FIG. 21 shows an example of an alpha mask pattern: the number of changes is 2.
FIG. 22 shows an example of an alpha mask pattern: the number of changes is 0.
FIG. 23 is a flowchart of a modification of the contour extraction processing according to the third embodiment of the present invention.
FIG. 24 is an example in which an object is present at an end of an image.
FIG. 25 is an example in which padding is provided at an end of an image to expand the image.
FIG. 26 is an example in which a region of a first object is extracted and a mask is set.
FIG. 27 is a flowchart of contour extraction processing according to the fourth embodiment of the present invention.
FIG. 28 is a flowchart of contour extraction processing according to the fifth embodiment of the present invention.
FIG. 29 is a flowchart of contour extraction processing according to the sixth embodiment of the present invention.
FIG. 30 is an example in which child blocks are arranged.
FIG. 31 is an example in which a parent block is obtained.
FIG. 32 is an example in which parent blocks are arranged.
FIG. 33 is an example in which a child block is obtained.
FIG. 34 is a view for explaining the concept of processing for enlarging image data and an alpha mask;
FIG. 35 is a view for explaining the concept of an alpha mask reduction process.
FIG. 36 is a flowchart of contour extraction processing according to the seventh embodiment of the present invention.
FIG. 37 is an example in which expansion and contraction are performed in a schematic shape.
FIG. 38 is a diagram showing a sampling area when a color table is created.
FIG. 39 is an example of locally creating a color table.
FIG. 40 is an exemplary view for explaining the schematic configuration of a computer used in each embodiment of the present invention;
[Explanation of symbols]
031 Ship whose contour is to be extracted
032 Another ship in the vicinity of ship 031
051 Outline of ship 031
081, 091, 161, 244, 303, 331 Child Block
082,093,121,162,245,311,321 Parent block
114 Part of outline of ship 032
115 Part of the outline of ship 031
111, 132 Child block
112,133 Reduced parent block
113, 122, 134 Difference
131, 135 Part of the boundary line of the label data
151 Label Data Boundary
241 images
242 object
243 Schematic shape of object 242
253 Expansion area by padding processing
301 Part of object contour
302 Part of schematic shape

Claims (17)

A first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data;
A second step of setting a plurality of search reference blocks so as to be shifted from each other along the outline of the general shape represented by the shape data;
For each of the search reference blocks, a symbol in a region larger than the search reference block and corresponding to this block on the image data is most similar to a symbol in an area corresponding to the search reference block on the image data. A third step of searching for similar blocks that
And a fourth step of repeatedly performing a process of replacing the shape data of the area corresponding to each search reference block with a reduced version of the shape data of the area corresponding to each similar block until a predetermined condition is satisfied,
In the first step,
Also receives label data indicating the certainty of the existence of the target object in each pixel in the image data,
In the third step,
Referring to the label data, calculating the weighted sum of the errors of the entire block by ignoring or neglecting the error between pixels having a low possibility that the target object is present, and searching for a similar block in which the weighted sum is minimized. A method for extracting an object image, comprising: The shape data is
Represents a region of the approximate shape of the target object, each pixel of which is in one-to-one correspondence with each pixel of the image data,
The label data is
The image data is divided into at least two types of regions out of three types of a target object fixed region, a non-target object fixed region, and an undetermined region, and each pixel corresponds to each pixel of the image data on a one-to-one basis. And
In the third step,
For each of the plurality of candidate blocks that are candidates for the similar block,
The data of the area corresponding to the search reference block in the image data, and the error of each corresponding point of the data obtained by reducing the area corresponding to the candidate block in the image data,
If the weighted sum is, both of the corresponding points are present in the area divided into the target object defined area by the label data, or both are present in the area divided into the non-target object defined area, Ignoring or neglecting the error about the corresponding point,
2. The object extraction method according to claim 1, wherein a candidate block that minimizes the weighted sum of the errors is determined as the similar block. In the third step,
For each of the plurality of candidate blocks that are candidates for the similar block,
The data of the area corresponding to the search reference block in the image data, and the error of each corresponding point of the data obtained by reducing the area corresponding to the candidate block in the image data,
The weighted sum, if both the corresponding points are present in the area divided into the target object confirmed area by the label data, or if both are present in the area divided into the non-target object confirmed area, A weighting factor for multiplying the error for the corresponding point by 0 or a small value is obtained,
3. The object extraction method according to claim 2, wherein a candidate block having a minimum weighted sum of the errors is determined as the similar block. A first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data;
A plurality of search reference blocks are set so as to be shifted from each other along the outline of the general shape represented by the shape data, and among these, a pattern of an area corresponding to the search reference block on the shape data is a predetermined pattern. The second step of canceling the setting of the pattern of
For each of the search reference blocks, a third similar block which is larger than the search reference block and in which the design of the region corresponding to the block on the image data is most similar to the design of the region corresponding to the search reference block is searched. Steps and
And a fourth step of repeatedly performing a process of replacing data of an area corresponding to each of the search reference blocks in the shape data with reduced data of an area corresponding to each similar block until a predetermined condition is satisfied. Object image extraction method. In the second step,
Along the outline of the schematic shape represented by the shape data, a plurality of search reference blocks are set at different positions,
For an inspection block having the same center position as each search reference block, a pattern of an area corresponding to the inspection block on the shape data is examined,
5. The object image extracting method according to claim 4, wherein the setting of the search reference block is canceled when the pattern of the inspection block is a predetermined pattern. A first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data;
A second step of setting a plurality of search reference blocks so as to be shifted from each other along the outline of the general shape represented by the shape data;
For each search reference block, while searching for a similar block that is larger than the search reference block and the symbol of the region corresponding to the block on the image data is most similar to the symbol of the region corresponding to the search reference block, A third step of canceling the setting of a similar block in which a pattern of a corresponding area on the shape data is a predetermined pattern and a search reference block corresponding to the similar block;
And a fourth step of repeatedly performing a process of replacing data of an area corresponding to each of the search reference blocks in the shape data with reduced data of an area corresponding to each similar block until a predetermined condition is satisfied. Object image extraction method. Receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data, and defining at least one of the upper, lower, left, and right sides of both the image data and the shape data by a predetermined width A first step to extend only
A second step of setting a plurality of search reference blocks so as to be shifted from each other along the outline of the general shape represented by the shape data;
For each of the search reference blocks, a third similar block which is larger than the search reference block and in which the design of the region corresponding to the block on the image data is most similar to the design of the region corresponding to the search reference block is searched. Steps and
And a fourth step of repeatedly performing a process of replacing data of an area corresponding to each of the search reference blocks in the shape data with reduced data of an area corresponding to each similar block until a predetermined condition is satisfied. Object image extraction method. A first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data;
A second step of setting a plurality of search reference blocks so as to be shifted from each other along the outline of the general shape represented by the shape data;
For each search reference block, a third similar block that is smaller than the search reference block and that has a pattern in the area corresponding to the block on the image data most similar to the pattern in the area corresponding to the search reference block is searched. Steps and
And a fourth step of repeatedly performing a process of replacing the data of the area corresponding to each similar block with the reduced data of the area corresponding to each search reference block in the shape data until a predetermined condition is satisfied. Object image extraction method. A first method of receiving image data including a target object to be extracted and shape data representing a region of a schematic shape of the target object in the image data, and enlarging these to generate enlarged image data and enlarged shape data; Steps and
A second step of setting a plurality of search reference blocks so as to be shifted from each other along the outline of the general shape represented by the enlarged shape data;
For each of the search reference blocks, a search is performed for a similar block that is larger than the search reference block, and in which the design of the region corresponding to the block on the enlarged image data is most similar to the design of the region corresponding to the search reference block. Step 3 of repeating the process of replacing the data of the area corresponding to each similar block with the reduced data of the area corresponding to each search reference block in the enlarged shape data until a predetermined condition is satisfied. 4 steps,
A fifth step of reducing the enlarged shape data obtained in the fourth step to the original size. A first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data;
A second step of generating shape data in which the shape of the target object is extracted using the shape data and the image data;
A third step of reducing the extracted shape data to a predetermined size and generating reduced shape data in which a pixel value of one pixel is obtained by using pixel values of a plurality of corresponding pixels of the extracted shape data; An object image extraction method comprising: A first step of receiving image data including a target object to be extracted and shape data representing an area of a schematic shape of the target object in the image data, and enlarging these to generate enlarged image data and enlarged shape data When,
A second step of generating enlarged shape data obtained by extracting the shape of the target object using the enlarged shape data and the enlarged image data;
A third step of reducing the extracted enlarged shape data to an original size and generating shape data in which a pixel value of one pixel is obtained by using pixel values of a plurality of corresponding pixels of the extracted shape data; An object image extraction method comprising: On the computer,
A first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data;
A second step of setting a plurality of search reference blocks so as to be shifted from each other along the outline of the general shape represented by the shape data;
For each of the search reference blocks, a similar block that is a block larger than the search reference block and whose symbol of the area corresponding to this block on the image data is most similar to the symbol of the area corresponding to the search reference block A third step of searching;
And a fourth step of repeatedly performing a process of replacing the shape data of the area corresponding to each search reference block with a reduced version of the shape data of the area corresponding to each similar block until a predetermined condition is satisfied,
In the first step,
Also receives label data indicating the certainty of the existence of the target object in each pixel in the image data,
In the third step,
Referring to the label data, ignore or neglect the error between pixels having a low possibility that the target object is present, calculate the weighted sum of the errors of the entire block, and search for a similar block in which the weighted sum is minimized. An object image extraction program characterized by being performed. The shape data is
Represents the schematic shape of the target object, each pixel of which is in one-to-one correspondence with each pixel of the image data;
The label data is
The image data is divided into at least two types of regions out of three types of a target object fixed region, a non-target object fixed region, and an undetermined region, and each pixel corresponds to each pixel of the image data on a one-to-one basis. age,
In the third step,
For each of the plurality of candidate blocks that are candidates for the similar block,
The data of the area corresponding to the search reference block in the image data, and the error of each corresponding point of the data obtained by reducing the area corresponding to the candidate block in the image data,
The weighted sum is determined when the corresponding points are both present in an area that is divided into target object defined areas by the label data, or when both are present in an area that is partitioned into non-target object defined areas. Ignoring or neglecting errors about points,
13. The non-transitory computer-readable storage medium according to claim 12, wherein a candidate block that minimizes the weighted sum of the errors is determined as the similar block. On the computer,
A first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data;
A plurality of search reference blocks are set so as to be shifted from each other along the outline of the general shape represented by the shape data, and among these, a pattern of an area corresponding to the search reference block on the shape data is a predetermined pattern. The second step of canceling the setting of the pattern of
For each of the search reference blocks, a third similar block which is larger than the search reference block and in which the design of the region corresponding to the block on the image data is most similar to the design of the region corresponding to the search reference block is searched. Steps and
And performing a fourth step of repeatedly performing a process of replacing data of an area corresponding to each of the search reference blocks in the shape data with reduced data of an area corresponding to each similar block until a predetermined condition is satisfied. An object image extraction program to be used. In the second step,
Along the outline of the schematic shape represented by the shape data, a plurality of search reference blocks are set at different positions,
For an inspection block having the same center position as each search reference block, a pattern of an area corresponding to the inspection block on the shape data is examined,
15. The object image extraction program according to claim 14, wherein the setting of the search reference block is canceled when the pattern of the inspection block is a predetermined pattern. On the computer,
A first step of receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data;
A second step of setting a plurality of search reference blocks so as to be shifted from each other along the outline of the general shape represented by the shape data;
For each search reference block, while searching for a similar block that is larger than the search reference block and the symbol of the region corresponding to the block on the image data is most similar to the symbol of the region corresponding to the search reference block, A third step of canceling the setting of a similar block in which a pattern of a corresponding area on the shape data is a predetermined pattern and a search reference block corresponding to the similar block;
And performing a fourth step of repeatedly performing a process of replacing data of an area corresponding to each of the search reference blocks in the shape data with reduced data of an area corresponding to each similar block until a predetermined condition is satisfied. An object image extraction program to be used. On the computer,
Receiving image data including a target object to be extracted, and shape data representing a region of a schematic shape of the target object in the image data, and defining at least one of the upper, lower, left, and right sides of both the image data and the shape data by a predetermined width A first step of extending a plurality of search reference blocks along the outline of the schematic shape represented by the shape data, and
For each of the search reference blocks, a third similar block which is larger than the search reference block and in which the design of the region corresponding to the block on the image data is most similar to the design of the region corresponding to the search reference block is searched. Steps and
And performing a fourth step of repeatedly performing a process of replacing data of an area corresponding to each of the search reference blocks in the shape data with reduced data of an area corresponding to each similar block until a predetermined condition is satisfied. An object image extraction program to be used.

Images