JP2009211607A - Object extraction device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow an object extraction device to correctly extract an object while reducing the load on a user. <P>SOLUTION: The object extraction device includes: an initial region setting means (210) that encircles an object region in an image with an encircling line to set the region within the encircling line as an initial region including the object region; a background region estimation means (300) that estimates a pixel in a region apart from the encircling line by a distance shorter than a distance between the pixel and the object as a background pixel constituting the background region of the object region, and estimates a region in the initial region that is constituted by pixels whose background pixel and pixel information are similar to each other in comparison with those of the object region as the background region; and object region determination means (400) that determines a region where the estimated background region is excluded from the initial region as the object region. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technical field of an object extracting apparatus and an object extracting method for extracting an object such as a specified person, vehicle, or building from an image such as a photograph.

  As this type of object extraction apparatus, there is an apparatus that automatically estimates and extracts an outline or the like for an object in an image designated by a user. For example, Patent Document 1 discloses a technique in which when a user designates a point on an object, a pixel similar to that point is searched and the object is extracted. Patent Document 2 discloses a technique of estimating an object outline by using a dynamic outline method (snakes method) and extracting the object. Non-Patent Document 1 discloses a technique in which a small region and a middle region are set around a contour designated by a user, and the contour of an object is estimated using similarity (fractality) of an image in each region. .

JP 2002-92613 A JP 2006-190307 A “Self-Affine Mapping System and Its Application to Object Contour Extraction”, IEEE Transaction on Image Processing, vol.9, No11, November, 2000

  However, since the technique according to Patent Document 1 is a form in which a user designates one point when designating an object, for example, when an object is composed of a plurality of regions having different colors, the object is accurately extracted. There is a risk that it will not be possible. Moreover, since the technique according to Patent Document 2 requires the user to specify a contour with relatively high accuracy, the burden on the user increases. In addition, since the techniques according to Patent Document 2 and Non-Patent Document 1 both require relatively complicated processing, there is a risk of increasing processing time and complication of the apparatus configuration.

  As described above, the above-described technique has a technical problem that it is difficult to achieve both improvement in extraction accuracy and reduction of the burden on the user.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an object extraction device and an object extraction method that can extract an object with high accuracy while reducing a user's burden.

  In order to solve the above-described problem, the object extraction device of the present invention surrounds an outer area of an object area in an image with a surrounding line at a distance from the object area. Initial region setting means for setting as an initial region including, and when the initial region is set, a background region that is a background of the object region is a pixel in a region separated from the enclosing line by a distance shorter than the distance. A background area estimating means for estimating a background pixel as a background area by estimating a background pixel in the initial area and a pixel having pixel information similar to that of the object area in the initial area; An object area determination that determines an area excluding the estimated background area from the object area as the object area And means.

  In order to solve the above-described problem, the object extraction method of the present invention surrounds the outside of the object area in the image with a surrounding line at a distance from the object area, so that the area inside the surrounding line is the object area. An initial region setting step for setting as an initial region including: and when the initial region is set, a background region that is a background of the object region is a pixel in a region that is separated from the enclosing line by a distance shorter than the distance. A background region estimation step for estimating a background pixel to be configured, and estimating, as the background region, a region formed by pixels having pixel information similar to the background region and pixel information in the initial region; and the initial region An object area determination that determines an area excluding the determined background area as the object area from And a step.

  The operation and other advantages of the present invention will become apparent from the best mode for carrying out the invention described below.

  Hereinafter, an object extraction apparatus and an object extraction method as the best mode for carrying out the invention will be described.

  In order to solve the above-described problem, the object extraction device of the present embodiment surrounds the outside of the object area in the image with a surrounding line at a distance from the object area, so that the area inside the surrounding line is the object. An initial area setting means for setting as an initial area including an area; and when the initial area is set, a background area that is a background of the object area is a pixel in an area separated from the enclosing line by a distance shorter than the distance A background region estimation means for estimating, as the background region, a region composed of pixels in which the pixel information of the background region is similar to that of the object region in the initial region; An object area that determines an area excluding the estimated background area from the area as the object area And a determination unit.

  According to the object extraction device of the present embodiment, at the time of the operation, first, the outside of the object area in the image is surrounded by a surrounding line at a distance from the object area, so that the area inside the surrounding line includes the object area. Set as the initial area. The encircling line is drawn on the image, for example, when the user operates a mouse, a touch pen, or the like. Here, the enclosing line should be drawn as a closed curve that does not overlap the object area. However, if it is not drawn as a closed curve, it is also possible to complement by connecting both ends of the line with straight lines or curves.

  When the initial area is set, the pixels in the area that is separated from the enclosing line by a distance shorter than the distance described above (that is, the distance between the enclosing line and the object area) constitutes the background area that is the background of the object area. This is estimated as a background pixel. That is, since the surrounding line is drawn at a distance from the object (in other words, so as not to overlap), a pixel near the surrounding line can be estimated as a background pixel. Here, the pixel estimated as the background pixel is typically a pixel within a range of several pixels from the surrounding line. Further, the pixels estimated as background pixels may be all pixels within a range of several pixels along the surrounding line, for example, pixels within a range of several pixels from several points on the surrounding line. There may be. That is, the pixel estimated as the background pixel may be a part of the pixel in the vicinity of the surrounding line. In this case, it is possible to make the process of estimating the background area described later relatively simple.

  When the background pixel is estimated, an area composed of pixels whose pixel information is similar to that of the object area in the initial area is estimated as the background area. That is, an area formed by pixels whose pixel information is closer to the background pixel than the object area is estimated as the background area. Specifically, for example, a threshold value for the pixel information is set in advance, and it is determined whether the pixel information is similar based on the threshold value. And the area | region comprised by the pixel determined that pixel information is similar to a background pixel is estimated as a background area. The “pixel information” here is information that changes for each pixel depending on the displayed image, such as color tone, luminance, and texture (frequency component). For example, when color tone is used as pixel information, It is possible to determine whether or not they are similar by comparing the RGB histogram of neighboring pixels with the RGB value of each pixel.

  When the background area is estimated, an area excluding the background area from the initial area is determined as the object area. The determined object area is output as, for example, image data as the object area or image data obtained by cutting out the object area. The output data is displayed on a screen, for example, or stored in a memory or the like. Alternatively, it may be distributed via the Internet or the like.

  In the present embodiment, in particular, the background region is estimated with high accuracy by the background region estimation means, so that the object region can be determined with high accuracy. That is, it is possible to extract the object area while reducing the possibility that the object area to be extracted is partially cut or the area that should not be extracted is extracted. Further, since the object area is determined based on the background area, the object can be suitably extracted even when the object area is composed of a plurality of areas having different colors.

  Furthermore, since the background area is estimated with high accuracy as described above, the accuracy at the time of setting the initial area may not be relatively high. That is, even when the surrounding line is drawn roughly to some extent, it is possible to extract an object with high accuracy. In other words, as long as the surrounding line is drawn as a closed curve including the object region, the effect of the present embodiment is exhibited. Therefore, the burden on the user can be reduced. However, it is also possible to increase the extraction accuracy by drawing a surrounding line in accordance with the contour of the object (that is, along the contour).

  As described above, according to the object extraction device according to the present embodiment, it is possible to accurately extract an object region while reducing the burden on the user.

  In one aspect of the object extraction device of the present embodiment, the background region estimation means sets a background pixel acquisition region so as to at least partially overlap the surrounding line, and pixel information of pixels in the background pixel acquisition region is obtained. , Background pixel acquisition means for acquiring the pixel information of the background pixel, and setting a background pixel search region so as to at least partially surround the background pixel acquisition region, and the acquired pixel in the background pixel search region Background pixel search means for searching for pixels having pixel information similar to the information and the object area.

  According to this aspect, when the initial area is set by the initial area setting means, first, the background pixel acquisition area is set by the background pixel acquisition means. A plurality of background pixel acquisition regions are typically set as regions including several pixels centered on one point on the enclosing line. When the background pixel acquisition area is set, the pixel information of the pixels in the background pixel acquisition area is acquired as the pixel information of the background pixels.

  Subsequently, a background pixel search area is set by the background pixel search means. The background pixel search area is typically set as a circular area centered on the background pixel acquisition area. However, it may be set as an area having a rectangular shape or a more complicated shape. The size of the area is determined by the size of the entire image, for example. Alternatively, it may be determined according to the size of the set initial area. A plurality of background pixel search areas are typically set, but the sizes may be different from each other.

  When the background pixel search region is set, pixels having pixel information similar to the pixel information acquired as the background pixel and the object region are searched in the background pixel search region. That is, in the background pixel search area, a pixel having pixel information closer to the background pixel than the object area is searched. As a result, the background area can be reliably estimated. In addition, if the search range is narrowed down to the inside of the surrounding line in the background pixel search area, the search process can be lightened.

  A plurality of background pixel acquisition areas and background pixel search areas described above are set so that adjacent background pixel search areas partially overlap each other. In other words, if the background pixel search area can be set so as to overlap, the background pixel acquisition area and the background pixel search area can be set with a certain interval. By increasing the interval between the background pixel acquisition region and the background pixel search region, the number of background pixel acquisition regions and background pixel search regions set as a whole decreases. Therefore, it is possible to reduce processing for pixel acquisition and search.

  As described above, according to the object extraction device according to this aspect, it is possible to reliably estimate the background region while making the above-described processes relatively light. Therefore, it is possible to extract the object region more preferably.

  In the aspect in which the background pixel estimation means includes the background pixel acquisition means and the background pixel search area, the background pixel search means obtains pixel information similar to the acquired pixel information in the pixels in the background pixel search area. You may comprise so that the magnitude | size of the said background pixel search area may be changed based on the ratio of the pixel which has.

  According to this configuration, first, a pixel search is performed, and a ratio of pixels having pixel information similar to the pixel information acquired as the background pixel in the pixels in the background pixel search region is obtained. Based on this ratio, the size of the background pixel search area is changed. When the size of the background pixel search area is changed, the pixel search is performed again.

  Specifically, for example, when the ratio of pixels similar to the background pixel is large, there is a high possibility that the search range does not sufficiently cover the boundary of the object region, so the background pixel search region is enlarged. By increasing the background pixel search area, the background area can be estimated more appropriately. On the other hand, when the ratio of pixels similar to the background pixel is small, it is considered that there are many portions related to the object region, and therefore the background pixel search region is reduced. When the background pixel search area is made small, the search range is narrowed, so that the processing for searching becomes light.

  As described above, by changing the size of the background pixel search area, it becomes possible to search for pixels more suitably. Therefore, the background area estimation means can more suitably estimate the background area.

  In another aspect of the object extraction device of the present embodiment, the object region determination unit includes an object region estimation unit that estimates an object region based on the estimated background region, the estimated background region, and the estimated Clustering means for clustering whether each pixel in the initial area is the background area or the object area by comparing with pixel information in the object area. In the clustering means, An area constituted by pixels that are determined to be present is determined as an object area.

  According to this aspect, when the background area is estimated, the object area is estimated by the object area estimating means. Then, each pixel in the initial region is compared with the pixel information in the estimated background region and the estimated object region by the clustering means, and clustering (that is, classification) is performed as to whether the pixel is the background region or the object region. The

  After clustering is performed, an area configured by pixels classified as an object area is determined as the object area. In this aspect, since the clustering is further performed after the background area is estimated, the object area can be extracted with high accuracy.

  Note that the clustering method is preferably a method capable of nonlinear separation and allowing rejection. In the case of a technique that allows rejection, pixels that are not classified into either the background area or the object area are classified as rejects and are excluded from the object area. Thereby, it is possible to further improve the extraction accuracy of the object region.

  In clustering, the initial region may be divided into regions (vertical division, horizontal division, cross division, etc.). Thereby, for example, even when the object region or the background region is composed of a plurality of regions having different colors, the object region can be suitably extracted.

  In another aspect of the object extraction device of the present embodiment, effective range setting means for setting an area having a smaller diameter than the image in the image as an effective range in which the encircling line is effective, and exceeding the effective range Correction means for correcting the drawn encircling line as a line drawn toward the edge of the image is further provided.

  According to this aspect, before the initial region is set (that is, before the surrounding line is drawn), an area having a smaller diameter than the image in the image is set as an effective range in which the surrounding line is valid. The effective range is typically set as a region whose diameter is several millimeters to several centimeters smaller than the entire image (that is, a region slightly smaller than the image).

  When the encircling line is drawn after the effective range is set, the encircling line drawn beyond the effective range is corrected as a line drawn toward the edge of the image. More specifically, it is corrected as, for example, a line extending in the extension direction of the effective surrounding line or a perpendicular to the edge. The line reaching the edge of the image is connected to the line reaching the edge of another image by a line along the edge of the image. That is, even if it is corrected, the surrounding line is a closed curve.

  If configured as described above, for example, when the object area is an area that touches the edge of the image, it is not necessary to draw a surrounding line that touches the edge of the image. That is, the user can draw a surrounding line surrounding the object area in the image without being aware of the edge of the image, and the surrounding line is automatically corrected to be along the edge of the image. Therefore, the initial region can be set appropriately. Therefore, it is possible to suitably extract the object area without increasing the burden on the user.

  In another aspect of the object extraction device of the present embodiment, in a moving image composed of a plurality of temporally continuous images, an object region determined in one image is enlarged, and the enlarged object region is changed to the one object. Automatic initial area setting means for setting as an initial area in another image displayed next to the first image.

  According to this aspect, when an object area is determined in one image in a moving image composed of a plurality of temporally continuous images, the determined object area is first enlarged by the automatic initial area setting means. The enlarged object area is determined as an initial area in another image displayed next to one image. In other words, the object area determined in one image is enlarged so that the object area is included in the newly set initial area even when the object area is moved in the next displayed image. The Although the enlargement ratio and the enlargement direction may be set in advance as predetermined values, for example, the accuracy can be improved by setting the enlargement ratio and the enlargement direction based on the moving speed and direction of the object area.

  As described above, when the initial region in another image is set, the background region is similarly estimated in the other image, and the object region is determined. That is, an object area is determined for each of a plurality of temporally continuous images. Therefore, for example, if the object area is surrounded by a surrounding line on the screen where the user is paused (that is, one image), the initial area is automatically set and the object area is extracted in the subsequent image. Therefore, it is possible to suitably extract an object from a moving image without increasing the burden on the user.

  In order to solve the above-described problem, the object extraction method of the present embodiment surrounds the outside of the object area in the image with a surrounding line at a distance from the object area, so that the area inside the surrounding line is the object. An initial region setting step for setting as an initial region including a region; and when the initial region is set, a background region that is a background of the object region is a pixel in a region that is separated from the surrounding line by a distance shorter than the distance A background region estimation step of estimating, as the background region, a region formed by pixels having pixel information similar to the background region and the pixel information in the initial region; An object area that determines an area obtained by removing the determined background area from the area as the object area And a determination step.

  According to the object extraction method of the present embodiment, as in the case of the object extraction device of the present embodiment described above, the background region is estimated with high accuracy by setting the initial region. Can be extracted. That is, it is possible to extract the object region with high accuracy while reducing the burden on the user.

  Note that the object extraction method of the present embodiment can also adopt various aspects similar to the various aspects of the object extraction device of the present embodiment described above.

  The effect | action and other gain of this embodiment are clarified from the Example demonstrated below.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First embodiment>
First, the configuration of the object extraction device according to the first embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the object extracting apparatus according to the first embodiment.

  In FIG. 1, an object extraction apparatus according to the first embodiment includes an image input unit 110, an image storage unit 120, a control unit 130, an image display unit 140, an operation unit 150, an initial region setting unit 210, The effective range setting unit 220, the correction unit 230, the background region estimation unit 300, and the object region determination unit 400 are configured.

  The image input unit 110 is, for example, a tuner that receives and decodes broadcast waves, Internet input, and a photographing device such as a camera, and inputs image data to the control unit 130.

  The image storage unit 120 is a storage medium such as an HDD (Hard Disc Drive) or a memory device, and inputs the stored image data to the control unit 130.

  The control unit 130 is, for example, a control unit including an arithmetic circuit and the like, and controls other parts by performing arithmetic processing on the input data and outputting it. Further, the apparatus or the whole system including the apparatus may be controlled.

  The image display unit 140 is a display monitor, for example, and displays images input from the image input unit 110 and the image storage unit 120. Moreover, you may comprise as a touchscreen which can input a surrounding line by touching.

  The operation unit 150 is, for example, a mouse or a touch pen, and is a tool for the user to input a surrounding line in the image.

  The initial area setting unit 210 is an example of the “initial area setting unit” in the present invention, and sets the inside of the enclosing line drawn on the operation unit 150 or the like as an initial area.

  The effective range setting unit 220 is an example of the “effective range setting unit” of the present invention, and sets an effective range in which a writing line is effective for an image displayed on the image display unit 140 or the like. The effective range may be stored in a memory or the like, for example, or may be calculated from the size of the image.

  The correction unit 230 is an example of the “correction unit” of the present invention, and corrects the surrounding line that exceeds the effective range by performing arithmetic processing.

  The background region estimation unit 300 is an example of the “background region estimation unit” of the present invention, and estimates the background region in the initial region by performing arithmetic processing or the like. In addition, a background pixel acquisition unit 310 that is an example of the “background pixel acquisition unit” of the present invention and a background pixel search unit 320 that is an example of the “background pixel search unit” of the present invention are provided.

  The object area determination unit 400 is an example of the “object area determination unit” of the present invention, and determines an object area from the estimated background area. Further, an object region estimation unit 410 that is an example of the “object region estimation unit” of the present invention and a clustering unit 420 that is an example of the “clustering unit” of the present invention are provided.

  Next, an object extraction method according to the first embodiment will be described with reference to FIGS. 2 to 11 in addition to FIG. FIG. 2 is a flowchart showing a flow of a series of processes of the object extraction method according to the first embodiment, and FIG. 3 is a plan view showing objects and enclosing lines drawn in the image. FIG. 4 is a plan view showing an effective range set in an image, and FIG. 5 is a plan view showing correction of a surrounding line. FIG. 6 is a plan view showing a background pixel acquisition region and a background pixel search region together with a partially enlarged view, and FIG. 7 is a plan view showing a setting method of the background pixel acquisition region and the background pixel search region. FIG. 8 is a plan view showing the estimated background area and object area, and FIG. 9 is a plan view showing each area classified in clustering. FIG. 10 is a plan view showing area division in clustering, and FIG. 11 is a plan view showing object areas to be extracted. In the following, the object extraction method according to the present embodiment will be described together with the operation of the object extraction apparatus described above.

  1 and 2, in the object extraction method according to the first embodiment, first, the image data input from the image input unit 110 or the image data stored in the image storage unit 120 is output to the control unit 130. The control unit 130 outputs the input image data to the image display unit 140 to display an image (step S1).

  When the image is displayed, the user operates the operation unit 150 to enclose the object area to be extracted with a surrounding line. When a surrounding line is input (step S2: YES), the initial area setting unit 210 sets the inside of the surrounding line as an initial area (step S3).

  In FIG. 3, for example, when a person object Oh is to be extracted, a surrounding line S that surrounds the person object Oh may be drawn as shown in the figure. The encircling line is preferably a closed curve that does not overlap the person object Oh. However, it does not have to be accurately drawn along the outline, and may be somewhat rough as shown in the figure, for example. An area including the person object Oh surrounded by a surrounding line is set as the initial area D.

  In FIG. 4, the effective range Ar is set by the effective range setting unit 220 in the displayed image. Here, for example, when trying to extract the star-shaped object Os in the figure, the object region cannot touch the outside of the object region because it is in contact with the edge of the screen. In such a case, the user has only to draw the surrounding line S within the range that can be drawn, and for example, the surrounding line S that does not touch the edge of the image as shown in the figure may be drawn.

  In FIG. 5, a surrounding line S drawn beyond the effective range Ar is corrected as a line toward the edge of the screen by the correction unit 230. At this time, as shown in the drawing, the line may be corrected as a line extending in the extending direction of the encircling line in the effective range Ar, or may be corrected as a perpendicular to the edge, for example. Thus, by setting the effective range Ar in advance, it is possible to suitably set the initial region D for the object region that is in contact with the edge of the image.

  Returning to FIG. 2, when the initial region is set, the background pixel acquisition unit 310 sets, for example, a range of several pixels from the surrounding line as the background pixel acquisition region. And the pixel information of the pixel in a background pixel acquisition area is acquired (step S4). The pixel information includes color tone, luminance, texture, and the like. For example, when obtaining the color tone as pixel information, an RGB histogram is created.

  Subsequently, the background pixel search unit 320 sets a background pixel search region centered on the background pixel acquisition region. Then, a pixel having pixel information similar to the acquired pixel information is searched from the pixels in the background pixel search region (step S5). Specifically, similar pixels are searched by comparing the RGB histogram described above with the RGB values of each pixel. For determining whether or not they are similar, for example, a preset threshold value or the like is used. Further, the threshold value may be determined based on the acquired pixel information.

  In FIG. 6, the background pixel acquisition area described above is set as a circular area Ci centered on one point on the encircling line S, for example, and the background pixel search area is set as an outer circle Co centered on the same point as Ci. Is done. Here, assuming that pixel information acquisition and pixel search are performed in the background pixel acquisition area Ci and the background pixel search area Co shown in the figure, pixels in the area shown in the enlarged view at the lower right of the figure are acquired. The pixel information has pixel information similar to the pixel information.

  As shown in FIG. 7, a plurality of background pixel acquisition areas Ci and background pixel search areas Co are set along the enclosing line S. The acquisition of pixel information and the search for similar pixels by the background pixel acquisition unit 310 and the background pixel search unit 320 are executed in each area shown in the figure. Therefore, a pixel similar to the background pixel is searched for around the object to be extracted. The background pixel acquisition area Ci and the background pixel search area Co are set at intervals such that adjacent background pixel search areas Co partially overlap as shown in the figure. Typically, the search accuracy is improved by setting many background pixel acquisition areas Ci and background pixel search areas Co. However, if the background pixel acquisition area Ci and the background pixel search area Co are set to be small, the search processing can be lightened.

  The size of the background pixel search area Co may be changed from the initially set size. For example, you may change according to the ratio of the pixel similar to a background pixel calculated | required as a result of a search. Specifically, when the ratio of pixels similar to the background pixel is large, there is a high possibility that the search range is not sufficiently covered up to the boundary of the object region, so the background pixel search region Co is increased. By increasing the background pixel search area Co, the background area can be estimated more appropriately. On the other hand, when the ratio of pixels similar to the background pixel is small, it is considered that there are many portions related to the object region, and therefore the background pixel search region Co is reduced. When the background pixel search area Co is made small, the search range is narrowed, so that the processing for searching becomes light. In this way, by changing the size of the background pixel search area Co, it becomes possible to more suitably search for pixels.

  Returning to FIG. 2 again, the background pixel estimation unit 300 estimates a pixel similar to the background pixel as a background region constituting the background of the object region (step S6). Then, the object area estimation unit 410 estimates an area excluding an area estimated as a background area from the initial area D (hereinafter referred to as “estimated background area” as appropriate) as an object area (step S7).

  In FIG. 8, for example, the image shown in FIG. 7 is classified into an estimated background area Be and an area Oe estimated as an object area (hereinafter referred to as “estimated object area” as appropriate) as a result of the search. The estimated object area includes, for example, an area where the background pixel search area Co has not reached (that is, a side peripheral portion or a neck peripheral portion of the person object Oh).

  Returning again to FIG. 2, in the object extraction method according to the present embodiment, clustering by color is performed here (step S8). Specifically, the clustering unit 420 first creates an RGB histogram from the pixel information in the estimated background region Be (hereinafter referred to as “background histogram” as appropriate). Subsequently, an RGB histogram is created from the pixel information in the estimated object region Oe (hereinafter referred to as “object histogram” as appropriate). Then, the RGB value of each pixel in the initial region D is examined and compared with the background histogram and the object histogram. More specifically, assuming that the corresponding bin value of the background histogram for each pixel is Nb, the corresponding bin value of the object histogram is No, and a predetermined threshold value Th, when Nb ≧ Th and No <Th, the background The pixels are classified as pixels (that is, pixels constituting the background area). When Nb <Th and No ≧ Th, the pixel is classified as an object pixel (that is, a pixel constituting the object region). Further, when Nb <Th and No <Th, and when Nb ≧ Th and No ≧ Th, the pixels are classified as rejects (that is, pixels that do not belong to either the background region or the object region). Thereby, each pixel in the initial region D is classified as one of the background pixel, the object pixel, and the reject pixel. The threshold value Th used above may be increased or decreased according to the number of pixels, for example, or may be set separately for Nb and No. The clustering method described above is merely an example, and other clustering methods that can obtain the same effect may be used.

  In FIG. 9, for example, as shown in FIG. 8, when the estimated background region Be and the estimated object region Oe are estimated, the side peripheral portion and the neck peripheral portion of the person object Oh are classified as reject Re by the clustering described above. The Pixels classified as reject Re are treated in the same manner as the background area and excluded from the object area. From this result, the object area determination unit 400 finally determines the background area Br and the object area Or (step S9).

  In FIG. 10, at the time of clustering, for example, the accuracy can be improved by dividing the initial region into a plurality of regions. Such an effect is particularly prominent when the object region and the background region include a plurality of colors. Moreover, although the example of the vertical direction division | segmentation is shown in the figure, the division | segmentation method is not limited, For example, you may use a horizontal direction division | segmentation, a cross division, etc.

  In FIG. 11, the determined object area Or may be output as the object area Or itself (that is, image data of only the person object Oh) as shown in the figure, or conversely, the object area Or is cut out from the image. It may be output as image data. Such data is output in various forms such as storage by the image storage unit 120, display on the image display unit 140, or distribution to the Internet.

  As described above, according to the object extraction device and the object extraction method according to the first embodiment, it is possible to realize the extraction of the object area with higher accuracy while reducing the burden on the user who designates the object area from the image.

<Second embodiment>
Next, an object extraction apparatus and method according to the second embodiment will be described with reference to FIGS. The second embodiment is different from the first embodiment described above in that it targets moving images, and the other configurations are generally the same. For this reason, in the second embodiment, a specific configuration and processing when extracting an object from a moving image will be described in detail, and the description overlapping with the other first embodiment will be omitted as appropriate.

  First, the configuration of the object extraction device according to the second embodiment will be described with reference to FIG. FIG. 12 is a block diagram showing the configuration of the object extracting apparatus according to the second embodiment. In FIG. 12, the same reference numerals are assigned to the same components as those in the first embodiment shown in FIG.

  In FIG. 12, the object extracting apparatus according to the second embodiment is configured to include an automatic initial area setting unit 500 in addition to the components of the object extracting apparatus according to the first embodiment shown in FIG. .

  The automatic initial area setting unit 500 is an example of the “automatic initial area setting unit” of the present invention, and enlarges the object area Or determined by the object area determination unit 400 and outputs it as initial area data. That is, the object area determination unit 400 in the second embodiment can output the extraction result to the automatic initial area setting unit 500.

  Next, an object extraction method according to the second embodiment will be described with reference to FIG. 2 used above and FIGS. 13 to 15 in addition to FIG. FIG. 13 is a flowchart showing the flow of a series of processes of the object extraction method according to the second embodiment. FIG. 14 is a process diagram illustrating the initial region setting method in successive frames step by step, and FIG. 15 is a process diagram illustrating the process diagram shown in FIG. 14 together with each region in detail. In the following, the object extraction method according to the present embodiment will be described together with the operation of the object extraction apparatus described above.

  12 and 2, in the object extraction method according to the second embodiment, first, the processing from step S1 to step S9 is executed as in the first embodiment described above, and the object region Or in one image is determined. The The one image is an image displayed as a still image on the screen when the moving image is paused, for example. That is, when the user specifies an object area to be extracted, the user draws a surrounding line S after temporarily pausing the moving image.

  In FIG. 13, when the object area Or in one image is determined, a parameter k that defines the number of frames is set to k = 2 (step S11). That is, if one image in which the object area Or is determined is the first frame, the target image moves to the second frame displayed next to the first frame.

  Subsequently, the automatic initial area setting unit 500 enlarges the object area Or of the (k−1) th frame (that is, the first frame here) (step S12). The enlargement here is performed so that the object area Or in the second frame is included in the object area of the enlarged first frame. The enlargement ratio here is typically stored in advance in a memory or the like in the automatic initial area setting unit 500 as a predetermined value. However, it may be changed as appropriate according to the moving speed and direction of the object. Specifically, if the movement of the object is fast, the enlargement ratio may be increased, or the enlargement ratio may be increased only in the direction in which the object moves.

  The automatic initial area setting unit 500 sets the enlarged object area of the (k−1) th frame as the initial area D of the kth frame (step S13). That is, an area in which the object area Or of the first frame is enlarged is set as the initial area D of the second frame.

  After the initial area is set, the object area Or is determined in the same manner as in the first embodiment described above. That is, similarly to the processing shown in steps S4 to S9 in FIG. 2, the processing from step S14 to step S19 is executed for the kth frame. Therefore, for the second frame, the object region Or is extracted even if the user does not draw the surrounding line S as in the first frame.

  When the object area Or of the k-th frame is determined, it is determined whether or not the object area Or has been determined for all the frames (step S20). In addition, all the frames here may be all the frames which comprise a moving image, for example, may point out all the frames of the number set beforehand, or the number set by the user etc. . Here, if it is determined that the object region Or has been determined for all the frames (step S20: YES), the series of processing ends. That is, the object area Or in the moving image is extracted. On the other hand, when it is determined that the object area Or has not been determined for all frames (step S20: NO), the parameter k is set to k = k + 1 (step S20). That is, the parameter k is incremented, and the process moves to the next frame. Then, the above-described processing from step S12 to step S20 is repeatedly executed. As a result, the object region is also extracted for subsequent successive frames.

  In addition, although the case where the process is performed on the frame after the frame specified by the user has been described above, for example, by setting the process in step S21 to k = k−1, Similarly, the object area Or can be extracted. Hereinafter, the series of processes described above will be described in detail using specific examples.

  14 and 15, for example, when a ball-shaped object Ob targets a moving image that rolls down an inclined surface, the user pauses the moving image and surrounds the ball-shaped object Ob with a surrounding line S. Enclose. In the first frame, the inside of the encircling line S is set as the initial area D, and the object area Or is determined.

  Subsequently, the object area Or determined in the first frame is enlarged to Er1. The enlarged area Er1 is an initial area D in the subsequent second frame. Since the object area Or is enlarged, the enlarged area Er1 is an area including the ball-shaped object Ob even when the ball-shaped object Ob moves to the lower right in the image, as shown in the drawing. In other words, the object area Or may be enlarged so as to include the object in the next frame.

  In the second frame, processing is performed on the inside of the enlarged area Er1, and the object area Or is determined. Similarly for the subsequent frames, the enlarged regions Er2, Er3... And the object region Or determined in each frame are enlarged and set as the initial region D. As a result, the object area Or is also automatically determined in subsequent successive frames.

  As described above, according to the object extraction device and the object extraction method according to the second embodiment, even if the target is a moving image (that is, even when the object area moves), the object in the user The area designating operation may be substantially the same as that for a still image. That is, the object area can be extracted with higher accuracy while reducing the burden on the user who designates the object area.

  The above-described object extraction apparatus and object extraction method of the present invention can be used for, for example, software for editing an image or a moving image, or a device to which these are applied. More specifically, it can be used for image editing software for personal computers, DVD recorders, cameras, TV phones, and the like.

  The present invention is not limited to the above-described embodiments, and can be appropriately changed without departing from the gist or concept of the invention that can be read from the claims and the entire specification. An object extraction method is also included in the technical scope of the present invention.

It is a block diagram which shows the structure of the object extraction apparatus which concerns on 1st Example. It is a flowchart which shows the flow of a series of processes of the object extraction method which concerns on 1st Example. It is a top view which shows the object and surrounding line which are drawn on an image. It is a top view which shows the effective range set in an image. It is a top view which shows correction | amendment of a surrounding line. It is a top view which shows a background pixel acquisition area | region and a background pixel search area | region with a partial enlarged view. It is a top view which shows the setting method of a background pixel acquisition area | region and a background pixel search area | region. It is a top view which shows the background area | region and object area | region estimated. It is a top view which shows each area | region classified in clustering. It is a top view which shows the area | region division in clustering. It is a top view which shows the object area | region extracted. It is a block diagram which shows the structure of the object extraction apparatus which concerns on 2nd Example. It is a flowchart which shows the flow of a series of processes of the object extraction method which concerns on 2nd Example. It is process drawing which shows the setting method of the initial region in a continuous frame later on. It is process drawing which shows the process drawing shown in FIG. 14 in detail with each area | region.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 110 ... Image input part, 120 ... Image storage part, 130 ... Control part, 140 ... Image display part, 150 ... Operation part, 210 ... Initial area setting part, 220 ... Effective range setting part, 230 ... Correction part, 300 ... Background Area estimation unit, 310 ... background pixel acquisition unit, 320 ... background pixel search unit, 400 ... object region determination unit, 410 ... object region estimation unit, 420 ... clustering unit, 500 ... automatic initial region setting unit

Claims (7)

Initial area setting means for setting an area inside the surrounding line as an initial area including the object area by enclosing the outside of the object area in the image with a surrounding line at a distance from the object area;
When the initial region is set, a pixel in a region that is separated from the surrounding line by a distance shorter than the distance is estimated as a background pixel that constitutes a background region that is a background of the object region, and in the initial region, A background area estimation means for estimating, as a background area, an area constituted by pixels whose pixel information is similar to that of the object area as compared to the background area;
An object extraction apparatus comprising: an object area determination unit that determines an area obtained by removing the estimated background area from the initial area as the object area. The background region estimation means includes
A background pixel acquisition unit that sets a background pixel acquisition region so as to at least partially overlap the surrounding line, and acquires pixel information of pixels in the background pixel acquisition region as pixel information of the background pixel;
A background pixel search area is set so as to at least partially surround the background pixel acquisition area, and pixels having pixel information similar to the acquired pixel information in the background pixel search area compared to the object area The object extraction device according to claim 1, further comprising background pixel search means for searching.   The background pixel search means changes the size of the background pixel search area based on a ratio of pixels having pixel information similar to the acquired pixel information in the pixels in the background pixel search area. The object extraction device according to claim 2, wherein The object area determination means includes
Object region estimation means for estimating an object region based on the estimated background region;
Clustering means for clustering whether each pixel in the initial region is the background region or the object region by comparing with pixel information in the estimated background region and the estimated object region ,
The object extraction apparatus according to any one of claims 1 to 3, wherein the clustering unit determines, as an object area, an area constituted by pixels that are determined to be the object area. Effective range setting means for setting an area having a smaller diameter than the image in the image as an effective range for validating the enclosing line;
The correction unit for correcting the surrounding line drawn beyond the effective range as a line drawn toward an edge of the image, further comprising: The object extraction device described.   In a moving image composed of a plurality of temporally continuous images, an object region determined in one image is enlarged, and the enlarged object region is initially set in another image displayed next to the one image. 6. The object extraction apparatus according to claim 1, further comprising automatic initial area setting means for setting as an area. An initial area setting step of setting an area inside the surrounding line as an initial area including the object area by enclosing the outside of the object area in the image with a surrounding line at a distance from the object area;
When the initial region is set, a pixel in a region that is separated from the surrounding line by a distance shorter than the distance is estimated as a background pixel that constitutes a background region that is a background of the object region, and in the initial region, A background region estimation step of estimating, as a background region, a region constituted by pixels whose pixel information is similar to that of the object region compared to the background pixel;
An object area determining step of determining an area obtained by removing the estimated background area from the initial area as the object area.

Images