JP2003256849A - Object extracting system and method and its program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object extracting system and method and its program capable of further improving the reliability of the extraction of an object. <P>SOLUTION: An object extracting device 101 configuring an object extracting system 100 divides an area by an area dividing part 111 by using colors, luminance or textures or the like as references, and calculates the moving vector of each area by a movement calculating part 112, and calculates the similarity of the movement of each area by a similarity calculating part 113. A similarity storing part 114 stores the results across a plurality of frames so that even when any temporarily similar movement is detected, an integral validity/invalidity judging part 115 can correctly judge the integral validity/ invalidity of the areas. Thus, it is possible to increase the reliability of the extraction of the object. The position of the area in the next frame can be predicted from the calculation result of the movement as the reference of the area division in the next frame. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object extraction system, an object extraction method and an object extraction program for extracting an object, and more particularly to an object extraction system, an object extraction method and an object extraction method for detecting an object by detecting a motion vector. It relates to an object extraction program.

[0002]

2. Description of the Related Art Information indicating how much each element in a video is moving in which direction is called a motion vector. In order to extract a specific object from the video, the screen is divided into multiple areas, such as block-type areas, and areas that have motion vectors with similar motion are integrated into a continuous area. The method of doing is often adopted.

Various object extraction systems using such a technique related to object extraction have been conventionally proposed. For example, JP-A-2-118884 and JP-A-3-246
Such a technique is disclosed in Japanese Patent Laid-Open No. 75-75, Japanese Patent Laid-Open No. 4-148283, or the like.

However, when an object is to be extracted based on an image actually taken by a television camera or the like, not only the density of the object but also the effect of the brightness appears in each of the divided areas of the screen. For example, when the object is rotating or the light source is moving, the motion vector of each divided area may not be correctly obtained. Therefore, if the regions are simply integrated by using the motion vector obtained for each region, the regions that should originally be integrated may not be integrated, or the regions that should not be integrated may be integrated. Incorrect integration occurs. This makes it impossible to correctly extract the object region.

Therefore, it has been proposed to consider not only the similarity of motion vectors but also the reliability of motion vectors in each region. This is disclosed in JP-A-7-129.
This method is disclosed in Japanese Patent Publication No. 777, etc., and the regions are relaxedly integrated from the regions having high motion vector reliability.

FIG. 9 shows an outline of the object extraction system disclosed in this publication. The object extraction system inputs an image 11 at a predetermined time and an image 12 at a unit time before the predetermined time into the first or second frame memories 13 and 14, respectively, and the correlation value detection device 15 calculates the correlation values of the both. calculate. Then, the result is input to the motion vector detecting device 16 and the reliability detecting device 17 to detect the motion vector and the reliability. These results are input to the high reliability motion vector determination device 18 and the low reliability motion vector determination device 19. High reliability motion vector determination device 18
Extracts a motion vector whose reliability is higher than a first threshold value set in advance based on the distribution of reliability as an initial motion vector. The low-reliability motion vector determination device 19 removes a motion vector whose reliability is lower than the second threshold value lower than the first threshold value to determine a motion vector whose reliability is equal to or higher than the second threshold value. Output. The area integration device 20 extracts a motion vector, which is the same as or similar to the initial motion vector, from the output motion vectors by searching the area around the position having the initial motion vector,
The area having the extracted motion vector is integrated to obtain the area.

[0007]

However, even with the method disclosed in Japanese Patent Laid-Open No. 7-129777, erroneous integration still occurs when extracting objects, and the reliability cannot be sufficiently improved.

FIG. 10 shows an example of an input image from which an object is to be extracted. Screen 3 as shown in this figure
It is assumed that a person 32 and a car 33 are photographed in 1.
Motion vector 3 where these person 32 and car 33 are similar
4 and 35, and their reliability is high. For example, if similar motion vectors were obtained before a predetermined unit time from the screen 31 shown in FIG. 10, the reliability of the motion vectors of the person 32 and the automobile 33 is both high.

FIG. 11 shows a state of object integration by a conventional method. Since the two motion vectors 34 and 35 in FIG. 10 are similar and have high reliability, the object extraction system determines that they are the same object. As a result, these are integrated and are extracted in the wrong form as the object 36.

Therefore, an object of the present invention is to provide an object extraction system, an object extraction method, and an object extraction program which can further improve the reliability of object extraction.

[0011]

According to a first aspect of the present invention, (a) an area dividing means for dividing an input image for each frame into a plurality of areas having uniform features,
(B) a motion vector calculating means for calculating a motion vector representing a change in the motion of each area divided by the area dividing means; and (c) a motion of each area calculated by the motion vector calculating means. By referring to the vector in the same frame, the similarity calculation means for calculating the similarity of movement between the areas divided by the area division means, and (d) the similarity calculated by the similarity calculation means at the time of input. Similarity degree accumulation means for accumulating over a plurality of different frames, and (e) integration possibility determination means for determining whether or not adjacent areas are integrated with reference to the similarity degree accumulated in the similarity degree accumulation means,
(H) The object extraction system is provided with an object region image generation unit that generates a region image for each object by applying the determination result of the integration possibility determination unit to the plurality of regions divided by the region division unit.

That is, according to the first aspect of the invention, the image for each frame input by the region dividing means is divided into a plurality of regions having regions having uniform features, and the motion vector is divided for each of the divided regions. Is calculated by the motion vector calculation means. Then, the similarity calculating means refers to the motion vectors of the respective areas calculated by the motion vector calculating means in the same frame, thereby calculating the similarity of the movements of the areas divided by the area dividing means, and calculating the similarity. The accumulating unit accumulates the similarity calculated by the similarity calculating unit over a plurality of frames at different input times. The integration possibility determination unit determines whether or not adjacent regions are integrated with reference to the similarity accumulated in the similarity accumulation unit, and the object region image generation unit uses the determination result to integrate the regions. By performing or not performing, it is possible to generate a region image for each object with high reliability. That is, if the objects are different, the same movement may occur at a certain point in time, but different movements may occur at other points in time. This is because the similarity accumulating unit accumulates the similarities over a plurality of frames, so that the integration availability determining unit can more accurately determine the integration availability.

According to a second aspect of the invention, in the object extraction system according to the first aspect, the region prediction for predicting the position of each region of the next frame using the motion vector of each region calculated by the motion vector calculating means. It is characterized by further comprising means.

That is, according to the second aspect of the invention, assuming that the motion vectors of the respective areas are often similar even in the adjacent frames, the next frame is calculated by using the motion vectors of the respective areas calculated by the motion vector calculating means. The position of each area is predicted to make the area division processing more reliable.

According to a third aspect of the present invention, in the object extraction system according to the first aspect, the part having a uniform feature serving as a reference for division by the area dividing means is one of the color, luminance, and texture of the input image. It is characterized by being part or whole.

That is, according to the third aspect of the present invention, whether or not the areas are the same area is determined with reference to a part or all of the color, brightness and texture of the input image. This is because the colors and textures are often similar to each other in the same portion, and the effects of illumination are the same when the same surface is formed. The features to be referred to here are examples, and the present invention is not limited to these.

According to the fourth aspect of the present invention, (a) an area dividing step of dividing the input image for each frame into a plurality of areas having uniform features, and (b) the area dividing step. A motion vector calculation step of calculating a motion vector representing the change of the motion for each divided area, and (c) a motion vector of each area calculated in this motion vector calculation step is referred to in the same frame. Accordingly, the similarity calculation step of calculating the similarity of the movements of the regions divided in the region division step, and (d) the similarity calculation of the similarity calculated in the similarity calculation step over a plurality of different frames at each input time point. Degree accumulation step,
(E) an integration availability determination step of determining whether or not adjacent areas are integrated with reference to the similarity stored in the similarity storage step, and (e) a determination result obtained by the integration availability determination step The object extracting method includes an object region image generating step of generating a region image for each object by applying the plurality of regions divided by the dividing step.

That is, in the fourth aspect of the present invention, the image for each frame input in the region dividing step is divided into a plurality of regions having regions having uniform features, and the motion vector is divided for each of the divided regions. Is calculated in the motion vector calculation step. And
In the similarity calculation step, the motion vector of each area calculated in the motion vector calculation step is referred to in the same frame to calculate the motion similarity between the areas divided in the area division step, and the similarity accumulation step. The similarity calculated in the similarity calculating step is accumulated over a plurality of frames at different input times. In the integration possibility determination step, it is determined whether or not adjacent areas are integrated with reference to the similarity accumulated in the similarity accumulation step. Therefore, in the object area image generation step, the area integration is performed using this determination result. By performing or not performing, it is possible to generate a region image for each object with high reliability. That is, if the objects are different, the same movement may occur at a certain point in time, but different movements may occur at other points in time. This is because the similarity is accumulated in a plurality of frames in the similarity accumulating step, so that the integration propriety can be determined more accurately in the integration propriety determining step.

According to a fifth aspect of the present invention, the object extraction method according to the fourth aspect uses the motion vector of each area calculated in the motion vector calculating step to predict the position of each area of the next frame. It is characterized by further comprising steps.

That is, according to the fifth aspect of the invention, assuming that the motion vectors of the respective regions are often similar even in adjacent frames, the next frame is calculated using the motion vectors of the respective regions calculated in the motion vector calculating step. The position of each area is predicted to make the area division processing more reliable.

According to the sixth aspect of the present invention, in the object extracting method according to the fourth aspect, a part having uniform features as a reference when dividing in the region dividing step is one of the colors, luminances and textures of the input image. It is characterized by being a part or all of.

That is, according to the sixth aspect of the invention, whether or not the areas are the same area is determined with reference to a part or all of the color, brightness, and texture of the input image. This is because the colors and textures are often similar to each other in the same portion, and the effects of illumination are the same when the same surface is formed. The features to be referred to here are examples, and the present invention is not limited to these.

In the object extraction program according to the seventh aspect of the present invention, (a) an area division process for dividing an input image for each frame into a plurality of areas having uniform features, and B) A motion vector calculation process for calculating a motion vector representing the change in the motion of each region divided by the region division process,
(C) A similarity calculation process of calculating the motion similarity between the regions divided by the region division process by referring to the motion vectors of the respective regions calculated by the motion vector calculation process in the same frame. D) Similarity accumulation processing for accumulating the similarity calculated by this similarity calculation processing over a plurality of frames at different input times,
(E) An integration feasibility determination process for determining whether or not to integrate adjacent regions by referring to the similarity accumulated in the similarity accumulation process, and (e) a determination result obtained by the integration feasibility determination process for the region. It is characterized in that the object area image generation processing for generating the area image for each object is executed by applying the plurality of areas divided by the division processing.

That is, according to the seventh aspect of the present invention, the image for each frame input by the region division processing which constitutes the object extraction program is divided into a plurality of regions having uniform features, and these regions are divided. A motion vector is calculated for each region by a motion vector calculation process. Then, by referring to the motion vector of each region calculated in the motion vector calculation process in the similarity calculation process in the same frame, the similarity of motion between the regions divided in the region division process is calculated, and the similarity accumulation is performed. By the processing, the similarity calculated in the similarity calculation processing is accumulated over a plurality of frames at different input times. In the integration possibility determination process, it is determined whether or not adjacent regions are integrated by referring to the similarity accumulated in the similarity accumulation process. Therefore, in the object region image generation process, the region integration is performed using this determination result. By performing or not performing, it is possible to generate a region image for each object with high reliability. That is, if the objects are different, the same movement may occur at a certain point in time, but different movements may occur at other points in time. Since the similarity accumulation process accumulates the similarity over multiple frames,
This is because the integration availability determination in the integration availability determination processing can be performed more accurately.

According to the object extraction program of the invention described in claim 8, the position of each area of the next frame is determined by using the motion vector of each area calculated by the motion vector calculation processing in the object extraction program of claim 7. The feature is that the region prediction process for prediction is further executed.

That is, according to the invention of claim 8, on the premise that the motion vectors of the respective areas are often similar even in adjacent frames, the next frame is calculated using the motion vectors of the respective areas calculated by the motion vector calculation processing. The position of each area is predicted to make the area division processing more reliable.

[0027]

DETAILED DESCRIPTION OF THE INVENTION

[0028]

EXAMPLES The present invention will be described in detail below with reference to examples.

FIG. 1 shows an outline of an object extraction system according to an embodiment of the present invention. The object extraction system 100 includes an object extraction device 101, an image input device 102 that inputs an image for each frame to the object extraction device 101, and an image output device 103 that outputs an image of the object extraction device 101. ing. Here, the image input device 102 converts the image data 104 into the object extraction device 10
The device for inputting 1 may generate an image by itself, or the image data 104 received or supplied from a communication network or the like (not shown) may be used as the object extracting device 10.
It may be sent to one. Image output device 103
May send out the extracted image data 105 of the object to the outside, or may itself display or record an image.

The object extracting device 101 receives the image data 104 for each frame and divides the image data 104 into regions.
11, a motion calculation unit 112 that calculates a motion vector for each of the divided regions, and a similarity calculation unit 113 that calculates the similarity of the motion of each region calculated by the motion calculation unit 112.
And a similarity accumulation unit 114 that accumulates the similarity calculated by the similarity calculation unit 113 over a plurality of frames,
An integration possibility determination unit 115 that determines whether or not adjacent regions are integrated by referring to the similarity accumulated in the similarity accumulation unit 114, and an image for each region divided by the region division unit 111 are stored. Using the area image storage unit 116 to store the information, the information about the integration availability of the areas determined by the integration availability determination unit 115, and the image divided into the areas stored in the area image storage unit 116, An object area image generation unit 117 is provided that generates an image of the object area by performing or not integrating. Also, the motion calculation unit 112
Based on the motion data 118 calculated for each area,
Area prediction unit 11 that predicts each area in the next frame
9 is also provided. The image data 105 output from the object area image generation unit 117 is supplied to the image output device 103. The area prediction data 121 output from the area prediction unit 119 is input to the area division unit 111 and becomes data for the next area division.

Now, from the image input device 102 to the object extraction device 101, image data 104 consisting of a color image in frame units taken by a video camera (not shown).
Is supplied. Area division unit 11 in the object extraction device 101
1 divides an image of one frame into a plurality of areas by including image parts having uniform features in the same area and distinguishing the other areas from other areas. Here, as an example of a feature that serves as a reference for area division, there is color, brightness, or texture. Here, the texture refers to a pattern attached to the surface of a figure or drawing for expressing a texture.

As a method of area division in the area division unit 111, for example, an area growth method can be used.
As disclosed in Japanese Unexamined Patent Publication No. 7-99660, the area growing method defines a pixel of interest and a pixel adjacent to the pixel of interest as one area when they have the same characteristics. This is a method of sequentially executing the processing of integration.
As a result, regions having the same characteristics grow little by little, and finally the entire image is divided into regions. For example, well-known concrete processing of the region growing method is performed in the following procedure.

First, the screen is scanned to find a pixel that has not yet been divided into regions and set it as a pixel of interest. A new area is generated starting from this pixel of interest. Next, among a plurality of pixels adjacent to the pixel of interest, the distance between pixel values with a pixel that has not been divided into regions is calculated, and if the difference is equal to or less than a threshold value, it is integrated into one region. The newly integrated pixel is processed as the target pixel. The above-mentioned operations and are repeated until the area cannot be expanded any more. When the region cannot be expanded, the process returns to search for a pixel which is a starting point of a new region. If all the pixels can be divided into any of the areas in this way, the process is terminated at that point.

However, using this method, if there is a portion where the distance between adjacent pixel values gradually changes,
There is a problem that even pixels having a large distance between pixel values that are not appropriate to be regarded as the same area are integrated into one area. Therefore, in order to prevent such over-integration of regions, in the calculation of the distance between pixel values described in,
You may make it compare the average of the value of the pixel in the already integrated area | region with the value of the adjacent pixel.

The area dividing unit 111 can also divide the area using a method other than the area growing method.
As such a method, there are known methods such as a division method, a division integration method, a method using clustering, a method using a histogram, and a method using edge information.

FIG. 2 shows a processing flow of the object extracting apparatus of this embodiment. First, as described above, the area dividing unit 111 divides the area (step S201). After this area division is performed, the area image is generated by giving the area number to which each pixel forming the screen belongs (step S202).
The area image storage unit 11 generates the area image for each area.
6 is stored (step S203).

At the same time, the motion calculator 112 calculates the motion of each area in the area image (step S204). As a motion calculation method, a template matching method is used in which a two-dimensional motion vector (u, v) is calculated by matching the next frame image with a color or density pattern inside the region as a template. Can be adopted.

The motion of each area in the area image can be calculated by other than the template matching method. For example, the motion vector (u, v) is calculated in advance for all the pixels in the frame, the average of the motion vectors among the pixels belonging to each area is calculated, and this is used as the motion vector of that area. It is also possible to use various methods. In addition, the movement of each region is calculated using an affine transformation model (a,
There is also a method described by b, c, d, e, f). This method is described in, for example, Japanese Patent Application Laid-Open No. 9-121356.

The motion of each area calculated by the motion calculation unit 112 is supplied to the similarity calculation unit 113. Similarity calculator 1
In step 13, the motion similarity Sij is calculated for all pairs i and j of mutually adjacent areas (step S20).
5). Here, the motion similarity Sij can be expressed by the following equation (1), for example. Here, the movement of each area is described by two-dimensional vectors (ui, vi) and (uj, vj).

[0040]   Sij = 1 / (| ui, vi | + | uj, vj |) (1)

Similarly, the movement of each area is calculated by using affine transformation models (ai, bi, ci, di, ei, fi), (aj,
bj, cj, dj, ej, fj),
The following equation (2) can be used.

[0042]   Sij = 1 / (| ai-aj | + | bi-bj | + | ci-cj | + | di- dj | + | ei-ej | + | fi-fj |) (2)

For a set of areas that are not adjacent to each other, a special value such as -1 is given to the motion similarity Sij to store the fact that they are not adjacent to each other.

The similarity degree accumulating section 114 includes a similarity degree calculating section 11
The similarity calculated in 3 is accumulated over a plurality of frames (step S206). The manner of accumulation in the similarity accumulation unit 114 will be described next.

FIG. 3 shows a frame of an input image at a certain time. In the frame 131, an upper region 132 and a lower region 133 of a person, a body region 134, an automobile body region 135, and a window glass region 136 are divided as constituent regions. By the area division in step S202, n (n is an integer of 2 or more) areas (five areas 132 to 136 in this example) are obtained.
Each arrow 139 shown in FIG.
36 speeds are shown. In this figure, the divided areas 132 to 136 are moving to the right in the figure at almost the same speed.

FIG. 4 shows how the degree of similarity is accumulated. First, as shown in this FIG.
An upper triangular matrix M of × N is prepared. Next, the upper triangular matrix M is updated based on the area division result in the current frame. Specifically, the areas 132 and 13 shown in FIG.
3, when a pair of regions i and j (here, i <j) are adjacent to each other like the regions 135 and 136 or the regions 134 and 135, Mij (pixel 141 in FIG. 4). Is added to the motion similarity Sij.

On the other hand, when the regions i and j are not adjacent to each other like the regions 133 and 135, the motion similarity S
ij is subtracted. Even if the regions i and j are not exactly adjacent to each other, they may be considered to be actually adjacent to each other. For example, in the case where the distance between the centers of gravity exists within a certain range, such as the regions 133 and 134,
The similarity may be added assuming that the regions i and j are adjacent to each other.

The above is the processing performed for each frame. When the processing for one frame is performed in this way, it is determined whether or not it is the final frame (step S207 in FIG. 2). If the processing of the last frame has not been performed (N), the processing of the next frame will be performed. At this time, the area prediction unit 119 illustrated in FIG. 1 predicts the position of each area 132 to 136 in the next frame from the motion in the previous frame (step S208). The region dividing unit 111 grows a region with the predicted position thus obtained as a nucleus, and generates a region image of the next frame (step S202). After this, the same processing as the previous frame described above is performed on the next frame. The same applies hereinafter.

On the other hand, if it is determined in step 207 that the current frame is the final frame (Y), the integration possibility determination unit 115 analyzes the similarity accumulated over a plurality of frames to determine the area to be integrated. Determine (Step S
209). For example, the similarity Mi, j accumulated for the areas i, j is divided by the total number of frames, and if the value is larger than a certain threshold value, the areas i, j are integrated. You can Other methods are of course possible.

It is considered that the regions which have a high accumulated similarity and are determined to be integrated belong to the same object. Therefore, in the object area image generation unit 117, the area image storage unit 11
Area integration processing is performed on the entire area image stored in 6 to generate an object area image, which is output to the image output device 103 (step S210).

FIGS. 5 to 7 are for explaining an operation example of the integration feasibility determining unit and the object area image generating unit described above. In the frame 131 shown in FIG. 3, it is assumed that the regions 132, 133, and 134 representing a person and the regions 135 and 136 representing an automobile perform temporarily similar movements. As a result, in the frame 151 shown in FIG. 5 at a later time, the regions 132, 133, 134 representing the person and the regions 135, 136 representing the car show different movements. ing. Therefore, the frame 131 shown in FIG. 3 and the frame 131 shown in FIG.
When compared with the frame 151 shown in FIG. 3, the similarity accumulated for each of the regions 132, 133, and 134 representing a person has a high value. Each area representing a car 13
The same applies to 5, 136. However,
For example, with respect to the regions 134 and 135, the accumulated similarity remains low.

Therefore, the integration possibility determination unit 115 determines that the areas 132, 133, and 134 representing the person should be integrated. Areas 135 and 13 representing automobiles
It is also determined that 6 should be integrated. On the other hand, in FIG. 3, the areas 134 and 135 that were adjacent to each other have a low degree of similarity.
It is determined that the group of 4 and the groups of the regions 135 and 136 should not be integrated.

FIG. 6 shows the generation result of the object area image by the object area image generation unit 117 for the frame 131 shown in FIG. 3 as a result of accumulating a plurality of frames in this way. The object area image 152 is shown in FIG.
The areas 132, 133, and 134 related to the person shown in FIG.
It is integrated as one area 153. Similarly, areas 135 and 136 related to automobiles are integrated as another area 154.

FIG. 7 similarly shows the frame 1 shown in FIG.
5 shows the generation result of the object region image by the object region image generation unit 117 for 51. Object area image 1
Reference numeral 56 denotes areas 132 and 13 related to the person shown in FIG.
3,134 are integrated as one area 153. Similarly, areas 135 and 136 related to automobiles are integrated as another area 154.

As described above, in this embodiment, the motion similarity is accumulated over a plurality of frames. Therefore, in the case where the regions i and j have similar movements constantly and are adjacent to each other, the similarity Mi, j is added. Therefore, in the case where there is a similar motion only temporarily or there is an adjacent motion, the overall similarity Mi,
j remains relatively low. Therefore, accurate area integration can be performed.

Modification

FIG. 8 shows an outline of an object extraction system in a modified example of the present invention. In FIG. 8, FIG.
The same parts as those in are denoted by the same reference numerals, and the description thereof will be appropriately omitted.

The object extraction system 100A of this modified example
Is an object extraction device 101A, an image input device 102, an image output device 103, and a storage medium 30 connected to the object extraction device 101A.
It is composed of 1. Here, the object extraction device 10
1A is composed of a CPU (central processing unit) 302, a RAM (random access memory) 303, an input / output interface circuit (not shown), and an input / output device such as a keyboard and a display (not shown) if necessary. ing. The storage medium 301 is a storage medium such as a magnetic disk, an optical disk, or a semiconductor memory.
The storage medium 301 stores a control program,
When the CPU 302 executes this, each unit of the object extracting apparatus 101 shown in FIG. 1 is functionally realized. The RAM 303 is used as a memory area for loading a control program stored in the storage medium 301 and as a memory area for temporarily storing work data.

Of course, the storage medium 301 is in the form of a ROM (Read Only Memory), and the object extracting apparatus 101A
May be stored inside the computer, or the necessary control program may be downloaded from a storage medium (not shown) such as the Internet via a communication device (not shown). Further, the object extracting apparatus 101A may be one that performs some or all of the processing on the web by using this communication apparatus.

Further, although the CPU 302 is arranged in this modified example, it goes without saying that its name is not particularly limited, such as a processor and a data processing device.

[0061]

As described above, according to the invention described in claim 1, even if the objects have different movements at a certain time point, different movements may occur at other time points. Paying attention to the points, the similarity accumulating unit accumulates the similarities over a plurality of frames, so that there is an advantage that the integration availability determination of the integration availability determination unit can be performed more accurately by a relatively simple process.

According to the invention described in claim 2, in the object extraction system according to claim 1, the position of each area of the next frame is determined by using the motion vector of each area calculated by the motion vector calculation means. Since we decided to further equip the area prediction means to predict, even if there is noise in the image in some frames or the image becomes temporarily unclear, it is possible to accurately divide the area, or even extract the object. You can do it well.

Further, according to the invention described in claim 4, attention is paid to the fact that even if different objects have similar movements at a certain time point, different movements may occur at other time points. Since the degree of similarity is accumulated in a plurality of frames in the degree accumulating step, there is an advantage that the possibility of integrating in the possibility of integrating can be more accurately determined by a relatively simple process.

According to the invention described in claim 5, in the object extraction system according to claim 4, the position of each area of the next frame is determined by using the motion vector of each area calculated in the motion vector calculating step. Since we decided to further include a region prediction step to predict, even if there is noise in the image in some frames or the image becomes temporarily unclear, the region division, or even the extraction of the object is accurate. You can do it well.

Further, according to the invention described in claim 7, attention is paid to the fact that, if different objects have different movements at a certain time point, different movements may occur at other time points. Since the degree of similarity is accumulated in a plurality of frames in the degree accumulating process, there is an advantage that it is possible to more accurately determine the integration possibility in the integration possibility determination process with a relatively simple process.

According to the invention described in claim 8, in the object extraction system according to claim 7, the position of each area of the next frame is determined by using the motion vector of each area calculated by the motion vector calculation processing. Since we decided to further execute the region prediction process to predict, even if there are noises in the image in some frames or the image becomes temporarily unclear, the region division, or even the extraction of the object is accurate. You can do it well.

[Brief description of drawings]

FIG. 1 is a block diagram showing an outline of an object extraction system according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a processing flow of the object extracting apparatus of this embodiment.

FIG. 3 is an explanatory diagram showing a frame of an input image at a certain time.

FIG. 4 is an explanatory diagram showing how the degree of similarity is accumulated in this embodiment.

FIG. 5 is an explanatory diagram showing a frame of an input image at a time different from that of FIG.

6 is a plan view showing a generation result of an object area image by an object area image generation unit for the frame shown in FIG.

7 is a plan view showing a generation result of an object area image by an object area image generation unit for the frame shown in FIG.

FIG. 8 is a block diagram showing an outline of an object extraction system in a modified example of the present invention.

FIG. 9 is a block diagram showing an outline of a conventionally proposed object extraction system.

FIG. 10 is an explanatory diagram showing an example of an input image from which an object is to be extracted.

FIG. 11 is an explanatory diagram showing a state of object integration by a conventional method.

[Explanation of symbols]

100, 100A object extraction system 101, 101A object extraction device 102 image input device 103 image output device 111 Area Division 112 Motion calculator 113 similarity calculator 114 similarity accumulation unit 115 Integration possibility determination unit 116 area image storage unit 119 Area prediction unit 132-136, 153, 154 areas

Claims (8)

[Claims] 1. An area dividing means for dividing an inputted image for each frame into a plurality of areas having parts having uniform features, and a change in movement of each area divided by the area dividing means. By referring to the motion vector calculating means for calculating the motion vector representing the motion vector and the motion vector of each area calculated by the motion vector calculating means in the same frame. The similarity calculation means for calculating the similarity, the similarity accumulation means for accumulating the similarity calculated by the similarity calculation means over a plurality of frames at different input times, and the similarity accumulated in the similarity accumulation means An integration availability determination means for determining whether or not to integrate adjacent areas with each other, and a determination result by the integration availability determination means Object extraction system, characterized by comprising the object area image generating means for generating area image for each object by applying a plurality of regions divided by the region dividing means. 2. The object according to claim 1, further comprising area prediction means for predicting a position of each area of the next frame using the motion vector of each area calculated by the motion vector calculation means. Extraction system. 3. The part having a uniform feature serving as a reference for division by the area dividing means is a part or all of the color, brightness, and texture of the input image.
The described object extraction system. 4. An area dividing step of dividing an input image for each frame into a plurality of areas having uniform features, and a change in movement of each area divided by the area dividing step. By referring to the motion vector of each area calculated in this motion vector calculation step in the same frame, the motion vector calculation step of calculating the motion vector The similarity calculation step of calculating the similarity, the similarity accumulation step of accumulating the similarity calculated by the similarity calculation step over a plurality of frames at different input times, and the similarity accumulated in the similarity accumulation step Refer to the integration availability determination step to determine whether to integrate adjacent areas. And an object region image generating step of generating a region image for each object by applying the determination result of the integration feasibility determining step to the plurality of regions divided by the region dividing step. Extraction method. 5. The object according to claim 4, further comprising an area prediction step of predicting a position of each area of the next frame using the motion vector of each area calculated in the motion vector calculating step. Extraction method. 6. The part having a uniform feature as a reference when dividing in the region dividing step is a part or all of the color, brightness, and texture of the input image. The described object extraction method. 7. A computer is divided into a plurality of regions each having a uniform feature, and the image for each frame inputted is divided into a plurality of regions, and the divided regions are divided into regions. The motion vector calculation processing for calculating a motion vector representing a change in motion and the motion vectors of the respective areas calculated by this motion vector calculation processing are referred to in the same frame, so that the areas divided by the area division processing are Similarity calculation processing for calculating the similarity of motions of the human body, similarity accumulation processing for accumulating the similarity calculated by the similarity calculation processing for a plurality of frames at different input times, and the similarity accumulation processing for accumulation. An integration availability determination process for determining whether or not to integrate adjacent areas by referring to the similarity, and an integration availability determination process An object extraction program for executing an object region image generation process for generating a region image for each object by applying a determination result based on a theory to a plurality of regions divided by the region division process. 8. The area prediction process for predicting the position of each area of the next frame using the motion vector of each area calculated by the motion vector calculation processing is further executed. Object extraction program.