JP2001043381A - Moving object contour extracting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accelerate entire processing and to speedily and highly accurately extract a moving object area out of a moving image by detecting the position of a moving object from a differential image among timewise continued plural images and determining the initial contour of the moving object on the basis of that position. SOLUTION: The differential image is prepared from the difference of timewise continued two figure images and binarizing processing is performed. The pixel of pixel value = 255 is searched from the upper left position of the binary image and the place of discovery is defined as top. The pixels of pixel value = 255 are searched respectively inside from the left and right ends of the binary image and the lateral position of a figure is detected. In this case, a range from the position of the top to the lower end of the image is divided into three search ranges H1-H3, for example, and the points of pixel value = 255 located in the closest position from ends within these ranges are detected. With these positions as reference, an initial contour form for contour extraction is determined.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving object contour extracting method for extracting a moving object region such as a person or a vehicle from a moving image including a moving object such as a person or a vehicle.

[0002]

2. Description of the Related Art For example, as a method for extracting a person region from a moving image including a person, there is a method based on threshold processing of an edge image. This is a method of obtaining an edge image by applying a Sobel filter to an input image and extracting a contour of a person who is a target object from the result. Assuming that the area surrounded by the extracted contour line is a person area, the subsequent processing is performed.

As a method of dynamically extracting a contour, there is a technique of a dynamic contour model (Active Control Model) called Snakes. This method is a method in which a plurality of control points are arranged in an image in advance, and these control points are moved so as to converge on a human contour. The movement of the control point is basically based on the movement in the direction in which the density gradient of the image is large. At this time, the control point moves so as to converge to a nearby contour line.

Further, as a simple method, there is a chroma key method using a blue background (blue background) as a method for cutting out a person area.

[0005]

However, a method based on threshold processing of an edge image does not always provide a connected contour. In addition, extra lines other than the originally intended lines are often extracted, and processing for correcting these lines is indispensable. Furthermore, even if the correction processing is performed, the contour lines are not always connected.

In addition, the active contour extraction method (Snakes) has a problem that if the initial contour is not appropriate, it does not converge on a desired person area. There is also a problem that the amount of calculation is significantly increased.

Further, in the chroma key method using the blue background, it is necessary to separately prepare a blue system as a background, and there is a problem that the scale becomes large.

[0008] Among them, the active contour extraction method (Sn
akes) and the two methods of the chroma key method using the blue background are mainly processing for still images.
However, in recent years, a process of extracting a target object (particularly, a person region in this case) from a moving image as well as a still image has been required. It is not practical because it requires hardware and takes time to process.

Therefore, the present invention provides a moving object contour extraction method which can reduce the amount of extra calculation, speed up the entire processing, and can quickly and accurately extract a moving object region from a moving image. The purpose is to provide.

[0010]

A moving object contour extracting method according to the present invention is a method for extracting a moving object region from a moving image including a moving object, wherein the image including the moving object is continuously input; Generating a difference image between the plurality of temporally continuous images input, detecting a position of the moving object from the generated difference image, and determining an initial contour of the moving object based on the position And extracting a moving object region from the input image based on the determined initial contour.

In the moving object contour extracting method according to the present invention, a moving object region is extracted from a moving image including a moving object. Generating a difference image between a plurality of images that are continuous in time, detecting a position of the moving object from the generated difference image, and determining an initial contour of the moving object based on the position; Determining the initial contour at the current time based on the initial contour determined at an earlier time; and determining the input image based on the determined initial contour. And extracting a moving object region from.

In the moving object contour extracting method according to the present invention, a moving object image is extracted from a moving image including a moving object. Generating a difference image between a plurality of temporally continuous images, and binarizing the generated difference image to determine whether or not the moving object in the input image has motion; The determining step, and when the moving object has a large movement, the position of the moving object is detected from the generated difference image, and the initial contour of the moving object is determined based on the position. If there is no motion, determining an initial contour at the current time based on the initial contour determined at a time earlier than that, extracting a moving object area from the input image based on the determined initial contour To do Consisting of a flop.

In the moving object contour extracting method according to the present invention, in a method for extracting a person region from a moving image including a person, a step of continuously inputting an image including a person includes the steps of: Generating a difference image between a plurality of continuous images; detecting a position of a person from the generated difference image; and determining an initial contour of the person based on the position; Extracting a person region from the input image based on the contour.

According to the moving object contour extraction method of the present invention, in a method for extracting a human body region from a moving image including a person, a step of continuously inputting an image including a person is performed. Generating a difference image between a plurality of images that are continuous with each other, detecting a position of the person from the generated difference image, and determining an initial contour of the person based on the position; and When determining the initial contour, a step of determining an initial contour at the current time based on the initial contour determined at a time earlier than that, and a person region from the input image based on the determined initial contour. Extracting.

In the moving object contour extraction method according to the present invention, in a method for extracting a person region from a moving image including a person, a step of continuously inputting an image including a person includes the steps of: Generating a difference image between a plurality of continuous images, detecting the positions of the shoulders and the crown of the person from the generated difference image, and defining an initial contour outside the person region based on those positions. Setting and extracting a person region from the input image based on the set initial contour.

Furthermore, the moving object contour extracting method of the present invention further comprises:
In the step of extracting a person region from a moving image including a person, a step of continuously inputting an image including a person, and a step of generating a difference image between the input temporally continuous images. , And the generated difference image
Determining whether the person in the input image has a motion by converting the value into a value, and detecting the position of the person from the generated difference image when the person has a large motion by the determination. Determining an initial contour of the person based on the position, and, if the person does not move significantly, determining an initial contour at the current time based on the initial contour determined at an earlier time; and Extracting a person region from the input image based on the determined initial contour.

According to the present invention, an initial contour is determined by using a difference image between successive images, so that when only a moving object (for example, a person) is moving in the image, it may occur. Since the difference is considered to be due to the motion of the moving object, the entire process can be sped up by setting the initial contour to an appropriate shape and reducing the amount of extra calculation.

Also, by utilizing the feature of a moving image and determining the initial contour at the current time from the contour extracted in the previous processing, efficient initial contour arrangement can be performed. Thus, the entire processing can be speeded up by reducing the amount of extra calculation.

Therefore, the moving object region can be quickly and accurately extracted from the moving image.

[0020]

Embodiments of the present invention will be described below with reference to the drawings.

Here, a description will be given of a case where the outermost contour of a person region is extracted under the following constraint conditions.

(1) The subject image is a person (front) upper body image (2) The subject number is one person in the image (3) The background is a plain background (the color is not limited) FIG. 1 shows a moving object according to the present invention. 1 schematically shows a configuration of a moving object contour extraction apparatus to which a contour extraction method is applied.
The apparatus includes an imaging unit 11 such as a video camera that continuously inputs images including the upper body of a person (front), and an imaging unit 11.
A / D converter 12 for converting an analog image signal output from the A / D converter into digital image data, and image storage for temporarily storing digital image data (hereinafter sometimes simply referred to as an image) output from the A / D converter 12 Unit 13, a person region extraction unit 14 that performs a person region extraction process based on the image in the image storage unit 13 and the image output from the A / D conversion unit 12, and displays the extraction result of the person region extraction unit 14. It is composed of an extraction result display unit 15.

That is, the analog image signal input by the image pickup unit 11 is converted into digital image data by the A / D converter 12. The converted image data is
Since time-series processing is sometimes performed, the image data is temporarily stored in the image storage unit 13.

At the same time, the input image output from the A / D conversion unit 12 is sent to a person region extraction unit 14, where a person region is extracted from a target image. Using the extracted person region, the extraction result display unit 15 cuts out and displays only the person region, for example.

In recent years, it has become possible to directly perform image pickup using digital signals. In such a case,
The configuration is such that the imaging unit 11 and the A / D conversion unit 12 are integrated into one.

When a time-series image such as a moving image is input, a person region is extracted by the person region extraction unit 14 by temporarily using the image immediately before being stored in the image storage unit 13. .

The person area extraction unit 14 performs processing by the background subtraction method when the background is simple, and when it is desired to extract a person area from an ordinary natural image or the like, for example, the outside of the person area as described later is used. For example, a contour is extracted, and processing is performed on the inside.

The extraction result display section 15 can be roughly classified into two types of displays. One of them is to use only the extracted person area in subsequent processing. Examples are as follows.

(1) A person area is combined with another background.

(2) When it is not desired to show the background to the other party using a videophone or the like, only the person area is displayed.

(3) Used for an expression technique (chroma key) used in broadcasting and the like.

The other is a case where a person area is erased from an image. This may not be used very often, but it is a possible application.

Next, a processing method according to the first embodiment in the person area extraction unit 14 will be described. In the first embodiment, an initial contour (hereinafter, sometimes referred to as an initial contour shape) is set from a difference image, and a case is described in which a human contour is extracted and the inside is regarded as a human area. .

First, two temporally continuous images G
After creating a difference image from the differences of [t] (x, y) and G [t-1] (x, y), the gradation difference Gd (x, y) = G
[T] (x, y) -G [t-1] (x, y) is equal to or larger than a threshold, and the pixel value Gd (x, y) = 255. , Y) = 0, and a binarization process is performed.

Next, as shown in FIG. 2, the top and right and left positions of the person are detected from the difference image, and an initial contour shape is set based on the positions. That is, first, 2
After performing the binarization process, a pixel having a pixel value Gd (x, y) = 255 is searched from the upper left position of the binary image, and the found position is defined as the crown 16 (see FIG. 2). However, in the difference image, the vicinity of the center of the head may be interrupted. Therefore, in practice, a pixel search is performed by giving a margin to the value.

Next, a pixel having a pixel value Gd (x, y) = 255 is searched inside from the left and right ends of the binary image, and the left and right positions of the person are detected. It is better to search the left and right positions in a small range so that control points can be placed as close to the person as possible. However, since there are parts in the difference image that are interrupted by binarization processing, the Come out.

Therefore, the area from the detected position of the crown 16 to the lower end of the image is divided into, for example, three search ranges H1, H2, and H3 as shown in FIG.
2 and H3, a point having a pixel value Gd (x, y) = 255 closest to the end is detected. Based on these positions, an initial contour shape R1 for contour extraction is determined as shown in FIG.

When an initial contour is formed from the extracted point sequence, each point may be connected by a straight line, or a curve may be applied.

In order to search the left and right ends, the image is divided into three search ranges for the sake of convenience, but the number of divisions is not limited thereto.

Next, a processing method according to the second embodiment in the person area extraction unit 14 will be described. In the second embodiment, a case will be described in which a contour line of a previous frame is used as an initial contour, a person contour is extracted, and the inside is considered to be a person area.

In the case of a moving image instead of a still image, if the image capturing interval is short between temporally consecutive frames,
It is considered that the person area hardly moves largely. Therefore, the human outline extracted in the previous frame is used as the initial outline shape for extracting the outline at the current time.

In this case, the pixel value of the difference image is "255".
Since the number of pixels of is smaller than the predetermined threshold, it is considered that the outline of the person has not moved much with respect to the previous frame,
As shown in FIG. 5, the contour line R2 of the previous frame is used as the initial contour shape R3 of the active contour. Actually, the person may be moving a little, and if the shape of the outline in the previous frame is not changed, it may be arranged inside the person. Therefore, as shown in FIG. Is slightly extended outward. A normal vector in a direction opposite to the convergence direction of the control point is obtained, and a moving amount of the control point is obtained by multiplying this by a certain coefficient. By using this method, the active contour can be arranged at a position very close to the human contour, and the processing time can be greatly reduced.

Here, a method of extracting a person contour will be described. Since this person contour extraction method is a known technique, it will be briefly described.

As shown in FIG. 6, a plurality of control points are arranged on the initial contour. The final contour may be constituted by line segments connecting these control points, or a curve (for example, a Bezier curve) complementing these control points may be applied.

It is assumed that three kinds of forces are acting on the control point, and the amount of movement according to these forces determines the movement vector of the control point each time. The movement amount of the k-th control point is calculated based on the following equation.

V (k) = α · V1 (k) + β · V2 (k) + γ · V3 (k) V1: Vector toward the maximum edge strength point near control point k V2: By the tension received from both sides of control point k Vector V3 toward the balance position V3: Normal vector toward the convergence direction of the control point α, β, γ: Weighting coefficient The edge force coefficient α and the forcing force coefficient γ change depending on the edge strength near the control point. When the control point is near the true human contour, the average edge value in the vicinity is high, but in the background region, the average edge value in the vicinity is low even in the vicinity of local noise. Therefore, in the former case, the edge force works strongly, and in the latter case, the forcing force works strongly.
α and γ are set. Accordingly, even when the control point is in the background portion, the control point can be moved so as to quickly move toward the human outline. Note that a fixed value is used as the tension coefficient β.

In the flow of the contour search processing, the interpolation of the control point, the convergence determination, the calculation of the movement vector, and the movement of the control point are repeatedly performed until the convergence is determined.

Next, a processing method according to the third embodiment in the person area extraction unit 14 will be described. According to the third embodiment, as described above, a difference image is created, the number of pixels in which a gradation difference appears is counted, and when the number of pixels exceeds a certain threshold, that is, the person in the image is If the motion is large, an initial contour is set from the difference image.If the number of pixels is lower than the threshold, that is, if the motion of the person in the image is small, the contour of the previous frame is set as the initial contour. Is what you do.

The description of the setting of the respective initial contours in the third embodiment is the same as that described in the first and second embodiments described above. The overall processing will be briefly described with reference to processing examples using actual images shown in FIG.

First, in step S1, an image including the upper body of a person is captured, and in step S2, a difference image of the captured image is generated. Next, in step S3, the difference image is binarized to generate an edge image (binary image).

Next, in step S4, based on this edge image, it is checked whether or not a person in the input image is moving. If the person is moving (when the person's movement is large), the above-described first step is performed. An initial contour setting process similar to that of the first embodiment is performed. If the person is not moving (when the motion of the person is small), an initial contour setting process similar to that of the second embodiment is performed.

That is, in step S4, if the person is moving (when the person moves greatly), step S4 is executed.
Proceeding to step 5, a plurality of control points are arranged on the difference image generated in step S2. Next, a contour search is performed in step S6, and a labeling process is performed in step S7. Next, in step S8, an outline image is displayed, and the process ends.

On the other hand, if it is determined in step S4 that the person is not moving (the person's movement is small),
Proceeding to 9, it is checked whether or not it has been determined in the previous frame that the contour has penetrated in the direction of the person. As a result of this check, if it is determined that the contour has penetrated in the direction of the person in the previous frame, a plurality of control points are arranged with respect to the contour shape of the previous frame in step S10, and the process proceeds to step S11. Then, a contour search is performed, and in step S12,
Perform labeling processing.

Next, in step S13, a difference image between the person area label and the previous person area label is generated, and in step S14, it is checked whether or not the contour has entered the person area. If the result of this check is that the contour has not invaded the person area, a contour image is displayed in step S8, and the process ends. As a result of the above check, if the contour line has entered the person area,
The process ends.

On the other hand, if it is not determined in step S9 that the contour line has penetrated in the direction of the person in the previous frame, then in step S15 a plurality of control points are located at the control point positions where they were arranged from the previous difference image. The control points are arranged, a contour search is performed in step S16, a labeling process is performed in step S17, a contour image is displayed in step S8, and the process ends.

As described above, according to the above-described embodiment, by determining the initial contour using the difference image between successive images, when only a person is moving in the image, Since the generated difference is considered to be caused by the motion of the person, the entire process can be speeded up by setting the initial contour to an appropriate shape and reducing the amount of extra calculation.

Further, by utilizing the feature of a moving image and determining the initial contour at the current time from the contour extracted in the previous processing, efficient initial contour arrangement can be performed. Thus, the entire processing can be speeded up by reducing the amount of extra calculation.

Therefore, the extraction of a person region from a moving image can be performed quickly and with high accuracy.

Further, by extending the method of arranging the initial contour for extracting a person, the person contour extraction processing which has been performed on a still image can be extended to a moving image.

In the above embodiment, the case where the moving object is a person has been described. However, the present invention is not limited to this. For example, the moving object may be a vehicle such as an automobile or an animal other than a person. However, the same can be applied.

[0061]

As described above in detail, according to the present invention, the amount of extra calculation can be reduced, the entire processing can be speeded up, and the extraction of the moving object region from the moving image can be performed quickly and accurately. A method of extracting a contour of a moving object that can be performed can be provided.

[Brief description of the drawings]

FIG. 1 is a block diagram schematically showing a configuration of a moving object contour extracting apparatus to which a moving object contour extracting method according to the present invention is applied.

FIG. 2 is a diagram illustrating a process of detecting a top position of a person.

FIG. 3 is a view for explaining processing for detecting the left and right positions of a person.

FIG. 4 is a view for explaining determination of an initial contour shape for contour extraction.

FIG. 5 is a view for explaining determination of an initial contour in a next frame using a result of extracting a contour in a previous frame;

FIG. 6 is an explanatory diagram regarding movement of a control point.

FIG. 7 is a flowchart for explaining a person contour extraction process from an image.

FIG. 8 is a diagram for explaining a processing example using an actual image.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 ... Imaging part 12 ... A / D conversion part 13 ... Image storage part 14 ... Person area extraction part 15 ... Extraction result display part 16 ... Top part

Claims (7)

[Claims] 1. A method for extracting a moving object region from a moving image including a moving object, comprising: continuously inputting an image including a moving object; Generating a difference image, detecting a position of the moving object from the generated difference image, and determining an initial contour of the moving object based on the position, and inputting the input image based on the determined initial contour. Extracting a moving object region from the obtained image. 2. A method for extracting a moving object region from a moving image including a moving object, comprising: continuously inputting an image including a moving object; Generating a difference image of the moving object; detecting a position of the moving object from the generated difference image; determining an initial contour of the moving object based on the position; and determining an initial contour at the current time. Determining an initial contour at the current time based on the initial contour determined at an earlier time; and extracting a moving object region from the input image based on the determined initial contour. A method for extracting a contour of a moving object, comprising: 3. A method for extracting a moving object region from a moving image including a moving object, the method comprising: continuously inputting an image including a moving object; Generating a difference image of the following; and binarizing the generated difference image to determine whether the moving object in the input image has a motion. When there is a large movement, the position of the moving object is detected from the generated difference image, and the initial contour of the moving object is determined based on the position. Determining an initial contour at the current time based on the initial contour determined at the time, and extracting a moving object region from the input image based on the determined initial contour. Do Moving body contour extraction method. 4. A method for extracting a person region from a moving image including a person, wherein the image including the person is continuously input, and a difference image between a plurality of the temporally continuous input images is provided. Generating an image, detecting a position of a person from the generated difference image, and determining an initial contour of the person based on the position; and calculating a person from the input image based on the determined initial outline. Extracting a region; and a method for extracting a contour of a moving object, comprising: 5. A method for extracting a human body region from a moving image including a person, comprising the steps of: continuously inputting an image including a person; and calculating a difference between the input temporally continuous images. Generating an image; detecting a position of the person from the generated difference image; and determining an initial contour of the person based on the position; and determining an initial contour at the current time. Determining an initial contour at the current time based on the initial contour determined at a previous time; and extracting a person region from the input image based on the determined initial contour. The moving object contour extraction method 6. A method for extracting a person region from a moving image including a person, wherein a step of continuously inputting an image including a person is performed, and a difference image between the input temporally continuous images. And detecting the positions of the shoulders and the crown of the person from the generated difference image, and setting an initial contour outside the person region based on those positions. Extracting a person region from the input image based on an outline. 7. A method for extracting a person region from a moving image including a person, comprising the steps of: continuously inputting an image including a person; and a differential image between a plurality of the temporally continuous inputted images. Generating, and binarizing the generated difference image to determine whether or not the person in the input image has a motion. In the case, the position of the person is detected from the generated difference image, and the initial contour of the person is determined based on the position.If the person does not move significantly, the initial contour determined at a time earlier than that is determined. Determining an initial contour at the current time based on the current time, and extracting a human region from the input image based on the determined initial contour. .