JP2010113466A - Object tracking apparatus, object tracking program, and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object tracking apparatus, an object tracking program, and a method thereof, capable of reducing a process load relating to tracking of an object in a moving image. <P>SOLUTION: A predetermined frame in the moving image is selected as a first frame, then an outline of a predetermined object is detected from the first frame, then the detected outline is determined as the outline in the first frame. A frame arranged at the predetermined number after the first frame is selected as a second frame, then an outline which satisfy a predetermined condition within a predetermined distance from the determined outline relating to the frame arranged at the predetermined number before the second frame is detected as an outline candidate relating to the second frame, then difference between a gray level histogram within each outline of the outline candidate relating to the second frame and a gray level histogram in an outline of a frame arranged at the predetermined number before the second frame is calculated, then the outline candidate whose difference of the gray level histogram within the calculated outline is the smallest among outline candidates is determined as the outline of the second frame. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for tracking an object in a moving image.

2. Description of the Related Art Conventionally, a technique for detecting an object for each frame is known as a technique for tracking a certain object such as a human face in a moving image. Further, as a technique related to such a technique, a technique is known in which a moving image is sequentially captured in units of frames, a human face in each frame is detected, and tracking is performed between a plurality of different frames (for example, patents). Reference 1).
JP 2008-199549 A

  However, as described above, in tracking an object, detecting the object for each frame has a problem that the processing load is large.

  The present invention has been made to solve such problems, and an object of the present invention is to provide an object tracking device, an object tracking program, and a method capable of reducing the processing load required for tracking an object in a moving image. And

  An object tracking apparatus according to the present invention is an apparatus that tracks a predetermined object in a moving image composed of a plurality of frames arranged in time series, and is a first device that selects a predetermined frame in the moving image. A first determination that detects a contour of the predetermined object from a selection unit and a frame selected by the first selection unit, and determines the detected contour as a contour in the frame selected by the first selection unit. A second selection unit that selects a frame arranged after a predetermined number of frames selected by the first selection unit, and a predetermined number of frames before the frame selected by the second selection unit. A contour that satisfies a predetermined condition within a predetermined distance from the contour determined for the frames arranged in a frame is selected by the first detection unit and the second selection unit. A density value histogram in the contour of each of the contour candidates and a density value histogram in the contour of a frame arranged a predetermined number of frames before the frame selected by the second selection unit. A frame in which a first candidate that calculates a difference and a contour candidate that has the smallest difference in density value histogram in the contour calculated by the first calculator out of the contour candidates is selected by the second selector. A second determination unit that determines the outline of

  The object tracking program is a program for tracking a predetermined object in a moving image composed of a plurality of frames arranged in time series, and a first selection step of selecting a predetermined frame in the moving image And a first determination step for detecting a contour of the predetermined object from the frame selected in the first selection step, and confirming the detected contour as a contour in the frame selected in the first selection step; A second selection step of selecting a frame arranged after a predetermined number of frames selected by the first selection step, and an arrangement of the frame selected by the second selection step before the predetermined number of frames. A contour that satisfies a predetermined condition within a predetermined distance from the contour determined for the selected frame is defined as a contour candidate. A density value histogram in each contour of the contour candidate and a predetermined number of frames before the frame selected by the second selection step with respect to the frame selected by the first detection step to be detected and the second selection step. A first calculation step of calculating a difference between the density value histogram in the contour and the frame arranged in the contour, and among the contour candidates, the difference between the density value histograms in the contour calculated in the first calculation step is the largest. Causing the computer to execute a second determination step for determining a small contour candidate as the contour of the frame selected in the second selection step.

  A method for tracking a predetermined object in a moving image composed of a plurality of frames arranged in time series, the first selecting step for selecting a predetermined frame in the moving image, and the first selecting step A first determination step for detecting a contour of the predetermined object from the frame selected by the step, and confirming the detected contour as a contour in the frame selected by the first selection step; and the first selection step. A second selection step for selecting a frame arranged after a predetermined number of selected frames, and a determination for a frame arranged before the predetermined number of frames for the frame selected by the second selection step A first detection step of detecting a contour satisfying a predetermined condition within a predetermined distance from the contour as a contour candidate; For the frame selected by the two selection step, the density value histogram in each contour of the contour candidate and the frame within the contour of the frame arranged a predetermined number of times before the frame selected by the second selection step A first calculation step for calculating a difference from the density value histogram; and a second candidate for selecting a contour candidate having the smallest difference between the density value histograms in the contour calculated by the first calculation step among the contour candidates. And a second confirmation step for confirming the outline of the frame selected by (1).

  According to the present invention, it is possible to reduce the processing load required for tracking an object in a moving image.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. First, the hardware configuration of the target tracking device according to the present embodiment will be described. FIG. 1 is a diagram illustrating a hardware configuration of an object tracking device according to the present embodiment.

  The object tracking apparatus according to the present embodiment is realized by the personal computer 10 shown in FIG. The personal computer 10 includes at least a CPU 101 and a memory 102, and is connected to a storage device 103. Note that the storage device 103 stores a moving image that is a processing target in the present embodiment. Further, the personal computer 10 reads out the data from the storage device 103 and processes the moving image, and writes the processed data into the storage device 103. In addition, it is assumed that a moving image to be processed is composed of a plurality of frames arranged in time series.

  Next, the object tracking device according to the present embodiment will be described. FIG. 2 is a block diagram showing a configuration of the object tracking device according to the present embodiment.

  As illustrated in FIG. 2, the target tracking device 20 according to the present embodiment includes a selection unit 201 (first selection unit, second selection unit), a detection unit 202 (first determination unit), and a determination unit 203 (first Determination unit, second determination unit), density vector calculation unit 204 (first detection unit, first calculation unit), determination unit 205 (first detection unit, second detection unit), distance calculation unit 206 (second calculation unit) ). The selection unit 201 selects each frame of a moving image to be processed. The detection unit 202 detects and matches the contour of the tracking target in the moving image. In addition, the determination unit 203 determines or updates the contour vicinity region and the contour position based on the contour detected by the detection unit 202. Further, the density vector calculation unit 204 calculates the sum of absolute values of density gradients in the normal direction of each point on the contour in the contour vicinity region. The distance calculation unit 206 calculates distances between contours in different frames and distances between contours and contour candidates. Further, the determination unit 205 performs determination related to the contour based on the sum of absolute values of the density gradient calculated by the density vector calculation unit or the distance calculated by the distance calculation unit 206. In the present embodiment, the target tracking device 20 is intended for a specific person, and stores only a frame in which the target is included in the moving image. Each unit in the target tracking device 20 is substantially a function executed by the CPU 101 and the memory 102 of the personal computer 10.

  Next, the operation of the target tracking device according to this embodiment will be described. 3 to 5 are flowcharts showing the operation of the object tracking device according to the present embodiment. Moreover, FIGS. 6-9 is used as a figure for supplementing description of the operation | movement by this flowchart. FIG. 6 is a diagram showing an outline indicated by a curve. FIG. 7 is a diagram showing a normal vector of the contour. FIG. 8 is a diagram showing a movement vector. FIG. 9 is a diagram showing contour candidates that satisfy condition 2. In the flowcharts shown in FIGS. 3 to 5, it is assumed that an image of a specific person is designated in advance as a tracking target.

  As shown in FIG. 3, first, when a moving image to be processed is selected (S101), the selection unit 201 selects the first frame of the selected moving image (S102, first selection step), For the selected frame, the detection unit 202 detects the outline of the person (S103, first determination step). Note that this contour detection method may be any method. Further, the detection unit 202 determines whether or not a human contour is detected by this detection (S104).

  When the outline of the person is detected (S104, YES), the detection unit 202 performs a comparison between the person designated in advance and the detected person (S105), and whether or not the detected person is a person designated in advance. Is determined (S106). Note that this collation method may be any method similar to the contour detection method.

  When the detected person is a person designated in advance (S106, YES), the determination unit 203 determines the contour by setting a contour template as shown in FIG. 6 for the detected person. Then, the contour vicinity region of the contour is determined (S107, first determination step).

  Here, the outline template and the outline vicinity area will be described. First, it is assumed that the contour template is based on a contour whose features are extracted from images of a plurality of people by, for example, the Adaboost learning method. Further, it may be based on average face, neck and shoulder edge portions based on images of different people. The contour template is approximated by a curve based on vector data based on the contour extracted by the Adaboost learning method or the average face, neck and shoulder edge portions. Further, in the present embodiment, as the contour template, a template indicating the contours of the face, neck, and shoulder is used, but for example, a template indicating the contours of the trunk and legs can also be used. By increasing the number of contours to be used, it is possible to realize more accurate tracking processing. Further, for example, the follow-up process can be executed for a frame in which the face and neck are outside the frame and only the body and legs are in the frame. Further, as shown in FIG. 7, the contour vicinity region is a region within a predetermined distance centered on the curve of the contour template.

  Next, the selection unit 201 selects the next frame (S108, second selection step), and the density vector calculation unit 204 shows the selected frame as shown in FIG. Then, the sum of absolute values of the density gradient in the normal direction of each point of the contour template is calculated (S109, first detection step). Specifically, the density vector calculation unit 204 shifts the contour in the x direction and the y direction shown in FIG. Calculate the sum of the absolute values of the concentration gradient in the normal direction. In this processing, the density vector calculation unit 204 calculates in advance, as an array, the x and y components of the tangent direction of each contact point of the contour template curve and the x and y components of the unit vector in the normal direction to the tangent line. These x and y components are used for calculating the absolute value of the concentration gradient. Thus, by using the x and y components based on the curve shown by the vector data for the calculation of the absolute value of the concentration gradient, the accuracy does not depend only on the resolution of the frame. That is, a more accurate value can be obtained as compared with the case where the contour indicated by the raster data is used. In this process, the position shift, enlargement, and reduction with respect to the contour may be performed for each part (face, neck, shoulder) constituting the contour or for each curve. Further, each part constituting the contour template may be rotated in addition to shifting the position, enlarging or reducing the position. In addition, the aspect ratio is not fixed when the outline template is enlarged or reduced.

  Next, as illustrated in FIG. 4, the determination unit 205 causes the density gradient sum to be larger than the threshold A in the contour that satisfies the condition 1, that is, the contour that has been shifted and expanded in step S109. It is determined whether or not there is a contour (S110). In the present embodiment, the threshold A is a value that is ½ of the total density gradient of the contour in the previous frame.

  When there is a contour that satisfies the condition 1 (S110, YES), the determination unit 205 sets the contour that satisfies the condition 1 as a candidate for the contour (S111, first detection step). That is, it is determined whether there is a frame two frames before (S112). Here, the previous frame refers to the previous frame of the selected frame in a series of frames continuing from the collated frame to the frame processed in S112 (selected frame). Therefore, if no person is detected or verified in the predetermined frame of the selected moving image, the frame before the predetermined frame is not suitable as the previous frame in the subsequent processing. Further, in the present embodiment, the determination as to the presence or absence of the previous frame is determined based on whether or not the contour is determined for a frame other than the frame that has been detected and verified.

  Next, when there is a previous frame (S112, YES), the distance calculation unit 206 calculates a difference between the movement vector between each contour candidate and the previous contour, and the movement vector between the previous and previous contours, Normalization is performed (S113). Next, the determination unit 205 determines that a contour candidate satisfying the condition 2 in the contour candidates, that is, a ratio (vector ratio) between a normalized motion vector difference and a motion vector between the previous and previous contours is a predetermined value. It is determined whether there is a contour candidate that is equal to or less than the threshold value B (S114). In the present embodiment, when the selected frame is a frame that is four frames after the collated frame, the threshold B is the contour candidate that is finally determined as the contour in the processing of the previous frame of the selected frame. The value is not more than twice the vector ratio to.

  Here, step S113 will be described with reference to FIGS. As shown in FIG. 8, when there are a contour candidate A, a contour candidate B, and a contour candidate C as contour candidates satisfying the condition 1, as shown in FIG. 9, the center of gravity of the contour or the contour candidate is used as a reference for movement. The difference between the motion vector between these contour candidates and the contour of the previous frame and the motion vector between the contour of the previous frame and the contour of the previous frame is calculated for each contour candidate. In FIG. 9, contour candidates other than the contour candidate C having the largest movement vector difference are shown as contour candidates that satisfy the condition 2.

  When there is a contour candidate that satisfies the condition 2 (S114, YES), only the contour candidate that satisfies the condition 2 is set as a contour candidate (S115), and the density vector calculation unit 204 sets the contour with the previous frame for each of the contour candidates. The difference between the histograms of the density values is calculated (S116, first calculation step). The difference between the histograms of density values is the sum of the absolute values of the differences for each density in the contours as contour candidates in the selected frame and the contours of the previous frame. It is calculated by dividing by the sum of. Next, the determination unit 202 determines whether there is a contour candidate satisfying the condition 2, that is, a contour candidate whose density value histogram difference is smaller than the threshold value C (S117). It should be noted that a predetermined value is used in advance as the threshold C for the frame next to the collated frame. For a frame subsequent to the next frame after the collated frame, the threshold C is a value twice as large as the difference in the density histogram calculated for the contour determined for the previous frame. To do.

  When there is a contour candidate that satisfies condition 3 (S117, YES), the determination unit 203 selects a contour candidate that has the smallest difference in density value histogram from among contour candidates that satisfy condition 3 (S118, second determination step). Then, the selected contour candidate is confirmed as the contour in the selected frame (S119, second determination step), and based on this contour, the contour vicinity region is updated (S120), and the frame in which the person who has designated the selected frame is shown (S121), and the selection unit 201 selects the next frame (S108, second selection step).

  In step S117, when there is no contour candidate satisfying the condition 3 (S117, NO), the selection unit 201 selects the next frame (S122, first selection step).

  In step S114, if there is no contour candidate that satisfies condition 2 (S114, NO), the selection unit 201 selects the next frame (S122, first selection step).

  In step S112, when there is no previous frame (S112, NO), the density vector calculation unit 204 calculates a histogram difference of density values in the contour from the previous frame for each contour candidate (S116). ).

  In step S110, when there is no contour candidate that satisfies condition 1 (S110, NO), the selection unit 201 selects the next frame (S122, first selection step).

  In step S106, when the detected person is not a person designated in advance (S106, NO), the selection unit 201 selects the next frame (S122, first selection step).

  If no human contour is detected in step S104 (S104, NO), the selection unit 201 selects the next frame (S122, first selection step).

  As described above, the target tracking device 20 according to the present embodiment first detects the contour of a predetermined frame in a moving image. Further, the object tracking device 20 has the smallest difference in the density histogram in the contour with respect to the previous frame among the contour candidates that satisfy the predetermined condition in the vicinity region of the detected contour in the next frame. This contour is determined as the contour. Thus, the object tracking device 20 does not need to detect the contour for each frame by determining the contour in the selected frame based on the contour determined in the previous frame. Further, according to such processing, in determining the contour of the selected frame, by limiting the processing to a region near the contour of the previous frame, it is possible to reduce the processing load required for tracking the target. .

  In addition, the target tracking device 20 can further narrow down the contour candidates of this frame according to different conditions (condition 2) for frames subsequent to the frame in which the contour of the tracking target is detected. According to such processing, it is possible to further improve the accuracy of the contour to be determined.

  In the present embodiment, the tracking target is a person, but the tracking target may be any target as long as the contour can be detected. Further, in the present embodiment, processing for moving images is executed for each frame, but processing may be performed every several frames.

  Moreover, the object tracking program according to the present invention can be provided by providing a program that causes a computer to execute the operations shown in the flowcharts and steps described in the above embodiments. This program can be recorded on a computer-readable medium, for example, a portable storage medium such as a CD-ROM, a flexible disk, a DVD disk, a magneto-optical disk, an IC card, a database, and the like, and can be executed by the computer.

It is a figure which shows the hardware constitutions of the object tracking apparatus which concerns on this Embodiment. It is a block diagram which shows the structure of the object tracking apparatus which concerns on this Embodiment. It is a flowchart which shows operation | movement of the object tracking apparatus which concerns on this Embodiment. It is a flowchart which shows operation | movement of the object tracking apparatus which concerns on this Embodiment. It is a flowchart which shows operation | movement of the object tracking apparatus which concerns on this Embodiment. It is a figure which shows an outline. It is a figure which shows the normal vector of an outline. It is a figure which shows a movement vector. It is a figure which shows the outline candidate which satisfy | fills the conditions 2. FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Personal computer, 20 Object tracking apparatus, 101 CPU, 102 Memory, 103 Storage device, 201 Selection part, 202 Detection part, 203 Determination part, 204 Density vector calculation part, 205 Determination part, 206 Distance calculation part

Claims (5)

In a moving image composed of a plurality of frames arranged in time series, a target tracking device that tracks a predetermined target,
A first selection unit for selecting a predetermined frame in the moving image;
A first determination unit for detecting a contour of the predetermined object from a frame selected by the first selection unit, and determining the detected contour as a contour in the frame selected by the first selection unit;
A second selection unit for selecting frames arranged after a predetermined number of frames selected by the first selection unit;
For a frame selected by the second selection unit, a contour satisfying a predetermined condition is detected as a contour candidate within a predetermined distance from a contour determined for a frame arranged a predetermined number of frames before the frame. A first detector that
Concentration between a density value histogram in each contour of the contour candidate and a frame arranged a predetermined number of frames before the frame selected by the second selection unit with respect to the frame selected by the second selection unit A first calculation unit for calculating a difference from the density value histogram in the
A second determination unit that determines a contour candidate having the smallest difference in density value histograms in the contour calculated by the first calculation unit as the contour of the frame selected by the second selection unit, out of the contour candidates; An object tracking device comprising: The object tracking device according to claim 1,
The second selection unit, when the contour is determined by the second determination unit, to select a frame after a predetermined number of frames with the determined contour as a selection frame,
When the contour candidate of the selected frame is detected by the first detection unit, the contour of the previous frame, which is a frame arranged a predetermined number of times before the selected frame, and the frame arranged a predetermined number of times before the previous frame A second calculation unit that calculates a difference between a movement vector with a contour and a movement vector between the contour of the previous frame and each of the contour candidates of the selected frame;
A second detection unit for detecting, as a contour candidate of the selected frame, a contour candidate whose difference in movement vectors calculated by the second calculation unit is smaller than a first threshold that is a predetermined threshold among the contour candidates; An object tracking device further provided. In the object tracking device according to claim 1 or 2,
The contour is constituted by a curve composed of a plurality of contact points,
The second detection unit has the predetermined condition that a total value, which is a sum of density gradients in a normal direction of tangents of the contact points of the contour, is larger than a second threshold value which is a threshold value different from the first threshold value. When the contour is moved within a predetermined distance from the contour determined for a frame arranged a predetermined number of frames before the frame selected by the second selection unit, the total value is less than the second threshold value. An object tracking device that detects a contour that becomes larger as a contour candidate. An object tracking program for tracking a predetermined object in a moving image composed of a plurality of frames arranged in time series,
A first selection step of selecting a predetermined frame in the moving image;
A first determination step of detecting a contour of the predetermined object from the frame selected in the first selection step, and confirming the detected contour as a contour in the frame selected in the first selection step;
A second selection step of selecting frames arranged after a predetermined number of frames selected in the first selection step;
For a frame selected in the second selection step, a contour that satisfies a predetermined condition within a predetermined distance from a contour determined for a frame arranged a predetermined number of times before the frame is detected as a contour candidate. First detecting step,
Concentration between a density value histogram in each contour of the contour candidate and a frame arranged a predetermined number of frames before the frame selected by the second selection step with respect to the frame selected by the second selection step A first calculation step of calculating a difference from the density value histogram in
A second confirmation step of confirming, as the contour of the frame selected in the second selection step, a contour candidate having the smallest difference between the density value histograms in the contour calculated in the first calculation step among the contour candidates; A target tracking program that causes a computer to execute. An object tracking method for tracking a predetermined object in a moving image composed of a plurality of frames arranged in time series,
A first selection step of selecting a predetermined frame in the moving image;
A first determination step of detecting a contour of the predetermined object from the frame selected in the first selection step, and confirming the detected contour as a contour in the frame selected in the first selection step;
A second selection step of selecting frames arranged after a predetermined number of frames selected in the first selection step;
For a frame selected in the second selection step, a contour that satisfies a predetermined condition within a predetermined distance from a contour determined for a frame arranged a predetermined number of times before the frame is detected as a contour candidate. First detecting step,
Concentration between a density value histogram in each contour of the contour candidate and a frame arranged a predetermined number of frames before the frame selected by the second selection step with respect to the frame selected by the second selection step A first calculation step of calculating a difference from the density value histogram in
A second confirmation step of confirming, as the contour of the frame selected in the second selection step, a contour candidate having the smallest difference between the density value histograms in the contour calculated in the first calculation step among the contour candidates; An object tracking method comprising:

Images