JP2007287006A - Moving object tracking device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect and track objects with the use of relevance to adjacent subareas. <P>SOLUTION: A moving object tracking device comprises an image generation means 1 for dividing input images into subareas of fixed shape, and a subarea relevance calculation means 7 for computing the relevance of any subarea to adjacent subareas. The subarea relevance calculation means 7 computes the relevance of any subarea to adjacent subareas on the preceding frame and the relevance of each subarea to adjacent subareas on the current frame. The subareas having the highest relevance between the preceding frame and the current frame are tracked as the same object. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving object tracking device and a program for individually detecting a plurality of moving objects by image processing from an image input from a camera or the like and automatically tracking the detected moving objects, and in particular, a moving object using a camera. The present invention relates to a moving object tracking device and a program for quantitatively expressing a relationship between a block of interest in a previous frame and a scanning target block of a current frame in tracking and determining an optimum moving destination in combination with similarity between blocks and a moving state. .

  Conventionally, various methods for tracking a moving object by image processing have been proposed. Typical methods include a background difference method, a template matching method, a local correlation method, and the like. A conventional example will be described below.

(1): Description of tracking a moving object by a combination of the background difference method and the template matching method This tracking of a moving object includes a difference binary image generation unit, a person dictionary, a tracking table, etc. The object is to accurately track a moving object, and a moving object region is extracted by a background subtraction method. In the case of a newly appearing object, the shading pattern is registered in the person dictionary. In the case of an existing object, template matching with a person dictionary is performed to determine whether the object is a correct moving object. And the moving object area | region of the present flame | frame is registered into a person dictionary (refer patent document 1).

(2): Explanation of tracking a moving object by the local correlation method This tracking of a moving object has an image input means, a moving image encoding / decoding means, a moving object detection means, an image composition means, etc. When detecting moving objects and detecting object colors by image processing, the frame acquired by the camera is divided into macroblocks composed of multiple pixels for the purpose of reducing the amount of calculation, calculation time, and cost. . Then, the difference absolute value sum is calculated between the previous frame and the current frame, and static motion determination is performed for each macroblock from the threshold comparison (see Patent Document 2).

(3): Explanation of tracking a moving object by a combination of local correlation method, band image correlation, and template matching method This tracking of the moving object is performed by image input means, optical flow estimation means, reliability evaluation means (moving object detection). ), Template management means, template matching means, object recognition means, etc., for the purpose of recognizing the intrusion of objects from the input video, the input image is divided into blocks (8 × 8 pixels), optical flow estimation A block having motion is detected by (local correlation method). Further, the band image obtained by the wavelet transform is divided into blocks, and a place with high correlation is found from the normalized correlation with the previous frame. A region where both match is regarded as a moving body, and template registration and matching are performed (see Patent Document 3).

(4): Description of tracking a moving object by a combination of a time series difference method and a local correlation method This tracking of a moving object is performed by an image input unit, a difference image creation unit, a change region extraction integration unit (an adjacent change region In order to identify monitoring targets in a short time and with high accuracy even in an environment with disturbance, (1) time t-1, t A difference image is generated by taking a difference between pixel values of the two frame images. (2) The difference image is binarized and a change area caused by the moving object is detected. The separated change areas that are close to each other are put together to set a circumscribed rectangle. (3) Grayscale template matching using a circumscribed rectangle as a template is performed between two frames, and a region with a low similarity is regarded as a moving body region and a large portion as a disturbance. The moving body is tracked by repeating these (1) to (3) (see Patent Document 4).

Among the above methods, the background subtraction method and the local correlation method use the difference in luminance value of the local region of interest,
-Background extraction and tracking of moving objects are not possible for areas with low contrast.

  When multiple moving objects overlap, the correlation of the block of interest deteriorates for both moving objects in the vicinity of these boundaries, making tracking impossible.

  There were problems such as. Further, the template matching method has a problem that when objects overlap each other, only a part of the template can be matched, so that accuracy is deteriorated and a moving object cannot be detected.

  On the other hand, there are the following conventional methods (see Patent Documents 5 and 6) as a method of continuing the tracking process using a non-shielded observation area as a clue even when objects are overlapped and a shielded part is generated.

(5): Description of a method of continuing tracking processing using an unoccluded observation area as a clue even when objects are overlapped with each other, tracking this moving object divides the image into blocks composed of a plurality of pixels Processing, processing for assigning a moving body ID to a block and generating an object map, processing for estimating the movement of an object when the time changes from t1 to t2, based on the image at time t1, the object map, and the image at time t2 If the moving object does not overlap even if the object overlaps, if the moving object does not overlap, it is tracked by the background difference method and the local correlation method, and each block that can be divided into a grid An object map in which a moving body ID is assigned to is created. When an object overlaps at time t2, it is determined using the next index which object the block near the boundary of the object belongs to.

  (1) The number of blocks having the same moving object ID around the movement source of the block of interest at time t1 (<t2).

  (2) Correlation value by block matching between the target block at time t2 and the movement source block at time t1.

(3) The correlation value between the movement vector of the block of interest and the movement vector of the adjacent block.
JP-A-6-231252 Japanese Patent No. 3262534 Japanese Patent Laid-Open No. 2004-13615 Japanese Patent No. 3423861 JP 2003-263626 A Japanese Patent Laid-Open No. 2004-207786

  The conventional device has the following problems.

  (1): The background subtraction method and local correlation method use the difference in luminance value of the local area of interest, so multiple areas of movement where background extraction and moving object tracking are not possible for areas with low shading When the bodies overlap, the correlation of the block of interest deteriorates with respect to either moving body in the vicinity of these boundaries, so that tracking cannot be performed.

  (2): The template matching method has a problem that when objects overlap each other, only a part of the template can be matched, so that the accuracy is deteriorated and a moving object cannot be detected.

  (3): Even when objects are overlapped with each other and a shielding part is generated, the method of continuing the tracking process using the observation area that is not shielded as a clue can track even when the moving object overlaps. However, since the relationship between the block of interest and the adjacent block is performed by matching the moving object ID, there is no verification means when an incorrect ID is given, and an ID is assigned such as a new object or a shielded part of an existing object. No evaluation means was provided when a new area appeared.

  An object of the present invention is to solve such a conventional problem and to provide an apparatus for detecting and automatically tracking a plurality of moving objects by image processing. In particular, the object is to detect and track an object with high accuracy even when the pixel values of a part or all of a moving object are uniform in density or even when objects overlap each other.

  FIG. 1 is an explanatory diagram of a moving object tracking device of the present invention. In FIG. 1, 1 is an image generation unit (image generation unit), 2 is a sub-partition attribute storage unit, 3 is a moving object attribute storage unit, 4 is a movement vector calculation unit (movement vector calculation unit), and 5 is a local correlation value calculation. Part (local correlation value calculation means), 6 is a sub-partition feature calculation part, 7 is an adjacent sub-partition relation degree calculation part (sub-partition relation degree calculation means), 8 is an adjacent sub-partition relation degree comparison part, and 9 is A movement probability calculation unit 10 is a moving object tracking unit.

  In order to solve the above-described conventional problems, the present invention is configured as follows.

  (1): an image generation unit 1 that divides an input image into sub-partitions having a fixed shape, and a sub-partition relation degree calculation unit 7 that obtains a relation degree between a certain sub-partition and an adjacent sub-partition, The inter-sub-partition relationship calculation means 7 obtains the degree of relation between the sub-partition of the previous frame and its adjacent sub-partition and the degree of relation between the sub-partition of the current frame and its adjacent sub-partition. The sub-partitions having the closest frame relation are tracked as the same. For this reason, it is possible to detect and track an object with high accuracy using the degree of relationship with adjacent sub-partitions.

  (2): In the moving object tracking device of (1), the movement vector calculation means 4 for obtaining the similarity between the movement vector of the sub-section of the previous frame and the movement vector of the sub-section of the current frame, and the sub-section of the previous frame And a local correlation value calculation means 5 for determining the similarity between the current frame and the sub-section of the current frame, the similarity index determined by the movement vector calculation means 4 and the similarity index determined by the local correlation value calculation means 5 The moving object is tracked by combining the index based on the degree of relation obtained by the degree-of-sub-partition relation degree calculating means 7. For this reason, an object can be detected and tracked more accurately using a plurality of indices.

  (3): In the moving object tracking device according to (1) or (2), the moving object tracking device includes sub-partition aggregate generation means for changing the size of the subpartition, and generates a sub-partition aggregate for the same subpartition with the same technology. The movement destination of the sub-partition aggregate is estimated and the moving object is tracked. For this reason, even if the pixel values of some or all of the moving object are uniform in density, the object can be detected and tracked with high accuracy.

  (4): In the moving object tracking device of (1) to (3), when a new sub-partition that is an area that does not exist in the previous frame appears, the degree of relationship between the new sub-partition and its adjacent sub-partition is obtained. New sub-partition determining means is provided, and the new sub-partition determining means determines whether the new sub-partition belongs to an existing object, a background region, or a new object according to the obtained degree of relationship, and tracks a moving object . For this reason, even when objects overlap each other, the objects can be detected and tracked with high accuracy.

  The present invention has the following effects.

  (1): The degree-of-sub-partition relationship calculation means obtains the degree of relation between the sub-partition of the previous frame and its adjacent sub-partition and the degree of relation between the sub-partition of the current frame and its adjacent sub-partition. Since the sub-partition having the closest relationship between the frame and the current frame is tracked as the same, the object can be detected and tracked with high accuracy using the degree of relation between the adjacent sub-partitions.

  (2): A moving object by combining an index based on similarity obtained by the movement vector calculating means, an index based on similarity obtained by the local correlation value calculating means, and an index based on the degree of relation obtained by the sub-partition relationship degree calculating means Therefore, it is possible to detect and track an object with higher accuracy using a plurality of indices.

  (3): In order to track a moving object by generating a sub-partition aggregate for a sub-partition where the texture is the same, and estimating the movement destination of the sub-partition aggregate, the pixel values of some or all of the moving objects Even if the shade is uniform, the object can be detected and tracked with high accuracy.

  (4): The new sub-partition determining means determines whether the new sub-partition belongs to an existing object, a background region, or a new object according to the obtained degree of relationship, and tracks the moving object. Even in this case, the object can be detected and tracked with high accuracy.

  The present invention quantitatively represents the relationship between the block of interest in the previous frame and the scanning target block of the current frame in tracking of a moving object using a camera or the like, and optimal movement in combination with the similarity and movement state between the blocks By determining the destination, the moving object is reliably tracked.

(1): Description of Moving Object Tracking Device FIG. 1 is an explanatory diagram of a moving object tracking device. In FIG. 1, the moving object tracking device includes an image generation unit 1, a sub-partition attribute storage unit 2, a moving object attribute storage unit 3, a movement vector calculation unit 4, a local correlation value calculation unit 5, a sub-part feature calculation unit 6, An adjacent sub-partition relationship calculation unit 7, an adjacent sub-partition relationship comparison unit 8, a movement probability calculation unit 9, and a moving object tracking unit 10 are provided.

(1) Description of Image Generating Unit 1 The image generating unit 1 includes an image input unit and a dividing unit that divides the input image into fixed shapes (sub-sections). In the image generation unit 1, from a camera, a video playback device (video tape on which video is recorded, a medium such as a CD, a DVD), a video file stored in a computer, or video data communicated via a network, Images are generated at regular sampling time intervals (eg, 33 ms).

  For example, the image is given by a rectangle of 720 pixels wide × 480 pixels vertically, and in the case of a color image, one pixel has gradation values of a red component (R), a green component (G), and a blue component (B). Given. In the case of a black-and-white image, it is given by the brightness gradation value (luminance). For example, the gradation values of the red component (R), green component (G), blue component (B), and luminance (I) of the pixel at the coordinates (i, j) indicated by the integers i and j are respectively digital values. R (i, j), G (i, j), B (i, j), I (i, j), etc. The pixel values used in the following calculation use luminance (monochrome gray value), but other R, G, B, RGB, or other color system (HSV color system) obtained by converting RGB values. System, YUV color system, etc.) may be used.

  Normally, the time series of images is processed in the order of oldest image to newest image. However, the time is renewed by playing back the video in reverse or by taking all the images into the image storage means once. You may perform the process with respect to an old image in order from an image.

(2) Description of sub-partition attribute storage unit 2 The sub-partition attribute storage unit 2 divides an image into sub-partitions of a fixed shape (for example, a grid of 8 × 8 pixels), and a moving object ID is assigned to each subpart. By giving them, the position of the detected moving object is represented. For example, when an image of horizontal 720 pixels × vertical 480 pixels is divided into 8 × 8 pixel sub-partitions, it becomes 90 horizontal parts × 60 vertical parts, giving a total of 5400 sub-partitions.

An area occupied by a moving object in an image is represented by a set of sub-partitions having the same moving object ID. The sub-partition is expressed by an array expression and stores various attribute values. For example, the k-th attribute value of the sub-part of the p-th row and the q-th column at time t is
^{Attrib_SubBlock (t) [p] [} q] [k]
0 ≦ p <SizeX / n, 0 ≦ q <SizeY / n, 0 ≦ k <kmax
Here, SizeX and SizeY are the horizontal and vertical sizes of the frame image, n is the number of pixels on one side of the sub-partition, and kmax is the number of attributes.
Given in. FIG. 2 is an explanatory diagram of sub-partition attributes. In FIG. 2, the attributes of the sub-partition include types such as a moving object ID, upper left pixel coordinates, a movement vector, a histogram, and a sub-partition similarity Stotal.

(3) Explanation of Moving Object Attribute Storage Unit 3 The area of moving objects in the image is represented by a set of sub-partitions that store the same moving object ID. FIG. 3 is an explanatory diagram of the relationship between sub-partitions and moving objects. In FIG. 3, it is an example which showed the relationship between a sub division and a moving object. When a new moving object is detected in the field of view of the camera, a new moving object ID = Z is allocated and stored in the sub-partition arrangement. The attributes of the moving object at time t are stored in the moving object attribute storage unit 3 in the following array expression.

Attrib_TrackBlock ^(t) [Z] [k]
k indicates the k-th attribute value. The attributes of the moving object include the position coordinates of the upper left end point and the lower right end point of the rectangle indicating the range of the circumscribed rectangle of the sub-partition, and a movement vector representing the speed and direction of movement from the previous sampling time to the current sampling time. FIG. 4 is an explanatory diagram of moving object attributes. The moving object attribute table of FIG. 4 includes the upper left corner coordinates of the circumscribed rectangle, the lower right corner coordinates of the circumscribed rectangle, the movement vector, and the number of sub-partitions included (9 in the example of FIG. 3).

  FIG. 5 is an explanatory diagram of examples of sub-partitions before and after movement. As shown in FIG. 5, when the previous frame image and the current frame image are given, it is estimated where the sub-partition to which the moving object ID of the previous frame has been moved moves in the current frame. The estimated movement destination is given at a position where the movement probability calculated based on the following index becomes the maximum. Each index is obtained by the calculation unit and comparison unit indicated in brackets [].

(Indicator 1) Probability based on the similarity (similarity) between the movement vector of the sub-partition of the previous frame and the movement vector of the sub-partition of the current frame. [Movement vector calculation unit 4]
(Indicator 2) Probability based on the similarity (similarity) between the movement-source sub-partition of the previous frame and the movement-destination sub-partion of the current frame. When the similarity is maximum, the probability is also maximum. [Local correlation value calculation unit 5]
(Indicator 3) Probability based on the degree of relationship between the sub-part of the previous frame and the adjacent part, and the degree of relation between the sub-part of the current frame and the adjacent part. [Sub-partition feature calculation unit 6, adjacent sub-partition relationship calculation unit 7, adjacent sub-partition relationship comparison unit 8]
Hereinafter, details of each calculation unit and comparison unit will be described.

(4) Description of Movement Vector Calculation Unit 4 In the movement vector calculation unit 4, when the sub-partition of the previous frame (time t-1) moves to the sub-partition where the current frame (time t) is located, the upper left end point of the sub-partition Is used to calculate the movement vector V.

V = (v _x , v _y ) = (i _LT −i _LT ′, j _LT −j _LT ′)
However, (i _LT ', j _LT ') is the coordinates of the upper left pixel of the sub-section of the previous frame,
(I _LT , j _LT ) are the coordinates of the upper left pixel of the sub-section of the current frame.

Assume that the sub-section of the current frame of interest (the upper left pixel coordinates (i _LT , j _LT )) is at the following position at the previous sampling time and the previous sampling time.

Sampling before: (i _LT ″, j _LT ″)
Pre-sampling: (i _LT ', j _LT ')
At this time, the use of calculation expression of the movement vector described above, the movement vector V of the previous frame _{'= (v x', v} y ') movement vector _{V = (v x, v y} ) of the current frame by the following equation .

V ′ = (v _x ′, v _y ′) = (i _LT ′ −i _LT ″, j _LT ′ −j _LT ″)
V = (v _x , v _y ) = (i _LT −i _LT ′, j _LT −j _LT ′)
The similarity between the vectors V ′ and V is evaluated by the following two values.

  The closer the difference value A (index A) is to 0 and the closer the inner product B (index B) is to 1, the higher the similarity of the movement vectors.

(5) Description of Local Correlation Value Calculation Unit 5 The local correlation value calculation unit 5 calculates the luminance value of the pixel at the same position included in the sub-partition between the previous frame (time t-1) and the current frame (time t). The difference is taken and the sum C of absolute values is calculated.

In the above equation, I _t−1 (i ′, j ′) is the luminance value of the pixel included in the sub-partition of the previous frame, I _t (i, j) is the luminance value of the pixel included in the sub-partition of the current frame, R _t-1 indicates that it is in the sub-partition area of the previous frame, and R _t indicates that it is in the sub-partition area of the current frame.

  The smaller the sum C, the greater the similarity between the sub-section of the previous frame and the current frame sub-section, and the smaller the sum C (index C), the smaller the similarity.

(6) Description of the sub-partition feature calculation unit 6 The sub-partition feature calculation unit 6 acquires the luminance value of the pixel included in the sub-part of interest, and uses the luminance value as a class, and the appearance of pixels included in the same class The histogram _Hsub obtained by integrating the number of times is used as the sub-partition feature. The histogram _Hsub is expressed as follows.

H _sub [I] = N _I
I is the luminance value, N _I are included in the focused sub-zone, the total number of pixels having luminance values I.

(7) Description of Adjacent Sub-partition Relation Degree Calculation Unit 7 Since the luminance value distribution of the local part of the moving object region generally changes gently, adjacent sub-partitions have similar characteristics. For example, when the torso portion of a person is divided into sub-sections, the luminance distribution of each sub-section is almost constant. On the other hand, when one of the adjacent sub-sections is included in the moving object and the other is included in the background area, the similarity is low. In order to quantitatively express the relationship between such adjacent sub-partitions, the degree of relation between adjacent sub-partitions is calculated by the degree-of-sub-partition relation calculation unit 7.

(Description of the first method)
The first method for obtaining the degree of relationship between adjacent sub-partitions is a method for calculating the overlap of histograms used for sub-partition features. In the first method, the degree of relationship REL between adjacent sub-partitions of the sub-partition sub1 and the sub-partition sub2 is obtained by the following equation.

    Here, min is a function for selecting the smaller value.

FIG. 6 is an explanatory diagram of an example of calculating a histogram. In FIG. 6, the image of the previous frame is shown in FIG. 6 (a), the histograms are shown in FIGS. 6 (b) to (d), the image of the current frame is shown in FIG. 6 (e), and the histograms are shown in (f) to (h). ing. Two squares superimposed on the image represent a sub-section (4 pixels × 4 pixels) and are located in the same portion of the person photographed in the previous and current frames. When the histogram of each sub-section is obtained, it becomes as shown in FIGS. 6 (b), (c), (f), and (g). FIG. 6 (d) (h) is a result of plotting min (H _sub1 [I], H _sub2 [I]) in the above equation. For example, in FIG. 6 (d), the smaller one is selected in the histograms of FIG. 6 (b) and FIG. 6 (c).

  The degree of relationship between adjacent sub-partitions corresponds to the area of FIGS. 6 (d) and 6 (h). Since (d) and (h) in FIG. 6 have substantially the same value, it can be seen that the degree of relationship between the sub-partitions assigned to the same part on the object is maintained even if the object moves. .

(Explanation of the second method)
The second method is a method using the distance between the peak values of the luminance value distribution in the sub-partition. In the second method, the relationship REL between adjacent sub-partitions of the sub-partition sub1 and the sub-partition sub2 is obtained by the following equation.

    Here, max is a function for selecting the maximum value.

(Explanation of the third method)
The third method is a method using a distance between luminance values corresponding to the center of gravity of the luminance value distribution in the sub-partition. In the third method, the degree of relationship REL between adjacent sub-partitions of the sub-partition sub1 and the sub-partition sub2 is obtained by the following equation.

    Here, N is the number of pixels in the sub-partition.

  That is, the difference in the center of gravity between adjacent sub-sections is used as the degree of relationship.

(Explanation of the fourth method)
The fourth method uses a distance between luminance values corresponding to the median value of the luminance value distribution in the sub-partition. In the fourth method, the degree of relationship REL between adjacent sub-partitions of the sub-partition sub1 and the sub-partition sub2 is obtained by the following equation.

M _sub is the median value of the luminance distribution, and is determined as follows. First, the luminance values I1 to IN in the sub-partition are arranged in ascending order.

I1≤I2≤ ... ≤IN N is the number of pixels in the sub-partition When the number of pixels N in the sub-partition is an even number, the median value _Msub is given by the following equation. When the number N of pixels in the sub-partition is an odd number, there is a median value, which is used.

(8) Description of the relationship comparison unit 8 between adjacent sub-partitions The sub-partition (p ′, q ′) of the previous frame (time t−1) is changed to the sub-partition (p, q) where the current frame (time t) is located. When moving, the degree of relationship with the sub-partition (p + a, q + b) (where a = -1, 0, 1, b = -1, 0, 1) adjacent to the target sub-partition is compared as follows: Calculate the output value. The combination of the values a and b indicating the adjacent sections is given as follows.

In the case of four adjacent sections: (a, b) = (0, −1), (−1, 0), (1, 0), (0, 1)
In the case of 8 adjacent sections: (a, b) = (0, −1), (−1, 0), (1, 0), (0, 1), (−1, −1), (−1, 1), (1, -1), (1,1)
Whether to use the four adjacent sections or the eight adjacent sections is determined in advance before executing the process.

  Hereinafter, the degree of relationship with the sub-section (a, b) = (0, −1) in contact with the upper side of the target sub-section will be described. Other adjacent sections can be described in the same procedure.

  First, the degree of relationship REL (sub (p ′, q ′), sub between adjacent sub-partitions between the sub-partition (p ′, q ′) of the previous frame and the sub-partition (p ′, q′−1) adjacent on the upper side. (p ', q'-1)) is calculated. Next, the sub-partition (p ′, q ′) of the previous frame and the adjacent sub-partition of the sub-partition (p, q) that is the destination candidate of the current frame on the upper side (p, q−1) The degree of interrelationship REL (sub (p ′, q ′), sub (p, q−1)) is calculated. Finally, the difference between the obtained degrees of relationship between two adjacent sub-partitions is calculated and set as an output value.

REL (sub (p ', q'), sub (p ', q'-1))-REL (sub (p', q '), sub (p, q-1))
When the degree of relationship between adjacent sub-partitions is obtained by the first method,
REL (sub (p ', q'), sub (p ', q'-1)) = 0
REL (sub (p ', q'), sub (p, q-1)) = 0
In this case, the output value is N (number of pixels in the sub-partition). The sub-partition (p ′, q ′) adjacent four sections are four sections in the upper and lower sides and the left and right sides. Therefore, an index D based on the degree of relationship between adjacent sub-partitions is expressed by the following equation.

    Here, R ′ represents a set of four adjacent sections of the sub-partition (p ′, q ′) of the previous frame, and R represents a set of four adjacent sections of the sub-partition (p, q) of the current frame.

  The closer the index D is to zero, the higher the similarity between the subframe of the previous frame and the current frame subsection, and the lower the value from zero, the lower the similarity.

  In the above description, the index D is described as one of the first to fourth methods. However, a plurality of indices such as D1, D2,... Are calculated by combining several calculation methods. Then, these indicators may be used in combination.

(9) Description of Movement Probability Calculation Unit 9 Using the index obtained by the above method, the similarity between the sub-partition (p ′, q ′) of the previous frame and the sub-partition (p, q) of the current frame is obtained. . The most similar sub-partition in the current frame is considered to have the maximum movement probability, and is the destination of sub-partition (p ′, q ′). The degree of similarity Stotal between sub-partitions is expressed by the following formula using indices A, B, C, and D.

Stotal (p ′, q ′, p, q) = − α | A | + β (B−1) −γC−δ | D |
α, β, γ, and δ are weight coefficients given in advance.

The destination (pdst, qdst) is
Stotal (p ', q', pdst, qdst) = max (Stotal (p ', q', p, q))
0 ≦ p <SizeX / n, 0 ≦ q <SizeY / n
Meet. When the sub-section (p, q) of the current frame is given, the processing time can be shortened by limiting the scanning range. For example, when scanning in the range of up, down, left, and right r sections centering on the sub section of the previous frame, the value of the sub section (p, q) is set to
p'-r≤p≤p '+ r, q'-r≤q≤q' + r
It is good to make it. The range of the r section becomes larger as the moving speed becomes faster (for example, with an enlarged zoom).

  When the moving objects overlap, the movement destinations of two different sub-partitions (p1 ′, q1 ′) and (p2 ′, q2 ′) of the previous frame are the same sub-partition (pdst) of the current frame near the boundary of the object. , Qdst). As described above, when there are a plurality of movement sources, the one having the higher similarity Stotal is selected.

Stotal (p1 ′, q1 ′, pdst, qdst) ≧ Stotal (p2 ′, q2 ′, pdst, qdst) → The source is the sub-partition (p 1 ′, q1 ′)
Stotal (p1 ', q1', pdst, qdst) <Stotal (p2 ', q2', pdst, qdst)-> move source is sub-partition (p2 ', q2')
Even when the number of movement sources of the previous frame becomes three or more, processing can be performed by sequentially comparing the movement source selected by the above equation and the next movement source.

(10) Description of Moving Object Tracking Unit 10 When the movement probability calculation unit 9 determines the movement source subsection of the subsection (p, q) of the current frame, the moving object tracking section 10 uses the movement source ID as the movement object ID. Inherit the ID value.

Attrib_SubBlock ^(t) [pdst] [qdst] [kID] = Attrib_SubBlock ^(t-1) [p '] [q'] [kID]
kID is the attribute number of the storage destination of the moving object ID.

  When the movement sources of all the sub-partitions in the current frame are determined and the moving object ID is set in the sub-partition attribute storage unit, the area of each moving object in the current frame can be determined. For example, the region of the object with the moving object ID = Z is represented by a collection of sub-partitions that satisfy Attrib_SubBlock [p] [q] [kID] = Z.

  The moving object can be tracked by repeating the above process every time the sampling time elapses and a new frame image is acquired.

(2): Description of Moving Object Tracking Process FIG. 7 is a moving object tracking process flow. Hereinafter, the moving object tracking process of FIG. 7 will be described according to the processes S1 to S23.

  S1: As an initial setting, various constants (image size SizeX, SizeY, number of pixels n on one side of sub-partition, etc.) are set in advance in processing units such as the image generation unit 1 and each calculation unit, and the process proceeds to processing S2.

  S2: The image generation unit 1 starts photographing with the camera, and proceeds to processing S3.

  S3: The image generation unit 1 acquires an image at each sampling time, and proceeds to processing S4.

  S4: The image generation unit 1 moves the data stored in the current frame image buffer to the previous frame image buffer, stores the image acquired in the processing S3 in the current frame image buffer, and proceeds to processing S5.

  S5: The movement vector calculation unit 4 extracts one sub-partition (p ′, q ′) of the previous frame, and proceeds to processing S6.

  S6: The movement vector calculation unit 4 extracts one sub-partition (p, q) of the current frame, and proceeds to process S7.

S7: The movement vector calculation unit 4 calculates the upper left pixel coordinates (i _LT ', j _LT ') of the sub-section (p ', q') of the previous frame and the upper left end of the sub-section (p, q) of the current frame. The movement vector V is calculated from the pixel coordinates (i _LT , j _LT ), and the process proceeds to processing S8.

  S8: The movement vector calculation unit 4 takes out the movement vector V ′ of the sub-partition (p ′, q ′) of the previous frame from the sub-partition attribute storage part, and obtains the movement vector V of the current frame obtained in the processing S7. The vector magnitude difference value A (index A) and the inner product B (index B) are calculated, and the process proceeds to step S9.

  S9: The local correlation value calculation unit 5 calculates the local correlation value from the pixel of the previous frame image buffer included in the sub-partition (p ′, q ′) and the pixel of the current frame image buffer included in the sub-partition (p, q). C (index C) is calculated, and the process proceeds to step S10.

  S10: The degree-of-sub-partition relationship calculation unit 7 receives the histogram Hp ′, q ′ [I] of the sub-partition (p ′, q ′) of the previous frame and the adjacent part (p ′) from the sub-partition attribute storage unit. + A, q ′ + b) (a, b = −1, 0, 1) histogram Hp ′ + a, q ′ + b [I] is extracted, and the degree of relationship REL (sub (p ′, q '), sub (p' + a, q '+ b)) are calculated, and the process proceeds to step S11.

  S11: The relationship calculation unit 7 between adjacent sub-partitions includes the current frame included in the sub-partition (p, q) and the adjacent section (p + a, q + b) (a, b = −1, 0, 1) of the current frame. Histograms Hp, q [I], Hp + a, q + b [I] are obtained from the pixels of the image buffer, and the degree of relationship REL (sub (p, q), sub (p + a, q + b) between the two partitions )) Is calculated, and the process proceeds to step S12.

  S12: The relationship comparison unit 8 between adjacent sub-partitions compares the relationship REL (sub (p ′, q ′), sub (p ′ + a, q ′ + b)) of the previous frame and the relationship REL ( An index D based on the degree of relationship between adjacent sub-partitions is calculated from sub (p, q), sub (p + a, q + b)), and the process proceeds to step S13.

  S13: The relationship comparison unit 8 between adjacent sub-partitions determines whether or not the degree of relation between adjacent sub-partitions for all the adjacent sections has been calculated. If it is determined that the degree of relationship between adjacent sub-partitions for all adjacent partitions is calculated, the process proceeds to step S14. If all the calculations are not completed, the process returns to step S10 (the degree of relationship between adjacent sub-partitions for all adjacent partitions is determined). The processes S10 to S12 are repeated until calculation is performed).

  S14: The movement probability calculation unit 9 obtains the similarity Stotal (p ′, q ′, p, q) between the sub-partitions using the indices A, B, C, and D, and proceeds to processing S15.

  S15: The movement probability calculation unit 9 determines whether or not the similarity Stotal has been calculated for all the sub-sections of the current frame that is the scanning target. If the similarity Stotal is calculated for all sub-parts of the current frame that is the scan target in this determination, the process proceeds to step S16. If all the calculations are not completed, the process proceeds to process S6 (all the current frames that are the scan target). Steps S6 to S14 are repeated until the similarity Stotal is calculated for the sub-parts).

  S16: The moving object tracking unit 10 sets the sub-partition (p, q) of the current frame that maximizes the similarity Stotal (p ′, q ′, p, q) between the sub-partitions as the movement destination (pdst, qdst). Then, the process proceeds to process S17.

  S17: The moving object tracking unit 10 receives the moving object ID Attrib_SubBlock (t−1) [p ′] [q ′] [kID of the sub-partition (p ′, q ′) of the previous frame from the sub-partition attribute storage unit. ] Is taken out and the process proceeds to step S18.

  S18: The moving object tracking unit 10 extracts the moving object ID Attrib_SubBlock (t) [pdst] [qdst] [kID] of the movement destination sub-partition (pdst, qdst) of the current frame from the sub-partition attribute storage unit, and processes it. The process moves to S19.

  S19: If the moving object ID has already been stored in the moving destination sub-part (Attrib_SubBlock (t) [pdst] [qdst] [kID]) of the current frame, the moving object tracking unit 10 proceeds to processing S20, If stored, the process proceeds to step S22.

  S20: The moving object tracking unit 10 extracts the similarity between the sub-partitions of the movement-destination sub-partition (pdst, qdst) of the current frame from the sub-partition attribute storage unit, sets it as Stotal_save, and proceeds to the process S21.

  S21: The moving object tracking unit 10 compares the similarity Stotal (p ′, q ′, p, q) between the sub-partitions obtained in the process S16 with the similarity Stotal_save in the process S20, and Stotal (p ′, If q ′, pdst, qdst) ≧ Stotal_save, the process proceeds to process S22, and if Stotal (p ′, q ′, pdst, qdst) <Stotal_save, the process proceeds to process S23.

  S22: The moving object tracking unit 10 stores the ID value Attrib_SubBlock (t−1) [p ′] [q ′] [kID] of the movement source and the similarity Stotal (p ′, q between sub-partitions) in the sub-partition attribute storage unit. ', pdst, qdst) is set, and the process proceeds to step S23.

  S23: The moving object tracking unit 10 returns to the process S3 if all the sub-sections of the previous frame have been scanned, and returns to the process S5 if not scanned.

  In this way, when information regarding a moving object is required, data is read from the sub-partition attribute storage unit or the moving object attribute storage unit.

(3): Description of background area In the above process, the detection of the background area was not mentioned for the sake of simplicity, but the background area was treated as a stationary moving object and moved appropriately to the sub-section of the background area. By giving the object ID, it can be distinguished from other moving objects. Since the background sub-section is stationary and the movement vector component is zero, the indices A and B are set to zero. The indicators C and D can be obtained by the method described above.

(4): Explanation of a case where the moving object includes a region with little variation in density The sub-compartment assembly generation means that makes the size of the sub-partition variable is provided. In particular, the density is uniform (the density is uniform) For sub-partitions that include the same pattern even if they are not included (ie, the same texture), create a sub-partition aggregate that combines subpartitions that have a certain degree of relationship between adjacent subpartitions, and move the aggregate The moving object is tracked by estimating the destination.

If the moving object includes a region with little variation in shading, the destination cannot be determined uniquely. FIG. 8 is an explanatory diagram in the case where the moving object includes a region with less variation in shading. For example, in the sub-parts b ₅ and b ₈ of the moving object in FIG. 8A, the degree of relationship between the adjacent sub-parts and the local correlation value match between the previous frame and the current frame, so that a plurality of destination candidates are generated. , Can not be determined uniquely.

  Thus, sub-partitions that satisfy a certain degree of variation in density and have a certain degree of relationship between adjacent sub-partitions are combined to create a sub-partition aggregate, and a movement destination is estimated for each aggregate. When the following equation is satisfied, it is determined that the sub-division is a uniform gray shade subdivision with little gray shade variation.

σ (p, q) ² <F _th
Here, σ (p, q) ² is the variance of the pixel values of the sub-partition and is given by the following equation. F _th is a threshold value for determining whether or not it is uniform (dispersion is uniform when the value is small, and non-uniform when the value is large).

    Here, N is the number of pixels in the sub-partition, I (i, j) is the luminance value of the pixel included in the sub-partition (p, q), R is the sub-partition area, and I bar is the pixel included in the sub-partition. Indicates the average luminance value.

  If it is determined that a certain sub-partition and its adjacent sub-partitions are uniform gray sub-partitions, and the degree of relationship between adjacent sub-partitions satisfies the following condition, the sub-partitions are combined to generate an aggregate.

Here, R _th is a threshold value for determining whether or not to generate an aggregate.

  The indices A, B, C, and D when scanning in units of sub-partition aggregates are obtained as follows.

(Explanation of indicators A and B)
A movement vector is calculated using a rectangular upper left end point (see FIG. 8B) circumscribing the uniform gray subdivision aggregate. The upper left end point (iRECT _LT , jRECT _LT ) of the circumscribed rectangle is obtained by the following equation.

iRECT _LT = min (i _LT (p, q) | (p, q) ∈ R _flat )
jRECT _LT = min (j _LT (p, q) | (p, q) ∈ R _flat )
Here, R _flat represents the region of the uniform gray sub-compartment aggregate.

  Indexes A and B are calculated from the movement vector V ′ of the sub-partition (p ′, q ′) of the previous frame and the movement vector V of the current frame obtained by the following equation using the same method as the movement vector calculation unit described above. .

_{V = (v x, v y} ) = (i LT -iRECT LT ', j LT -jRECT LT')
(The movement destination of the upper left sub-part of the circumscribed rectangle of the sub-part aggregate is defined as sub-part (p, q))

(Explanation of indicator C)
For the pixels at the same position included in the sub-partition aggregate, the luminance value difference is calculated between the previous frame (time t-1) and the current frame (time t), and the sum of absolute values is calculated.

In the above equation, I _t-1 (i ′, j ′) is the luminance value of the pixel included in the sub-partition set of the previous frame, and I _t (i, j) is the sub-partition (p, q) of the current frame. The luminance value of the pixel included in the range of the sub-partition aggregate of the previous frame as a reference, R _t-1 is in the sub-partition aggregate area of the previous frame, and R _t is in the sub-partition aggregate area of the current frame Represents.

(Explanation of indicator D)
The degree of relationship between adjacent sub-partitions of a sub-partition aggregate is calculated at a location where the aggregate is in contact with a non-aggregate. For example, in FIG. 8B, an aggregate indicated by a thick line, such as REL (sub _b1 , sub _a2 ), REL (sub _b2 , sub _a3 ), REL (sub _b3 , sub _a4 ) _,. The degree of relationship between adjacent sub-partitions is obtained between the sub-partitions that sandwich the non-aggregate boundary.

  An index D based on the degree of relationship between adjacent sub-partitions is as follows.

    Here, R ′ represents a set of sub-partitions of the previous frame adjacent to the sub-partition set, and R represents a set of sub-partitions of the current frame adjacent to the sub-partition set.

  In this way, the movement probability calculation unit 9 determines the movement destination using the obtained indexes A, B, C, and D.

  FIG. 9 is an explanatory diagram of the moving object tracking device when the sub-partition aggregate is used. In FIG. 9, the moving object tracking device includes an image generation unit 1, a sub-partition attribute storage unit 2, a moving object attribute storage unit 3, a movement vector calculation unit 4, a local correlation value calculation unit 5, a sub-partition feature calculation unit 6, Adjacent sub-partition relationship degree calculation unit 7, adjacent sub-partition relation degree comparison unit 8, movement probability calculation unit 9, moving object tracking unit 10, sub-partition assembly generation unit 11, movement vector calculation unit 14, local correlation value calculation 15, a sub-part feature calculation unit 16, a relationship calculation unit 17 between adjacent sub-parts, and a relationship comparison unit 18 between adjacent sub-parts are provided.

  Here, the movement vector calculation unit 4, the local correlation value calculation unit 5, the sub-part feature calculation unit 6, the relationship degree between adjacent sub-parts calculation unit 7, and the relation degree comparison unit 8 between adjacent sub-parts are for a single sub-partition. , The same as FIG. The movement vector calculation unit 14, local correlation value calculation unit 15, sub-part feature calculation unit 16, adjacent sub-partition relationship degree calculation unit 17, and adjacent sub-partition relationship degree comparison unit 18 are for sub-partition aggregates. The indexes A, B, C, and D are obtained for the sub-compartment aggregate generated by the partition aggregate generation unit 11.

  FIG. 10 is a moving object tracking processing flow when the sub-partition aggregate is used. Hereinafter, the moving object tracking process of FIG. 10 will be described according to the processes S31 to S44.

  S31: As an initial setting, various constants (image size SizeX, SizeY, number of pixels n on one side of the sub-partition, etc.) are set in advance in the processing unit such as the image generation unit 1 or each calculation unit, and the process proceeds to processing S32.

  S32: The image generation unit 1 starts photographing with the camera, and proceeds to processing S33.

  S33: The image generation unit 1 acquires an image at each sampling time, and proceeds to processing S34.

  S34: The image generating unit 1 moves the data stored in the current frame image buffer to the previous frame image buffer, stores the image acquired in the process S33 in the current frame image buffer, and proceeds to process S35.

  S35: The movement vector calculation unit 4 extracts one sub-partition (p ′, q ′) of the previous frame, and proceeds to processing S36.

  S36: The movement vector calculation unit 4 extracts one sub-partition (p, q) of the current frame, and proceeds to processing S37. The processes S31 to S36 are the same as the processes S1 to S6 in FIG.

  S37: The sub-partition aggregate generation unit 11 determines whether the sub-partition (p ′, q ′) of the previous frame is a uniform gray sub-partition. If it is determined that the sub-section is uniform, the process proceeds to step S38. If the sub-section is not uniform, the process proceeds to step S7 in FIG.

  S38: The sub-partition aggregate generation unit 11 generates a uniform subpartition aggregate from the degree of inter-subpartition relation between adjacent sub-partitions, and proceeds to processing S39.

  S39: The movement vector calculation unit 14 calculates the movement vector V from the upper left pixel coordinates of the circumscribed rectangle of the sub-partition set of the previous frame and the upper left pixel coordinates of the sub-partition (p, q) of the current frame, Control goes to step S40.

  S40: The movement vector calculation unit 14 extracts the movement vector V ′ of the sub-partition (p ′, q ′) of the previous frame from the sub-partition attribute storage part, and obtains the movement vector V of the current frame obtained in the processing S39. The vector magnitude difference value A (index A) and the inner product B (index B) are calculated, and the process proceeds to step S41.

  S41: The local correlation value calculation unit 15 includes the pixel of the previous frame image buffer included in the sub-partition aggregate of the previous frame and the current sub-part aggregate based on the sub-partition (p, q). The local correlation value C (index C) is calculated from the pixels in the frame image buffer, and the process proceeds to step S42.

S42: The relationship calculation unit 17 between the adjacent sub-partitions calculates the degree of relation REL ′ (sub ₁ , sub ₂ ) for the sub-partition where the sub-partition aggregate (aggregate) and the non-aggregate are in contact with each other in the previous frame. After calculating, the process proceeds to step S43.

S43: The relationship calculation unit 17 between the adjacent sub-partitions determines the degree of relation REL (sub ₁ , sub for sub-parts where a range corresponding to the sub-partition aggregate of the previous frame and a section outside the range are in contact with each other in the current frame. ₂ ) is calculated, and the process proceeds to step S44.

S44: The relation degree comparison unit 18 between the adjacent sub-partitions calculates an index D based on the relation degree between the sub-partitions from the relational degree REL ′ (sub ₁ , sub ₂ ) and the relational degree REL (sub ₁ , sub ₂ ). 7 is transferred to the process S14.

(5): Explanation when a region that does not exist in the previous frame appears When a region (new sub-partition) that does not exist in the previous frame, such as a new object or a shielded part of an existing object, appears, the new sub-partition and its adjacent sub New sub-partition evaluation means (new sub-partition determination unit) for determining the degree of relationship with the section, and by determining whether it belongs to an existing object, background, or new object (new object determination) To track.

  In the sub-section of the current frame, the section to which no moving object ID is given is a new sub-section that has no correspondence with the sub-section of the previous frame.

FIG. 11 is an explanatory diagram of a new sub-partition, and FIG. 11A is an explanation of an example of existing object and background determination. In FIG. 11 (a), a part of the existing object that has been occluded appears as a new sub-section. At this time, it is determined whether the new sub-partition belongs to the existing object or the background region by calculating the degree of relationship between the adjacent sub-partitions between the new sub-partition and its adjacent sections. First, the total value SUM _{EX of the} degree of relationship between adjacent sub-partitions between a new sub-partition and an existing object is calculated as follows.

Here, sub _new represents a new sub section, and R _EX represents a sub section of an existing object adjacent to the new sub section. N _EX is the number of sub-sections of adjacent existing objects.

Similarly, the total value SUM _{BK of the} degree of relationship between adjacent sub-partitions between the new sub-partition and the background region is calculated as follows.

Here, sub _new represents a new sub-partition, and _RBK represents a sub-partition of the background area adjacent to the new sub-partition. N _BK is the number of sub-divisions of adjacent background regions.

If the degree of relationship between adjacent sub-partitions is large, the similarity between the sub-partitions is high, so the new section is considered to belong to a region having a larger total value. That means
SUM _EX ≧ SUM _BK : belongs to an existing object SUM _EX <SUM _BK : belongs to a background area. That is, in FIG. 11A, the black square at the center is the new sub-compartment, the sub-compartments (1), (2), and (4) in the shaded area are the existing objects, and the sub-compartments (3), (5), (6), (7) (8) is the background. At this time, the degree of relationship between the new sub-partition and the existing object sub-partition (1) (2) (4) is large as shown in the right figure (the histogram of brightness distribution etc. is close). 3) Since the degree of relationship with (5) (6) (7) (8) is small as shown in the right figure (the histogram of brightness distribution etc. is separated), this new sub-partition is determined to belong to the existing object The

  FIG. 11B illustrates an example of new object determination. In FIG. 11B, a new moving object appears on the screen. At this time, it is determined whether the new sub-partition belongs to the new object or the background area by calculating the degree of relationship between the adjacent sub-partitions between the new sub-partition and its adjacent sections. First, the total value SUM of the degree of relationship between the new sub-partition and the adjacent sub-partition is calculated.

Here, sub _new represents a new sub-partition, and R _next represents a sub-partition adjacent to the new sub-partition.

If new sub-zone the background, the total value SUM becomes large the degree of relationship between the adjacent sub-compartments also have a large value, if the new object, the total value SUM has a small value, where, by the threshold SUM _th Whether a new object or a background area is determined. That means
SUM ≧ SUM _th : Background area SUM <SUM _th : New object That is, in FIG. 11 (b), the central black square is the new sub-partition, and the other sub-partitions (1) (2) (3) (4) (5) (6) (7) (8) are the background. And At this time, the relationship between the new sub-section and the background area sub-section (1) (2) (3) (4) (5) (6) (7) (8) is ) Since it is small, this new sub-partition is determined to be a new object.

  FIG. 12 is an explanatory diagram of the moving object tracking device when the new sub-partition determining unit is used. In FIG. 12, the moving body tracking device includes an image generation unit 1, a sub-partition attribute storage unit 2, a moving object attribute storage unit 3, a movement vector calculation unit 4, a local correlation value calculation unit 5, a sub-part feature calculation unit 6, An adjacent sub-partition relationship calculation unit 7, an adjacent sub-partition relationship comparison unit 8, a movement probability calculation unit 9, a moving object tracking unit 10, and a new sub-partition determination unit 20 are provided.

  Here, the image generation unit 1, the sub-partition attribute storage unit 2, the moving object attribute storage unit 3, the movement vector calculation unit 4, the local correlation value calculation unit 5, the sub-part feature calculation unit 6, and the relationship calculation unit between adjacent sub-parts 7, the adjacent sub-partition relationship degree comparison unit 8, the movement probability calculation unit 9, and the moving object tracking unit 10 are the same as those in FIG. Is a new addition.

  FIG. 13 is a moving object tracking processing flow when the new sub-partition determining unit is used. Hereinafter, the moving object tracking process of FIG. 13 will be described according to the processes S51 to S61.

  S51: The new sub-partition determination unit 20 obtains Attrib_SubBlock (t) [pdst] [qdst] [kID], which is the moving object ID of the destination sub-partition (pdst, qdst) of the current frame, from the sub-partition attribute storage unit 2. Then, the process proceeds to step S52.

  S52: The new sub-partition determination unit 20 determines whether the moving object ID has already been stored in Attrib_SubBlock (t) [pdst] [qdst] [kID]. If it is determined that the moving object ID has already been stored, the process proceeds to step S61. If not, the process proceeds to process S53.

S53: The new sub-partition determination unit 20 calculates the total value SUM _{EX of the} degree of relationship between adjacent sub-partitions with the moving object, and proceeds to process S54.

S54: The new sub-partition determination unit 20 calculates the total value SUM _{BK of the} degree of relationship between adjacent sub-partitions with the background area. If there is an existing object and a background area in the adjacent sub-partitions, the process proceeds to process S55, If it is only an existing object, the process proceeds to process S56, and if the adjacent sub-section is only a background area, the process proceeds to process S57.

S55: The new sub-partition determination unit 20 determines whether or not SUM _EX ≧ SUM _BK (belongs to an existing object). If it is determined that SUM _EX ≧ SUM _BK , the process proceeds to step S58. If SUM _EX ≧ SUM _BK is not satisfied, the process proceeds to step S59.

S56: The new sub-partition determination unit 20 determines whether SUM _EX ≧ SUM _th . If it is determined that SUM _EX ≧ SUM _th , the process proceeds to step S58. If SUM _EX ≧ SUM _th , the process proceeds to step S60.

S57: The new sub-partition determination unit 20 determines whether SUM _BK ≧ SUM _th . If it is determined that SUM _BK ≧ SUM _th , the process proceeds to step S59. If SUM _BK ≧ SUM _th , the process proceeds to step S60.

  S58: The new sub-partition determination unit 20 stores the (existing) moving object ID in the sub-partition attribute storage unit 2, and proceeds to the process S61.

  S59: The new sub-partition determination unit 20 stores the background area ID in the sub-partition attribute storage unit 2, and proceeds to processing S61.

  S60: The new sub-partition determination unit 20 stores the new moving object ID in the sub-partition attribute storage unit 2, and proceeds to processing S61.

  S61: The new sub-partition determination unit 20 determines whether all the sub-partitions of the current frame have been scanned. If all the sub-partitions of the current frame have been scanned by this determination, the process ends. If all the sub-partitions of the current frame have not been scanned, the process returns to step S51.

  As described above, according to the present invention, the relationship between adjacent sub-partitions is quantitatively expressed using the degree of relation between adjacent sub-partitions, and combined with an index based on the similarity of sub-partitions between frames. Since an optimal destination is determined, a moving object can be reliably tracked. In addition, even when a moving object includes a region with little shading or a region that does not exist in the previous frame, such as a new object or a shielded portion of an existing object, tracking can be performed reliably. Note that the processing flow described separately in FIGS. 7, 10, and 13 is actually one processing flow.

(6): Description of program installation Image generation unit (image generation unit) 1, sub-partition attribute storage unit 2, moving object attribute storage unit 3, movement vector calculation unit (movement vector calculation unit) 4, local correlation value calculation unit ( (Local correlation value calculation means) 5, sub-part feature calculation part 6, adjacent sub-partition relation degree calculation part (sub-partition relation degree calculation means) 7, adjacent sub-partition relation degree comparison part 8, movement probability calculation part 9, Moving object tracking unit 10, sub-partition assembly generation unit (sub-partition assembly generation unit) 11, movement vector calculation unit (movement vector calculation unit) 14, local correlation value calculation unit (local correlation value calculation unit) 15, sub-partition The feature calculation unit 16, the relationship calculation unit 17 between adjacent sub-partitions, the relationship comparison unit 18 between adjacent sub-partitions, a new sub-partition determination unit (new object determination means) 20 and the like can be configured by a program, and a main control unit (CPU) But To be executed and stored in the main memory. This program is processed by a computer (information processing apparatus). The computer includes hardware such as a main control unit, main memory, a file device, an output device such as a display device, and an input device.

  The program of the present invention is installed on this computer. In this installation, these programs are stored in a portable recording medium such as a floppy disk or a magneto-optical disk, and the recording medium provided in the computer is accessed via a drive device. Alternatively, it is installed in a file device provided in the computer via a network such as a LAN. Accordingly, it is possible to easily provide a moving object tracking device that can detect and track an object with high accuracy.

It is explanatory drawing of the moving object tracking apparatus of this invention. It is explanatory drawing of the sub division attribute of this invention. It is explanatory drawing of the relationship between the sub division of this invention and a moving object. It is explanatory drawing of the moving object attribute of this invention. It is explanatory drawing of the example of a sub division before and behind the movement of this invention. It is explanatory drawing of the example of calculation of the histogram of this invention. It is a moving object tracking processing flow of this invention. It is explanatory drawing in case the area | region with little shading fluctuation | variation is contained in the moving object of this invention. It is explanatory drawing of the moving object tracking apparatus in the case of using the subcompartment aggregate | assembly of this invention. It is a moving object tracking process flow in the case of using the subcompartment aggregate of the present invention. It is explanatory drawing of the novel subdivision of this invention. It is explanatory drawing of the moving object tracking apparatus in the case of using the novel sub-partition determination part of this invention. It is a moving object tracking process flow in the case of using the novel sub-partition determination part of this invention.

Explanation of symbols

1 Image generation unit (image generation means)
2 sub-partition attribute storage unit 3 moving object attribute storage unit 4 movement vector calculation unit (movement vector calculation means)
5 Local correlation value calculation unit (local correlation value calculation means)
6 sub-partition feature calculation unit 7 relationship calculation unit between adjacent sub-partitions (sub-partition relation calculation means)
8 Relational comparison part between adjacent sub-parts 9 Movement probability calculation part 10 Moving object tracking part

Claims (5)

Image generating means for dividing the input image into sub-sections of a certain shape;
A sub-compartment relationship degree calculating means for obtaining a degree of relation between a certain sub-partition and an adjacent sub-partition;
The degree-of-sub-partition relationship calculation means obtains the degree of relation between the sub-partition of the previous frame and its adjacent sub-partition and the degree of relation between the sub-partition of the current frame and its adjacent sub-partition, A moving object tracking device that tracks sub-sections having the closest relationship between frames as the same one. A movement vector calculation means for obtaining a similarity between the movement vector of the sub-partition of the previous frame and the movement vector of the sub-partition of the current frame;
A local correlation value calculating means for obtaining a similarity between the sub-section of the previous frame and the sub-section of the current frame;
A moving object is obtained by combining an index based on similarity obtained by the movement vector calculating means, an index based on similarity obtained by the local correlation value calculating means, and an index based on the degree of relationship obtained by the inter-sub-partition relationship degree calculating means. The moving object tracking device according to claim 1, wherein tracking is performed. A sub-partition assembly generating means for changing the size of the subpartition,
The moving object tracking device according to claim 1, wherein a sub-compartment aggregate is generated for the same sub-partition, the moving destination of the sub-partition aggregate is estimated, and the moving object is tracked. A new sub-partition determining means for obtaining a degree of relationship between the new sub-partition and its adjacent sub-partition when a new sub-partition that does not exist in the previous frame appears;
2. The new sub-partition determining means determines whether the new sub-partition belongs to an existing object, a background region, or a new object based on the obtained degree of relationship, and tracks a moving object. The moving object tracking apparatus in any one of -3. Image generating means for dividing the input image into sub-sections of a certain shape;
A sub-compartment relationship degree calculating means for obtaining a degree of relation between a certain sub-partition and an adjacent sub-partition;
The degree-of-sub-partition relationship calculation means obtains the degree of relation between the sub-partition of the previous frame and its adjacent sub-partition and the degree of relation between the sub-partition of the current frame and its adjacent sub-partition, As a means to track the closest sub-partitions with the same frame relationship,
A program that allows a computer to function.

Images