WO2010113417A1 - Moving object tracking device, moving object tracking method, and moving object tracking program - Google Patents

Abstract

A moving object tracking device (20) detects a moving object in a time series image by performing image processing. The moving object tracking device (20) comprises an object map storage unit (26), a contour extraction unit (30), and a correction unit (31). The object map storage unit (26) allocates an identification code corresponding to the moving object to each of a plurality of blocks obtained by dividing each frame in the time series image and stores the identification code. The contour extraction unit (30) extracts the contour of the moving object from the time series image. The correction unit (31) corrects the identification code allocated to the block on the basis of the contour.

Description

Moving object tracking device, moving object tracking method, and moving object tracking program

The present invention relates to a moving object tracking device, a moving object tracking method, and a moving object tracking program for tracking a moving object (for example, a car, a bicycle, and an animal) in an image by processing a time-series image.
This application claims priority based on Japanese Patent Application No. 2009-090489 filed in Japan on April 2, 2009, the contents of which are incorporated herein by reference.

In recent years, there has been a demand for a technique for accurately detecting a moving object in an image by processing an image captured by a camera. For example, early detection of traffic accidents can not only increase the success rate of lifesaving by quick rescue activities, but also speed up on-site inspection by the police to alleviate traffic jams. For this reason, automation of recognition for various traffic accidents is expected. In order to increase the recognition rate of traffic accidents, it is necessary to accurately detect a moving object by performing image processing on an image captured by a camera.

31A to 31D schematically show images at time t = 1 to 4 captured by a camera installed above the center line of the expressway.

When vehicles frequently overlap on the image, there is a problem that it becomes difficult to track each vehicle by image processing. In order to solve this problem, there is a method in which a plurality of cameras are installed along the road and the images photographed by these cameras are comprehensively processed.

However, since this method needs to include a plurality of cameras and image processing apparatuses, the cost becomes high. Further, in this case, the image processing becomes complicated because the image processing of each camera must be related and comprehensively processed.

Therefore, the inventors of the present application solved the above problem by a method of detecting a moving object retroactively as described in Patent Document 1 and Patent Document 2 below.

First, the time series images at time t = 1 to 4 are temporarily stored. At time t = 4, the vehicles M1 and M2 are identified, and the motion vectors of these vehicles M1 and M2 are obtained. An image at time t = 3 in which the vehicles M1 and M2 are identified is assumed by moving the vehicles M1 and M2 in the image at time t = 4 based on these motion vectors. From the correlation between the assumed image and the actual image, the vehicles M1 and M2 in the image at time t = 3 are identified.

Next, for the images at time t = 3 and time t = 2, the vehicles M1 and M2 in the image at time t = 2 are identified by similar image processing. Next, for the images at time t = 2 and time t = 1, the vehicles M1 and M2 in the image at time t = 1 are identified by similar image processing.

By performing such image processing, the vehicles M1 and M2 can be tracked with a single camera.

JP 2002-133421 A Japanese Patent Laid-Open No. 2004-207786

In the above-described prior art, an image captured with the camera fixed, that is, an image with a fixed background is subjected to image processing to accurately detect a moving object in the image.

On the other hand, when the camera is panned or zoomed, the background image that has been fixed up to that time also varies depending on the camera panning or zooming. When image processing is performed on an image whose background changes in this way, there is a problem in that the boundary between the image of the moving object and the background image is not clear, and the moving object in the image cannot be accurately detected.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a moving object tracking device, a moving object tracking method, and a moving object tracking program that can accurately detect a moving object in an image whose background changes. .

The present invention employs the following means in order to solve the above problems and achieve the object.
(1) The moving object tracking device of the present invention is a moving object tracking device that detects a moving object in a time-series image by image processing, and an identification code corresponding to the moving object is assigned to each frame of the time-series image. Is assigned to each block divided into a plurality of blocks, and is stored; an outline extraction unit that extracts an outline of the moving object from the time-series image; and the blocks assigned to the blocks based on the outlines A correction unit that corrects the identification code.

(2) In the moving object tracking device according to (1), the contour extraction unit calculates the contour of the moving object based on an image area corresponding to the moving object stored in the object map storage unit. A target region setting unit that sets a target region to be extracted; and a contour extraction processing unit that extracts a contour of the moving object from the time-series image with respect to the target region.

(3) In the moving object tracking device according to (2) above, the number of pixels corresponding to an edge in the target region is determined by the contour extraction unit with respect to an image obtained by performing edge extraction processing on the time-series image. And a target area correction unit that corrects the target area for each coordinate axis based on the histogram, and the contour extraction processing unit applies the corrected target area to the corrected target area. Then, the contour of the moving object is extracted from the time-series image; a configuration may be adopted.

(4) In the moving object tracking device according to (1), the contour extraction unit sets a plurality of moving objects in which occlusion has occurred as an integrated moving object, and extracts the integrated moving object from the time-series image. An outline may be extracted.

(5) In the moving object tracking device according to (4), the correction unit is based on identification codes corresponding to a plurality of moving objects in which the occlusion occurs, which is stored in the object map storage unit. Based on the information indicating the boundaries of the plurality of moving objects and the contour of the moving object that is extracted by integrating the plurality of moving objects in which the occlusion is generated by the contour extraction unit, the object map storage unit stores The stored identification code may be corrected.

(6) The moving object tracking device according to (1) above, a determination unit that determines whether or not the background image is fluctuating; and when the determination unit determines that the background image is fluctuating, A first control unit that causes the contour extraction unit to extract a contour and causes the correction unit to correct the identification code stored in the object map storage unit.

(7) The moving object tracking device according to (1) detects the size of the moving object or the amount of change in the movement amount stored in the object map storage unit for each unit time based on the identification code. A moving object fluctuation amount detecting unit that detects the moving object size or movement amount of the moving object per unit time detected by the moving object fluctuation amount detection unit is larger than a predetermined size or movement amount fluctuation amount And a second control unit that causes the contour extraction unit to extract a contour and causes the correction unit to correct the identification code stored in the object map storage unit.

(8) In the moving object tracking device according to (1), the moving object tracking device stores the identification code and the object map storage unit based on the result of image processing of the time-series image. And a moving object tracking unit that updates the motion vector of the moving object, and the moving object tracking unit is adjacent to each of the consecutive N images (N ≧ 2) of the time-series images. An identification code providing step of attaching different identification codes to the moving objects that overlap each other on an image by attaching the same identification code to blocks whose absolute values of motion vector differences of the blocks are within a predetermined value; Each of the first object, which is a block group to which a first identification code is attached, and the second object, which is a block group to which a second identification code is attached, and A determination step of determining whether or not the degree of correlation between the first objects of temporally adjacent images with respect to an image is greater than or equal to a predetermined value; and when the determination step determines affirmative, A tracking step of tracking the first object and the second object; and a data stored in the object map storage unit based on the first object and the second object tracked back in time by the tracking step. An update process for updating the identification code and the motion vector may be employed.

(9) The moving object tracking method of the present invention is a moving object tracking method for detecting a moving object in a time-series image by image processing, and an identification code corresponding to the moving object is assigned to each frame of the time-series image. Assigning to a block divided into a plurality of blocks, storing the blocks, extracting a contour of the moving object from the time-series image, correcting the identification code assigned to the block based on the contours And comprising.

(10) The moving object tracking program of the present invention provides a computer as a moving object tracking device that detects a moving object in a time-series image by image processing, and assigns an identification code corresponding to the moving object to each of the time-series images. Allocating and storing a frame into a plurality of divided blocks; extracting a contour of the moving object from the time-series image; and correcting the identification code allocated to the block based on the contour And a process is executed.

According to the present invention, by correcting the block identification code stored in the object map storage unit based on the extracted contour of the moving object, the moving object in the image can be detected even when the background of the image fluctuates. It can be detected accurately.

1 is a schematic diagram of a moving object tracking system using a moving object tracking device according to an embodiment of the present invention. It is a block diagram which shows the structure of the moving object tracking apparatus which concerns on the same embodiment. It is a figure which shows ID of the moving object provided to the slit and block which were each set to the four entrances to an intersection and the four exits from an intersection in a frame image. It is a figure which shows typically the image in the time t-1 with a block boundary line. It is a figure which shows the image in the time t typically with a block boundary line. It is a figure which shows typically the image in the time t-1 with a pixel boundary line. It is a figure which shows the image in the time t typically with a pixel boundary line. It is a figure which shows typically the image in the time t-1 with the motion vector provided to the block. It is a figure which shows typically the image in the time t with the motion vector provided to the block. It is a figure which shows typically the motion vector and object boundary which were provided to the object map at the time t-1. It is a figure which shows typically the motion vector and object boundary provided to the object map at the time t. It is a flowchart which shows the estimation method of an undetermined motion vector. FIG. 10 is a diagram schematically showing motion vectors and object boundaries assigned to an object map at time t−1 when the same estimation method is applied. It is a figure which shows typically the motion vector and object boundary provided to the object map at the same time t. It is a figure which shows typically the 1st step of the motion vector and object boundary provided to the object map at the time of applying the estimation method. It is a figure which shows the said 2nd step typically. It is a figure which shows the said 3rd step typically. FIG. 6 is a state transition diagram showing state transition of modes in the moving object tracking device 20. 12 is a flowchart illustrating an operation of the moving object tracking device 20 in the correction mode of FIG. 11. It is a figure which shows area term Earea. It is a figure which shows the 1st step of an example of the processing result of Snakes. It is a figure which shows the 2nd step. It is a figure which shows the said 3rd step. It is a figure which shows the 1st step of an example of generation of edge distribution and a histogram. It is a figure which shows the 2nd step. It is a figure which shows the said 3rd step. It is a figure which shows the example of correction of the object map by Snakes. It is a figure which shows the modification example. It is a figure which shows the modification example. It is a figure which shows the modification example. It is a figure which shows the detection result in the flame | frame 80 in the case where the camera 10 fluctuates, when not applying an inter-layer cooperation algorithm. FIG. 10 is a diagram showing a detection result in the same frame 80. It is a figure which shows the detection result in the same frame 95. FIG. It is a figure which shows the detection result in the same frame 95. FIG. It is a figure which shows the detection result in the same frame 107. FIG. It is a figure which shows the detection result in the same frame 107. FIG. It is a figure which shows the detection result in the same frame 115. FIG. It is a figure which shows the detection result in the same frame 115. FIG. It is a figure which shows the detection result in the same frame 142. FIG. It is a figure which shows the detection result in the same frame 142. FIG. It is a figure which shows the detection result in the same frame 153. FIG. It is a figure which shows the detection result in the same frame 153. FIG. It is a figure which shows the detection result in the flame | frame 80 in the case of applying the cooperation algorithm between layers, when the camera 10 fluctuates. FIG. 10 is a diagram showing a detection result in the same frame 80. It is a figure which shows the detection result in the same frame 95. FIG. It is a figure which shows the detection result in the same frame 95. FIG. It is a figure which shows the detection result in the same frame 107. FIG. It is a figure which shows the detection result in the same frame 107. FIG. It is a figure which shows the detection result in the same frame 115. FIG. It is a figure which shows the detection result in the same frame 115. FIG. It is a figure which shows the detection result in the same frame 142. FIG. It is a figure which shows the detection result in the same frame 142. FIG. It is a figure which shows the detection result in the same frame 153. FIG. It is a figure which shows the detection result in the same frame 153. FIG. It is a figure which shows the detection result of the moving object in case the occlusion has generate | occur | produced when the camera 10 fluctuates. It is a figure which shows the same detection result. It is a figure which shows typically the image in the time t = 1 imaged with the camera installed above the center line of the highway. It is a figure which shows typically the image in the same time t = 2. It is a figure which shows typically the image in the same time t = 3. It is a figure which shows typically the image in the same time t = 4.

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic diagram showing a configuration of a moving object tracking system using a moving object tracking device 20 according to an embodiment of the present invention. As shown in FIG. 1, the moving object tracking system includes an electronic camera 10 that captures an intersection and outputs an image signal, and a moving object tracking device 20 that processes the image and tracks the moving object. .

The time-series images captured by the electronic camera 10 are stored in an image memory 21 (described later) included in the moving object tracking device 20 at a rate of 12 frames / second. In this image memory 21, the oldest frame is rewritten with a new frame image. The electronic camera 10 can change an image area to be photographed by panning or zooming. The panning or zooming with respect to the electronic camera 10 may be controlled by the moving object tracking device 20 or may be controlled by a host control device that controls the moving object tracking system.

The moving object tracking device 20 performs image processing on a time series image (a time series image stored in an image memory 21 described later) taken by the electronic camera 10 to detect a moving object in the image.

Next, the configuration of the moving object tracking device 20 will be described with reference to FIG.

The image conversion unit 22 copies each frame image in the image memory 21 to the frame buffer memory 23, and converts the corresponding frame image in the image memory 21 into a spatial difference frame image using the copied image data. . This conversion is performed in two stages.

If the pixel value (luminance value) of the i-th row and j-th column of the original frame image is G (i, j), the pixel value H (i, j) of the i-th row and j-th column after the conversion in the first stage. Is represented by the following formula (1).

H (i, j) = Σneighberpixcels | G (i + di, j + dj) −G (i, j) | (1)

Here, Σneighberpixcels means the sum over di = −c to c and dj = −c to c, where c is a natural number. For example, when c = 1, Σneighberpixcels is the sum of eight pixels adjacent to the pixel in the i-th row and j-th column. When the illuminance changes, the pixel value G (i, j) and the neighboring pixel value G (i + di, j + dj) change similarly. For this reason, the image of H (i, j) is invariant to the change in illuminance.

Here, the absolute value of the difference between adjacent pixels is generally larger as the pixel value is larger. In order to increase the success rate of moving object tracking, it is preferable to acquire edge information almost equivalently to the case where the pixel value and the difference are large even when the pixel value is small and the difference is small. Therefore, H (i, j) is normalized as in the following formula (2).

H (i, j) = Σneighberpixcels | G (i + di, j + dj) −G (i, j) | / (Gi, j, max / Gmax) (2)

Here, Gi, j, max is the maximum value of the original pixel value used in the calculation of H (i, j). For example, when c = 1, Gi, j, max is the maximum value of 3 × 3 pixels centered on the pixel in the i-th row and j-th column. Gmax is 255 when the pixel value G (i, j) can take a maximum value, for example, when the pixel value is represented by 8 bits. Hereinafter, a case where c = 1 and Gmax = 255 will be described.

The maximum value that H (i, j) can take differs for each moving object. For example, if G (i, j) = Gmax and the values of 8 pixels adjacent to the pixel in the i-th row and j-th column are all 0, H (i, j) = 8 Gmax, so H (i, j, j) cannot be represented by 8 bits.

On the other hand, when a histogram of the values of H (i, j) at the edge portion of the moving object was created, it was found that most of the frequency was included in the range of H = 50 to 110. That is, as the value of H is larger than about 110, the number of pieces of edge information for tracking a moving object is small, so the importance is low.

Therefore, it is preferable to perform image processing at high speed by suppressing the portion where the value of H is large and shortening the bit length of the converted pixel. Therefore, as a second stage, this H (i, j) is converted to I (i, j) by the following equation (3) using a sigmoid function.

I = Gmax / {1 + exp [−β (H−α)]}〕 (3)

Sigmoid function has good linearity around H = α. Therefore, the threshold value α is set to the most frequent value of the frequency distribution of H having edge information, for example, 80.

Based on the above equations (2) and (3), the image conversion unit 22 converts the image having the pixel value G (i, j) into a spatial difference frame image having the pixel value I (i, j). Store in the image memory 21.

The background image generation unit 24, the ID generation / annihilation unit 25, and the moving object tracking unit 27 perform processing based on the spatial difference frame image in the image memory 21. Hereinafter, the spatial difference frame image is referred to as a frame image.

The background image generation unit 24 includes a storage unit and a processing unit. The processing unit accesses the image memory 21 and creates a histogram of pixel values for the corresponding pixels of all the frame images for the past 10 minutes. The processing unit generates an image having the mode value (mode) as the pixel value of the pixel as a background image in which no moving object is present, and stores this in the storage unit. The background image is updated by performing this process periodically.

As shown in FIG. 3, the ID generation / annihilation unit 25 includes the positions of the slits EN1 to EN4 and EX1 to EX4 arranged at the four entrances to the intersection and the four exits from the intersection in the frame image. Size data is preset. The ID generation / annihilation unit 25 reads the image data in the entrance slits EN1 to EN4 from the image memory 21, and determines whether there is a moving object in these entrance slits in units of blocks. The meshes in FIG. 3 represent blocks, and one block is composed of 8 × 8 pixels. When one frame is composed of 480 × 640 pixels, one frame is divided into 60 × 80 blocks. Whether or not there is a moving object in a block is determined by whether or not the sum of the absolute values of the differences between the pixels in the block and the corresponding pixels in the background image is greater than or equal to a predetermined value. This determination is similarly performed in the moving object tracking unit 27.

When the ID generation / annihilation unit 25 determines that there is a moving object in the block, it gives a new object identification code (hereinafter referred to as ID) to this block. If the ID generation / annihilation unit 25 determines that a moving object is present in a block adjacent to the ID-assigned block, the ID generation / annihilation unit 25 assigns the same ID as the assigned block to this adjacent block. This ID-added block includes a block adjacent to the entrance slit. For example, ID = 1 is assigned to the block in the entrance slit EN1 in FIG.

ID is assigned to the corresponding block in the object map storage unit 26. In the above-described case, the object map storage unit 26 stores an object map of 60 × 80 blocks. Each block is provided with a flag indicating whether an ID is assigned, and when an ID is assigned, the number and a motion vector of a block described later are assigned as block information. Note that without using this flag, if ID = 0, it may be determined that no ID is assigned. The most significant bit of the ID may be used as a flag.

The moving object tracking unit 27 assigns and moves IDs for the blocks in the moving direction and disappears IDs for the blocks in the opposite direction with respect to the cluster that has passed through the entrance slit (that is, tracking process of the cluster). The tracking process by the moving object tracking unit 27 is performed up to the exit slit for each cluster.

The ID generation / annihilation unit 25 further checks whether or not IDs are assigned to the blocks in the exit slits EX1 to EX4 based on the contents of the object map storage unit 26. If an ID is assigned, the ID is extinguished when the cluster passes through the exit slit. For example, when the block in the exit slit EX1 in FIG. 3 is changed from the state in which ID = 3 is assigned to the state in which no ID is assigned, ID = 3 is extinguished. The disappeared ID can be used as an ID to be generated next time.

Based on the object map at time t−1 stored in the object map storage unit 26 and the frame images at times t−1 and t stored in the image memory 21, the moving object tracking unit 27 performs time t−1. Are created in the object map storage unit 26. Hereinafter, this operation will be described.

4A to 7B schematically show images at times t-1 and t. The dotted lines in FIGS. 4A, 4B, and 6A to 7B are block boundaries. The dotted lines in FIGS. 5A and 5B are pixel boundaries.

The block in the i-th row and the j-th column is denoted as B (i, j), and the block in the i-th row and the j-th column at the time t is denoted as B (t: i, j). Let MV be the motion vector of block B (t−1: 1, 4). First, the block at time t that most corresponds to the area where block B (t−1: 1, 4) has been moved by MV is found. In the case of FIG. 4B, this block is B (t: 1, 5). As shown in FIGS. 5A and 5B, the degree of correlation between the image of the block B (t: 1, 5) and the image of the block size area AX at time t−1 is 1 in the area AX within the predetermined range AM. It is obtained every time the pixel is moved (that is, block matching).

Here, the range AM is larger than the block, and one side thereof is 1.5 times the number of pixels on one side of the block. The center of the range AM is a pixel at a position where the center of the block B (t: 1, 5) is moved by MV.

The correlation degree is a spatio-temporal texture correlation degree, and the larger the evaluation value UD that is the sum of absolute values of the difference between the block B (t: 1, 5) and the corresponding pixel value in the region AX, the larger the correlation degree.

Next, an area AX having a maximum correlation within the range AM is obtained. A vector starting from the center of this area and ending at the center of block B (1, 5) is determined as the motion vector of block B (t: 1, 5). Further, the ID of the block at time t−1 that is closest to the region AX where the degree of correlation is maximum is determined as the ID of the block B (t: 1, 5).

The moving object tracking unit 27 assigns the same ID to blocks whose absolute values of motion vector differences between adjacent blocks are equal to or less than a predetermined value. Thereby, one cluster is divided into a plurality of objects (moving objects) having different IDs. In FIG. 6A and FIG. 6B, the boundary between objects is shown by the thick line.

Although the image of the moving object does not exist on the object map, in FIG. 6A and FIG. 6B, the moving object is schematically drawn on the object map for easy understanding. FIG. 7A and FIG. 7B show object boundaries in bold lines on the object map, and correspond to FIG. 6A and FIG. 6B.

When one cluster is detected at the entrance slit EN1 of FIG. 3 and is not divided into a plurality of objects, and then divided into a plurality of objects as described above at time t1, the time is traced back from time t1. The object map is obtained in the same way as when is in the positive direction. Thereby, the process which divides | segments an object into a some object is performed with respect to the object map before time t1. Thereby, an object that could not be divided can be divided and recognized, and individual objects can be tracked.

In the above-mentioned Patent Document 1, each object is traced back in time after one cluster is separated into a plurality of clusters. However, according to the present embodiment, before separation into a plurality of clusters, for example, FIG. From t = 2 in FIG. 31A to FIG. 31D before t = 4, individual objects can be traced back in time. Therefore, the storage capacity of the image memory 21 can be reduced, and the amount of image processing can be reduced to reduce the load on the CPU.

In the above description, the case where the motion vector of the block in the cluster is obtained has been described. However, as shown in FIG. 9A, when there is a block for which the motion vector cannot be obtained, depending on the position of the block, It may be unknown if it belongs. If the color of each pixel in a block belonging to a certain moving object is almost the same, the motion vector cannot be determined by the block matching described above. For example, if an image (spatial difference frame image) is converted into a binary image and the number of “1” pixels in the block is equal to or smaller than a predetermined value, the block is not suitable for obtaining a motion vector by the above method. It is determined.

The motion vector of such a block is estimated by the method shown in FIG.

(Step S1)
If there is a block whose motion vector is undetermined, the process proceeds to step S2, and if it does not exist, the undetermined motion vector estimation process is terminated.

(Step S2)
The determined motion vectors MV1 to MVn are extracted from the eight blocks around the block B (i, j) whose motion vectors are undetermined.

(Step S3)
If the motion vector determined in step S2 exists, the process proceeds to step S4, and if not, the process proceeds to step S6.

(Step S4)
The motion vectors MV1 to MVn are divided into groups in which the absolute value of the difference between the vectors is within a predetermined value.

(Step S5)
The average value of the motion vectors of the group having the largest number of motion vectors is estimated as the motion vector of the block B (i, j). When there are a plurality of groups having the largest number of motion vectors, the average value of the motion vectors of any one group is estimated as the motion vector of the block B (i, j). Next, the process returns to step S1.

Note that since the motion vectors of the same group are substantially equal to each other, any one of the motion vectors of the same group may be estimated as the motion vector of the block B (i, j).

(Step S6)
The motion vector estimated in step S5 is set as the determined motion vector, and the process returns to step S1.

This process makes it possible to uniquely estimate an undetermined motion vector.

Next, a specific example of this estimation method will be described. In FIG. 9A, the motion vector of the block B (i, j) in the i-th row and j-th column is denoted as MV (i, j). In FIG. 9A, the motion vectors of blocks B (2,2), B (2,4) and B (3,3) are undetermined.

The motion vectors of the blocks around the block B (2, 2) are the group of MV (2, 1), MV (3, 1), MV (3, 2) and MV (2, 3), and MV (1 , 2) and MV (1, 3). Therefore, select the former group,
MV (2,2) = (MV (2,1) + MV (3,1) + MV (3,2) + MV (2,3)) / 4
Estimated.

The motion vectors of the blocks around the block B (2, 4) are a group of MV (2, 3), MV (3, 4) and MV (3, 5), and MV (1, 3), MV (1 4), MV (1, 5) and MV (2, 5). So select the latter group,
MV (2,4) = (MV (1,3) + MV (1,4) + MV (1,5) + MV (2,5)) / 4
Estimated.

The block motion vectors around block B (3, 3) are MV (2, 3), MV (3, 2), MV (4, 2), MV (4, 4) and MV (3,4). Because it is one group of
MV (3,3) = (MV (2,3) + MV (3,2) + MV (4,2) + MV (4,4) + MV (3,4))) / 5
Estimated.

In this way, an object map as shown in FIG. 9B is generated. In FIG. 9B, the boundary of the object is indicated by a thick line.

Even if the number of undetermined motion vectors is large as shown in FIG. 10A, steps S1 to S5 are repeated until a negative determination is made in step S3. As a result, the motion vector is uniquely estimated as shown in FIG. 10B. Next, in step S6, the estimated motion vector is regarded as the determined motion vector, and steps S1 to S5 are executed again. Thereby, the motion vector of the block B (3, 4) is uniquely estimated as shown in FIG. 10C. Next, by assigning the same ID to blocks whose absolute values of motion vector differences between adjacent blocks are equal to or less than a predetermined value, one cluster is divided into a plurality of objects having different IDs.

The moving object tracking unit 27 stores the time series of the object map stored in the object map storage unit 26 in a hard disk (not shown) as a tracking result.

Through the processing described above, each image of the time-series image is divided into a plurality of blocks in the object map storage unit 26, and an identification code indicating the moving object in the image is attached to the block corresponding to the moving object. In addition, the motion vector of the moving object corresponding to the block is attached to the block and stored.

Then, the moving object tracking unit 27 updates the block identification code and the motion vector stored in the object map storage unit 26 based on the result of image processing of the time-series image as described above. Specifically, the moving object tracking unit 27 updates the block identification code and the motion vector stored in the object map storage unit 26 according to the following procedures (1) to (4).

(1) For each of consecutive N images (N ≧ 2) of time-series images, the same identification code is attached to a block in which the absolute value of the motion vector difference between adjacent blocks is within a predetermined value. An identification code procedure for attaching different identification codes to moving objects that overlap each other.

(2) In each of the N images, a first object that is a block group to which a first identification code is attached is in contact with a second object that is a block group to which a second identification code is attached, and the time for the N image A determination procedure for determining whether or not the degree of correlation between first objects of adjacent images is equal to or greater than a predetermined value.

(3) A tracking procedure in which the first object and the second object are traced back in time after it is determined affirmative in the determination procedure.

(4) An update procedure for updating the block identification code and the motion vector stored in the object map storage unit 26 based on the first object and the second object tracked back in time by the tracking procedure.

The contour extraction unit 30 extracts the contour of the moving object from the time-series image. The contour extracting unit 30 extracts a plurality of moving objects having occlusions as a single moving object, and extracts a contour of the moving object as a single unit from a time-series image.

The contour extraction unit 30 includes a target region setting unit 301, a target region correction unit 302, and a contour extraction processing unit 303.

The target area setting unit 301 sets a target area for extracting the outline of the moving object based on the image area corresponding to the block of the moving object stored in the object map storage unit 26.

The contour extraction processing unit 303 extracts the contour of the moving object from the time-series image for the target region set by the target region setting unit 301.

The target area correction unit 302 projects, for each coordinate axis, a histogram of the number of pixels corresponding to the edges in the target area set by the target area setting unit 301 with respect to an image obtained by performing edge extraction processing on a time-series image. And generate. Based on the histogram generated for each coordinate axis, the target area set by the target area setting unit 301 is corrected for each coordinate axis.

Then, the contour extraction processing unit 303 described above may extract the contour of the moving object from the time-series image with respect to the target region corrected by the target region correction unit 302.

The correction unit 31 corrects the block identification code and the motion vector stored in the object map storage unit 26 based on the contour extracted by the contour extraction processing unit 303 of the contour extraction unit 30.

The correction unit 31 also includes information indicating the boundaries of the plurality of moving objects based on the identification codes corresponding to the plurality of moving objects in which occlusion occurs and stored in the object map storage unit 26, and contour extraction. The block identification code and the motion vector stored in the object map storage unit 26 are corrected based on the extracted outline of the moving object obtained by integrating the plurality of moving objects in which occlusion has occurred.

The determination unit 32 determines whether or not the background image is changing. The determination unit 32 determines whether or not the background image has changed based on an input signal indicating that the camera 10 has been panned or zoomed.

The determination unit 32 may compare the background image generated by the background image generation unit 24 with the image input from the camera 10 and determine whether or not the background image is fluctuating. Whether or not the background image is fluctuated by detecting a change in the position of the marker in the image included in the image input from the camera 10 by the marker being embedded in the background region in advance. It may be determined whether or not.

When the determination unit 32 determines that the background image is fluctuating, the control unit 34 (first control unit) controls the contour extraction unit 30 every predetermined period or every predetermined frame. Then, the contour is extracted, and the correction unit 31 is controlled to correct the block identification code and the motion vector stored in the object map storage unit 26.

The moving object fluctuation amount detection unit 33 will be described later.

<Background block>
In the above description, whether or not an object exists is checked by comparing the input image with the background image in units of blocks. For this reason, the processing method of a background image and an object differs. For example, since the background image is generated based on the captured images for the past 10 minutes, when the camera shakes, the shake cannot be reflected in the background image.

Therefore, an object map may be created by regarding the background image as an object. The object map generation method is different from the background image only in determining whether or not a moving object exists in the block. Since the background image is also regarded as an object, block matching is performed on all the blocks to assign IDs and determine MVs.

Note that an ID predetermined for the background image may be given to the background image. The background ID and the moving object can be easily identified by this predetermined ID.

As described above, even if the background image is set as one block and an ID is assigned to the background image, the block belongs between the background image and the moving object as shown in FIGS. 4A to 7B. The image can be determined.

As described above, by setting the background image as one block, even when the background image fluctuates in response to panning or zooming of the camera, processing can be performed in the same manner as when the background image is fixed.

<Operation of Moving Object Tracking Device 20>
Next, the operation of the moving object tracking device 20 will be described with reference to FIGS. 11 and 12. The operation mode of the moving object tracking device 20 will be described with reference to FIG.

First, the moving object tracking device 20 is in the camera fixed mode. In this camera fixing mode, the background image is not given an ID as an object, and the image generated by the background image generation unit 24 is used as the background image. This is because the background image does not change because the camera is fixed.

In this camera fixing mode, since the background image is fixed, the moving object tracking unit 27 identifies the moving object.

(Step S1)
Next, in accordance with the panning or zooming of the camera, the determination unit 32 determines that the background image is fluctuating. In response to this determination, the control unit 34 shifts from the camera fixing mode to the camera variation mode, registers the background image in the object, and assigns an ID.

(Step S2)
In the camera variation mode, the control unit 34 shifts to the correction mode every predetermined period or every predetermined frame. In this correction mode, the control unit 34 controls the contour extraction unit 30 to extract a contour, and controls the correction unit 31 to correct the block identification code and motion vector stored in the object map storage unit 26. .

(Step S3)
When the correction is completed in the correction mode, the control unit 34 transitions from the correction mode to the camera fluctuation mode.

Thereafter, during a period in which the camera 10 is changing, the control unit 34 alternately switches between the camera change mode and the correction mode.

(Step S4)
Thereafter, when the determination unit 32 determines that the fluctuation of the camera 10 has stopped, the control unit 34 deletes the ID assigned to the background image, and uses the image generated by the background image generation unit 24 as the background image. Use.

It should be noted that a predetermined period such as 10 minutes is required for the background image generation unit 24 to generate the background image. Therefore, it is desirable to use the background image as an object until the background image is generated by the background image generation unit 24.

Next, referring to FIG. 12, in the correction mode of FIG. 11, the control unit 34 controls the contour extraction unit 30 to extract a contour, controls the correction unit 31, and is stored in the object map storage unit 26. The operation for correcting the block identification code and motion vector will be described.

(Step S1201)
First, the occlusion detection unit 35 determines whether or not occlusion has occurred based on the object map stored in the object map storage unit 26.

(Step S1202)
If it is determined in step S1201 that no occlusion has occurred, the target area setting unit 301 sets a target area for a moving object without occlusion.

(Step S1203)
Next, the target area correction unit 302 generates the above-described histogram, and corrects the target area set by the target area setting unit 301 for each coordinate axis based on the generated histogram.

(Step S1204)
Next, the contour extraction processing unit 303 extracts the contour of the moving object from the time-series image for the target region corrected by the target region correction unit 302.

(Step S1205)
Next, the correction unit 31 corrects the block identification code and the motion vector stored in the object map storage unit 26 based on the contour extracted by the contour extraction unit 30.

(Step S1212)
On the other hand, if it is determined in step S1201 that occlusion has occurred, the target area setting unit 301 of the contour extraction unit 30 sets a plurality of moving objects in which occlusion has occurred as an integrated moving object.

(Step S1213)
Next, the target area setting unit 301 sets a target area for an integrated moving object.

(Step S1214)
Next, the target area correction unit 302 generates the above-described histogram for the integrated moving object, and corrects the target area set by the target area setting unit 301 for each coordinate axis based on the generated histogram.

(Step S1215)
Next, the contour extraction processing unit 303 extracts the contour of the moving object integrated with the target region corrected by the target region correction unit 302 from the image of the time series image.

(Step S1216)
Next, as described with reference to FIG. 4A to FIG. 7B, the correction unit 31 performs object detection based on the boundary of the moving object in which the detected occlusion occurs and the contour extracted by the contour extraction unit 30. The block identification code and motion vector stored in the map storage unit 26 are corrected.

After the processing described above is completed for all moving objects stored in the object map storage unit 26, the control unit 34 transitions from the correction mode to the camera variation mode.

As described above with reference to FIGS. 11 and 12, the moving object tracking device 20 can track a moving object even when the camera is fixed or fluctuates. In the camera fixing mode, since the background image is not registered in the object, it is not necessary to execute the processing as illustrated in FIGS. 4A to 7B on the object of the background image. The amount of processing or load can be reduced.

Hereinafter, the identification code and motion vector for each block stored in the object map storage unit 26 will be described as “space-time MRF (Markov Random Field)”. Next, the operation and results described with reference to FIG. 12 will be described with reference to FIGS. 13 to 30B.

<Snakes>
First, extraction of the contour of the moving object from the image by the contour extraction unit 30 will be described. Of the configuration of the contour extraction unit 30, first, an example of a technique for extracting the contour of a moving object by the contour extraction processing unit 303 will be described in detail. Here, the case where Snakes (refer nonpatent literature 1 below) is used as a technique which extracts an outline is demonstrated.

(Non-Patent Document 1) Kass et.al “Snakes: Active contour models”, Proc. Of 1st ICCV, pp.259-268, 1987

First, the outline of Snakes will be described. In general, Snakes expresses a spline (set of control points) v (s) = (x (s), y (s)) (0 ≦ s ≦ 1) expressed as a parameter on the image plane (x, y), This is a contour extraction model that is deformed so as to minimize the energy function defined by the following equation (4) and whose shape is determined as a minimum state of energy.

The first term Eint in this equation (4) is internal energy. Thus, the Snakes spline has a property of smoothly contracting into a convex shape. The theoretical definition is expressed by the following formula (5). The first term in the following formula (5) has the property that the spline becomes smooth in a convex shape, and the spline contracts by the second term.

Next, the second term Eimage of the above formula (4) is image energy. This image energy has a property that the value of the image energy becomes smaller as the ratio of the image energy existing on the edge (a portion such as a contour having a large luminance gradient) increases. This image energy is defined by the following equation (6) by the luminance I (v (s)) of the image. This time, in order to stably extract the contour edge regardless of the illuminance, the illuminance invariant filter image developed so far by the present inventors was used as the image energy.

Here, the illuminance invariant filter image is an image converted by the image conversion unit 22 according to the above formulas (1) to (3).

And the third term Econ in the equation (4) is external energy. This external energy is used when force is forcedly applied to Snakes from the outside. This energy is defined as needed. This time, an area term (see Non-Patent Document 2 below) proposed for extracting a concave contour that was difficult to extract due to the influence of Eint used for internal energy was defined as external energy. The area term Area is derived by the following equation (7) (see FIG. 13).

(Non-patent literature 2) Shoichi Araki, Naokazu Yokoya, Hidehiko Sasaiwa, Haruo Sasatakemura: “Dynamic contour model splitting by intersection judgment for the purpose of extracting multiple objects”, Journal of the Institute of Electronics, Information and Communication Engineers (D-II) Vol.J79-DII, No.10, pp1704-1711 (Oct, 1996)

14A to 14C show the processing results by Snakes. First, initial control points are arranged around the object whose contour is to be extracted (FIG. 14A). Next, Snakes begins to contract (FIG. 14B). Finally, the contraction stops near the contour line (FIG. 14C).

Here, the initial control points shown in FIG. 14A correspond to the target area set by the target area setting unit 301.

<Analysis of edge distribution in local region>
Next, analysis of edge distribution in the local region will be described. Snakes is an algorithm that finishes the search when the spline is deformed to minimize the energy function _{E_snakes} and _reaches a minimum state. If there are many background edges and the initial control points are placed away from the contour, the spline will be caught by the background edges before converging on the contour of the object, and the energy will be minimal. As a result, the outline extraction of the object may fail.

Therefore, some initial control points must be placed near the contour of the object. On the other hand, in the local region of the person obtained by the spatiotemporal MRF, the boundary between the background object and the person object may become ambiguous during panning.

Therefore, the edge distribution (binary distribution of the illuminance invariant filter image) in the surrounding rectangle of the human object obtained by the object map is analyzed, and the human area is estimated. Thereby, the accuracy of contour extraction is improved by arranging the initial control points around it. The analysis of the edge distribution is performed by projecting onto the horizontal axis and the vertical axis and generating a histogram (see FIGS. 15A to 15C).

Next, an example of an edge distribution analysis procedure will be described. The edge distribution analysis is executed by the following steps STEP1 to STEP3.

(STEP 1: Preprocessing of edge distribution)
The edge image is labeled, and a small area is excluded as noise.

(STEP 2: Horizontal axis histogram analysis)
A horizontal axis histogram is generated from the edge distribution obtained in STEP 1. Since a person has long vertical edges, after removing edges with low vertical continuity, the person is orthogonally projected once to narrow down the horizontal area of the person from a strong vertical distribution.

After that, within the narrowed area, the obtained edge distribution is projected on the horizontal axis to generate a histogram. Then, the window is scanned in the obtained horizontal axis histogram. Thus, by obtaining a continuously distributed region, the distribution region of the human edge in the horizontal component is estimated (FIG. 15A).

(STEP3: Vertical axis histogram analysis)
A vertical axis histogram is generated from the edge distribution obtained in STEP 1. Since a person has a continuous edge that is somewhat long in the horizontal direction, after removing edges with low continuity in the horizontal direction, the person is projected once onto the vertical axis, and the vertical region of the person is narrowed down from the distribution in the horizontal direction.

After that, within the narrowed area, the edge distribution is projected onto the vertical axis, and a histogram is generated. Then, the window is scanned in the obtained horizontal axis histogram. Thus, the distribution area of the person edge in the horizontal component is estimated by obtaining the continuously distributed area (FIG. 15B).

In STEP 3, a vertical axis histogram is generated from the edge distribution obtained in STEP 1. Alternatively, the vertical axis histogram may be generated from the edge distribution in the local region narrowed in STEP2.

By referring to the edge distribution information through the above steps, a more accurate circumscribed rectangular region of the person can be obtained (FIG. 15C).

Note that the histogram threshold (initial control point) may be set as follows.
First, the histogram frequency values are clustered into two groups. The clustering method may be any method such as a k-mean method (one-dimensional). Thereby, the frequency value of a histogram is divided into a high frequency cluster and a low frequency cluster.

Next, when searching from the both ends of the image to the inside and hitting the frequency value belonging to the high-frequency cluster for the first time, the boundary between the previous low-frequency position and this high-frequency position is set as a boundary. In this case, there may be a low frequency inside the high frequency.

<Inter-layer cooperation algorithm>
Here, a processing step of tracking by cooperation between layers of the spatio-temporal MRF and Snakes will be described. Hereinafter, correcting the object map using the spatio-temporal MRF and Snakes will be referred to as inter-layer cooperation.

First, in the case of object map correction (without occlusion) by Snakes, that is, processing corresponding to steps S1202 to S1205 in FIG. 12 will be described.

(STEP1)
As an output of the spatiotemporal MRF, an object map is received, and information on a circumscribed rectangular area in each object is obtained.

(STEP2)
For each object, edge distribution analysis is performed in the local region obtained in STEP 2, and Snakes initial control points are arranged around the contour of the object.

(STEP3)
Execute Snakes on each object. Compared to the size of the circumscribed rectangle obtained in STEP 1, the object map is not corrected for an object whose spline has contracted too much. For other objects, the processing result of Snakes is reflected and the object map is corrected.

Next, in the case of object map correction by Snakes (with occlusion), that is, processing corresponding to steps S1212 to S1215 in FIG. 12 will be described.

(STEP1)
As an output of the spatio-temporal MRF, an object map is received, and information on a circumscribed rectangular area in each object is obtained. For objects in which occlusion is detected (having overlapping areas with circumscribed rectangles of other objects), the circumscribed rectangular area is obtained with the objects that are occluded as one group.

(STEP2)
Edge distribution analysis is performed in the local region obtained in STEP 2, and Snakes initial control points are arranged around the contour of the object.

(STEP3)
Execute Snakes. Compared to the size of the circumscribed rectangle obtained in STEP 1, the object map is not corrected for an object whose spline has contracted too much. For other objects, the processing result of Snakes is reflected and the object map is corrected. At that time, in the labeling of the ID number of each object in the inner region surrounded by the contour extracted by Snakes, the labeling is performed based on the output result of the spatiotemporal MRF model. However, the label is changed from the background object to indefinite for the block recognized as the background object in this contour inner region. Thereby, ID allocation by the spatio-temporal MRF model is performed in the next frame. Or, in the current frame, only this block can be assigned ID again by the spatio-temporal MRF model.

FIG. 16A to FIG. 17B show examples of object map correction by Snakes. 16A and 16B are examples in the case where there is no occlusion, and FIGS. 17A and 17B are examples in the case of occlusion.

17A and 17B, first, the persons with ID numbers 6 and 7 are grouped together, and the outline of the group (the boundary between the background and the person) is obtained by Snakes. The region division within the group reflects the output information by the spatiotemporal MRF.

<Correction of object map by Snakes>
Next, with reference to FIGS. 18A to 29B, effects of the inter-layer cooperation algorithm when the camera 10 changes will be described. Here, as shown in FIGS. 18A to 29B, the processing results of moving object detection in the case of no inter-layer cooperation and inter-layer cooperation for the same frame of the same scene will be described.

FIG. 18A to FIG. 23A show the processing results when inter-layer cooperation is not performed, and FIG. 18B to FIG. 23B are object maps thereof. FIG. 24A to FIG. 29A show the processing results when inter-layer cooperation is performed, and FIG. 24B to FIG. 29B are object maps thereof.

カ メ ラ Camera panning starts immediately after frame number 80. When the inter-layer cooperation algorithm is not performed, although the tracking of more than 20 frames has been successful, the boundary between the person object and the background object has gradually become ambiguous (frame 95 and frame 107).

On the other hand, when coordinated by Snakes, the boundary between objects can be corrected by referring to the output of the spatio-temporal MRF, and tracking can be performed for a long time. In addition, even when zooming, there is an effect of the cooperation algorithm between layers (the right two columns of the frame 142 and the frame 153).

FIG. 30A and FIG. 30B are results of detecting a moving object when the camera 10 fluctuates when occlusion occurs. In FIG. 30A, no occlusion has occurred. Thereafter, in FIG. 30B, occlusion has occurred. In any case, the moving object tracking device 20 according to the present embodiment can track the moving object.

As described above, the moving object tracking device 20 according to the present embodiment changes the background by correcting the block identification code and the motion vector stored in the object map storage unit based on the extracted contour. Even from an image, a moving object in the image can be accurately detected.

In the description of the above embodiment, the motion vector is corrected together with the block identification code stored in the object map storage unit. However, only the identification code may be corrected. Even in this way, similarly, a moving object in an image can be accurately detected from an image whose background changes.

The moving object fluctuation amount detection unit 33 described above detects the size of the moving object or the fluctuation amount of the movement amount stored in the object map storage unit 26 based on the identification code or the motion vector for each unit time.

Then, the control unit 34 (second control unit) is configured such that the size of the moving object or the amount of movement of the moving object per unit time detected by the moving object fluctuation amount detection unit 33 is a predetermined size or movement amount. Is larger than the fluctuation amount, the contour extraction unit 30 is controlled to extract the contour, and the correction unit 31 is controlled to correct the block identification code and motion vector stored in the object map storage unit 26.

As described above, the block identification code and the motion vector stored in the object map storage unit 26 may be corrected in accordance with the size of the moving object stored in the object map storage unit 26 or the fluctuation amount of the movement amount. Good.

As a result, the control unit 34 compares the object map at a timing at which it is likely to fail to detect a moving object, as compared with the case where the control unit 34 simply corrects the object map every predetermined period or every predetermined frame. Can be corrected. Therefore, a moving object can be detected and tracked more accurately.

Note that the storage unit such as the frame buffer memory 3, the image memory 21, or the object map storage unit 26 in FIG. 2 is a non-volatile memory such as a hard disk device, a magneto-optical disk device, a flash memory, or a CD-ROM. May be configured by a storage medium that can only be stored in the memory, a volatile memory such as a RAM (Random Access Memory), or a combination thereof.

2, the image conversion unit 22, the background generation unit 24, the ID generation / annihilation unit 25, the moving object tracking unit 27, the contour extraction unit 30, the correction unit 31, the determination unit 32, the moving object fluctuation amount detection unit 33, The processing unit called the control unit 34 or the occlusion detection unit 35 may be realized by dedicated hardware. The processing unit may be configured by a memory and a CPU (central processing unit), and the function may be realized by loading a program for realizing the function of the processing unit into the memory and executing the program.

Further, in FIG. 1, the image conversion unit 22, the background generation unit 24, the ID generation / annihilation unit 25, the moving object tracking unit 27, the contour extraction unit 30, the correction unit 31, the determination unit 32, the moving object variation amount detection unit 33, By recording a program for realizing the function of the processing unit such as the control unit 34 or the occlusion detection unit 35 on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium The processing by this processing unit may be executed. The “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” is a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in a computer system.
Furthermore, “computer-readable recording medium” is a program that dynamically holds a program for a short time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system that serves as a server or a client in this case includes a program that holds a program for a certain period of time.
Further, the program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. .

As mentioned above, although one embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment alone, and includes a design and the like within a scope not departing from the gist of the present invention. It is.

According to the present invention, by correcting the block identification code stored in the object map storage unit based on the extracted contour of the moving object, the moving object in the image can be detected even when the background of the image fluctuates. It can be detected accurately.

DESCRIPTION OF SYMBOLS 10 Camera 20 Moving object tracking device 21 Image memory 22 Image conversion part 23 Frame buffer memory 24 Background image generation part 25 ID production | generation / extinction part 26 Object map memory | storage part 27 Moving object tracking part 30 Contour extraction part 31 Correction | amendment part 32 Determination part 33 Moving object variation detection unit 34 Control unit 35 Occlusion detection unit 301 Target region setting unit 302 Target region correction unit 303 Contour extraction processing unit

Claims (10)

1. A moving object tracking device for detecting a moving object in a time-series image by image processing,
   An object map storage unit for storing an identification code corresponding to the moving object by assigning and storing the identification code corresponding to a block obtained by dividing each frame of the time-series image into a plurality of blocks;
   A contour extraction unit that extracts a contour of the moving object from the time-series image;
   A correction unit for correcting the identification code assigned to each block based on the contour;
   A moving object tracking device comprising:
2. The contour extraction unit
   A target area setting unit that sets a target area for extracting an outline of the moving object based on an image area corresponding to the moving object stored in the object map storage unit;
   A contour extraction processing unit that extracts a contour of the moving object from the time-series image with respect to the target region;
   The moving object tracking device according to claim 1, comprising:
3. The contour extraction unit generates and generates a histogram for the number of pixels corresponding to an edge in the target region for an image obtained by performing edge extraction processing on the time-series image, based on the histogram. A target area correction unit that corrects the target area for each coordinate axis;
   The contour extraction processing unit extracts the contour of the moving object from the time-series image with respect to the corrected target region;
   The moving object tracking device according to claim 2.
4. The movement according to claim 1, wherein the contour extraction unit extracts a contour of the integral moving object from the time-series image by using a plurality of moving objects in which occlusion has occurred as an integral moving object. Object tracking device.
5. The correction unit is
   Information indicating boundaries of the plurality of moving objects based on the identification codes corresponding to the plurality of moving objects in which the occlusion occurs, which is stored in the object map storage unit, and the occlusion is generated by the contour extraction unit Based on the contour of the moving object extracted as a plurality of moving objects that are integrated,
   The moving object tracking device according to claim 4, wherein the identification code stored in the object map storage unit is corrected.
6. A determination unit for determining whether the background image is fluctuating;
   First, when the determination unit determines that the background image is fluctuating, the contour extraction unit extracts a contour, and the correction unit corrects the identification code stored in the object map storage unit. A control unit;
   The moving object tracking device according to claim 1, further comprising:
7. A moving object fluctuation amount detection unit that detects a fluctuation amount of the size or movement amount of the moving object stored in the object map storage unit per unit time based on the identification code;
   If the moving object size or moving amount per unit time detected by the moving object changing amount detection unit is larger than a predetermined size or moving amount variation, a contour is added to the contour extracting unit. A second control unit that causes the correction unit to extract and correct the identification code stored in the object map storage unit;
   The moving object tracking device according to claim 1, further comprising:
8. A moving object tracking unit that updates the identification code and a motion vector of the moving object stored in the object map storage unit based on a result of image processing of the time-series image;
   The moving object tracking unit is
   For each successive N images (N ≧ 2) of the time-series images, the same identification code is attached to the blocks whose absolute values of motion vector differences between adjacent blocks are within a predetermined value. An identification code providing step of attaching different identification codes to the moving objects that overlap each other;
   In each of the N images, a first object that is a block group to which a first identification code is attached is in contact with a second object that is a block group to which a second identification code is attached, and the N image is temporally related. A determination step of determining whether or not the degree of correlation between the first objects of images adjacent to each other is equal to or greater than a predetermined value;
   A tracking step of tracking the first object and the second object retroactively when it is determined affirmative in the determination step;
   An update step of updating the identification code and the motion vector stored in the object map storage unit based on the first object and the second object traced back in time by the tracking step;
   The moving object tracking device according to claim 1, wherein:
9. A moving object tracking method for detecting a moving object in a time-series image by image processing,
   Assigning and storing an identification code corresponding to the moving object to a block obtained by dividing each frame of the time-series image into a plurality of blocks;
   Extracting the contour of the moving object from the time-series image;
   Correcting the identification code assigned to the block based on the contour;
   A moving object tracking method comprising:
10. To a computer as a moving object tracking device that detects moving objects in time-series images by image processing,
    Assigning and storing an identification code corresponding to the moving object to a block obtained by dividing each frame of the time-series image into a plurality of blocks;
    Extracting the contour of the moving object from the time-series image;
    Correcting the identification code assigned to the block based on the contour;
    Moving object tracking program to execute.

Images