JP2010244207A - Moving object tracking device, moving object tracking method, and moving object tracking program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving object tracking device which detects a moving object in an image accurately, even in an image whose background changes. <P>SOLUTION: A moving object tracking device, which is a device for tracking a moving object in an image, detects the moving object in the image by applying image processing to time series images, and includes an object map storage unit, a contour extraction unit, and a correction unit. The object map storage unit allows each image in the time series images to be divided into a plurality of blocks, attaches an identification code indicating the moving object in the image to a block corresponding to the moving object, and stores the identification code. The contour extraction unit extracts the contour of the moving object from the time series images. The correction unit corrects the identification code of the block stored in the object map storage unit based on the contour extracted by the contour extraction unit. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a moving object tracking device, a moving object tracking method, and a moving object tracking program that process time-series images and track moving objects (movable objects such as cars, bicycles, and animals) in the images.

  In recent years, there has been a demand for a technique for accurately detecting a moving object in an image by processing an image captured by a camera. For example, early detection of traffic accidents can not only increase the success rate of lifesaving through quick rescue activities, but also reduce traffic jams by speeding up on-site inspection by the police. Recognition automation is expected. Thus, in order to increase the recognition rate of traffic accidents, it is necessary to accurately detect a moving object by performing image processing on an image captured by a camera.

  FIG. 20 schematically shows an image at time t = 1 to 4 captured by a camera installed above the center line of the expressway.

  Since vehicles frequently overlap each other on an image, it is difficult to track each vehicle by image processing. In order to solve this problem, it is necessary to install a plurality of cameras along the road and comprehensively process these images.

  However, since it is necessary to provide a plurality of cameras and image processing apparatuses, the cost increases. In addition, the image processing becomes complicated because the captured images of the respective cameras must be related to each other and comprehensively processed.

  Therefore, the inventors of the present application solved this problem by detecting the moving object by going back in time as follows (see Patent Document 1 and Patent Document 2).

  For example, time-series images at time t = 1 to 4 are temporarily stored, vehicles M1 and M2 are identified starting from time t = 4, and motion vectors of the vehicles M1 and M2 are obtained. Assuming an image at time t = 3 in which the vehicles M1 and M2 are identified by moving the vehicles M1 and M2 in the image at t = 4, from the correlation between this and the actual image at time t = 3 The vehicles M1 and M2 in the image at time t = 3 are identified.

  Next, vehicles M1 and M2 in the image at time t = 2 are identified by similar image processing for the images at time t = 3 and time t = 2. Next, vehicles M1 and M2 in the image at time t = 1 are identified by similar image processing for the images at time t = 2 and time t = 1.

  Such image processing makes it possible to track the vehicles M1 and M2 with a single camera.

JP 2002-133421 A Japanese Patent Laid-Open No. 2004-207786

  In such a conventional technique, an image picked up with a camera fixed, that is, an image with a fixed background is subjected to image processing to accurately detect a moving object in the image.

  On the other hand, when the camera is panned or zoomed, the background image that has been fixed so far also changes according to the panning or zooming of the camera. When image processing is performed on an image whose background changes in this way, there is a problem that the boundary between the image area of the moving object and the background image is not clear, and the moving object in the image cannot be accurately detected.

  The present invention has been made in view of such circumstances, and an object thereof is a moving object tracking device and a moving object tracking method capable of accurately detecting a moving object in an image even from an image with a changing background. And providing a moving object tracking program.

  The present invention has been made to solve the above-described problems, and the invention according to claim 1 is an on-image moving object tracking device that detects a moving object in an image by processing a time-series image. Each of the time-series images is divided into a plurality of blocks, and an identification code indicating a moving object in the image is attached to the block corresponding to the moving object and stored therein, and the object map storage unit, A contour extracting unit that extracts the contour of the moving object from a time-series image, and a correction that corrects the block identification code stored in the object map storage unit based on the contour extracted by the contour extracting unit. And a moving object tracking device.

  According to a second aspect of the present invention, the contour extracting unit extracts a target region from which the contour of the moving object is extracted based on an image region corresponding to the block of the moving object stored in the object map storage unit. A target region setting unit to be set; and a contour extraction processing unit that extracts a contour of the moving object from the image of the time-series image with respect to the target region set by the target region setting unit. A moving object tracking device according to claim 1.

  According to a third aspect of the present invention, the contour extraction unit has the number of pixels corresponding to an edge in the target region set by the target region setting unit with respect to an image obtained by performing edge extraction processing on the time-series image. A target area correction unit that generates a histogram for the number of projections for each coordinate axis and corrects the target area set by the target area setting unit based on the histogram generated for each coordinate axis, for each coordinate axis, The moving object according to claim 2, wherein the contour extraction processing unit extracts a contour of the moving object from the image of the time-series image with respect to the target region corrected by the target region correction unit. A tracking device.

  According to a fourth aspect of the present invention, the contour extracting unit extracts a plurality of moving objects in which occlusion occurs as an integral moving object, and extracts the contour of the integral moving object from the images of the time-series images. The moving object tracking device according to claim 1, wherein the moving object tracking device is a moving object tracking device according to claim 1.

  According to a fifth aspect of the present invention, the correction unit includes a boundary between the plurality of moving objects based on identification codes corresponding to the plurality of moving objects in which the occlusion occurs, which is stored in the object map storage unit. Of the block stored in the object map storage unit based on the information indicating the contour of the moving object extracted by integrating the plurality of moving objects in which the occlusion is generated by the contour extraction unit. 5. The moving object tracking device according to claim 4, wherein the identification code is corrected.

  According to a sixth aspect of the present invention, when the determination unit determines whether or not the background image is fluctuating, and the determination unit determines that the background image is fluctuating, the contour extraction unit is controlled. A first control unit that extracts a contour and controls the correction unit to correct the identification code of the block stored in the object map storage unit. 5. The moving object tracking device according to 5.

  According to a seventh aspect of the present invention, there is provided a moving object fluctuation amount detection unit that detects a fluctuation amount of the size or movement amount of the moving object stored in the object map storage unit per unit time based on the identification code. The contour extraction is performed when the moving object size or moving amount variation amount per unit time detected by the moving object variation detecting unit is larger than a predetermined size or moving amount variation amount. And a second control unit that controls the unit to extract a contour and controls the correction unit to correct the identification code of the block stored in the object map storage unit. The moving object tracking device according to any one of claims 1 to 6.

  In the invention according to claim 8, in the object map storage unit, the identification code is attached to the block corresponding to the moving object, and the motion vector of the moving object corresponding to the block is the block. And a moving object tracking unit that updates a block identification code and a motion vector stored in the object map storage unit based on the result of image processing of the time-series image. The moving object tracking unit, for each of the consecutive N images (N ≧ 2) of the time-series images, has the same identification code as the block whose absolute value of the motion vector difference between adjacent blocks is within a predetermined value. In each of the N images, an identification code procedure for attaching different identification codes to moving objects that overlap each other on the image, A first object that is a group of blocks to which another code is attached and a second object that is a group of blocks to which a second identification code is attached, and between the first objects of images that are temporally adjacent to the N images A determination procedure for determining whether or not the degree of correlation is greater than or equal to a predetermined value, and a tracking procedure for tracking the first object and the second object retroactively after it is determined affirmative in the determination procedure; An update procedure for updating a block identification code and a motion vector stored in the object map storage unit based on the first object and the second object tracked back in time by the tracking procedure; The moving object tracking device according to claim 1, wherein:

  The invention according to claim 9 is an on-image moving object tracking method used in an on-image moving object tracking device that performs time-series image processing to detect a moving object in an image, wherein the contour extraction unit includes the contour extracting unit, The contour extraction procedure for extracting the contour of the moving object from the image of the time-series image, and the correction unit divides each image of the time-series image into a plurality of blocks, and the identification code indicating the moving object in the image is A correction procedure for correcting the block identification code stored in the object map storage unit attached to the block corresponding to the moving object based on the contour extracted by the contour extraction unit. This is a moving object tracking method.

  According to a tenth aspect of the present invention, in a computer as an on-image moving object tracking device that performs image processing on a time-series image and detects a moving object in the image, a contour extraction unit includes the movement from the image of the time-series image. A contour extraction procedure for extracting the contour of the object is executed, and the correction unit divides each image of the time-series image into a plurality of blocks, and an identification code indicating the moving object in the image corresponds to the moving object. Movement for causing the block identification code stored in the object map storage unit attached to the block to be corrected based on the contour extracted by the contour extraction unit It is an object tracking program.

  According to the present invention, by correcting the block identification code stored in the object map storage unit based on the extracted contour, it is possible to accurately detect a moving object in the image even from an image whose background changes. There is an effect that can be done.

It is a block diagram which shows the outline of the moving object tracking system using the intersection and the moving object tracking apparatus of this invention arrange | positioned at this. It is a block diagram which shows the structure as an example of the moving object tracking apparatus of FIG. It is explanatory drawing which shows ID of the moving object provided to the slit and block which were each set to the four entrances to an intersection and the four exits from an intersection in a frame image. (A) And (B) is a figure which shows typically the image of the time t-1 and t with a block boundary line, respectively. (A) And (B) is a figure which shows typically the image of the time t-1 and t with a pixel boundary line, respectively. (A) And (B) is a figure which shows typically the image of the time t-1 and t with the motion vector provided to the block, respectively. (A) and (B) are diagrams schematically showing motion vectors and object boundaries assigned to the object maps at times t-1 and t, respectively. It is a flowchart which shows the estimation method of an undetermined motion vector. (A) And (B) is a figure which shows typically the motion vector and object boundary provided to the object map for demonstrating the process of FIG. (A)-(C) is a figure which shows typically the motion vector and object boundary provided to the object map for demonstrating the process of FIG. FIG. 6 is a state transition diagram showing state transition of modes in the moving object tracking device 20. It is a flowchart figure which shows operation | movement of the moving object tracking device 20 in the correction mode of FIG. It is explanatory drawing which shows the area term Earea. It is explanatory drawing which shows the Snakes process result as an example. It is explanatory drawing which shows the generation of edge distribution and a histogram as an example. It is explanatory drawing which shows the example of correction of the object map by Snakes. It is a result figure which shows the result of a moving object detection in case the camera 10 fluctuates, when there is no cooperation algorithm between layers. It is a result figure which shows the result of a moving object detection in case the camera 10 fluctuates, when there exists a cooperation algorithm between layers. It is a result figure which shows the result of a moving object detection in case the occlusion has generate | occur | produced when the camera 10 fluctuates. It is a figure which shows typically the image of the time t = 1-4 imaged with the camera installed above the center line of the highway.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing a configuration of a moving object tracking system using a moving object tracking device 20 according to an embodiment of the present invention. As shown in FIG. 1 as an example, this moving object tracking system includes an electronic camera 10 that captures an intersection and outputs an image signal, and a moving object tracking device 20 that processes the image and tracks the moving object. ing.

  A time-series image captured by the electronic camera 10 is stored in an image memory 21 (described later) of the moving object tracking device 20 at a rate of 12 frames / second, for example, and the oldest frame is rewritten with a new frame image. The electronic camera 10 can be panned or zoomed to change the image area to be photographed. The panning or zooming for the electronic camera 10 may be controlled by, for example, the moving object tracking device 20 or may be controlled by a host control device that controls the moving object tracking system.

  The moving object tracking device 20 performs image processing on a time-series image (a time-series image stored in an image memory 21 to be described later) taken by the electronic camera 10 and performs image processing on the time-series image to move the moving object in the image. Is detected.

  Next, a configuration as an example of the moving object tracking device 20 will be described with reference to FIG.

  The image conversion unit 22 copies each frame image in the image memory 21 to the frame buffer memory 23, and uses the copied image data to convert the corresponding frame image in the image memory 21 into a spatial difference frame image. Convert to This conversion is performed in two stages.

  If the pixel value (luminance value) of the i-th row and j-th column of the original frame image is G (i, j), the pixel value H (i, j) of the i-th row and j-th column after the conversion in the first stage. Is expressed by the following equation.

  H (i, j) = Σneighberpixcels | G (i + di, j + dj) −G (i, j) | (Expression 1)

  Here, Σneighberpixcels means the sum of di = −c to c and dj = −c to c, where c is a natural number. For example, when c = 1, 8 adjacent to the pixel in the i-th row and j-th column. The sum over the pixels. When the illuminance changes, the pixel value G (i, j) and the neighboring pixel value G (i + di, j + dj) change in the same manner, so the image of H (i, j) is invariant to the change in illuminance. .

  Here, the absolute value of the difference between adjacent pixels is generally larger as the pixel value is larger. In order to increase the success rate of moving object tracking, it is preferable to acquire edge information almost equivalently to the case where the pixel value and the difference are large even when the pixel value is small and the difference is small. Therefore, H (i, j) is normalized as follows.

  H (i, j) = Σneighberpixcels | G (i + di, j + dj) −G (i, j) | / (Gi, j, max / Gmax) (Expression 2)

  Here, Gi, j, max is the maximum value of the original pixel value used in the calculation of H (i, j). For example, when c = 1, the pixel in the i-th row and j-th column is the center. Gmax is the maximum value that can be taken by the pixel value G (i, j), for example, 255 when the pixel value is represented by 8 bits. Hereinafter, a case where c = 1 and Gmax = 255 will be described.

  The maximum value that H (i, j) can take differs for each moving object. For example, if G (i, j) = Gmax and the values of 8 pixels adjacent to the pixel in the i-th row and j-th column are all 0, H (i, j) = 8Gmax and H (i, j ) Cannot be represented by 8 bits.

  On the other hand, when a histogram of the value of H (i, j) at the edge portion of the moving object was created, it was found that most of the frequency was included in the range of H = 50 to 110. That is, since the number of edge information for tracking a moving object is smaller as the value of H is larger than about 110, the importance is lower.

  Therefore, it is preferable to perform image processing at high speed by suppressing the portion with a large value of H and shortening the bit length of the converted pixel. Therefore, as a second stage, this H (i, j) is converted to I (i, j) by the following equation using a sigmoid function.

  I = Gmax / {1 + exp [−β (H−α)]} (Equation 3)

  The sigmoid function has good linearity in the vicinity of H = α. Therefore, the threshold value α is set to the most frequent value of the frequency distribution of H having edge information, for example, 80.

  Based on the above equations (2) and (3), the image conversion unit 22 converts the image having the pixel value G (i, j) into a spatial difference frame image having the pixel value I (i, j). Store in the image memory 21.

  The background image generation unit 24, the ID generation / annihilation unit 25, and the moving object tracking unit 27 perform processing based on the spatial difference frame image in the image memory 21. Hereinafter, the spatial difference frame image is simply referred to as a frame image.

  The background image generation unit 24 includes a storage unit and a processing unit. The processing unit accesses the image memory 21 and creates a histogram of pixel values for pixels corresponding to all frame images for the past 10 minutes, for example. An image having the mode value (mode) as the pixel value of the pixel is generated as a background image in which no moving object exists, and is stored in the storage unit. The background image is updated by performing this process periodically.

  As shown in FIG. 3, the ID generation / annihilation unit 25 has positions and sizes of slits EN1 to EN4 and EX1 to EX4 arranged at four entrances to the intersection and four exits from the intersection, respectively, in the frame image. Data is preset. The ID generation / annihilation unit 25 reads the image data in the entrance slits EN1 to EN4 from the image memory 21, and determines in block units whether or not a moving object exists in these entrance slits. The meshes in FIG. 3 are blocks, and one block is, for example, 8 × 8 pixels. When one frame is 480 × 640 pixels, one frame is divided into 60 × 80 blocks. Whether there is a moving object in a certain block is determined by whether the sum of absolute values of differences between the pixels in the block and the corresponding pixels in the background image is equal to or greater than a predetermined value. This determination is also performed in the moving object tracking unit 27.

  If the ID generation / annihilation unit 25 determines that there is a moving object in the block, the ID generation / annihilation unit 25 assigns a new object identification code (ID) to the block. If the ID generation / annihilation unit 25 determines that a moving object is present in a block adjacent to the ID-assigned block, the ID generation / annihilation unit 25 assigns the same ID as the assigned block to this adjacent block. This ID-added block includes a block adjacent to the entrance slit. For example, ID = 1 is assigned to the block in the entrance slit EN1 in FIG.

  The ID is assigned to the corresponding block in the object map storage unit 26. The object map storage unit 26 is for storing an object map of 60 × 80 blocks in the above-described case, and each block has a flag indicating whether or not an ID is assigned. The number and a block motion vector described later are given as block information. Instead of using the flag, it may be determined that no ID is assigned when ID = 0. The most significant bit of the ID may be used as a flag.

  For the cluster that has passed through the entrance slit, the moving object tracking unit 27 assigns an ID to the block in the moving direction and extinguishes the ID of the block in the direction opposite to the movement, that is, a cluster tracking process. The tracking processing by the moving object tracking unit 27 is performed up to the inside of the exit slit for each cluster.

  The ID generation / annihilation unit 25 further checks whether an ID is assigned to the blocks in the exit slits EX1 to EX4 based on the contents of the object map storage unit 26, and if so, the cluster has passed through the exit slit. Sometimes the ID disappears. For example, when the block in the exit slit EX1 in FIG. 3 changes from a state where ID = 3 is assigned to a block where no ID is assigned, ID = 3 is extinguished. The disappearance ID can be used as the next generation ID.

  The moving object tracking unit 27 is based on the object map at time (t−1) stored in the object map storage unit 26 and the frame image at time (t−1) and t stored in the image memory 21. Thus, an object map at time t is created in the object map storage unit 26. This will be described below.

  4 to 7 schematically show images at times t-1 and t. The dotted lines in FIGS. 4, 6 and 7 are block boundaries, and the dotted lines in FIG. 5 are pixel boundaries.

  The block in the i-th row and the j-th column is denoted as B (i, j), and the block in the i-th row and the j-th column at the time t is denoted as B (t: i, j). It is assumed that the motion vector of the block B (t−1: 1, 4) is MV. Find the block at time t that most corresponds to the area where block B (t−1: 1, 4) has been moved by MV. In the case of FIG. 4B, this block is B (t: 1, 5). As shown in FIG. 5, the correlation between the image of the block B (t: 1, 5) and the image of the block size area AX at time t−1 is moved by one pixel within the predetermined range AM. Calculate every block (block matching).

  The range AM is larger than the block, and one side thereof is, for example, 1.5 times the number of pixels on one side of the block. The center of the range AM is a pixel at a position obtained by moving the center of the block B (t: 1, 5) by approximately −MV.

  The correlation degree is, for example, a spatio-temporal texture correlation degree, and it is assumed that the smaller the evaluation value UD, which is the sum of absolute values of the difference between corresponding pixel values in the block B (t: 1, 5) and the region AX, is larger.

  An area AX having the maximum degree of correlation within the range AM is obtained, and a vector starting from the center of this area and ending at the center of the block B (1, 5) is defined as a motion vector of the block B (t: 1, 5). And decide. Further, the ID of the block at time t−1 that is closest to the region AX where the degree of correlation is maximum is determined as the ID of the block B (t: 1, 5).

  The moving object tracking unit 27 assigns the same ID to blocks whose absolute value of the difference between the motion vectors of adjacent blocks is a predetermined value or less. Thereby, even one cluster is divided into a plurality of objects (moving objects) having different IDs. In FIG. 6, the boundary between objects is indicated by a bold line.

  Although an image of a moving object does not exist on the object map, in FIG. 6, the moving object is schematically drawn on the object map for easy understanding. FIG. 7 shows the boundary of an object in the object map with a thick line, and corresponds to FIG.

  For example, when one cluster is detected at the entrance slit EN1 of FIG. 3 and is not divided into a plurality of objects, and then divided into a plurality of objects as described above at time t1, the time goes back from time t1, By obtaining the object map in the same way as when the time is in the positive direction, the object map before time t1 is divided into a plurality of objects. Thereby, an object that could not be divided can be divided and recognized, and individual objects can be tracked.

  In Patent Document 1, individual objects are traced back in time after one cluster is separated into a plurality of clusters. However, according to the present embodiment, before separation into a plurality of clusters, for example, FIG. Since t = 2 before t = 4, each object can be traced back in time, so that the storage capacity of the image memory 21 can be reduced, and the image processing amount can be reduced to reduce the burden on the CPU. Can be lightened.

  In the above description, the case where the motion vector of the block in the cluster is obtained has been described. However, as shown in FIG. 9A, when there is a block for which the motion vector cannot be obtained, which block depends on the position. It is unknown whether it belongs to any object. When the color of each pixel in a block belonging to a certain moving object is almost the same, the motion vector cannot be determined by the block matching described above. For example, if an image (spatial difference frame image) is converted into a binary image, and the number of “1” pixels in the block is equal to or less than a predetermined value, the block is determined to be unsuitable for obtaining a motion vector by the above method. To do.

  The motion vector of such a block is estimated by the method shown in FIG.

  (S1) If there is an undetermined motion vector, the process proceeds to step S2, and if not, the undetermined motion vector estimation process is terminated.

  (S2) The determined motion vectors MV1 to MVn are extracted from the eight blocks around the block B (i, j) whose motion vector is undetermined.

  (S3) If there is a motion vector determined in step S2, the process proceeds to step S4, and if not, the process proceeds to step S6.

  (S4) The motion vectors MV1 to MVn are divided into groups in which the absolute value of the difference between the vectors is within a predetermined value.

  (S5)) The average value of the motion vectors of the group having the largest number of motion vectors is estimated as the motion vector of the block B (i, j). When there are a plurality of groups having the largest number of motion vectors, the average value of the motion vectors of any one group is estimated as the motion vector of the block B (i, j). Next, the process returns to step S1.

  Since the motion vectors of the same group are substantially equal to each other, any one of the motion vectors of the same group may be estimated as the motion vector of the block B (i, j).

  (S6) The motion vector estimated in step S5 is regarded as the determined motion vector, and the process returns to step S1.

  By such processing, the undetermined motion vector can be estimated uniquely.

  Next, a specific example will be described. In FIG. 9A, the motion vector of the block B (i, j) in the i-th row and j-th column is denoted as MV (i, j). In FIG. 9A, the motion vectors of the blocks B (2,2), B (2,4) and B (3,3) are undetermined.

The motion vectors of the blocks around the block B (2, 2) are the group of MV (2, 1), MV (3, 1), MV (3, 2) and MV (2, 3), and MV (1 , 2) and MV (1, 3), so select the former group,
MV (2,2) = (MV (2,1) + MV (3,1) + MV (3,2) + MV (2,3)) / 4
Estimated.

The motion vectors of the blocks around the block B (2, 4) are a group of MV (2, 3), MV (3, 4) and MV (3, 5), and MV (1, 3), MV (1 4), MV (1,5) and MV (2,5), so select the latter group,
MV (2,4) = (MV (1,3) + MV (1,4) + MV (1,5) + MV (2,5)) / 4
Estimated.

The block motion vectors around block B (3, 3) are MV (2, 3), MV (3, 2), MV (4, 2), MV (4, 4) and MV (3,4). Because it is one group of
MV (3,3) = (MV (2,3) + MV (3,2) + MV (4,2) + MV (4,4) + MV (3,4))) / 5
Estimated.

  In this way, an object map as shown in FIG. 9B is generated. In FIG. 9B, the boundary of the object is indicated by a bold line.

  Even when the number of undetermined motion vectors is large as shown in FIG. 10A, when steps S1 to S5 are repeated until a negative determination is made in step S3, motion vectors are uniquely estimated and FIG. B). Next, in step S6, the estimated motion vector is regarded as the determined motion vector, and by executing steps S1 to S5 again, the motion vector of the block B (3,4) is uniquely estimated. 10 (C). Next, by assigning the same ID to blocks whose absolute values of motion vector differences between adjacent blocks are equal to or less than a predetermined value, one cluster is divided into a plurality of objects having different IDs.

  The moving object tracking unit 27 stores the time series of the object map stored in the object map storage unit 26 in a hard disk (not shown) as a tracking result.

  Through the processing described above, each image of the time-series image is divided into a plurality of blocks in the object map storage unit 26, and an identification code indicating the moving object in the image is attached to the block corresponding to the moving object. In addition, the motion vector of the moving object corresponding to the block is attached to the block and stored.

  Then, the moving object tracking unit 27 updates the block identification code and the motion vector stored in the object map storage unit 26 based on the result of image processing of the time-series image as described above. Specifically, the moving object tracking unit 27 updates the block identification code and the motion vector stored in the object map storage unit 26 according to the following procedures (1) to (4).

(1) For each of consecutive N images (N ≧ 2) of time-series images, by adding the same identification code to blocks whose absolute values of motion vector differences between adjacent blocks are within a predetermined value, An identification code procedure for attaching different identification codes to moving objects that overlap each other.

(2) In each of the N images, a first object that is a block group to which a first identification code is attached is in contact with a second object that is a block group to which a second identification code is attached, and the time for the N image A determination procedure for determining whether or not the degree of correlation between first objects of adjacent images is equal to or greater than a predetermined value.

(3) A tracking procedure in which the first object and the second object are traced back in time after it is determined affirmative in the determination procedure.

(4) An update procedure for updating the block identification code and the motion vector stored in the object map storage unit 26 based on the first object and the second object tracked back in time by the tracking procedure.

  The contour extraction unit 30 extracts the contour of the moving object from the time-series image. The contour extracting unit 30 extracts a plurality of moving objects having occlusions as a single moving object, and extracts a contour of the moving object as a single unit from a time-series image.

  The contour extraction unit 30 includes a target region setting unit 301, a target region correction unit 302, and a contour extraction processing unit 303.

  The target area setting unit 301 sets a target area for extracting the contour of the moving object based on the image area corresponding to the block of the moving object stored in the object map storage unit 26.

  The contour extraction processing unit 303 extracts the contour of the moving object from the time-series image for the target region set by the target region setting unit 301.

  The target area correction unit 302 projects, for each coordinate axis, a histogram of the number of pixels corresponding to the edges in the target area set by the target area setting unit 301 with respect to an image obtained by performing edge extraction processing on a time-series image. The target area set by the target area setting unit 301 is corrected for each coordinate axis based on the histogram generated for each coordinate axis.

  The contour extraction processing unit 303 described above may extract the contour of the moving object from the time-series image for the target region corrected by the target region correction unit 302.

  The correction unit 31 corrects the block identification code and the motion vector stored in the object map storage unit 26 based on the contour extracted by the contour extraction processing unit 303 of the contour extraction unit 30.

  In addition, the correction unit 31 includes information indicating boundaries of the plurality of moving objects based on identification codes corresponding to the plurality of moving objects in which occlusion occurs and stored in the object map storage unit 26, and contour extraction The block identification code and the motion vector stored in the object map storage unit 26 are corrected based on the extracted outline of the moving object obtained by integrating the plurality of moving objects in which occlusion has occurred.

  The determination unit 32 determines whether or not the background image is changing. For example, the determination unit 32 receives a signal indicating that the camera 10 has been panned or zoomed from the camera 10, and determines whether or not the background image has changed based on this signal.

  Alternatively, the determination unit 32 may compare the background image generated by the background image generation unit 24 with the image input from the camera 10 and determine whether or not the background image has changed. Alternatively, a marker is embedded in the background region in advance, and the determination unit 32 detects a change in the position of the marker image included in the image input from the camera 10, thereby changing the background image. It may be determined whether or not.

  When the determination unit 32 determines that the background image is fluctuating, the control unit 34 (first control unit) controls the contour extraction unit 30 for example every predetermined period or every predetermined frame. The contour is extracted by control, and the correction unit 31 is controlled to correct the block identification code and the motion vector stored in the object map storage unit 26.

  The moving object fluctuation amount detection unit 33 will be described later.

<Background block>
In the above description, whether or not an object exists is examined by comparing it with the background image in units of blocks, so the background image must be treated specially. Further, for example, since the background image is generated based on the photographed images for the past 10 minutes, when the camera shakes, this shake cannot be reflected in the background image.

  Therefore, the object map may be created by regarding the background image as an object. The object map generation method is different from the background image only in determining whether or not a moving object exists in the block. Since the background image is also regarded as an object, block matching is performed on all the blocks to assign IDs and determine MVs.

  The background image may be given a predetermined ID for the background image. The background ID and the moving object can be easily identified by this predetermined ID.

  As described above, even if the background image is set as one block and an ID is given to the background image, the image to which the block belongs is displayed between the background image and the moving object as shown in FIGS. It is possible to determine.

  In this way, by making the background image into one block, even when the background image fluctuates according to the panning or zooming of the camera, it can be processed in the same manner as when the background image is fixed. Become.

<Operation on Moving Object Tracking Device 20>
Next, the operation of the moving object tracking device 20 will be described with reference to FIGS. 11 and 12. First, the operation mode of the moving object tracking device 20 will be described with reference to FIG.

  First, it is assumed that the moving object tracking device 20 is in the camera fixing mode. In this camera fixing mode, an image generated by the background image generation unit 24 is used as a background image without being given an ID as an object. This is because the camera is fixed and the background image does not change.

  In the camera fixing mode, the moving object tracking unit 27 identifies the moving object because the background image is fixed.

  Next, in response to the panning or zooming of the camera, the determination unit 32 determines that the background image has changed. In response to this determination, the control unit 34 shifts from the camera fixing mode to the camera variation mode (step S1), registers the background image in the object, and assigns an ID.

  In the camera variation mode, the control unit 34 makes a transition to the correction mode every predetermined period or every predetermined frame (step S2). In this correction mode, the control unit 34 controls the contour extraction unit 30 to extract a contour, and controls the correction unit 31 to correct the block identification code and motion vector stored in the object map storage unit 26. .

  When the correction is completed in the correction mode, the control unit 34 transitions from the correction mode to the camera fluctuation mode (step S3).

  Thereafter, during a period in which the camera 10 is changing, the control unit 34 alternately changes between the camera change mode and the correction mode.

  Thereafter, in response to the determination unit 32 determining that the fluctuation of the camera 10 has stopped, the control unit 34 deletes the ID assigned to the background image, and the image generated by the background image generation unit 24. Is used as a background image.

  In addition, for the background image generation unit 24 to generate the background image, a predetermined period such as 10 minutes is required. Therefore, it is desirable to use the background image as an object until the background image is generated by the background image generation unit 24.

  Next, referring to FIG. 12, in the correction mode of FIG. 11, the control unit 34 controls the contour extraction unit 30 to extract a contour, controls the correction unit 31, and is stored in the object map storage unit 26. An operation for correcting the block identification code and the motion vector will be described.

  First, the occlusion detection unit 35 determines whether or not occlusion has occurred based on the object map stored in the object map storage unit 26 (step S1201).

  If it is determined in step S1201 that no occlusion has occurred, the target area setting unit 301 sets a target area for a moving object without occlusion (step S1202).

  Next, the target region correction unit 302 generates the above-described histogram, and corrects the target region set by the target region setting unit 301 for each coordinate axis based on the generated histogram (step S1203).

  Next, the contour extraction processing unit 303 extracts the contour of the moving object from the time-series image for the target region corrected by the target region correction unit 302 (step S1204).

  Next, the correction unit 31 corrects the block identification code and the motion vector stored in the object map storage unit 26 based on the contour extracted by the contour extraction unit 30 (step S1205).

  On the other hand, if it is determined in step S1201 that occlusion has occurred, the target area setting unit 301 of the contour extraction unit 30 sets a plurality of moving objects in which occlusion has occurred as an integrated moving object (step S1212). .

  Next, the target area setting unit 301 sets a target area for an integrated moving object (step S1213).

  Next, the target area correction unit 302 generates the above-described histogram for the moving object integrated, and corrects the target area set by the target area setting unit 301 for each coordinate axis based on the generated histogram (step S1214). ).

  Next, the contour extraction processing unit 303 extracts the contour of the moving object integrated with the target region corrected by the target region correction unit 302 from the image of the time-series image (step S1215).

  Next, the correction unit 31 performs an object map based on the boundary of the moving object where the occlusion detected as described with reference to FIGS. 4 to 7 and the contour extracted by the contour extraction unit 30 are performed. The block identification code and the motion vector stored in the storage unit 26 are corrected (step S1216).

  After the processing described above is completed for all moving objects stored in the object map storage unit 26, the control unit 34 transitions from the correction mode to the camera variation mode.

  As described above with reference to FIGS. 11 and 12, the moving object tracking device 20 can track a moving object regardless of whether the camera is fixed or fluctuating. In the camera fixed mode, since the background image is not registered in the object, it is not necessary to execute the processes shown in FIGS. 4 to 7 on the object of the background image. The amount or load can be reduced.

  Hereinafter, the identification code and motion vector for each block stored in the object map storage unit 26 will be referred to as “space-time MRF (Markov Random Field)”. Next, the operation and the result described with reference to FIG. 12 will be specifically described with reference to FIGS.

<Snakes>
First, extraction of the contour of the moving object from the image by the contour extraction unit 30 will be described. Of the configuration of the contour extraction unit 30, first, a technique as an example of extracting the contour of a moving object by the contour extraction processing unit 303 will be described in detail. Here, the case where Snakes (refer literature 1) is used as a technique which extracts an outline is demonstrated.

(Reference 1) Kass et.al “Snakes: Active contour models”, Proc. Of 1st ICCV, pp.259-268, 1987

  First, an outline of the Snakes will be described. In general, Snakes expresses a spline (set of control points) v (s) = (x (s), y (s)) (0 ≦ s ≦ 1) expressed as a parameter on the image plane (x, y), This is a contour extraction model that is deformed so as to minimize the energy function defined by the following equation (4) and whose shape is determined as a minimum state of energy.

  The first term Eint of the equation (4) is called internal energy, and has the property that the Snakes spline contracts smoothly into a convex shape. The theoretical definition is expressed by the following equation (5). The spline is smoothed by the first term in equation (5), and the spline is contracted by the second term.

  Next, the second term Eimage of Equation (4) is called image energy, and the larger the ratio of the spline as a whole on the edge (the portion where the luminance gradient such as the outline is large), the smaller the value. Has properties. This energy is defined by the following equation (6) according to the luminance I (v (s)) of the image. This time, in order to extract contour edges stably regardless of illuminance, we used the illuminance invariant filter image that we have developed so far as image energy.

  Here, the illuminance invariant filter image is an image converted by the image conversion unit 22 according to Expressions 1 to 3 described above.

  The third term Econ in the formula (4) is called external energy, and is used when force is applied to Snakes from the outside. This energy can be defined as needed. This time, the area term proposed for extracting the contour of the concave shape, which was difficult to extract due to the influence of Eint used for internal energy (see Document 2), was defined as external energy. The area term Earea is derived by the following equation (7) (see FIG. 13).

(Reference 2) Shoichi Araki, Naokazu Yokoya, Hidehiko Iwasa, Haruo Takemura: “A dynamic contour model that splits by intersection detection for the purpose of extracting multiple objects”, IEICE Transactions (D-II) Vol. J79-DII, No.10, pp1704-1711 (Oct, 1996)

  FIG. 14 shows the Snakes processing result. First, initial control points are arranged around the object whose contour is to be extracted (FIG. 14A). Next, Snakes starts to contract (FIG. 14B), and finally stops contracting near the contour line (FIG. 14C).

  Here, the initial control points shown in FIG. 14 correspond to the target area set by the target area setting unit 301.

<Analysis of edge distribution in local region>
Next, analysis of edge distribution in the local region will be described. Snakes is an algorithm that transforms a spline so as to minimize the energy function _{E_snakes} and ends the search with a minimal state. If there are many background edges and the initial control points are placed away from the outline, the spline will be caught by the background edges before converging on the object outline, and the energy will be minimal, extracting the object outline May fail.

  Therefore, the initial control points must be arranged to some extent near the contour of the object. On the other hand, in the local region of the person obtained by the spatio-temporal MRF, it is sometimes seen that the boundary between the background object and the person object becomes ambiguous during panning.

  Therefore, by analyzing the edge distribution (binary distribution of the illuminance invariant filter image) in the surrounding rectangle of the human object obtained by the object map, estimating the human area and placing the initial control points around it, the contour extraction Improve accuracy. The analysis of the edge distribution is performed by projecting onto the horizontal axis and the vertical axis and generating a histogram (see FIG. 15).

  Next, an example of an edge distribution analysis procedure will be described. The edge distribution analysis is executed by the following steps STEP1 to STEP3.

(STEP 1: Preprocessing of edge distribution)
The edge image is labeled, and a small area is excluded as noise.

(STEP 2: Horizontal axis histogram analysis)
A horizontal axis histogram is generated from the edge distribution obtained in STEP 1. Since a person has long continuous edges, after removing edges with low vertical continuity, the person is orthogonally projected once to narrow down the horizontal region of the person from a strong vertical distribution.

  Thereafter, the obtained edge distribution is projected onto the horizontal axis in the narrowed region, and a histogram is generated. Then, a window is scanned in the obtained horizontal axis histogram to obtain a continuously distributed region, thereby estimating a human edge distribution region in the horizontal component (FIG. 15A).

(STEP3: Vertical axis histogram analysis)
A vertical axis histogram is generated from the edge distribution obtained in STEP 1. Since a person has continuous edges that are somewhat long in the horizontal direction, after removing edges with low horizontal continuity, the person is projected once onto the vertical axis, and the vertical region of the person is narrowed down from the distribution in the horizontal direction.

  Thereafter, within the narrowed region, the edge distribution is projected onto the vertical axis, and a histogram is generated. Then, a window is scanned in the obtained horizontal axis histogram to obtain a continuously distributed region, thereby estimating a human edge distribution region in the horizontal component (FIG. 15B).

  In this STEP 3, the vertical axis histogram is generated from the edge distribution obtained in STEP 1. However, the vertical axis histogram can also be generated from the edge distribution in the local region narrowed in STEP 2.

  By referring to the edge distribution information through the above steps, a more accurate circumscribed rectangular region of the person can be obtained (FIG. 15C).

Note that the histogram threshold (initial control point) may be set as follows.
First, the histogram frequency values are clustered into two groups. The clustering method may be any method such as a k-mean method (one-dimensional). Thereby, it will be divided into a high frequency cluster and a low frequency cluster.

  Next, when searching from the both ends of the image to the inside and hitting a frequency value belonging to the high frequency cluster for the first time, a boundary is set between the immediately preceding low frequency position and the high frequency position. In this case, there is no problem even if there is a low frequency inside the high frequency.

<Inter-layer cooperation algorithm>
Here, a processing step of tracking by cooperation between layers of the spatio-temporal MRF and Snakes will be described. Hereinafter, correcting the object map using the spatio-temporal MRF and Snakes will be referred to as inter-layer cooperation.

  First, in the case of object map correction (without occlusion) by Snakes, that is, processing corresponding to steps S1202 to S1205 in FIG. 12 will be described.

STEP 1: An object map is received as an output of the spatio-temporal MRF, and information on a circumscribed rectangular area in each object is obtained.

STEP 2: For each object, an edge distribution analysis is performed in the local region obtained in STEP 2, and Snakes initial control points are placed around the contour of the object.

STEP 3: Execute Snakes in each object. Compared with the size of the circumscribed rectangle obtained in STEP 1, the object map is not modified for objects whose splines are contracted too much, and the object map is reflected for the other objects. Correct it.

  Next, in the case of object map correction by Snakes (with occlusion), that is, processing corresponding to steps S1212 to S1215 in FIG. 12 will be described.

STEP 1: An object map is received as an output of the spatio-temporal MRF, and information on a circumscribed rectangular area in each object is obtained. For objects in which occlusion is detected (having overlapping areas with circumscribed rectangles of other objects), the circumscribed rectangular area is obtained with the objects that are occluded as one group.

STEP 2: An edge distribution analysis is performed in the local region obtained in STEP 2, and initial control points of Snakes are arranged around the contour of the object.

STEP3: Execute Snakes. Compared to the size of the circumscribed rectangle obtained in STEP 1, the object map is not modified for objects whose splines are contracted too much, and the object map is reflected for the other objects. Correct it. At that time, in the labeling of the ID number of each object in the inner region surrounded by the contour extracted by Snakes, the labeling is performed based on the output result of the spatio-temporal MRF model. However, for the block recognized as a background object in the contour inner region, the ID is assigned by the spatiotemporal MRF model in the next frame by changing the label from the background object to indefinite. Alternatively, in the current frame, it is also possible to perform ID assignment by the spatiotemporal MRF model only for that block.

  FIG. 16 shows a modification example of the object map by Snakes. 16 (1-a) and (1-b) are examples in the case where there is no occlusion, and FIGS. 16 (2-a) and (2-b) are examples in the case of occlusion.

  In FIGS. 16 (2-a) and (2-b), the persons with ID numbers 6 and 7 are once made into one group, and the outline of the group (the boundary between the background and the person) is obtained by Snakes. The area division within the group reflects the output information by the spatiotemporal MRF.

<Correction of object map by Snakes>
Next, FIG. 17 and FIG. 18 are used to show the effect of the inter-layer cooperation algorithm when the camera 10 changes. Here, as shown in FIGS. 17 and 18, the processing results of moving object detection in the case of no inter-layer cooperation and inter-layer cooperation for the same frame of the same scene will be described.

  FIG. 17A shows a processing result without cooperation between layers, and FIG. 17B shows an object map thereof. FIG. 18A shows a processing result with cooperation between layers, and FIG. 18B shows an object map thereof.

  Camera panning begins immediately after frame number 80. When the inter-layer cooperation algorithm is not performed, although the tracking of more than 20 frames is successful, the boundary between the person object and the background object gradually becomes ambiguous (frame 95 and frame 107).

  On the other hand, when coordinated by Snakes, it is understood that the boundary between objects can be corrected by referring to the output of the spatiotemporal MRF, and tracking for a long time is possible. In addition, even when zooming, it can be seen that there is an effect of the cooperation algorithm between layers (the right two columns of the frame 142 and the frame 153).

  FIG. 19 shows the result of detecting a moving object when the camera 10 fluctuates when occlusion occurs. In FIG. 19A, no occlusion has occurred, and thereafter, in FIG. 19B, occlusion has occurred. In any case, it can be seen that the moving object tracking device 20 according to the present embodiment can track the moving object.

  As described above, the moving object tracking device 20 according to the present embodiment changes the background by correcting the block identification code and the motion vector stored in the object map storage unit based on the extracted contour. Even from an image, a moving object in the image can be accurately detected.

  In the above description of the embodiment, the motion vector is corrected together with the block identification code stored in the object map storage unit. However, only the identification code may be corrected. Even in this case, similarly, a moving object in an image can be accurately detected from an image with a changing background.

  The moving object variation detection unit 33 described above detects the size of the moving object or the variation of the movement amount stored in the object map storage unit 26 based on the identification code or the motion vector for each unit time.

  Then, the control unit 34 (second control unit) determines that the size of the moving object or the amount of movement of the moving object per unit time detected by the moving object fluctuation amount detection unit 33 is a predetermined size or movement amount. When it is larger than the fluctuation amount, the contour extraction unit 30 is controlled to extract the contour, and the correction unit 31 is controlled to correct the block identification code and the motion vector stored in the object map storage unit 26.

  As described above, the block identification code and the motion vector stored in the object map storage unit 26 may be corrected in accordance with the size of the moving object stored in the object map storage unit 26 or the variation amount of the movement amount. .

  As a result, the control unit 34 corrects the object map at a timing at which it is likely to fail to detect the moving object, as compared with the case where the control unit 34 simply corrects the object map every predetermined period or every predetermined frame. It becomes possible to do. Therefore, it becomes possible to detect and track a moving object more accurately.

  The storage unit such as the frame buffer memory 3, the image memory 21, or the object map storage unit 26 of FIG. 2 described above is a non-volatile memory such as a hard disk device, a magneto-optical disk device, a flash memory, or a CD-ROM. Only a storage medium that can only be used, a volatile memory such as a RAM (Random Access Memory), or a combination thereof.

  2, the image conversion unit 22, the background generation unit 24, the ID generation / annihilation unit 25, the moving object tracking unit 27, the contour extraction unit 30, the correction unit 31, the determination unit 32, the moving object fluctuation amount detection unit 33, and the control unit. The processing unit 34 or the occlusion detection unit 35 may be realized by dedicated hardware, and this processing unit is configured by a memory and a CPU (central processing unit) to realize the function of the processing unit. The function may be realized by loading a program to be executed into a memory and executing the program.

  Also, the image conversion unit 22, the background generation unit 24, the ID generation / annihilation unit 25, the moving object tracking unit 27, the contour extraction unit 30, the correction unit 31, the determination unit 32, the moving object variation detection unit 33, and the control in FIG. By recording a program for realizing the function of the processing unit called the unit 34 or the occlusion detection unit 35 on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium, You may perform the process by this process part. The “computer system” here includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within a scope not departing from the gist of the present invention.

  DESCRIPTION OF SYMBOLS 10 ... Camera, 20 ... Moving object tracking apparatus, 21 ... Image memory, 22 ... Image conversion part, 23 ... Frame buffer memory, 24 ... Background image generation part, 25 ... ID production | generation / annihilation part, 26 ... Object map storage part, 27 ... Moving object tracking unit, 30 ... Contour extraction unit, 31 ... Correction unit, 32 ... Determination unit, 33 ... Moving object fluctuation amount detection unit, 34 ... Control unit, 35 ... Occlusion detection unit, 301 ... Target region setting unit, 302: Target area correction unit, 303 ... Contour extraction processing unit

Claims (10)

A moving object tracking device on an image that detects a moving object in an image by processing a time-series image,
Each image of the time-series image is divided into a plurality of blocks, and an identification code indicating a moving object in the image is attached to the block corresponding to the moving object, and is stored.
A contour extracting unit that extracts a contour of the moving object from the image of the time series image;
A correction unit that corrects the identification code of the block stored in the object map storage unit based on the contour extracted by the contour extraction unit;
A moving object tracking device comprising: The contour extraction unit
A target area setting unit that sets a target area for extracting the contour of the moving object based on an image area corresponding to the block of the moving object stored in the object map storage unit;
A contour extraction processing unit that extracts a contour of the moving object from the image of the time-series image with respect to the target region set by the target region setting unit;
The moving object tracking device according to claim 1, comprising: The contour extraction unit
A histogram of the number of pixels corresponding to the edge in the target region set by the target region setting unit is generated by projecting the image of the time series image by edge extraction processing for each coordinate axis, A target region correction unit that corrects the target region set by the target region setting unit for each coordinate axis based on a histogram generated for each coordinate axis;
Have
The contour extraction processing unit
For the target region corrected by the target region correction unit, the contour of the moving object is extracted from the image of the time series image.
The moving object tracking device according to claim 2. The contour extraction unit
A plurality of moving objects in which occlusion occurs are defined as an integral moving object, and the contour of the integral moving object is extracted from the image of the time series image.
The moving object tracking device according to claim 1, wherein the moving object tracking device is a moving object tracking device. The correction unit is
Information indicating boundaries of the plurality of moving objects based on identification codes corresponding to the plurality of moving objects in which the occlusion occurs, which is stored in the object map storage unit, and the occlusion is generated by the contour extraction unit Correcting the identification code of the block stored in the object map storage unit based on the outline of the extracted moving object integrated with a plurality of moving objects that are integrated,
The moving object tracking device according to claim 4. A determination unit that determines whether or not the background image is fluctuating;
When the determination unit determines that the background image is fluctuating, the contour extraction unit is controlled to extract a contour, and the correction unit is controlled to store the block stored in the object map storage unit. A first control unit for correcting the identification code;
The moving object tracking device according to claim 1, further comprising: A moving object fluctuation amount detection unit that detects a fluctuation amount of the size or movement amount of the moving object stored in the object map storage unit per unit time based on the identification code;
The contour extracting unit when the moving object size or moving amount variation amount per unit time detected by the moving object variation detecting unit is larger than a predetermined size or moving amount variation amount. A second control unit that extracts the contour by controlling the correction unit, and controls the correction unit to correct the identification code of the block stored in the object map storage unit;
The moving object tracking device according to claim 1, further comprising: In the object map storage unit,
The identification code is attached to the block corresponding to the moving object, and a motion vector of the moving object corresponding to the block is attached to the block and stored.
A moving object tracking unit that updates a block identification code and a motion vector stored in the object map storage unit based on a result of image processing of the time-series image;
Have
The moving object tracking unit includes:
For each of consecutive N images (N ≧ 2) of the time-series images, by attaching the same identification code to blocks whose absolute values of motion vector differences between adjacent blocks are within a predetermined value, An identification code procedure for attaching different identification codes to overlapping moving objects;
In each of the N images, a first object that is a block group to which a first identification code is attached is in contact with a second object that is a block group to which a second identification code is attached, and the N image is temporally related. A determination procedure for determining whether or not the degree of correlation between the first objects of images adjacent to each other is equal to or greater than a predetermined value;
A tracking procedure for tracking the first object and the second object retroactively after being determined affirmative in the determination procedure;
An update procedure for updating a block identification code and a motion vector stored in the object map storage unit based on the first object and the second object tracked back in time by the tracking procedure;
The moving object tracking device according to claim 1, wherein: An on-image moving object tracking method used in an on-image moving object tracking device that detects time-series images and detects a moving object in the image,
A contour extracting unit that extracts a contour of the moving object from the image of the time-series image;
An object map storage unit in which each image of the time-series image is divided into a plurality of blocks and an identification code indicating the moving object in the image is attached to the block corresponding to the moving object and stored in the correction unit A correction procedure for correcting the identification code of the block stored in the image based on the contour extracted by the contour extraction unit;
A moving object tracking method comprising: To the computer as a moving object tracking device on the image that detects the moving object in the image by processing the time series image,
A contour extraction unit executes a contour extraction procedure for extracting the contour of the moving object from the image of the time series image,
An object map storage unit in which each image of the time-series image is divided into a plurality of blocks and an identification code indicating the moving object in the image is attached to the block corresponding to the moving object and stored in the correction unit Executing a correction procedure for correcting the identification code of the block stored in the block based on the contour extracted by the contour extraction unit;
Moving object tracking program to make

Images