JP2006024149A - On-image moving object recognition method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an on-image moving object recognition method and device recognizing a moving object quickly and accurately. <P>SOLUTION: Each of time serial images is divided into a plurality of blocks, and the blocks are compared with corresponding blocks of a background image by a block unit to calculate nonsimilarity between both blocks, and the presence of the moving object within the block is determined when the nonsimilarity is a reference value or more. An outline of a moving object area is recognized quickly in the block unit, in the first step, and only the area is corrected in the block unit, in the second step. The reference value is made larger in the second step than that in the first step, a block size is made smaller in the second step than that in the first step, the isolated moving object block is integrated with a moving object cluster near thereto in the second step, alternatively only a brightness component is used in the first step and a color component is used in the second step. When the same moving object is separately recognized even executing any thereof, the reference value is made smaller in the second step than that in the first step. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an on-image moving object recognition method and apparatus for processing a time-series image and recognizing a moving object (movable object such as a car, a bicycle, or an animal) in the image.

  By capturing moving objects with a video camera, image processing and recognizing and tracking moving objects on the image, detection of traffic conditions and traffic accidents, analysis of entering and exiting cars in parking lots, movement analysis of customers in stores, etc. Can be done automatically.

  As a method of recognizing and tracking a moving object on an image, for example, a 480 × 640 pixel frame is divided into 8 × 8 pixel blocks to form 60 × 80 block images, and each block image is a corresponding block of a background image. The moving object is detected in units of blocks, and the identification code (ID) and motion vector (MV) of the moving object are obtained in units of blocks based on the space-time correlation of the frame images at times t-1 and t. There are methods (for example, Patent Documents 1 to 3 below). Thereby, the same ID is given to the blocks recognized as the same moving object.

  In such a method, since it is necessary to track and process a plurality of moving objects in the frame image in real time, it is necessary to increase the processing speed. In particular, when measuring the speed of a moving object, all frame images of 30 frames / second are processed in order to improve measurement accuracy, and thus higher speed is required.

  If a monochrome image is used to meet this requirement, it becomes difficult to distinguish a moving object and its shadow. In addition, there are many moving objects without patterns, and the feature amount of the block at the middle part of the moving object is particularly small.Therefore, in the monochrome image, the middle part of the moving object may not be distinguished from the background. However, the front part and the rear part are recognized as different moving objects. On the other hand, when two moving objects having similar densities overlap, it may be recognized as one moving object.

  In order to solve such problems, when a high-resolution color image is processed, real-time processing cannot be performed unless frames are thinned out, and the measurement accuracy of physical quantities such as the position and speed of a moving object is lowered. That is, there is a trade-off between speeding up the processing and accuracy of moving object recognition.

  In Patent Document 4 described below, primary feature data in an image is extracted at high speed in order to automatically extract a target image simply by roughly specifying a region to be extracted on the image. By performing region segmentation, image regions that can be extracted are roughly extracted, and then region growth based on secondary features (features with different primary features, types, attributes, etc.) centered on candidate regions A method of performing finer extraction by performing is disclosed. Primary features include low-resolution data (for example, after reducing the resolution by thinning or averaging the image, and then detecting a motion vector, or dividing the image data into large blocks and moving the motion vector for each block) The secondary features use color components or high resolution data of the same type. From the evaluation of the homogeneity of the motion vector to the setting of the region growth nucleus is performed at a low resolution, and the region growth is performed at a high resolution to perform fine segmentation. Region growth means a process of merging or separating partial regions based on homogeneity.

However, it is not disclosed how to specifically process the secondary feature value. In order to cut out a plurality of moving objects on an image and measure the position, speed, etc. with higher accuracy, it is necessary to perform image processing efficiently. A specific method that satisfies this requirement is disclosed in Patent Document 4. Not.
JP 2002-133421 A JP 2003-006655 A JP 2003-263626 A JP 09-185720 A

  In view of the above problems, an object of the present invention is to provide a moving object recognition method and apparatus on an image that can recognize a moving object at high speed and more accurately.

  In the on-image moving object recognition method according to the present invention, each of the time-series images is divided into a plurality of blocks, and the dissimilarity between them is calculated in comparison with the corresponding blocks of the background image in units of blocks. In the on-image moving object recognition method in which it is determined that a moving object exists in the block if it is equal to or greater than a reference value, the outline of the moving object area is recognized at a high speed in blocks in the first stage, and only the area in the second stage. By correcting in block units, moving objects are recognized more quickly and accurately.

In the first aspect of the present invention,
(A) By recognizing the presence or absence of a moving object in units of blocks using the reference value as a first value, a region in which blocks in which a moving object exists is connected is recognized;
(B) The reference value is set to a second value larger than the first value, and the presence / absence of a moving object in the region is determined in units of blocks, whereby the shadow determined as the moving object in step (a) is determined. The region is narrowed by step (b).

In the second aspect of the on-image moving object recognition method according to the present invention,
(A) Recognizing a region where blocks where moving objects exist are linked by determining the presence or absence of a moving object based on the dissimilarity with the block size as a first value,
(B) Dividing the peripheral block of the region into blocks having a second value smaller than the first value, and determining whether or not there is a moving object based on the dissimilarity for each of the divided blocks .

In the third aspect of the on-image moving object recognition method according to the present invention,
(A) By recognizing the presence or absence of a moving object in units of blocks using the reference value as a first value, a region in which blocks in which a moving object exists is connected is recognized;
(B) If the two areas are close to each other by a predetermined value or more and the difference between the motion vectors of the two areas is not more than a predetermined value, a rectangle in which the two areas are inscribed is obtained.
(C) If the dissimilarity is not less than a second value smaller than the first value for each block in the rectangle, it is determined that a moving object exists in the block.

  In the fourth aspect of the moving object recognition method on the image according to the present invention, the number of blocks in the first area is equal to or less than a predetermined value for the first and second areas in which the blocks where the moving object exists are connected, and If the second region exists within a predetermined distance from the first region, it is considered that there is a moving object in the first and second regions and the region between them, and the first and second regions It is assumed that the same moving object exists in the region and the region between the two.

In the fifth aspect of the moving object recognition method according to the present invention,
(A) using the luminance component to temporarily determine the presence or absence of a moving object in units of blocks using the method, recognizing a region where blocks that have been temporarily determined to have a moving object are connected,
(B) With respect to the area, the presence / absence of a moving object is determined in block units by the method using color components.

  According to the configuration of the first to fifth aspects, the correction process is performed in the second stage only with respect to the moving object existence area recognized in the first stage.

  According to the configuration of the first aspect, the comparison reference value for the dissimilarity is set larger than that in the first stage. Therefore, even if the shadow of the moving object is recognized as a moving object in the first stage, the reference value is determined in the second stage. Even if a part or the whole is not recognized as a moving object, and a new moving object non-existing block is generated inside the area excluding the edge of the area in the second stage, these blocks can be treated as the same moving object. Since this is possible, it is possible to suppress an increase in processing time and to achieve accurate movement object recognition.

  According to the configuration of the second aspect, since the block inside the moving object area recognized in the first stage is not subdivided, the peripheral edge shape of the area is obtained faster and more accurately than in the past, and Since a new moving object non-existing block is not generated inside the area in the second stage, it is not necessary to perform a process of filling it with the moving object existing block.

  According to the configuration of the third or fourth aspect, even if the same moving object is separated and recognized as different moving objects in the first stage of the first or second aspect, they can be recognized as the same moving object. It becomes possible.

  According to the configuration of the fifth aspect, the dissimilarity between the image of the moving object region recognized in the first stage and the background image is increased in the second stage for the block in which the moving object actually exists. In the second stage, a shadow block recognized as a moving object in the first stage is removed, or a block that is part of the moving object but is determined not to be a moving object in the first stage is referred to as a moving object existence block. It becomes possible to judge. Therefore, even if another moving object overlaps one shadow of the two moving objects, and the two moving objects and the shadow are recognized as one moving object, the two moving objects are separated. It becomes possible to recognize.

  Other objects, configurations and effects of the present invention will become apparent from the following description.

  Embodiments of the present invention will be described below with reference to the drawings. In the figure, the same or similar elements are denoted by the same or similar reference numerals.

  FIG. 1 is a schematic functional block diagram of a moving object tracking apparatus 20 that processes an image captured by a video camera (for example, an ITV camera) 10 and tracks a moving object. For example, the video camera 10 is arranged near an intersection, and FIG. 2 shows a schematic frame image captured by the video camera 10.

  The moving object tracking device 20 other than the storage unit can be configured by computer software, dedicated hardware, or a combination of computer software and dedicated hardware.

  The time-series images captured by the video camera 10 are stored in the image memory 21 at a rate of, for example, 30 frames / second, and the oldest frame is rewritten with a new frame image.

  In the present invention, the image data stored in the image memory 21 is not only processed in real time by the components 23 to 26 but also stored in an external storage device (not shown), and only necessary portions are processed later. It can also be applied to cases.

  Further, even if the image (original image) stored in the image memory 21 is directly processed by the constituent elements 23 to 26, it is converted into a spatial difference frame image by applying a filter such as a Laplacian filter or two-dimensional Fourier transform In addition, a configuration in which the components 23 to 26 are processed (transformed images) obtained by performing inverse Fourier transform after emphasizing components of a predetermined frequency or higher is also possible. Hereinafter, “image” means an original image or a converted image.

  The background image generation unit 22 includes a storage unit and a processing unit. The processing unit accesses the image memory 21, and for example, the pixel values of the corresponding pixels of the frame image of all or thinned out objects in the past one minute. A histogram is created, and an image having the mode value (mode) as the value of the pixel is generated as a background image and stored in the storage unit. The background image is updated by performing this process periodically.

  Here, in the frame 30, as shown in FIG. 2, entrance slits EN1 to EN4 and exit slits EX1 to EX4 are arranged at four entrances to the intersection and four exits from the intersection, respectively. In the ID generation / annihilation unit 23, the position and size data of the entrance slits EN1 to EN4 and the exit slits EX1 to EX4 in the frame 30 are set in advance. The ID generation / annihilation unit 23 reads the image data in the entrance slits EN1 to EN4 from the image memory 21, and determines whether or not there is a moving object in the entrance slits in units of the blocks. Whether there is a moving object in a certain block is determined by whether the block dissimilarity NS is greater than or equal to the reference value RV1. The block dissimilarity NS is, for example, the sum of absolute values of differences between each pixel in the block and the corresponding pixel in the background image.

  When the ID generation / annihilation unit 23 determines that there is a moving object in the block, the ID generation / annihilation unit 23 assigns a new moving object identification code (ID) to the block. If the ID generation / annihilation unit 23 determines that a moving object exists in a block adjacent to the ID-assigned block, the ID generation / annihilation unit 23 assigns the same ID as the assigned block to this adjacent block. This ID-added block includes a block adjacent to the entrance slit. For example, ID = 1 is assigned to the block in the entrance slit EN1 in FIG. Hereinafter, a block of blocks to which an ID is assigned is referred to as a cluster.

  The ID is assigned to the corresponding block of the object map in the storage unit 24. The object map storage unit 24 is for storing the moving object information of each block of 60 × 80 blocks in the above-described case. The moving object information includes a flag F indicating whether or not an ID is assigned, In the case where it is shown that it is assigned, the ID and the motion vector MV of the block are included. The flag may not be used or may be used, and when ID = 0, it may be determined that no ID is assigned. The most significant bit of the ID may be used as a flag. For example, the motion vector MV of a block whose motion vector MV cannot be obtained by block matching is set equal to the average value of the motion vectors MV of the blocks having the same ID around it.

  A tracking process is performed on the cluster that has passed through the entrance slit by the moving object recognition unit 25 by a known method. For example, for high-speed processing, the approximate movement range of the cluster at time t is estimated based on the motion vector MV at time t-1 (time is the serial number of the frame, for example), and the above-mentioned moving object existence is determined within this range. The determination is performed for each block, the ID is assigned to the block on the movement direction side, and the ID on the block on the opposite direction to the movement is deleted, and the cluster at time t is determined. Further, the motion vector MV of the block to which the ID is assigned at time t is obtained by block matching between the frame images at time t−1 and t. For high-speed processing, the range for performing block matching is determined by prediction with the motion vector MV at time t-1. The ID and motion vector MV of each block can be determined simultaneously using an evaluation function (for example, Patent Document 3 above). The update of the ID and the motion vector MV is performed on the object map in the storage unit 24.

  The tracking process by the moving object recognition unit 25 is performed up to the exit slit for each cluster.

  The ID generation / annihilation unit 23 further checks whether an ID is assigned to the blocks in the exit slits EX1 to EX4 based on the object map in the storage unit 24, and if so, the cluster has passed through the exit slit. Sometimes the ID disappears. For example, when the block in the exit slit EX1 in FIG. 2 changes from a state in which ID = 3 is assigned to a state in which no ID is assigned, ID = 3 is extinguished. The disappearance ID can be used as the next generation ID.

  Next, problems to be solved by the first embodiment will be described.

  If the reference value RV1 is too large, the same moving object becomes a plurality of clusters as shown in FIG. 5A and is recognized as a different moving object. If the reference value RV1 is reduced to avoid this, the shadow 32 of the moving object 31 is also recognized as a moving object as shown in FIG. Since there are moving objects with various luminances, it is difficult to determine the reference value RV1 so that both problems do not occur.

  Therefore, if the surrounding block image is taken into consideration for the determination of the presence or absence of the moving object of each block, or if a color image is used, the processing becomes slow. In order to avoid this, it is necessary to reduce the frame rate or use high-performance and expensive hardware.

  In the first embodiment, in order to solve such a problem, in FIG. 1, the moving object recognition correcting unit 26 is applied to the object map generated in the storage unit 24 by the ID generation / annihilation unit 23 and the moving object recognition unit 25. The moving object in each cluster is recognized more accurately as follows.

First, the moving object recognition / correction unit 26 obtains a rectangle 33 in which the cluster is inscribed, as shown in FIG. 4A, for each cluster on the ID object map. This is done by scanning the ID object map and obtaining the minimum value X0j and maximum value Xmj of the X coordinate and the minimum value Y0j and maximum value Ymj of the Y coordinate for each IDj, j = 1 to N. ,
ID1, (X01, Y01), (Xm1, Ym1)
ID2, (X02, Y02), (Xm2, Ym2)
・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・
IDN, (X0N, Y0N), (XmN, YmN)
As obtained. These rectangular IDs 1 to IDN are the same as the IDs assigned to the moving object blocks in the rectangle.

  FIG. 3 is a schematic flowchart showing the process of the moving object recognition / correction unit 26 for a rectangle inscribed by any one cluster.

  The diagonal coordinates of the rectangle 33 are (X0, Y0) and (Xm, Ym), the ID and dissimilarity NS of the block (X, Y) are ID (X, Y) and dissimilarity NS (X, Y).

  (S0) Substituting initial values X0 and Y0 for X and Y, respectively.

  (S1) If ID (X, Y) ≠ 0, that is, if there is a moving object in the block (X, Y), the process proceeds to step S2, and if not, the process proceeds to step S4.

  (S2) If NS (X, Y) ≦ RV2, the process proceeds to step S3. Otherwise, the process proceeds to step S4.

  The reference value RV2 is larger than the reference value RV1 used in the moving object recognition unit 25, and is for recognizing only a moving object as a moving object and recognizing its shadow 32 as not a moving object. The reference value RV2 is set in advance before the moving object tracking process.

  In FIG. 1, the setting is performed by setting means including an input device IN such as a pointing device such as a mouse or a keyboard, a display device OUT for confirming the setting, and an input processing program included in the moving object recognition correcting unit 26. Performed based on human operation.

  For example, the reference value RV2 is determined by the operator from the distribution by displaying the moving object selected on the screen in an enlarged manner and displaying the dissimilarity NS of each block superimposed on the image. In this case, a histogram of the dissimilarity NS may be created and displayed on the screen. Since a peak corresponding to the shadow appears on the human gram, the reference value RV2 may be automatically determined based on the non-similarity NS of the peak or the bottom of the peak.

  (S3) 0 is substituted for ID (X, Y).

  (S4) The block coordinates (X, Y) in the rectangle 33 are updated.

  (S5) If the updated coordinates are within the rectangle 33, the process returns to step S1, and if not, the process proceeds to step S6.

  Through the processing described above, the ID distribution in the rectangle 33 as shown in FIG. 4A is as shown in FIG. 4B, and it is determined that part or all of the shadow is not a moving object.

  (S6, S7) Since the luminance of the moving object is close to the luminance of the background, the cluster may be separated as shown in FIG. The moving object block is supplemented as follows.

  That is, when each row in the rectangle 33 is scanned in the X direction, and the block moves from the block where the moving object exists to the block where the moving object exists, and moves again to the block where the moving object exists, the moving object is not in the meantime. The ID of the moving object given to the rectangle is given to the existing block (gap). As a result, as shown in FIG. 5B, the gap between the clusters is filled and one moving object cluster is obtained. That is, when the clusters are separated into two or more or there are gaps that are not separated, these gaps are filled with moving object blocks having the same ID.

  When the reference value is set to RV2 from the beginning, the probability that the same moving object is separated into a plurality of clusters increases, and it is not easy to collectively recognize the separated clusters as one moving object, and the processing time is long. Become.

  In the first embodiment, the reference value of the dissimilarity is set to a relatively small RV1, and the presence / absence of the moving object is determined in units of blocks, thereby recognizing the area where the blocks where the moving object exists are connected. Since the reference value is set to RV2 larger than RV1 and the presence / absence of a moving object is determined in units of blocks, an increase in processing time is suppressed, and a part or all of the shadow of the moving object is determined not to be a moving object. Recognition can be made accurate.

  FIG. 6 is a schematic functional block diagram of a moving object tracking apparatus 20A according to the second embodiment of the present invention that processes an image captured by the video camera 10 to track a moving object.

  The difference from the first embodiment is that the block size is increased, for example, the length of one side of the block is doubled as shown in FIG. 8A, and the ID generation / annihilation unit 23 and the moving object recognition unit 25 In the moving object recognition correction unit 26A, the moving object is converted into the following using the data of the image memory 21 and the background image generation unit 22 for the peripheral part of each moving object region on the object map as follows. It is a point to recognize more accurately.

  Since the amount of information included in one block increases by increasing the block size, the probability of erroneously determining that a moving object exists in the block but does not exist as shown in FIG. 5A is reduced. Can do.

  However, since the block size is large, the area determined to be a moving object becomes too wide.

  In order to recognize a moving object more accurately, if the block size of the entire area determined to be a moving object is reduced, the amount of information included in one block is reduced, so that the above-described gap is generated, and a plurality of one moving object is generated. It takes time to correct the cluster.

  Therefore, as described in the first embodiment, the moving object recognition / correction unit 26A first obtains the rectangle 33 inscribed by the cluster, and then performs the process shown in FIG. 7 for each rectangle.

  The diagonal point coordinates of the rectangle 33 are expressed as (X0, Y0) and (Xm, Ym), and the moving object presence / absence flag of the block (X, Y) is expressed as F (X, Y).

  (S10) Substituting initial values X0 and Y0 for X and Y, respectively.

  (S11) If F (X, Y) = 1, that is, if there is a moving object in the block (X, Y), the process proceeds to step S12; otherwise, the process proceeds to step S15.

  (S12) The value of the flag F of the 3 × 3 block centering on the block of interest (X, Y) is added, and this is set to n. The flag F outside the rectangle 33 is ignored.

  (S13) If n ≦ 6, that is, if there are no moving objects in three or more blocks among the eight blocks around the block of interest, the process proceeds to step S14, and if not, the process proceeds to step S15. For example, the blocks BL1 and BL2 have n = 4 and n = 7, respectively.

  (S14) As shown in FIG. 8B, the block (X, Y) is divided into four, and the dissimilarity NS is obtained for each of them to determine the presence / absence of a moving object and determined to be present. The ID of the rectangle 33 is given only to the block.

  (S15) The block coordinates (X, Y) in the rectangle 33 are updated.

  (S16) If the updated coordinates are within the rectangle 33, the process returns to step S11, and if not, the process ends.

  In the second embodiment, the block size is set to a relatively large first value, and the presence / absence of the moving object is determined in units of blocks based on the dissimilarity NS. For example, recognition is performed in about 1/4 of the conventional time, and the peripheral block of the region is divided into blocks having a size of a second value smaller than the first value, and each divided block is dissimilar. Since the presence / absence of the moving object is determined based on the degree NS, the shape of the peripheral portion can be obtained more accurately as shown in FIG. 8B, and the inner block of the moving object is not divided, so a new gap is created. It is not formed and there is no need to fill it.

  In addition, since the block of interest in which the number of moving object non-existing blocks in contact with the block of interest is a predetermined number or more is determined as the peripheral block, it is possible to divide only the blocks that need to be divided by simple processing. It is possible to accurately recognize a moving object in a short time.

  When the process proceeds from step S11 or step S13 to step S15, the target block (X, Y) is divided before moving to step 15, and the ID before the division (the ID of the block having no moving object is 0). ) May be given. In this way, the result shown in FIG. 9 is obtained by the processing of FIG. 7, and the data structure in the rectangle 33 is simplified, so that other processing in the rectangle 33 is facilitated.

  Further, after the process of FIG. 7, the processes of steps S6 and S7 of FIG. 2 may be performed.

  Even if the reference value RV1 is set to a relatively small value or the block size is increased in the first or second embodiment, the brightness is similar to the background depending on the moving object. One moving object may be recognized as two clusters.

  In the third embodiment, in order to solve this problem, the moving object recognition / correction unit 26 in FIG. 1 performs the process shown in FIG. 10 on adjacent rectangle pairs among a plurality of rectangles inscribed by the cluster.

  (S20) An X-direction distance ΔX and a Y-direction distance ΔY between the rectangles 34 and 35 (0 when the rectangles are in contact with each other, and negative when they overlap) are obtained. In the case of FIG. 11A, ΔX> 0 and ΔY <0.

  (S21a, S21b) If ΔX ≦ D1 and ΔY ≦ D2, the process proceeds to step S21c; otherwise, the process ends. Both D1 and D2 are positive values, are determined empirically so as to obtain the effects of the third embodiment, and are set in advance by the setting means described above.

  (S21c) The motion vector representative values of the blocks in the respective rectangles, for example, average values M1 and M2 are obtained, and the difference ΔM = | M1−M2 | If ΔM ≦ D3, the process proceeds to step S22, and if not, the process ends. This D3 is also determined in the same manner as D1 and D2, and is set in advance.

  (S22) As shown in FIG. 11B, a rectangle 36 in which both rectangles are inscribed is obtained. Further, the ID of the moving object block in the rectangle 36 is unified to one value IDj, and IDj is given to the rectangle 36.

  (S23) RV1-Δ is set as a new reference value RV3. Δ is a positive value, is determined in the same manner as D1 to D3 described above, and is set in advance by the setting means described above. The reference value RV1 is a value used by the moving object recognition unit 25 for creating an object map. Also, one diagonal point coordinate X0 and Y0 of the rectangle is substituted for X and Y, respectively.

  (S24) If ID (X, Y) = 0, the process proceeds to step S25; otherwise, the process proceeds to step S27.

  (S25) If NS (X, Y) ≧ RV3, it is determined that there is a moving object in this block, and the process proceeds to step S26. Otherwise, the process proceeds to step S27.

  (S26) IDj assigned to the rectangle is substituted for ID (X, Y).

  (S27) The block coordinates (X, Y) in the rectangle 36 are updated.

  (S28) If the updated coordinates are within the rectangle 36, the process returns to step S24; otherwise, the process proceeds to step S29.

  By the above processing, an object map with an ID as shown in FIG. 11B is obtained.

  (S29, 29A) The same processing as steps S6 and S7 in FIG. 3 is performed.

  According to the third embodiment, an object map with an ID as shown in FIG. 12 is obtained by the above processing, and a moving object can be recognized more accurately.

  When one moving object is recognized as being separated into three or more clusters, the above processing is performed for the two clusters, and this is repeated, so that it is recognized as one moving object.

  In any of the above-described embodiments, as shown in FIG. 14A, the ID object map generated by the moving object recognition unit 25 is an isolated moving object block 37 (hatched portion) in which no moving object block exists. May exist. The probability that such a small moving object exists independently is generally small. In addition, the isolated moving object block 37 has a high probability of belonging to the same moving object as the nearby moving object block.

  Therefore, in the third embodiment of the present invention, when such an isolated moving object block 37 exists, the moving object recognition correction unit 26 or 26A performs the process shown in FIG. 13 for each isolated moving object block.

  (S30) It is checked whether or not an isolated moving object block exists in the rectangle 36. If it exists, the process proceeds to step S31, and if not, the process ends.

(S31) It is checked whether another moving object block exists within the distance d from the isolated moving object block. The unit of d is 1 block width. For example, when d = 2, whether or not there is a moving object block whose side or vertex is in contact with the 3 × 3 local region 38 indicated by a thick line in FIG. Check out. The value of d is similarly set in advance by the above-described setting means (S32). A moving object non-existing block between this isolated moving object block and the moving object block in contact therewith is regarded as a moving object existing block. An ID is assigned or changed so that the connected moving object blocks have the same ID. As a result, the object map of ID becomes as shown in FIG.

  (S33) Even if such processing is performed, there may be a gap block. Therefore, for each of the moving object non-existing blocks in the rectangle 36, the moving object block having the same ID as the block whose side or vertex touches this block. It is determined whether there are m or more, and if there is, the same ID is assigned to the moving object non-existing block. For example, m is 5.

  The process of FIG. 13 is performed, for example, between steps S5 and S6 of FIG. In this case, step S33 in FIG. 13 may be omitted, and the gap may be filled by the processing in steps S6 and S7 in FIG.

  Further, the same effect can be obtained even if a cluster composed of k or less moving object blocks is set as an isolated cluster and the same process as the process for the above isolated moving object block is performed. The value of k is determined empirically based on the average size and block size of moving objects on the image.

  In the fourth embodiment, for the first and second areas in which blocks in which a moving object exists are connected, the number of blocks in the first area is equal to or less than a predetermined value, and the second area is the first area. If the moving object exists within a predetermined distance from the region, the moving object is considered to exist in the first and second regions and the region between them, and the connected region is regarded as one moving object. It can be recognized more accurately.

  In the fifth embodiment of the present invention, a color video camera is used as the video camera 10 of FIG. For simplification, it is assumed that the output signal of the video camera 10 is a luminance signal Y and color difference signals I and Q.

  In order to increase the processing speed, the ID generation / annihilation unit 23 and the moving object recognition unit 25 create an object map using only the image (Y image) of the luminance signal Y. At this time, the non-similarity NSy for each block of the Y image is stored in the storage unit 24.

  The moving object recognition / correction unit 26 </ b> A has images of color difference signals I and Q (I images) stored in the image memory 21 and the background image generation unit 22 only for a rectangular region including each cluster on the created object map. And Q image). The moving object recognition / correction unit 26A obtains the above-described dissimilarity NSi and NSq for each of the I image and the Q image in each block in the rectangular region, and determines the dissimilarity NSy of the Y image that has already been obtained. The sum of the similarities NSi and NSq is used as the block dissimilarity NS, and this is compared with a reference value to determine whether a moving object exists.

  As a result, the difference from the background image increases, that is, the dissimilarity NS increases, so that the shadow block of the moving object can be removed from the cluster, or the gap as shown in FIG. It becomes possible. In addition, since other moving objects overlap with the shadow of the moving object, if these are recognized as one moving object only by the Y image processing, the moving object is moved to the shadow area by the I image and Q image processing. It is determined that it does not exist and is recognized as two separate moving objects.

  It should be noted that the color image dissimilarity between adjacent blocks is greater than or equal to a predetermined value and the difference in motion vectors between the adjacent blocks is greater than or equal to a predetermined range for an area where blocks determined to have moving objects exist. For example, a configuration may be adopted in which moving objects overlapped on an image are recognized as different moving objects by determining that both blocks are different moving objects.

  Note that the present invention includes various other modifications.

  For example, in the first to fourth embodiments, the case where a monochrome image is used has been described. In particular, if a color image is used in step S14 in FIG. 7, a moving object can be recognized more accurately. Further, a configuration in which the fourth embodiment is performed with a color image and this is combined with the fifth embodiment may be adopted.

1 is a schematic functional block diagram of a moving object tracking device that processes an image and tracks a moving object according to Embodiment 1 of the present invention. FIG. It is a figure which shows the general | schematic frame image for showing the set entrance slit and exit slit. It is a schematic flowchart which shows the process by the moving object recognition correction part in FIG. It is a figure for demonstrating the process of FIG. It is a figure for demonstrating the process of FIG. 1 is a schematic functional block diagram of a moving object tracking device that processes an image and tracks a moving object according to Embodiment 1 of the present invention. FIG. It is a schematic flowchart which shows the process by the moving object recognition correction part in FIG. It is a figure for demonstrating the process of FIG. It is a figure for demonstrating the process of FIG. It is a schematic flowchart which shows the process by the moving object recognition correction part which concerns on Example 3 of this invention. It is a figure for demonstrating the process of FIG. It is a figure for demonstrating the process of FIG. It is a schematic flowchart which shows the process by the moving object recognition correction part which concerns on Example 3 of this invention. It is a figure for demonstrating the process of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Video camera 30 Frame 31 Moving object 32 Shadow 33-36 Rectangle 37 Isolated moving object block 38 Local area | region 20, 20A Moving object tracking device 21 Image memory 22 Background image generation part 23 ID production | generation part / annihilation part 24 Object map storage part 25 Movement Object recognition unit 26, 26A Moving object recognition correction unit EN1 to EN4 Entrance slit EX1 to EX4 Exit slit RV, RV1 to RV3 Reference value NS Non-similarity MV Motion vector

Claims (10)

Each time-series image is divided into a plurality of blocks, and compared with the corresponding block of the background image in block units to calculate the dissimilarity between them. If the dissimilarity is equal to or greater than a reference value, the moving object is In the moving object recognition method on the image that is determined to exist in
(A) By recognizing the presence or absence of a moving object in units of blocks using the reference value as a first value, a region in which blocks in which a moving object exists is connected is recognized;
(B) The reference value is set to a second value larger than the first value, and the presence / absence of a moving object in the region is determined in units of blocks, whereby the shadow determined as a moving object in step (a) is determined. A moving object recognition method on an image, characterized in that the region is narrowed by step (b). Each time-series image is divided into a plurality of blocks, and compared with the corresponding block of the background image in block units to calculate the dissimilarity between them. If the dissimilarity is equal to or greater than a reference value, the moving object is In the moving object recognition method on the image that is determined to exist in
(A) Recognizing a region where blocks where moving objects exist are linked by determining the presence or absence of a moving object based on the dissimilarity with the block size as a first value,
(B) Dividing the peripheral block of the region into blocks having a second value smaller than the first value, and determining whether or not there is a moving object based on the dissimilarity for each of the divided blocks And a moving object recognition method on an image.   3. The step (b) is characterized in that, in the region, the target block in which the number of moving object non-existing blocks in contact with the target block is a predetermined number or more is determined as the peripheral block. Moving object recognition method for images. (C) When there are two continuous regions of blocks determined to have a moving object in step (b), the same moving object exists in the two regions and the region between the two regions. The on-image moving object recognition method according to any one of claims 1 to 3, further comprising: Each time-series image is divided into a plurality of blocks, and compared with the corresponding block of the background image in block units to calculate the dissimilarity between them. If the dissimilarity is equal to or greater than a reference value, the moving object is In the moving object recognition method on the image that is determined to exist in
(A) By recognizing the presence or absence of a moving object in units of blocks using the reference value as a first value, a region in which blocks in which a moving object exists is connected is recognized;
(B) If the two areas are close to each other by a predetermined value or more and the difference between the motion vectors of the two areas is not more than a predetermined value, a rectangle in which the two areas are inscribed is obtained.
(C) If the dissimilarity is greater than or equal to a second value smaller than the first value for each block in the rectangle, it is determined that a moving object is present in the block. Method. (D) When there are two continuous regions of blocks determined to have a moving object in step (c), the same moving object exists in the two regions and the region between the two regions. The on-image moving object recognition method according to claim 5, further comprising: Each time-series image is divided into a plurality of blocks, and compared with the corresponding block of the background image in block units to calculate the dissimilarity between them. If the dissimilarity is equal to or greater than a reference value, the moving object is In the moving object recognition method on the image that is determined to exist in
For the first and second areas in which blocks having moving objects are connected, the number of blocks in the first area is equal to or less than a predetermined value, and the second area is within a predetermined distance from the first area. If the same moving object exists in the region between the first and second regions and both, the moving object is considered to exist in the region between the first and second regions and the region. A moving object recognition method on an image characterized by Each time-series image is divided into a plurality of blocks, and compared with the corresponding block of the background image in block units to calculate the dissimilarity between the two. If the dissimilarity is greater than or equal to a reference value, the moving object is the block In the moving object recognition method on the image that is determined to exist in
(A) using the luminance component to temporarily determine the presence or absence of a moving object in units of blocks using the method, recognizing a region where blocks that have been temporarily determined to have a moving object are connected,
(B) A method for recognizing a moving object on an image, characterized in that the presence / absence of a moving object is determined in block units by the method using the color component. Each time-series image is divided into a plurality of blocks, and compared with the corresponding block of the background image in block units to calculate the dissimilarity between them. If the dissimilarity is equal to or greater than a reference value, the moving object is In the moving object recognition apparatus on the image determined to exist in
Setting means for setting a first value of the reference value and a second value smaller than the first value;
A first recognizing unit for recognizing a region in which blocks each having a moving object are connected by determining whether the moving object exists in units of blocks by setting the reference value as the first value;
A second recognizing unit that sets the reference value to the second value and determines the presence or absence of a moving object in the region in units of blocks;
A moving object recognizing apparatus on an image, wherein the second recognizing unit narrows a shadow area determined as a moving object by the first recognizing unit. Each time-series image is divided into a plurality of blocks, and compared with the corresponding block of the background image in block units to calculate the dissimilarity between the two. If the dissimilarity is greater than or equal to a reference value, the moving object is the block In the moving object recognition apparatus on the image determined to exist in
Setting means for setting a block size of a first value and a second value smaller than the first value;
First recognition means for recognizing a region in which blocks where moving objects exist are linked by determining the presence or absence of moving objects based on the dissimilarity with the block size as the first value;
A second recognition for dividing the peripheral block of the region into blocks having a size of a second value smaller than the first value, and determining the presence or absence of a moving object for each of the divided blocks based on the dissimilarity Means,
An on-image moving object recognition apparatus comprising: