JP5253101B2 - Object tracking method and image processing apparatus - Google Patents

Description

  The present invention relates to an object tracking method and an image processing apparatus for tracking an object in a processing target image.

  There are technologies (Patent Documents 1, 2, and 3) for determining whether or not a person is a suspicious person by analyzing a behavior pattern of a person in a monitored area such as a parking lot. For example, an area to be monitored is photographed with a camera, the image is divided into a plurality of blocks, a block having the maximum amount of motion (motion vector) of an object in the block is obtained, and the position of the maximum block is moved to an adjacent block There is a video monitoring method that detects a continuously moving pattern and determines whether this pattern is an abnormal behavior pattern that violates a rule (see Patent Document 3).

  In addition, there is a monitoring camera system that performs tracking by obtaining position information such as the center of gravity of a moving object in a captured image and comparing the movement of the position information from time-series monitoring images (for example, Patent Documents). 4).

JP 2004-328622 A JP 2005-354594 A JP 2005-284651 A JP 2003-259337 A

  When monitoring large parking lots based on camera images, because there are multiple people, vehicles, obstacles, etc. in the image, objects being tracked (for example, people) cross each other or objects being tracked May temporarily hide behind an obstacle (for example, a parked vehicle). In the method of tracking an object based on temporal and spatial continuity as in the techniques disclosed in Patent Documents 3 and 4, the continuity is lost when the above-described intersection or hiding occurs. There is a problem that the same object cannot be continuously tracked as the same object before and after the intersection and before and after appearing temporarily hidden behind an obstacle.

  An object of the present invention is to solve the above-described problem, and an object tracking method capable of continuously tracking the same object as the same object before and after an intersection or a temporary hiding behind an obstacle, and An object of the present invention is to provide an image processing apparatus.

  The gist of the present invention for achieving the object lies in the inventions of the following items.

[1] In an object tracking method for detecting and tracking an object in a processing target image,
Detecting an object in the image to be processed;
When a new tracking target object is detected, a plurality of search blocks having a predetermined size for the tracking target are generated in the area of the object, and each search block has a feature of the object in the search block existing area. Storing information in association with each other;
For each search block, the object and the object to be tracked by the search block based on the comparison between the feature information of the object existing in the search block existence area and the feature information stored in association with the search block Calculating the fitness with
A determination is made as to whether or not an object existing in the search block existence area is a tracking target object of the search block based on the fitness of the search block, and if the determination result is true, the search block Determining a tracking target object as a tracking success object;
For each tracking success object, probabilistically replacing a search block with a low fitness with a search block with a high fitness between a plurality of search blocks targeting the object,
Updating the feature information stored in association with the search block for the search block whose tracking target is the tracking target;
Moving each search block based on a combined motion of the object to be tracked in the search block and a random motion;
The repeated line physician
The process of probabilistically replacing a search block having a low fitness with a search block with a high fitness between a plurality of search blocks of the same object as the tracking target is the sum of the fitness of all search blocks having the object as a tracking target. A roulette with the ratio of the fitness of each search block to the appearance probability of the search block is generated, a random number lottery based on the roulette is performed for each search block whose tracking target is the object, and the search block is selected by the lottery An object tracking method, wherein the object tracking method is performed by replacing a search block corresponding to the result .

  In the above invention, for each object to be tracked, a plurality of search blocks having characteristic information of the object are generated. The search block is a unit area for searching for an object to be tracked. After the occurrence, these search blocks are moved by adding random movement to the movement of the object (following by adding random elements), and feature information of the objects in each search block's existence area and corresponding search blocks Based on the comparison with the feature information of the tracking target object that is stored in addition (high or low fitness), it is determined whether or not the object existing in the search block existence region is the tracking target object of the search block . As a result, even if an object to be tracked disappears from the image due to an intersection or hiding behind an obstacle, it continues to search for the lost object by the moving search block, so the object that appears again is found by the search block. The same object can be tracked as the same object before and after crossing and hiding.

  For a tracking success object, a search block with a low fitness is replaced with a search block with a high fitness between a plurality of search blocks whose tracking targets are the object. In other words, the search block on the tracking target object has a high fitness and the search block outside the tracking target object has a low fitness, so that a search block with a low fitness is replaced with a search block with a high fitness. Thus, many search blocks exist on the tracking target object, and move so as to follow the tracking target object.

  On the other hand, for a search block whose tracking target is an object that has failed to be tracked, a process for replacing a search block with a low fitness with a search block with a high fitness is not performed. Become wider.

[2] The degree of matching is such that the smaller the difference between the feature information of the object (image) in the search block existing area and the feature information stored in association with the search block, and the position of the search block and The object tracking method according to [1], wherein the calculation is performed such that the shorter the distance from the center of gravity of the object to be tracked in the search block, the higher the distance.

  In the above invention, the coincidence (similarity) between the feature information of the object (image) in the existence area of the search block and the feature information of the tracking target object, and the distance between the existence position of the search block and the object gravity center of the tracking target The fitness is calculated based on the above.

[ 3 ] Divide the image to be processed into a plurality of small areas,
In the determining step, for each small area where an object exists, if a search block exists in the small area, the object to be tracked by the search block having the highest fitness among the search blocks present in the small area is The object tracking method according to [1] or [2] , wherein the object tracking method is determined to exist in the small area.

  In the above invention, the object is tracked and searched for each small area, so that the processing is simplified and the amount of calculation can be reduced. Note that the entire image of an object is recognized as a set of small areas belonging to the object.

[ 4 ] Any one of [1] to [ 3 ], wherein an object in which the tracking failure state has continued for a predetermined period or longer is excluded from the tracking target, and all search blocks having the object as the tracking target are deleted. Object tracking method described in one.

  In the above invention, when the disappearance state (tracking failure state) continues for a predetermined period or longer, the further search is stopped and the object is removed from the tracking target.

[ 5 ] An image processing apparatus that tracks an object in an image to be processed by the object tracking method according to any one of [1] to [ 4 ].

  According to the object tracking method and the image processing apparatus according to the present invention, even when the tracking target objects intersect with each other and the tracking target object temporarily hides behind the obstacle, It can be tracked as the same object.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a schematic configuration of an image processing apparatus 10 to which an object tracking method according to an embodiment of the present invention is applied. The image processing apparatus 10 includes a camera unit 11 for capturing an image to be processed, a processing unit 12 that performs processing such as detecting an object by analyzing an image captured by the camera unit 11 and tracking its movement. And an output unit 13 for outputting the processing result of the processing unit 12 and an input unit 14 for inputting instructions regarding various settings and operations for the image processing apparatus 10. Hereinafter, an object recognized in the image data is referred to as an object.

  The camera unit 11 captures a moving image. The camera unit 11 functions to capture a moving image by capturing several tens of frames of images per frame (a still image for one screen) per second. The processing unit 12 is characterized by a CPU (Central Processing Unit), a ROM (Read Only Memory) in which a program executed by the CPU and fixed data are stored, image data captured by the camera unit 11, and a tracking target object. An area for storing quantity data, a RAM (Random Access Memory) used as a work area when the CPU executes a program, and the like are mainly configured.

  The output unit 13 includes an interface circuit for outputting a processing result data signal to the outside, various setting screens and operation screens, a display device for visually displaying processing results, and the like. The input unit 14 includes various operation switches in addition to an interface circuit for inputting signals relating to various settings and instructions regarding operations.

  The image processing apparatus 10 has a function of tracking the same object as the same object before and after the occurrence of a phenomenon such as the intersection of the tracking target objects or the temporary hiding of the tracking target object behind the obstacle. It has. The image processing apparatus 10 is incorporated in, for example, a system that monitors a suspicious person in a large parking lot, and is used to track an object such as a person or a vehicle.

  FIG. 2 shows a state transition of an object tracked by the processing unit 12 of the image processing apparatus 10. The object status is undetected UNDETECTED, PENDING determining whether the object detected in the UNDETECTED state is an object that can be tracked, ACTIVE being tracked and successfully tracked, and tracking It is managed in four states: LOST, which is the target object, but tracking has failed (disappearing or searching).

  While the object is not detected, the UNDETECTED state continues (E1 in the figure). When an object is detected in the UNDETECTED state, the state shifts to the PENDING state (E2). In the PENDING state, the duration is counted (E4), and when a predetermined time (PENDING_TIME) elapses in the PENDING state, the object transitions to the ACTIVE state (E5). An object that disappears from the image before transitioning from the PENDING state to the ACTIVE state transitions to the UNDETECTED state (E3), and is excluded from the tracking target.

  In the ACTIVE state, the duration is counted (E6). If tracking of an object in the ACTIVE state fails (the object disappears), the object transitions to the LOST state, and the duration of the ACTIVE state is reset (E7). Even in the LOST state, the duration is measured (E9), and when the predetermined duration (LOST_TIME) elapses, the object is removed from the tracking target and becomes the UNDETECTED state (E10). An object found by the search by the search block within a predetermined time (LOST_TIME) after entering the LOST state returns to the ACTIVE state (E8).

  FIG. 3 shows each processing phase in the object tracking process performed by the processing unit 12 of the image processing apparatus 10. FIG. 4 shows the flow of the object tracking process. As shown in FIG. 3, the object tracking process includes an object detection phase P1 and a tracking / search phase P2. The object detection phase P1 corresponds to steps S1 to S4 in the flowchart of FIG. 4, and the tracking / search phase P2 corresponds to steps S5 to S9.

  First, the frame time t is initialized to 0 (FIG. 4; step S1), and then an object candidate area is detected from the processing target image captured by the camera unit 11 (step S2). The object candidate area is detected by a known method such as a difference process between frames or a binarization process.

  Next, it is confirmed whether or not one or more object candidate regions have been detected in the processing target image (step S3). When one or more object candidate areas are detected (step S3; Yes), the process proceeds to step S5. If no object candidate area is detected (step S3; No), it is confirmed whether or not there is an object whose state is LOST (hereinafter referred to as a LOST object) (step S4). The LOST object is an object in which an object detected as a tracking target in the past disappears (tracking failure), the duration of the disappearing state is less than a predetermined time (LOST_TIME), and is an object being searched by the search block.

  If the LOST object does not exist (step S4; No), the frame time t = t + 1 is set (step S9), the process returns to step S2, and the same processing is performed on the next frame image captured by the camera unit 11. As described above, the object detection phase P1 is a state in which it is confirmed whether or not one or more object candidate areas exist, and when there is no object candidate area, the detection of the object candidate area is awaited.

  In step S5, a tracking / search process is performed on the object candidate area detected in the object detection phase P1. In the object tracking / search process, a tracking process for associating an object in the image at the previous time with an object in the image at the current time and a search process for searching for an object that disappears due to hiding or crossing are performed. The tracking process is performed by associating an object in the image at the previous time with an object in the image at the current time by a known method such as a motion vector.

  The search process for searching for a lost object generates a plurality of search blocks around the center of gravity of a newly detected tracking target object, and then causes each search block to follow the tracking target object. Tracking is performed by paying attention to a local area in the object area (in this case, an area in units of search blocks). The search block is a unit area for tracking and searching for an object. If the search for the lost object is not successful within a predetermined time (LOST_TIME), the searched object is regarded as having been lost and is set to the UNDETECTED state, and is excluded from the tracking target. Details of the object tracking / search process will be described later.

  For an object that has been successfully tracked / searched in the object tracking / searching process (an object in the ACTIVE state), feature quantities such as shape, motion, and texture are extracted (calculated) (step S6). Extraction of object feature quantities such as shape and movement may be performed by any known method. The extracted object feature data is sequentially accumulated and updated or newly generated as time series data (step S7). The feature amount data accumulated in time series for each object is used, for example, to determine whether the object is a person or a vehicle.

  Thereafter, according to the state transition of FIG. 2, the state (status) of each object is updated (step S8), the frame time is incremented and updated with t = t + 1 (step S9), the process returns to step S2, and is captured by the camera unit 11. The same processing is performed on the next frame image.

  Next, details of the object tracking / searching process in step S5 of FIG. 4 will be described.

  In the object tracking / search process, tracking of an object and searching for an object (LOST object) that has failed (disappeared) in tracking are performed by a known method. The search is performed by a plurality of search blocks that move so as to track the disappeared tracking target object.

  FIG. 5 shows an outline of the search by the search block. For a newly detected object A to be tracked (person A in the example of FIG. 5), a plurality of search blocks D having a predetermined size for which the object A is tracked are generated, and each of these search blocks D is The characteristic information of the tracking target object A is associated and stored (t1). In FIG. 5, each shaded rectangular area is a search block D. Each object to be tracked is assigned a unique object ID as identification information for uniquely identifying the object, and the object ID of the object to be tracked is registered by registering the object ID of the object to be tracked in each search block. Is associated with the search block.

  After that, the object is tracked by an existing method such as matching of the image of the frame at the previous time and the image of the frame at the current time, and each search block D is tracked to the object to be tracked (the object's object). Move so as to follow the motion vector (FIG. 5; t2).

  If tracking of an object fails (disappears) due to crossing or hiding (FIG. 5; t3), the movement of the search block D with the object being tracked is continued according to a predetermined movement rule. For example, the movement is continued at random, or the movement is continued by combining the movement of the object before disappearance with the random movement. In FIG. 5, the object A is hidden behind the obstacle 21 at time t3, and each search block D moves with a motion that combines the motion of the object A up to the current time and a random motion. The influence of the motion before the disappearance of the object A gradually decreases, and the random motion becomes dominant, and the search block gradually moves so as to be spread over a wide range (t4). In FIG. 5, the character “t3” is attached to each search block at time t3, and the character “t4” is attached to each search block at time t4.

  As described above, when an object exists at the position (existing region) of the search block continuously moved so as to search for the lost object, the feature information of the object is stored in association with each search block. The feature information of the tracking target object is compared, and based on the comparison result, it is determined whether or not the object existing at the position of the search block is the tracking target object of the search block. If the object is a tracking target, the object ID of the tracking target object registered in the search block is assigned to the object. Thereby, the same object is tracked as the same object before and after disappearance.

  In the example of FIG. 5, the object A is found (detected) by the search block D1 at time t4. After the object to be tracked disappears, each search block moves and spreads with the random movement dominant, so if the object A appears at the position A ′ indicated by the broken line, It will be discovered by the search block D2 which has moved to the position. In this way, even if an intersection or a temporary hiding behind an obstacle occurs, the same object is tracked as the same object (with the same object ID) before and after that.

  In the present embodiment, as shown in FIG. 6, an image to be processed (image for one frame) is divided into a plurality of small areas (divided blocks Bk), and an object is captured in units of divided blocks. Thereby, the amount of calculation related to tracking is reduced. Here, the image to be processed is divided by a grid-like boundary line so that each divided block Bk is rectangular.

  In the example of FIG. 6, the object A is divided into 33 rectangular divided blocks Bk that are shaded. Tracking / searching of the object A is performed for each divided block Bk. The object A is recognized as an aggregate of the 33 divided blocks Bk, and the feature information such as the shape, the center of gravity position, and the movement of the object A as a whole is grasped. Here, when the area of the object occupying the divided block Bk is equal to or greater than a predetermined occupancy ratio (for example, 50% or more), the divided block is regarded as a divided block where the object exists.

  FIG. 7 shows the flow of the object tracking / search process (FIG. 4; step S5). In object tracking (FIG. 7; step S21), an object ID is assigned to a newly detected object and an object data table 30 as shown in FIG. 8 is generated.

  For objects that have been detected at the past time and have already been assigned an object ID, tracking is performed by a method such as associating an object in the previous time image with an object in the current time image by a known method such as a motion vector. To do.

  Next, search block generation processing for generating a search block for an object that is newly tracked is performed (step S22). Details of the search block generation process are shown in FIG. A state (status) that is present in the image to be processed is ACTIVE (step S42; Yes), and there is no search block for which the object is a tracking target (step S43; Yes). A plurality of search blocks are generated around the center of gravity (steps S44 to S55).

  A search block is a unit area for tracking / searching an object to be tracked. Whether or not an object existing in the search block's existing area is the object to be tracked by the search block, with the search block as a unit. To be judged. A predetermined number (BLOCKFACTOR) of search blocks is generated for each divided block in which an object to be tracked exists. That is, the number of search blocks corresponding to the size of the tracking target object is generated. The number of BLOCKFACTOR is one or more and is a number that can be arbitrarily changed, and may be set to an appropriate number based on the ratio of the area size of the divided block and the area size of the search block. For example, the number of BLOCKFACTOR is set to 2 to 5. FIG. 10 shows the relationship between the origin (px0, py0) and the center position (px, py) of the search block. The search block is a pixel area of BLOCK_SIZE × BLOCK_SIZE (5 × 5 pixel area in the example of FIG. 10). Here, although the search block is rectangular, the shape of the search block is not limited to this, and may be another shape such as a circle.

  Further, a search block data table 40 as shown in FIG. 11 is generated for each generated search block. In the search block data table 40, information indicating the attribute of the corresponding search block is registered. The following values are registered in the search block data table 40 as initial values when they occur.

-Object ID: obj_id → Object ID to be tracked
Search block state (status): state ^obj_id _{n_block} → ON
Search block position (x coordinate): bx ^obj_id _{n_block} → x
^-Search block position (y coordinate): by ^obj_id _{n_block} → y
Search block motion vector (horizontal direction): dx ^obj_id _{n_block} → dx (gx, gy)
Search block motion vector (vertical direction): dy ^obj_id _{n_block} → dy (gx, gy)
・^Fitness of search block: cor ^obj_id _{n_block} → 1.0

Further, an HSV color space cumulative histogram as shown in FIG. 12 is generated from the HSV value (pixel value expressed in the HSV color space) of each pixel in the search block existence region. This cumulative histogram is derived by scanning the search block area in units of pixels. For pixel value i, if j ≧ i,
hist _{H, S, V} (t, j) ^obj_id _{n_block} = hist _{H, S, V} (t, j) ^obj_id _{n_block} + 1
And That is, the possible values of the pixel value are divided into a predetermined number of classes, and if the pixel value of the pixel is i, “1” is added to each of the accumulated values of the classes to which i belongs. This process is performed for all the pixels in the search block.

  The HSV color space is a color space composed of three components: hue, saturation (saturation / chroma), and brightness (brightness / lightness / value), and the HSV color space cumulative histogram is the H (hue) component. , S (saturation) component and V (lightness) component. Each value related to the HSV color space cumulative histogram is stored in association with the search block as feature information of the object (image) at the position (existing region) of the search block.

  Next, the fitness for all search blocks is updated (FIG. 7: step S23, details are shown in FIG. 13). In the process of updating the fitness level, the fitness level is updated (steps S64 to S67) for all search blocks whose search block status (status) is ON (FIG. 13; step S63; Yes). The goodness of fit is an index indicating the degree of coincidence between the object existing at the position (existing area) of the search block and the tracking target object of the search block. The degree of fitness is the feature information of the object existing in the existence area of the search block (feature information as an image such as color and the feature information related to the center of gravity position) and the feature information of the object stored in association with the search block (the relevant It is calculated based on a comparison with the feature information of the object existing at the previous time position of the search block. In this embodiment, a case where an HSV color space cumulative histogram is used will be described. Specifically, by the product of the time difference of the HSV color space cumulative histogram in the search block existence area and the distance coefficient based on the distance between the center of gravity of the object to be tracked by the search block and the center of the search block. To derive. The goodness of fit ranges from 1 to 0. Specifically, it is as the following formula.

The time difference total_diff of the HSV color space cumulative histogram is expressed by the following equation, for example.

For example, the distance coefficient is expressed by the following equation.

  Therefore, the closer the distance between the center of the search block and the center of gravity of the object area is, the higher the fitness of the search block is. For the disappearing object (LOST object), the distance coefficient is obtained by using the center of gravity position at the frame time immediately before the LOST state as the center of gravity of the object.

  FIG. 14 exemplifies the relationship between the distance between the center of the search block and the center of gravity of the object to be tracked and the fitness. Since the search blocks D1 and D2 are not on the tracking target object, the distance from the object center of gravity is large (distance factor (dist_factor) → small), and the degree of coincidence of the object (image) feature information (HSV color space cumulative histogram) ) Is low (total_diff → small), so the fitness is low (approaching 0). Since the search blocks D3 and D4 are on the object to be tracked, the distance from the object center of gravity is small (distance factor (dist_factor) → large), and the degree of coincidence of object (image) feature information (HSV color space cumulative histogram) Since the degree of matching is also high (total_diff → large), the degree of matching is high (close to 1).

  Next, a divided block area ID assignment process is performed (FIG. 7: step S24, details are shown in FIG. 15). In this process, the divided block Bk in which some object exists in the divided block area and one or more search blocks exist has the highest priority (having the highest fitness) among those search blocks. By assigning the tracking target object ID registered in the search block, the object ID is assigned to the divided block Bk, and the object of the object ID is set to the ACTIVE state. The object that was originally in the ACTIVE state continues to be in the ACTIVE state.

  When the object ID registered in the search block whose tracking target is the LOST object is assigned to any one of the divided blocks Bk by the above processing, the tracking target object in the LOST state is found (detection) Was). Note that an object that is a tracking target object but has not been assigned an object ID to any of the divided blocks Bk is determined to be a LOST object in the object state update process (S8) of FIG. 4, and the state is maintained or updated. .

  Next, search block erasure processing is performed (FIG. 7: step S25, details are shown in FIG. 16). If the object state is UNDETECTED, or if the state is LOST and the object duration obj_time is longer than LOST_TIME (LOST state lasts longer than LOST_TIME), it belongs to the object (track the object) Delete all search blocks. Deletion is performed by clearing all search block data tables belonging to the object.

  Next, search block resampling processing is performed (FIG. 7: step S28, details are shown in FIG. 17). The search block resampling process is performed for all search blocks belonging to the object in the ACTIVE state (FIG. 7; step S27; Yes). In the search block resampling process, a search block with a low fitness is replaced with a search block with a high fitness between a plurality of search blocks with the same tracking target object.

  Here, a roulette is generated in which the ratio of the fitness of each search block to the sum of the fitness of all search blocks that track the same object (for example, object A) is the appearance probability of the search block, A random number lottery based on the roulette is performed for each search block whose object is to be tracked, and the search block is replaced with a search block corresponding to the result of the lottery result. That is, the randomness lottery is performed the same number of times as the number of all search blocks for which the object is tracked based on the roulette, and replaced by the search block corresponding to the winning point for each lottery, The search block having a low match is easily replaced with a search block having a high fitness.

More specifically, the following processing is performed for each object whose state (status) is ACTIVE among the objects existing in the processing target image. First, for the object obj_id of interest, a roulette is generated as follows based on the fitness cor ^obj_id _{n_block} of the search block n_block belonging to the object.

Search block n_block state (status) If state ^obj_id _{n_block} is ON,
roulette ^obj_id _{n_block} = roulette ^obj_id _{n_block-1} + cor ^obj_id _{n_block}
Search block n_block state (status) If state ^obj_id _{n_block} is OFF,
roulette ^obj_id _{n_block} = roulette ^obj_id _{n_block-1}

  Next, after generating random number data for each search block belonging to the object of interest obj_id and determining the roulette output for the search block based on the random number data, the search block corresponds to the output Replace with search block.

  FIG. 18 shows an example of a roulette generated for an arbitrary object. There are a total of M search blocks that use this object as a tracking target. On the roulette where the total fitness of all search blocks from search blocks 1 to M is 360 degrees, the output of each search block is arranged at an occupation angle corresponding to the ratio of the fitness. For example, the occupancy angle of the search block 1 is ((the suitability of search block 1) / (the sum of the suitability of all search blocks)) × 360 degrees.

  Since the search block with a high degree of matching has a large occupancy angle, the probability of winning in the random number lottery is high. On the other hand, a search block with a low degree of fitness has a small occupancy angle and a low probability of winning. Therefore, a search block with a low fitness is easily replaced with a search block with a high fitness.

  Here, replacing the search block means copying the contents of the search block data table 40 for items other than the search block ID.

  As described above, the suitability of a search block is high when the search block exists on the tracking target object, and is low when the search block is at a position outside the tracking target object. Accordingly, as shown in FIG. 19A, when search block replacement (search block resampling) is performed for the ACTIVE object, search blocks D5 to D7 on the tracking target object A and search blocks not on the object A There is a large difference in fitness between D1 and D4. Therefore, the roulette generated by the search block resampling is as shown in FIG. 19B (the occupied angle is small for D1 to D4 and the occupied angle is large for D5 to D7).

  When random number lottery is performed for the number of search blocks (in this case, seven times) using this roulette, the result is as shown in, for example, (c) of FIG. Is replaced with a search block having a high value. As shown in FIG. 19D, many of the search blocks that were not on the tracking target object are replaced with search blocks on the tracking target object, and the search blocks are easily aggregated on the tracking target object.

  Next, search block update processing is performed (FIG. 7: step S29, details are shown in FIG. 20). In the search block update process, the movement of each search block and the update of the feature information regarding the tracking target object stored in association with each search block are performed. The position information and motion vector information of all search blocks existing in the image are updated (steps S147 and S148). Further, a search block belonging to an object whose status (status) is ACTIVE (with the object as a tracking target), the status is ON, and the ID of the tracking target object and the location of the search block are divided For the search block that matches the object ID assigned to the block, the HSV color space cumulative histogram is also updated (step S146).

The location information is updated as follows.
Here, rnd_x and rnd_y indicate random numbers in the x and y directions, respectively. By adding a random number, a random motion is added to the search block.

The update of the motion vector information is as follows, and is a linear load sum of the motion vector of the search block at the previous time and the motion vector in the divided block at the current search block location.
Here, k represents an update rate (inertia constant). The value of k can be changed. If tracking has succeeded, the motion vector of the object becomes dominant, and if tracking has failed (the tracking target object has disappeared (LOST state)), movement by random numbers becomes dominant. .

  By tracking and searching for an object as described above, even if the object to be tracked is temporarily hidden behind the obstacle 21 as shown in FIG. That is, the same object ID can be assigned to the same object before and after being hidden by the obstacle 21 for tracking.

  Similarly, even when tracking target objects intersect, tracking can be performed by assigning the same object ID to the same object before and after the intersection (FIGS. 21A to 21D). In the intersection area, a search block having a high degree of matching has an advantage. In FIG. 21B, since the object A passes behind the object B, the search block that tracks the object B on the front side is superior. As shown in FIG. 5C, when the objects A and B are separated from each other from the intersection state, the search block belonging to the object A follows the object A, and the search block belonging to the object B follows the object B. (Fig. 4D).

  As described above, in the object tracking / search method according to the present embodiment, when a new object to be tracked is detected, a plurality of search blocks are generated near the center of gravity of the object, and each search block has its search block. Is stored in association with the feature information of the tracking target object in the existing region, and after that, the search block is moved so as to follow the tracking target object and the feature information of the tracking target object stored in the search block is stored. After updating and tracking fails, the search block continues to move so that the random movement becomes dominant, and the tracking target object that disappeared is judged by determining the fitness based on the local area called the search block By doing so, the same object is tracked as the same object before and after crossing and hiding.

  Note that the feature amount data indicating the feature information about the object such as the shape, movement, and texture extracted from the tracked object in step S6 in FIG. 4 is in a FIFO (First in First Out) format as shown in FIG. The time-series data is stored in the storage area 50 (FIG. 4, step S7). The storage area 50 in FIG. 22 stores feature data for each time in a time series from the current time up to Lmax times. FIG. 22A shows a state in which feature amount data is accumulated up to the middle of the storage area 50 (accumulation for L times: L is an arbitrary positive integer less than Lmax), and FIG. 22B shows feature amount data for Lmax times. A state in which the storage area 50 is filled is shown. The feature amount data stored in time series in the storage area 50 is used, for example, to determine whether each object in the image is a person or a vehicle. In addition, since the same object can be tracked as the same object even if there is a crossing or hiding, even if there is a crossing or hiding during tracking, the time series feature amount data of the same object can be collected.

  The embodiment of the present invention has been described with reference to the drawings. However, the specific configuration is not limited to that shown in the embodiment, and there are changes and additions within the scope of the present invention. Are also included in the present invention.

  For example, in the embodiment, the object is tracked / searched in divided block units, but tracking may be performed in object units.

  In the embodiment, the search block is moved by synthesizing the movement of the object to be tracked and the random movement (position change based on the random number, Equation 4), and the search block targeted for the search block belonging to the ACTIVE object Combined with resampling (probabilistic replacement of search block with low fitness with search block with high fitness), the search block moves to track the tracking target object while tracking is successful The search block is moved so that the random motion becomes dominant during the disappearance, but the method of moving the search block is not limited to the method exemplified in the embodiment. That is, the tracking target object may be tracked during tracking success, and it may be moved according to the movement rule that random motion becomes dominant during disappearance.For example, according to the tracking target motion vector without random elements during tracking success. It may be configured to move and move only with random movement while eliminating the inertial movement during disappearance.

  As the rule of movement during disappearance, it is preferable to add a random motion to the motion of the object immediately before the disappearance, and it is preferable that the random motion becomes more dominant as time passes.

  In the embodiment, the fitness of each search block is determined based on the time difference of the HSV color space cumulative histogram and the distance coefficient based on the distance between the object and the search block. May be determined. The degree of matching is preferably determined using the degree of coincidence as an image and the positional relationship between the object and the search block as elements.

1 is a block diagram showing a system configuration of an image processing apparatus according to an embodiment of the present invention. It is a state transition diagram of an object. It is explanatory drawing which shows the process phase in the object tracking process which the image processing apparatus which concerns on embodiment of this invention performs. It is a flowchart which shows the object tracking process which the image processing apparatus which concerns on embodiment of an invention performs. It is explanatory drawing which shows the outline of the search by a search block. It is explanatory drawing which shows an example of the state which divided | segmented the image (image for 1 frame) of a process target into the some division block. It is a flowchart which shows an object tracking and search process (FIG. 4; step S5). It is explanatory drawing which shows an object data table. It is a flowchart which shows search block generation processing (FIG. 7; step S22). It is explanatory drawing which shows the relationship between the origin (px0, py0) and center position (px, py) of a search block. It is explanatory drawing which shows a search block data table. It is explanatory drawing which shows an example of an HSV color space accumulation histogram. It is a flowchart which shows the detail of a fitness update process (FIG. 7: step S23). It is explanatory drawing which shows the relationship between the distance between search block center and the object gravity center of tracking object, and a fitness. It is a flowchart which shows the detail of a division | segmentation block area ID allocation process (FIG. 7: step S24). It is a flowchart which shows the detail of search block erasure | elimination process (FIG. 7: step S25). It is a flowchart which shows the detail of search block resampling process (FIG. 7: step S28). It is explanatory drawing which shows an example of the roulette produced | generated by the search block resampling process. It is explanatory drawing which shows an example of the replacement condition of the search block by search block resampling processing. It is a flowchart which shows the detail of search block update process (FIG. 7: step S29). It is explanatory drawing which shows an example of the tracking and search condition of the object by the search block before and behind an intersection. It is explanatory drawing which shows an example of the memory area which memorize | stores the feature-value data of an object in time series.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus 11 ... Camera part 12 ... Processing part 13 ... Output part 14 ... Input part 21 ... Obstacle 30 ... Object data table 40 ... Search block data table 50 ... Storage area P1 ... Object detection phase P2 ... Tracking / search Phase

Claims (5)

In an object tracking method for detecting and tracking an object in a processing target image,
Detecting an object in the image to be processed;
When a new tracking target object is detected, a plurality of search blocks having a predetermined size for the tracking target are generated in the area of the object, and each search block has a feature of the object in the search block existing area. Storing information in association with each other;
For each search block, the object and the object to be tracked by the search block based on the comparison between the feature information of the object existing in the search block existence area and the feature information stored in association with the search block Calculating the fitness with
A determination is made as to whether or not an object existing in the search block existence area is a tracking target object of the search block based on the fitness of the search block, and if the determination result is true, the search block Determining a tracking target object as a tracking success object;
For each tracking success object, probabilistically replacing a search block with a low fitness with a search block with a high fitness between a plurality of search blocks targeting the object,
Updating the feature information stored in association with the search block for the search block whose tracking target is the tracking target;
Moving each search block based on a combined motion of the object to be tracked in the search block and a random motion;
The repeated line physician
The process of probabilistically replacing a search block having a low fitness with a search block with a high fitness between a plurality of search blocks of the same object as the tracking target is the sum of the fitness of all search blocks having the object as a tracking target. A roulette with the ratio of the fitness of each search block to the appearance probability of the search block is generated, a random number lottery based on the roulette is performed for each search block whose tracking target is the object, and the search block is selected by the lottery An object tracking method, wherein the object tracking method is performed by replacing a search block corresponding to the result . The degree of matching is such that the smaller the difference between the feature information of the object (image) in the search block existing area and the feature information stored in association with the search block, and the position of the search block and the search block The object tracking method according to claim 1, wherein the object tracking method is calculated so that the distance from the center of gravity of the tracking target object is shorter. Divide the image to be processed into multiple small areas,
In the determining step, for each small area where an object exists, if a search block exists in the small area, the object to be tracked by the search block having the highest fitness among the search blocks present in the small area is the object tracking method according to claim 1 or 2, characterized in that it is determined that present in the small region. The object according to any one of claims 1 to 3 , wherein an object whose tracking failure state has continued for a predetermined period or longer is excluded from the tracking target, and all search blocks having the object as the tracking target are deleted. Tracking method. The object tracking method according to any one of claims 1 to 4, an image processing apparatus for tracking an object in an image to be processed.