JP2016024534A - Moving body tracking device, moving body tracking method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving body tracking system and the like capable of robustly making collation of a moving image even if background change and occlusion of non-rigid body occur with the moving image containing the non-rigid body which accompanies deformation with no interruption, like a walker.SOLUTION: A moving body tracking system 1 includes a cluster extracting part 101 which divides a template image representing a moving body and an ROI image that is to be collated into partial regions respectively, an inter-cluster correlation calculation part 102 for acquiring correlation between partial regions, and an integral correlation calculation part 103 for acquiring correlation between the template image and the ROI image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technical field for tracking or collating a moving object included in a moving image. In particular, the present invention relates to a tracking technique for a moving body such as a person included in a moving image captured by a monitoring camera, an in-vehicle camera, or the like in a congested environment with a large background fluctuation.

  In recent years, image processing technology has been utilized in various fields as the information society advances. For example, Patent Literature 1 discloses an image tracking device that monitors traffic on a road using an image processing technique. This image tracking apparatus divides an image set as a template into blocks. Then, the image tracking apparatus obtains a correlation value between the block and a region (corresponding region) corresponding to the block by scanning a collation image that is a collation target for each divided block. The image tracking apparatus outputs only the block showing a good correlation value among the obtained correlation values as the correlation value between the template and the scanning region. As described above, Patent Document 1 proposes a technique that enables robust matching even when the background around the tracking target changes or when the tracking target is hidden by an obstacle in the foreground.

  Patent Document 2 discloses a technique related to a method and apparatus for tracking a moving object in an image. This tracking device can track a moving object using moving images captured in a smaller time series.

JP 2000-259838 A Japanese Patent Laid-Open No. 2004-207786

  By the way, as an image processing technique, there is a technique for tracking a specific moving body such as a person or a car included in a moving image, for example. In this technique, a region representing a moving body included in an image (image frame) constituting the moving image is designated (hereinafter referred to as “template” in the present application). Then, in the moving image composed of a plurality of time-series images, the technology collates a region where the same object (moving body) appears from an image acquired at a different time from the image. However, in this technique, a correct region is obtained while ensuring robustness against a change in an image representing a tracking target region (hereinafter referred to as a “tracking target image”) in the image inside the template and the collation image. Collation is an issue.

  Japanese Patent Application Laid-Open No. 2004-151620 discloses a method for robust matching even when the background around the tracking target changes or when the tracking target including an obstacle is hidden in the foreground. However, this method is premised on not changing the shape or brightness of the tracking target image excluding the background and the hidden portion of the tracking target. Therefore, for example, when collating a non-rigid body (moving body) that moves while constantly changing its posture like a pedestrian, this method correctly collates (or tracks) a moving image representing the non-rigid body. Is difficult to do. Similarly, the technique disclosed in Patent Document 2 cannot solve this problem.

  The present invention provides a moving object that can robustly collate the moving image even when a background change or non-rigid hiding (occlusion) occurs in a moving image including a non-rigid body that is constantly deformed, such as a pedestrian. The main purpose is to provide a tracking system.

  In order to achieve the above object, a mobile tracking device according to an aspect of the present invention is characterized by including the following configuration.

That is, the mobile tracking device according to one aspect of the present invention is
A dividing unit that divides each of the first image region representing the moving object to be collated or tracked and the second image region representing the region of interest to be collated into partial regions;
Correlation calculating means for obtaining a correlation between partial areas between the first and second image areas;
Of the correlations between the obtained partial areas, a combination of specific partial areas is selected according to the importance given to the combination of the partial areas, and the correlation between the selected specific partial areas is integrated. And an integrated correlation calculating means for obtaining a correlation between the first and second image regions.

  In order to achieve the object, a moving body tracking method according to an aspect of the present invention includes the following configuration.

That is, the moving body tracking method according to an aspect of the present invention includes:
By electronic control unit,
A first image area representing a mobile object to be collated or tracked and a second image area representing an area of interest to be collated are each divided into partial areas;
Obtaining a correlation between partial areas between the first and second image areas;
Of the correlations between the obtained partial areas, a combination of specific partial areas is selected according to the importance given to the combination of the partial areas, and the correlation between the selected specific partial areas is integrated. To obtain a correlation between the first and second image regions.

  The object is also achieved by a computer program for realizing the mobile object tracking device and the mobile object tracking method having the above-described configurations by a computer, and a readable storage medium storing the computer program. Is done.

  According to the present invention, even when a background change or non-rigid occlusion occurs in a moving image including a non-rigid body that is constantly deformed, such as a pedestrian, the moving object tracking that can robustly match the moving image A system or the like can be provided.

It is a block diagram showing typically an example of composition of a moving body tracking system in a 1st embodiment of the present invention. It is a block diagram which shows the structural example of the mobile body tracking system in the 1st Embodiment of this invention. It is a figure which shows operation | movement in each function part of the mobile body tracking system in the specific example of 1st Embodiment. It is a figure which shows operation | movement with the correlation calculation part between clusters, and an integrated correlation calculation part among the function parts of the mobile tracking system in the specific example of 1st Embodiment. It is a conceptual diagram which shows the flow of the process which the function part of the mobile body tracking system in the specific example of 1st Embodiment performs. It is a flowchart which shows typically a detailed operation | movement in the case of adding the process which narrows down the group used as the object of the correlation calculation which the correlation calculation part between clusters in the specific example of 1st Embodiment adds. It is a flowchart which shows typically a detailed operation | movement in the case of performing a correlation calculation using the group of the representative cluster which the correlation calculation part between clusters in the specific example of 1st Embodiment performs.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
FIG. 1 is a block diagram schematically showing a configuration example of a moving body tracking system 1 according to the first embodiment of the present invention. Referring to FIG. 1, the moving body tracking system 1 includes an image output device 20, an electronic control device 10, and a terminal device 30. The moving body tracking system 1 is a system that tracks the movement of a moving body that is a detection target based on a captured image.

  In the following description, the moving body is, for example, a pedestrian or a preceding vehicle.

  The image output device 20 is a device that outputs at least image information to the electronic control device 10. The image output apparatus 20 illustrated in FIG. 1 is a camera device, for example. This camera device is an apparatus that captures an image in real time (hereinafter referred to as an “imaging apparatus” in the present application). In addition, the camera device continuously acquires images to be captured. For example, a video camera that outputs an NTSC (National Television Standards Committee) format or a PAL (Phase Alternating Line) format can be used as the imaging device.

  The image output device 20 may be an image capture device that reads image information stored in a storage medium, converts it into an NTSC output, PAL output, or other image format that can be read by the electronic control device 10 and outputs the image format. . The image output device 20 in this case can also be realized as a software program that operates inside a CPU (Central Processing Unit) 11 of the electronic control device 10. The image output device 20 is not limited to a fixed camera, and may be a moving camera. The present invention described by taking this embodiment as an example can be applied to a wide range of applications such as a monitoring system using a pan / tilt camera, driver assistance using a vehicle-mounted camera, environment recognition of a mobile robot, and the like.

  The terminal device 30 operates as a user interface that operates the electronic control device 10 and monitors the internal state and output of the electronic control device 10. The terminal device 30 is a display or the like that presents an input image, a recognition area, a symbol indicating a registered list of recognition targets, and the like. Further, the terminal device 30 is an input device (for example, a switch board, a keyboard, a mouse, etc.) that inputs commands to the electronic control device 10 such as start and end of image recognition, designation of a recognition target, selection of presentation information to be presented on a display, , Touch panel, etc.).

  For convenience of explanation, the mobile tracking system 1 will be described by taking a configuration having the terminal device 30 as an example, but the present invention described by taking this embodiment as an example is not limited to the configuration described above. The mobile body tracking system 1 can also implement the present invention in a configuration that does not include the terminal device 30 (the same applies to the following embodiments).

  In addition, the terminal device 30 can be connected to a device (not shown) that uses information obtained by the mobile tracking system 1 as an input. The function of the device can also be realized as a software program that operates inside the CPU 11 of the electronic control device 10.

  The electronic control device 10 is a computer that performs information processing for recognizing a moving object in an image. The electronic control device 10 is based on the image information sent from the image output device 20, and based on a predetermined program, the mobile body (for example, a pedestrian, a preceding vehicle, etc.) belonging to a specific category to be recognized included in the image Information processing for specifying the area representing) is performed.

  The electronic control unit 10 includes a central processing unit (CPU) 11, a storage device (Mem: Memory) 12, a storage device (DB: database) 13, an interface (Interface: I / F) 14, an interface 15, Have In FIG. 1, for convenience of explanation, the storage device (Mem) 12 and the storage device (DB) 13 are shown separately. However, these may be realized as one storage device.

  The interface 14 is a device that mediates exchange of information among the central processing unit 11, the storage device (Mem) 12, the storage device (DB) 13, and the image output device 20. In FIG. 1, as an example, the interface 14 is connected to the image output device 20 and the central processing unit 11. However, the interface 14 may be directly connected to the storage device (Mem) 12 and the storage device (DB) 13.

  The interface 15 passes the result of information processing performed inside the electronic control device 10 to the terminal device 30 connected to the outside, or receives a command input from the terminal device 30 to the electronic control device 10. Serves as an information broker.

  The storage device (Mem) 12 is a device that stores temporary data. The storage device (Mem) 12 is electrically connected to the central processing unit 11.

  The storage device (DB) 13 is a device that mainly stores a database (DB). The storage device (DB) 13 is electrically connected to the central processing unit 11. For convenience of explanation, in FIG. 1, the storage device (Mem) 12 and the storage device (DB) 13 will be described by taking a configuration built in the electronic control device 10 as an example. However, the embodiment according to the present invention is not limited to such a configuration. These devices may be external storage devices.

  The central processing unit 11 is a device that performs information processing. The central processing unit 11 is electrically connected to an interface 14, an interface 15, a storage device (Mem) 12, and a storage device (DB) 13. The central processing unit 11 refers to the information stored in the storage device (Mem) 12 and the storage device (DB) 13 based on the image information input from the image output device 20 via the interface 14. Information processing for collating or tracking an image area such as a moving object is performed. In the following description, for convenience of explanation, the matching of the image area is referred to as “image matching”, and the tracking of the image area is referred to as “image tracking”.

  Here, image collation (image tracking) will be described more specifically. In the following description, it is assumed that a reference image (template image, first image region) including an image representing a target of verification or tracking (a verification target or a tracking target) is given. Image collation (image tracking) is to estimate a region representing a collation or tracking target in an image (test image) of another scene different from the template image. Note that the reference image and the test image are images that are continuous in time series, such as a moving image.

  Further, such a template image can be defined as a region representing a partial image obtained by cutting out a minimum rectangular region centered on a target to be collated or tracked from an original image.

  In the following description, when time series continuity is not assumed between the template image and the test image, it is simply referred to as image collation or image search. In this case, image tracking is used when collating a template image acquired in the past with an image acquired at a later (that is, near) time in time series.

  In the following description, a method for collating a template image (first image region) and a region of interest (second image region) included in a test image that are effective in both applications of image collation and image tracking described above will be described. .

  In this embodiment, pedestrians are mainly examples of recognition targets. However, the present invention does not limit the recognition target to pedestrians. The present invention provides an effective technique for tracking a moving object that moves with a change in shape.

  In the following description, the region of interest included in the test image is referred to as a ROI (Region of Interest) image (hereinafter, the same applies to each embodiment).

  FIG. 2 is a block diagram illustrating a configuration example of the moving body tracking system 1 according to the first embodiment of the present invention.

  The electronic control unit 10 executes the software program in the central processing unit 11 to realize the functions of the respective units shown by blocks in FIG. The function of each unit realized in the electronic control device 10 may be realized as an individual device, a functional unit, or an electronic circuit. The software program supplied to the apparatus may be stored in a storage device such as a readable / writable storage device (Mem) 12 and storage device (DB) 13.

  In the above case, the software program supply method into the electronic control apparatus 10 can employ a general procedure at present. For example, there are a method of installing in the apparatus via various storage media such as a CD-ROM (Compact Disc Read Only Memory), a method of downloading from the outside via a communication line such as the Internet, and the like. In such a case, the present invention can be considered to be configured by a code constituting the software program or a storage medium storing the code.

  Referring to FIG. 2, the electronic control device 10 includes a cluster extraction unit (dividing unit) 101, an inter-cluster correlation calculation unit 102, and an integrated correlation calculation unit 103.

  These functional units function as a whole to calculate a correlation value (which may be referred to as similarity or coincidence) between the template image and the ROI image. In the entire system, the correlation value calculation is applied to one or more ROI images included in the test image. And as a whole system, it can use for judgment of a final collation area, such as determining a position with the highest correlation value as a collation area. Each of these functions generally operates as follows.

  The cluster extraction unit 101 executes region division for dividing each image region of the template image and the ROI image selected in the test image into partial regions.

  As the clustering means, for example, an existing K-means (k-means), hierarchical clustering, Graph-cut, or the like may be used.

  Clustering generally defines the distance (or similarity) between elements within data (one data or a group that is a subset of the whole). Next, clustering makes elements that are close in distance (or high in similarity) into the same group. On the other hand, clustering is known to perform grouping in which elements that are far away (or have a low similarity) are grouped together.

  In the K-average method, K centroids indicating the number of divisions are given as initial values, and then each element is associated with the nearest centroid. The K-average method updates each centroid with the average value of the associated elements. The K-means method continues these procedures until the change in the center of gravity converges. Thereby, the K-average method can group elements into K clusters.

  Hierarchical clustering is a method of creating a tree diagram of group classification by continuing the procedure of grouping into the same group in order from the closest group until all elements are finally integrated into one group.

  In the methods of the Graph-cut system, the problem of selecting a boundary to be divided is solved as a minimization problem that minimizes energy.

  The definition of the distance (or similarity) between elements used for clustering is the coordinate distance (Euclidean distance, pixel distance) included in the image, the proximity of brightness and color, and the motion vector that is the correspondence between pixels between time-series images. Similarity (or non-similarity) of features such as similarity (such as optical flow) can be used. More specifically, as an example, the cluster extraction unit 101 divides the template image and the ROI image into partial regions based on the template image and a criterion including similarity of flow vectors representing movement between pixels in the ROI image. You may employ | adopt the structure to do.

  In addition to the above-described means, an existing image segmentation technique can be used for cluster extraction. For example, as the method, the distance between pixels is defined from the difference between the trajectory and the geometric distance included in the image using the trajectory obtained by long-term linking motion vectors by optical flow, Realize area separation such as person and background.

  The inter-cluster correlation calculation unit 102 calculates a correlation value between the cluster obtained from the template image and the cluster obtained from the ROI image for each cluster acquired by the cluster extraction unit 101.

Here, the division element will be described. In the following description, the entire areas of the template image and the ROI image are A and B, respectively. In the following description, the entire area of the template image is referred to as a template area (A), and the entire area of the ROI image is referred to as an ROI area (B). When the region A is divided into M pieces and the region B is divided into N pieces, the respective divided elements can be defined as follows. That is,
A = {A ₁ , A ₂ ,..., A _M | M ≧ 1} (1),
B = {B ₁ , B ₂ ,..., B _N | N ≧ 1} (2).

In the inter-cluster correlation calculation unit 102, a combination (A _i , B _j ) of an arbitrary cluster element (hereinafter also simply referred to as “cluster” or “element”) of the region A of the template image and the region B of the ROI image. The correlation value R (A _i , B _j ) is calculated.

In general, the region shape of (A _i , B _j ) does not completely match. For this reason, the correlation calculation is performed by comparing feature vectors obtained by extracting features that are independent of the difference in the shape of each region.

  Here, as the feature vector, for example, a normalized histogram obtained by calculating a ratio for each type such as a pixel color and a gradient direction in each region can be used. In addition, for correlation between feature vectors, vector similarity indices such as Bhattacharya coefficients, various correlation indices such as normalized correlation, Euclidean distance, and Chebychev distance can be used.

  Here, the normalized correlation value Corr (range: [−1, +1]) and the distance Dist (value range: [0, + ∞]) are, for example, similarity R that takes a value from 0 to 1 as in the following equation: By converting to, the index can be unified.

R (A _i , B _j ) = exp [−Dist (A _i , B _j )] (3),
R (A _i , B _j ) = 2 * Corr (A _i , B _j ) −1.0 (4),
Here, exp is a natural logarithm. * Is an integration.

Note that the combination (A _i , B _j ) of the elements for which the correlation is obtained may be selected so that the correlation can be obtained for the brute force in all the combinations of the elements. On the other hand, the selection can also reduce the calculation cost by narrowing down the combination conditions of the elements.

  In the following description, a method for narrowing down combinations based on geometric similarity (geometric similarity) will be described as an example.

The following methods can be considered to narrow down such combinations. That is,
(1) Narrowing by the presence / absence of overlapping of regions for selecting only elements that overlap each other when the regions A and B are overlapped,
(2) Or, narrowing down by overlap ratio that narrows down to elements whose relative overlap area ratio of regions A and B exceeds the threshold, and (3) narrowing down by area ratio that narrows down to only those whose ratio of the area between elements does not exceed the threshold. .

  These are examples representing conditions for narrowing down combinations based on the evaluation of geometric similarity between clusters. This geometric similarity can be defined by the above-described overlap ratio, area ratio, or a combination thereof. That is, the geometric similarity can be defined by at least one of an overlap ratio and an area ratio.

The overlapping ratio of the regions can be defined by the following equation (5), for example. Moreover, an area ratio can be defined by following Formula (6), for example. However, area (X) shown in the equations (5) and (6) represents the area of the region X.
(5),
(6).

  The geometric similarity using the overlap ratio and the area ratio can be defined by the following equation (7), for example. However, (alpha) shown in Formula (7) is a weight with respect to an area ratio. * Is an integration.

(Geometric similarity) aff _g (A _i , B _j ) = (1−α) * (overlap ratio) + α * (area ratio)
(7)

As described above, the inter-cluster correlation calculation unit 102 calculates the correlation between the clusters by narrowing down in advance to a combination in which the similarity evaluation value between the clusters exceeds the threshold, such as the above-described aff _g (A _i , B _j ). Cost can be reduced. That is, the inter-cluster correlation calculation unit 102 can reduce the calculation cost by selecting a combination of partial areas based on the evaluation of geometric similarity. The process for narrowing down the combination will be described in detail later with reference to the flowchart shown in FIG.

  In addition, the inter-cluster correlation calculation unit 102 calculates the distance between the center of each image and the center of each cluster based on the assumption that the comparison target is the template image and the foreground region located at the center or approximately the center of the ROI image. By giving a weight inversely proportional to the correlation value to the correlation value, it is possible to place importance on the correlation between clusters located at the center. In the following description, the center or the approximate center is simply referred to as “center” for convenience of description.

  More specifically, for example, for the purpose of performing correlation calculation with a lower calculation amount, in the following description, it is assumed that the comparison target is a template image and a foreground region located in the center of the ROI image. explain. First, the cluster extraction unit 101 performs cluster division. Next, the inter-cluster correlation calculation unit 102 can select only the cluster closest to the center from among the divided clusters and calculate the correlation.

  The correlation value in this case is determined to be one at this point without going through the integrated correlation calculation unit 103. For this reason, the integrated correlation calculation unit 103 employs the correlation value obtained by the inter-cluster correlation calculation unit 102 as a correlation value between two images of the template image and the ROI image.

  Further, as the distance between the center of each image and the center of the cluster, which is a selection criterion for the cluster for calculating the correlation, a simple Euclidean distance between the center coordinates of the cluster and the image center can be used. In addition, a condition that limits the area of the cluster to a threshold value or more may be added to the selection criterion. Alternatively, the selection criterion defines an index that has a positive correlation with the size of the distance between the centers and a negative correlation with the size of the area of the cluster, and selects a cluster that minimizes them. You may make it do.

The energy (cluster selection energy) E _cluster (i) related to such cluster selection can be defined as follows as an example.
(8)

In equation (8), each parameter is as follows:
I: cluster index,
D (i): the distance on the image between the centers of the cluster and the template image or the ROI image,
Area (i): cluster area, and k: area weighting factor with respect to distance.

  In this way, the inter-cluster correlation calculation unit 102 can select a plurality of clusters in the order of increasing energy, for example, without limiting the number of representative clusters (representative clusters) to one.

  The integrated correlation calculation unit 103 determines the correlation value between the entire template image and the entire ROI image using the correlation value between the clusters acquired by the inter-cluster correlation calculation unit 102.

  The acquired correlation values between the clusters are selected according to the degree of importance determined by the contribution rate to the entire combination of the clusters, the correlation value, the positional relationship included in the image, and the like, and are integrated by weighting.

For example, the integrated correlation calculation unit 103 sets the maximum for each element (i) on the template image side among the correlations R _ij = R (A _i , b _j ) between the clusters acquired by the inter-cluster correlation calculation unit 102. A combination R (i, j) of specific clusters (partial regions) with an element on the ROI image side that takes the correlation value is selected. The integrated correlation calculation unit 103 uses the average value (correlation average between the largest clusters) for all the elements on the template image side obtained based on the correlation value of the selected combination as the correlation value R _all of each cluster as the following equation (9). Can be defined as follows.
(9)

  This is based on the magnitude of the correlation value, narrowing down the values of overlapping combinations of cluster elements one by one, and the correlation value for each set of clusters after selection is the same weight. It is a method of calculation.

In the integrated correlation calculation, instead of the average value, a weighted average corresponding to the importance for each combination of clusters after the duplication of cluster elements is removed may be taken. In this case, the correlation value R _all can be defined as, for example, “correlation average between weighted maximum clusters” shown in the following equation (10).
(10)

Here, the weight w _ij of the pair (i, j) between the clusters can be defined by, for example, Expression (11) as the area ratio W _ij of the element (i) in the template region (A). At this time, the relationship shown in Expression (12) is established.
(11),
(12).

Alternatively, the weight w _ij is defined as an area ratio W _ij of the cluster set (i, j) to the total area of the entire other cluster set used in the calculation, as shown in Expression (13).
(13)
However, C = {(A _k , B _l )} is a set of combinations of clusters between two images selected to be used for computation through the above-described combination overlap and threshold processing described later.

  In the integrated correlation calculation described above, the value of a set of elements whose correlation between clusters does not exceed the threshold value can be excluded.

For example, the correlation value R _all is a function K that takes 0 when the maximum correlation value (max _{j∈ (1, N)} (R _ij )) corresponding to a certain element (i) is lower than the threshold value TH _cr and 1 otherwise. _{Using ij} , it can be defined as “correlation average between weighted maximum clusters + excluded outlier” shown in the following equation (14).
(14)

However, when the correlation between all the clusters is less than or equal to the threshold value, the function K _ij must be set to 0 as an exception. Generally, when the set of correlations between most clusters is below a threshold value, the ROI image and the template image should not be collated.

Therefore, when the ratio of elements below the threshold is below the threshold, the value of the correlation value R _all is desirably 0. For example, the value of the correlation value R _all is set to 0 when the ratio of clusters that exceed the threshold TH _cr (effective cluster ratio shown in the following equation (15)) is lower than the threshold TH _acl .
(15)

In this case, the effective cluster ratio takes a value from 0 to 1. Therefore, the threshold value TH _acl may be set to any value from 0 to 1. As a reference for setting, the threshold TH _acl may be set to a lower ratio based on the ratio of the image area representing the recognition target inside the area in the template image.

  As described above, according to the moving body tracking system 1 according to the present embodiment, even when a background change or non-rigid occlusion occurs in a moving image including a non-rigid body with constant deformation such as a pedestrian, The moving images can be collated robustly. That is, according to the moving body tracking system 1, it is possible to perform robust matching against an apparent shape change of a non-rigid body that is a target of verification (tracking target) in addition to a change of the background. The reason is as described below.

  When the moving object tracking system 1 collates image areas, the moving object tracking system 1 divides the area into irregular small areas. This is because the mobile tracking system 1 can obtain a correlation value (between clusters) in units of divided small areas and integrate the results to obtain a (integrated) correlation value between the entire areas. That is, the mobile tracking system 1 can obtain the overall correlation value by selecting the correlation values obtained for each cluster.

  Further, the moving body tracking system 1 can obtain only the correlation value between the low clusters, and does not reflect the correlation result of the area corresponding to the changed background area when obtaining the integrated correlation value. That is, the moving body tracking system 1 can obtain the result of extracting and collating the features of the non-rigid body from the template image and the ROI image without being affected by the shape change of the non-rigid body. Thereby, the moving body tracking system 1 can suppress a decrease in the correlation value between the template region and the correct corresponding region due to the background change. In addition, the moving body tracking system 1 performs the correlation calculation between the small areas on the small area (cluster) clustered on the basis of the feature of the input image instead of the fixed lattice area unit. This is because a small area can be defined flexibly.

(Concrete example)
Next, a specific example based on the mobile tracking system 1 according to the first embodiment of the present invention described above will be described. In the following description, the characteristic part according to this example will be mainly described. At this time, the same reference numerals are assigned to the same configurations as those in the above-described embodiments, and duplicate descriptions are omitted.

  A moving body tracking system 1 according to a specific example of the present invention will be described with reference to FIGS. For convenience of explanation, as an example, in the following explanation, an operation when the mobile tracking system 1 collates image areas will be explained.

  FIG. 3 is a diagram illustrating an operation in each functional unit of the mobile tracking system 1 in the specific example of the first embodiment.

  Referring to FIG. 3, the cluster extraction unit 101 uses the template image of the input image acquired at time (t−1) and the test image acquired at time (t) (hereinafter also referred to as “collation image”). Each ROI image included is divided into a set of clusters.

  In the case of tracking, the cluster extraction unit 101 divides the ROI image included in the tracking result area acquired at time (t) and the test image acquired at time (t + 1) into a set of clusters. To do.

Next, the inter-cluster correlation calculation unit 102 calculates a correlation value R (A _i , B _j ) for each combination of the extracted template area cluster set (A) and ROI area cluster set (B). Calculate.

  The integrated correlation calculation unit 103 selects a specific cluster combination pattern that maximizes the correlation value for each cluster combination. The integrated correlation calculation unit 103 integrates the remaining correlation values by weighted averaging or the like through a threshold process for rejecting correlation values equal to or lower than the threshold value among the correlation values of the selected pattern. The integrated correlation calculation unit 103 employs a value obtained by the integration as an integrated correlation value between the two images.

  FIG. 4 is a diagram illustrating operations of the inter-cluster correlation calculation unit 102 and the integrated correlation calculation unit 103 among the functional units of the mobile tracking system 1 in the specific example of the first embodiment.

  The diagram on the left side of the drawing shown in FIG. 4 is a diagram conceptually illustrating an aspect in which a combination of clusters having the maximum value is selected for each cluster among the obtained correlation values between the clusters. 4 is a diagram conceptually illustrating an aspect in which a combination of clusters having low correlation is excluded by threshold processing.

  In this example, the inter-cluster correlation calculation unit 102 obtains a correlation value for a combination between all clusters. Then, the inter-cluster correlation calculation unit 102 leaves only the maximum correlation value for each cluster among the obtained correlation values between the clusters. The inter-cluster correlation calculation unit 102 leaves only the correlation values that exceed a preset threshold value (for example, 0.4 here) among the remaining correlation values. That is, the inter-cluster correlation calculation unit 102 eliminates a set of clusters having a low correlation by threshold processing.

  The integrated correlation calculation unit 103 calculates an integrated correlation value based on a statistical value (such as an average value) based on the correlation value of the cluster combination left by the inter-cluster correlation calculation unit 102. That is, the remaining correlation values are used for calculation of integrated correlation values such as averaging according to the weights given to the combinations of the clusters.

In this example, a set of selected cluster sets is defined as C.
C = {(A ₁ , B ₁ ), (A ₃ , B ₃ ), (A ₄ , B ₅ ), (A ₅ , B ₁ )}
(16)

Further, here, an example in which the integrated correlation value R _all is obtained by a weighted average using the area ratio W _ij shown in the following equation (17) as a weight with respect to the entire set (A _i , B _j ) of each cluster included in C. This is shown in equation (18).
(17),
(18).

As described above, the mobile tracking system 1 evaluates the similarity of the ROI image on the collation image with respect to the template image based on the correlation value (integrated correlation value R _all ) obtained by integrating the correlation values obtained selectively. . Thereby, even if the moving body tracking system 1 is a case where the shape change and the background of the non-rigid body which are objects are changed, the correct area | region can take a high correlation value robustly. As a result, in the mobile body tracking system 1, it can be expected that correct area collation or image tracking is performed.

(Operation)
The operation of the mobile tracking system 1 according to a specific example of the present invention will be described in detail with reference to the drawings.

  FIG. 5 is a conceptual diagram showing a flow of processing performed by the functional unit of the mobile tracking system 1 in the specific example of the first embodiment.

  Referring to FIG. 5, first, the cluster extracting unit 101 first extracts a template area (A) extracted from a moving body image (input image) including a recognition target and an ROI area extracted from a test image to be verified. Using (B) as an input, each area is divided into small areas (clusters) (Step 1).

  Next, the inter-cluster correlation calculation unit 102 calculates a correlation value for each combination of small regions of the divided template region (A) and ROI region (B) (Step 2).

  Finally, the integrated correlation calculation unit 103 calculates the final result of the correlation value between the template area (A) and the ROI area (B) by summing up the correlation between clusters calculated in Step 2 (Step 3).

  The above is the outline of the operation in the mobile tracking system 1 according to the specific example of the first embodiment.

  Next, in the following description, an operation in the case of adding a process of narrowing down the group for performing the correlation calculation to the operation shown in Step 2 of FIG. 5 will be described.

  FIG. 6 is a flowchart schematically showing an example of a detailed operation in the case where a process for narrowing down the group for performing the correlation calculation is added in Step 2.

  Referring to FIG. 6, first, the inter-cluster correlation calculation unit 102 selects a combination of elements (small regions) from the template region (A) and the ROI region (B) (Step 201). The inter-cluster correlation calculation unit 102 evaluates geometric similarity with respect to the selected set of elements (Step 202). As a result, the inter-cluster correlation calculation unit 102 determines whether or not the value exceeds the threshold value (Step 203).

  When the inter-cluster correlation calculation unit 102 determines that the value does not exceed the threshold value, the inter-cluster correlation calculation unit 102 advances the process to Step 205 (“NO” in Step 203). That is, the inter-cluster correlation calculation unit 102 skips Step 204.

  On the other hand, when it is determined that the value exceeds the threshold value, the inter-cluster correlation calculation unit 102 advances the process to Step 204 (“YES” in Step 203). The inter-cluster correlation calculation unit 102 calculates the correlation between the clusters (Step 204).

  Finally, the inter-cluster correlation calculation unit 102 determines whether selection has been performed for all element combinations. When the inter-cluster correlation calculation unit 102 determines that the selection has been completed for all the combinations of elements, the inter-cluster correlation calculation unit 102 ends the processing (“YES” in Step 205). On the other hand, if the inter-cluster correlation calculation unit 102 determines that the selection has not been completed for all combinations of elements, the process returns to Step 201 ("NO" in Step 205). In other words, the inter-cluster correlation calculation unit 102 sequentially performs processing on the unselected element set from Step 201.

  Next, in the following description, one representative cluster is selected according to the distance from the center of each image (input image and test image) and the size of the area in the operation shown in Step 2 of FIG. The operation in the case of performing the correlation calculation using only the above will be described.

  FIG. 7 is a flowchart schematically showing an example of detailed operation in the case where the correlation calculation is performed using only the set of selected representative clusters in Step 2.

  Referring to FIG. 7, the inter-cluster correlation calculation unit 102 calculates the selection energy of all clusters in each of the divided template area (A) and ROI area (B) (Step 201A). Next, the inter-cluster correlation calculation unit 102 selects a cluster that takes the minimum energy (Step 202A). The inter-cluster correlation calculation unit 102 calculates the correlation between the selected clusters (Step 203A).

  By adopting the configuration as described above, the mobile tracking system 1 can collate image regions that are robust against changes in the background and changes in the target shape.

  The present invention has been described above with reference to the embodiments and specific examples. However, the present invention is not limited to the above-described embodiments and specific examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Mobile body tracking system 10 Electronic controller 11 Central processing unit 12 Memory | storage device (Mem)
13 Storage device (DB)
DESCRIPTION OF SYMBOLS 14 Interface 15 Interface 20 Image output apparatus 30 Terminal device 101 Cluster extraction part 102 Inter-cluster correlation calculating part 103 Integrated correlation calculating part

Claims (10)

A dividing unit that divides each of the first image region representing the moving object to be collated or tracked and the second image region representing the region of interest to be collated into partial regions;
Correlation calculating means for obtaining a correlation between partial areas between the first and second image areas;
Of the correlations between the obtained partial areas, a combination of specific partial areas is selected according to the importance given to the combination of the partial areas, and the correlation between the selected specific partial areas is integrated. Integrated correlation calculation means for obtaining a correlation between the first and second image regions;
A moving body tracking device comprising: The correlation calculation means includes
The correlation value is obtained based on a weight inversely proportional to the distance between the center of the partial area and the center of the first or second image area divided into the partial area, and the correlation value exceeds a threshold value. The mobile body tracking device according to claim 1, wherein the correlation of the combinations of the partial areas is obtained. The correlation calculation means includes
For each set of partial areas of the first and second image areas, the distance between the center of the partial area and the center of the first or second image area divided into the partial areas. An energy function having a positive correlation in size and a negative correlation in the area size of the partial regions is defined, and the values obtained based on the energy function are the smallest values of the partial regions. The single correlation or a plurality of combinations of the partial areas are selected so that the values are in ascending order, and the correlation of the combinations of the selected partial areas is obtained. The mobile tracking device described. The correlation calculation means includes
Based on the evaluation of the geometric similarity of at least one of the overlapping ratio and the overlapping area ratio between the partial areas, for each set of partial areas of the first and second image areas, the partial areas The mobile tracking apparatus according to claim 1, wherein a combination of the selected partial areas is selected and the correlation of the selected partial areas is determined. The integrated correlation calculation means includes
The combination of the specific image regions whose correlation value exceeds a threshold value is selected from the set of correlations in the combination of the image regions obtained by the correlation calculation means. Item 5. The moving object tracking device according to any one of Items 4 to 6. The integrated correlation calculation means includes
The correlation between the selected specific image regions is integrated according to a weight determined by an area ratio of the combination of the specific partial regions with respect to the combination of the partial regions used when obtaining the correlation. The mobile body tracking device according to any one of claims 1 to 5. The integrated correlation calculation means includes
The correlation between the selected specific image regions is integrated according to a weight determined by an area ratio of the combination of the specific partial regions to the total area of the combinations of all the selected specific partial regions. The mobile body tracking device according to any one of claims 1 to 5. The dividing means includes
The first and second image regions are divided into partial regions based on a criterion including similarity of flow vectors representing movement between pixels in the first and second image regions. The moving body tracking device according to claim 7. By electronic control unit,
A first image area representing a mobile object to be collated or tracked and a second image area representing an area of interest to be collated are each divided into partial areas;
Obtaining a correlation between partial areas between the first and second image areas;
Of the correlations between the obtained partial areas, a combination of specific partial areas is selected according to the importance given to the combination of the partial areas, and the correlation between the selected specific partial areas is integrated. To obtain a correlation between the first and second image regions,
The moving body tracking method characterized by this. A function of dividing a first image area representing a mobile object to be collated or tracked and a second image area representing an area of interest to be collated into partial areas;
A function of obtaining a correlation between partial areas between the first and second image areas;
Of the correlations between the obtained partial areas, a combination of specific partial areas is selected according to the importance given to the combination of the partial areas, and the correlation between the selected specific partial areas is integrated. A computer program for causing a computer to realize a function of obtaining a correlation between the first and second image areas.