JP2014211846A - Target tracking apparatus and target tracking program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately track an irregularly moving target.SOLUTION: A target tracking apparatus comprises: observation data extraction means extracting observation data that serves as a target candidate from image data acquired from an imaging sensor; gating processing means narrowing down the extracted observation data to observation data in a gate area near a prediction position in a next frame of a target; correlation processing means associating each tracked target with any of the observation data obtained by narrowing down the extracted observation data; tracking filter means estimating a position of the target and dispersion in a current frame on the basis of the observation data associated with the target and process noise, and predicting a position of the target and dispersion in a next frame; and process noise determination means making the process noise greater if a luminance value of the target is higher. The gate area is set wider as the process noise is greater, and the gating processing means narrows down the extracted observation data to observation data exceeding a luminance threshold corresponding to the luminance value of the target in the gate area.

Description

  The present invention relates to a target tracking device and a target tracking program.

  The target tracking device is a device that tracks a target in image data captured by an image sensor. For example, an aircraft monitors the arrival of a target that is another aircraft from a long distance. For this purpose, the target tracking device images a plurality of targets at a long distance using an imaging sensor such as an infrared sensor, and tracks the movement of the target. One pixel with high luminance in the captured image data is observed as a point target, and the luminance of the point target pixel gradually increases as it approaches.

  The target tracking method is described in Patent Documents 1 and 2, for example.

  In the target tracking device, observation data including the position in the target target image and its brightness are extracted from the image data captured by the image sensor, and the target prediction predicted in the previous frame for each target is extracted. The observation data in the gate region centered on the position is narrowed down as the observation data to be correlated. Then, each target estimated in the previous frame is associated with the observation data extracted and narrowed down in the current frame. In this association, assignment is made to each target so that the observation data does not overlap, and the combination of assignments that maximizes the likelihood is selected from a plurality of assignment combinations. Finally, a tracking filter such as a Kalman filter estimates the position, velocity, and variation of each target from the predicted position of the target and the observed data in the current frame based on the association between each target and the observed data. Based on the above, the predicted position of the target in the next frame is predicted. In the next frame, based on the predicted position of the target, the observation data is narrowed down, correlated, and estimated and predicted by the tracking filter, and these processes are repeated for each frame.

JP 2012-127890 A JP-A-5-303624

  In order to track an irregularly moving target, it is necessary to increase the process noise of the Kalman filter and track the irregular target. When the process noise is increased, the gate area for narrowing the observation data in the next frame becomes wider. This is because the gate region is determined based on a normalized distance measure called Mahalanobis distance obtained based on process noise.

  However, if the gate area becomes wider, observation data with high brightness generated by noise is likely to enter the gate area in addition to the observation data for the target, and the number of observation data that are candidates for association increases and the amount of computation for association increases. At the same time, it becomes difficult to separate the observation data corresponding to the target from the observation data generated by noise, and the number of errors in association increases. Therefore, there is a trade-off between widening the gate region and reducing the amount of calculation of association and increasing the accuracy of association.

  Therefore, an object of the present invention is to provide a target tracking device and a target tracking program that can suppress the increase in the amount of calculation and increase the detection accuracy so that the number of candidate observation data does not increase even if the process noise is increased. There is to do.

The first aspect of the target tracking device according to the embodiment is as follows:
Observation data extraction means for extracting observation data as target candidates from image data acquired from an image sensor;
Gating processing means for narrowing down the extracted observation data to observation data in the gate region near the predicted position in the next frame of the target;
Correlation processing means for associating each tracking target with one of the narrowed observation data;
Tracking filter means for estimating the position and variation of the target in the current frame based on observation data associated with the target and process noise, and predicting the variation in position of the target in the next frame;
Process noise determining means for increasing the process noise as the target luminance value is higher,
The larger the process noise, the wider the gate region is set.
The gating processing means narrows down to observation data exceeding a luminance threshold value corresponding to the target luminance value in the gate region.

  According to the first aspect, even if the process noise is increased, an increase in calculation amount can be suppressed and detection accuracy can be increased.

It is a block diagram of the target tracking apparatus in this Embodiment. It is a figure which shows schematic structure of a target tracking apparatus. It is a flowchart figure which shows the outline of a process of a target tracking program. It is a figure which shows the example of observation data. It is a flowchart figure of a gating process means. It is a figure explaining the process by a gating process means and a correlation process means. It is a flowchart figure of the process of the tracking filter means 244. It is a figure explaining the process of a tracking filter means. FIG. 6 is a diagram showing a residual d (t) between an observation position z (t) and an observation prediction position Hx− (t). It is a block diagram of the target tracking apparatus in this Embodiment. It is a flowchart figure of the target tracking program in this Embodiment. It is a flowchart figure of the process by a brightness | luminance prediction filter means. It is a figure explaining the determination of the brightness | luminance threshold value Vth by the determination means of a brightness | luminance threshold value. It is a flowchart figure of the process by a process noise determination means. It is a figure explaining calculation of the probability P (Vth) that observation data exceeds the brightness | luminance threshold value Vth. It is a flowchart figure of the process of the gating process means in this Embodiment.

  FIG. 1 is a configuration diagram of a target tracking device according to the present embodiment. The target tracking device 20 acquires image data from an imaging sensor such as an infrared ray, tracks a target in the image data, and displays the target on the display device 30. In the target tracking device 20, a CPU 21, which is a processor, a memory 22, an input / output device 23, and a recording medium 24 for storing a target tracking program are connected via a bus.

  The CPU 21 executes the target tracking program, analyzes the image data, and executes processing for tracking the target.

  FIG. 2 is a diagram showing a schematic configuration of the target tracking device. FIG. 3 is a flowchart showing an outline of the processing of the target tracking program. The outline of the target tracking process will be described with reference to these.

  The imaging sensor 10 captures a plurality of targets approaching from a long distance and outputs image data for each frame. This image data has luminance values of a plurality of pixels. The target is observed with a size of one pixel, and the luminance of the pixel where the target exists is high, and is called a point target. As the point target approaches, the brightness gradually increases. The point target approaches as the pixel position changes.

  As shown in FIG. 2, the target tracking device includes observation data extraction means 241, gating processing means 242, correlation processing means 243, and tracking filter means 244. The observation data extraction unit 241 acquires the image data captured by the image sensor 10, and extracts the position of the point target image on the target candidate image and its luminance as observation data. The point target is a bright pixel with high luminance as described above. However, the extracted observation data includes actual target point targets and point targets generated by noise.

  FIG. 4 is a diagram illustrating an example of observation data. The observation data includes a plurality of bright pixels Z in a two-dimensional image of the X and Y axes. FIG. 4 includes point target pixels Z1 to Z7 of high luminance pixels that are target candidates and low luminance pixels Zn. The observation data extraction means 241 extracts point targets Z1-Z7 from these pixels. FIG. 4 shows target T1 (t) and T2 (t) detected in the previous frame.

  Next, a target in the current frame image for the targets T1 (t) and T2 (t) detected in these previous frames is selected from the extracted target target point targets Z1-Z7, and Are associated with target candidates in the current frame image. However, if each target is associated with all observation data, the number of point targets that are target candidates increases, and the amount of calculation for the association increases. Therefore, the number of target candidates is reduced by narrowing down observation data extracted by the gating means.

  FIG. 5 is a flowchart of the gating processing means. FIG. 6 is a diagram for explaining processing by the gating processing means and the correlation processing means. As shown in FIG. 5, the gating processing means 242 performs observation in the gate region G centered on the predicted position with respect to the target from the extracted observation data Z1-Z7 for each target T1 (t), T2 (t). A process of narrowing down to data is performed (S11, S12 in FIG. 5).

  As shown in FIG. 6, the target candidate in the current frame (t + 1) with respect to the target T1 (t) in the previous frame is a gate centered on the predicted position T1 (t + 1) with respect to the target T1 (t). It is narrowed down to point targets Z1, Z2, and Z3 of observation data in region G1. Further, the target candidates for the target T2 (t) are narrowed down to the point targets Z2 and Z3 of the observation data in the gate region G2 centered on the predicted position T2 (t + 1) with respect to the target T2 (t). Then, the point targets Z4-Z7 are excluded from the target candidates. The predicted positions T1 (t + 1) and T2 (t + 1) are obtained by using the target filter T1 (t), T2 (t) estimated by the tracking filter unit 244 described later in the previous frame (t) based on, for example, constant velocity linear motion. ) By calculation. The sizes of the gate regions G1 and G2 are obtained as regions where the Mahalanobis distance obtained from the residual covariance matrix S obtained from the process noise Q, which will be described later, is below a certain threshold. The gate regions G1 and G2 shown in FIG. 6 are not a perfect circle but an ellipse.

  The correlation processing means 243 determines the target candidates Z1, Z2, Z2 of the plurality of observation data in the current frame (t + 1) from the positional relationship between the positions of the target candidates Z1, Z2, Z3 of the observation data in the gate region and the predicted positions. Decide which target in the previous frame (t) to associate with Z3. In the example of FIG. 6, the observation data with the highest likelihood is selected from the observation data Z1, Z2, and Z3 of the target candidate for the target T1 (t), and is associated or associated with the target T1 (t). Similarly, the observation data with the highest likelihood is selected from the observation data Z2 and Z3 of the target candidate for the target T2 (t), and is associated with or associated with the target T2 (t).

  The correlation processing means 243 ensures that the narrowed observation data Z1, Z2, and Z3 in the current frame (t + 1) do not overlap with each target T1 (t) and T2 (t) in the previous frame, and , One observation data is assigned. That is, a plurality of combinations of observation data assigned to each target are obtained by calculation, and the combination with the maximum likelihood is selected from the plurality of combinations. This selection is performed by calculating the likelihood that each observation data is observed for each target, and selecting the one that maximizes the likelihood among the combinations of target and observation data assignments. . The likelihood increases as the distance between the predicted position and the position of the observation data (Mahalanobis distance) decreases.

  For correlation processing, methods such as GNN, JPDA, and MHT are often used. For example, MHT (Multiple Hypothesis Tracking) generates hypotheses that assign observation data in the gate region to the target. These hypotheses are held for several consecutive times (several frames). Which hypothesis to select is determined retrospectively after obtaining observation data for the number of steps specified in advance.

  When new observation data is narrowed down and input, a plurality of hypotheses are further generated under the plurality of hypotheses generated and held for the previous observation data, and these are held. When the above processing is executed every time new observation data is input, the number of hypotheses increases exponentially as the number of input observation data increases. Therefore, when the number of observation data inputs reaches a predetermined number n, a hypothesis is selected by going back the time (frame) n-1 times, and the hypothesis that has not been selected is discarded. Hypothesis selection, as described above, compares the likelihood calculated from the hypothesis likelihood after inputting the latest observation data among multiple hypotheses generated n-1 times earlier, and calculates the maximum likelihood. This is done by selecting a hypothesis having

  In this way, in the correlation processing by MHT, (1) generation of a plurality of hypotheses to assign (associate) observation data to a target, (2) the likelihood of each hypothesis is calculated using the likelihood of the hypothesis generated last time, (3) A process of selecting a hypothesis by going back the history of hypotheses generated so far by n steps and discarding a non-selected hypothesis is executed. A detailed description of the processing is well known and will be omitted.

  FIG. 7 is a flowchart of processing of the tracking filter unit 244. FIG. 8 is a diagram for explaining the processing of the tracking filter means. As shown in FIG. 7, the tracking filter unit 244 calculates the difference between the target data T1 (t) and T2 (t) in the previous frame (time t) and the observation data of the current frame (latest frame, time t + 1). The target position, velocity, and variation (covariance matrix) in the current frame (t + 1) are estimated using the Kalman filter, for example, for the association result (S21). This estimation is also referred to as fiber to current frame, update (or correction of predicted value). Further, the tracking filter means uses the estimated information to calculate and predict the target predicted position, predicted speed, and variation (covariance matrix) in the next frame (t + 2) (S22).

  When the Kalman filter uses a constant velocity model in which the target moves linearly at a constant velocity, the model is modeled as follows, and prediction processing and estimation (update) processing are performed.

When using the constant velocity model, the following transition model (modeled on the image coordinates with the state quantity as the image coordinate, the target state quantity (position, velocity) and variation (error covariance matrix) transition from frame to frame. Model) and an observation model (a model of an observation amount (position) modeled on the image coordinate and obtained from the state quantity, with the observation amount as an image coordinate). The Kalman filter is an algorithm for estimating a state quantity based on the obtained observation quantity while noise is on each of the state quantity and the observation quantity, and uses two models, a transition model and an observation model.
Transition model: x (t + 1) = Fx (t) + w (t) (11)
Observation model: z (t) = Hx (t) + v (t) (12)
Here, each variable is defined as follows.
x (t): State quantity (position, speed)

z (t): Observed quantity (position)

w (t): Process noise (transition noise) (normal distribution with mean value 0 and covariance matrix Q (t))
v (t): Observation noise (normal distribution with mean value 0 and covariance matrix R (t))
F: State transition matrix, state transition matrix F transitions state quantity x (t) to state quantity x (t + 1). Here, transition means time transition.

H: Observation matrix, observation matrix H maps state quantity x (t) to observation quantity z (t).

  Therefore, the transition model of Equation (11) and the observation model of Equation (12) are as follows.

  Furthermore, the model of process noise Q in constant speed motion is as follows.

Here, dt is a time step, and qx and qy are process noise magnitudes in the X and Y directions, respectively.

  The tracking filter means prediction process S22 shown in FIG. 7 is a process for obtaining the state prediction value of the next frame (t + 1) from the state estimation value of the current frame (t). Specifically, in the following equation (1), the estimated value of the state quantity x (t) of the current frame (t) is multiplied by the state transition matrix F to obtain the state quantity x of the next frame (t + 1). The prediction value of (t + 1) is obtained, and the error in the next frame (t + 1) is calculated from the error covariance matrix P (t) and the process noise Q in the current frame (t) using Equation (2). A predicted value of the covariance matrix P (t + 1) is obtained. Note that the error covariance matrix P (t) indicates the covariance of the state quantity and indicates how much the state quantity varies.

  Next, the tracking filter means estimation (update) process S21 is a process for obtaining the estimated value of the current frame from the observation data of the current frame (t) and the predicted value predicted from the previous frame by the following equation.

  Specifically, in the above equation (3), the residual covariance matrix S (t) at the current frame (t) is obtained from the predicted value of the error covariance matrix and the covariance matrix R of the observed noise. The residual covariance matrix S (t) is the covariance matrix of the residual d (t), which shows the variation in error from the observed observable, and the predicted value of the state variable covariance matrix It is obtained by adding the covariance matrix of the observation noise to the one mapped to the observation space. Further, the Kalman gain K is obtained from Equation (4).

Then, in Formula (5), the state estimated value x of the current frame (t) ^- a (t), the observation position z (t) and the observed predicted position Hx ^- the residuals (t) (d (t)) The state estimated value x ^ (t) is updated by adding the value multiplied by the Kalman gain K. Furthermore, in equation (6), the predicted value P of the error covariance matrix of the current frame (t) ^- from (t), the observation prediction value HP ^- by subtracting the value obtained by multiplying the Kalman gain K in (t) , Find the error covariance matrix P (t).

9, the observation position z (t) and observation prediction position Hx ^- is a diagram showing a residual d (t) of (t). As shown in the equation (5), by multiplying the residual d (t) by the Kalman gain K and adding it to the predicted value x ⁻ (t), the state estimated value x ^ (t) is obtained.

As shown in FIG. 7, the tracking filter means firstly, the observation data z (t) of the current frame associated with the target of the previous frame, and the predicted value x ⁻ (t) of the current frame predicted from the previous frame. Then, an estimation (update) process S21 for obtaining an estimated value is performed. Then, a prediction process S22 for generating predicted values x ⁻ (t + 1) and P ⁻ (t + 1) used in the gating process in the next frame (t + 1) from the estimated value is performed.

Referring to the drawing explaining the processing of the tracking filter means in FIG. 8, the estimation (update) processing S21 and the prediction processing S22 by the tracking filter means described above will be clarified. That is, the state estimation value x ^ (t-1) and the error covariance matrix for the target T1 (t-1) in the previous frame (t-1) are estimated by the prediction process in the previous frame (t-1). From the value P (t−1), the state predicted value x ⁻ (t) and the error covariance matrix predicted value P ⁻ (t) at the current frame (t) are obtained.

And by the estimation (update) processing in the current frame (t), the observation data z (t) of the current frame (t) associated with the target T1 (t-1) of the previous frame and the prediction from the previous frame state estimated value x of the present frame ^- (t) predicted value P and the error covariance matrix ^- since (t), the state estimate x ^ (t), obtaining an estimate of the error covariance matrix P (t). Then, prediction processing is performed at the same time, and prediction values x ⁻ (t + 1), P used for gating processing etc. in the next frame (t + 1) from the estimated values x ^ (t), P (t) ^-Generate (t + 1).

  Further, the same estimation process is performed for the next frame (t + 1), and the prediction process for obtaining the predicted value for the next frame (t + 2) is performed.

  With reference to the flowchart of the target tracking program shown in FIG. 3, the outline of the target tracking process is again organized as follows. First, observation data is extracted in the first frame (time), and a target is set from a point target with high brightness (S1). Then, in the next frame (time +1) (S2), observation data is extracted (S3), candidate observation data is narrowed down to observation data in the gate region by gating processing (S4), and target processing is performed by correlation processing. And observation data are associated (S5). Further, the tracking filter means estimates the state quantity x (t) and error covariance matrix P (t) in the current frame, and the state quantity x (t + 1) and error covariance matrix P ( t + 1) is predicted (S6). Then, in the next frame (time t + 1) (S2), the above processing is repeated using the predicted value obtained in the previous frame (t).

The above Kalman filter is a generally known technique, and is described in, for example, the following documents.
http://en.wikipedia.org/wiki/%E3%82%AB%E3%83%AB%E3%83%9E%E3%83%B3%E3%83%95%E3%82%A3%E3 % 83% AB% E3% 82% BF% E3% 83% BC
[Target tracking processing in this embodiment]
The target tracking device in the present embodiment increases the process noise Q of the Kalman filter that constitutes the tracking filter so that a target with irregular motion can be properly tracked. That is, as shown in the transition model, observation model, prediction process, and estimation (update) process described above, the target of irregular motion can be tracked by increasing the process noise Q.

  However, increasing the process noise Q increases the gate area. As a result, (1) in the next frame, the number of observation data narrowed down by the gate area near the predicted position increases, and the increase in the number of observation data that can be candidates for correlation processing increases the amount of calculation for association However, (2) it becomes difficult to discriminate between observation data generated from noise and the target, and errors in association increase. In this way, when tracking a target that moves irregularly, increasing the process noise Q increases the amount of computation and errors.

  Hereinafter, the relationship between the process noise Q and the gate region will be described.

  In the Kalman filter, it is assumed that the target of the next frame is observed according to the center of the following predicted position (formula (7)) and the normal distribution variation with the residual covariance matrix as S (t). Therefore, if the residual between the observation position z (t) and the predicted position (formula (7)) is d (t) (formula (8)) below, the following Mahalanobis distance (formula (9)) is It is a scale that represents the proximity to equation (7). Therefore, a region where the Mahalanobis distance (equation (9)) is equal to or less than the threshold value T is used as the gate region. That is, the boundary of the gate region becomes an ellipse whose Mahalanobis distance (Equation (9)) is the threshold T from the predicted center position (Equation (7)). This ellipse is as shown in FIG.

As described above, the Mahalanobis distance (formula (9)) is calculated using the residual covariance matrix S (t). The residual covariance matrix S (t) is calculated based on the process noise covariance matrix Q as shown in equations (2) and (3). Therefore, when the process noise covariance matrix Q increases, the prediction value of the error covariance matrix P (t) also increases according to equation (2), and the residual covariance matrix S (t) also increases according to equation (3). (The reciprocal S (t) ^-1 becomes smaller). The Mahalanobis distance (Equation (9)) is obtained by normalizing the residual d (t) of the distance with the residual covariance matrix S (t), so if the residual covariance matrix S (t) is large, , Mahalanobis distance (equation (9)) becomes smaller. As a result, the gate region where the Mahalanobis distance (equation (9)) is equal to or smaller than the threshold value T is widened.

Therefore, the target tracking device in the present embodiment adjusts the process noise Q as follows in accordance with the magnitude of the target luminance value in order to appropriately track a target that performs irregular motion.
(1) If the target luminance value is large, increase the process noise Q. As a result, the gate region becomes wider. If the target luminance value is small, the process noise Q is reduced. Accordingly, the gate region becomes narrower. Desirably, the magnitude of the process noise Q is determined so that the number of observation data to be associated among the observation data generated by noise in the gate region is equal to or less than a certain number.
(2) Observation data having a low luminance value (below the luminance threshold Vth) having a certain difference or more from the target luminance value being tracked is excluded from the observation data to be associated.

  In general, the brightness value increases as the target gets closer. If the luminance value is large, even if a low luminance value (below the luminance threshold Vth) having a difference of a certain value or more from the luminance value is excluded from the association candidates, there is little error in tracking. Therefore, when the target luminance value is large, the process noise Q is increased so that the target that performs irregular motion can be appropriately tracked. However, since the gate area becomes wide, the observation data in the gate area is excluded from the observation data with a low luminance value (below the luminance threshold Vth) that differs from the target luminance value by a certain value or more, and the association process is performed. . As a result, the adverse effect of widening the gate region due to the increased process noise Q is suppressed.

  FIG. 10 is a configuration diagram of the target tracking device according to the present embodiment. The target tracking device of FIG. 10 is a luminance predictor in addition to the observation data extraction means 241, the gating processing means 242, the correlation processing means 243, and the tracking filter means 244 shown in the schematic configuration of the target tracking device of FIG. Filter means 245, luminance threshold value determining means 246, and process noise determining means 247 are provided. Then, the brightness prediction filter means 245 predicts the target predicted brightness value V and its variance (variation) σ (V) in the next frame by the same Kalman filter as the tracking filter means 244, and the brightness threshold value determination means 246 , The luminance threshold value Vth is determined from the predicted luminance value V and its variance σ (V), and the process noise determining unit 247 determines the process noise Q from the predicted luminance value V and the luminance threshold value Vth, and sends it to the tracking filter unit 244. Set the process noise Q.

  As described above, the process noise determination unit 247 increases the process noise Q when the target luminance value is large, and enables the tracking filter unit 244 to track a target that performs irregular motion. That is, the tracking filter unit 244 performs a filter process corresponding to the process noise Q determined by the process noise determination unit 247. However, since the gate area is increased by increasing the process noise Q, the gating processing unit 242 excludes the observation data below the luminance threshold Vth from the candidate observation data to be associated with the association processing target, Preferably, the number of observation data should be constant so that it does not increase excessively.

  The observation data extraction unit 241, the gating processing unit 242, the correlation processing unit 243, and the tracking filter unit 244 in FIG. 10 perform the same processes as those described in FIG. However, unlike FIG. 2, the tracking filter means 244 performs the filtering process corresponding to the process noise Q determined by the process noise determination means 247, and the gating means 242 transmits the observation data below the luminance threshold Vth. It differs from Fig. 2 in that it is excluded from the candidate observation data.

  FIG. 11 is a flowchart of the target tracking program in the present embodiment. The processes S1, S2, S3, and S5 in the flowchart of FIG. 11 are the same as the processes S1, S2, S3, and S5 of FIG. 2, but the processes S7, S8, and S9 of FIG. 11 are added to FIG. The processes S4X and S6X in FIG. 11 are partially different from the processes S4 and S6 in FIG.

  In the target tracking process by the target tracking program of FIG. 11, first, observation data is extracted at the first frame (time), and a target is set from a point target with high luminance (S1). Then, in the next frame (time +1) (S2), the observation data is extracted (S3), and the observation data in the gate region by the gating process has a candidate for the observation data having a luminance value higher than the luminance threshold Vth. It is narrowed down (S4X), and correlation of observation data narrowed down to the target is performed by correlation processing (S5).

  Next, the luminance prediction filter means 245 estimates the luminance value from the predicted luminance value predicted in the previous frame and the luminance value observed in the current frame in the same manner as the Kalman filter of the tracking filter, and the luminance value in the next frame Is predicted (S7). This estimation and prediction is also performed for the amount of change in the luminance value and the variance (variation) σ (V) of the luminance value. Then, the luminance threshold Vth is determined by the luminance threshold determining means 246. Further, the process noise determining means 247 determines the process noise Q using the predicted luminance value V and the luminance threshold value Vth (S9). Desirably, the process noise Q is set so that the area A of the gate region when the gating means 242 narrows down the observation data having a luminance value exceeding the luminance threshold Vth is an area where the number of observation data to be narrowed down is a certain number. To decide.

  Further, the tracking filter means estimates the state quantity (position and velocity) and error covariance matrix in the current frame based on the determined process noise Q, and obtains the state quantity and error covariance matrix in the next frame. Predict (S6X). Then, in the next frame (time) (S2), the above processing is repeated using the predicted value obtained in the previous frame.

  FIG. 12 is a flowchart of processing by the luminance prediction filter means. The luminance prediction filter unit 245 is a Kalman filter in which the state quantity in the tracking filter unit 244 is replaced with a target luminance value and a luminance change rate. Therefore, the luminance prediction filter means estimates the target luminance value in the current frame from the target predicted luminance value and the observed luminance value associated with the target (S31). In addition to the luminance value, the luminance change speed and variation are also estimated. Further, the brightness prediction filter means predicts the predicted brightness value, predicted brightness change speed, and variation in brightness value (variance σ (V)) of the next frame from the estimated target brightness value and brightness change speed (S32). .

  FIG. 13 is a diagram for explaining determination of the luminance threshold Vth by the luminance threshold determination means. FIG. 13 shows the distribution of target luminance values in the next frame, with the horizontal axis representing the luminance and the vertical axis representing the probability of the luminance on the horizontal axis. The luminance threshold value determining means 246 determines the luminance threshold value Vth of the observation data associated with the target in the next frame. In the next frame, the gating means 242 narrows the observation data in the gate region that exceeds the luminance threshold Vth as candidate observation data associated with the target.

As shown in FIG. 13, the luminance threshold value determining unit 246 assumes a normal distribution in which the predicted luminance V predicted by the luminance prediction filter unit 245 is an average value, and the predicted luminance variation is variance σ (V), and the normal distribution A luminance threshold Vth at which the upper probability is 99% is calculated. That is, the brightness threshold value Vth is as follows.
Vth = V−2.32 × σ (V)
By defining the brightness threshold Vth according to the magnitude of the predicted brightness V in this way, the observed data narrowed down from the observed data observed in the next frame is used as the average value of the predicted brightness V and the variance of σ (V) 99% of luminance values in the normal distribution having It is desirable to fix the number of observation data to be narrowed to a constant N.

  FIG. 14 is a flowchart of processing by the process noise determining means. The process noise determining means 247 determines the process noise Q using the predicted luminance value V and the luminance threshold value Vth (S9). That is, the process noise Q (qx, qy) is determined to increase as the predicted luminance value V increases. Accordingly, the tracking filter unit 244 performs estimation (update) and prediction based on the large process noise Q, and therefore can track a target that moves irregularly. However, if the process noise Q is increased, the gate region becomes larger, so the gating means 242 narrows the observation data that exceeds the luminance threshold Vth from the observation data in the gate region as candidate observation data.

  Preferably, the process noise determining unit 247 preferably has an area where the area A of the gate region when the gating unit 242 narrows down observation data having a luminance value exceeding the luminance threshold Vth is a certain number. Determine the process noise Q to be Hereinafter, the process for determining the process noise by this desirable method will be described with reference to the flowchart of FIG.

  The process noise determining means 247 sets the process noise qx, qy to the maximum values qxmax, qymax (S41). Next, the process noise determining means calculates a probability P (Vth) that the observation data generated based on the noise exceeds the luminance threshold value Vth (S42).

  FIG. 15 is a diagram for explaining the calculation of the probability P (Vth) that the observation data exceeds the luminance threshold Vth. Assume a model where the noise is white noise. When the noise is white noise, the model has a normal distribution with the variance Is centered on the background luminance Vbg, which is an ideal luminance without white noise. That is, the distribution of luminance values of all observation data imaged by the imaging sensor is the normal distribution of FIG. On the other hand, in such a luminance value distribution model, the probability P (Vth) exceeding the luminance threshold Vth is given as an upper probability that is (Vth−Vbg) / Is or higher in the normal distribution.

Next, the process noise determining means calculates the area A of the gate region where the expected value of the number of observation data exceeding the luminance threshold Vth among the observation data generated from the noise is a constant value N (S43). .
P (Vth) × A = N
Therefore, A = N / P (Vth)
Then, the process noise determining means obtains the area B of the gate region from the currently set process noises qx and qy (S44). In this calculation, the covariance matrix S (t) is obtained from the process noise Q (qx, qy) according to Equations (2) and (3), the Mahalanobis distance is obtained, and the area of the gate region where the Mahalanobis distance is the threshold T Ask for.

  The process noise determining means multiplies the process noise qx, qy by a coefficient less than 1 to reduce qx, qy until the area B becomes smaller than the area A (YES in S45) (S46), and processes S44, S45. repeat. When area B <area A (YES in S45), the process noise determining means determines and outputs the current process noise qx, qy as process noise (S47).

  In the processing shown in FIG. 15, the larger the predicted luminance value V, the larger the luminance threshold Vth, and the probability P (Vth) exceeding the luminance threshold Vth in FIG. 15 decreases, and as a result, the area A of the gate region increases. Become. Accordingly, the process noise Q (qx, qy) of the area B corresponding to the area A of the gate region is also largely determined. Conversely, the smaller the predicted luminance value V is, the smaller the luminance threshold Vth is, and the probability P (Vth) is increased. As a result, the area A of the gate region is decreased. Accordingly, the process noise Q (qx, qy) of the area B corresponding to the area A of the gate region is also determined to be small.

  That is, the process noise determining means determines the process noise Q (qx, qy) to be a large value when the predicted luminance value V increases, and determines the process noise Q (qx, qy) to be a small value when the predicted luminance value V decreases. To do.

  Furthermore, the process noise determining means preferably uses the distribution model of the luminance value of white noise in FIG. 13 in order to fix the number of candidate observation data narrowed down by the gating processing means to a constant value N, Determine process noise.

  FIG. 16 is a flowchart of processing of the gating processing means in the present embodiment. The gating processing means narrows down the observation data acquired by the imaging sensor to observation data having a luminance value exceeding the luminance threshold Vth in the gate region having an area corresponding to the determined process noise centered on the predicted position with respect to the target. (S51). This process is performed for all targets (S52).

  If the predicted brightness value V is high, the brightness threshold Vth is also increased, and even if the process noise Q (qx, qy) is set large and the gate area is widened, the number of candidate observation data to be narrowed down by the gating processing means is small. It won't be too big. Conversely, if the predicted luminance value V is low, the luminance threshold Vth is also low, and if the process noise Q (qx, qy) is set small and the gate area becomes narrow, the gate processing means is narrowed down by the luminance threshold Vth. Is low and the number of candidate observation data is sufficient.

  The processing of the tracking filter unit shown in FIG. 7 is similarly performed in the present embodiment. Since the estimation (update) and prediction by the tracking filter means are performed by the error covariance matrix P (t) obtained based on the process noise Q, if the process noise Q is set large, the error covariance matrix P (t ) Is set to be large, and the target tracking with irregular motion can be appropriately performed by the estimation and prediction processing according to equations (1) to (6).

  In the above embodiment, the Kalman filter in which the target moves linearly as an example of the tracking filter means has been described, but it goes without saying that an extended Kalman filter in which the target moves nonlinearly may be used.

  As described above, according to the target tracking device and the target tracking program of the present embodiment, target tracking with irregular movement can be performed while suppressing an increase in the amount of calculation while suppressing the probability of erroneous determination. it can.

  The above embodiment is summarized as follows.

(Appendix 1)
Observation data extraction means for extracting observation data as target candidates from image data acquired from an image sensor;
Gating processing means for narrowing down the extracted observation data to observation data in the gate region near the predicted position in the next frame of the target;
Correlation processing means for associating each tracking target with one of the narrowed observation data;
Tracking filter means for estimating the position and variation of the target in a current frame based on the observation data associated with the target and process noise, and predicting the position and variation of the target in the next frame;
Process noise determining means for increasing the process noise as the target luminance value is higher,
The larger the process noise, the wider the gate region is set.
The target tracking device, wherein the gating processing means narrows down observation data exceeding a luminance threshold corresponding to the target luminance value in the gate area.

(Appendix 2)
In Appendix 1,
The process noise determination unit is a target tracking device that determines the process noise based on the luminance threshold value so that the number of observation data to be narrowed is constant even when the luminance threshold value changes.

(Appendix 3)
In Appendix 2,
Brightness value prediction filter means for predicting the target brightness value;
Brightness threshold determination means for determining the brightness threshold higher as the predicted brightness value is higher,
The process noise determination means is a target tracking device that increases the process noise as the predicted luminance value increases.

(Appendix 4)
A target tracking program for causing a computer to execute target tracking processing for tracking a target for image data acquired from an imaging sensor,
The target tracking process is:
An observation data extraction process for extracting observation data as target candidates from image data acquired from an image sensor;
A gating process that narrows down the extracted observation data to observation data in the gate region near the predicted position in the next frame of the target;
Correlating each tracking target with one of the narrowed observation data;
A tracking filter step of estimating the position and variation of the target in the current frame based on the observation data associated with the target and process noise, and predicting the position and variation of the target in the next frame;
A process noise determination step of increasing the process noise as the target luminance value is higher,
The larger the process noise, the wider the gate region is set.
In the gating processing step, a target tracking program that narrows down to observation data that exceeds a luminance threshold corresponding to the target luminance value in the gate region.

(Appendix 5)
In Appendix 4,
In the process noise determination step, a target tracking program for determining the process noise based on the luminance threshold value so that the number of observation data to be narrowed is constant even if the luminance threshold value changes.

(Appendix 6)
In Appendix 5,
A luminance value prediction filter step for predicting the target luminance value;
A luminance threshold value determining step of determining the luminance threshold value higher as the predicted luminance value is higher;
In the process noise determination step, a target tracking program for increasing the process noise as the predicted luminance value increases.

241: Observation data extracting means 242: Gating processing means 243: Correlation processing means 244: Tracking filter means 245: Luminance prediction filter means 246: Luminance threshold determining means 247: Process noise determining means

Claims (4)

Observation data extraction means for extracting observation data as target candidates from image data acquired from an image sensor;
Gating processing means for narrowing down the extracted observation data to observation data in the gate region near the predicted position in the next frame of the target;
Correlation processing means for associating each tracking target with one of the narrowed observation data;
Tracking filter means for estimating the position and variation of the target in a current frame based on the observation data associated with the target and process noise, and predicting the position and variation of the target in the next frame;
Process noise determining means for increasing the process noise as the target luminance value is higher,
The larger the process noise, the wider the gate region is set.
The target tracking device, wherein the gating processing means narrows down observation data exceeding a luminance threshold corresponding to the target luminance value in the gate area. In claim 1,
The process noise determination unit is a target tracking device that determines the process noise based on the luminance threshold value so that the number of observation data to be narrowed is constant even when the luminance threshold value changes. In claim 2, further,
Brightness value prediction filter means for predicting the target brightness value;
Brightness threshold determination means for determining the brightness threshold higher as the predicted brightness value is higher,
The process noise determination means is a target tracking device that increases the process noise as the predicted luminance value increases. A target tracking program for causing a computer to execute target tracking processing for tracking a target for image data acquired from an imaging sensor,
The target tracking process is:
An observation data extraction process for extracting observation data as target candidates from image data acquired from an image sensor;
A gating process that narrows down the extracted observation data to observation data in the gate region near the predicted position in the next frame of the target;
Correlating each tracking target with one of the narrowed observation data;
A tracking filter step of estimating the position and variation of the target in the current frame based on the observation data associated with the target and process noise, and predicting the position and variation of the target in the next frame;
A process noise determination step of increasing the process noise as the target luminance value is higher,
The larger the process noise, the wider the gate region is set.
In the gating processing step, a target tracking program that narrows down to observation data that exceeds a luminance threshold corresponding to the target luminance value in the gate region.

Images