JPH04279983A - Adaptive type state vector estimation processing system - Google Patents

Abstract

PURPOSE:To estimate a state vector from an observation vector having a two-dimensional array with a high precision by selecting and outputting one of state vectors of a specific Kalman filter. CONSTITUTION:The observation vector obtained by observation is successively inputted by an input means 100, and a state vector is estimated from the observation vector by a Kalman filter 200 and is outputted. A state vector selecting means selects one of state vectors which minimizes the difference between the estimate value of the observation vector and the inputted observation vector. That is, the Kalman filter is defined in each of directions where coordinate indicating the positions on the two-dimensional array as the estimate object and one of nearby coordinates already subjected to state vector estimate processing are connected, and state vectors are estimated respectively. In this case, the difference between the observation vector estimate value obtained in the calculation process and the successively inputted observation vector is calculated, and one state vector in the direction where this difference is minimum is selected.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention is applicable to CC
The present invention relates to an adaptive state vector estimation processing method for estimating the state vector of an object from observation vectors obtained from sensors such as D cameras and range sensors.

Specifically, the distance to the object is estimated by observing the movement (optical flow) of the object from a camera, or the inclination of the surface is estimated by observing the distance from a range sensor. becomes possible.

[0003] Computer vision is used to estimate three-dimensional motion, three-dimensional shape, etc. from two-dimensional data (range data, image data).

0004

2. Description of the Related Art A Kalman filter is a filter algorithm proposed based on the state space representation of a linear stochastic system and the theory of orthogonal projection. References [1] (
Toru Katayama “Applied Kalman Filter”, Asakura Shoten, 1983
). The popularization of Kalman filters was largely due to the efforts of many researchers to apply Kalman filters and optimal control theory in space development programs centered on the US Apollo program in the 1960s. Currently, applications of Kalman filters are found not only in space engineering, control engineering, and communication engineering, but also in many fields such as civil engineering, economics, and operations research. To summarize, the Kalman filter uses (1) the dynamic characteristics of the system that generates the signal, (2) the statistical properties of the noise, (3) a priori information about the initial value, and observation data given from time to time to evaluate the system. It can be said to be an online data processing algorithm that sequentially provides least squares estimates of states. In the above application fields, most Kalman filters use one-dimensional time-series observation data. However, when dealing with observation data that has a two-dimensional array such as an image, it is not possible to perform highly accurate estimation that takes advantage of the two-dimensional structure using conventional methods.

[0005] In the prior art, a Kalman filter is known that uses correlation coefficients between pixels for the purpose of restoring an image, which is used for observation data in a two-dimensional array. Reference [2] (Ali Habibi,
“Two-dimensional Bayesia
n estimate of images”, Pro
ceedings of the IEEE, vol.
．． 60, No. 7, July 1972).

[0006] This principle and drawbacks will be explained below. FIG. 6 shows a layout diagram of sample points. In FIG. 6, the horizontal coordinate of the sample point is j, and the vertical coordinate is i, which is expressed as (i, j). The sample points are arranged in the direction of (0,1), (0,2) from the coordinates (0,0), and the estimated state vector value at the current (i+1, j+1) is determined. This method can be formulated as follows as a system that receives a p-dimensional observation vector y (i, j) and estimates an n-dimensional state vector x (i, j).

[0007]

[Equation 1] Here, Ht and Dt are p×n and p×p matrices that depend only on time t, respectively, wt and vt are m-dimensional and p-dimensional Gaussian white noise vectors, respectively, ρ1 and ρ2
are correlation coefficients between vertically and horizontally adjacent pixels, and are predetermined constants.

A Kalman filter can be applied to a system described as a linear probability system as described above. In this conventional method, the average state vector of three neighboring state vectors for which estimation processing has already been completed for the target sample is calculated, and a one-dimensional Kalman filter is operated using this value as the state vector of one sample before. The one-dimensional Kalman filter framework is used as is by estimating the next state vector.

[0009]

[Problem to be solved by the invention] However, the constants ρ1, ρ
2, and ignores the local characteristics of the image, which has the disadvantage that estimation accuracy decreases at edges and the like.

An object of the present invention is to solve the above-mentioned drawbacks and provide a processing method for estimating a state vector with high accuracy from an observed vector having a two-dimensional array.

[0011]

[Means for Solving the Problems] FIG. 1 shows a diagram of the basic configuration of the present invention. Reference numeral 100 in the figure is an input means, into which observation vectors sequentially observed are input. 200 is a Kalman filter that estimates and outputs a state vector from an observed vector. 300 is a state vector selection means which selects one of the state vectors that minimizes the difference between the estimated value of the observation vector and the input observation vector.

In the present invention, the following measures are taken to estimate a state vector from an observation vector having a two-dimensional array. In other words, a Kalman filter is defined for each direction connecting the coordinates representing the position on the two-dimensional array to be estimated and one of the neighboring coordinates for which the state vector estimation process has already been completed, and the state vector is Estimate each. At this time, the difference between the observed vector estimate obtained in the calculation process and the sequentially input observed vectors is calculated, and one state vector in the direction in which this difference is minimized is selected.

FIG. 2 shows an example of directions (four directions) for defining a Kalman filter. In FIG. 2, consider coordinates (i, j) where j is the horizontal direction and i is the vertical direction, similar to FIG. 6. The current (i+1, j+1) is the coordinate of the state vector to be found.

The configuration will be explained in detail below. First, we will show how to estimate the state vector using a Kalman filter that operates in one direction. Assume that the defined direction is considered as the time axis, and that the observation vector and state vector in the system are expressed by the following equation.

xt+1 =Ft xt +Gt wt
(3) yt = Ht xt
+vt, t=0,1,... (4) Here, xt is an n-dimensional state vector, yt is a p-dimensional observation vector, wt and vt are m-dimensional and p-dimensional Gaussian white noise vectors, respectively, Ft, Gt,
Ht are n×n, n×m, and p×n matrices that depend only on time t, respectively. Then, the Kalman filter is given by the following (1) to (4).

[0016]

[Math 2] means the estimated value of x.

(1) Filter equation

[Math 3]

(2) Kalman gain

[Math 4]

(3) Estimation error covariance matrix

[Math 5]

(4) Initial state

[Math 6]

[0021] First, the initial state

[Math 7] Given, from equation (10)

[Math. 8] Establish.

Next, K0 is calculated using equation (7). t
Observe y0 at =0, and from equations (6) and (9)

[Math. 9] seek. Then, from equations (5) and (8),

[Number 10
] Calculate K1 from equation (7). y1 at t=1
Observe and again from equations (6) and (9)

Since Equation 11 is obtained, the estimated value of the state vector is calculated using the same procedure. This shows that the state vector is estimated along the sample sequence in one direction.

If this procedure is similarly applied to sample sequences in other directions, state vectors can be estimated for each direction.

Next, select one of the plurality of state vector estimates.
We will explain how to select one. In the calculation process of equation (6),

The observed vector estimated value is calculated from the state vector by [Equation 12], and the difference from the observed vector yt is

(13) is obtained for each direction. The above νt is taken as the evaluation value, and the state vector obtained from the direction in which it is smallest is taken as the estimated value at the target coordinates.

In the above, we have not explicitly stated what the observation vector and state vector physically indicate. Specifically, examples include estimating the distance to an object by observing its movement (optical flow) from a camera, or estimating the slope of a surface by observing the distance from a range sensor. can give.

For example, when estimating the distance to an object from the optical flow, the observation vector corresponds to the optical flow obtained from the brightness value obtained from the camera, and the state vector corresponds to the distance to the object. The observation matrix Ht is a matrix that describes the relationship between the distance to the object and the optical flow, and the state transition matrix Ft is a matrix that describes the relationship between the distances to the object between adjacent pixels.

From the above, it has been shown that a state vector can be estimated from an observation vector having a two-dimensional array using a Kalman filter defined along sample sequences in a plurality of directions.

FIG. 3 shows a flowchart of an embodiment of the invention. As an example, a case will be described in which the camera is moved parallel to its horizontal scanning line and the distance to the object is estimated from the obtained optical flow. Here, it is assumed that the amount of movement of the camera itself is known.

In FIG. 3, first, in process 1, optical flow is determined from luminance values. In process 2, the reciprocal of the distance from the optical flow to the object (hereinafter referred to as inverse distance) is estimated for each direction, and in process 3, the evaluation value is estimated.

Select the inverse distance obtained from the direction that minimizes [Equation 14] and find the distance to the object. This process is repeated until the observation is completed for each sample point.

First, the part for determining the optical flow from the luminance value will be described. Now, the coordinates at time t (
Let the luminance of i, j) be I(i, j, t). time interval T
Let U be the movement in the horizontal (h) direction. In this embodiment, since the movement is in the horizontal direction, the vertical movement V is zero.

Next, h of the image at point (i, j, t)
The brightness gradient in the direction

[Formula 15] The difference between frames is It=I(i,j,t+T)−I(i,j
, t) (12), the constraint equation regarding the motion field is IhU+It=0
(1
3). This is the one-dimensional case of the well-known gradient method constraint equation. Therefore, the horizontal movement U of each pixel is U=-It/Ih
(1
4). Since It and Ih are observable quantities, the horizontal movement U can be determined from the above equation.

FIG. 4 shows a flowchart of the processing 2 part of FIG. 3, that is, the part of estimating the inverse distance from the optical flow to the object along a sample sequence in a certain direction using a Kalman filter. Since the amount of movement of the camera is known, the distance to the object can be estimated from the optical flow. Here, the amount of movement of the camera is D, the distance to the object is d, the inverse distance is x, and the focal length is f. Based on the principle of triangulation, the flow is U=Df/d=Dfx
(15)
It can be written as

A system in which the optical flow is the observation vector and the inverse distance to the object is the state vector can be formulated as follows, and the state vector can be estimated sequentially according to the processing procedure described above.

[Equation 16] Here, the state transition matrix Ft is a 1×1 unit matrix. The observation matrix Ht is 1×1

It is expressed as [Equation 17]. Also, the state vector is a one-dimensional vector x,
The observation vector is the optical flow U. The white noise vector Gt wt is an appropriate value according to prior knowledge of the target object, and vt is an appropriate value assumed to be an observation error of a camera or the like. In Figure 4, first, the inverse distance to the target object is

The initial value of the estimated error covariance matrix Pt is set as follows (processing 2-1).

[0034] The initial value is set to an appropriate value using prior knowledge. Next, from equation (7), the initial value of Kalman gain K0
is calculated (process 2-2). Also, observe the optical flow y0 at coordinates (0,0) at t=0, and use equations (6) and (9) to estimate the distance at t=0.

[
[Equation 19] Find the estimated error covariance matrix P0/0 (Process 2-3)
. Then, from equations (5) and (8), the estimated value of the distance at the next time t=1, that is, the coordinates (0, 1)

[Formula 20] Estimation error covariance matrix P1/0 (process 2-4), equation (7
), K1 is calculated (process 2-2). Observe the optical flow y1 at t=1 and use equation (6
), from (9)

[Formula 21] is obtained (processing 2-3). Thereafter, the distance to the object is sequentially calculated using the same procedure until there is no more optical flow input to observe.

FIG. 5 shows a block diagram of an embodiment of the present invention. In FIG. 5, 1 is a luminance value input terminal, 2 is a sampling and quantization circuit, 3 is a luminance value memory, and 4 is an optical flow calculation circuit. 5, 6, 7 are each Kalman filter, 101 is an initial inverse distance/estimation error covariance matrix input terminal, 1
02 is an inverse distance/evaluation value calculation circuit, 103 is an inverse distance/estimation error covariance matrix memory, 104 is a switch, 8 is an inverse distance selection circuit, 9 is a reciprocal circuit, 1
0 is a distance output terminal.

First, analog luminance value data is input from the input terminal 1, and then converted into digital data by the sampling and quantization circuit 2. The data is written into the brightness value memory 3 and transferred to the optical flow calculation circuit 4. Here, the optical flow (horizontal movement) is determined using Equation (14) from the horizontal brightness gradient of the image and the inter-frame difference. As mentioned above, this circuit can be easily implemented because it only performs addition, subtraction, multiplication, and division operations. The optical flow obtained here is transferred to a plurality of Kalman filters 5, 6, 7, etc.

At the start of calculation in each Kalman filter, an inverse distance/estimated error covariance matrix is written from the initial inverse distance/estimated error covariance matrix input terminal 101. After inputting the initial data, the switch 104 is turned to the inverse distance/evaluation value calculation circuit 102 side. Therefore, since the circuit 102 uses equations (5) to (10), it can be realized only by addition, subtraction, multiplication, and division operations. The obtained inverse distance and estimated error covariance matrix are input to the inverse distance/estimated error covariance matrix memory 103 and the inverse distance selection circuit 8.

The inverse distance/estimation error covariance matrix memory 103 holds data until it is needed for each Kalman filter operation. The inverse distance selection circuit 8 receives each inverse distance x and evaluation value νt. The inverse distance selection circuit 8 outputs the distance obtained from the direction in which the evaluation value νt is the smallest. This circuit is realized using only addition, subtraction, multiplication, and division operations. Inverse distance is reciprocal circuit 9
The distance is converted into a distance and output via the distance output terminal 10.

With the above configuration, a state vector can be estimated from an observation vector having a two-dimensional array using a plurality of Kalman filters. With this, it is possible to select the most optimal estimated value in terms of the minimum error (in the observation space) represented by νt for estimation results whose estimation accuracy varies depending on the scanning direction. Furthermore, the disadvantage of ignoring local characteristics of data in conventional methods is solved by performing estimation processing in multiple directions and selecting one of them.

In this embodiment, the case where distance is estimated from flow has been described, but it is clear that a similar framework can be used when estimating distance from distance, such as when removing noise from a range sensor. Furthermore, when estimating a state vector from a certain observed vector, it is clear that this method can be applied to any data structure that has a two-dimensional data structure.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing the principle configuration of the present invention.

FIG. 2 is an example of the direction in which the Kalman filter of the present invention is defined.

FIG. 3 is a flowchart of an embodiment of the invention.

FIG. 4 is a flowchart of a portion of calculating the distance to an object.

FIG. 5 is a block diagram showing an embodiment of the present invention.

FIG. 6 is a diagram showing the arrangement of pixels in a conventional method.

[Explanation of symbols]

1 Luminance value input terminal 2 Sampling quantization circuit 3 Luminance value memory 4 Optical flow calculation circuit 5, 6, 7 Kalman filter 101 Initial inverse distance/estimation error covariance matrix input terminal 102 Inverse distance/evaluation value calculation circuit 103
Inverse distance/estimation error covariance matrix memory 104
Switch 8 Inverse distance selection circuit 9 Reciprocal circuit 10 Distance output terminal 100 Input means 200 Kalman filter 300 State vector selection means

Claims (1)

[Claims] Claim 1: means for sequentially inputting observation vectors having a two-dimensional array; a plurality of Kalman filters for estimating state vectors for the sequentially input observation vectors; and means for estimating state vectors of the plurality of Kalman filters. This is an adaptive state vector estimation processing method that has a state vector selection means for selecting one from outputs, and each Kalman filter has coordinates representing a position on a two-dimensional array to be estimated, and state vector estimation processing that has already been performed. A state vector is estimated for each direction connecting one of the coordinates in the vicinity of the coordinate where the process has been completed, and the state vector selection means selects a combination of the observation vector estimate calculated by the plurality of Kalman filters and the observation vector inputted sequentially. An adaptive state vector estimation processing method characterized by selecting and outputting one Kalman filter state vector that minimizes the difference.