JPH1125276A - Traveling object detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect an accurate motion vector regardless of the size of a motion vector of a traveling object by detecting a motion vector and remainder based on a stored image and two images at time that is separated by sampling time and selecting motion vectors of two images which have the smallest remainder. SOLUTION: A TV camera 1 continuously picks up the image of a traveling object 2 in each sampling time Δt. An A/D converter 3 converts an image obtained by the camera 1 into a digital image. K pieces of image storing devices 11 to 13 store a digital image converted by the converter 3. K pieces of vector detectors 21 to 23 respectively detect a motion vector of the object 2, based on images at time at t-kΔt (k=1, 2,...,K) which are stored in the devices 11 to 13 and an image that is inputted by the camera 1. A motion vector selector 31 selects one motion vector, based on the accuracy of K pieces of mobile vectors detected by the detectors 21 to 23.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving object detection apparatus applied to a monitoring system for intruders and intruders in harbors and the like.

[0002]

2. Description of the Related Art Conventionally, in a monitoring system for a moving object such as an intruder or an intruder in a harbor or the like, a moving object detecting device which images a moving object and detects the image by image processing has been widely used.

FIG. 6 is a diagram showing the overall configuration of a conventional moving object detection device. As shown in FIG. 6, first, at time t-
At Δt, an image including the moving object 2 is captured by the TV camera 1. The image signal I (t−Δt) including the moving object 2 obtained by imaging is output to the A / D converter 3,
It is quantized and output to the image storage 4. Therefore, the image storage 4 stores the quantized image I (t−t) shown in FIG.
Δt, i, j) are stored. Here, i and j are 1 ≦
i ≦ M, 1 ≦ j ≦ N, and M × N is the image size.

Next, at time t, which is delayed by sampling time Δt from time t−Δt, moving object 2 is
The camera 1 takes an image. This captured image signal I (t)
Is output to the A / D converter 3 and is quantized to obtain the signal shown in FIG.
Are output to the movement vector detector 5 as an image I (t, i, j) shown in FIG. Further, the image storage unit 4 outputs the stored image I (t−Δt, i, j) to the movement vector detector 5.

The movement vector detector 5 detects two images I (t−Δt, i, j) obtained at time t−Δt and time t,
Moving object 2 is obtained by the following method using I (t, i, j).
Of the motion vector (u, v) is detected.

First, a spatial differential image in the i direction of an image I (t, i, j) is I _x (i, j), a spatial differential image in the j direction is I _y (i, j), and two images I Let the time differential image of (t−Δt, i, j) and I (t, i, j) be I _t (i, j). Also, as shown in FIG.
Is set to a certain area W × H. This area is called a window S.

In FIG. 8, the moving object 2 is shown as a circle for simplicity. Although only one window S may be set, the moving object 2 is moved from one end to the other so that the entire area of the image can be covered by the window S because it is unknown where the object is located on the image. That is, FIG.
S as shown in _{(b) p (p = 1,2} , ..., P) moving vector (u _p, v _p) for the image data contained in the window S _p at each position as determined. The moving vector (u _p, v _p) deriving a is obtained by solving the following equation.

[0008]

(Equation 1)

[0009] In the above equation (1), sigma symbol means that the sum of all the pixels contained in a window S _p. If it does not contain moving objects in the window S _p becomes (1) the right side of the equation is a value close to 0 motion vector _{(u p, v p) ≒} (0,0). On the other hand, FIG.
I direction of travel during the sampling time Delta] t if it contains moving objects in the window S _p as (a), as the distance j direction (u _p, v _p) (pixel / Delta] t) is determined.

(U ₁ , v ₁ ) obtained in this manner
(U _P, v _P) by overlapping, moving vector (u, v) of time t and time t-Delta] t is obtained. This movement vector detection operation is performed for each sampling time Δt of the TV camera 1, whereby the movement amount and the movement direction of the moving object 2 are known, so that the moving object 2 can be detected.

[0011]

In the above-mentioned conventional moving object detecting apparatus, spatial brightness changes (spatial differential images I _x (i, j), I _y (i,
j)) and the temporal brightness change (temporal differential image I _t (i,
j)) can be used to determine the motion vector (u, v). The estimation accuracy of this movement vector is given by the following equation (2)
It depends on the size of the residual E _p given by.

[0012] close to _{E p = Σ Sp {I x} (i, j) u p + I y (i, j) v p + I t (i, j)} 2 (2) The value of the estimation accuracy E _p is 0 The error included in the value of the movement vector (u, v) is smaller as the value is larger, and a larger error is included when _Ep is larger.

For example, as shown in FIG. 8B, when the moving amount of the moving object 2 during the sampling time Δt is small, that is, when the moving speed of the moving object 2 is very small, the estimation is performed due to the influence of image noise. moving vector (u _p, v _p) errors in the value of the increases are accurate motion vector is a problem that can not be determined.

If the sampling time .DELTA.t is increased by several times to solve this problem, the movement amount becomes large, so that the movement vector can be correctly obtained. However, for the moving object 2 having a large moving amount, the moving object 2 is out of the range with respect to the window S set as shown in FIG. _Ep becomes large, and the error of the movement vector obtained by the equation (1) becomes very large.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and has as its object to provide a moving object detecting device capable of detecting an accurate moving vector regardless of the size of the moving vector of the moving object. Is to provide.

[0016]

SUMMARY OF THE INVENTION A moving object detecting apparatus according to the present invention comprises an image pickup means for continuously picking up a moving object at every sampling time .DELTA.t, and an image conversion means for converting an image obtained by the image pickup means into a digital image. / D conversion means and this A / D
K image storage means for storing the digital image converted by the conversion means, images at time tkΔt (k = 1, 2,..., K) stored in the K image storage means, and the imaging K moving vector detecting means for respectively detecting the moving vector of the moving object based on the image at time t inputted by the means, and accurate K moving vectors detected by the K moving vector detecting means. Moving vector selecting means for selecting one moving vector based on the motion vector.

According to a preferred embodiment of the present invention, the K number of motion vector detectors include an image at time tkΔt (k = 1, 2,..., K) stored in the K image storage means and the image pickup device. The movement vector and the residual are detected based on the two images at the time t separated by the sampling time kΔt input by the means, and the movement vector selecting means detects the two images having the smallest detected residual. Is selected.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an overall configuration of a moving object detection device according to one embodiment of the present invention. This moving object detection device includes a TV camera 1, an A / D
Converter 2, first image storage 11, second image storage 12 to
K-th image storage unit 13, first motion vector detector 21, second motion vector detector 22 to k-th motion vector detector 2
3. It is composed of a movement vector selector 31.

The TV camera 1 captures a scene including the moving object 2 at every sampling time Δt, and the captured image is output to the A / D converter 3. A / D converter 3
Quantizes the captured analog image signal and transfers it to the first image storage 11. Here, the first image storage 11
, An image signal quantized for each sampling time Δt is input, and at the same time a new image signal is input,
The already stored image signal Δt before the input new image signal is transferred to the second image storage 12. Similarly, at the same time as the new image signal is input to the second image storage 12, the already stored image signal is transferred to the third image storage (not shown), and is already stored in the (K-1) th image storage. The transmitted image signal is transferred to the K-th image storage unit 13. Therefore, at time t, the k-th image memory (k = 1,
2,..., K) are images I taken at time t−kΔt.
(Tk-t) is stored.

FIG. 2 shows an example of an image sequence stored in the first to K image storage units 11 to 13 and an image at time t transferred from the A / D converter 3. 2A shows an image I (t) at time t, and FIG. 2B shows a time t-Δt.
2 (c), and FIG.
FIG. 2D shows an image I (t−KΔt) at time t−KΔt, and FIG. 2D shows an image I (t−KΔt) at 2Δt.
The images before the time t shown in FIGS. 2B to 2D have been transferred from the first image storage unit 11, the second image storage unit 12, and the K-th image storage unit 13, respectively. The moving object 2 is indicated by a solid circle for simplicity, and the moving object 2 at time t is indicated by a dotted circle for comparison.

The first to K image storage units 11 to 13
The stored image signals are output to first to K-vector detectors 21 to 23 corresponding to the respective image storages 11 to 13.
On the other hand, image signals at time t, which are captured by the TV camera 1 and digitized by the A / D converter 3, are transferred to the first to K-vector detectors 21 to 23. That is, I (t)
And I (t−Δt) are sent to the first motion vector
(T) and I (t−2Δt) are the values of the second motion vector detector 2
2, I (t) and I (t−KΔt) are transferred to the K-th motion vector detector 23, respectively.

Next, the first to K-th motion vector detectors 21 to 21
23 is a k-th motion vector detector (k = 1, 2,..., K)
The operation will be described below. The k-th motion vector detector detects two images I (t) and I (tk-t)
A window _Sp is set for. This window S
FIG. 3 shows a setting example of _p . The window _Sp is shown in FIG.
As shown in FIG. 3, a size of W × H may be set at a specific position, or a window of size W × H may be sequentially scanned over the entire image as shown in FIG. the window _{S S 1, S 2, ...} , it may be used to set the P-number of window as S _P. It is also possible to scan the plurality of windows S _p are overlapped with each other.

Here, a case where a window is set as shown in FIG. 3B will be described below. Also, focus on one window Sp of the _P windows. Kth
Relative motion vector detector time t of the image inputted from the image and A / D converter 3 at time t-k.DELTA.t stored in the k-th image storage unit, the window S _p of the image I (t, i,
j), a spatial differential image I _x (i, j) in the i direction, a spatial differential image I _y (i, j) in the j direction, and two images I (t−Δ)
t, i, j) and I (t, i, j) of the time differential image I _t
(I, j) is obtained, and the operation shown in Expression (1) is performed based on the obtained differential images. Then, a movement vector ( _upk , _vpk ) of the moving object 2 is extracted based on the calculation result. Here, i and j are 1 ≦ i ≦ M and 1 ≦ j ≦ N, and M × N is the image size. Further, the k-th motion vector detector performs the operation shown in the expression (1) and simultaneously calculates the residual _Epk given by the expression (2).

Further, depending on the image in the window S _p, or no change in the brightness of the image, it may be present only on a straight line edges. For such images,
(1) The determinant | A | of the 2 × 2 matrix on the left side of the equation (hereinafter, this matrix is referred to as A) becomes a value close to 0, and the movement vector cannot be obtained correctly.

Then, the eigenvalues λ ₁ and λ ₂ of the matrix A are calculated. The eigenvalues λ ₁ and λ ₂ indicate a measure of how much the brightness changes in the i direction and the j direction, and λ ₁ ≦ λ
Assuming that ₂ , the greater the value of λ ₁ , the greater the change in the brightness of the image in the window S.
Therefore, lambda ₁ when the value is greater than the threshold value lambda _th there is only (1) the motion vector calculated by formula, when lambda _th smaller than 0 as the movement vector if not moving objects 2 in the window The movement vector detector outputs the value. In this case, the K motion vector detectors all output a value of 0, so the motion vector selector 10 automatically selects 0 as the motion vector.

[0026] Thus, at time t, 1 single movement vector for window S _p of the k motion vector detector (u _pk, v _pk) is obtained. Similarly, image I
When (t) is sent to the first, second,..., Kth motion vector detectors, the motion vectors ( _up1 , _vp1 ) to ( _upK , _vpK ) for the first to _Kth motion vector detectors. And the residual E _p1
ＥE _pK are obtained and sent to the motion vector selector 10.

The moving vector selector 10 compares the obtained values of the residuals E _{p1 to} E _pK . Then, motion vector (u _PK0 in k = k ₀ as the smallest residuals, v
_pk0 ) is selected as the optimal movement vector. And
This movement vector _{(u pk0, v pk0) (} u 'p, v'
_p ). Here, K residuals E _pk (k =
1,2, ..., is greater than the threshold value Th which is the minimum value E _pk of K), it determines that all the mobile vectors Motoma' against window S _p is not accurate, the movement vector is not selected.

The above is a calculation and selection process of the movement vector was performed in the window S _p, the series of processing is performed for all windows S ₁ to S _P. In other words, each window _{S p (p = 1,2, ...} , P) motion vector (u _'1, based on images at different times for each,
v ′ ₁ ) to (u ′ _P , v ′ _P ) are obtained. The process of selecting these movement vectors will be described briefly with reference to FIG.

FIG. 4 shows a circular moving object 41, a triangular moving object 42 and a moving vector in a window in the image. FIG. 4A shows an image I (t−KΔt) at time t−KΔt, and FIG. 4B shows a time t−Δt.
4 (c) shows the image I (t-Δt) at time t, and FIG. 4 (c) shows the image I (t) at time t. At time t in FIG. 4C, the object 1 and the object 2 are included in the windows S ₁ and S ₂ , respectively.
Shows _1, the process of calculating and selecting a movement vector in S ₂ below.

As shown by the solid line in FIG.
Located moving object 41 in the window S ₁ in Kderutati, in the S ₂ moving object 42 is positioned in S ₂ out not located. In this case, the moving object 41 at time t,
The moving object 41 has an appropriate moving distance for calculating the moving vector with respect to the position of 42, and from these two images,
The movement vector V _K (t) is calculated. That is, FIG.
As shown in (d), V _K (t) = (u _1K , v _1K ), and when the residual E _1K corresponding to this movement vector has the minimum value, the movement vector is (u ′ ₁ , v ′). ₁ ) However, the actual moving speed is not (u ₁ , v ₁ ),
(U ₁ / K, v ₁ / K) (pixels / Δt).

On the other hand, since the moving object 42 is not located in the S ₂ area and is far away from the position at the time t, the residual E _2K becomes large, and it is difficult to calculate an accurate moving vector. , The calculated motion vector (u _2K , v
_2K ) is not selected by the motion vector selector 10.

Next, time t-Δt and time t in FIG.
3 shows an image comparison with FIG. As shown by the solid line, the window S
_The moving objects 41 and 42 are located in ₁ and S ₂ , respectively.
Is hardly changed with respect to the position of the moving object 42 with respect to the position of, and the error estimated due to the influence of the image noise is large with respect to the amount of change of the moving object 42 in both images. is there. Therefore, the movement vector (u _1t , v _1t ) is not selected by the movement vector selector 31.

On the other hand, since the moving object 42 has an appropriate moving distance with respect to the position at the time t, these two
The movement vector V ₁ (t) can be calculated from the images. Therefore, as shown in FIG. 4E, V ₁ (t) =
(U ₂₁ , v ₂₁ ), and when the residual E ₂₁ corresponding to this movement vector has the minimum value, the movement vector is (u ′)
₂ , v ′ ₂ ).

V ₁ (t) obtained as described above
Of V _K (t), the smallest optimum motion vector residual E _pK is selected for each window. Then, V _a (t) obtained by superimposing the motion vectors of the selected windows is the output of the motion vector selector 10 finally selected. That is, as shown in FIG. 4F, V _a (t) is (u ′ ₁ , v ′ ₁ ) to (u ′ _P ,
v ′ _P ). Images I (t), I (t + Δt), I (t + 2Δ) obtained at every sampling time Δt until the data processing until obtaining V _a (t)
t),... are executed at each time Δt. Therefore, as _a result, V _a (t) is obtained for each sampling time Δt. FIG. 5 shows the flow of data in this series of processing.

In FIG. 5, the captured image, the output of the first motion vector detector, the output of the second motion vector detector,
.., Data on the output of the K-th motion vector detector and the output of the motion vector selector, and the horizontal direction is the time axis. Arrows indicate the flow of data, for example, at time t.
The first motion vector detector output V ₁ (t) in the obtained on the basis of the captured image I (t-Δt) in the captured image I (t) and time t-Delta] t at time t.

As described above, even when the image is hardly changed during the sampling time Δt of the TV camera 1 and the moving amount of the moving object 2 is so small that it is difficult to correctly estimate the moving vector, the first to the first cases. The image which is traced every Δt with respect to the time t stored in the K-th image storage units 11 to 13 and A /
The first to Kth motion vector detectors 21 to 23 calculate the motion vector and the residual E from the current image which is the output of the D converter 3, and the motion vector selector 31 calculates the optimal image for estimating the motion vector. Since the movement vector can be determined by selecting the value of the residual E _pk , it is possible to accurately detect the movement vector, that is, the movement amount and the movement direction of the moving object 2 without being buried in the errors included in the two image signals. It becomes possible.

On the other hand, for an object having a large moving amount, an image most suitable for estimating a moving vector can be similarly selected from an image retroactive at every Δt and a current image. It is possible.

In this embodiment, the case where the moving object detecting apparatus is applied to an intruder or an intruder monitoring system has been described.
It is needless to say that the present invention can be applied to a vehicle monitoring system at an intersection and the like.

[0039]

As described above, according to the present invention, K image storage means for storing a digital image, and time tkΔt (k = 1, k) stored in the K image storage means.
2,..., K) and two images at a time t separated by a sampling time kΔt. A moving vector selecting means for selecting a moving vector based on the accuracy of the detected K moving vectors, the moving speed is small, and there is almost no image change during the sampling time Δt;
Even when it is difficult to correctly estimate the motion vector, the motion vector can be obtained by selecting the image most suitable for estimating the motion vector from the image retroactive at every Δt with respect to time t and the current image. It is possible to accurately extract a movement vector without being buried in an error included in the movement vector.

On the other hand, for an object having a large moving object, an optimum image can be similarly selected and a moving vector can be obtained. Therefore, the most accurately obtained moving vector can be extracted. Objects can be detected.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an overall configuration of a moving object detection device according to an embodiment of the present invention.

FIG. 2 is a view showing an example of a continuous image in the embodiment.

FIG. 3 is an exemplary view showing an example of setting a window in the embodiment.

FIG. 4 is an exemplary view showing a process of detecting and selecting a movement vector in the embodiment.

FIG. 5 is an exemplary view showing a flow of data continuously processed by each component in the moving object detection device according to the embodiment.

FIG. 6 is a block diagram showing the overall configuration of a conventional moving object detection device.

FIG. 7 is a view showing an image captured by a TV camera.

FIG. 8 is a diagram showing an example in which a motion vector is obtained and an example in which a motion vector is not obtained in a conventional moving object detection device.

[Explanation of symbols]

 Reference Signs List 1 TV camera 2, 41, 42 Moving object 3 A / D converter 11 First image storage 12 Second image storage 13 Kth image storage 21 First movement vector detector 22 Second movement vector detector 23 K movement vector detector 31 Movement vector selector

Claims (1)

[Claims] An imaging unit for continuously imaging a moving object at every sampling time Δt, an A / D conversion unit for converting an image obtained by the imaging unit into a digital image, and an A / D conversion unit. K image storage means for storing the converted digital image, and the image at time t−kΔt (k = 1, 2,..., K) stored in the K image storage means and input by the imaging means K moving vector detecting means for respectively detecting the moving vector of the moving object based on the image at the time t, and the accuracy of the K moving vectors detected by the K moving vector detecting means. And a moving vector selecting means for selecting one moving vector.