JPH0894320A - Traveling object-measuring instrument - Google Patents

Abstract

PURPOSE: To improve the accuracy in a traveling object measuring instrument for measuring the number of passing automobiles and the speed by analyzing a road image. CONSTITUTION: With an inputted animation image 11, a vehicle part is detected by a traveling object detection means 12 and is converted to a three-dimensional position on a road by a 2D/3D conversion means 13 and is retained by a 3D information retention means 15. Based on the three-dimensional position information and the traveling information until that time, the three-dimensional position of the vehicle at the next time is predicted by a prediction/fusion means 16, is inputted to a 3D/2D conversion means 14 via the 3D information retention means 15 and is converted to a position on an image, and then the position is inputted to a traveling object detection means 12. The traveling object detection means 12 detects the vehicle preferentially to reduce detector error.

Description

Detailed Description of the Invention

[0001]

[Industrial applications]
_{The present invention relates to a moving object measuring device, and in particular,} a traffic monitoring device that uses a camera installed on a road and measures traffic information such as the number of vehicles passing through it, vehicle type, vehicle speed, and a camera installed in a building. The present invention relates to a moving object measuring device that detects and analyzes a moving object using an image, such as a monitoring device that detects a person passing therethrough.

[0002]

2. Description of the Related Art As a conventional example of a moving object measuring device using an image, there is a technique disclosed in Japanese Patent Laid-Open No. 4-313198, "Movable object measuring device". The main purpose of this technology is to measure traffic information, and it consists of the following processes. Figure 3
As shown in (1), an edge is extracted from the road image by image processing to create an edge image 31, and the edge points included in the edge image are projected on a projection axis 32 along the road on the image to create integrated data 33. After that, the projection data 34 is obtained by binarizing with an appropriate threshold value. Due to the fact that many edges are observed in the part where the vehicle exists on the road, this projection data 34 is data in which the part where the vehicle exists on the line along the road has a value of 1 at a certain time. This one-dimensional data is temporally arranged to create two-dimensional data called a DTT image. Since the position of the vehicle moves every moment on the projection data 34, the trajectory of one vehicle is recorded as one band in the DTT image. However, since the processing described so far is not perfect, there is a chip or a hole in the band that indicates the trajectory of the vehicle, and a small area of value 1 occurs in areas other than the band. Therefore, the noise region is removed and the band is shaped by performing image processing such as isolated point removal or hole filling on the DTT image. After that, the speed is calculated from the number of passing vehicles from the number of belts and the inclination of the belts.

[0003]

Generally, an error is included in the result of image processing, and coping with this error becomes a big problem in making an object detecting device by image processing. In processing using a moving image, an error in image processing is often detected and corrected using temporal continuity of the position of a moving object. In the above example, by creating an image with one axis taken as time, the continuity of the position of the moving object with respect to time is converted into a data representation represented by the neighborhood on the image, and isolated points are set for this image. By performing neighborhood processing in the field of image processing such as removal and hole filling, as a result, the continuity of the motion of the object is used as a clue to correct the error in the image processing for vehicle detection.

However, since the motion of an object in the real world is governed by Newton's law of motion, in order to correctly model the continuity of the motion of an object and detect and correct errors in image processing more effectively. Needs to incorporate this law of motion as knowledge in the processing. For this purpose, it is necessary to convert the law of motion in the three-dimensional space into the law of motion on the image. The equation of motion has a simple shape in a three-dimensional space in which a moving object exists, but because of the above-mentioned necessity, it becomes very complicated when converted into a relation on an image by projective transformation. Due to this complexity, the law of motion could not be effectively utilized as knowledge in processing.

[0005]

SUMMARY OF THE INVENTION The first aspect of the present invention, by analyzing each frame of _moving image, the moving object detection means for detecting an area of the moving object present in the image,
_The 2D / 3D conversion means for restoring the spatial position of the moving object from the area detected by the _{moving object detecting means,} and the current position and the motion are predicted from the information on the spatial position and the motion of the moving object in the past, A prediction / fusion unit that fuses the predicted position / motion information and the position information output from the 2D / 3D conversion unit;
Holding the position and motion information fused _{by the prediction-fusion means} is a moving object measuring apparatus _{you and a} 3D information holding means for outputting in accordance with a request from the outside.

[0006] The second invention is a moving object measuring apparatus of the first aspect of the invention, is characterized in the use of the Kalman Filter based on the equation of motion of the object as _the prediction-fusion means.

A third invention is the moving object measuring apparatus of the second aspect of the invention, the variance of the measurement values as an input of the Kalman filter used as _the prediction-fusion means, the dispersion of errors estimated position information on the image , _the 2D / 3D
The calculation is performed by using a differential value of the relationship between the position on the image and the spatial position used by the conversion means.

[0008] The fourth invention is the moving object measuring apparatus of the first aspect of the invention, by using the information about the motion that is predicted by _the prediction-fusion means, information on the position provided by _the 2D / 3D conversion unit, they It is characterized by having a mechanism for deciding whether or not to fuse the information.

A fifth invention is a moving object detecting means for detecting a region of a moving object existing in the image by analyzing each frame of the moving image,
_The 2D / 3D conversion means for restoring the spatial position of the moving object from the area detected by the _{moving object detecting means,} and the current position and the motion are predicted from the information on the spatial position and the motion of the moving object in the past, The predicted position / motion information and the position information output from the 2D / 3D conversion unit are fused, and the prediction / fusion unit that predicts the future position of the moving object is processed by the prediction / fusion unit. Holds the position / motion information of moving objects,
3D information holding means for outputting _{according to a request from the outside, and} 3D / 2 for converting the position of the moving object predicted by the prediction / fusion means into two-dimensional information on the image by projective conversion.
D conversion means, _and inputs the predicted two-dimensional information to the moving object detection means, and based on the two-dimensional information
To change the contents of processing performed by _the moving object detecting means
_A moving object measuring device _{characterized by the} above.

[0010] According to a sixth aspect, in the moving object measuring apparatus of the fifth invention, it is characterized in the use of the Kalman Filter based on the equation of motion of the object as _the prediction-fusion means.

[0011] A seventh invention is in the sixth moving object measuring apparatus of the invention, the variance of the measurement values as an input of the Kalman filter used as _the prediction-fusion means, error variance, estimated in the position information on the image from _the 2D / 3
It is characterized in that the calculation is performed by using a differential value or a Jacobian matrix of the relationship between the position on the image and the spatial position used in the D conversion means.

[0012] An eighth invention is the in 5 moving object measuring apparatus of the invention, _the moving object detecting means constituted by the image edge calculation means and binarizing means, the image on provided by _the 3D / 2D conversion unit Depending on the location information of
It is characterized in that the threshold value used in the binarizing means is changed.

[0013] A ninth aspect of the invention, the moving object measuring apparatus of the fifth invention, by using the information about the motion that is predicted by _the prediction-fusion means, information on the position provided by _the 2D / 3D conversion unit, It is characterized by having a mechanism for determining whether or not to fuse such information.

[0014]

EXAMPLE In order to incorporate the law of motion of an object as a knowledge into the process for measuring a moving object by an image, conversion into a rule on an image by projective transformation results in a complicated relationship. Therefore, conversely, the position on the image is converted into the position in the three-dimensional space by the inverse operation of the projective transformation, and the equation of motion is applied in the three-dimensional space. Of course, since the projective transformation is a degenerate transformation, there is no inverse, and some kind of constraint is required to perform the inverse operation. For example, the 1993 Autumn Meeting of the Institute of Electronics, Information and Communication Engineers, page 6-333, "By Real Space Feature Value" As described in "Surveillance method", by using the constraint that the object exists on the ground surface, the positional information on the image can be converted into the information in the three-dimensional space.

In this way, the three-dimensional position of the moving object can be calculated momentarily from the processing result of each frame of the moving image. Next, some mechanism for integrating these many pieces of positional information with the law of motion is integrated. is necessary. Since the Kalman filter is suitable as one of the mechanisms, this is used in the embodiments described later.

The Kalman filter is composed of time update processing and observation update processing. The law of motion is used in the time update process, which predicts the position of the object at the next time step. This prediction mechanism can be used not only to detect and correct an error in image processing in the moving object detection process but also to positively reduce the error. That is, a stable detection result can be obtained by converting the predicted position in the three-dimensional space into a position on the image by projective transformation and increasing the detection sensitivity of the moving object in that portion in the image processing.

Next, the vehicle monitoring device according to the present invention will be described. The vehicle monitoring device is a device that detects and tracks a vehicle as a moving object from a road image and measures the position and speed of the moving object.

The configuration _of the first embodiment _of the present invention is shown in FIG. Each frame of the moving image 11 is the image processing by the moving object detection unit 2 _1, area of the vehicle are extracted. Then 2
Converted by D / 3D conversion unit 1 ₃ in position on the road, is held by the 3D information holding means 1 _5, the observed value of the Kalman filter is a prediction-fusion means 1 _6. In the Kalman filter, time update processing is performed using the estimated value of the motion information of the previous time step held in the 3D information holding unit 15, and the current observed value is obtained as the current predicted value by the observation update processing. To merge. The fused motion information is again held by the 3D information holding means 15 and output as a final measurement value. Next, each means constituting this embodiment will be described.

The function of the moving object detecting means 21 is to detect the moving object area from the image. It is assumed that the input image is an image taken from the road with the optical axis directed in the traveling direction of the vehicle. The input image is a Sobel operator
ator) and the like, and then binarized by an appropriate threshold value L1 to form an edge image (see FIG. 3). Next, after the number of edge points is integrated in the horizontal direction of the image to project on the projection axis 32 along the lane, it is binarized by an appropriate threshold value L ₂ and becomes projection data 34. The portion of the projection data 34 where the value 1 is continuous corresponds to the vehicle. Here, it is assumed that the tail of the vehicle is tracked, and the position y _c (t) at which the continuous portion of the value 1 ends is output as the vehicle position on the image at time t.

The 2D / 3D converting means 13 converts the vehicle position y _c (t) on the image into a position on the road. A side view of a road in a three-dimensional space is shown in FIG. Here, H is the height of the focus of the camera from the road surface 41, and θ is the road surface 41.
Is the depression angle of the camera, f is the focal length, and Z _c (t) is the position of the vehicle on the road when the point directly below the camera is the origin. Position y _c (t) on vehicle image and position Z _c (t) on road
The relationship of is expressed by the following equation with y ₀ as the y coordinate of the image center.

[0021]

Y '= 2y _c (t) -y ₀ Z _c (t) is calculated by this relational expression, and is input to the prediction / fusion unit 16 via the 3D information holding unit 15.

A Kalman filter is used as the prediction / fusion means 16. The state vector x of the Kalman filter is a three-dimensional vector composed of a position Z _c (t), a velocity v (t), and an acceleration a (t), and a system noise is added to the acceleration a (t). To use. Then, the time update processing is as follows (Tetsu Katayama, "Applied Kalman Filter", Asakura Shoten (1983), p. 77).

[0024] ^{x - (t + 1) =} Fx + (t) (2) P - (t + 1) = FP + (t) F T + Q In addition, the observation update processing is as follows.

[0025] ^{K (t) = P - (} t) H T (HP - (t) H T + p z (t)) -1 (3) x + (t) = x - (t) + K (t) ( y _c (t) −Hx ⁻ (t)) P ⁺ (t) = P ⁻ (t) −K (t) HP ⁻ (t) where x (t) = (Z _c (t), v (t ), a (t)) ^T , (4)

[0026]

Where p _a is the variance of the system noise of the acceleration a, p _z (t) is the variance of the noise contained in the position Z _c (t) on the road, which is the observed value, and P (t) is the estimated value. It is the variance of x (t), and +/- on the shoulders of x (t) and P (t) represent the values before and after the observation update. The unit of time is the frame interval of the moving image. The variance p _z (t) of the observation values can be set to a constant value, but more correctly, the position y on the image
_The following equation is used to calculate the error variance p _y (t) contained in _c (t) and the differential value of Z _c (t) with respect to y _c (t).

P _z (t) = (dZ _c / dy _c ) ² p _y (t) (5) The derivative used here is derived from the relationship used in the 2D / 3D conversion means 2, but in the present embodiment In the case of, the equation (1) may be differentiated by y _c (t). The differential value used in the actual calculation is the value at the object position y _c (t) on the detected image. The value of p _z (t) itself is either constant or estimated from the result of image processing.

The Kalman filter operates effectively when the noise included in the observed value is close to Gaussian noise. However, the observed values obtained by image processing often include sporadic values that cannot be modeled as Gaussian noise. In such a case, robust estimation that eliminates abnormal values is effective. To do this,
Time updating the predicted value after x ^- a first component (t) Z _c ^- as _{(t), d z (t} ) = | Z c (t) -Z c - (t) | Get (6) However, when d _z (t) is a certain value or more, it is achieved by not performing the observation update processing as an abnormal value and using the predicted value as the estimated value at that time. In addition, since the variance is calculated for the observed value and the predicted value, this can be reflected in the above judgment criteria. This is
It is achieved by calculating _z (t).

D _z (t) = | Z _c (t) −Z _c ⁻ (t) | / (p _z (t) + p _z ⁻ (t)) ^−1/2 (7) where p _z ⁻ (t) is a 1-row, 1-column component of P ⁻ (t).

The 3D information holding means holds the information on the position output by the 2D / 3D conversion means and the information on the motion calculated by the prediction / fusion means, and outputs it as the measurement result of the device. Generally, the first component Z _c ⁺ (t) and the second component v ⁺ (t) of the estimated value x ⁺ (t) after the observation update processing are used as the measured values of the current position and speed of the vehicle, respectively.

Next, a second embodiment of the present invention will be described. In the above-mentioned first embodiment, the prediction / fusion unit 16
Predicted values of the spatial position of the vehicle at the next time step by Z _c ^- (t) is computed. Therefore, by feeding back this information to the moving object detection means ₁₂ , it is possible to reduce errors in image processing and to perform stable and accurate measurement. An example of realizing this is the device shown in FIG. In the second embodiment, since the moving object detecting means 12 and 3D / 2D conversion unit 1 ₄ other parts are the same as those of the first embodiment, described below for these parts.

The 3D / 2D converting means 14 calculates the prediction value x after the time update process of the Kalman filter of the predicting / fusion means 16.
^- to convert (t) ^- first component Z _c of (t) ^- the predicted value of the position on the image by the projective transformation of the (t) y _c. By solving the equation (1) for y _c (t), the equation of this projective transformation can be obtained.

[0034] In the moving object detection unit 12 prediction value y _c ^- (t)
Is received, and processing is performed so that the tail of the vehicle is easily detected at this portion. This process is as follows. As shown in FIG. 3, when the projection data 34 is obtained from the integrated data 33 obtained by integrating the number of edges of the edge image 31 in the horizontal direction of the image, binarization is performed. is changed as shown in FIG. 5 in the vicinity ^- threshold predicted value y _c of reduction. That is, the threshold value is increased by d _L2 above L ₂ in the area of width w on the left side (+ direction) of y _c ⁻ and decreased by d _{L2 in} the area of width w on the right side (− direction). This y _c ^- car tail around is likely to be detected. A variety of ways of changing the threshold value can be considered, and the same effect can be obtained even when the threshold value is changed as shown in FIG. The optimum way of changing is to be determined according to the characteristics of the edge detection processing.

Each means constituting the present invention can be realized by an electronic computer including a microprocessor and a memory, and a program describing a calculation procedure corresponding to each means described above. In addition, T
It can also be realized by combining logic elements such as TL so as to execute the above-mentioned calculation procedure.

[0036]

By using the present invention, it is possible to realize a moving object measuring apparatus which can obtain stable and accurate measurement results.

[Brief description of drawings]

FIG. 1 is a diagram showing a second embodiment of the present invention.

FIG. 2 is a diagram showing a first embodiment of the present invention.

FIG. 3 is a diagram showing moving object detection means.

FIG. 4 is a diagram showing a correspondence relationship between an object on a road and a point on an image.

FIG. 5 is a first threshold value used in the moving object detection means.
It is a figure which shows the example of.

FIG. 6 is a second threshold value used in the moving object detecting means.
It is a figure which shows the example of.

[Explanation of symbols]

 11 Moving Image 12 Moving Object Detection Means 13 2D / 3D Exchange Means 14 3D / 2D Conversion Means 15 3D Information Holding Means 16 Prediction / Fusion Means 17 Position / Motion Information 21 Moving Object Detection Means 31 Edge Image 32 Projection Axis 33 Integrated Data 34 Projection data 41 Road surface 42 Photographed image

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G08G 1/01 CK 1/04 DH 04N 7/18 DK

Claims (9)

[Claims] 1. A moving object detecting means for detecting an area of a moving object existing in an image by analyzing each frame of a moving image, and a spatial movement of the moving object from the area detected by the _{moving object detecting means.} 2D / 3D to restore position
The conversion unit and the current position and motion are predicted from information about the spatial position and motion of the moving object in the past, and the predicted position / motion information and the position information output from the 2D / 3D conversion unit are calculated. Prediction and fusion means to fuse,
_{The prediction and} holds the position and motion information fused _{by fusion means,} moving object measuring apparatus _{you and a} 3D information holding means for outputting in accordance with a request from the outside. 2. A moving object measuring apparatus according to claim 1, characterized by using a Kalman Filter based on the equation of motion of the object as _the prediction-fusion means. Wherein the variance of the input to become the measurement values of the Kalman filter used as _the prediction-fusion means, the dispersion of errors estimated position information on the image, _the 2D / 3
The moving object measuring device according to claim 2, wherein the moving object measuring device is calculated by using a differential value of a relationship between a position on an image and a spatial position used by the D conversion means. 4. A information on motion that is predicted by _the prediction-fusion means, using _said information on the position provided by 2D / 3D conversion unit, a mechanism for judging whether to fuse the information The moving object measuring device according to claim 1, which has. 5. A moving object detecting means for detecting an area of the moving object existing in the image by analyzing each frame of the moving image, and a spatial movement of the moving object from the area detected by the _{moving object detecting means.} 2D / 3D to restore position
The conversion unit and the current position and motion are predicted from information about the spatial position and motion of the moving object in the past, and the predicted position / motion information and the position information output from the 2D / 3D conversion unit are calculated. A 3D information holding unit that fuses and holds the position / motion information of the moving object processed by the prediction / fusion unit and the prediction / fusion unit that predicts the future position of the moving object, and outputs _{according to a request from the outside.} And 3D / 2D conversion means for converting the position of the moving object predicted by the prediction / fusion means into two-dimensional information on the image by projective conversion, _and detecting the predicted two-dimensional information by the moving object detection. Fill in means, moving object measuring _{apparatus characterized by} changing the contents of the processing performed by _the moving object detecting means based on the two-dimensional information. _6. The moving object measuring device according to claim 5, wherein a Kalman filter based on an equation of motion of the object is used as the predicting / merging means. 7. The dispersion of the input to become the measurement values of the Kalman filter used as _the prediction-fusion means, the dispersion of errors estimated position information on the image, _the 2D / 3
7. The moving object measuring device according to claim 6, wherein the moving object measuring device is calculated by using a differential value or a Jacobian matrix of a relationship between a position on an image and a spatial position used in the D conversion means. 8. constituted by _the image edge calculation unit moving object detecting means and binarizing means, the position information on the image provided by _the 3D / 2D conversion unit,
The moving object measuring device according to claim 5, wherein the threshold value used in the binarizing means is changed. 9. A information on motion that is predicted by _the prediction-fusion means, using _said information on the position provided by 2D / 3D conversion unit, a mechanism for judging whether to fuse the information The moving object measuring device according to claim 5, which has.