JPH09114987A - Moving object detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize the moving direction and shape of a moving object while using a moving vector (optical flow) provided from the image of the moving object. SOLUTION: The moving object is photographed by a camera 10 and supplied to an ECU 12. A DSP 12d calculates the moving vector for every small area in the provided image and stores it in a RAM 12f. Based on a prescribed value Z1 concerning the distance from the camera 10 set to one of small areas, a CPU 12e calculates an estimated value Za concerning the distance from the camera 10 provided for the other small area in the same plane. Based on the prescribed value Z1 and an estimated value Za, the inclining angles of planes consisting of the moving object to the camera 10 are calculated and the planes having the different inclining angles are mutually coupled so that the shape of the moving object can be constituted.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to a moving object detection device,
In particular, the present invention relates to a device that detects a moving object using a movement vector (optical flow) obtained from an image.

[0002]

2. Description of the Related Art Conventionally, a motion vector (optical flow: hereinafter simply referred to as OPF) from images obtained at different times.
The method of extracting a moving object has been proposed. For example, Japanese Laid-Open Patent Publication No. 5-233813 discloses a method of calculating an OPF from a moving image obtained by a moving camera and detecting the movement of the camera and a moving object using information indicating the reliability of the OPF. Has been done.

However, this method is premised on the fact that the movement parameters of the camera, which is the reference when extracting a moving object, can be detected with high accuracy, and this premise may not always hold in an actual environment.

Therefore, the applicant of the present application has previously filed Japanese Patent Application No. 7-11.
In 8564, when a small region composed of a plurality of adjacent OPF groups is formed and it is determined that the start point and the component of the OPF group satisfy a predetermined relationship between the small regions and are similar, these small regions are grouped. A method of extracting the surface of a moving object is proposed. According to this, a moving object can be easily and surely extracted from the OPF obtained from the image regardless of the moving state of the camera.

[0005]

However, in this method, the surface of the moving object is assumed to be a plane perpendicular to the optical axis of the camera. There is a problem that it is not possible to know how far the extracted surface is with respect to the camera, that is, how much the surface of the moving object is tilted with respect to the optical axis of the camera, only by being able to detect the distance. there were.

The present invention has been made in view of the above-mentioned problems of the prior art, and an object thereof is to detect the tilt angle of a surface constituting a moving object with respect to an image pickup means such as a camera, and further to detect the three-dimensional shape of the moving object. An object is to provide a moving object detection device whose shape can be specified.

[0007]

In order to achieve the above object, a first aspect of the present invention is to provide an image pickup means for photographing an object and a movement vector for each small area of the obtained image, and to calculate each movement vector. In a moving object detection device having a surface detection unit that detects a surface forming an object by grouping the small areas based on the similarity of the above, the moving object detection apparatus is set for one of the grouped small areas. Distance calculation means for calculating an estimated value Za concerning the distance from the image pickup means which is included in another small area in the same group based on the predetermined value Z1 regarding the distance from the image pickup means, and the predetermined value Z
Angle calculation means for calculating the tilt angle of the surface forming the object with respect to the imaging means based on 1 and the estimated value Za.

Further, in order to achieve the above object, the second invention is the first invention, further comprising shape specifying means for specifying a three-dimensional shape of the object from a plurality of surfaces for which the tilt angles are calculated. It is characterized by having.

The basic principle of tilt angle calculation in the present invention will be apparent by referring to the following embodiments.

[0010]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a block diagram of the configuration of this embodiment. A camera 10 (designated as a focal length f) is provided as an imaging unit that captures an environment around the vehicle, and the obtained image is supplied to the electronic control unit ECU 12. Camera 10
The installation position may be the front side of the vehicle or the door mirror. The ECU 12 includes an A / D converter 12a,
VRAM 12b, 12c, digital signal processing circuit DSP
12d, CPU 12e, RAM 12f, ROM 12g,
And I / O 12h. A / D converter 1
2a sequentially converts the analog image signal from the camera 10 into a digital signal and outputs it to the VRAM 12b or 12c for storage. Image signals captured at different times are stored in the VRAMs 12b and 12c, and the OPF is calculated by the difference calculation in the digital signal processing circuit DSP12d and stored in the RAM 12f. Then, the CPU 12e as the distance calculation means and the angle calculation means performs a predetermined calculation process described later using the calculated OPF to calculate the tilt angle of the surface forming the moving object with respect to the camera 10. Further, the CPU 12e recognizes the three-dimensional shape of the moving object as a shape specifying unit and determines whether the moving object is a vehicle. The determination result is the alarm device 1 via the I / O 12h.
4 is output.

FIG. 2 shows a processing flowchart in this embodiment. First, the VRAMs 12b and 12
OPF is calculated from the images of different times stored in c (the first image and the second image) (S101). In this calculation, as in Japanese Patent Application No. 7-118564, the position where the preset window in the first image moves in the second image is determined by the correlation method (block matching) at the position with the highest correlation. It is calculated by It should be noted that although there is a case where the correlation cannot be obtained well in the window in which the change in the brightness contrast is small, it is preferable to remove such an OPF having low reliability by using a threshold parameter such as the degree of change in the correlation value.

After the OPF is calculated, the movement vector of the vertical plane of the small area including the OPF group having relatively high reliability of the boundary area of the moving body and the inclination angle of the surface are calculated (S1).
02). This processing is the core processing of this embodiment, and this processing will be described in detail below.

FIG. 3 shows an observation coordinate system used in this embodiment. The X-axis, Y-axis, and Z-axis are fixed with the installation position of the camera 10 as the origin. The optical axis of the camera 10 is Z
It matches the axis. Further, it is assumed that the XZ plane is parallel to the road surface. The projection point of one point P (X, Y, Z) of the object on the two-dimensional image is defined as p (x, y). Image side is X-
It is parallel to the Y plane and Z = f. When the point P (X, Y, Z) is translated by the motion vector (U, V, W), the motion vector of the projection point p, that is, OPF is t.
(U, v). It should be noted that even if the point P is rotating, this rotating motion vector is not important for extracting a moving object and is ignored in this embodiment.

When viewed from the camera 10, the OPF caused by an approaching object (or an object moving away) is always a radial vector centered on one point,

[Expression 1] u = f · U / Z−x · W / Z v = f · V / Z−y · W / Z (1) The formula (1) is expressed in Japanese Patent Application No. 7-118.
It is equivalent to (Equation 5) and (Equation 6) of No. 564. This equation (1) is non-linear with respect to a parameter (hereinafter referred to as depth parameter) Z related to distance. Therefore, Z on both sides
And a predetermined value Z1 as Z (for example, Z1 =
Setting 1) gives a linear equation as follows.

[0016]

## EQU00002 ## U−x / f · W = Z1 · u V−y / f · W = Z1 · v (2) Here, as shown in FIG. 4, the moving object is a vehicle, and Consider the four OPFs in a small area located in one moving vertical plane (constituting the vehicle as a solid consisting of four vertical planes) that composes. It is assumed that two of these four OPFs are on the same vertical line and the Z parameter is unknown Za, and the remaining two OPFs are on the same vertical line and the Z parameter is known Z1. By applying the above equation (2) to these four OPFs, a redundant linear equation is obtained with respect to unknown parameters U, V, W and Za.

[0017]

Ω · a = Ψ (3) where

(Equation 4)

(Equation 5)

(Equation 6) It is. In this way, the matrix Ω and the vector Ψ have known parameters or measured values as components. Actually, since the matrix Ω and the component of the vector Ψ include a measurement error, it is necessary to evaluate the vector a by the least square method.

[0018]

## EQU00007 ## A = inv [t.OMEGA..OMEGA.]. OMEGA..psi. (7) where A is the evaluation value of a, and inv [t.OMEGA..OMEGA.] Is the inverse matrix of the matrix t.OMEGA..OMEGA. , TΩ is a transposed matrix of Ω. As a result, the motion vector T (U, V, W) and the arbitrary depth parameter Za can be obtained.

Further, in FIG. 4, X- of the moving vertical plane is used.
The inclination angle D of the straight line of the orthogonal projection on the Z plane is

## EQU8 ## D = arctan [(Za-Z1) / (Xa-X1)] (8) On the other hand, the point P (X, Y, Z) and the point p (x,
between y),

Since there is a relation of x = −f · X / Z y = −f · Y / Z (9), the tilt angle D is eventually calculated from the equations (8) and (9).

## EQU10 ## D = arctan [f (Za-Z1) / (X2.Z1-X1.Za)] ... (10) As a result, the inclination of the moving vertical plane with respect to the Z axis can be obtained from the evaluation value Za, the predetermined value Z1, and the measured values X1 and X2. In addition, motion parameters (U, V, W)
Ratio of the vertical plane and the direction D of the vertical plane are set value Z1 of the depth parameter.
It is clear that the value of is not affected by itself. Further, in the present embodiment, the calculation is performed for four OPFs in the small area, but it goes without saying that the calculation is not limited to this.

As described above, the motion vector (U, V, W) of the small area and the tilt angle D are calculated, and the process of S102 ends. When this process is executed for each small area and the motion vector and tilt angle are calculated for all small areas,
Extract a small area group with the same motion vector ratio and slope,
These are grouped to reconstruct a plane (S10).
3). Since the small areas in each of the four vertical planes that form the vehicle should have the same motion vector ratio and the same tilt angle, the vertical planes that will form the vehicle by this grouping process. Can be reliably extracted.

After extracting the vertical planes, it is ascertained whether these vertical planes do indeed constitute the vehicle. That is,
It is determined whether or not the extracted planes are adjacent to each other and the angle between the planes is substantially right, and the planes that satisfy the conditions are combined to configure a vehicle as a moving object (S10).
4). In this way, the three-dimensional shape of the moving object (vehicle in this embodiment) can be grasped. After recognizing the shape of the moving object, the distance from the position on the screen to the recognized vehicle is estimated and the actual speed is calculated (S105).

In this embodiment, the vehicle as a moving object is assumed to be composed of four vertical planes, but it goes without saying that other models can be applied. However, this 4
The surface solid model has an advantage that the motion vector of the moving body can be obtained from several OPFs around the boundary of the vehicle. From the three-dimensional shape composed of the extracted vertical plane and its size,
There is also an advantage that it can be easily determined whether or not the moving body is a vehicle.

[0023]

As described above, according to the present invention,
The moving direction of the moving object can be detected, and the three-dimensional shape of the moving object can be recognized.

[Brief description of the drawings]

FIG. 1 is a configuration block diagram of an embodiment of the present invention.

FIG. 2 is a processing flowchart of the embodiment.

FIG. 3 is an explanatory diagram of an observation system of the same embodiment.

[Fig. 4] OP of the moving vertical plane and image plane of the same embodiment
F is an explanatory diagram.

[Explanation of symbols]

 10 camera, 12 ECU, 14 alarm device.

Claims (2)

[Claims] 1. An image pickup means for photographing an object, a movement vector is calculated for each small area of the obtained image, and the object is configured by grouping the small areas based on the similarity of each movement vector. In a moving object detecting device having a surface detecting means for detecting a surface to be moved, in a same group based on a predetermined value Z1 related to a distance from the image pickup means set for one of the grouped small areas. Based on the distance calculation means for calculating an estimated value Za relating to the distance from the image pickup means included in another small area, and the tilt angle of the surface constituting the object with respect to the image pickup means, based on the predetermined value Z1 and the estimated value Za. A moving object detecting device comprising: an angle calculating unit for calculating. 2. The moving object detecting apparatus according to claim 1, further comprising shape specifying means for specifying a three-dimensional shape of the object from the plurality of surfaces for which the tilt angles have been calculated. Detection device.