JP2011248830A - Moving body behavior detection device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving body behavior detection device, etc. that can distinguish moving body's behavior in the yaw direction from one in the lateral direction with a high degree of accuracy.SOLUTION: A moving body behavior detection device acquires images imaged by a frontward imaging camera and a backward imaging camera and measures a frontward feature point Pf included in the frontward direction of a vehicle 1 and a backward feature point Pb included in the backward direction different from the frontward direction. A comparison is made between the movement direction (Pf'→Pf) of the frontward feature point Pf included in the frontward direction of the vehicle 1 and the movement direction (Pb'→Pb) of the backward feature point Pb included in the backward direction of the vehicle. If the movement directions of these points are different, it is determined that the vehicle 1 is moving in the lateral direction; if the movement directions are the same, it is determined that the vehicle 1 is moving in the yaw direction.

Description

  The present invention relates to a moving body behavior detection apparatus and method.

  Conventionally, as a technique for detecting a change in behavior of the host vehicle, a vehicle behavior detection device described in Patent Document 1 below is known.

  This vehicle behavior detection apparatus detects a feature point far away from the host vehicle as a feature point using an image obtained by imaging the front of the vehicle. This vehicle behavior detection device monitors the movement of the distant feature point and detects a change in the behavior of the host vehicle. This vehicle behavior detection device detects the behavior of the host vehicle in the yaw direction based on the movement of the remote feature point by utilizing the fact that the feature point in the distance is not affected by the behavior change in the forward direction of the host vehicle. ing.

JP 2006-338272 A

  When the above-described vehicle behavior detection device cannot detect feature points far away, all the feature points in the image are affected by the behavior of the host vehicle and move in the image. For this reason, the vehicle behavior detection device described above has a problem that the behavior of the host vehicle in the yaw direction and the lateral direction cannot be distinguished.

  Therefore, the present invention has been proposed in view of the above-described circumstances, and an object of the present invention is to provide a moving body behavior detection apparatus and method capable of distinguishing the behavior of a moving body in the yaw direction and the lateral direction with high accuracy. .

  The present invention compares the moving directions of the first feature point and the second feature point included in different directions with respect to the moving body, and when the moving directions of the both are different, the moving body moves in the lateral direction. If both moving directions are the same, it is determined that the moving body is moving in the yaw direction.

  According to the present invention, the moving direction of the moving object can be determined based on whether the moving direction of the feature points included in different directions with respect to the moving object is different or the same. Can be distinguished.

In the vehicle behavior detection system shown as 1st Embodiment of this invention, it is a top view which shows arrangement | positioning of a front imaging camera and a rear imaging camera. It is a block diagram which shows the functional structure of the vehicle behavior detection system shown as 1st Embodiment of this invention. In the vehicle behavior detection system shown as 1st Embodiment of this invention, it is a figure which shows the moving direction of the feature point when a moving body moves to a horizontal direction and a rotation direction, (A) is a moving body moving to a horizontal direction. (A) Front feature point, (b) Movement of rear feature point, (B) (a) Front feature point, (b) Movement of rear feature point when moving body moves in rotation direction Indicates. It is a flowchart which shows the process sequence which estimates the behavior of a moving body in the vehicle behavior detection system shown as 1st Embodiment of this invention. It is a flowchart which shows the process sequence which estimates the position of a feature point in the vehicle behavior detection system shown as 1st Embodiment of this invention. In the vehicle behavior detection system shown as 1st Embodiment of this invention, it is a figure explaining the gap | deviation of a prior estimated value and an actual value. In the vehicle behavior detection system shown as 1st Embodiment of this invention, it is a figure which shows the positional relationship of the attention feature point with respect to a moving body. In the vehicle behavior detection system shown as 1st Embodiment of this invention, it is a figure which shows a mode that coordinate conversion was performed so that an attention feature point might become az axis direction. In the vehicle behavior detection system shown as 1st Embodiment of this invention, it is a figure which shows a mode that it set with another attention feature point in the opposite direction of a attention feature point. In the vehicle behavior detection system shown as the first embodiment of the present invention, (A) shows (a) the front feature point and (b) the movement of the rear feature point when the moving body moves in the lateral direction, (B) (A) shows the movement of the front feature point and (b) the rear feature point when the moving body moves in the rotation direction. In the vehicle behavior detection system shown as 2nd Embodiment of this invention, it is a top view explaining that a back imaging camera images with the breadth of the range of 90 degree | times with respect to the imaging direction of a front imaging camera.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
The vehicle behavior detection system shown as the first embodiment of the present invention detects the behavior of a moving object. This vehicle behavior detection system detects the behavior of the host vehicle, for example. For example, the vehicle behavior detection system detects the behavior of the host vehicle in order to warn the driver based on the behavior of the host vehicle.

  As shown in FIG. 1, the vehicle behavior detection system includes a front imaging camera 12 provided in front of the host vehicle 1 and a rear imaging camera 13 provided in the rear of the host vehicle 1. As shown in FIG. 2, the vehicle behavior detection system includes a calculation device 11, a front imaging camera 12, a rear imaging camera 13, and a surrounding feature storage unit 14.

  The front imaging camera 12 is attached with the front of the host vehicle 1 as the imaging direction. The front imaging camera 12 captures front captured images that are temporally continuous every predetermined time and supplies the captured images to the arithmetic device 11. The front imaging camera 12 is attached to the host vehicle 1 and corresponds to first imaging means that captures a first image that is temporally continuous.

  The rear imaging camera 13 is attached with the rear of the host vehicle 1 as the imaging direction. The rear imaging camera 13 captures rear captured images that are temporally continuous at predetermined time intervals and supplies the captured images to the arithmetic device 11. The rear imaging camera 13 is attached to the host vehicle 1 so as to be in the imaging direction opposite to the imaging direction of the first imaging unit, and corresponds to a second imaging unit that captures a temporally continuous second image.

  The front imaging camera 12 and the rear imaging camera 13 are attached with an angle of 180 degrees between the optical axis direction of the front imaging camera 12 and the optical axis direction of the rear imaging camera 13.

  The surrounding feature storage unit 14 stores a feature point around the host vehicle 1 and position information of the feature point in association with each other. The surrounding feature storage unit 14 includes, for example, a map database in which feature points and position information measured in advance are registered. The feature points include partial images obtained by performing edge detection processing on surrounding images such as edges and corners of an object. The surrounding feature storage unit 14 is an information recording / reproducing apparatus capable of recording / reproducing information on / from a recording medium such as a hard disk.

  It should be noted that the surrounding feature storage unit 14 may perform processing for sequentially storing feature points and position information online using an existing method such as Motion Stereo while the host vehicle 1 is moving.

  The arithmetic device 11 includes a front feature detection unit 21, a front position acquisition unit 22, a front imaging range acquisition unit 23, a front feature selection unit 24, a forward movement measurement unit 25, a rear feature detection unit 26, a rear position acquisition unit 27, and a rear imaging. A range acquisition unit 28, a rear feature selection unit 29, a rear movement measurement unit 30, and a moving body behavior determination unit 31 are provided.

  The arithmetic device 11 is actually composed of a ROM, a RAM, a CPU, and the like, but functions that can be realized by the CPU performing a process according to a vehicle behavior detection program stored in the ROM as a block. explain. That is, hereinafter, although the physical configuration of the arithmetic device 11 is single, the functional units 21 to 31 of the arithmetic device 11 will be described in accordance with the processing contents that can be executed by the arithmetic device 11 by a program.

  The front feature detection unit 21 is supplied with a front captured image captured by the front imaging camera 12. The front feature detection unit 21 performs an edge detection process or the like on the front captured image and detects a front feature point included in the front captured image. This forward feature point is a partial image detected from an actual forward captured image, like the edges and corners stored in the surrounding feature storage unit 14.

  Similarly, the rear imaging range acquisition unit 28 is supplied with a rear captured image captured by the rear imaging camera 13. The rear imaging range acquisition unit 28 performs edge detection processing or the like on the rear captured image and detects a rear feature point included in the rear captured image. The rear feature point is a partial image detected from the actual rear captured image, like the edges and corners stored in the surrounding feature storage unit 14.

  The front position acquisition unit 22 estimates the position of the front feature point in the front captured image captured by the front imaging camera 12.

  Similarly, the rear position acquisition unit 27 estimates the position of the rear feature point in the rear captured image captured by the rear imaging camera 13.

  The front position acquisition unit 22 and the rear position acquisition unit 27 estimate the position of the host vehicle 1 in advance of actually capturing images with the front imaging camera 12 and the rear imaging camera 13 and detecting feature points. The prior estimated value of the position of the host vehicle 1 can be obtained by various methods. For example, the position of the host vehicle 1 may be estimated using an infrastructure such as GPS (Global Positioning System), or the position of the host vehicle 1 may be estimated directly from the rotation amount of the wheel of the host vehicle 1. .

  The forward imaging range acquisition unit 23, the forward feature selection unit 24, and the forward movement measurement unit 25 detect the moving direction of the forward feature points detected by the forward feature detection unit 21.

  The front imaging range acquisition unit 23 acquires the front imaging range of the front imaging camera 12 based on the position of the host vehicle 1 estimated by the front position acquisition unit 22 and the attachment position of the front imaging camera 12. The front feature selection unit 24 compares the position information acquired by the surrounding feature storage unit 14 with the front imaging range, and selects a front feature point included in the front imaging range. The forward movement measuring unit 25 determines the moving direction of the forward feature point from the deviation between the forward feature point included in the forward captured image captured by the forward imaging camera 12 and the forward feature point selected by the forward feature selecting unit 24. calculate.

  That is, the forward movement measuring unit 25 includes the position of the forward feature point estimated to appear in the imaging range of the front imaging camera 12 among the feature points stored in the surrounding feature storage unit 14, and the estimated forward feature point. The moving direction of the actual front feature point with respect to the position is measured.

  Similarly, the rear imaging range acquisition unit 28, the rear feature selection unit 29, and the rear movement measurement unit 30 detect the movement direction of the rear feature point detected by the rear feature detection unit 26.

  The rear imaging range acquisition unit 28 acquires the rear imaging range of the rear imaging camera 13 based on the position of the host vehicle 1 estimated by the rear position acquisition unit 27 and the attachment position of the rear imaging camera 13. The rear feature selection unit 29 compares the position information acquired by the surrounding feature storage unit 14 with the rear imaging range, and selects a rear feature point included in the rear imaging range. The backward movement measuring unit 30 determines the moving direction of the backward feature point from the deviation between the backward feature point included in the backward captured image captured by the backward imaging camera 13 and the backward feature point selected by the backward feature selecting unit 29. calculate.

  That is, the backward movement measuring unit 30 includes the position of the rear feature point estimated to appear in the imaging range of the rear imaging camera 13 among the feature points stored in the surrounding feature storage unit 14, and the estimated rear feature point. The moving direction of the actual rear feature point with respect to the position is measured.

  When the front imaging range acquisition unit 23 and the rear imaging range acquisition unit 28 acquire an imaging range, the prior estimated value of the host vehicle position acquired by the front position acquisition unit 22 and the rear position acquisition unit 27 is used. Thereby, the front imaging range acquisition unit 23 and the rear imaging range acquisition unit 28 calculate an area that can be observed from the front imaging camera 12 and the rear imaging camera 13. At this time, the front imaging range acquisition unit 23 and the rear imaging range acquisition unit 28 determine the positions of the feature points on the world coordinates stored in the surrounding feature storage unit 14 in advance, and the front imaging camera 12 and the rear imaging camera 13. Based on the attachment position, the position of the feature point in the front imaging range and the rear imaging range is estimated. Further, the front imaging range acquisition unit 23 and the rear imaging range acquisition unit 28 consider the attachment direction of the front imaging camera 12 and the rear imaging camera 13 and camera internal parameters (view angle and distortion) with higher accuracy. It is also possible to estimate the position of the feature point within the range and the rear imaging range.

  The moving body behavior determination unit 31 compares the moving direction of the forward feature point measured by the forward movement measuring unit 25 with the moving direction of the backward feature point measured by the backward movement measuring unit 30. The moving body behavior determination unit 31 determines that the host vehicle 1 is moving in the lateral direction when the two moving directions are different. The moving body behavior determination unit 31 determines that the host vehicle 1 is moving in the yaw direction when both of the movement directions are the same.

  Here, as shown in FIG. 3, the behavior of the feature point in the yaw direction of the host vehicle 1 and the behavior of the feature point in the lateral direction of the host vehicle 1 are the movement direction of the front feature point and the movement direction of the rear feature point. Appear as different movements.

  When the host vehicle 1 moves in the lateral direction, as shown in FIG. 3A, the feature point P ′ of the pre-estimated position in the front imaging range 40f is actually a feature point that has moved in the left direction. Appears at position P. On the other hand, when the host vehicle 1 moves in the lateral direction, the feature point P ′ of the pre-estimated position in the rear imaging range 40b is actually in the right direction as shown in FIG. It appears at the position of the moved feature point P. In this way, when the host vehicle 1 moves in the lateral direction, the feature point P is shifted in the “different” movement direction between the front and the rear.

  When the host vehicle 1 moves in the rotational direction, as shown in FIG. 3B, the feature point P ′ of the pre-estimated position in the front imaging range 40f is actually a feature point that has moved in the left direction. Appears at position P. On the other hand, when the host vehicle 1 moves in the rotational direction, the feature point P ′ of the prior estimated position in the rear imaging range 40b is actually leftward as shown in FIG. It appears at the position of the moved feature point P. As described above, when the host vehicle 1 moves in the rotation direction, the characteristic points are shifted in the “same” movement direction between the front and the rear.

  By utilizing this, the vehicle behavior detection system can discriminate between the behavior of the host vehicle 1 in the yaw direction and the behavior of the host vehicle 1 in the lateral direction.

  Next, a processing procedure for estimating the behavior of the host vehicle 1 in the vehicle behavior detection system described above will be described with reference to the flowcharts of FIGS. 4 and 5. Note that the vehicle behavior detection system estimates the behavior of the host vehicle 1 every predetermined time by executing the processes after step S1 every predetermined time.

  First, in step S1, a front captured image is acquired by the front imaging camera 12, and a rear imaging range is acquired by the rear imaging camera 13 in step S4. Here, the acquisition timing of the front captured image by the front imaging camera 12 and the acquisition timing of the rear captured image by the rear imaging camera 13 are synchronized.

  Next, in step S2, the position in the front captured image of the front feature point is estimated, and in step S5, the position in the rear captured image of the rear feature point is estimated.

Steps S2 and S5 perform the processing shown in FIG. First, in step S11, the front position acquisition unit 22 and the rear position acquisition unit 27 estimate the position of the host vehicle 1 (moving body) in advance. The pre-estimation of the vehicle position does not use images captured by the front imaging camera 12 and the rear imaging camera 13. The front position acquisition unit 22 and the rear position acquisition unit 27 use, for example, a motion model of the host vehicle 1, a control command value for generating a torque for moving the host vehicle 1, and other sensors such as GPS, and the world coordinates The center-of-gravity position E of the host vehicle 1 is obtained with respect to the system. Here, the center-of-gravity position E of the host vehicle 1 is expressed as shown in Equation 1 below.

  In Equation 1, R represents a 3 × 3 rotation matrix, and t represents a third-order movement vector. The position of the host vehicle 1 can be estimated by various other methods.

The front position acquisition unit 22 and the rear position acquisition unit 27 estimate the position of the host vehicle 1 in advance using the control command value of the host vehicle 1, for example. Assume that the center-of-gravity position E _t-1 of the host vehicle 1 before the control of the host vehicle 1 is obtained, and a straight-running control command value V _Z [m / s] as a control command value of the host vehicle 1 and a rotation control command value ω. Assume that control by [rad / s] is performed. However, there is an error in the control of the host vehicle 1, and even if the host vehicle 1 is controlled as controlled, the host vehicle 1 may not move as controlled due to friction with the road surface with respect to the host vehicle 1 or the like. Absent. Therefore, the expected value of the host vehicle position after the control is performed,
E _prior = M _c · E _t−1
This is set as the prior estimated value E _prior of the _host vehicle 1. Here, _Mc is expressed by the following Equation 2.

  In the next step S12, the front imaging range acquisition unit 23 and the rear imaging range acquisition unit 28, the pre-estimated value of the host vehicle position acquired by the front position acquisition unit 22 and the rear position acquisition unit 27, the front imaging camera 12, and The imaging ranges of the front imaging camera 12 and the rear imaging camera 13 are acquired from the attachment position of the rear imaging camera 13.

Here, the attachment position of the front imaging camera 12 and the rear imaging camera 13 with respect to the vehicle 1 of the gravity center position E and C _f. At this time, the pre-estimated value of the attachment position of the front imaging camera 12 and the rear imaging camera 13 in the world coordinate system is
C _fprior = C _f · E _prior
Can be expressed as At this time, the imaging range imaged from the pre-estimated value C _fprior of the attachment positions of the front imaging camera 12 and the rear imaging camera 13 is the world coordinates of the pre-estimated value C _fprior of the attachment positions of the front imaging camera 12 and the rear imaging camera 13. Represented by converting to centered coordinates. The imaging ranges of the front imaging camera 12 and the rear imaging camera 13 are equal to or greater than a certain value in the world coordinate system (x, y, z), and the absolute values of x / z and y / z are certain values. It can be expressed as a range below.

In the next step S <b> 13, the front feature selection unit 24 and the rear feature selection unit 29 are within the imaging range of the front imaging camera 12 and the rear imaging camera 13 among the feature points stored in the surrounding feature storage unit 14. Select the feature points considered. Here, it is assumed that the feature points are represented in advance as position information P _i as a world coordinate system in the surrounding map. When the position information P _i in the world coordinate system is expressed in the camera coordinate system, the position information P _{ic in the} camera coordinate system is
P _ic = C _fprior ⁻¹ · P _i
Can be expressed as The front feature selection unit 24 and the rear feature selection unit 29 use the position information P _ic of the camera coordinate system, and the movement vector z component of the position information P _ic34 of the camera coordinate system is greater than or equal to a certain value, P _ic14 / P _ic34 , A feature point that is considered to be within the imaging range of the front imaging camera 12 and the rear imaging camera 13 is selected on condition that the absolute value of P _ic24 / P _{ic 34} is below a certain value. _{Here, P IC 14} is the motion vector x component of the position information _{P ics} of the camera coordinate _{system, P IC 24} is the motion vector y component of the position information _{P ics} of the camera coordinate system.

Returning to FIG. 4, in step S <b> 3, the estimated position of the forward feature point in the forward captured image estimated in step S <b> 2 by the forward movement measuring unit 25 and the position of the forward feature point in the actually captured forward captured image. The deviation d _p is measured. At this time, the forward movement measurement unit 25 acquires the forward feature point and the position in the image detected from the actual forward captured image detected by the forward feature detection unit 21. Similarly, in step S6, the estimated position of the rear feature point in the rear captured image estimated in step S5 by the rear movement measuring unit 30 and the position of the rear feature point in the actually captured rear captured image. The deviation d _p is measured. At this time, the backward movement measurement unit 30 acquires the backward feature point and the position in the image detected from the actual backward captured image detected by the backward feature detection unit 26.

In the next step S7, the behavior of the host vehicle 1 is estimated by the moving body behavior determination unit 31 from the deviation d _p measured in steps S3 and S6. At this time, mobile behavior determination unit 31, a moving direction indicated by the deviation d _p in the front characteristic points measured in step S3, is represented by the deviation d _p in the rear feature point that is measured at step S6 Compare the moving direction. The moving body behavior determination unit 31 determines that the host vehicle 1 is moving in the yaw direction when the movement direction of the front feature point is the same as the movement direction of the rear feature point. The moving body behavior determination unit 31 determines that the host vehicle 1 is moving in the lateral direction when the moving direction of the front feature point is different from the moving direction of the rear feature point. Thereby, the vehicle behavior detection system can estimate the behavior of the host vehicle 1.

In step S7, the amount of movement of the host vehicle 1 deviates from the amount of movement of the host vehicle 1 estimated in advance from the deviation d _p between the front feature point and the rear feature point obtained in step S3 and step S6. Estimate whether it is fun.

  Next, a process for estimating the behavior of the host vehicle 1 by the vehicle behavior detection system will be specifically described.

  First, as described above, the front imaging range of the front imaging camera 12 and the rear imaging range of the rear imaging camera 13 are opposite to each other, and the moving body behavior determination unit 31 captures the front imaged by the front imaging camera 12. Calculation of the amount of rotation of the host vehicle 1 based on the difference between the movement of the front feature point in the captured image and the movement of the rear feature point in the rear captured image captured by the rear imaging camera 13 will be described.

In the three-dimensional coordinate system shown in FIG. 6, the prior estimated value E _prior (40 ′) of the _host vehicle position when the center of gravity position E of the host vehicle 1 is 41 ′ and the center of gravity position E of the _host vehicle 1 are 41. As for the deviation from the actual value (40) of the host vehicle position at that time, the deviation in the x direction is represented as M _x , the deviation in the z direction is represented as M _z , and the deviation in the rotation direction is represented as M _θ . In FIG. 6, the previous direction of the host vehicle 1 is represented by 42 ′, and the actual direction of the host vehicle 1 is represented by 42.

Further, as shown in FIG. 7, select any feature point of the plurality of feature points selected in step S13, the selected feature points and feature point P _1, feature point in the camera coordinate system The position in the x direction where P ₁ exists is P _1x , and the position in the z direction is P _1z . In this case, deviation _{d P1x} in the x direction in the image of the target feature point _{P 1} is
d _P1x = −aM _θ −f (M _x , M _z , P _x , P _z ) (Formula 3)
Can be expressed as Function f in the above formula 3 indicates the amount of movement in the image by the own displacement of the vehicle 1 other than the deviation M _theta in the direction of rotation. “A” in the above formula is a constant determined by the angle of view of the front imaging camera 12 and the rear imaging camera 13.

Here, in order to facilitate the behavior calculation of the vehicle 1, as shown in FIG. 8, with a focus on the vehicle 1 of the gravity center position E, such as the central axis of the target feature point P ₁ is the z-axis direction Perform coordinate transformation. When the rotation angle of the coordinate transformation is sufficiently small, the deviation d _px1 the feature point P ₁ in the image after the coordinate transformation _'includes a shift d _px1 the feature point P ₁ conversion before the image shown in FIG. 7 Almost equal,
d _px1 ' _{≈d px1} (Formula 4)
It becomes. The deviation (−M _θ ) in the rotation direction of the host vehicle 1 is not affected by coordinate conversion due to the rotation of the host vehicle 1. X position after the conversion of the target feature point _{P 1 P 1x} ', z direction position _{P 1z'} is due to the presence of target feature point _{P 1} in the z-axis direction,
P _1x '= 0 (Formula 5)
P _1z ′ = {((P _1x ) ² + (P _1z ) ² )} ^1/2 (Formula 6)
Represented as: When the position of the feature point P ₁ after conversion is P _1x ′ = 0 as shown in Equation 5, on the image, the movement of the displacement M _z ′ of the host vehicle 1 is not affected, and the displacement d _p on the image Is determined only by the deviation M _θ in the rotation direction of the host vehicle 1 and the deviation M _x ′ after conversion in the x direction of the host vehicle 1. Since the movement of the image due to the deviation M _x ′ after conversion in the x direction of the _host vehicle 1 is inversely proportional to the distance (P _1z ′) from the _host vehicle 1 to the feature point P ₁ in the z direction,
-M _x '/ P _1z ' (Formula 7)
Represented as: As a result, the deviation dP _1x of the feature point of interest P ₁ in the x direction in the image is expressed as
_{d P1x = -aM θ -M x '} / P 1z' ( Eq. 8)
Represented as:

'And the self displacement M _x after the conversion in the x direction of the vehicle _1' deviation M _theta converted in the rotation direction theta of the vehicle 1 for obtaining a feature point of the different position from the feature point P ₁ P ₂ Select. The feature point P _2, from the viewpoint of accuracy of measuring the movement direction of forward feature point and the rear feature point, in the neighborhood of z axis after transformation, it is desirable that as far as possible from the feature point P ₁ .

This, if there is a current feature point P ₂ on the z axis after the conversion, even if the movement of the z-axis parallel to the vehicle 1 after conversion, feature point P ₂ in the image does not move . Therefore, even when the vehicle 1 moves, it is possible to consider ignoring the z direction of movement of the feature point P ₂ in the image. On the other hand, if are out of target feature point P ₂ from the z-axis after the transformation, when the parallel vehicle 1 and z axis after the conversion is moved, the image position of the target feature point P ₂ is or x direction y It will shift in the direction. For this reason, the precision which measures the moving direction of a front feature point and a back feature point will fall.

In step S7 described above, the behavior of the vehicle 1, based on the deviation d _P1x in the x direction of the feature point P ₁ in the image, the yaw movement or is lateral movement of the discrimination. This determination utilizes the fact that the movement of the own vehicle 1 in the same direction acts on both the front imaging camera 12 and the rear imaging camera 13. For this reason, there is no difference between the front feature point and the rear feature point in the term of the deviation M _θ ′ after conversion in the rotation direction θ of the host vehicle 1. Therefore, in order to calculate the shift M _θ ′ after conversion in the rotational direction θ of the _host vehicle 1 and the shift M _x ′ after conversion in the x direction of the _host vehicle 1, M _x ′ / P _1z in Equations 7 and 8 A difference needs to be generated in the term ', and the larger the difference, the more accurately the behavior of the host vehicle 1 can be obtained. Therefore, it is desirable to select a feature point having a distance as large as possible from the target feature point P ₁ as the target feature point P ₂ .

In this case, when the feature point of interest P ₁ is present in the front captured image captured by the front imaging camera 12, the feature point of interest P ₂ is present in the opposite direction of the front imaging camera 12 with respect to the host vehicle 1. It is desirable. For example, as shown in FIG. 9, among the feature points captured by the rear image pickup camera 13, it selects feature points located in the vicinity of the z 'axis of the converted as feature point P _2. Shift amount _{d p2x} of the feature point _{P 2,} as shown in Equation 9 below,
_{d P2x = -aM θ -M x '} / P 2z' ( Equation 9)
Represented as: Formula 8, with respect to formula 9, x-direction of the deviation _{d p1x} in the image of the target feature point _{P 1,} x direction position _{P 1x} the feature point _{P 1} is present, z direction position _{P 1z} which feature point _{P 1} is present , the constant a is determined by the angle of the front imaging camera 12 and the rear imaging camera 13, the deviation amount _{d P2x} in the x direction of the feature point _{P 2,} z direction position P2 _z where feature point _{P 2} is present in a known is there. Therefore, the difference between the deviation d _P1x the feature point P ₁ in the x-direction in the image obtained by the Equation 8, a deviation d _P2x of feature point P ₂ in the x-direction in the image obtained by the formula 9 Thus, it is possible to determine the movement amount M _{θ in} the rotation direction of the host vehicle 1 and the movement amount M _x ′ in the x direction of the host vehicle 1. Further, the movement amount M _x in the x direction of the vehicle 1 _', the movement amount M _x in the x direction in the pre-conversion _coordinates, since a synthesis component of the movement amount M _z in the z-direction, before the coordinate transformation by projecting the coordinate axis, the movement amount M _x in the x direction in the pre-conversion _coordinates, it is possible to determine the movement amount M _z in the z-direction.

As described above, according to this vehicle behavior detection system, when the host vehicle 1 moves in the lateral direction, as shown in (a) of FIG. 10A, the feature of the pre-estimated position in the front imaging range _50f . The feature point P _f ′ estimated in advance appears in actuality at the position of the feature point P _f moved in the left direction. In contrast, as shown in FIG. 10A (b), the feature point P _b ′ estimated in advance in the rear imaging range 50 _b is actually at the position of the feature point P _b moved in the right direction. appear. Thus, when the host vehicle 1 moves in the lateral direction, the pre-feature feature point P ′ and the actual feature point P are shifted in the “different” movement directions in the forward and backward directions.

When the host vehicle 1 moves in the rotational direction, as shown in (a) of FIG. 10B, the feature point P _f ′ estimated in advance in the front imaging range 50 _f actually moves to the left. It appears at a position of the feature point _{P f.} On the other hand, as shown in (b) of FIG. 10A, the feature point P _b ′ estimated in advance in the rear imaging range 50 _b is actually at the position of the feature point P _b moved in the left direction. appear. As described above, when the host vehicle 1 moves in the rotation direction, the characteristic points are shifted in the “same” movement direction between the front and the rear.

  As described above, the vehicle behavior detection system automatically determines whether the surrounding feature points stored in advance and the feature points appearing in the captured images captured by the front imaging camera 12 and the rear imaging camera 13 have the same or different moving directions. The behavior of the vehicle 1 can be detected.

  As described above in detail, according to the vehicle behavior detection system shown as the first embodiment, when the movement direction of the front feature point and the movement direction of the rear feature point are compared, the movement directions of the two feature points are different. It can be determined that the own vehicle 1 is moving in the lateral direction, and if both the moving directions are the same, it can be determined that the own vehicle 1 is moving in the yaw direction. Thus, according to the vehicle behavior detection system, the behavior of the host vehicle 1 in the yaw direction and the behavior in the lateral direction can be distinguished with high accuracy, and the position of the host vehicle 1 can be obtained.

  For example, when realizing a parking operation support function of the host vehicle 1, it is very important to obtain what positional relationship the host vehicle position is with respect to the target position (Localization). For example, when parking in the same way at the position where the host vehicle 1 was parked before, it is necessary to accurately grasp the position where the host vehicle 1 was parked before and the current position of the host vehicle 1. On the other hand, the vehicle behavior detection system can detect the behavior of the host vehicle 1 and accurately detect the position of the host vehicle 1. In addition, a movement locus can be obtained by moving while always performing position detection. If this trajectory is followed from the next time, it becomes possible to control the host vehicle 1 without performing complicated route calculation.

  In addition, according to the vehicle behavior detection system, the feature points are stored in advance, and the movement direction of the front feature point and the movement direction of the rear feature point are determined from the feature points obtained in advance and the actually obtained feature points. Thus, the behavior of the host vehicle 1 in the yaw direction and the behavior in the lateral direction can be distinguished with high accuracy, and the position of the host vehicle 1 can be obtained.

  Furthermore, according to this vehicle behavior detection system, since the front imaging range of the front imaging camera 12 and the rear imaging range of the rear imaging camera 13 are opposite, when the host vehicle 1 moves in the lateral direction, By using the fact that feature points shift in different movement directions at the rear and the own vehicle 1 moves in the rotation direction, the feature points shift in the same movement direction at the front and rear, thereby detecting the behavior of the host vehicle 1 with high accuracy. it can.

  Furthermore, according to this vehicle behavior detection system, the movement of the front feature point in the front captured image captured by the front imaging camera 12 and the movement of the rear feature point in the rear captured image captured by the rear imaging camera 13 Since the rotation amount of the own vehicle 1 is calculated based on the difference between the two, the lateral movement and the yaw direction rotation of the own vehicle 1 can be algebraically separated regardless of the motion state of the own vehicle 1. Thereby, the vehicle behavior detection system can obtain the behavior of the host vehicle 1 with a low calculation load.

[Second Embodiment]
Next, another process for estimating the behavior of the host vehicle 1 will be specifically described as the vehicle behavior detection system according to the second embodiment.

  In this vehicle behavior detection system, the rear imaging camera 13 captures an image in a 90-degree direction with respect to the imaging direction of the front imaging camera 12. Specifically, as shown in FIG. 11, the optical axis direction of the front imaging camera 12 is in front of the host vehicle 1. On the other hand, the optical axis direction of the rear imaging camera 13 is shifted by 180 degrees from the optical axis direction of the front imaging camera 12, but the imaging range of the rear imaging camera 13 is the optical axis direction of the front imaging camera 12. With a spread of 90 degrees.

  In this vehicle behavior detection system, the moving body behavior determination unit 31 calculates the amount of rotation of the host vehicle 1 based on the movement of the feature points included in the rear captured image captured by the rear imaging camera 13.

In such a vehicle behavior detection system, the deviation between the rear feature point estimated to be included in the rear imaging range of the rear imaging camera 13 and the rear feature point actually included in the rear imaging image is the deviation in the rotation direction. It is thought that it _depends only on Mθ. Therefore, in this case, the deviation d _P1x observed by the rear imaging camera 13 is
d _P1x = −M _θ (Formula 10)
Thus, the rotation amount of the host vehicle 1 can be obtained by the equation (10).

  Here, in the parking operation for stopping the host vehicle 1 on the parking frame, a situation is considered in which the host vehicle 1 is stopped at a position expected to be a stop position in the parking frame. At this time, the computing device 11 can recognize that the host vehicle 1 is not traveling in the straight direction by detecting the wheel rotation amount of the host vehicle 1 or the like. Further, it is assumed that it is possible to confirm that the stop position of the host vehicle 1 in the straight traveling direction is correct from the feature point group imaged by the front imaging camera 12 and the rear imaging camera 13.

  In this situation, the vehicle behavior detection system can accurately determine whether the own vehicle 1 is displaced laterally with respect to the stop position or whether the own vehicle 1 is displaced in the rotational direction with respect to the stop position. Thereby, the vehicle behavior detection system can detect the vehicle behavior in order to accurately set up a route plan for redoing the parking operation.

  As described above, according to the vehicle behavior detection system shown as the second embodiment, as in the first embodiment, the moving direction of the front feature point and the moving direction of the rear feature point are compared, and the own vehicle 1 movement in the lateral direction and movement in the yaw direction can be discriminated, and the behavior in the yaw direction and the behavior in the lateral direction of the host vehicle 1 can be distinguished with high accuracy, and the position of the host vehicle 1 can be obtained.

  Further, according to this vehicle behavior detection system, the rear imaging camera 13 is configured to capture a rear captured image in a direction of 90 degrees with respect to the imaging direction of the front imaging camera 12. Thereby, according to the vehicle behavior detection system, since the position of the own vehicle 1 can be estimated using the rear imaging camera 13, the position of the own vehicle 1 can be determined with high accuracy when the own vehicle 1 is stopped. It can be obtained.

  Furthermore, according to this vehicle behavior detection system, the amount of rotation of the host vehicle 1 can be calculated by the moving body behavior determination unit 31 based on the movement of the feature points included in the rear captured image captured by the rear imaging camera 13. Thereby, the vehicle behavior detection system can obtain the behavior of the host vehicle 1 with a low calculation load.

  The above-described embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and various modifications can be made depending on the design and the like as long as the technical idea according to the present invention is not deviated from this embodiment. Of course, it is possible to change.

  That is, although the above-described vehicle behavior detection system has been described for detecting the behavior of the host vehicle 1, it may be one that detects the behavior of another moving body, such as a robot.

DESCRIPTION OF SYMBOLS 1 Own vehicle 11 Computation device 12 Front imaging camera 13 Back imaging camera 14 Ambient feature memory | storage part 21 Front feature detection part 22 Front position acquisition part 23 Front imaging range acquisition part 24 Front feature selection part 25 Forward movement measurement part 26 Back feature detection part 27 rear position acquisition unit 28 rear imaging range acquisition unit 29 rear feature selection unit 30 backward movement measurement unit 31 moving body behavior determination unit

Claims (6)

A first imaging means attached to the moving body and capturing a first image that is temporally continuous;
A second imaging unit that is attached to the moving body so as to be in an imaging direction opposite to the imaging direction of the first imaging unit, and that captures a second image that is temporally continuous;
First feature detection means for detecting a first feature point from a first image imaged by the first imaging means;
Second feature detection means for detecting a second feature amount from the second image imaged by the second imaging means;
First movement measuring means for measuring a moving direction of the first feature point detected by the first feature detecting means;
Second movement measuring means for measuring the moving direction of the second feature point detected by the second feature detecting means;
When the moving direction of the first feature point measured by the first movement measuring unit and the moving direction of the second feature point measured by the second movement measuring unit are compared, and the two moving directions are different Comprises a moving body behavior determining means for determining whether the moving body is moving in the lateral direction, and determining that the moving body is moving in the yaw direction when both moving directions are the same. A moving body behavior detection device. Surrounding feature acquisition means for acquiring feature points around the preset moving body and position information of the feature points;
Position estimating means for estimating the position of the moving body;
First imaging range acquisition means for acquiring a first imaging range of the first imaging means based on the position of the moving body estimated by the position estimation means and the mounting position of the first imaging means;
Second imaging range acquisition means for acquiring the second imaging range of the second imaging section based on the position of the moving body estimated by the position estimation means and the mounting position of the second imaging section;
A first feature selection unit that compares the position information acquired by the surrounding feature acquisition unit with the first imaging range and selects a feature point included in the first imaging range;
A second feature selection unit that compares the position information acquired by the surrounding feature acquisition unit with the second imaging range and selects a feature point included in the second imaging range;
The first movement measuring means is a moving direction of the first feature point from a feature point included in the first image picked up by the first image pickup means and a feature point selected by the first feature selection means. To calculate
The second movement measuring means determines a moving direction of the second feature point from a feature point included in the second image picked up by the second image pickup means and a feature point selected by the second feature selection means. The mobile body behavior detection device according to claim 1, wherein the mobile body behavior detection device is calculated.   The moving body behavior determination unit includes a movement of the first feature point in the first captured image captured by the first imaging unit and a second feature point in the second captured image captured by the second imaging unit. The moving body behavior detecting apparatus according to claim 1, wherein the amount of rotation of the moving body is calculated based on a difference from the movement of the moving body.   The moving body behavior detection apparatus according to claim 1, wherein the second imaging unit captures an image in a direction 90 degrees with respect to an imaging direction of the first imaging unit.   The said moving body behavior discrimination | determination means calculates the rotation amount of a moving body based on the motion of the feature point contained in the back captured image imaged by the said 2nd imaging means. Moving body behavior detection device. Measuring a first feature point included in the first direction of the moving body and a second feature point included in a second direction different from the first direction;
The moving direction of the first feature point included in the first direction of the moving body is compared with the moving direction of the second feature point included in the second direction of the moving body,
If the two moving directions are different, it is determined that the moving body is moving in the lateral direction, and if both moving directions are the same, it is determined that the moving body is moving in the yaw direction. A moving body behavior detection method characterized.

Images