JP5894413B2 - Collision judgment method and collision judgment program - Google Patents

Description

  Embodiments described herein relate generally to a collision determination method and a collision determination program.

  Recently, as a technique for supporting driving of a vehicle such as an automobile, development of a technique for determining the possibility of collision between the vehicle and an object is desired. As this type of technology, for example, there is a technology for determining the possibility of collision between a vehicle and an object based on an image captured by a camera mounted on the vehicle. When using an image captured by a camera mounted on a vehicle, it is possible to use digitized image data as compared with the case of using a radar. Complex judgments such as the approach angle can be made, and the approach of the object can be easily detected.

Japanese Patent Laid-Open No. 2004-56763

  However, when the own vehicle has a rotation component, the own vehicle and the object may approach each other by the rotation component while moving closer to the translation component of the own vehicle, or may move away from the object. For this reason, it is difficult to predict a temporal change in the positional relationship between the vehicle and the object.

  Therefore, when the vehicle is moving with a rotation component, the approach of the object can be detected using the image captured by the camera mounted on the vehicle, but whether or not it collides. It is difficult to determine the time until judgment and collision.

In order to solve the above-described problem, a collision determination method according to an embodiment of the present invention includes a step in which a camera provided in a vehicle images the surroundings of the vehicle, and an object included in an image captured by the camera. A step of detecting a motion vector, a step of determining a translation component and a rotation component of the vehicle based on the motion vector, a movement amount by the translation component, a movement amount by the rotation component at the position of the current object, and a predetermined amount from the current movement and that forms if the amount of movement by the rotation component in the position of an object at a predetermined rate assuming a movement of only the translation component after hours, determining a synthesized composite moving amount of the synthetic movement amount And determining whether or not the vehicle and the object collide within a predetermined time based on the method.

The block diagram which shows an example of the collision determination apparatus for enforcing the collision determination method which concerns on one Embodiment of this invention. The flowchart which shows the procedure at the time of performing a collision determination in consideration of the rotation component of the own vehicle by CPU of the main control part shown in FIG. Explanatory drawing which shows an example of the relationship between a coordinate system and a rotation direction. The subroutine flowchart which shows the first half part of the procedure of the collision determination and the collision time prediction process performed by the collision determination part in step S5 of FIG. The subroutine flowchart which shows the second half part of the procedure of the collision determination and the collision time prediction process performed by the collision determination part in FIG.3 S5. Explanatory drawing which shows an example of the relationship between average rotational movement amount ABR and synthetic | combination movement amount RT of this average rotational movement amount ABR and translation component T. FIG. Explanatory drawing which shows an example of a mode that a side collision is estimated. Explanatory drawing which shows an example of the relationship between the correct track | orbit and an approximate calculation track | orbit in the case of moving to the direction which faces a collision surface in the movement of only the rotation component in an initial position. Explanatory drawing which shows an example of the relationship between the correct track | orbit and an approximate calculation track | orbit in the case of moving in the direction away | separated with respect to a collision surface in the movement of only the rotation component in an initial position.

  Embodiments of a collision determination method and a collision determination program according to the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating an example of a collision determination apparatus 10 for carrying out a collision determination method according to an embodiment of the present invention.

  The collision determination device 10 includes a camera 11, a main control unit 12, a vehicle information acquisition unit 13, a lighting device 16, a horn 17, a speaker 18, and a display device 19.

  The camera 11 is composed of a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 11 captures a video around a vehicle such as a private car and generates image data to be supplied to the main controller 12. .

  For example, when monitoring the rear, the camera 11 is disposed slightly downward from a line parallel to the road surface in the vicinity of the license plate at the rear of the vehicle. A wide-angle lens or a fisheye lens may be attached to the camera 11 so that a wider range of outside-vehicle images can be acquired. When monitoring the side of the vehicle, the camera 11 is disposed near the side mirror. In addition, a wide range of vehicle surrounding images may be captured by using a plurality of cameras 11.

  The main control unit 12 is configured by a microcontroller including a CPU, a RAM, and a ROM, for example. The CPU of the main control unit 12 loads a collision determination program stored in a storage medium such as a ROM and data necessary for the execution of the program into the RAM, and considers the rotation component of the vehicle according to the program. A process for performing the determination is executed. The RAM of the main control unit 12 provides a work area for temporarily storing programs and data executed by the CPU. A storage medium such as a ROM of the main control unit 12 stores a collision determination program and various types of data necessary for executing these programs.

  A storage medium such as a ROM has a configuration including a recording medium readable by a CPU, such as a magnetic or optical recording medium or a semiconductor memory, and a part of programs and data in the storage medium. Or you may comprise so that all may be downloaded via an electronic network via the network connection part which is not shown in figure.

  In this case, the network connection unit implements various information communication protocols according to the network form, and the main control unit 12 and electric devices such as ECUs of other vehicles are connected via the electronic network according to the various protocols. Connect. For this connection, an electrical connection via an electronic network can be applied. Here, an electronic network means an information communication network using telecommunications technology in general, in addition to a wireless / wired LAN (Local Area Network) and the Internet network, a telephone communication line network, an optical fiber communication network, a cable communication network, Includes satellite communications networks.

  The vehicle information acquisition unit 13 acquires at least information on the current acceleration of the host vehicle and outputs the information to the main control unit 12. The vehicle information acquisition part 13 may be comprised, for example with an acceleration sensor, and may have a vehicle information acquisition function generally used in CAN (Controller Area Network). In this embodiment, the vehicle information acquisition part 13 is used in order to utilize the movement parameter of the own vehicle. For this reason, when the main control unit 12 can calculate the movement parameter of the own vehicle based on the image captured by the camera 11, the vehicle information acquisition 13 may not be provided.

  The lighting device 16 is configured by a general headlight, and is controlled by the main control unit 12 to blink (so-called passing), for example, to warn the outside of the host vehicle.

  The horn 17 is controlled by the main control unit 12 and outputs a warning sound to the outside of the host vehicle.

  The speaker 18 is provided in the host vehicle and is controlled by the main control unit 12 to respond to various information such as a beep and information for notifying the driver of the host vehicle that a danger is imminent. Is output.

  The display device 19 is provided at a position visible to the driver, and a display output device such as a general vehicle-mounted display, a car navigation system, or a HUD (head-up display) can be used. In accordance with the control, various information such as a captured image of the camera 11 and information for notifying that the danger is imminent are displayed.

  Next, the configuration of the function realization unit by the CPU of the main control unit 12 will be described.

  As shown in FIG. 1, the CPU of the main control unit 12 uses at least a plane image generation unit 21, a motion vector detection unit 22, a vehicle movement calculation unit 23, a stationary vector detection unit 24, and a collision determination unit 25 according to a collision determination program. And functions as an alarm unit 26. Each unit 21-26 uses a required work area in the RAM as a temporary storage location for data. In addition, you may comprise these function implementation parts by hardware logics, such as a circuit, without using CPU.

  The planar image generation unit 21 generates a planar image based on the image captured by the camera 11. For example, when a fisheye lens is attached to the camera 11, the planar image generation unit 21 normalizes the fisheye image captured by the camera 11 into a normal two-dimensional image, and based on the normalized image in advance. A plane image of a predetermined viewpoint is generated.

  The motion vector detection unit 22 detects a motion vector for each pixel from a plurality of plane images generated by the plane image generation unit 21 using a block matching method, a gradient method, or the like.

The own vehicle movement calculation unit 23 obtains the movement parameters of the own vehicle (the translation component T and the rotation component R 1 of the own vehicle) based on the motion vector detected by the motion vector detection unit 22. When the collision determination device 10 includes the vehicle information acquisition unit 13, the own vehicle movement calculation unit 23 may acquire the movement parameter of the own vehicle from the vehicle information acquisition unit 13. In addition, the own vehicle movement calculation unit 23 calculates a movement vector of the own vehicle from the movement parameters of the own vehicle.

Stationary vector detecting unit 24 compares the angle of the motion vector of the motion vector and the vehicle of the object, the difference in angle becomes under a predetermined angle Sa以, also, the motion vector size the larger object Is detected as a stationary object.

  The collision determination unit 25 assumes the amount of movement due to the translation component, the amount of movement due to the rotation component at the current position of the object, and the rotation component at the position of the object when assuming only movement of the translation component after a predetermined time from the current time. The combined movement amount obtained by combining the movement amount obtained by averaging the movement amounts by the above is obtained. Then, the collision determination unit 25 predicts a time (hereinafter referred to as a collision time) until the own vehicle and the target object collide based on the combined movement amount, and the own vehicle and the target object within a predetermined time. To collide with each other.

  When the collision determination unit 25 determines that the host vehicle and the target object collide within a predetermined time, the alarm unit 26 performs at least one of flashing of the lighting device 16 (passing) and warning sound output via the horn 17. The danger can be notified to a person outside the vehicle. When the collision determination unit 25 determines that the host vehicle and the target object collide within a predetermined time, the alarm unit 26 notifies the driver of the host vehicle via a speaker 18 a beep sound or a collision. A sound that informs the danger can be output. Further, the alarm unit 26 can display information (warning) indicating that there is a risk of collision on the display device 19. The alarm unit 26 may perform at least one of beep sound output, sound output, and warning display as danger notifications to the driver of the host vehicle, and may perform all of them simultaneously. Moreover, both the danger notification for the person outside the vehicle and the danger notification for the driver of the host vehicle may be performed simultaneously, or only one of them may be performed.

  Next, an example of operation | movement of the collision determination apparatus 10 which concerns on this embodiment is demonstrated.

  FIG. 2 is a flowchart illustrating a procedure when the CPU of the main control unit 12 illustrated in FIG. 1 performs the collision determination in consideration of the rotation component of the own vehicle. In FIG. 2, a symbol with a number added to S indicates each step of the flowchart.

  First, in step S <b> 1, the planar image generation unit 21 generates a planar image based on the image captured by the camera 11.

  Next, in step S <b> 2, the motion vector detection unit 22 detects a motion vector for each pixel from the plurality of planar images generated by the planar image generation unit 21 using a block matching method, a gradient method, or the like.

  Next, in step S 3, the own vehicle movement calculation unit 23 obtains the movement parameter of the own vehicle based on the motion vector detected by the motion vector detection unit 22 or the information from the vehicle information acquisition unit 13. A movement vector of the own vehicle is calculated based on the movement parameter of the car.

Here, the calculation method of the own vehicle movement vector based on the own vehicle movement parameter by the own vehicle movement calculation unit 23 will be described.

  FIG. 3 is an explanatory diagram illustrating an example of the relationship between the coordinate system and the rotation direction. In the following description, as shown in FIG. 3, an example of using a coordinate system in which the traveling direction is the z axis, the road surface normal direction is the y axis, and the direction orthogonal to the traveling direction and the road surface normal direction is the x axis. Show about. In FIG. 3, the rotations about the x-axis, y-axis, and z-axis are shown as Rx, Ry, and Rz, respectively.

  As shown in FIG. 3, the image captured by the camera 11 is developed on the camera projection plane 31 ideally represented by y = f by the plane image generation unit 21. Here, f is the focal length of the camera 11.

  The rotation direction and translation direction of the screen when the vehicle is moving are assumed to be R (Rx, Ry, Rz) and T (Tx, Ty, Tz), respectively. At this time, the rotation direction and translation direction of the own vehicle movement are (−Rx, −Ry, −Rz) and (−Tx, −Ty, −Tz).

When the position of the image on the camera projection surface 31 is (x, y, f), the own vehicle movement vector (x, y, cu, cv) = (x coordinate position, y coordinate position, x direction vector) according to a known formula. , Y direction vector) can be obtained from the own vehicle movement parameter as follows.
wu =-(x ・ y / f) ・ Rx + ((f ・ f + x ・ x) / f) ・ Ry-y ・ Rz
wv =-((f ・ f + y ・ y) / f) ・ Rx + (x ・ y / f) ・ Ry + x ・ Rz
tu = (f ・ Tx-x ・ Tz) / z
tv = (f ・ Ty-y ・ Tz) / z
cu = tu + wu
cv = tv + wv

  wu is the x direction of the rotation component of the own vehicle movement vector, wv is the y direction of the rotation component of the own vehicle movement vector, tu is the x direction of the translation component of the own vehicle movement vector, tv is y of the translation component of the own vehicle movement vector Represent each direction. z is the depth in the screen direction. A certain distance (for example, 10 m) may be set to 1, and only the road surface that is necessarily closer than that may be set to a value smaller than 1. The magnitude of the translation component (Tx, Ty, Tz) of the own vehicle movement parameter changes in proportion to the set distance.

  Next, in step S4, the static vector detection unit 24 compares the angle of the motion vector detected by the motion vector detection unit 22 and the movement vector of the host vehicle, and the difference in angle is equal to or less than a predetermined angle difference. Then, an object having a larger motion vector is detected as a stationary object. For example, if the motion vector detected by the motion vector detection unit 22 is (x, y, pu, pv) = (x coordinate position, y coordinate position, x direction vector, y direction vector), (pu−wu, pv) -The difference in angle between (wv) and (tu, tv) is less than or equal to a predetermined angle difference, and the magnitude of (pu-wu, pv-wv) is greater than or equal to a predetermined ratio from the magnitude of (tu, tv) If it is larger, the stationary vector detection unit 24 determines that this motion vector is a stationary object vector.

  Next, in step S5, the collision determination unit 25 predicts a time until the own vehicle and the object collide (hereinafter referred to as a collision time), and the own vehicle and the object collide within a predetermined time. The process which determines whether to do is performed.

  According to the above procedure, the collision determination can be performed in consideration of the rotation component of the own vehicle.

  4 and 5 are subroutine flowcharts showing the procedure of the collision determination / collision time prediction process executed by the collision determination unit 25 in step S5 of FIG. FIG. 4 shows the first half of the procedure, and FIG. 5 shows the second half of the procedure.

  In step S11, the collision determination unit 25 specifies the direction of the collision surface.

More specific description will be given below. The direction of the collision surface is M (Mx, My, Mz). Since the collision surface is basically perpendicular to the road surface, the direction M can be written as follows using the convergence point FOE (FOEx, FOEy).
Mx = FOEx
My = FOEy
Mz = f

Using the direction M of the collision surface, the equation for the collision surface is as follows.
Mx ・ x + My ・ y + Mz ・ z = 0

  Next, in step S12, the collision determination unit 25 specifies the position of the stationary object.

More specific description will be given below. First, z of a stationary object is calculated using a stationary object vector.
pu-wu = tu
pv-wv = tv

  Z is calculated with z as a variable, but two z are derived. The larger one is in pu-wu and pv-wv, or the larger one is in tu and tv when z is 1. Or average.

From the obtained z and the position (x, y, f) of the obtained motion vector, the position A (Ax, Ay, Az) of the stationary object is as follows.
Ax = z ・ x / f
Ay = z ・ y / f
Az = z

  Next, in step S <b> 13, the collision determination unit 25 calculates a collision time t <b> 1 in the movement of only the translational component.

The position AT (ATx, ATy, ATz) of only the translational component of the stationary object is as follows (t is a variable).
ATx = Ax + t ・ Tx
ATy = Ay + t ・ Ty
ATz = Az + t ・ Tz

The intersection between the above position and the collision surface is the value at t when the AT position is substituted into the surface equation. Further, t at that time is a collision time (number of frames) to the surface. If the time at that time is the temporary collision time t1,
Mx ・ (Ax + t1 ・ Tx) + My ・ (Ay + t1 ・ Ty) + Mz ・ (Az + t1 ・ Tz) = 0
→ t1 = (-Mx ・ Ax-My ・ Ay-Mz ・ Az) / (Mx ・ Tx + My ・ Ty + Mz ・ Tz)

  It becomes. Assuming that the warning time (warning time) is ta, if the temporary collision time t1 is longer than the warning time ta, then t1 = ta. When t1 is negative, t1 = 0. In the case of 0 ≦ t1 ≦ ta, it remains t1.

  Next, in step S14, the collision determination unit 25 calculates a collision position in the movement of only the translational component.

A position B (Bx, By, Bz) after t1 with only the translational component of the stationary object is as follows.
Bx = Ax + t1 ・ Tx
By = Ay + t1 ・ Ty
Bz = Az + t1 ・ Tz

  Next, in step S15, the collision determination unit 25 determines the movement amount AR due to the rotation component at the current position A of the object and the movement amount BR due to the rotation component at the position B after t1 with only the translational component of the stationary object. The average rotational movement amount ABR is calculated.

  FIG. 6 is an explanatory diagram showing an example of the relationship between the average rotational movement amount ABR and the combined movement amount RT of the average rotational movement amount ABR and the translation component T. FIG. 6 shows an example in which t1 = ta · 3/2> ta and t1 is larger than ta, and therefore t1 = ta.

First, the rotational movement amount AR (ARx, ARy, ARz) in one frame at the initial position A is as follows.
ARx = Az ・ Ry-Ay ・ Rz
ARy = Ax ・ Rz-Az ・ Rx
ARz = Ay ・ Rx-Ax ・ Ry

Further, the rotational movement amount BR (BRx, BRy, BRz) in one frame at the position B after t1 with only the translational component of the stationary object 42 is as follows.
BRx = Bz ・ Ry-By ・ Rz
BRy = Bx ・ Rz-Bz ・ Rx
BRz = By ・ Rx-Bx ・ Ry

  As described above, as the object position becomes closer, the movement amount and direction of the rotation component change. In the present embodiment, the movement amount AR in the rotation component at the position A (t = 0) where the object is detected and the movement in the rotation component at the position B when assuming the movement of only the translation component after the time t1. The average ABR with the amount BR is treated as the amount of movement in the rotational component in one frame. Since the time until the collision is short, there is almost no error even if the averaged rotational movement amount ABR is used.

  In this embodiment, an example in which the average of AR and BR is ABR will be described. However, the ABR is not limited to the average of AR and BR, and may be any one instead of the average depending on the situation. In that case, it may be determined by actual measurement. For example, a combination of AR and BR weighted at a predetermined ratio may be used as ABR, or a combination of a predetermined ratio and then processed under a predetermined condition may be used as ABR. As a case where an ABR is processed under this predetermined condition, for example, as shown below, the processing in the case of satisfying the condition that only the rotational component is in a direction away from the collision surface (to the collision surface) (ABR2) obtained by processing the average value ABR2 with the rotation component with the component 0 set to 0 is used as the ABR.

The average ABR (ABRx, ABRy, ABRz) of the movement amount in the rotation component is as follows.
ABRx = (ARx + BRx) / 2
ABRy = (ARy + BRy) / 2
ABRz = (ARz + BRz) / 2

  Note that only the rotational component may be in a direction away from the collision surface. When the direction is away, the parallel direction is usually closer to the collision surface, and the rotation component is reduced accordingly. For this reason, when it becomes a reverse side with respect to a collision surface, the average with the rotation component which set the component with respect to a collision surface to 0 is used. By doing so, it becomes a near collision position and collision time. Since the collision time may be slightly shortened, the time ta to be warned may be slightly shortened to check whether the collision occurs in time.

A vector N (Nx, Ny, Nz) perpendicular to the collision surface of the average movement amount ABR of the rotation component is as follows.
Nx = ABRx + Mx ・ t
Ny = ABRy + My ・ t
Nz = ABRz + Mz ・ t

Since the above vector only needs to intersect the plane passing through (0, 0, 0) in the direction of the collision surface, t at the time of intersection is derived from the following equation.
Mx ・ Nx + My ・ Ny + Mz ・ Nz = 0

Assuming that the derived t is t2, the average value ABR2 (ABR2x, ABR2y, ABR2z) with the rotation component with the component to the collision surface being zero is as follows.
ABR2x = ABRx + Mx ・ t2 / 2
ABR2y = ABRy + My ・ t2 / 2
ABR2z = ABRz + Mz ・ t2 / 2

  Accordingly, when the rotational component alone moves in a direction away from the collision surface, this ABR2 may be calculated as ABR. Moreover, although the average of the rotation component which set the component to ABR and its collision surface to 0 was made into an average, it may be made from either instead of an average depending on the situation. In that case, it may be determined by actual measurement. The ratio may be changed according to the parallel position or the vertical position of the collision surface.

  Therefore, in step S16, the collision determination unit 25 determines whether or not the rotation component alone is a direction away from the collision surface. If the direction is closer, the process proceeds to step S21 in FIG. On the other hand, if the direction is away, the average ABR2 with the rotation component with the component to the collision surface being zero is calculated (step S17), and the process proceeds to step S21 in FIG.

  Subsequently, in step S21 in FIG. 5, the collision determination unit 25 calculates a combined movement amount RT obtained by combining the translation movement amount and the average movement amount ABR of the rotation component.

When the combined movement amount of the movement amount of the translation component and the average movement amount of the rotation component in one frame is represented by the translation component, this combined movement amount RT (RTx, RTy, RTz) is as follows.
RTx = Tx + ABRx
RTy = Ty + ABRy
RTz = Tz + ABRz

  Next, in step S22, the collision determination unit 25 calculates a collision time t2 with the combined movement amount RT.

The position ART (ARTx, ARTy, ARTz) in consideration of the rotation component of the object is as follows (t is a variable).
ARTx = Ax + t ・ RTx
ARTy = Ay + t ・ RTy
ARTz = Az + t ・ RTz

The intersection between the position ART and the collision surface is a value at t when the position of the ART is substituted into the surface equation. Further, t at that time is a collision time (number of frames) to the surface. If the time at that time is t2,
Mx ・ (Ax + t2 ・ RTx) + My ・ (Ay + t2 ・ RTy) + Mz ・ (Az + t2 ・ RTz) = 0
→ t2 = (-Mx ・ Ax-My ・ Ay-Mz ・ Az) / (Mx ・ RTx + My ・ RTy + Mz ・ RTz)

  It becomes. At this time, if the polarity of the time to collision (number of frames) t2 is positive and less than the time ta to be warned, it is determined as a collision candidate.

  Next, in step S23, the collision determination unit 25 calculates the collision position C at the combined movement amount RT.

The position C (Cx, Cy, Cz) of the stationary object 42 at t = t2 when moving with the combined movement amount RT is as follows.
Cx = Ax + t2 ・ RTx
Cy = Ay + t2 ・ RTy
Cz = Az + t2 ・ RTz

Next, in step S24, the collision determination unit 25 determines whether the stationary object 42 collides with the front of the host vehicle. Specifically, for example, the collision determination unit 25 determines that the position C at t = t2 is within the specified range for the stationary object 42 that is a collision candidate because the time t2 calculated in step S22 is less than ta. Determine whether or not. Since the collision time may be slightly shortened, the time ta to be warned may be slightly shortened to check whether the collision occurs in time.

For example, assuming that the height from the camera is H and the collision range on the left and right of the collision surface in front of the host vehicle is ± W, the collision occurs when the following conditions are satisfied.
-W <= Cx <= W
Cy> = -H

  The collision range may be changed depending on the position of the convergence point FOE or the deviation in the upward direction. In that case, Cx, Cy, and Cz may be rotated and calculated around the origin. Further, the collision range and shape may be changed.

If it is determined in step S24 that a frontal collision has occurred, the collision determination unit 25 gives information to that effect to the alarm unit 26 in step S25. Then, the alarm unit 26 displays, for example, information indicating that there is a risk of a frontal collision with the stationary object 42 on the display device 19. Further, information on the time t2 until the collision and the collision position may be displayed.

On the other hand, if it is determined in step S24 that no frontal collision occurs, in step S26, the collision determination unit 25 determines whether or not the stationary object 42 collides with the side surface of the host vehicle.

  FIG. 7 is an explanatory diagram illustrating an example of how a side collision is predicted.

Since the front direction basically moves forward, there is no collision unless the translational component is directed to the stationary object 42 . However, even if the side surface is a translational component and does not go to the stationary object 42 , it may collide with a rotational component. Therefore, as shown in FIG. 7, the calculation in the front direction (calculation up to the front collision surface, see the left side of FIG. 7) and the calculation in the side (calculation after the front collision surface, see the right side of FIG. 7) are separated. And calculate. FIG. 7 shows an example in which the front collision surface is set to z = 0.

  First, the calculation up to the front collision surface (calculation in the front direction) uses the same method as the method of calculating the front collision described so far, and uses the collision position C0 (C0x, C0y, C0z) and the collision time t0 are measured. At this time, if the front collision surface is too far away from the camera, z = 0 may be used as the collision surface.

The time tb that you want to warn on the side is the collision time t0 on the front,
tb = ta-t0

  It becomes. Similar to the calculation formula at the front, t1 at the side is calculated. When t1 is negative, it is set to tb from the time to warn and the time in front. After that, the same calculation is performed.

For example, the equation of the left collision surface is as follows when the axis is rotated by −90 ° y-axis from the front surface direction, assuming that it is inclined in the x direction with respect to the Mup ratio.
-Mz ・ (x-Wl) + Mup ・ f ・ y + Mx ・ z = 0

  Here, Wl represents the position in the x direction from the camera (because it is on the left, it becomes negative).

Similarly, the expression of the collision surface on the right side is as follows when rotated by 90 ° y-axis from the direction of the front surface assuming that it is inclined in the x direction with respect to the Mup ratio.
Mz ・ (x-Wr)-Mup ・ f ・ y-Mx ・ z = 0

  Here, Wr represents the position in the x direction from the camera (it is positive because it is on the right).

As described above, the side surface is designated, and it is further confirmed whether the collision position is within the range of the own vehicle. For example, assuming that the collision position on the side surface is C (Cx, Cy, Cz), the height from the camera is H, and the distance from the camera to the front side D is the collision range, the side surface will satisfy the following conditions: collide.
Cy> = -H
-D <= Cz <= 0

  The collision range may be changed depending on the position of the convergence point FOE or the deviation in the upward direction. In this case, Cx, Cy, and Cz may be rotated and calculated around the origin. Further, the collision range and shape may be changed.

  The collision determination apparatus 10 according to the present embodiment can perform the collision determination in consideration of the rotation component of the own vehicle. For this reason, compared with the case of ignoring the rotation component, it is possible to obtain a very accurate collision determination result and to predict the collision time.

  If the collision determination is an area instead of a point, select n points at both ends of the area (minimum 2 points, for example, 2 points at both ends on the horizontal line, or 4 points at the upper left, upper right, lower left, lower right, etc. ), Collision judgment is performed for each. If it is determined that any of these collisions, the one with the shortest time until the collision is set as the collision determination result of this region.

If it is determined that none of the collisions occur, it is assumed that no collision occurs if any of the collision times t2 on the front surface is greater than the time ta at which it is desired to warn of the collision. On the other hand, if at least one collision time t2 is shorter than the time ta at which a collision is to be warned, the smaller position E (Ex, Ey, Ez) at ta and t2 and position F (Fx, Fy, Fz) at ta Calculate each one. If E and F are created for each point on the area (if E and F are the same, one is deleted), the minimum is n points, which is the number of points on the area, and the maximum is on the area 2.n points, which is twice the number of points, are obtained. Two points are selected from the obtained points and set as a line segment, and it is determined whether the line segment intersects with the area of the vehicle that is the collision area. It tries with the combination of all the two points, and when any line segment crosses the area of the own vehicle, it determines with it colliding. The collision time may be the shortest of the collision times t2 in front of the two points that create the line segment from among the line segments intersecting all the areas of the host vehicle.
(Example)

  The result of performing a simulation to verify the correctness of the collision prediction calculation according to the present embodiment is shown below.

  FIG. 8 is an explanatory diagram showing an example of the relationship between the correct trajectory and the approximate calculation trajectory when moving in the direction toward the collision surface when only the rotational component is moved at the initial position. FIG. 9 is an explanatory diagram showing an example of the relationship between the correct trajectory and the approximate calculation trajectory when moving in the direction away from the collision plane when only the rotational component is moved at the initial position. 8 and 9 are examples of trajectory comparison results from the viewpoint directly above, and the markers in the drawings represent positions for each frame.

  First, in order to simplify the calculation, the road surface is y = 0, the collision surface is z = 0, the translation component of the vehicle is only Tz, and the rotation component is only Ry. In addition, the depth z of the object is fixed to 0.3, and the time ta to warn until the collision is 15 frames.

  The longer the time until the collision, the more error is included in the calculation results. For this reason, the translation component Tz is set to −0.02 (/ frame) so that a calculation error appears.

  Further, in order to see the effect of the portion in consideration of the rotation component in the collision prediction calculation according to the present embodiment, the state having the most rudder angle is set to Tz so that the rotation component is near the maximum allowed in the behavior of the vehicle. 3 times. At this time, the rotation component is Ry = −0.06. In addition, the movement of the rotation component which does not accompany a translational component does not occur on the behavior of the vehicle.

Assuming that the position after n frames is X (n) and Z (n), the calculation formula of the correct trajectory is as follows.
X (n) = X (n-1)-Z (n-1) ・ Ry
Z (n) = Z (n-1) + Tz + X (n-1) ・ Ry

On the other hand, the calculation formula for the approximate calculation trajectory is as follows.
X (n) = X (n-1) + (-z ・ Ry + 0) / 2
Z (n) = Z (n-1) + Tz + (x ・ Ry + x ・ Ry) / α
Based on the above conditions, verification is performed by shifting x.

  From the results of the trajectory comparison, it was found that the variable α should be about 2.0 in the direction in which the rotation component is directed. FIG. 8 shows an example in which α = 2. On the other hand, it was found from the result of the trajectory comparison that the variable α should be about 4.0 in the direction in which the rotation component is directed. FIG. 9 shows an example in which α = 4.

  As is apparent from FIGS. 8 and 9, it is understood that the collision position and the collision time can be made substantially the same between the trajectory in the approximate calculation and the correct trajectory by appropriately selecting α.

  In addition, although some embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  Further, in the embodiment of the present invention, each step of the flowchart shows an example of processing that is performed in time series in the order described. The process to be executed is also included.

DESCRIPTION OF SYMBOLS 10 Collision determination apparatus 11 Camera 12 Main control part 21 Plane image generation part 22 Motion vector detection part 23 Own vehicle movement calculation part 24 Still vector detection part 25 Collision determination part 26 Alarm part

Claims (7)

A camera provided in the vehicle images the surroundings of the vehicle;
Detecting a motion vector of an object included in an image captured by the camera;
Obtaining a translation component and a rotation component of the vehicle based on the motion vector ;
The amount of movement due to the translation component, the amount of movement due to the rotation component at the current position of the object, and the movement due to the rotation component at the position of the object assuming a movement of only the translation component after a predetermined time from the present time. a step of determining movement and that forms if the amount at a predetermined ratio, the synthesized synthesis amount of movement,
Determining whether the vehicle and the object collide within the predetermined time based on the combined movement amount; and
A collision determination method. Obtaining a temporary collision time until the vehicle and the object collide when assuming the movement of only the translational component;
When the temporary collision time is smaller than a predetermined warning time, the temporary collision time is set as the predetermined time, and when the temporary collision time is equal to or longer than the warning time, the warning time is set to the predetermined time. Step to set as
The collision determination method according to claim 1, further comprising: The step of determining whether or not the vehicle and the object collide based on the combined movement amount,
Determining whether the front of the vehicle and the object collide based on the combined movement amount; and
When it is determined that the front surface of the vehicle and the object do not collide, the step of determining whether or not the side surface of the vehicle and the object collide,
Having
The collision determination method according to claim 1 or 2. Predicting a collision time between the vehicle and the object based on the combined movement amount;
The collision determination method according to any one of claims 1 to 3, further comprising: The step of determining whether or not the vehicle and the object collide within the predetermined time based on the combined movement amount,
A step of selecting a plurality of end points of the object and determining whether or not each of these end points collides with the vehicle within the predetermined time based on the combined movement amount;
The collision determination method according to any one of claims 1 to 4. When it is determined that the vehicle and the object collide within the predetermined time, the driver of the vehicle visually recognizes the sound output and buzzer output through the speaker of the vehicle and the driver of the vehicle. Informing the danger by at least one of warning indications for a display device provided at a possible position;
The collision determination method according to any one of claims 1 to 5, further comprising: On the computer,
Obtaining an image around the vehicle imaged by a camera provided in the vehicle;
Detecting a motion vector of an object included in an image captured by the camera;
Obtaining a translation component and a rotation component of the vehicle based on the motion vector ;
The amount of movement due to the translation component, the amount of movement due to the rotation component at the current position of the object, and the movement due to the rotation component at the position of the object assuming a movement of only the translation component after a predetermined time from the present time. a step of determining movement and that forms if the amount at a predetermined ratio, the synthesized synthesis amount of movement,
Determining whether the vehicle and the object collide within the predetermined time based on the combined movement amount; and
Collision determination program for executing

Images