JP5731801B2 - Image processing apparatus for vehicle and image processing method for vehicle - Google Patents

Description

  Embodiments described herein relate generally to a vehicle image processing apparatus and a vehicle image processing method.

  Recently, as a technique for supporting driving of a vehicle such as an automobile, development of a technique for determining the possibility of collision between the vehicle and an object is desired. As this type of technology, for example, there is a technology for determining the possibility of collision between a vehicle and an object based on an image captured by a camera mounted on the vehicle. When using an image captured by a camera mounted on a vehicle, it is possible to use digitized image data as compared with the case of using a radar, so that it is possible to make a complicated determination such as an approach angle of an object. .

JP 11-353565 A

  However, the conventional technology assumes that the vehicle is moving straight without vibration, and does not consider the three types of rotational components (yawing, rolling, and pitching) of the vehicle. For this reason, it is difficult to apply the conventional technique when the vehicle turns or when the vehicle vibrates.

In order to solve the above-described problem, a vehicle image processing device according to an embodiment of the present invention includes a camera, a motion vector calculation unit, a translational component calculation unit with a scale factor, a z position acquisition unit, and a collision determination. And a collision risk determination unit. The camera is provided in the vehicle and images the periphery of the vehicle. The motion vector calculation unit obtains a motion vector between a plurality of images captured by the camera. The translation component with a scale factor calculation unit calculates a translation component with a scale factor in each of the three axis directions of the camera coordinate system based on the motion vector corresponding to the object included in the image and the rotation component of the vehicle. The z position acquisition unit acquires the optical axis direction coordinate value of the object in the camera coordinate system. The collision determination unit is an expression that expresses the distance from the center of the camera to the object as a function of time using a three-dimensional coordinate value including the optical axis direction coordinate value of the object in the camera coordinate system and a translation component with a scale factor. Based on this, it is determined whether or not the closest distance from the center of the camera to the object is equal to or less than a predetermined distance. The collision risk determination unit determines that the vehicle and the object collide when the closest distance is equal to or less than a predetermined distance.

1 is an overall configuration diagram showing an example of a vehicle image processing apparatus according to an embodiment of the present invention. Explanatory drawing which shows the relationship between the camera projection surface in a camera coordinate system, and a target object. (A) is ^an explanatory diagram showing an example of the relationship between ^r 2 -r0 ² and the time t when the minimum value of r 2 -r0 ² is negative, (b) the minimum value positive in ^r 2 -r0 ² explanatory view showing an example of the relationship between ^r 2 -r0 ² and the time t in the case. 7 is a flowchart showing a procedure when the CPU of the collision prediction ECU determines the possibility of collision with an object in consideration of the influence of vehicle curvature and vehicle vibration based on an image acquired by a camera.

  Embodiments of a vehicle image processing apparatus and a vehicle image processing method according to the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is an overall configuration diagram showing an example of a vehicle image processing apparatus 10 according to an embodiment of the present invention.

  The vehicle image processing device 10 includes a camera 11, a collision prediction ECU (Electronic Control Unit) 12, a storage unit 13, a display unit 14, and a speaker 15.

  The camera 11 is constituted by a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

  For example, when monitoring the rear, the camera 11 is disposed slightly downward from a line parallel to the road surface in the vicinity of the license plate at the rear of the vehicle. A wide-angle lens or a fisheye lens may be attached to the camera 11 so that a wider range of outside-vehicle images can be acquired. When monitoring the side of the vehicle, the camera 11 is disposed near the side mirror. Of course, a wide range of outside-vehicle surrounding images may be captured by using a plurality of cameras 11.

  The collision prediction ECU 12 is configured by a storage medium such as a CPU, a RAM, and a ROM. The CPU of the collision prediction ECU 12 loads a collision prediction program stored in a storage medium such as a ROM and data necessary for executing the program into the RAM, and based on the image acquired by the camera 11 according to the program. The possibility of collision with the object is determined in consideration of the influence of the vehicle turning and the vehicle vibration.

  The RAM of the collision prediction ECU 12 provides a work area for temporarily storing programs and data executed by the CPU. The storage medium such as the ROM of the collision prediction ECU 12 stores a collision prediction program and various data necessary for executing these programs.

  A storage medium such as a ROM has a configuration including a recording medium readable by a CPU, such as a magnetic or optical recording medium or a semiconductor memory, and a part of programs and data in the storage medium. Or you may comprise so that all may be downloaded via an electronic network via the network connection part which is not shown in figure.

  In this case, the network connection unit implements various information communication protocols according to the form of the network, and the collision prediction ECU 12 and electric devices such as ECUs of other vehicles are connected via the electronic network according to the various protocols. Connecting. For this connection, an electrical connection via an electronic network can be applied. Here, an electronic network means an information communication network using telecommunications technology in general, in addition to a wireless / wired LAN (Local Area Network) and the Internet network, a telephone communication line network, an optical fiber communication network, a cable communication network, Includes satellite communications networks.

  The storage unit 13 is a non-volatile memory in which data can be read and written by the collision prediction ECU 12, and information on FOE (x0, y0) of a vanishing point (FOE), a distance margin r0, and a first time are stored in advance. A threshold value s1, a second time threshold value s2, and an approach threshold value d0 are stored. These pieces of information may be updated via an electronic network or a portable storage medium such as an optical disk.

  The vanishing point information FOE (x0, y0) is used when the collision prediction ECU 12 calculates the rotational component of the vehicle.

  The distance margin r0, the first time threshold value s1, the second time threshold value s2, and the approach threshold value d0 are used by the collision prediction ECU 12 to determine the risk of collision between the vehicle and the object.

  The display unit 14 is provided at a position where the driver can visually recognize, and a display output device such as a general vehicle-mounted display, a car navigation system, or a HUD (head-up display) can be used. Various information such as collision warning information is displayed.

  The speaker 15 outputs sound corresponding to various information such as collision warning information in accordance with the control of the collision prediction ECU 12.

  As shown in FIG. 1, the CPU of the collision prediction ECU 12 executes at least a motion vector calculation unit 21, a rotation component calculation unit 22, a translation component with scale factor 23, an Euclidean distance estimation unit 24, and a collision determination unit 25 according to a collision program. , Z position acquisition unit 26, collision time estimation unit 27, collision risk determination unit 28, and warning presentation unit 29. Each of the units 21 to 29 uses a required work area of the RAM as a temporary storage location for data. In addition, you may comprise these function implementation parts by hardware logics, such as a circuit, without using CPU.

  Here, the camera coordinate system used in this embodiment will be briefly described.

  FIG. 2 is an explanatory diagram showing the relationship between the camera projection surface 31 and the object 32 in the camera coordinate system.

  Since the vehicular image processing apparatus according to the present embodiment uses only the camera coordinate system and does not use the world coordinate system, processing such as mutual conversion of the coordinate system is unnecessary.

  The camera coordinate system (x, y, z) is a left-handed coordinate system with the center of the camera 11 as the origin. The z axis of the camera coordinate system is set to coincide with the optical axis of the camera 11.

  The camera projection plane 31 is a plane with z = f in the camera coordinate system. f is the focal length of the camera 11. The camera 11 captures an image around the vehicle projected on the camera projection plane 31.

When the three-dimensional coordinate of the camera coordinate system of the point P with the object 32 is P (X, Y, Z), the coordinate of the point p projected onto the camera projection surface 31 of the point P is p (x, y, f ) If the z coordinate (optical axis direction coordinate) of the point P is defined as z (Z = z), X = (z / f) · x, Y = (z / f) · y, and Z = z. At this time, the distance r can be written as r ² = (z ² / f ² ) (x ² + y ² + f ² ). Note that x and y are xy coordinate values on the projection plane 31 and z is a z coordinate value of the object 32. Since the xy coordinate values on the projection plane 31 correspond to the pixels of the camera 11, they can be easily obtained from the image.

  The motion vector calculation unit 21 calculates a motion vector (u, v) for each pixel from a plurality of images captured by the camera 111 using a block matching method, a gradient method, or the like.

The rotation component calculation unit 22 calculates rotation components R _x , R _y , R _z based on the motion vector (u, v) and the vanishing point information FOE (x0, y0). More specific description will be given below.

It is known that the motion vector (u, v) on the camera projection plane 31 can be written as follows.
U = xy (1 / f) R _x − (x ² + f ² ) (1 / f) R _y + yR _z −f (T _x / z) + x (T _z / z) (1)
V = (y ² + f ² ) (1 / f) R _x −xy (1 / f) R _y −xR _z −f (T _y / z) + y (T _z / z) (2)

Here, x and y are xy coordinate values on the projection surface 31, and z is a z coordinate value of the object 32. Also, R _x , R _y , R _z , T _x , T _y , T _z are motion parameters, R _x , R _y , R _z represent the rotational components of the vehicle, and T _x , T _y , T _z are A relative translation component between the own vehicle and the object 32 is shown.

The rotation component _{Rx of the} vehicle is a rotation component about the x-axis, and represents the pitching component of the own vehicle. R _y is a rotational component about the y-axis and represents the yawing component of the host vehicle. Yawing occurs mainly at the steering angle (when the vehicle is turning). R _z is a rotation component about the z axis and represents a rolling component of the vehicle.

Equations (1) and (2) can be modified as follows.
u−xy (1 / f) R _x + (x ² + f ² ) (1 / f) R _y −yR _z = −f (T _x / z) + x (T _z / z) (3)
v− (y ² + f ² ) (1 / f) R _x + xy (1 / f) R _y + xR _z = −f (T _y / z) + y (T _z / z) (4)

  In Equation (3) and Equation (4), the left side is obtained by correcting the motion vector (u, v) with a rotation component.

When the object 32 is a stationary object, it is known that the ratio of the expression (3) to the expression (4) is equal to the inclination from the pixel (x, y) to the FOE (x0, y0). (Takashi Sakimoto et al., Proceedings of Fire Country Information Symposium 2003, “Research on Moving Vehicle Detection and Tracking Using Extended Focus” (2003)).
For this reason, the following formula (5) is derived.
[u−xy (1 / f) R _x + (x ² + f ² ) (1 / f) R _y −yR _z ] / [v− (y ² + f ² ) (1 / f) R _x + xy (1 / f) R _y + xR _z ]
= [x-x0] / [y-y0] (5)

Accordingly, if R _x , R _y , and R _z are considered as three unknowns, R _x , R _y , and R _z can be calculated if motion vectors of at least three pixels corresponding to the stationary object can be obtained.

Therefore, the rotation component calculation unit 22 acquires the vanishing point information FOE (x0, y0) from the storage unit 13, and formula (5) regarding the motion vectors (u, v) of three or more pixels corresponding to the stationary object. Based on this, rotation components R _x , R _y , R _z are calculated. That is, the rotation component calculation unit 22 can calculate the rotation component from the image acquired by the camera 11.

  The rotation component can also be obtained from vehicle information such as vehicle speed data and steering angle data, a gyro sensor, and the like.

The translation component with scale factor calculation unit 23 calculates the scale of the object 32 based on the motion vector (u, v) corresponding to the object 32 included in the image and the vehicle rotation components R _x , R _y , R _z. A relative translation component with a factor (hereinafter referred to as a translation component with a scale factor) T _x / z, T _y / z, and T _z / z is calculated.

The translation components T _x / z, T _y / z, and T _z / z with scale factors are obtained by dividing the relative translation components T _x , T _y , T _z of the object 32 by the z coordinate value of the object 32. . For this reason, if the relative translation components T _x , T _y , and T _z are the same, the farther away (the larger the z coordinate value), the smaller the translation component with a scale factor. Regardless of whether the point P of the object 32 is stationary or moving, the point P moves on the image if it moves with the translation components T _x / z, T _y / z, and T _z / z with scale factors. Observed. More specific description will be given below.

The rotation components R _x , R _y , and R _z are parameters that are common at arbitrary positions in the screen. For this reason, the following equation is established for an arbitrary pixel based on the equations (3) and (4).
−f (T _x / z) + x (T _z / z) = a (6)
−f (T _y / z) + y (T _z / z) = b (7)

Here, a is an observed value that represents the left side of Equation (3), and b is an observed value that represents the left side of Equation (4). If observation values a and b for a plurality of pixels (x, y) of the object 32 are obtained, the unknowns T _x / z, T _y / z, and T _z / z are obtained from the equations (6) and (7). Can be obtained.

Therefore, the translation component with scale factor calculation unit 23 includes vehicle rotation components R _x , R _y , R _z , motion vectors (u, v) for a plurality of pixels corresponding to the object 32 included in the image, Based on the above, by calculating a and b in the equations (6) and (7), the translation components T _x / z, T _y / z, and T _z / z with the scale factor of the object 32 are calculated.

In other words, the translation component with scale factor calculation unit 23 obtains the translation components with scale factors T _x / z, T _y / z, and T _z / z of the object 32 without using the z coordinate value of the object 32. be able to.

The Euclidean distance estimation unit 24 uses the three-dimensional coordinate values (X, Y, Z) of the object 32 in the camera coordinate system and the translation components T _x / z, T _y / z, T _z / z with scale factors. An expression expressing the distance r from the center of the camera 11 to the object 32 as a function of time t is established. More specific description will be given below.

Assuming that the vehicle and the object 32 move in the same way from the current time to time t, the square of the distance r between the origin and the point P after time t seconds is expressed as a Euclidean distance as a quadratic function at time t. Can do.
r ² = (X−T _x t) ² + (Y−T _y t) ² + (Z−T _z t) ² (8)

As described above, if the z coordinate (optical axis direction coordinate) of the point P is defined as z (Z = z), X = (z / f) · x, Y = (z / f) · y, Z = z I can write. Substituting this into equation (8) yields:
r ² = (xz / f−T _x t) ² + (yz / f−T _y t) ² + (z−T _z t) ² (9)

Equation (9) can be modified as follows.
r ² / z ² = At ² −2Bt + C (10)
Where A = (T _x / z) ² + (T _y / z) ² + (T _z / z) ² ,
B = (1 / f) (x T _x / z + yT _y / z + fT _z / z),
C = (1 / f ² ) (x ² + y ² + f ² ) respectively.

Since A in Expression (10) is positive, r ² is a quadratic function of t convex downward. From equation (10), each time t the minima ^r 2 min and minimum value ^r 2 min of ^{r 2} can be expressed as follows.
r ² min = (z / f) ² ((x ² + y ² + f ² ) − (xT _x / z + yT _y / z + fT _z / z) ² / (T _x ² / z ² + T _y ² / z ² + T _z ² / z ² )} (11)
t = (1 / f) (xT _x / z + yT _y / z + fT _z / z) / (T _x ² / z ² + T _y ² / z ² + T _z ² / z ² ) (12)

As is clear from equation (11), the minimum value of r ² is calculated if the translation components T _x / z, T _y / z, T _z / z with scale factors and the z coordinate value of the object 32 are obtained. be able to. Further, as is clear from the equation (12), the time t at which r ² becomes the minimum value can be calculated if the translation components T _x / z, T _y / z, and T _z / z with scale factors are obtained. it can.

that r ² is zero means that the point P reaches the origin (camera center). Therefore, the minimum value of r ² can be used as an index for determining the risk of collision between the vehicle and the object 32.

In addition, it is considered effective to determine whether or not the vehicle and the object are within a predetermined distance (hereinafter referred to as a distance margin r0) in determining the collision risk. Distance the vehicle and the object is given (hereinafter, the distance margin r0 hereinafter) to determine whether the approaches within, first make the following equation obtained by subtracting the distance margin r0 ² from both sides of the equation (9).
r ² −r0 ² = (xz / f−T _x t) ² + (yz / f−T _y t) ² + (z−T _z t) ² −r0 ² (13)

From equation ^(13), the minimum value ^r 2 ^-r0 2 min of r 2 -r0 ² is expressed as follows. r ² −r0 ² min = (z / f) ² ((x ² + y ² + f ² ) − (xT _x / z + yT _y / z + fT _z / z) ² / (T _x ² / z ² + T _y ² / z ² + T _z ² / z ² )} −r0 ² (14)

Note that the time t for which r ² −r 0 ² takes the minimum value is expressed by Expression (10).

In the following description, the Euclidean distance estimation unit 24 acquires the distance margin r0 from the storage unit 13, and the three-dimensional coordinate value (X, Y, Z) of the object 32 in the camera coordinate system and the translation component T _x with a scale factor. Formula (14) based on Formula (8) using / z, T _y / z, and T _z / z is established, and an example of giving this formula to the collision determination unit 25 will be described.

The collision determination unit 25 uses the equation (14) based on the equation (8) to determine whether or not the closest distance (minimum value of r) from the center of the camera 11 to the object 32 is equal to or less than the distance margin r0. That is, it is determined whether or not the minimum value of r ² −r 0 ² is equal to or less than zero in Expression (14). The collision determination unit 25 may further determine that the vehicle and the object 32 collide when the closest distance from the center of the camera 11 to the object 32 is equal to or less than the distance margin r0. In addition, when the minimum value of r ² −r 0 ² is positive, the collision determination unit 25 determines whether or not this minimum value is smaller than the approach threshold d1 stored in the storage unit 13 (provided that d1> 0). Determine.

In order to solve Expression (14), the z-coordinate value of the object 32 is necessary in addition to the translation components T _x / z, T _y / z, and T _z / z with scale factors. The z coordinate value of the object 32 is given from the z position acquisition unit 26 to the collision determination unit 25.

The z position acquisition unit 26 acquires the z coordinate value of the object 32 and gives it to the collision determination unit 25. For example, when the object 32 is a stationary object, the relative translation components T _x , T _y , T _z are separately obtained, and the relative translation components T _x , T _y , T _z and the translation component T _x / with a scale factor are obtained. z, T _y / z, and T _z / z. The relative translation components T _x , T _y , and T _z can be obtained from an image, or can be obtained from vehicle information such as vehicle speed data and steering angle data of the vehicle, an acceleration sensor, and the like. Note that the z-coordinate value of the object 32 can also be obtained by a distance measurement method using a radar, a radar laser, a light cutting method, a stereo camera, or the like when a member other than one camera 11 is used.

3 (a) ^is an explanatory diagram showing an example of the relationship between ^r 2 -r0 ² and the time t when the minimum value of r 2 -r0 ² is negative, (b) it is ^r 2 -r0 ² tiny value is an explanatory diagram showing an example of the relationship between r ² -r0 ² and the time t in the case of a positive.

The collision time estimation unit 27 calculates the time when the distance r from the center of the camera 11 to the object 32 becomes the distance margin r0 using the equation (14) based on the equation (8). Incidentally, the distance r from the center of the camera 11 to the object 32 is the distance margin r0 ^is only the minimum value of r 2 -r0 ² is less than or equal to zero (see Figure 3 (a)).

  Specifically, the collision time estimation unit 27 obtains the real roots t1 and t2 of the equation with the left side of the equation (14) being zero, and sets the smaller root t1 as a time TTC (Time To Collision) until the collision.

  The collision risk determination unit 28 determines the vehicle and the object 32 according to the closest distance from the center of the camera 11 to the object 32 and the time TTC when the distance r from the center of the camera 11 to the object 32 becomes r0. The collision risk is classified into multiple stages. In the following description, an example in which the collision risk is classified into three stages will be described.

  More specifically, when the closest distance is r0 or less and the TTC is smaller than the first time threshold value s1 stored in the storage unit 13, the collision risk level determination unit 28 sets the risk level to “high”. judge. If the closest distance is r0 or less and TTC is equal to or greater than the first time threshold s1 and less than the second time threshold s2 stored in the storage unit 13, the risk is determined as “medium”, and s2 If it is above, the degree of risk is determined as “low”.

Further, the collision risk judging section 28, and larger than the closest distance r0, if r ² -r0 ² of minimum values has been less than approaching threshold d1 stored in the storage unit 13 the risk "medium" determination If it is greater than or equal to d1, the risk is determined to be “low”.

  The warning presenting unit 29 warns the driver via the display unit 14 and the speaker 15 according to the collision risk classified by the collision risk determination unit 28. For example, when the degree of danger is “high”, the driver may be notified that there is a danger of a collision with a video through the display unit 14 and a voice through the speaker 15 so that the driver is recognized to be urgent. If the degree of danger is “medium”, it is preferable to notify that there is a danger of collision with an image through the display unit 14 and a sound through the speaker 15 to the extent that the driver is alerted. When the degree of risk is “low”, the driver need not be warned.

  Next, an example of the operation of the vehicle image processing apparatus and the vehicle image processing method according to the present embodiment will be described.

  FIG. 4 shows a procedure when the CPU of the collision prediction ECU 12 determines the possibility of collision with an object in consideration of the influence of vehicle curvature and vehicle vibration based on the image acquired by the camera 11. It is a flowchart. In FIG. 4, reference numerals with numbers added to S indicate steps in the flowchart.

  First, in step ST <b> 1, the motion vector calculation unit 21 acquires image data of a captured image from the camera 11.

  Next, in step ST2, the motion vector calculation unit obtains a motion vector (u, v) between a plurality of images captured by the camera 11 for each pixel based on the acquired image data.

Next, in step ST3, the rotation component calculation unit 22 acquires vanishing point information FOE (x0, y0) from the storage unit 13, and formulas for three or more motion vectors (u, v) corresponding to the stationary object. Based on (5), rotation components R _x , R _y , R _z are calculated.

Next, in step ST4, the translation component with scale factor calculation unit 23 calculates the rotational components R _x , R _y , R _z of the vehicle and motion vectors (u) for a plurality of pixels corresponding to the object 32 included in the image. , V) and a and b in the equations (6) and (7) based on the above, the translational components T _x / z, T _y / z, T _z / z with the scale factor of the object 32 are calculated. Is calculated.

Next, in step ST5, the Euclidean distance estimation unit 24 acquires the distance margin r0 from the storage unit 13, and the three-dimensional coordinate value (X, Y, Z) of the object 32 in the camera coordinate system and the translation component with a scale factor. Formulas (9) and (10) based on Formula (8) using T _x / z, T _y / z, and T _z / z are established.

Next, in step ST6, the Euclidean distance estimation unit 24 establishes Expression (13) obtained by subtracting the distance margin r0 from both sides of Expression (9). Further, the Euclidean distance estimation unit 24 sets a formula (14) for determining the on basis ^r 2 -r0 ² minima formula (13) gives the equation (14) to the collision determination unit 25.

Next, in step ST7, the collision determination unit 25 acquires the z coordinate value of the object 32 from the z position acquisition unit 26, and the z coordinate value and the translation components T _x / z, T _y / z with scale factors, By substituting T _z / z and solving equation (14), it is determined whether or not the minimum value of r ² −r 0 ² is less than or equal to zero. If r ² -r0 ² minimum value is less than zero, the process proceeds to step ST8. On the other ^hand, if the minimum value of r 2 -r0 ² is positive, the process proceeds to step ST13.

  Next, in step ST8, the collision time estimation unit 27 obtains the real roots t1 and t2 of the equation with the left side of the equation (14) being zero, and uses the smaller root t1 as a time TTC (Time To Collision) until the collision. presume.

  Next, in step ST <b> 9, the collision risk determination unit 28 determines whether or not TTC is smaller than the first time threshold s <b> 1 stored in the storage unit 13. If TTC is smaller than the first time threshold s1, the process proceeds to step ST10. On the other hand, if the TTC is equal to or greater than the first time threshold s1, the process proceeds to step ST11.

  Next, in step S10, the collision risk determination unit 28 determines that the collision risk is “high”, and proceeds to step ST15.

  On the other hand, if it is determined in step ST9 that TTC is equal to or greater than the first time threshold s1, the collision risk determination unit 28 determines in step ST11 that the TTC is greater than the second time threshold s2 stored in the storage unit 13. It is determined whether or not it is small. If TTC is smaller than the second time threshold s2, the process proceeds to step ST12. On the other hand, if the TTC is equal to or greater than the second time threshold s2, the process proceeds to step ST14.

  Next, in step ST12, the collision risk determination unit 28 determines that the collision risk is “medium”, and the process proceeds to step ST15.

  In step ST13, the collision risk determination unit 28 determines that the collision risk is “low”, and the process proceeds to step ST15.

On the other hand, when the minimum value of ^r 2 -r0 ² is determined to be positive at step ST7, at step ST14, the collision determination unit 25 approaches threshold minimum value of ^r 2 -r0 ² is stored in the storage unit 13 It is determined whether it is smaller than d1. When the minimum value of r ² −r 0 ² is smaller than the approach threshold value d 1, the process proceeds to step S 12 and the collision risk determination unit 28 determines that the collision risk is “medium”. On the other hand, if the minimum value of r ² −r 0 ² is greater than or equal to the approach threshold value d 1, the process proceeds to step S 14 and the collision risk determination unit 28 determines that the collision risk is “low”.

  In step S <b> 15, the warning presentation unit 29 issues a warning to the driver via the display unit 14 and the speaker 15 according to the collision risk classified by the collision risk determination unit 28.

  According to the above procedure, the possibility of collision with an object can be determined based on the image acquired by the camera 11 in consideration of the influence of vehicle curvature and vehicle vibration.

The vehicular image processing apparatus 10 according to the present embodiment calculates the motion vector (u, v) when calculating the translation components T _x / z, T _y / z, and T _z / z with the scale factor of the object 32. Equations (3) and (4) corrected with the rotation components R _x , R _y , and R _z are used. For this reason, the influence of the curvature and vibration of the vehicle is eliminated. Therefore, according to the vehicular image processing apparatus 10, it is possible to accurately determine the possibility of collision with the object 32 in consideration of the influence of the curvature of the own vehicle and the vibration of the own vehicle.

  Further, the vehicle image processing apparatus 10 determines the collision risk based on the distance r between the vehicle and the object 32. Therefore, the collision risk can be accurately determined regardless of whether the object 32 is a stationary object or a moving object. Further, if the object 32 is a stationary object, the z-coordinate value can be obtained from only one camera 11, and therefore the vehicle image processing apparatus 10 can be configured simply.

  The vehicle image processing apparatus 10 uses only a single coordinate system (camera coordinate system). For this reason, compared with the case where the camera coordinate system and the world coordinate system are used in combination, processing such as mutual conversion of the coordinate system becomes unnecessary, and the amount of calculation required for collision risk prediction can be reduced.

  Further, the vehicle image processing apparatus 10 determines whether the vehicle and the object are in accordance with the closest distance from the center of the camera 11 to the object 32 and the time TTC when the distance r from the center of the camera 11 to the object 32 is r0. The risk of collision with the object 32 can be classified into a plurality of stages, and an appropriate warning corresponding to this classification can be given to the driver.

  In addition, although some embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  Further, in the embodiment of the present invention, each step of the flowchart shows an example of processing that is performed in time series in the order described. The process to be executed is also included.

DESCRIPTION OF SYMBOLS 10 Vehicle image processing apparatus 11 Camera 12 Collision prediction ECU
DESCRIPTION OF SYMBOLS 13 Memory | storage part 21 Motion vector calculation part 22 Rotation component calculation part 23 Translation component calculation part 25 with a scale factor Collision determination part 26 z position acquisition part 27 Collision time estimation part 28 Collision risk determination part 31 Camera projection plane 32 Target object

Claims (6)

A camera provided in the vehicle for imaging the periphery of the vehicle;
A motion vector calculation unit for obtaining a motion vector between a plurality of images captured by the camera;
Wherein based on the rotational component of the vehicle and the motion vector corresponding to the object included in the image, the scale factor with translational component calculation unit for calculating a three-axis directions of the scale factor with the relative translational component of camera coordinate system ,
A z-position acquisition unit that acquires optical axis direction coordinate values of the object in the camera coordinate system;
A distance from the center of the camera to the object is expressed as a function of time using a three-dimensional coordinate value including the optical axis direction coordinate value of the object in the camera coordinate system and the relative translation component with the scale factor. A collision determination unit that determines whether or not the closest distance from the center of the camera to the object is equal to or less than a predetermined distance based on the equation;
A collision risk determination unit that determines that the vehicle and the object collide when the closest distance is equal to or less than the predetermined distance;
An image processing apparatus for a vehicle, comprising: The object is a stationary object;
A rotation component calculation unit that calculates the rotation component based on vanishing point information and a motion vector of at least three pixels;
The vehicular image processing apparatus according to claim 1, further comprising: The z position acquisition unit
The translation component of the vehicle is obtained from an image captured by the camera, and the translation component of the vehicle is divided by the relative translation component with the scale factor to obtain the coordinate value in the optical axis direction of the object in the camera coordinate system. get,
The image processing apparatus for vehicles according to claim 1 or 2. Using the three-dimensional coordinate value including the coordinate value in the optical axis direction of the object in the camera coordinate system and the relative translation component with the scale factor, the distance from the camera center to the object is a quadratic function of time. A collision time estimator that calculates a time when the distance from the center of the camera to the object is the predetermined distance, based on the expression expressed as:
The vehicle image processing apparatus according to claim 1, further comprising: The collision risk determination unit
Classifying the risk of collision between the vehicle and the object into a plurality of stages according to the closest distance and a time when the distance from the center of the camera to the object is the predetermined distance;
The image processing apparatus for vehicles according to claim 4. A vehicle image processing method using a vehicle image processing apparatus including a motion vector calculation unit, a translation component calculation unit with a scale factor, a z position acquisition unit, a collision determination unit, and a collision risk determination unit,
A camera provided in the vehicle images the periphery of the vehicle;
The motion vector calculating unit obtaining a motion vector between a plurality of images captured by the camera for each pixel;
Wherein with scale factor translational component calculation unit, on the basis of the motion vector corresponding to the object contained in the rotary component and the image of the vehicle, the three axis directions of the scale factor with the relative translational component of camera coordinate system A calculating step;
The z-position acquisition unit acquires an optical axis direction coordinate value of the object in the camera coordinate system;
The collision determination unit uses a three-dimensional coordinate value including the optical axis direction coordinate value of the object in the camera coordinate system and a relative translation component with the scale factor to a distance from the camera center to the object. Determining whether the closest distance from the center of the camera to the object is equal to or less than a predetermined distance based on an expression expressed as a function of time;
The collision risk determination unit determining that the vehicle and the object collide when the closest distance is equal to or less than the predetermined distance;
An image processing method for a vehicle, comprising:

Images