JP5663411B2 - Ranging device - Google Patents

Description

  The present invention relates to a distance measuring device that measures a distance between an object existing around a vehicle and a host vehicle using a captured image of an in-vehicle camera.

  For the purpose of driving support of the vehicle, a captured image around the vehicle is acquired by an in-vehicle camera mounted on the front part of the vehicle, and based on the captured image, an object such as a pedestrian existing around the vehicle and the own vehicle A technique for measuring the distance between the two is conventionally known.

  As the distance measuring technique, for example, as shown in Patent Document 1, a technique for measuring the distance to an object based on the parallax of the object in a stereo image captured by a stereo camera is generally known.

  As a distance measuring technique using a single camera, for example, as seen in Patent Document 2, the distance to the object is measured based on the temporal change rate of the size of the local area of the image of the object. Techniques have been proposed by the applicant.

JP 2000-283755 A JP 2009-265882 A

  As shown in Patent Document 1, a distance measuring technique using a stereo camera requires a plurality of cameras and high-precision calibration of the direction of the optical axis of these cameras. For this reason, a ranging system using a stereo camera is generally expensive and disadvantageous in terms of cost.

  In addition, the distance measuring technique as disclosed in Patent Document 2 does not require a plurality of cameras, but in order to increase the reliability of measurement of the temporal change rate of the size of the local area of the image of the object, It is necessary to measure the temporal change rate of the size at a time interval in which the change in size appears prominently. For this reason, the distance to the object estimated from the temporal change rate tends to cause a delay with respect to the actual distance. Furthermore, in order to measure the temporal change rate of the size, local regions of the image of the object extracted from the captured images captured at different times tend to vary among the captured images. For this reason, it is difficult to increase the reliability of the distance to the object estimated from the temporal change rate.

  The present invention has been made in view of such a background, and it is an inexpensive system configuration that uses an image captured by a single vehicle-mounted camera, and is provided between an object captured by the vehicle-mounted camera, the host vehicle, and the object. An object of the present invention is to provide a distance measuring device capable of performing distances with high reliability.

  First, basic technical matters relating to the present invention will be described.

  As shown in FIG. 1A, a situation is assumed in which a vehicle 1 equipped with an in-vehicle camera 2 is traveling on a flat road surface 51. In this situation, there is an object 53 (pedestrian in the illustrated example) on which the distance D from the vehicle 1 (the host vehicle 1) is to be measured on the road surface 52 in front of the vehicle 1.

  In this case, the road surface 52 on which the object 53 is present (the road surface 52 on which the object 53 is grounded) is the road surface on the same plane as the road surface 51 on which the vehicle 1 is present (the road surface 51 on which the vehicle 1 is grounded) ( Road surface 51 and the same road surface). Hereinafter, the road surfaces 51 and 52 may be referred to as the own vehicle existence road surface 51 and the target object existence road surface 52, respectively.

  Moreover, FIG.1 (b) has shown roughly the picked-up image imaged with the vehicle-mounted camera 2 in the condition shown to Fig.1 (a). The portions denoted by reference numerals 51a, 52a, and 53a indicate images of the own vehicle existence road surface 51, the object existence road surface 52, and the object 53, respectively.

  In the situation shown in FIG. 1A, as shown, the size of the object 53 in the real space (here, the total length in the vertical direction (height)) is H53. In addition, the size (full length in the vertical direction) of the target object 53 in the captured image indicated by the entire image 53a of the target object 53 in the captured image in this situation is defined as h53a.

  At this time, the relationship of the following equation (1) is established between these H53 and h53a and the distance D of the object 53 from the host vehicle 1 (distance in a direction parallel to the host vehicle existing road surface 51). To do. Note that F is a focal length of the in-vehicle camera 2.

D = F · (H53 / h53a) (1)

Therefore, if the value of the size H53 of the object 53 in the real space (hereinafter, sometimes referred to as the object size H53) is specified, the value of H53 and the captured image obtained from the image 53a of the object 53 are included. From the size h53a of the target object 53 (hereinafter, sometimes referred to as the target size h53a in the image) and the value of the focal length F of the in-vehicle camera 2 (this can generally be set in advance as a known value), ), The distance D between the object 53 and the host vehicle 1 can be estimated.

  Here, in many cases, the general object size H53 is a value around a certain standard size corresponding to the type of the object 53. Therefore, if the standard size value of the object 53 (hereinafter, sometimes referred to as the object standard size value H53s) is used as the value of the object size H53, the object 53 and the object size H53 are automatically calculated by the following equation (1a). The distance D of the vehicle 1 can be roughly estimated. That is, by assuming that the actual object size H53 substantially matches the object standard size value H53s, the object 53 and the object 53 are automatically compared with each other based on the ratio between the object standard size value H53s and the object size h53a in the image. The distance D to the vehicle 1 can be roughly estimated by the following equation (1a).

D = F · (H53s / h53a) (1a)

In this case, as the object standard size value H53s of the object 53, an average value of a generally known object size H53 may be used for the object type to which the object 53 belongs.

  Supplementally, in the above description, the object size H53 is the total length in the vertical direction of the object 53. However, depending on the type of the object 53, H53 may be the horizontal length of the object 53 or an oblique direction. It may be the length in the direction. For example, when the object 53 is a vehicle (another vehicle), H53 may be the total length (vehicle width) of the object 53 in the width direction. The object size H53 basically has a relatively small variance (variation) in the value of the object type to which the object 53 belongs, and the size of the object 53 in the captured image (the object in the image). The object size) is preferably a length that can easily specify h53a (a length in a predetermined direction determined in advance for each type of the object 53 to be measured for the distance D to the host vehicle 1).

  In the following description, the method for estimating the distance D between the object 53 and the host vehicle 1 by the equation (1a) as described above is referred to as a first distance measuring method. The distance D estimated by the first distance measuring method is referred to as a first distance estimated value D1. In the first distance measuring method, the error of the first distance estimated value D1 estimated thereby depends on the error of the actual object size H53 value with respect to the object standard size value H53s, and the vehicle 1 travels when the vehicle 1 travels. There is a characteristic that it is difficult to be affected by the change in the direction of the optical axis Lc of the in-vehicle camera 2 and the change in the gradient between the object existing road surface 52 and the host vehicle existing road surface 51.

  Therefore, the error of the first distance estimated value D1 falls within a certain error range. More specifically, assuming that the known maximum value of the general object size H53 is H53max and the minimum value is H53min, the error of the first distance estimation value D1 is ((H53s / H53max ) -1) · 100 [%] and ((H53s / H53min) -1) · 100 [%]. In this case, H53s / H53max is equal to the ratio of the first estimated distance value D1 to the distance D value calculated by the equation (1) assuming that H53 = H53max. Similarly, H53s / H53min is equal to the ratio of the first estimated distance value D1 to the distance D value calculated by the equation (1) on the assumption that H53 = H53min.

  Here, the error of the first distance estimated value D1 is an error defined by the proviso of Expression (2). In this case, however, Dr being written means the actual distance D (the true value of the distance D) between the object 53 and the host vehicle 1.

((H53s / H53max) -1) · 100 [%]
≦ Error of first distance estimated value D1 ≦ ((H53s / H53min) −1) · 100 [%]
(2)
However,
Error ≡ ((D1 / Dr) −1) · 100 [%] of first distance estimated value D1

As a specific example, when the object 53 is an adult pedestrian in Japan and the object size H53 is the total length (height) of the object 53 in the vertical direction, it is generally known. The maximum value H53max and the minimum value H53min of the object size H53 are about 210 [cm] and 120 [cm], respectively. Moreover, the average height of a generally known Japanese adult is 164.7 [cm]. In this case, when the average height (164.7 [cm]) of a Japanese adult is used as the object standard size value H53s, the first distance estimation error is −21.6 [ %] And 37.3 [%].

-21.6 [%] ≤ error of first distance estimated value D1 ≤ 37.3 [%] (2a)

Next, in the situation shown in FIG. 1A, as shown, the angle formed by the optical axis Lc of the in-vehicle camera 2 with respect to the own vehicle existing road surface 51 is α, and the ground point P53 of the object 53 from the in-vehicle camera 2 is shown. The angle formed by the straight line L1 reaching (the contact point between the object 53 and the object existence road surface 52) with respect to the optical axis Lc of the in-vehicle camera 2 is γ, and the height of the in-vehicle camera 2 from the own vehicle existence road surface 51 is Hc. And

  Here, the height Hc of the in-vehicle camera 2 (hereinafter sometimes referred to as the camera height Hc) is more specifically the height of the optical center C of the in-vehicle camera 2 from the own vehicle existing road surface 51. More specifically, the straight line L1 related to the angle γ is a straight line from the optical center C of the in-vehicle camera 2 to the ground point P53 of the object 53. More specifically, the angles α and γ are angles around the axis in the normal direction of a plane that includes the optical axis Lc of the in-vehicle camera 2 and stands upright with respect to the road surface 51 where the vehicle exists. The angles α and γ correspond to the angle in the pitch direction of the vehicle 1 (that is, the angle around the axis of the vehicle 1 in the vehicle width direction) in the situation shown in FIG.

  The angle α is a situation in which the optical axis Lc of the in-vehicle camera 2 extends obliquely downward with respect to a plane parallel to the own vehicle existing road surface 51 through the optical center C of the in-vehicle camera 2 (FIG. 1A). In the situation shown in FIG. 2) and a negative angle in a situation where the optical axis Lc of the in-vehicle camera 2 extends obliquely upward with respect to the plane. The angle γ is a positive angle when the straight line L1 extends obliquely downward with respect to the optical axis Lc of the in-vehicle camera 2, and the straight line L1 is obliquely downward with respect to the optical axis Lc of the in-vehicle camera 2. A negative angle is assumed in the situation extending to

  In FIG. 1A, α ≠ 0 [deg], but α = 0 [deg] may be used.

  When the angles α and γ are defined as described above, the distance D of the object 53 from the host vehicle 1 (the distance in the direction parallel to the host vehicle existing road surface 51), the angles α and γ, and the camera height Hc The relationship of following Formula (3) is materialized between.

D = Hc / tan (α + γ) (3)

Further, as shown in FIG. 2 (a), the gradient of the road surface ahead of the vehicle 1 changes, and the object existing road surface 52 becomes a road surface (inclined road surface) having a gradient with respect to the own vehicle existing road surface 51. Assuming the situation. This situation is different from the situation shown in FIG. 1A only in the gradient state of the object existence road surface 52.

  In addition, FIG.2 (b) has shown roughly the picked-up image imaged with the vehicle-mounted camera 2 in the condition shown to Fig.2 (a). Similarly to the case of FIG. 1B, the portions denoted by reference numerals 51 a, 52 a, and 53 a indicate images of the own vehicle existence road surface 51, the object existence road surface 52, and the object 53, respectively.

  In the situation shown in FIG. 2A, angles α and γ in the figure are angles defined in the same manner as in the situation shown in FIG. In addition, in this situation, a point located below the in-vehicle camera 2 in the own vehicle existence road surface 51 (specifically, a vertical line dropped from the optical center C of the in-vehicle camera 2 to the own vehicle existence road surface 51 is the own vehicle existence road surface. The angle formed by the straight line L2 extending from the intersection 51 to the ground contact point P53 of the object 53 with respect to the own vehicle existing road surface 51 is β (> 0).

  Note that the angle β is similar to the angles α and γ and includes an optical axis Lc of the in-vehicle camera 2 and an angle around a normal axis of a plane that stands perpendicular to the vehicle-existing road surface 51 ( In the situation shown in FIG. 1B, the angle of the vehicle 1 in the pitch direction).

  When the angle β is defined in addition to the angles α and γ as described above, the distance D from the host vehicle 1 to the object 53, the angles α, γ, β, and the camera height Hc are as follows. The relationship of Formula (4) is materialized.

D = Hc / (tan (α + γ) + tanβ) (4)

In this case, in the equation (2), if β = 0, the equation (4) matches the equation (3). The situation shown in FIG. 1A corresponds to the situation where β in FIG. 2A is set to “0”.

  Accordingly, in any of the situations shown in FIGS. 1A and 2A, the value of the camera height Hc of the in-vehicle camera 2 and the values of the angles α, γ, and β (values in units of [rad]) ), The distance D between the object 53 and the host vehicle 1 can be specified by the calculation of the right side of the above equation (2) from these values.

  Here, since the camera height Hc is defined by the mounting position of the in-vehicle camera 2 with respect to the vehicle body of the vehicle 1, in general, the value can be set in advance as a known value.

  Further, the angle γ can be estimated based on the captured image of the in-vehicle camera 2. Specifically, as shown in FIG. 1B or FIG. 2B, the on-image object grounding point P53a, which is a point formed by projecting the grounding point P53 of the object 53 onto the captured image, and the captured image. Assuming that the vertical position deviation between the image center Pca (intersection of the imaging surface of the in-vehicle camera 2 and the optical axis Lc) is Δy, the following equation (5) is established. Note that F is the focal length of the in-vehicle camera 2 as described above.

tanγ = Δy / F (5)

Therefore, in the captured image, if the vertical position deviation Δy between the on-image object ground point P53a and the image center Pca is specified, the above equation (5) is obtained from the value of Δy and the value of the focal length F. Based on this, the angle γ can be estimated.

  Supplementally, on the object existence road surface 52, an in-vehicle camera 2 represents an arbitrary stationary point (a point fixed with respect to the object existence road surface 52) at which the distance from the vehicle 1 is the same as the ground contact point P53 of the object 53. For example, as shown in FIG. 1B or FIG. 2B, the point projected on the captured image extends in the horizontal direction (horizontal direction) of the captured image through the object grounding point P53a on the image. This is a point on the line L53a (for example, the point P1 in the figure).

  In this case, the vertical distance between this point (for example, P1) and the image center Pca coincides with the vertical distance Δy between the image object grounding point P53a and the image center Pca. Therefore, the angle γ can also be estimated based on the equation (5) from the interval Δy between the point (for example, P1) obtained by projecting the stationary point on the captured image and the image center Pca.

  Therefore, if the values of the angles α and β are estimated by an appropriate method or set to predetermined values, the estimated values or set values of the angles α and β and the captured image of the in-vehicle camera 2 are used as described above. By using the value of the angle γ estimated as described above, the distance D between the object 53 and the host vehicle 1 can be estimated by the above formula (3) or (4). In the following description, the method for estimating the distance D between the object 53 and the host vehicle 1 using the estimated value of the angle γ is referred to as a second distance measuring method. Further, the estimated value of the distance D by the second distance measuring method is referred to as a second distance estimated value D2.

  In the second distance measuring method, in the situation where the vehicle 1 is traveling on a flat road surface where there is no change in the gradient or the change in the gradient is sufficiently small, the type and size of the object 53 Regardless of H53, the distance D between the object 53 and the host vehicle 1 can be estimated with relatively high accuracy.

  However, the second distance estimated value D2 based on the second distance measuring method is, as can be seen from the above formula (3) or (4), the change in the posture of the vehicle body when the vehicle 1 is traveling (and consequently the own vehicle traveling road surface). 51 is greatly affected by fluctuations in the angle α accompanying the fluctuation in the direction of the optical axis Lc of the in-vehicle camera 2 with respect to 51 and changes in the gradient of the road surface on which the vehicle 1 is traveling. Accordingly, the angle α, the estimated value of the change in the slope of the road surface, or the error of those set values greatly affects the accuracy of the second distance estimated value D2 by the second distance measuring method.

  More specifically, as the estimated value or set value of the angle α, or the estimated value of the change in the slope of the road surface (the estimated value of the angle β or a parameter that defines this) or the error of the set value increases, the second distance There is a tendency that the error range of the estimated value D2 rapidly increases.

  For example, when the second distance estimated value D2 is calculated by the equation (4), the error range of the second distance estimated value D2 is a combined angle of the angles α and β as illustrated in FIG. With a slight increase in error (angle θ shown in FIG. 2A), it expands rapidly.

  FIG. 3 shows an example in which the actual distance D (the true value of the distance D) between the object 53 and the host vehicle 1 is 30 [m], for example. Further, the error on the horizontal axis (α + β error) is an absolute value. The upper curve a1 and the lower curve a2 in FIG. 3 are curves indicating the second distance estimation error when the horizontal axis error polarity is positive and negative, respectively. In this case, the error of the second distance estimated value D2 is defined by an error defined in the same manner as the error of the first distance estimated value D1 (an expression in which D1 in the proviso of the above equation (2) is replaced with D2. Error).

  Therefore, the estimated value or set value of the angle α, or the estimated value of the change in the slope of the road surface (the angle β or the estimated value of the parameter that defines it) or the error of the set value (hereinafter these are collectively referred to as the angle αβ When the error) increases to some extent, the error range of the second distance estimation value D2 also increases. As a result, in a situation where the angle αβ-related error is large to some extent, the error range of the second distance estimated value D2 is larger than the first distance estimated value D1, and the reliability of the second distance estimated value D2 is greater than the first distance estimated value D1. It becomes lower than the distance estimated value D1.

  For example, when the object 53 is an adult pedestrian in Japan, the error range of the first distance estimated value D1 shown by the equation (2a) is a range between the broken lines b1 and b2 in FIG. The error range of the second distance estimated value D2 is the error range of the first distance estimated value D1 when the absolute value of the error of the combined angle θ of the angles α and β exceeds about 0.8 [deg]. Bigger than.

  The present invention considers the characteristics of the first distance measurement method and the second distance measurement method as described above, and appropriately uses the first distance measurement method and the second distance measurement method, so The distance to the vehicle is estimated.

  The present invention will be described below based on the technical matters described above.

The present invention is a distance measuring device for measuring a distance between an object on a road surface imaged by an in-vehicle camera and a host vehicle on which the in-vehicle camera is mounted,
An object extraction means for extracting an image of an object to be measured from the captured image of the in-vehicle camera, and the distance from the host vehicle;
Based on the image of the object extracted from the captured image, the object size in the image, which is the size of the object in the captured image, is specified, and the specified object size in the image and the in-vehicle camera From the ratio of the object standard size value set in advance as the standard value of the object size, which is the size of the object in real space seen in the optical axis direction, and the setting value of the focal length of the in-vehicle camera First distance estimating means for determining a first distance estimated value that is a first estimated value of the distance between the object and the host vehicle at the imaging time t1 of the captured image;
A stationary point on the object existing road surface on which the object exists is a stationary point whose distance from the host vehicle is the same as the ground contact point of the object at the imaging time t1. A feature point extraction means for distance measurement that extracts from the captured image a feature point formed by projecting on the captured image captured in step, and specifies a position of the feature point in the captured image;
Based on the position of the feature point specified in the captured image, a first reference angle value that is an angle formed by a straight line from the in-vehicle camera to the stationary point with respect to the optical axis of the in-vehicle camera is estimated. 1 reference angle estimation means;
Based on at least the estimated value of the first reference angle and the set value of the height of the in-vehicle camera, a second estimated distance value that is a second estimated value of the distance between the object and the host vehicle is determined. A second distance estimating means;
The distance value estimated based on the ratio between the specified object size in the image and the maximum size setting value set in advance as the maximum value of the object size, and the specified object size in the image And a range between the distance value estimated based on a ratio of a minimum size setting value set in advance as a minimum value of the target object size as an allowable range of the second distance estimation value, Second distance estimated value judging means for judging whether or not the second distance estimated value is within the set allowable range;
If the determination result of the second distance estimated value determining means is affirmative, the second distance estimated value is determined as an estimated value of the distance between the object and the host vehicle, and the determination result is negative If it is, the Ru der which the basic configuration in that it comprises a distance estimate determining means for determining a first distance estimate as an estimate of the distance between the object and the vehicle.

The first reference angle above SL is more particularly a plane including the optical axis of the on-vehicle camera, and the normal direction to or substantially the same direction of the plane standing perpendicular to the vehicle there road Is the angle around the axis. The same applies to a second reference angle and a third reference angle described later.

  The size of the object in real space (object size) is a predetermined direction (vertical direction, horizontal direction, diagonal direction, etc.) determined in advance corresponding to the type of the object whose distance from the host vehicle is to be measured. ) Is the length of the actual object (object in real space). The size of the target object in the captured image (the target object size in the image) means the length in the predetermined direction in the image of the target object in the captured image.

According to the above basic configuration , the object (object to be measured for the distance to the host vehicle) extracted from the captured image (captured image at the imaging time t1) of the in-vehicle camera by the first distance estimating means. The object size in the image is specified based on the image. This in-image object size corresponds to the above-described in-image object size h53a.

  Further, the first distance estimating means determines the first distance estimated value from the ratio between the specified object size in the image and the object standard size value and the set value of the focal length of the in-vehicle camera. This first distance estimated value corresponds to the first distance estimated value D1 obtained by the first distance measuring method. Therefore, the first distance estimated value (D1) can be determined by calculating the right side of the equation (1a) using the specified object size in the image and the object standard size value.

  Further, a stationary point on the road surface on which the object exists is such that the distance from the host vehicle is the same as the ground contact point of the object at the imaging time t1 by the distance measurement feature point extracting means. The feature points formed by projecting on the captured image captured in (1) are extracted. Then, the position of the feature point in the captured image is specified.

  The feature point is, for example, a point formed by projecting the ground point of the object at the imaging time t1 onto the captured image (in the example shown in FIG. 1B or FIG. 2B, the point P53a). ) Can be extracted as a point having a local feature such as luminance on a horizontal line passing through (). As such a feature point, for example, a Harris corner point, a minimum eigenvalue point, or the like can be used. The feature point may be a point that coincides with a grounding point of an object (an object grounding point on the image) in the captured image at the imaging time t1.

  Further, the first reference angle estimation means estimates the value of the first reference angle based on the position of the feature point in the captured image. This first reference angle means the angle γ in FIG. 1 (a) or FIG. 2 (a). Therefore, the value of the first reference angle (γ) can be estimated based on the position of the feature point in the captured image. More specifically, the value of the first reference angle (γ) can be estimated based on the equation (5).

  Further, the second distance estimation means determines the second distance estimation value based at least on the estimated value of the first reference angle (γ) and the set value of the height of the in-vehicle camera. This second distance estimated value corresponds to the second distance estimated value D2 obtained by the second distance measuring method. Therefore, the second distance estimated value (D2) can be determined based on the formula (3) or (4).

  In this case, the value of the angle α in the equations (3) and (4) or the value of the angle β in the equation (4) may be estimated by an appropriate method (may be measured using a gyro sensor or the like). Alternatively, in a situation where the angle α or β is predicted to be maintained at a substantially constant value, a predetermined set value may be used as the value. For example, “0” may be used as the value of the angle β when the vehicle is traveling on a road surface where the change in the gradient is minute. Also, for example, in a situation where the vehicle is traveling on a flat road surface when the in-vehicle camera is attached to the vehicle body so that its optical axis is in the same direction as the longitudinal direction of the vehicle body of the vehicle. Also, “0” may be used as the value of the angle α. As described above, when known set values are used as the values of the angles α and β, it is not necessary to estimate (measure) the values of the angles α and β.

  Next, an allowable range of the second distance estimated value is set by the second distance estimated value determining means. In this case, the allowable range includes the distance value estimated based on the ratio between the specified object size in the image and the maximum size setting value, the specified object size in the image, and the minimum value. A range between the distance value estimated based on the ratio to the size setting value is set.

  Here, the value of the distance estimated based on the ratio between the object size in the specified image and the maximum size setting value is based on the assumption that the object size matches the maximum size setting value. Means the value of D estimated by the equation (1). Similarly, the distance value estimated based on the ratio between the specified object size in the image and the minimum size setting value is based on the assumption that the object size matches the minimum size setting value. Means the value of D estimated by the equation (1).

  The actual object size generally falls within the range between the maximum size setting value and the minimum size setting value. Therefore, the allowable range set by the second distance estimated value determination means means a range in which the actual distance value between the object and the host vehicle can exist with high accuracy. Such a range is set as an allowable range of the second distance estimated value (D2).

  For this reason, if the second distance estimated value (D2) determined by the second distance estimating means deviates from the allowable range, the second distance estimated value (D2) is actually The reliability of the distance D as an estimated value is poor compared to the first distance estimated value (D1).

  When the second distance estimated value (D2) is within the allowable range, the reliability of the second distance estimated value (D2) is the error of the first distance estimated value (D1). It can be regarded as being secured within the scope. In this case, the error of the estimated values or the set values of the angles α and β can be regarded as sufficiently small, so that the accuracy of the second distance estimated value (D2) is also ensured to be relatively high. Can be regarded as being.

Therefore, in the basic configuration of the present invention, the distance estimated value determination means determines the second distance estimated value as the object and the own vehicle when the determination result of the second distance estimated value determination means is affirmative. When the determination result is negative, the first distance estimated value is determined as an estimated value of the distance between the object and the host vehicle.

Thereby, according to the basic configuration of the present invention, it is possible to prevent the occurrence of an excessive error in the value determined (determined) as the estimated value of the distance between the object and the host vehicle. A reliable distance estimate can be determined. In this case, since the vehicle-mounted camera may be a single vehicle-mounted camera, the distance measuring device according to the first invention can have an inexpensive system configuration.

Therefore, according to the basic configuration of the present invention, a low-cost system configuration using an image captured by a single in-vehicle camera increases the distance between the object captured by the in-vehicle camera, the host vehicle, and the object. It can be done with reliability.

In the basic configuration of the present invention, in the processing of the second distance estimating means, the value of the angle α may be set to a predetermined value (for example, “0”). The direction generally changes as the posture of the vehicle body changes. Accordingly, in order to further improve the accuracy of the second distance estimated value (D2), the angle α is sequentially estimated, and the second distance estimated value (D2) is determined using the estimated value of the angle α. Also good.

  Here, the angle α can be estimated by various methods. For example, the value of the angle α can be estimated using a captured image of an in-vehicle camera.

In this case, for example, the following configuration is adopted in the basic configuration . That is, based on two or more captured images taken at different imaging times, camera motion measuring means for sequentially measuring the motion state of the in-vehicle camera including at least the translational movement direction of the in-vehicle camera, and the in-vehicle camera A second reference for estimating a value of a second reference angle that is an angle formed by the optical axis of the in-vehicle camera with respect to the road surface on which the vehicle exists at the imaging time t1 from the measured motion state Angle estimation means, and the second reference angle estimation means is based on at least the estimated value of the first reference angle, the estimated value of the second reference angle, and the set value of the height of the in-vehicle camera. The second distance estimation value is determined.

However, in this case, when the vehicle speed of the vehicle is a low vehicle speed that is equal to or lower than a predetermined value, the distance estimated value determining means does not depend on the determination result of the second distance estimated value determining means, It confirms one distance estimate as an estimate of the distance between the object and the vehicle (first invention).

According to the first aspect of the present invention , the camera motion measurement means sequentially measures the motion state of the in-vehicle camera including at least the translational movement direction of the in-vehicle camera. In this case, the measurement of the motion state of the in-vehicle camera is performed based on two or more captured images captured at different imaging times. Such measurement can be performed by a known image processing method based on, for example, Structure from Motion (hereinafter, sometimes referred to as SfM).

  Then, the second reference angle is estimated by the second reference angle estimation means from the measured motion state of the in-vehicle camera. This second reference angle means the angle α in FIG. 1 (a) or FIG. 2 (a).

  In this case, since the translational movement direction of the in-vehicle camera is substantially parallel to the road surface on which the host vehicle is present, the second reference angle (α) is basically the in-vehicle direction with respect to the optical axis of the in-vehicle camera. It is defined according to the translation direction of the camera. Therefore, the value of the second reference angle (α) can be estimated based on the motion state of the in-vehicle camera including at least the translational movement direction of the in-vehicle camera.

  Supplementally, the motion state of the in-vehicle camera measured by the camera motion measuring means may include an angle change amount of the posture of the in-vehicle camera (an angle change amount between different imaging times). In that case, the value of the second reference angle (α) can be estimated using both the translational movement direction of the in-vehicle camera and the movement change amount of the attitude of the in-vehicle camera.

In the first invention , the second reference angle estimation means includes at least an estimated value of the first reference angle (γ), an estimated value of the second reference angle (α), and a set value of the height of the in-vehicle camera. Based on the above, the second distance estimated value (D2) is determined. For example, the second distance estimated value (D2) is determined by the equation (3). By determining the second distance estimated value (D2) in this way, the second distance estimated value (D2) is reflected by reflecting the change in the direction of the optical axis of the in-vehicle camera accompanying the change in the posture of the vehicle body. Can be determined.

  Here, when the vehicle speed of the vehicle is low, a plurality of captured images captured within a short time are unlikely to have a difference between them. For this reason, in such a state, it is generally difficult for the camera motion measurement means to accurately measure the motion state of the in-vehicle camera, and the reliability of the motion state of the in-vehicle camera to be measured is reduced. As a result, the reliability of the estimated value of the second reference angle (α) and the estimated value of the second distance (D2) determined using the estimated value are also reduced.

Therefore, in the first invention , the distance estimated value determining means, when the vehicle speed of the vehicle is a low vehicle speed equal to or lower than a predetermined value, is not dependent on the determination result of the second distance estimated value determining means. The 1-distance estimated value is determined as the estimated value of the distance between the object and the vehicle.

  Thereby, it can prevent that the 2nd distance estimated value (D2) with low reliability is finally decided as an estimated value of the distance of a target object and the own vehicle.

In the first aspect of the present invention , when the vehicle is traveling on a road surface where there is no change in the slope of the road surface, or the road surface becomes minute, the value of the angle β may be set to “0”. It is also possible to sequentially estimate the value of the angle β using the captured image.

Specifically, for example, the following configuration can be adopted. That is, the distance measurement feature point extracting means includes a feature point obtained by projecting the still point onto a captured image captured at the imaging time t1, and a captured image captured at an imaging time t2 different from the imaging time t1. Is extracted from each of the two captured images, and the position of each feature point in each of the two captured images is specified. Means for successively measuring camera motion parameters representing changes in the position and orientation of the in-vehicle camera in a period between the respective imaging times t1 and t2 of the two captured images, and the camera motion parameters and the two captured images; From the position of the feature point in each, the height of the vehicle-mounted camera, and the point located below the vehicle-mounted camera on the road surface where the vehicle is present at the imaging time t1 The measured value of the camera motion parameter and the two imaging based on an arithmetic expression representing a relationship between a straight line leading to the stationary point and a third reference angle that is an angle formed with respect to the optical axis of the in-vehicle camera. Third reference angle estimation means for estimating the value of the third reference angle as an unknown in the calculation formula from the position of the feature point specified in each of the images and the set value of the height of the in-vehicle camera The second distance estimating means further includes an estimated value of the first reference angle, an estimated value of the second reference angle, an estimated value of the third reference angle, and a set value of the height of the in-vehicle camera. Is a means for determining the second distance estimation value on the basis of ( 2nd invention ).

In the second aspect of the invention , the third reference angle estimated by the third reference angle estimation means is located below the in-vehicle camera on the road surface of the host vehicle at the imaging time t1 of the first captured image. Since the straight line from the point to the stationary point is an angle formed with respect to the optical axis of the vehicle-mounted camera, the angle θ shown in FIG. ) Corresponding to the combined angle (= α + β) of the angle α and the angle β.

Here, the principle of the third reference angle (θ) estimation method in the second invention will be described below.

  First, as shown in FIG. 4A, it is assumed that a flat road surface S (plane) existing in front of the in-vehicle camera 2 is imaged at different times t1 and t2. A point obtained by projecting an arbitrary stationary point P on the road surface S onto a captured image at time t1 (hereinafter sometimes referred to as a first captured image) is P1, and a captured image at time t2 (hereinafter referred to as a first captured image). P2 is a point that is projected onto the second captured image).

  In this case, the projection point P1 in the first captured image and the projection point P2 in the second captured image are each extracted as a feature point in each captured image, and the position of each point in the captured image is specified. To do.

Then, a position vector representing the position of the projection point P1 in the first captured image in homogeneous coordinates is ↑ P1 (≡ [x1, y1,1] ^T ), and the position of the projection point P2 in the second captured image is the same. A position vector expressed in the next coordinate is represented by ↑ P2 (≡ [x2, y2,1] ^T ).

  Note that x1 and y1 are the coordinate position in the horizontal direction and the coordinate position in the vertical direction (vertical direction) of the projection point P1 in the first captured image, respectively. Similarly, x2 and y2 are the coordinate position in the horizontal direction and the coordinate position in the vertical direction (vertical direction) of the projection point P2 in the second captured image, respectively. The subscript “T” means transposition.

  Further, a translation vector representing the spatial translational displacement of the position of the in-vehicle camera 2 in the period between the time t1 and the time t2 is represented by ↑ t, and the in-vehicle camera 2 in the period between the time t1 and the time t2. Let R be a rotation matrix representing the spatial rotational displacement of the posture. In other words, the translation vector ↑ t and the rotation matrix R represent the spatial translational displacement and the rotational displacement of the camera coordinate system in the period between the time t1 and the time t2, respectively.

  Note that the rotation matrix R is an orthogonal matrix. The camera coordinate system is a coordinate system fixed to the vehicle-mounted camera 2. More specifically, the camera coordinate system has, for example, the optical center C of the in-vehicle camera 2 as the origin, the horizontal direction of the captured image of the in-vehicle camera 2 in the X-axis direction, the vertical direction (vertical direction) in the Y-axis direction, and light. This is a three-axis coordinate system in which the direction of the axis Lc is the Z-axis direction.

  Further, as shown in FIG. 4B, a unit vector in the normal direction of the road surface S (hereinafter simply referred to as a normal vector) is represented by ↑ n, one of the times t1 and t2, for example, at time t1. Let d be the distance between the optical center C of the in-vehicle camera 2 and the road surface S. These ↑ n and d are plane parameters that define the spatial posture and position of the road surface S (plane), respectively.

In addition, since the road surface S is generally substantially parallel to the vehicle width direction, the road surface S is substantially parallel to the lateral direction of the captured image of the in-vehicle camera 2 (X-axis direction of the camera coordinate system). Therefore, when the inclination angle of the road surface S with respect to the optical axis Lc of the in-vehicle camera 2 at time t1 is θ as shown in FIG. 4B, the normal vector ↑ n of the road surface S is the camera coordinate system at time t1. ↑ n = [0, cos θ, sin θ] ^T is expressed using the coordinate component at.

  Here, the relationship between the position vector ↑ P1 of the projection point P1 of the first captured image and the position vector ↑ P2 of the projection point P2 of the second captured image is associated by plane projective transformation. Specifically, H is a homography matrix (planar projection transformation matrix) for converting the position vector ↑ P1 of the projection point P1 of the first captured image into the position vector ↑ P2 of the projection point P2 of the second captured image. Then, as shown in the following equation (6), ↑ P2 is a vector obtained by multiplying H ・ ↑ P1 by a certain proportionality constant k.

↑ P2 = k ・ H ・ ↑ P1 (6)

The homography matrix H is given by the following equation (7) using the translation vector ↑ t, the rotation matrix R, and the plane parameters ↑ n, d of the road surface S.

H = ^{RT- RT} , ↑ t, (↑ n / d) ^T (7)

Therefore, the following formula (8) is obtained by applying the formula (7) to the formula (6).

↑ P2 = k · (R ^T −R ^T · ↑ t · (↑ n / d) ^T ) · ↑ P1 (8)

In this equation (8), if ↑ P1, ↑ P2, R, and ↑ t are known, the value of (cosθ) / d or (sinθ) / d that is the component of ↑ n / d is obtained as follows. be able to.

That is, the translation vector ↑ t seen in the camera coordinate system at time t1 is expressed as ↑ t = [t1, t2, t3] ^T using coordinate components. The component display of the transposed matrix ^RT of the rotation matrix R is given by the following equation (9).

Further, variables a and b are defined as a≡ (cos θ) / d and b≡ (sin θ) / d, respectively, and ↑ n / d = [0, a, b] ^T is set. At this time, the equation (8) is expressed by the following equation (10) using respective components of ↑ P1, ↑ P2, R ^T , ↑ t, and ↑ n / d.

  Then, when k is deleted from the first row and the second row using the equation in the third row of the equation (10), the following equations (11a) and (11b) are obtained.

  Note that the determinant of the coefficient matrix (second-order square matrix with (ef) as the first row component and (hi) as the second row component) of the variables a and b related to the equations (11a) and (11b) is “ Therefore, the values of the variables a and b cannot be determined using these equations (11a) and (11b) as simultaneous equations.

  On the other hand, since a≡ (cos θ) / d and b≡ (sin θ) / d, the following equation (11c) is established.

a ² + b ² = 1 / d ² (11c)

Therefore, the values of a and b can be calculated by using the equations (11a) to (11c) as simultaneous equations. For example, when b is deleted using the formula obtained by adding both sides of the formulas (11a) and (11b) and the formula (11c), the following formula (12) is obtained.

  Therefore, the value of the variable a can be obtained as a solution of the quadratic equation shown by the equation (12). In this case, there are generally two values of a that are the solutions of equation (12), but of these two values, the value of a that satisfies equations (11a) and (11b) Is finally selected as the value of a that satisfies all of the equations (11a) to (11c).

  Of course, it is possible to delete a from the equations (11a) to (11c) and determine the value of b in the same manner as described above. Further, the other value can be determined from one value of a and b by the formula (11a) or (11b).

  Thus, if ↑ P1, ↑ P2, R, and ↑ t are known, the value of the variable a or b can be calculated based on the equations (11a) to (11c). Since a≡ (cos θ) / d and b≡ (sin θ) / d, if the value of the variable a or b is calculated as described above, the following equation (13a) or (13b ), The inclination angle θ of the road surface S with respect to the optical axis Lc of the in-vehicle camera 2 can be obtained.

θ = cos ⁻¹ (a · d) (13a)
θ = sin ⁻¹ (b · d) (13b)

Next, it is assumed that the first captured image captured by the vehicle-mounted camera 2 at time t1 is an image captured in the situation of FIG. 2A described above, and the imaging time t2 of the second captured image is the time t1. It is assumed that it is immediately after or immediately before. It is assumed that the time difference between the times t1 and t2 is a short time difference that makes the change in the distance between the object 53 and the vehicle 1 within the time interval sufficiently small.

  The stationary point P projected onto the first captured image and the second captured image at times t1 and t2, respectively, is the same as or substantially the same as the ground point P53 of the object 53 at the distance from the host vehicle 1 at the time t1. It is assumed that the object is a stationary point on the road surface 52. The stationary point P may be a stationary point that coincides with the grounding point P53 of the object 53 at time t1.

  Further, in the situation shown in FIG. 2 (a), the normal direction of the plane that stands up perpendicularly to the own vehicle existing road surface 51 including the optical axis Lc of the in-vehicle camera 2 (the vehicle in the situation of FIG. 1 is assumed to be a virtual road surface parallel to the vehicle width direction (direction perpendicular to the paper surface) and including the straight line L2 as a virtual road surface on which the object 53 exists. This is referred to as an object existence virtual road surface Sa. Then, the normal vector of the object existence virtual road surface Sa and the distance from the optical center C of the in-vehicle camera 2 to the object existence virtual road surface Sa at time t1 are newly set as ↑ n and d, respectively.

  At this time, when the plane parameters ↑ n and d of the object existence virtual road surface Sa are applied to the right side of the equation (8), the position vector ↑ P1 of the projection point P1 of the first captured image and the second captured image It is considered that the relationship represented by the equation (8) is approximately established with the position vector ↑ P2 of the projection point P2.

  Accordingly, the translation vector ↑ t and the rotation matrix R representing the change in the position and orientation of the in-vehicle camera 2 in the period between the time t1 and the time t2, respectively, and the distance from the host vehicle 1 at the time t1 are the object 53. Position vectors ↑ P1 and ↑ P2 of projection points P1 and P2 obtained by projecting a stationary point P on the object presence road surface 52 that is the same as the ground contact point P53 to the first and second captured images, respectively. When the variable a (≡ (cos θ) / d) or b (≡ (sin θ) / d) is calculated as described above, the value of the variable a or b is approximately the object existing virtual road surface Sa. It corresponds to the value of the component of ↑ n / d corresponding to.

  The distance d between the object existence virtual road surface Sa and the optical center C of the in-vehicle camera 2 at time t1 is considered to approximately match the camera height Hc.

  Therefore, by using the camera height Hc as the value of d, from the value of d (= Hc) and the value of the variable a or b, the above equation (13a) or equation (13b) can be used at time t1. The inclination angle θ of the object existence virtual road surface Sa with respect to the optical axis Lc of the in-vehicle camera 2 (in other words, the inclination angle of the straight line L2 with respect to the optical axis Lc in FIG. 2A) can be estimated.

  On the other hand, as can be seen with reference to FIG. 2A, the inclination angle θ of the object existence virtual road surface Sa with respect to the optical axis Lc of the in-vehicle camera 2 is θ = α + β. Accordingly, the angle β necessary for measuring the distance D of the object 53 from the host vehicle 1 based on the equation (2) is the inclination angle θ of the object existence virtual road surface Sa estimated as described above. From the angle α, it can be calculated by the following equation (14).

β = θ−α (14)

As described above, it is possible to estimate the value of the angle β using two captured images captured by the in-vehicle camera 2 at different times t1 and t2.

  Then, by using the estimated value of the angle β, the distance D between the target object 53 and the host vehicle 1 can be estimated based on the equation (4).

Based on the technical matters described above, the second invention will be described below.

According to the second aspect of the present invention , the distance from the own vehicle is the same as the ground point of the object at the imaging time t1 of the first captured image of the two captured images by the distance measurement feature point extracting means. The feature points obtained by projecting the still points on the road surface where the object exists to the two captured images are extracted. Then, the position of the feature point in each of the two captured images is specified. As a result, the position vectors ↑ P1 and ↑ P2 described above are specified.

  The feature point in the first captured image is, for example, a point formed by projecting the ground point of the object at the imaging time t1 of the first captured image onto the first captured image (FIG. 1B or FIG. 2). (B) In the example shown, it can be extracted as a point having local features such as luminance on a horizontal line passing through the point P53a). Then, the feature point in the other captured image (second captured image captured at imaging time t2) is extracted as a point having the same characteristics as the feature point of the first captured image in the second captured image. be able to. As such a feature point, for example, a Harris corner point, a minimum eigenvalue point, or the like can be used.

  In addition, the camera motion measuring means represents camera motion representing changes in the position and orientation of the in-vehicle camera in the period between the imaging times t1 and t2 of the two captured images (first captured image and second captured image). Parameters are measured. The camera motion parameters are parameters that can define the translation vector ↑ t and the rotation matrix R, and may be the translation vector ↑ t and the rotation matrix R itself.

  Note that the camera motion parameters such as the translation vector ↑ t and the rotation matrix R can be measured by a known image processing method based on, for example, Structure from Motion (hereinafter sometimes referred to as SfM).

  Further, the value of the third reference angle is estimated by the third reference angle estimation means. This third reference angle corresponds to the inclination angle θ of the object existence virtual road surface (Sa) with respect to the optical axis of the in-vehicle camera.

  Therefore, between the camera motion parameter, the position of the feature point in each of the two captured images, the height of the vehicle-mounted camera, and the third reference angle, the equations (11a) to (11c) are used. The relationship shown is approximately established. Then, based on an arithmetic expression representing the relationship, from the measured value of the camera motion parameter, the position of the feature point specified in each of the two captured images, and the set value of the height of the in-vehicle camera The value of the third reference angle (θ) can be estimated.

  More specifically, for example, by the camera motion measuring means, a translation vector ↑ t that is a displacement vector of the position of the in-vehicle camera in a period between the imaging times of the two captured images, and the in-vehicle camera When the rotation matrix R representing the angle change amount of the posture is measured as the camera motion parameter, based on the equations (11a) to (11c), the measured value of the camera motion parameter (of the translation vector ↑ t) The value of each component and the value of each component of the rotation matrix R), the position of the feature point specified in each of the two captured images (the value of each component of the position vectors ↑ P1, ↑ P2), and As described above, the value of the variable a or b can be obtained. Then, from the value of the variable a or b and the set value (Hc) of the in-vehicle camera height as the value of d, the estimated value of the third reference angle (θ) by the above formula (11a) or (11b). Can be requested.

In the second invention , the second distance estimating means includes the estimated value of the first reference angle (γ), the estimated value of the second reference angle (α), and the estimated value of the third reference angle (θ). And the second distance estimation value (D2) is determined based on the height setting value of the in-vehicle camera. In this case, since the value of the angle β can be estimated by the equation (14) from the estimated value of the second reference angle (α) and the estimated value of the third reference angle (θ), the value of β Using the estimated value of the first reference angle (γ) and the estimated value of the second reference angle (α), the second distance estimated value (D2) can be determined based on Equation (4).

According to the second invention , when the second distance estimated value (D2) is finally determined as the estimated value of the distance between the object and the host vehicle, that is, the second distance estimated value (D2) It is possible to accurately estimate the distance between the object and the host vehicle in a situation where reliability can be regarded as being ensured.

In the basic configuration or the first aspect of the invention , the vehicle further includes a gradient detection unit that detects whether the object existing road surface has a gradient with respect to the own vehicle existing road surface on which the vehicle exists. The distance estimated value determining means, when the detection result of the gradient detecting means is affirmative, the first distance estimated value is determined based on the determination result of the second distance estimated value determining means. You may make it fix as an estimated value of the distance between a target object and the own vehicle ( 3rd invention ).

According to the third aspect of the present invention, when the gradient detecting unit detects that the object existing road surface has a gradient with respect to the own vehicle existing road surface, the determination by the second distance estimated value determining unit is performed. Regardless of the result, the first distance estimated value is determined as the estimated value of the distance between the object and the host vehicle. For this reason, for example, a calculation process (for example, by the above equation (3)) on the premise that the second distance estimating means has no change or slight change in the road surface gradient, and the second distance estimated value (D2). In the situation where the reliability of the second distance estimated value (D2) is poor, the second distance estimated value (D2) is the distance between the object and the host vehicle. It is prevented that the estimated value is determined.

In the basic configuration or the first to third inventions , based on the object image extracted by the object extraction means, the object is obtained from the object standard size value in the type to which the object belongs. It further comprises non-standard object judging means for judging whether or not it is a non-standard object having a non-standard size value that diverges, and the distance estimation value determining means is that the object is a non-standard object If it is determined, the second distance estimated value is determined as an estimated value of the distance between the object and the host vehicle without depending on the determination result of the second distance estimated value determining means. ( 4th invention ).

According to the fourth aspect of the invention, when the non-standard object estimation unit determines that the target object is a non-standard object, regardless of the determination result of the second distance estimated value determination unit, The second distance estimated value is determined as an estimated value of the distance between the object and the host vehicle. This prevents the unreliable first distance estimated value (D1) from being determined as an estimated value of the distance between the object and the host vehicle for an object having a non-standard size value. Is done.

FIG. 1A is a diagram illustrating an example of a state of a vehicle and an object, and FIG. 1B is a diagram illustrating an outline of a captured image captured by a vehicle-mounted camera in the state of FIG. FIG. 2A is a diagram illustrating another example of the state of the vehicle and the object, and FIG. 2B is a diagram illustrating an outline of a captured image captured by the in-vehicle camera in the situation of FIG. The graph which shows the characteristic of the estimation error of the distance between a target object and a vehicle. 4A is a diagram showing an imaging state at different times t1 and t2 by the in-vehicle camera, and FIG. 4B is a diagram showing an imaging state at time t1 in FIG. 4A. 1 is a block diagram showing a configuration of a distance measuring device according to a first embodiment of the present invention. The block diagram which shows the process of the 2nd distance estimation part shown in FIG. FIGS. 7A and 7B are diagrams for explaining processing of the second reference angle A-estimation unit in FIG. 6. The block diagram which shows the process of the distance estimated value determination part in 2nd Embodiment of this invention. The block diagram which shows the process of the distance estimated value determination part in 3rd Embodiment of this invention.

[First Embodiment]
A first embodiment of the present invention will be described below with reference to FIGS.

  With reference to FIG. 5, the distance measuring device 10 of the present embodiment is mounted on the vehicle 1 shown in FIGS. 1A and 2A and is connected to the in-vehicle camera 2. The arithmetic processing unit 11 is provided. In addition, in FIG. 1A and FIG. 2A, the vehicle 1 is schematically illustrated, but the actual vehicle 1 is, for example, an automobile.

  In the present embodiment, the in-vehicle camera 2 is mounted on the front of the vehicle 1 to monitor the front of the vehicle 1 and captures an image in front of the vehicle 1. In this case, the in-vehicle camera 2 is attached to the vehicle body of the vehicle 1 such that its optical axis Lc is directed in the front-rear direction of the vehicle 1. However, the optical axis Lc of the in-vehicle camera 2 may have a slight inclination with respect to the longitudinal direction of the vehicle 1.

  The in-vehicle camera 2 is constituted by a CCD camera, a CMOS camera, or the like, and generates and outputs an image signal of a captured image. The captured image may be either a color image or a monotone image.

  The in-vehicle camera 2 may be a camera mounted on the vehicle 1 so as to capture a rear or side image of the vehicle 1.

  The arithmetic processing unit 11 is an electronic circuit unit including a CPU, a RAM, a ROM, an interface circuit, etc. (not shown), and receives an image signal generated by the in-vehicle camera 2.

  As a function realized by executing the implemented program, the arithmetic processing unit 11 measures the distance D from the host vehicle 1 from the image acquisition unit 12 that acquires a captured image of the in-vehicle camera 2 and the captured image. A target extraction unit 13 that extracts an image of the target object 53 to be extracted, and a first distance estimation method that estimates a distance D between each target object and the host vehicle 1 using the first distance measurement method and the second distance measurement method. The distance estimation part 14 and the 2nd distance estimation part 15 and the distance estimated value determination part 16 which fixes and outputs the estimated value of the distance of each target object and the own vehicle 1 are provided. The arithmetic processing unit 11 estimates the distance D between each object 53 present in front of the vehicle 1 and the host vehicle 1 by sequentially executing the processing of each of these functional units at a predetermined arithmetic processing cycle. The value is sequentially determined and output.

  The details of the overall processing of the arithmetic processing unit 11 including the processing of each functional unit of the arithmetic processing unit 11 will be described below.

  The in-vehicle camera 2 captures an image in front of the vehicle 1 at a timing instructed by the arithmetic processing unit 11 at a predetermined arithmetic processing cycle. And the image signal produced | generated by the imaging is taken in sequentially by the image acquisition part 12 of the arithmetic processing unit 11 from the vehicle-mounted camera 2. FIG. The image acquisition unit 12 converts an image signal (an image signal for each pixel) that is an analog signal input from the in-vehicle camera 2 into digital data, and stores and stores the digital signal in an image memory (not shown).

  In this case, the image acquisition unit 12 periodically updates a plurality of captured images, for example, a set of two captured images captured at different times (times at time intervals of a predetermined calculation processing cycle) by the in-vehicle camera 2. However, it is stored in the image memory. In the following description, the imaging times of two captured images stored and held in the image memory are referred to as times t1 and t2, the captured image at time t1 is referred to as a first captured image, and the captured image at time t2 is referred to as a second captured image. Sometimes. Note that the time t2 may be either the time after the time t1 or the previous time.

  The first captured image of the two captured images (first captured image and second captured image) stored and held in the image memory by the processing of the image acquisition unit 12 is given to the object extraction unit 13.

  This target object extraction unit 13 extracts image portions of a predetermined type of target object 53 from which the distance D to the host vehicle 1 is to be measured (estimated) from the first captured image, and is image information regarding each target object. The object image information is determined and output.

  In the present embodiment, the types of the object 53 to be measured are roughly classified into two types: a pedestrian (person) and another vehicle. Furthermore, pedestrians are classified into adult pedestrians and child pedestrians. Therefore, the object extraction unit 13 extracts the image portion of the adult pedestrian, the child pedestrian, and the other vehicle from the first captured image as the object 53 to be measured.

  In this case, the object extraction unit 13 searches the first captured image for an image portion having a shape pattern or the like that is characteristic of the pedestrian or the vehicle, so that the pedestrian image portion included in the first captured image. And image parts of other vehicles are extracted. For example, in the situation shown in FIG. 1 (a) or 2 (a), the portion indicated by reference numeral 53a in FIG. 1 (b) or 2 (b) is extracted as the image portion of the pedestrian.

  Note that various methods are known for searching for image portions of pedestrians and other vehicles from captured images, and a pedestrian or target object 53 from the first captured image using the known methods. What is necessary is just to extract another vehicle.

  When the extracted object 53 is a pedestrian, the object extraction unit 13 further specifies whether the pedestrian is an adult or a child. This specification is performed based on, for example, the ratio between the length in the vertical direction (vertical direction) and the length in the horizontal direction of the image portion of the pedestrian.

  Specifically, in general, the ratio of the width (full length in the horizontal direction) to the height (full length in the vertical direction) of a child is larger than that of an adult. Therefore, in the present embodiment, the object extraction unit 53 obtains a ratio of the length in the horizontal direction to the length in the vertical direction of the extracted image portion of the pedestrian, and the ratio is equal to or greater than a predetermined threshold value. Whether or not the pedestrian is a child. Thereby, the extracted pedestrian is distinguished into an adult and a child.

  Then, the object extraction unit 13 determines and outputs object image information for each object extracted from the first captured image. Specifically, in the present embodiment, the object extraction unit 13 includes attribute information indicating the type of the extracted object 53, the in-image object size that is the size of each object 53 in the first captured image, Position of the object grounding point on the image obtained by projecting the grounding point of each object 53 (the contact point between the object 53 and the object existence road surface 52) onto the first captured image (position in the first captured image) Are output as object image information.

  In this case, the object size in the image is a length in a direction determined in advance for each type of the object 53. Specifically, when the object 53 is a pedestrian (adult or child), the object size in the image of the object 53 is the vertical direction (vertical direction) of the object 53 in the first captured image. Is the full length (h53a shown in FIG. 1B or FIG. 2B).

  When the object 53 is another vehicle, the object size in the image of the object 53 is the total length of the object 53 in the horizontal direction (vehicle width direction) in the first captured image.

  Further, in the present embodiment, the object extraction unit 13 specifies the lowest point of the image of each object 53 in the first captured image as the image object ground point, and the object ground point on the image. The position (position in the first captured image) is output. For example, in the situation shown in FIG. 1 (a) or FIG. 2 (a), regarding the image 53a of the object 53 (pedestrian), the point P53a shown in FIG. The ground point is specified, and the position of the point P53a in the first captured image is output.

  Next, the arithmetic processing unit 11 executes the processing of the first distance estimation unit 14 and the second distance estimation unit 15.

  The object image information is input from the object extraction unit 13 to the first distance estimation unit 14. And the 1st distance estimation part 14 determines the 1st distance estimated value D1 based on the said 1st ranging method for every target object 53 using the input target object image information.

  In the processing of the first distance estimation unit 14, first, for each target object 53, a standard value of the size (length in a predetermined direction) of the target object to which the target object 53 belongs according to the type. Determine an object standard size value. Specifically, when the object 53 is an adult pedestrian, a value (known value) generally known as an average value of the height (full length in the vertical direction) of adults existing in the country is used. , The object standard size value corresponding to the object 53 is determined.

  Similarly, when the object 53 is a pedestrian of a child, a value (known value) generally known as an average value of the height of a child existing in the country corresponds to the object 53. Determined as the object standard size value.

  Further, when the object 53 is another vehicle, a value (known value) generally known as an average value of the vehicle width (total length in the lateral direction) of the vehicle existing in the country is the object. The object standard size value corresponding to 53 is determined.

  The object standard size value for each type of the object 53 (in this embodiment, the average value of the object size for each type) is stored and held in advance in the storage device of the arithmetic processing unit 11.

  Then, the first distance estimating unit 14 determines, for each object 53, the own vehicle 1 of the object 53 based on the ratio between the object standard size value determined as described above and the object size in the image. A first distance estimated value D1 is determined as an estimated value of the distance from.

  Specifically, the first distance estimation unit 14 substitutes the object standard size value and the object size in the image for H53s and h53a on the right side of the equation (1a) for each object 53. Then, the first distance estimated value D1 is calculated by calculating the right side.

  In this case, the value of the focal length F of the in-vehicle camera 2 is stored and held in advance in the storage device of the arithmetic processing unit 11.

  As described above, the first distance estimating unit 14 determines the estimated value (first distance estimated value D1) of the distance between each object 53 and the host vehicle 1 by the first distance measuring method.

  The second distance estimation unit 15 receives the first captured image and the second captured image from the image acquisition unit 12 and the target image information from the target extraction unit 13. And the 2nd distance estimation part 15 determines the 2nd distance estimated value D2 based on the said 2nd ranging method for every target object 53. FIG. In this case, in the present embodiment, the second distance estimation unit 15 estimates not only the angle γ but also the angles α and θ, and uses the estimated values of the angles γ, α, and θ, and the second distance. Estimate value D2 is determined.

  The second distance estimating unit 15 is a main functional unit for estimating the distance between each object 53 and the host vehicle 1 from the first captured image and the second captured image by the second ranging method. Ranging feature point extraction unit 17 for extracting feature points to be used, camera motion estimation unit 18 for estimating (measuring) the motion state of in-vehicle camera 2, and first reference for estimating first to third reference angles, respectively. The angle estimation unit 19, the second reference angle estimation unit 20, the third reference angle estimation unit 21, and the second distance estimation value calculation unit 22 that calculates the second distance estimation value D <b> 2 between each object 53 and the host vehicle 1. With. The first to third reference angles correspond to the angles γ, α, and θ, respectively.

  The distance-measuring feature point extraction unit 17 for each target, the first captured image and the second captured image, and the position of the on-image target ground point P53a in the target image information (in the first captured image). Position) and the stationary point P on the object existence road surface 52 where the distance from the host vehicle 1 is the same (or substantially the same) as the ground contact point P53 of the object 53. The points respectively projected on the image are searched from each captured image as the characteristic points for distance measurement of the object 53.

  In the present embodiment, the in-vehicle camera 2 is mounted on the vehicle 1 so that the lateral direction of the captured image (the x-axis direction of the camera coordinate system) is substantially parallel to the own vehicle existing road surface 51.

  For this reason, the point formed by projecting a stationary point on the object existence road surface 52 such that the distance from the own vehicle 1 is the same distance as the object ground contact point P53 is basically the following: As shown in FIG. 1B or FIG. 2B, it exists on a straight line L53a extending in the horizontal direction (x-axis direction of the camera coordinate system) through the object grounding point P53a on the image.

  Therefore, the distance measurement feature point extraction unit 17 first exists in the image 52a of the object existence road surface 52 on the straight line L53a in the image region in the vicinity of the image 53a of the object 53 in the first captured image. A feature point obtained by searching for a feature point to be detected and projecting the point on the first captured image as a stationary point on the object existence road surface 52 (a stationary point whose distance from the vehicle 1 is the same as the object grounding point P53). Extract as a point.

  Such a feature point generally shows a characteristic (distinguishable from other local regions) size or change in state quantities such as luminance, saturation, and hue in the local region including the point. It is such a point. For example, a Harris corner point, a minimum eigenvalue point, or the like is searched for as the feature point.

  Thereby, for example, the point P1 in FIG. 1B or 2B is extracted as a feature point in the first captured image corresponding to the object 53.

  After extracting the feature points in the first captured image in this way, the distance measurement feature point extracting unit 17 next searches the second captured image for a point having the same feature as the feature point of the first captured image, The searched point is extracted as a feature point obtained by projecting the stationary point onto the second captured image. The search process in the second captured image may be performed using a known image processing method such as block matching.

  A feature obtained by projecting a stationary point on the object existence road surface 52 whose distance from the vehicle 1 is the same as the object grounding point P53 to the first captured image by the above-described processing of the distance measurement feature point extraction unit 17. A point and a feature point obtained by projecting the still point on the second captured image are extracted. In this way, examples of feature points extracted from the first captured image and the second captured image are the feature points P1 and P2 illustrated in FIG.

  Then, the distance measurement feature point extraction unit 17 obtains and outputs the positions of the extracted feature points P1 and P2 in the captured images (the coordinate components of the position vectors ↑ P1 and ↑ P2).

  Supplementally, in this embodiment, since the object 53 (pedestrian or other vehicle) is a moving object, the object grounding point is obtained at the imaging time t1 of the first captured image and the imaging time t2 of the second captured image. P53 generally does not match. For this reason, in this embodiment, a point formed by projecting a stationary point different from the object ground point P53 on each captured image is extracted from each captured image as the feature point.

  However, the object 53 whose distance is to be measured may not be a moving body such as a pedestrian, but may be a fixed installation object (stationary object). In that case, you may make it extract the point formed by projecting the grounding point of the fixed installation object as a target object on each captured image from each captured image as the said feature point.

  The camera motion estimation unit 18 receives a first captured image and a second captured image. Then, in this embodiment, the camera motion estimation unit 18 has two given captured images (first captured image and second captured image), a frame rate indicating the number of captured images per unit time of the in-vehicle camera 2, and From the vehicle speed V of the vehicle 1 detected by a vehicle speed sensor (not shown), an estimated value (measurement value) of a spatial position displacement vector of the in-vehicle camera 2 using a known image processing method based on SfM (Structure from Motion). ) And the rotation matrix indicating the estimated value (measured value) of the angular change amount of the spatial orientation of the in-vehicle camera 2 in the period between the imaging times t1 and t2 of the two captured images. R is calculated as a camera motion parameter indicating the motion state of the in-vehicle camera 2 (the motion state at time t1).

  The translation vector ↑ t and the rotation matrix R are obtained from feature points in each captured image of a stationary object (fixed installation) image projected on the first captured image and the second captured image, for example. Known methods based on SfM for optical flow (eg “Structure from Motion without Correspondence” / Frank Dellaert, Steven M. Seitz, Charles E. Thorpe, Sebastian Thrun / Computer Sience Department & Robotics Institute Canegie Mellon University, Pittsborgh PA 15213 ).

  Supplementally, the translational movement vector calculated based on SfM has an indefinite scale, but the scale of the translational movement vector ↑ t is determined based on the frame rate and the detected vehicle speed.

  The second distance estimation unit 15 performs the processing of the distance measurement feature point calculation unit 17 and the camera motion estimation unit 18 as described above, and then performs the first reference angle estimation unit 19 and the second reference angle estimation unit 20. The processing of the third reference angle estimation unit 21 is executed. Note that the processing of the first reference angle estimation unit 19 may be executed in parallel with or before the processing of the camera motion estimation unit 18. Further, the process of the second reference angle estimation unit 20 may be executed in parallel with or before the process of the distance measurement feature point calculation unit 17.

The position of the feature point P1 in the first captured image (or the position vector ↑ P1) is input from the ranging feature point extraction unit 17 to the first reference angle estimation unit 19 for each object 53. The first reference angle estimator 19 then, for each object 53, moves in the vertical direction (Y-axis direction of the camera coordinate system) between the image center Pca of the first captured image and the input feature point P1. Is calculated (see FIG. 1B or FIG. 2B). The first reference angle estimator 19 then calculates, for each object 53, the equation (5) from the positional deviation Δy and the value of the focal length F of the vehicle-mounted camera 2 stored and stored in advance in the storage device. Based on this, an estimated value (= tan ⁻¹ (Δy / F)) of the angle γ as the first reference angle is calculated.

  The third reference angle estimator 21 includes the positions (or the position vectors) of the feature points P1, P2 for each object 53 in the first captured image and the second captured image from the distance measurement feature point extraction unit 17, respectively. ↑ P1, ↑ P2) are input, and the translation movement vector ↑ t and each component value of the rotation matrix R are input from the camera motion estimation unit 18.

  The third reference angle estimation unit 21 then determines the component values of the position vectors ↑ P1, ↑ P2 of the feature points P1, P2 for each object 53 indicated by the input values from the distance measurement feature point extraction unit 17. And each component value of the translation vector ↑ t and the rotation matrix R input from the camera motion estimation unit 18 and d of the equation (11c) (the object existing virtual road surface Sa and the vehicle-mounted camera 2 at the time t1 From the value of the camera height Hc stored and held in advance in a storage device (not shown) as an approximate set value of the distance to the optical center C), the above formulas (11a) to (11c) are obtained for each object 53. Based on the above, an estimated value of the angle θ as the third reference angle is calculated.

  Specifically, for example, the values of the coefficients r, s, and t of the quadratic equation of the equation (12) are defined in the proviso for the equations (11a) to (11c) and the proviso for the equation (12). Is calculated from the component values of ↑ P1, ↑ P2, ↑ t, R. Then, using the values of the coefficients r, s, and t, the value of the solution of the quadratic equation of Expression (12) is calculated as the value of the variable a (≡ (cos θ) / d).

  In this case, as described above, there are generally two values of a which are the solutions of the equation (12), but among these two values, the equations (11a) and (11b) are satisfied. The value of a to be obtained is selected.

  Then, the estimated value of the third reference angle θ is calculated from the value of the variable a and the set value (Hc) of d by the equation (13a).

  Instead of the variable a, the value of the variable b (≡ (sin θ) / d) is calculated, and the value of the variable b and the set value (Hc) of d are calculated by the equation (13b). 3 An estimated value of the reference angle θ may be calculated.

  The second reference angle estimation unit 20 receives the translation movement vector ↑ t and the component values of the rotation matrix R from the camera motion estimation unit 18.

  The second reference angle estimator 20 executes a process for calculating the A-th provisional estimated value αa and the B-th tentative estimated value αb of the angle α as the second reference angle, respectively. 20a and a second reference angle B-th provisional estimation unit 20b.

  The second reference angle A provisional estimation unit 20a determines the second reference angle α of the second reference angle α based on the direction of the input translational movement vector ↑ t (direction relative to the optical axis Lc of the vehicle-mounted camera 2 at the imaging time t1). A provisional estimated value αa is calculated.

  In this case, the translation vector ↑ t can be regarded as a vector that is substantially parallel (substantially parallel to the traveling direction of the vehicle 1) to the own vehicle existing road surface 51 where the vehicle 1 is traveling at the time t1.

  Therefore, the direction of the translation vector ↑ t with respect to the optical axis Lc of the in-vehicle camera 2 (or the direction of the optical axis Lc with respect to the translation vector ↑ t) is such that the optical axis Lc of the in-vehicle camera 2 is relative to the own vehicle existing road surface 51. The angle formed depends on the second reference angle α.

  That is, as shown in FIGS. 7A and 7B, the translational movement vector ↑ t seen in the camera coordinate system of the vehicle-mounted camera 2 at time t1 is relative to the optical axis Lc (Z-axis) of the vehicle-mounted camera 2. The formed angle αa corresponds to the second reference angle α.

  Therefore, in the present embodiment, the second reference angle A-provisional estimation unit 20a determines the angle αa that the input translational movement vector ↑ t makes with the XZ plane of the camera coordinate system of the vehicle-mounted camera 2 at time t1. Calculated as a provisional estimated value of the second reference angle α.

Specifically, the second reference angle A provisional estimation unit 20a calculates the component value of the translation vector ↑ t (= [t1, t2, t3] ^T ) in the camera coordinate system of the vehicle-mounted camera 2 at time t1. The A-th provisional estimated value αa is calculated by the following equation (15).

  Further, the second reference angle B provisional estimation unit 20b calculates a B provisional estimated value αb of the second reference angle α based on the input rotation matrix R.

  Specifically, the second reference angle B provisional estimator 20b determines the amount of change in the angle of the spatial orientation of the in-vehicle camera 2 indicated by the input rotation matrix R (the imaging times t1 and t2 of the two captured images). Of the change in angle in the pitch direction of the vehicle 1 (the direction around the axis where the change in the second reference angle α occurs) is sequentially accumulated (accumulated addition) in the calculation processing cycle of the calculation processing unit 11. As a result, the B-th provisional estimated value αb of the second reference angle α is calculated.

  The B-th provisional estimated value αb is initialized when the vehicle 1 starts to operate. In this case, as the initial value of the B-th provisional estimated value αb, for example, a value (for example, “0”) determined in advance as the value of the second reference angle α when the vehicle 1 is stopped on a horizontal road surface. ) Or the value of the second reference angle α finally calculated during the previous driving of the vehicle 1 is set.

  Through the processing of the second reference angle A estimation unit 20a and the second reference angle B estimation unit 20b described above, the A provisional estimated value αa of the second reference angle α based on the translation vector ↑ t and the rotation matrix R And a B-th provisional estimated value αb of the second reference angle α based on is sequentially calculated at a predetermined calculation processing cycle.

  Here, since the A-th provisional estimated value αa based on the translational movement vector ↑ t is directly affected by the instantaneous fluctuation of the posture of the vehicle body of the vehicle 1, it is instantaneous with respect to the actual second reference angle. Although a deviation is likely to occur, a steady offset with respect to the actual second reference angle α generally does not occur.

  On the other hand, the waveform of the B-th provisional estimated value αb based on the rotation matrix R matches the waveform of the actual second reference angle α with relatively high accuracy, but has a steady offset with respect to the actual second reference angle α. Prone to occur. This offset is caused by an error in the initial value of αb or an accumulation of instantaneous errors.

  Therefore, in the present embodiment, based on the above-described properties of the A-th provisional estimated value αa and the B-th provisional estimated value αb, the second reference angle estimation unit 20 estimates the second reference angle α as described below. Are sequentially determined.

  That is, the second reference angle estimation unit 20 stores and holds the A-th provisional estimated value αa and the B-th provisional estimated value αb calculated in a period from the current time to a predetermined time before in time series. In this case, the stored A-th provisional estimated value αa and B-th provisional estimated value αb are updated each time new (latest) αa and αb are calculated.

  Then, the second reference angle estimation unit 20 firstly stores the latest data for a predetermined time interval (from the current time to a past time before the predetermined time interval) among the stored time-series data of αa and αb. Data within the period of time) is extracted, and the average value of the extracted data of αa and αb is calculated. That is, the second reference angle estimation unit 20 calculates the average value αa_ave of αa and the average value αb_ave of αb within a period from the current time to a past time before a predetermined time interval. The predetermined time interval may be a fixed time, but in the present embodiment, for example, it is set according to the vehicle speed of the vehicle 1 (the higher the vehicle speed, the shorter the predetermined time interval is set).

  Next, the second reference angle estimation unit 20 calculates a deviation (= αa_ave−αb_ave) between the average value αa_ave of the A-th provisional estimated value αa and the average value αb_ave of the B-th provisional estimated value αb. The second reference angle estimation unit 20 corrects the estimated value of the second reference angle α at the current time by correcting the B-th provisional estimated value αb (the latest value of αb) at the current time based on this deviation. decide. Specifically, the second reference angle estimation unit 20 determines the estimated value of the second reference angle α at the current time by adding the deviation to the B-th provisional estimated value αb at the current time.

  Through the above-described processing of the second reference angle estimation unit 20, the estimated value of the second reference angle α is sequentially determined. The estimated value of the second reference angle α determined in this way is an estimated value common to each object 53. The A-th provisional estimation is obtained by calculating the deviation between αa and αb at each time in the period from the current time to the past time before the predetermined time interval, and obtaining the average value of the deviation within the period. The deviation (= αa_ave−αb_ave) between the average value αa_ave of the values αa and the average value αb_ave of the B-th provisional estimated value αb may be calculated.

  A second Kalman filter is used to calculate the second reference angle α based on the A-th provisional estimated value and the non-cumulative B-thick estimated value (that is, the angle change amount per unit time of the attitude of the in-vehicle camera 2). The reference angle α may be estimated.

  The estimated value of the first reference angle γ, the estimated value of the second reference angle α, and the estimated value of the third reference angle θ obtained by the first to third reference angle estimators 19 to 21 as described above are Input to the 2-distance estimated value calculation unit 22. Then, for each object 53, the second distance estimated value calculation unit 22 calculates a second distance estimated value D2 as an estimated value of the distance between the object 53 and the host vehicle 1 from these input values. To do.

  Specifically, the second distance estimated value calculation unit 22 subtracts the estimated value of the second reference angle α from the estimated value of the third reference angle θ (according to the equation (14)), thereby obtaining FIG. ), That is, an estimated value of the angle β formed by the straight line L2 (or the object existence virtual road surface Sa) in the drawing with respect to the own vehicle existence road surface 51 is calculated. Then, the second distance estimated value calculation unit 22 calculates the above equation from the estimated value of the angle β, the estimated value of the first reference angle γ, the estimated value of the second reference angle α, and the value of the camera height Hc. By (4), the second distance estimated value D2 based on the second distance measuring method is calculated.

  As described above, after the processing of the first distance estimation unit 14 and the second distance estimation unit 15 is executed, the arithmetic processing unit 11 next executes the processing of the distance estimated value determination unit 16.

  The distance estimated value determination unit 16 receives the first distance estimated value D1 and the second distance estimated value D2 for each object 53 from the first distance estimating unit 14 and the second distance estimating unit 15, respectively. Further, a detected value of the vehicle speed V of the host vehicle 1 is input from a vehicle speed sensor (not shown).

  Then, as shown in FIG. 5, the distance estimated value determination unit 16 first executes the determination process of S16-1. In S16-1, it is determined whether or not the input detection value (current detection value) of the vehicle speed V is greater than a predetermined threshold value VL. When this determination result is negative, that is, when the detected value of the vehicle speed V is a low vehicle speed (including a stop state) equal to or less than the threshold value VL, the distance estimated value determination unit 16 determines each object 53. The first estimated distance value D1 input with respect to is determined and output as an estimated value of the distance D between the object 53 and the vehicle 1 (estimated value of the distance D at time t1).

  On the other hand, when the detected value of the vehicle speed V is larger than the predetermined threshold VL, the distance estimated value determination unit 16 next executes the determination process of S16-2. In S <b> 16-2, the distance estimated value determination unit 16 inputs an allowable range (a set of a lower limit value and an upper limit value) of the second distance estimated value D <b> 2 for each target object 53. Determine according to D1. Then, the distance estimated value determination unit 16 determines, for each object 53, whether or not the input second distance estimated value D2 is within the allowable range.

  In this case, the distance estimated value determination unit 16 determines the lower limit value and the upper limit value of the allowable range of the second distance estimated value D2 by the following expressions (16a) and (16b), respectively.

Lower limit value of allowable range of D2 = D1 / (1 + Emax) (16a)
Upper limit value of allowable range of D2 = D1 / (1 + Emin) (16b)

Here, Emax and Emin are the maximum value [%] (> 0) and the minimum value of the error range (error range for each type of the object 53) of the first distance estimated value D1 given by the equation (2), respectively. This is a value obtained by dividing [%] (<0) by “100”.

  Therefore, for each object 53, Emax is the object standard size H53s corresponding to the type, and the minimum value Hmin (<H53s) of the object size generally known for the type of object. From this, the value is calculated by the following equation (17a). Further, Emin is determined for each object 53 from the object standard size H53s corresponding to the type and the maximum value Hmax (> H53s) of the object size generally known for the type of object. Is a value calculated by the following equation (17b).

Emax = (H53s / Hmin) -1 (17a)
Emin = (H53s / Hmax) -1 (17b)

When the object 53 is a pedestrian (adult or child), H53s, Hmin, and Hmax are values related to height (the total length in the vertical direction). When the object 53 is another vehicle, H53s, Hmin, and Hmax are values related to the vehicle width (the total length in the lateral direction). Further, the values of Hmin and Hmax are stored and held in advance in the storage device of the arithmetic processing unit 11 for each type of the object 53 (for each adult pedestrian, child pedestrian, and other vehicle).

  Supplementally, the lower limit value and the upper limit value of the allowable range of the second distance estimated value D2 may be set by the following equations (18a) and (18b), respectively. The allowable range set by these equations (18a) and (18b) is the same as the allowable range set by equations (17a) and (17b).

Lower limit value of allowable range of D2 = D1 · (Hmin / H53s) = F · (Hmin / h53a)
...... (18a)
Upper limit value of allowable range of D2 = D1. (Hmax / H53s) = F. (Hmax / h53a)
...... (18b)

The allowable range set as described above includes the object size h53a in the image of each object 53 and Hmax as the set value (maximum size set value) of the maximum value of the object size in the type to which the object 53 belongs. The value of the distance D estimated based on the ratio (the value of D calculated by replacing H53s in the equation (1a) with Hmax), the object size h53a in the image of the object 53, and the object The value of the distance D estimated based on the ratio to the minimum value setting value (minimum size setting value) of the object size in the type to which 53 belongs (the expression in which H53s in the expression (1a) is replaced with Hmin. The value of D calculated by (1).

  Therefore, this allowable range corresponds to the error range of the first distance estimated value D1 caused by the difference between the actual object size H53 of the object 53 and the object standard size value H53s.

  Then, the distance estimated value determination unit 16 determines that the input second distance estimated value D2 is within the allowable range set as described above for each object 53 (the determination result in S16-2 is positive). The second distance estimated value D2 of the object 53 is determined as an estimated value of the distance D between the object 53 and the host vehicle 1 (estimated value of the distance D at time t1). Output.

  In addition, the distance estimated value determination unit 16 determines that the input second distance estimated value D2 deviates from the allowable range set as described above for each object 53 (the determination result in S16-2 is negative). The first distance estimated value D1 of the object 53 is determined as an estimated value of the distance D between the object 53 and the host vehicle 1 (estimated value of the distance D at time t1). And output.

  Therefore, in the process of the distance estimated value determination unit 16, when the detected value of the vehicle speed V is a low vehicle speed equal to or lower than the predetermined threshold VL, the first distance estimated value of each object 53 for any target object 53. D1 is determined as the estimated value of the distance D. In addition, when the detected value of the vehicle speed V is larger than the predetermined threshold VL, for each object 53, when the determination result of S16-2 is affirmative, the second distance estimated value D2 is the object. The estimated distance D is 53. When the determination result in S16-2 is negative, the first distance estimated value D1 is determined as the estimated value of the distance D of the object 53.

  The above is the detail of the process which the arithmetic processing unit 11 in this embodiment performs. By this processing, when there is an object 53 (in this embodiment, a pedestrian or another vehicle) whose distance to the host vehicle 1 is to be measured on the road surface in front of the vehicle 1, each object 53 and The distance D between the host vehicle 1 and the host vehicle 1 is estimated sequentially.

  Here, a supplementary description will be given of the correspondence between the present embodiment and the present invention. In the present embodiment, the object extracting unit 13, the first distance estimating unit 14, the second distance estimating unit 15 (or the second distance estimated value calculating unit 22), and the distance estimated value determining unit 16 are respectively used in the present invention. An object extracting means, a first distance estimating means, a second distance estimating means, and a distance estimated value determining means are realized.

  Further, the distance measurement feature point extraction unit 17, the camera motion estimation unit 18, the first reference angle estimation unit 19, the second reference angle estimation unit 20, and the third reference angle estimation unit 21 respectively perform distance measurement in the present invention. A feature point extraction unit, a first reference angle estimation unit, a second reference angle estimation unit, and a third reference angle estimation unit are realized.

  Furthermore, the second distance estimated value determination means in the present invention is realized by the determination processing in S16-2 of the distance estimated value determination unit 16.

  According to the embodiment described above, when the detected value of the vehicle speed V is not a low vehicle speed equal to or lower than the predetermined threshold VL, the second distance estimated value D2 based on the second distance measuring method is expressed by the equations (16a) and (16b). ) (Or within the allowable range set by the equations (18a) and (18b)), the second distance estimated value D2 is used as the estimated value of the distance D of each object 53 from the host vehicle 1. Is determined.

  Here, the allowable range corresponds to an error range of the first distance estimation value D1 based on the first distance measurement method for each object 53, and the second reference angle α and the third reference angle θ are Not affected. For this reason, the actual distance D (true value) of each object 53 from the host vehicle 1 falls within the allowable range with sufficiently high accuracy.

  Therefore, when the second distance estimated value D2 is within the allowable range, the error of the estimated values of the second reference angle α and the third reference angle θ is sufficiently small, and as a result, the second distance measuring method. It can be considered that the reliability of the second estimated distance value D2 calculated based on the above is sufficiently secured. In this case, since the second estimated distance value D2 is not affected by the actual object size, the error is considered to be sufficiently small.

  For this reason, when the second distance estimated value D2 is within the allowable range, the second distance estimated value D2 is determined as the estimated value of the distance D of the object 53 from the host vehicle 1. A reliable distance estimation value of the object 53 can be obtained.

  Further, when the second distance estimation value D2 based on the second distance measurement method deviates from the allowable range set by the equations (16a) and (16b) (or the equations (18a) and (18b)), That is, when the reliability of D2 is low, the first distance estimated value D1 based on the first distance measuring method is determined as the estimated value of the distance D of the object 53 from the host vehicle 1. Thereby, it is prevented that a value greatly deviating from the actual distance value is determined as the estimated value of the distance D of the target unit 53 from the host vehicle 1.

  Further, when the detected value of the vehicle speed V is a low vehicle speed equal to or lower than a predetermined threshold VL, the first distance estimation based on the first distance measuring method is used as the estimated value of the distance D of each object 53 from the host vehicle 1. The value D1 is determined.

  Here, in the present embodiment, the camera motion estimation unit 18 translates the translational motion vector indicating the motion state of the in-vehicle camera 2 based on the first captured image and the second captured image captured at different imaging times t1 and t2. ↑ t and rotation matrix R are calculated. Then, the second reference angle α and the third reference angle θ are estimated using the translation vector ↑ t and the rotation matrix R. Further, the second distance estimated value D2 is calculated using the second reference angle α and the third reference angle θ.

  On the other hand, when the vehicle speed V is low, the difference between the first captured image and the second captured image is unlikely to occur. Therefore, the reliability of the translation vector ↑ t and the rotation matrix R estimated using the captured images is high. It tends to decrease. As a result, the reliability of the second distance estimated value D2 is likely to decrease.

  For this reason, in this embodiment, when the detected value of the vehicle speed V is a low vehicle speed equal to or lower than the predetermined threshold VL, the distance from each vehicle 53 to the host vehicle 1 regardless of the determination result of S16-2. As an estimated value of D, a first distance estimated value D1 based on the first distance measuring method is determined. This prevents the second distance estimated value D2 from being determined as the estimated value of the distance D of the object 53 from the host vehicle 1 in a situation where the reliability of the second distance estimated value D2 is low. it can.

  Furthermore, in this embodiment, in addition to the estimated value of the first reference angle γ, the second reference angle α and the third reference angle θ are estimated, and the estimated values of these angles γ, α, and θ are used to estimate the first reference angle γ. A second distance estimation value D2 based on the two distance measurement method is determined.

  For this reason, in a situation where the second distance estimated value D2 is adopted as an estimated value of the distance D of the object 53 from the host vehicle 1 (a situation where the reliability of D2 is considered to be high), the road surface on which the vehicle 1 travels. The second distance estimated value D2 with high accuracy (small error) obtained by appropriately compensating for the influence of the change in the gradient of the vehicle and the change in the posture of the vehicle 1 (particularly, the change in the posture in the pitch direction) It can be determined as an estimated value of the distance D from 53 of the own vehicle 1.

  Further, in the process of estimating the second reference angle α used for determining the second distance estimated value D2, the B-th provisional estimated value αb calculated at a predetermined calculation processing cycle is set to a predetermined time interval from the current time. By correcting the deviation (= αa_ave−αb_ave) between the average value αa_ave of the A-th provisional estimated value αa and the average value αb_ave of the B-th provisional estimated value αb within the period up to the previous past time, The estimated value of the second reference angle α is sequentially determined.

  Thereby, the precision of the estimated value of the second reference angle α can be increased.

  Further, the predetermined time interval that defines the sampling periods of αa and αb used for obtaining the deviation (= αa_ave−αb_ave) is set to a shorter time as the vehicle speed increases. For this reason, the period of the predetermined time interval is set to a period in which a large variation is unlikely to occur in the posture of the vehicle 1 (particularly the posture in the pitch direction) within the period.

  As a result, the deviation (= αa_ave−αb_ave) is calculated using the average values αa_ave and αb_ave with high reliability (high S / N ratio) of the A-th provisional estimated value αa and the B-th provisional estimated value αb. be able to. For this reason, the precision of the estimated value of the second reference angle α can be effectively increased. As a result, the precision of the 2nd distance estimated value D2 based on a 2nd ranging method can be improved.

  As described above, according to the present embodiment, in a situation where the reliability of the second distance estimated value D2 based on the second distance measuring method is considered to be high, the second distance estimated value D2 is set as the object 53. Since the estimated value of the distance D to the host vehicle 1 is determined, the distance D can be estimated with high accuracy. Further, in a situation where the reliability of the second distance estimation value D2 based on the second distance measurement method is considered to be low, the first distance estimation value D1 based on the first distance measurement method is calculated as the distance between the object 53 and the host vehicle 1. Therefore, the estimated value of the distance D can be prevented from having an excessive error with respect to the actual distance D.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIG. This embodiment is different from the first embodiment only in a part of the processing of the arithmetic processing unit 11. For this reason, in the description of the present embodiment, the differences will be mainly described, and the description of the same matters as in the first embodiment will be omitted.

  In the first embodiment, the first distance estimated value calculator 22 of the second distance estimator 15 calculates the second distance estimated value D2 using the equation (4). In contrast, in the present embodiment, the first distance estimated value calculation unit 22 calculates the first reference angle γ and the second reference angle α from the estimated value of the camera height Hc and the equation (3). A second distance estimated value D2 is calculated. Then, the second distance estimation unit 15 outputs the second distance estimation value D2 to the distance estimation value determination unit 16, and calculates the second reference angle α and the third reference angle θ from the estimated values of the equation (14). The calculated estimated value of the angle β is output to the distance estimated value determination unit 16.

  Furthermore, in this embodiment, the distance estimated value determination unit 16 executes the process shown in FIG. 8 for each object 53, thereby estimating the distance D between each object 53 and the host vehicle 1. Confirm and output.

  Specifically, the distance estimated value determination unit 16 first executes the determination process of S16-1. The determination process of S16-1 is the same as the determination process of S16-1 in the first embodiment, and it is determined whether or not the detected value of the vehicle speed V is larger than a predetermined threshold value VL. When the determination result is negative, the first distance estimated value D1 is determined as an estimated value of the distance D between the object 53 and the host vehicle 1.

  If the determination result in S16-1 is affirmative, the distance estimated value determination unit 16 next executes the determination process in S16-3. In S16-3, the distance estimated value determining unit 16 determines whether or not the estimated value of the angle β input from the second distance estimating unit 15 is larger than a predetermined threshold value βL. The situation where the determination result is affirmative is a situation where the angle β is large to some extent, that is, a situation where the object existing road surface 52 has a gradient with respect to the own vehicle existing road surface 51.

  Here, in the present embodiment, since the second distance estimated value D2 is calculated by the equation (3), if there is a change in the slope of the road surface, the second distance estimated value D2 is converted to the subject vehicle 1 of the object 53. An error occurs with respect to the actual distance D from

  Therefore, in the present embodiment, when the determination result in S16-3 is affirmative, the first distance estimated value D1 is determined as the estimated value of the distance D between the object 53 and the host vehicle 1.

  On the other hand, the situation in which the determination result of S16-3 is negative is a situation where the angle β is “0” or close to it, that is, a situation where the object existence road surface 52 has almost no gradient with respect to the own vehicle existence road surface 51. It is. In this situation, as long as the second reference angle α is appropriately estimated, the reliability of the second distance estimated value D2 is high.

  Therefore, when the determination result in S16-3 is negative, the distance estimated value determination unit 16 next executes the determination process in S16-2. The determination process of S16-2 is the same as the determination process of S16-2 in the first embodiment, and it is determined whether or not the second distance estimated value D2 is within the allowable range. When the determination result is negative, the first distance estimated value D1 is determined as an estimated value of the distance D between the object 53 and the host vehicle 1. When the determination result is affirmative, the second distance estimated value D2 is determined as an estimated value of the distance D between the object 53 and the host vehicle 1.

  The present embodiment is the same as the first embodiment except for the matters described above.

  Also in this embodiment, in a situation where the reliability of the second distance estimated value D2 based on the second distance measuring method is considered to be high, the second distance estimated value D2 is set between the object 53 and the host vehicle 1. Therefore, the distance D can be estimated with high accuracy.

  In a situation where the reliability of the second distance estimation value D2 based on the second ranging method is considered to be low, including the situation where the object existence road surface 52 has a gradient with respect to the own vehicle existence road surface 51, the first Since the first distance estimated value D1 based on the distance measuring method is determined as the estimated value of the distance D between the object 53 and the host vehicle 1, the estimated value of the distance D is excessive with respect to the actual distance D. Can be prevented.

  Supplementally, in the present embodiment, the gradient detection means in the present invention is realized by the processing of S16-3.

[Third Embodiment]
Next, a third embodiment of the present invention will be described with reference to FIG. This embodiment is different from the first embodiment only in a part of the processing of the arithmetic processing unit 11. For this reason, in the description of the present embodiment, the differences will be mainly described, and the description of the same matters as in the first embodiment will be omitted.

  In the first embodiment, the pedestrian as the object 53 is classified into an adult pedestrian and a child pedestrian, and the distance D between the object 53 and the vehicle 1 is estimated for each. I made it. On the other hand, in this embodiment, when a pedestrian is extracted as the target object 53 by the target object extraction unit 13, the first distance estimation unit 14 assumes that the pedestrian is an adult pedestrian. Execute the process. That is, in the present embodiment, when the object 53 is a pedestrian, the first distance estimation unit 14 uses the object standard size H53s (an average value of the height of an adult pedestrian) related to an adult pedestrian. The first estimated distance value D1 is calculated from the equation (1a).

  In the present embodiment, the distance estimated value determination unit 16 includes object image information for each object 53 in addition to the first distance estimated value D1 and the second distance estimated value D2 for each object 53. It is given from the object extraction unit 13. In this embodiment, the object image information includes the length in the vertical direction and the length in the horizontal direction of the image of each object 53 in the first captured image.

  Furthermore, in this embodiment, the distance estimated value determination unit 16 performs the process shown in FIG. 9 for each object 53, thereby estimating the distance D between each object 53 and the host vehicle 1. Confirm and output.

  Specifically, the distance estimated value determination unit 16 first executes the determination process of S16-1. The determination process of S16-1 is the same as the determination process of S16-1 in the first embodiment, and it is determined whether or not the detected value of the vehicle speed V is larger than a predetermined threshold value VL. When the determination result is negative, the first distance estimated value D1 is determined as an estimated value of the distance D between the object 53 and the host vehicle 1.

  If the determination result in S16-1 is negative, the distance estimated value determination unit 16 next executes the determination process in S16-4. In S16-4, the distance estimated value determination unit 16 determines that the object 53 has a large deviation from the object standard size value based on the object image information given from the object extraction unit 13. It is determined whether or not the object is out of specification with a high possibility of being a product.

  In this case, this determination is made by comparing the ratio between the length in the vertical direction and the length in the horizontal direction of the image of the object 53 in the first captured image with a predetermined value determined in advance for each type of the object 53. . For example, when the object 53 is a pedestrian, the ratio of the length in the horizontal direction to the length in the vertical direction of the image is higher than that in the case where the pedestrian is a general adult. If you are a child, you will be bigger. Therefore, in the present embodiment, when the object 53 is a pedestrian, if the ratio of the horizontal length to the vertical length of the image is greater than a predetermined value, It is determined that the object 53 is a non-standard object.

  Further, when the object 53 is another vehicle, for example, when the ratio of the horizontal length to the vertical length of the image deviates from a predetermined range, the target It is determined that the object 53 is a non-standard object such as a special vehicle having an object size that deviates from a general object size.

  Then, when the determination result of S16-4 is affirmative, that is, when the reliability of the first distance estimated value D1 is low, the distance estimated value determination unit 16 determines the second distance estimated value D2 as the object. The estimated value of the distance D between the vehicle 53 and the host vehicle 1 is determined.

  On the other hand, when the determination result of S16-4 is negative, the distance estimated value determination unit 16 next executes the determination process of S16-2. The determination process of S16-2 is the same as the determination process of S16-2 in the first embodiment, and it is determined whether or not the second distance estimated value D2 is within the allowable range. When the determination result is negative, the first distance estimated value D1 is determined as an estimated value of the distance D between the object 53 and the host vehicle 1. When the determination result is affirmative, the second distance estimated value D2 is determined as an estimated value of the distance D between the object 53 and the host vehicle 1.

  The present embodiment is the same as the first embodiment except for the matters described above. Note that in S16-2, not only the ratio between the length in the vertical direction and the length in the horizontal direction of the image of the object 53, but based on, for example, the shape characteristics of the entire image of the object 53 or a local portion. It may be determined whether or not the object 53 is a non-standard object.

  In this embodiment, in a situation where the reliability of the first distance estimated value D1 based on the first distance measuring method is considered to be low, the first distance estimated value D1 is set between the object 53 and the host vehicle 1. Therefore, it is possible to prevent the estimated value from being excessively deviated from the actual value.

  Supplementally, in the present embodiment, the non-standard object judging means in the present invention is realized by the processing of S16-4.

  In each of the embodiments described above, the second reference angle α is sequentially estimated. However, in a situation where the vehicle 1 is traveling on a flat road surface, the value of the second reference angle α is , Maintained almost constant. The value of the second reference angle α is a known value defined by the mounting posture of the in-vehicle camera 2 with respect to the vehicle body. Therefore, in such a situation, even if the second reference angle α is not sequentially estimated by the processing of the second distance estimating unit 15, the value of α is set as a predetermined default value, and the equation (3) or ( The second distance estimated value D2 may be determined according to 4).

  In each of the above-described embodiments, the case where the object 53 to be measured is a pedestrian or another vehicle has been described as an example, but the object 53 may be an animal other than a pedestrian. Alternatively, the object 53 may be a fixed ground object (stationary object). In the case where the object 53 is a fixed ground object, in the process of the distance measurement feature point extraction unit 17, as described above, a point formed by projecting the ground point of the object 53 onto each captured image is the feature point P1. , P2 may be used.

  In the above embodiments, the camera motion parameters (specifically, the translation vector ↑ t and the rotation matrix R) representing the motion state of the in-vehicle camera 2 are estimated using the captured image. When an attitude sensor such as a gyro sensor for detecting the attitude of the vehicle or an acceleration sensor for detecting the acceleration of the vehicle body is mounted on the vehicle 1, for example, based on the output of these sensors, the camera motion parameters are set. You may make it measure.

  In the first embodiment, the second reference angle α and the third reference angle θ are estimated, and the second distance estimated value D2 is determined using the estimated values. However, the second reference angle α, Alternatively, the second distance estimated value D2 may be calculated by the above formula (3) or formula (4) with the value of the third reference angle θ set as a predetermined value.

  In the third embodiment, the second reference angle α and the third reference angle θ determined by the second reference angle estimator 20 and the third reference angle estimator 21, respectively, are calculated by the equation (14). Based on the angle β, in S16-4, it is detected whether the object existing road surface 52 has a gradient with respect to the own vehicle existing road surface 51. For example, a laser radar is mounted on the vehicle 1. In such a case, the presence or absence of a gradient may be detected based on the received intensity of the reflected wave of the laser radar.

  In the third embodiment, the second distance estimated value D2 may be determined by the equation (3). In that case, in the process of the distance estimated value determination unit 16, the determination process of S16-3 in the second embodiment is added, and if the determination result is affirmative, the process is the same as in the second embodiment. The 1-distance estimated value D1 may be determined as an estimated value of the distance D between the object 53 and the host vehicle 1.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 2 ... Car-mounted camera, 10 ... Distance measuring device, 13 ... Object extraction part (object extraction means), 14 ... 1st distance estimation part (1st distance estimation means), 15 ... 2nd distance estimation part (Second distance estimation means), 16 ... distance estimation value determination section (distance estimation value determination means), 17 ... distance measurement feature point extraction section (range measurement feature point extraction means), 18 ... camera motion estimation section (camera Motion measurement means), 19 ... first reference angle estimation unit (first reference angle estimation means), 20 ... second reference angle estimation unit (second reference angle estimation means), 21 ... third reference angle estimation unit (third Reference angle estimating means), S16-2 ... second distance estimated value judging means, S16-3 ... gradient detecting means, S16-4 ... non-standard object judging means.

Claims (4)

A distance measuring device that measures a distance between an object on a road surface imaged by an in-vehicle camera and the vehicle on which the in-vehicle camera is mounted,
An object extraction means for extracting an image of an object to be measured from the captured image of the in-vehicle camera, and the distance from the host vehicle;
Based on the image of the object extracted from the captured image, the object size in the image, which is the size of the object in the captured image, is specified, and the specified object size in the image and the in-vehicle camera From the ratio of the object standard size value set in advance as the standard value of the object size, which is the size of the object in real space seen in the optical axis direction, and the setting value of the focal length of the in-vehicle camera First distance estimating means for determining a first distance estimated value that is a first estimated value of the distance between the object and the host vehicle at the imaging time t1 of the captured image;
A stationary point on the object existing road surface on which the object exists is a stationary point whose distance from the host vehicle is the same as the ground contact point of the object at the imaging time t1. A feature point extraction means for distance measurement that extracts from the captured image a feature point formed by projecting on the captured image captured in step, and specifies a position of the feature point in the captured image;
Based on the position of the feature point specified in the captured image, a first reference angle value that is an angle formed by a straight line from the in-vehicle camera to the stationary point with respect to the optical axis of the in-vehicle camera is estimated. 1 reference angle estimation means;
Based on at least the estimated value of the first reference angle and the set value of the height of the in-vehicle camera, a second estimated distance value that is a second estimated value of the distance between the object and the host vehicle is determined. A second distance estimating means;
The distance value estimated based on the ratio between the specified object size in the image and the maximum size setting value set in advance as the maximum value of the object size, and the specified object size in the image And a range between the distance value estimated based on a ratio of a minimum size setting value set in advance as a minimum value of the target object size as an allowable range of the second distance estimation value, Second distance estimated value judging means for judging whether or not the second distance estimated value is within the set allowable range;
If the determination result of the second distance estimated value determining means is affirmative, the second distance estimated value is determined as an estimated value of the distance between the object and the host vehicle, and the determination result is negative A distance estimated value determining means for determining the first distance estimated value as an estimated value of the distance between the object and the host vehicle ,
Camera motion measurement means for sequentially measuring the motion state of the in-vehicle camera including at least the translational movement direction of the in-vehicle camera based on two or more captured images captured at different imaging times;
Based on the measured motion state of the in-vehicle camera, a value of a second reference angle that is an angle formed by the optical axis of the in-vehicle camera with respect to the road surface on which the vehicle exists is estimated at the imaging time t1. Second reference angle estimating means for
The second reference angle estimation means determines the second distance estimated value based on at least the estimated value of the first reference angle, the estimated value of the second reference angle, and the set value of the height of the in-vehicle camera. Means to
When the vehicle speed of the vehicle is a low vehicle speed equal to or lower than a predetermined value, the distance estimated value determining means determines the first distance estimated value as the target regardless of the determination result of the second distance estimated value determining means. A distance measuring device, wherein the distance measuring device is fixed as an estimated value of a distance between an object and the host vehicle . The distance measuring device according to claim 1 ,
The distance-measuring feature point extraction means projects the feature point obtained by projecting the still point onto the captured image captured at the imaging time t1, and the captured image captured at the imaging time t2 different from the imaging time t1. A feature point that is extracted from each of the two captured images and specifies the position of each feature point in each of the two captured images.
The camera motion measurement means is means for sequentially measuring camera motion parameters representing changes in position and posture of the in-vehicle camera in a period between the imaging times t1 and t2 of the two captured images,
The camera motion parameter, the position of the feature point in each of the two captured images, the height of the vehicle-mounted camera, and the point positioned below the vehicle-mounted camera on the road surface where the vehicle is present at the imaging time t1 And a measured value of the camera motion parameter based on an arithmetic expression representing a relationship between a straight line from the stationary point to the stationary point and a third reference angle that is an angle formed with respect to the optical axis of the in-vehicle camera, Third reference angle estimation means for estimating the value of the third reference angle as an unknown in the arithmetic expression from the position of the feature point specified in each captured image and the set value of the height of the in-vehicle camera Further comprising
The second distance estimating means, based on the estimated value of the first reference angle, the estimated value of the second reference angle, the estimated value of the third reference angle, and the set value of the height of the in-vehicle camera, A distance measuring apparatus which is means for determining a second distance estimated value. A distance measuring device that measures a distance between an object on a road surface imaged by an in-vehicle camera and the vehicle on which the in-vehicle camera is mounted,
An object extraction means for extracting an image of an object to be measured from the captured image of the in-vehicle camera, and the distance from the host vehicle;
Based on the image of the object extracted from the captured image, the object size in the image, which is the size of the object in the captured image, is specified, and the specified object size in the image and the in-vehicle camera From the ratio of the object standard size value set in advance as the standard value of the object size, which is the size of the object in real space seen in the optical axis direction, and the setting value of the focal length of the in-vehicle camera First distance estimating means for determining a first distance estimated value that is a first estimated value of the distance between the object and the host vehicle at the imaging time t1 of the captured image;
A stationary point on the object existing road surface on which the object exists is a stationary point whose distance from the host vehicle is the same as the ground contact point of the object at the imaging time t1. A feature point extraction means for distance measurement that extracts from the captured image a feature point formed by projecting on the captured image captured in step, and specifies a position of the feature point in the captured image;
Based on the position of the feature point specified in the captured image, a first reference angle value that is an angle formed by a straight line from the in-vehicle camera to the stationary point with respect to the optical axis of the in-vehicle camera is estimated. 1 reference angle estimation means;
Based on at least the estimated value of the first reference angle and the set value of the height of the in-vehicle camera, a second estimated distance value that is a second estimated value of the distance between the object and the host vehicle is determined. A second distance estimating means;
The distance value estimated based on the ratio between the specified object size in the image and the maximum size setting value set in advance as the maximum value of the object size, and the specified object size in the image And a range between the distance value estimated based on a ratio of a minimum size setting value set in advance as a minimum value of the target object size as an allowable range of the second distance estimation value, Second distance estimated value judging means for judging whether or not the second distance estimated value is within the set allowable range;
If the determination result of the second distance estimated value determining means is affirmative, the second distance estimated value is determined as an estimated value of the distance between the object and the host vehicle, and the determination result is negative A distance estimated value determining means for determining the first distance estimated value as an estimated value of the distance between the object and the host vehicle ,
Gradient detection means for detecting whether or not the object existence road surface has a gradient with respect to the own vehicle existence road surface on which the vehicle exists,
The distance estimated value determining means, when the detection result of the gradient detecting means is affirmative, uses the first distance estimated value as the object regardless of the determination result of the second distance estimated value determining means. A distance measuring device that is determined as an estimated value of a distance between the vehicle and the host vehicle . In the distance measuring device according to any one of claims 1 to 3 ,
Based on the image of the object extracted by the object extraction means, the object is a non-standard object having a non-standard size value that deviates from the object standard size value in the type to which the object belongs. A non-standard object judging means for judging whether or not there is,
When it is determined that the object is a non-standard object, the distance estimated value determining means determines the second distance estimated value without depending on the determination result of the second distance estimated value determining means. A distance measuring device, wherein the distance measuring device is fixed as an estimated value of a distance between an object and a host vehicle.