JP4876676B2 - POSITION MEASURING DEVICE, METHOD, AND PROGRAM, AND MOVEMENT DETECTION DETECTING DEVICE, METHOD, AND PROGRAM - Google Patents

Description

  The present invention relates to a position measurement apparatus, method and program, and a movement amount detection apparatus, method and program, and more particularly to a position measurement apparatus, method and method for measuring a relative position between a current position and an object from a plurality of images. The present invention relates to a program, a movement amount detection apparatus, a method, and a program.

  As a vehicle-mounted distance measuring device, one using a monocular camera has been proposed. This technique essentially calculates the distance from the positional relationship of an object projected on a plane as 3D information as 2D information. Therefore, there is a problem in that the distance measurement value changes greatly when the geometric parameter which is a precondition for calculating the distance changes. In particular, in a vehicle-mounted distance measuring device, the geometric parameter greatly changes due to a change in the behavior of the vehicle, so that the influence is great.

  Therefore, an in-vehicle distance measuring device that can accurately measure the distance even when the posture of the vehicle changes is disclosed (see Patent Document 1). The technique of Patent Literature 1 detects a monocular imaging unit mounted on a vehicle, a vehicle speed sensor for detecting the vehicle speed of the host vehicle, and a structure fixed near a road in an image obtained by the imaging unit. A structure detecting means for detecting the structure, a structure distance estimating means for estimating a distance from the imaging means to the structure based on a state of image change of the structure and a vehicle speed of the host vehicle, and running on the image Object distance estimating means for estimating a distance from the imaging means to the object based on a positional relationship on the image between an object present on the road and the structure.

Also, those using points as feature regions, those using points and straight lines, and those using only straight lines are disclosed (see Non-Patent Documents 1, 2, and 3).
JP 2003-247824 A Richard I. Hartley, “In Defense of the Eight-Point Algorithm” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.19, No.6, June 1997 Cordelia Schmid et al., "Automatic Line Matching across Views" Computer Vision and Pattern Recognition, 1997 Peregero O. A. (Pellejero OA) et al., "Automatic computation of the fundamental matrix from matched lines" Current Topics in Artificial Intelligence, April 2004

  However, in patent document 1, in order to obtain | require the change of the height of a roadside structure, it is necessary to see both the upper end and lower end of a roadside structure. For this reason, there exists a problem which cannot be measured when the upper end and the lower end are hidden by objects (for example, a pedestrian, a parked vehicle, etc.) other than a roadside structure.

  In Non-Patent Documents 1 and 2, calculation is performed using feature points corresponding to each other between images. From one image, the number of feature points detected is larger than that of a straight line, and there is a problem that the calculation cost for matching is high. Also, the feature point has less luminance information around the feature point that can be used for matching than the straight line. For this reason, it is easy to receive the influence of a noise and a correspondence mistake occurs easily. Furthermore, there is a problem that it is not known which feature points represent roadside structures.

  In Non-Patent Document 3, only a straight line is used, but since three-dimensional motion is assumed, (1) two or more planes are required in the imaged space, and (2) four from the plane. It is necessary to detect the above straight lines. For this reason, there exists a problem which can be used only for the position measurement of walls, such as a house.

  The present invention has been proposed in order to solve the above-described problem, and a position measuring device, method, and program for accurately measuring a relative position with respect to an object without incurring calculation costs, and a moving amount. An object of the present invention is to provide a detection device, a method, and a program.

The position measuring device according to the present invention is picked up at three different positions among an imaging unit composed of a monocular camera that captures the front of a vehicle and generates a plurality of images, and a plurality of images generated by the imaging unit. Vertical edge line detection means for detecting vertical edge lines from the first image, the second image, and the third image , respectively , and the vertical edge lines of the respective images detected by the vertical edge line detection means. The corresponding vertical edge line determining means for determining the corresponding vertical edge line from the above, the horizontal position on the image of the corresponding vertical edge line determined by the corresponding vertical edge line determining means, and the first image are generated The yaw angle and the amount of movement of the vehicle that have changed from the time of the generation of the second image to the time of the generation of the second image and the time of the generation of the second image to the time of the generation of the third image Yaw angle and vehicle Based on the moving amount, and a, a position measuring means for measuring the relative position information to the object the longitudinal edge line represents.

The imaging means images the front of the vehicle and generates a plurality of images at different locations. The vertical edge line detection means detects vertical edge lines from the first image, the second image, and the third image captured at three different positions . Each vertical edge line detected here may be a vertical edge line of a completely different object between images or a vertical edge line of the same object. Therefore, the corresponding vertical edge line determination means determines a corresponding vertical edge line from among the vertical edge lines of each image. The position measuring means includes the determined horizontal position on the image of the corresponding vertical edge line, the yaw angle changed from the time when the first image is generated to the time when the second image is generated, and the movement of the vehicle. Relative to the object represented by the vertical edge line based on the amount, the yaw angle changed from the time when the second image is generated to the time when the third image is generated, and the amount of movement of the vehicle. Measure location information.

Therefore, in the above invention, the front of the vehicle is imaged to generate a plurality of images , and the first image, the second image, and the third image captured at three different positions among the plurality of images. A vertical edge line is detected, a corresponding vertical edge line is determined from the detected vertical edge lines of each image, a horizontal position on the image of the determined corresponding vertical edge line, and a first The yaw angle and the amount of vehicle movement that have changed from the time when the image is generated to the time when the second image is generated, and the time from when the second image is generated to the time when the third image is generated By measuring the relative position information to the object represented by the vertical edge line based on the changed yaw angle and the amount of movement of the vehicle, the relative position of the object can be obtained from a plurality of images without incurring calculation costs. The position can be measured accurately.

  The present invention can also be applied to a position measurement method and program.

  The present invention can accurately detect the relative position of an object.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram illustrating the configuration of the position measurement apparatus according to the first embodiment. The position measurement device is mounted on a vehicle, for example, and measures the relative position between the vehicle (hereinafter referred to as “own vehicle”) and the object.

  The position measurement device includes an imaging unit 11 installed so as to image the front of the vehicle, a vertical edge line detection unit 12 that detects a vertical edge line from a vehicle front image generated by the imaging unit 11, and a plurality of vertical edge lines. Corresponding vertical edge line determination unit 13 that determines a corresponding vertical edge line (hereinafter referred to as “corresponding vertical edge line”), and a positional relationship calculation unit that calculates a relative positional relationship between the vehicle and the object 16.

  FIG. 2 is a diagram showing a coordinate system (hereinafter referred to as “vehicle coordinate system”) of a vehicle on which the position measuring device is mounted. The Z-axis direction corresponds to the traveling direction of the vehicle, and the X-axis direction corresponds to the lateral direction of the vehicle (a direction orthogonal to the traveling direction). Therefore, in the vehicle coordinate system, the coordinate position of the stationary object changes as the vehicle moves.

  FIG. 3 is a diagram illustrating a vehicle coordinate system based on a vehicle before exercise. FIG. 4 is a diagram illustrating a vehicle coordinate system based on a vehicle after exercise. Here, “′” is attached to the X axis and the Z axis of the vehicle coordinate system that is not the reference. A vehicle motion variable that represents the amount of movement of the vehicle is represented by a translational amount of movement and a yaw angle.

  For example, in the case of FIG. 3, the vehicle before the exercise is represented by a translational movement amount 0 and a yaw angle 0. The vehicle after the exercise is represented by a translational movement amount (+ tz) and a yaw angle (+ θ). The yaw angle is an angle around the vertical axis, and the counterclockwise direction is positive. In the case of FIG. 4, the vehicle after exercise is represented by a translational movement amount 0 and a yaw angle 0. The vehicle before exercise is expressed by a translational movement amount (−tz) and a yaw angle (−θ).

  The position measuring device configured as described above executes processing as follows when the vehicle moves.

  When the vehicle moves, the imaging unit 11 images the front of the vehicle at three different positions (viewpoint 1, viewpoint 2, and viewpoint 3), and generates a vehicle forward image at each position. In the present embodiment, the viewpoint 1 is a position before the movement of the vehicle, but the viewpoint 1 may be a predetermined position during movement.

  FIG. 5 is a diagram illustrating a relationship between the vehicle front image generated by the imaging unit 11 and the vehicle coordinate system. The vehicle front image is an image in the Z-axis direction of the vehicle coordinate system. Note that f represents the camera focal length of the imaging unit 11, and it is assumed that there is a projection plane at the position of Z = f. Note that the vehicle front image includes a road surface and a roadside structure.

  The vertical edge line detection unit 12 uses, for example, the characteristic of the vertical edge line that the luminance change in the horizontal direction is larger than that in the vertical direction, and the roadside structure of the vehicle front image at the viewpoints 1 to 3 generated by the imaging unit 11. Detect vertical edge lines from objects. The vertical edge line detection unit 12 detects, for example, the contour of a power pole if it is a power pole, or the vertical contour of a building if it is a building. Thereby, a plurality of vertical edge lines are detected from the vehicle front images of the viewpoints 1 to 3.

  Next, as shown in FIG. 5, the vertical edge line detection unit 12 detects a horizontal position u representing the X coordinate of each vertical edge line. In the present embodiment, the horizontal position u is the X coordinate of the vehicle coordinate system, but may be a horizontal distance from a predetermined reference position (for example, the left end or the right end of the vehicle front image) to the vertical edge line. The horizontal position u may be detected after the corresponding vertical edge line is determined by the corresponding vertical edge line determination unit 13.

  The corresponding vertical edge line determination unit 13 determines the corresponding vertical edge line between the vehicle front images at the viewpoints 1 to 3 detected by the vertical edge line detection unit 12. Specifically, the corresponding vertical edge line determination unit 13 detects the corresponding vertical edge line using the characteristic that the luminance distribution of the corresponding vertical edge line between the front images of the vehicle and the surrounding points is substantially constant. To do.

  For example, the corresponding vertical edge line determination unit 13 determines the vertical edge line of the same part of the power pole between the vehicle front images when the vehicle front images of the viewpoints 1 to 3 have a power pole. That is, the vertical edge line of the same part between the vehicle front images is determined.

FIG. 6 is a diagram illustrating the vertical edge lines p ₁ and p ₂ associated with the vehicle front image at (a) viewpoint 3, (b) viewpoint 2, and (c) viewpoint 1. FIG. The vertical edge line p ₁ is a vertical edge line of the same portion of the structure on the right road side. Vertical edge line p ₂ is a vertical edge lines of the same part of the structure of the left roadside. The subscript _i of pi represents the identification number of the vertical edge line that has been associated. In the following, the horizontal position u is represented by u _i and _a . The subscript a represents the numbers of the viewpoints 1 to 3. That is, u _i and _a represent the horizontal position of the vertical edge line of the identification number i at the viewpoint a.

The positional relationship calculation unit 16 determines the position of the vertical edge line p _{i with} respect to the viewpoint 3 based on the horizontal position u _i , _a , and the focal length f for the corresponding vertical edge line determined by the corresponding vertical edge line determination unit 13. calculate.

FIG. 7 is a diagram illustrating a vehicle coordinate system based on the position at the viewpoint 3. The positional relationship calculation unit 16 calculates θ and t in this vehicle coordinate system, and uses these to calculate the coordinates (X _i , Z _i ) of the vertical edge line p _i (i = 1 or 2) with respect to the viewpoint 3. .

Here, θ and t are θ ₁ : Yaw angle change amount when moving from viewpoint 1 to viewpoint 2 θ ₂ : Yaw angle change amount when moving from viewpoint 2 to viewpoint 3 t _z1 : From viewpoint 1 to viewpoint 2 Translational movement amount when moving t _z2 : Translational movement amount when moving from viewpoint 2 to viewpoint 3 . Also, it represents the straight line from the viewpoint a to the vertical edge line p _i l _i, in _a. For example, the straight lines l 2 and ₃ represent straight lines from the viewpoint 3 to the vertical edge line p ₂ .

Here, the straight lines l _i , 1, l _i , 2, l _i , 3 are represented by the following equations (1) to (3), respectively.

However, u ′ = u / f (u ′ _{i, a} = u _{i, a} / f), and f is a focal length. The positional relationship calculation unit 16 calculates, for example, the intersections of the straight lines l _i and 1 and the straight lines l _i and 3 and the intersections of the straight lines l _i and 2 and the straight lines l _i and 3, and the two intersections are the same. Utilizing this, θ ₁ , θ ₂ , t _z1 , and t _z2 when the error between the two intersections is minimized are calculated. Hereinafter, the derivation of θ and t will be described.

(Derivation of θ and t)
_Let t _z1 , t _z2 , θ ₁ , and θ ₂ be unknown parameters. A straight line expression is expressed as a vector from the viewpoints 1 to 3. First, a vector is defined as shown in equations (4) to (7).

  Furthermore, when the parameter r is introduced, Expression (10) is established from Expression (8).

  The number on the right shoulder represents the coordinate system number. These equations are expressed in the coordinate system of the viewpoint 3. When the rotation matrix R and the translation vector h are used, Expressions (11) and (12) are established.

  However, Expressions (13) to (15) are satisfied.

  When each component is displayed, the equations (16) to (18) are obtained.

When the x component and the z component are expanded and the parametric variable r is deleted and summarized, the above equation (1) to equation (3) can be obtained. Therefore, the x coordinate x ₁₃ of the intersection of the straight line l ₁ and the straight line l ₃ is expressed by equation (19), and the z coordinate z _{13 of the} intersection is expressed by equation (20).

Further, x coordinate x ₂₃ at the intersection of the straight line l ₂ and the straight line l ₃ is the formula (21) and, z coordinate z ₂₃ of the intersection will be equation (22).

  Then, using the fact that the intersection of the straight lines is equal, the error of the intersection is defined. The error is defined by, for example, Expression (23) to Expression (25).

Then, θ ₁ , θ ₂ , t _z1 , and t _z2 when the error is minimized may be obtained.

The positional relationship calculation unit 16 calculates θ ₁ , θ ₂ , t _z1 , and t _{z2 as described above,} and then calculates these values using the equations (18), (19) (or (20), and (21) )). Thereby, the coordinate in which the vertical edge exists in a vehicle coordinate system is calculated | required.

  As described above, the position measurement apparatus according to the first embodiment detects the vertical edge lines from the vehicle front images taken at three locations, and uses the horizontal positions of the corresponding vertical edge lines in the respective vehicle front images. Thus, the relative position of the object with respect to the host vehicle can be obtained from the intersection of a plurality of straight lines.

  Further, the position measuring device calculates a yaw angle from a plurality of vehicle front images without using a yaw rate sensor when determining the relative position of the object, and thus can function as a yaw angle detecting device.

In the present embodiment, two or more associated vertical edge lines are required to calculate the four vehicle motion variables t _z1 , t _z2 , θ ₁ , and θ ₂ . However, when t _z1 = t _z2 or θ ₁ = θ ₂ , the number of associated vertical edge lines is sufficient. Further, when θ = 0, it is not necessary to calculate sin θ and cos θ, so that the calculation speed can be increased. The same applies to the second and third embodiments described later.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the site | part same as 1st Embodiment, and the overlapping description is abbreviate | omitted.

  FIG. 8 is a block diagram illustrating a configuration of the position measurement apparatus according to the second embodiment. The position measuring device according to the second embodiment is obtained by adding a vehicle speed sensor 14 to the configuration shown in FIG.

  The vehicle speed sensor 14 is provided for each wheel provided in the vehicle, generates a signal corresponding to the rotational angular velocity of the wheel, and supplies the signal to the positional relationship calculation unit 16.

The positional relationship calculation unit 16 calculates the vehicle speed based on the signal from the vehicle speed sensor 14, and calculates the translational movement amounts t _z1 and t _z2 by integrating the vehicle speed over time. Then, the intersection of the straight line l _i , 1 and the straight line l _i , 3 and the intersection of the straight line l _i , 2 and the straight line l _i , 3 are calculated, and the fact that the two intersection points are the same, the two intersection points Θ ₁ and θ ₂ when the error is minimized are calculated.

Here, in the first embodiment, two vertical edge lines p ₁ and p ₂ are necessary to calculate t _z1 , t _z2 , θ ₁ , and θ ₂ . However, in the second embodiment, since only θ ₁ and θ ₂ need be calculated, only one vertical edge line is required.

  As described above, the position measuring device according to the second embodiment calculates the vehicle speed from the signal detected by the vehicle speed sensor 14 and measures the relative position of the object using the vehicle speed, thus reducing the calculation load. Thus, the relative position of the object can be measured at high speed.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the site | part same as 2nd Embodiment, and the overlapping description is abbreviate | omitted.

  FIG. 9 is a block diagram illustrating a configuration of a position measurement apparatus according to the third embodiment. The position measuring device according to the third embodiment is obtained by adding a roadside structure detection unit 15 to the configuration shown in FIG.

  The roadside structure detection unit 15 stores a template representing the luminance pattern of the roadside structure. The roadside structure detection unit 15 scans the vehicle front image generated by the imaging unit 11 while changing the size of the template, and calculates the degree of similarity by using a pattern matching method (for example, SVM, Viola & Jones method). The position and size of the structure on the vehicle front image are detected.

  The vertical edge line detection unit 12 detects the horizontal position of the vertical edge line only in the portion of the roadside structure on the vehicle front image detected by the roadside structure detection unit 15. Since the range in the vehicle front image to detect can be limited, calculation cost can be suppressed. Further, the positional relationship calculation unit 16 avoids using a vertical edge line (for example, a vertical edge line of a traveling vehicle) that is not detected from the roadside structure for calculation, and a vertical edge line detected from the roadside structure. Calculate using only.

  As described above, the position measurement apparatus according to the third embodiment uses the roadside structure detection unit 15 to improve the efficiency of the vertical edge line detection unit 12 and the vertical edge line used in the positional relationship calculation unit 16. Optimization can be achieved.

  The vehicle speed sensor 14 is used to detect the actual absolute amount on the road, but may be omitted if other means can detect the actual absolute amount on the road. For example, the diameter of the electric pole is about 30 cm, and when the roadside structure detection unit 15 detects the electric pole, the distance between the two vertical edge lines representing the outline of the electric pole detected by the vertical edge line detection unit 12 is 30 cm. It is determined. At this time, the positional relationship calculation unit 16 can also determine the absolute amount of other vertical edge lines using the diameter of the utility pole 30 cm in the vehicle front image as a scale.

  Note that the present invention is not limited to the above-described embodiment, and it is needless to say that the present invention can also be applied to a design modified within the scope of the claims.

It is a block diagram which shows the structure of the position measuring device which concerns on 1st Embodiment. It is a figure which shows a vehicle coordinate system. It is a figure which shows the vehicle coordinate system on the basis of the vehicle before exercise | movement. It is a figure which shows the vehicle coordinate system on the basis of the vehicle after exercise | movement. It is a figure which shows the relationship between the vehicle front image produced | generated by the imaging part, and a vehicle coordinate system. (A) viewpoint 3 is a diagram showing a (b) the viewpoint 2, (c) vertical edge line p ₁ associated with a vehicle front image at the viewpoint 1, p _2. It is a figure which shows the vehicle coordinate system on the basis of the position in the viewpoint 3. FIG. It is a block diagram which shows the structure of the position measuring device which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the position measuring device which concerns on 3rd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Image pick-up part 12 Vertical edge line detection part 13 Corresponding vertical edge line determination part 14 Vehicle speed sensor 15 Roadside structure detection part 16 Positional relationship calculation part

Claims (7)

Imaging means comprising a monocular camera that images the front of the vehicle and generates a plurality of images;
Vertical edge line detection means for detecting vertical edge lines from the first image, the second image, and the third image captured at three different positions among the plurality of images generated by the imaging means. When,
Corresponding vertical edge line determining means for determining a corresponding vertical edge line from among the vertical edge lines of each image detected by the vertical edge line detecting means;
The horizontal position on the image of the corresponding vertical edge line determined by the corresponding vertical edge line determination means , and the yaw changed from when the first image was generated to when the second image was generated. Based on the angle and the amount of movement of the vehicle, and the yaw angle and the amount of movement of the vehicle changed from when the second image is generated to when the third image is generated , the vertical edge line is Position measuring means for measuring relative position information to the object to be represented;
A position measuring device with Further comprising object detection means for detecting the object based on an image generated by the imaging means and a pattern of the object prepared in advance.
The position measuring device according to claim 1 , wherein the vertical edge line detection unit detects a vertical edge line from the object detected by the object detection unit. Capture the front of the vehicle with a monocular camera to generate multiple images,
A vertical edge line is detected from each of the first image, the second image, and the third image captured at three different positions among the plurality of generated images,
Determining a corresponding vertical edge line from among the detected vertical edge lines of each image;
The determined horizontal position of the corresponding vertical edge line on the image, the yaw angle and the amount of vehicle movement that have changed from when the first image is generated to when the second image is generated; , Relative to the object represented by the vertical edge line based on the yaw angle and the amount of vehicle movement that have changed from when the second image is generated to when the third image is generated. Position measurement method that measures accurate position information. On the computer,
A vertical edge line is detected from each of a first image, a second image, and a third image captured at three different positions among a plurality of images generated by capturing the front of the vehicle with a monocular camera. ,
A corresponding vertical edge line is determined from the vertical edge lines of each detected image,
The determined horizontal position of the corresponding vertical edge line on the image, the yaw angle and the amount of vehicle movement that have changed from when the first image is generated to when the second image is generated; , Relative to the object represented by the vertical edge line based on the yaw angle and the amount of vehicle movement that have changed from when the second image is generated to when the third image is generated. Position measurement program that measures accurate position information. Imaging means comprising a monocular camera that images the front of the vehicle and generates a plurality of images;
Vertical edge line detection means for detecting vertical edge lines from the first image, the second image, and the third image captured at three different positions among the plurality of images generated by the imaging means. When,
Corresponding vertical edge line determining means for determining a corresponding vertical edge line from among the vertical edge lines of each image detected by the vertical edge line detecting means;
The horizontal position on the image of the corresponding vertical edge line determined by the corresponding vertical edge line determination means , and the yaw changed from when the first image was generated to when the second image was generated. Detecting means for detecting an angle and a moving amount of the vehicle, and a yaw angle and a moving amount of the vehicle that have changed from when the second image is generated to when the third image is generated ;
A movement amount detection device comprising: Capture the front of the vehicle with a monocular camera to generate multiple images,
A vertical edge line is detected from each of the first image, the second image, and the third image captured at three different positions among the plurality of generated images,
Determining a corresponding vertical edge line from among the detected vertical edges of each image;
The determined horizontal position of the corresponding vertical edge line on the image, the yaw angle and the amount of vehicle movement that have changed from when the first image is generated to when the second image is generated; A movement amount detection method for detecting a yaw angle and a movement amount of a vehicle that have changed between when the second image is generated and when the third image is generated . On the computer,
A vertical edge line is detected from each of a first image, a second image, and a third image captured at three different positions among a plurality of images generated by capturing the front of the vehicle with a monocular camera. ,
A corresponding vertical edge line is determined from the vertical edges of each detected image,
The determined horizontal position of the corresponding vertical edge line on the image, the yaw angle and the amount of vehicle movement that have changed from when the first image is generated to when the second image is generated; A moving amount detection program for detecting a yaw angle and a moving amount of a vehicle that have changed from when the second image is generated to when the third image is generated .

Images