JP3081788B2 - Local positioning device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention provides localization data relating to the position, speed and attitude of an object in a local area, and more particularly, to stationary or running vehicles on the road. The present invention relates to a localization system that is particularly suitable for detecting the state of a vehicle.

[0002]

2. Description of the Related Art FIG. 30 shows a conventional local position detecting device used for an automobile. This conventional local position grasping device LLP includes an edge extractor 1P, a threshold setting device 3P, a threshold setting device 5P, a matching detector 9P, and a lane contour detector 1.
1P, an area limiter 13P, a current position detector 15P, a curvature detector 17P, and a yaw angle detector 19P.

The edge extractor 1P is connected to a digital image pickup device (FIG. 1). The digital imaging device is mounted on an automobile AM (FIG. 1) and generates a perspective image Vi of a scenery ahead in the traveling direction of the automobile as a digital image signal Si. This image Vi shows a road, a currently running lane Lm provided thereon, and white lines Lm1 and Lm2 that regulate both sides of the lane Lm. Furthermore, the white line Lm3 of the adjacent lane is also shown. The edge extractor 1P
From the digital image signal Si, these lanes Lm1,
An edge pixel corresponding to Lm2 and Lm3 is extracted to generate an extracted edge pixel signal Sx. The extracted edge pixel signal Sx displays an extracted edge image Vx including only the extracted edge pixels.

[0004] The threshold setter 3P uses a signal Sx 'to extract the lines of each lane Lm to be marked in a known manner.
To extract a pixel indicating the outline of the lane marking from the edge pixel data signal Sx ′.
Determine h ′. Then, a signal Sc ′ is generated by extracting outline pixels representing the outline of the lane marking.

The matching detector 9P finds a straight line or an arc that matches the contour line included in the contour extraction signal Sc 'from which the contour has been extracted as described above, and performs a matching operation that includes all of the straight lines or the arc-matched lines. Data Sm '
Generate Further, the lane contour extraction unit 11P compares the matching data with the dimensional features of the lane stored in advance, adopts a matching line satisfying the dimensional features as the contour of the lane, and outputs a lane extraction signal. Generate Smc '.

The area limiter 13P outputs a lane extraction signal Sm
Based on c ′, an area is set around the extracted lane, and an area signal Sr ′ defining this area is generated. The edge extractor 1 limits the area in the perspective image Vi to an area defined by the area signal Sr ′. The current position detector 15P detects the position on the road where the car AM is currently located. The curvature detector 17P detects the curvature of the traveling lane. The yaw angle detector 19P detects an inclination angle of the vehicle with respect to the lane.

[0007]

However, all the processing is performed based on the perspective image Vi in front of the vehicle obtained by the imaging device 100. Needless to say, in the perspective view, a three-dimensional space is represented on a two-dimensional plane, so that it is not possible to obtain correct dimensional information of an object, that is, a lane, from the perspective image Vi. In the perspective image Vi, the shape is changed and displayed as the distance from the imaging device 100 increases. In other words, objects that are far away
Nearby object. Expressed smaller.

[0008] As described above, it is impossible to obtain information on the correct dimensions of the object from the contour extraction signal Sc which is a perspective image. Further, based on such unreliable, inaccurate, and distorted dimension information, processing such as curvature and yaw angle detection is performed based on the local position information of the vehicle, and based on the local position information incorrectly obtained. Obviously, doing so is very dangerous.

A different local position grasping system is proposed in Japanese Patent Application Laid-Open No. 3-139706. By detecting tangents at two points at different distances from the vehicle with respect to the two guide lines extracted from the image captured from the imaging means, calculating the curvature of the guide lines from the detected tangents, the pre-stored guide lines are stored. The lateral displacement of the vehicle is detected from the relationship between the curvature and the vehicle state quantity. However, even in such a local position grasping device, when the curvature of the curve is small, an accurate curvature cannot be obtained, and when the interval between the white lines is changed, it cannot be dealt with. There was a problem that the position of the vehicle in the lane could not always be measured accurately.

[0010]

SUMMARY OF THE INVENTION Based on a digital image signal representing a perspective view of a local area in the direction of travel of the object, a local area of the object which can travel in a direction related to a lane on the local area. A local position grasping device for detecting a target position, extracting a contour of the lane from the digital image signal and generating contour data, and converting the contour data to represent a plan view. Converting means for generating coordinate conversion data accurately representing the dimensions of the extracted contour; and detecting dimensional characteristics of the extracted outline based on the coordinate conversion data to generate dimensional data. And a dimensional characteristic detecting means.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows a local positioning system LPS incorporating a local positioning device LP according to the present invention.
Is shown. This local position grasping system LPS is mounted on, for example, an automobile AM. The local positioning system LPS includes a local positioning device LP, a digital imaging device 100, a moving speed detector 200, a GPS (global positioning device) receiver 300, and an ECU (electric control unit).
400 and a navigation device 500.

The digital image pickup apparatus 100 includes an image pickup device, and can continuously obtain an image of an object by using a PhxPV pixel matrix (FIG. 3). Ph and PV
Represents the number of pixels arranged in the horizontal direction and the vertical direction, respectively. Therefore, if the number of pixels Ph and PV are increased, the resolution of the image can be increased, so that the accuracy of the operation such as the position grasp of the present invention can be improved. on the other hand,
Due to the problem of increased manufacturing costs, these numbers Ph and PV should be determined according to the required resolution and manufacturing costs. As an example, Ph and Pv are 428 and 268, respectively, but are not limited thereto.

The digital imaging device 100 is installed near the front end of the automobile AM. The digital imaging device 100.
A landscape image Vi in the traveling direction of the automobile AM is continuously photographed to generate a digital image signal Si. This image Vi
FIG. 2 is a front perspective view seen from inside the automobile AM. This image may be either a still image or a moving image.

FIG. 7B schematically shows a side view of an automobile AM on which the digital imaging device 100 is mounted. The optical axis Ax of the digital imaging device 100 matches the direction Z in which the automobile AM moves on the road, but is set to be downward at a predetermined angle θ with respect to the horizontal plane. In this case, the digital imaging device 10 having a sufficiently large angle of view to obtain an image Vi of the road surface immediately below the front end of the automobile AM.
0 is used.

FIG. 3 shows an automobile AM represented by a digital image signal Si obtained by the digital imaging device 100.
5 shows a perspective image Vi as viewed from the front in the traveling direction. Car A
M is traveling in the left lane of a two-lane road. The two-lane road is divided into two lanes by three lane separation lines Lm1, Lm2, and Lm3. That is, the right lane is guided by the separation lines Lm3 and Lm2, and the left lane is guided by the separation lines Lm1 and Lm2. Usually, these lane separation lines are painted on the road with white paint,
It is specified. In the figure, the bottom part of the perspective view Vi is
It is an image of the road just under the front end of the car AM.

FIG. 1 shows a moving speed detector 200 for detecting a moving speed of an automobile AM and generating a moving speed signal Sv.
It is shown. As the moving speed detector 200, a speedometer widely used in automobiles can be used. The local position grasping device LP is connected to the digital imaging device 100, the moving speed detector 200, and the GPS receiver 300, and respectively receives the digital image signal Si and the moving speed signal S.
v and the global positioning signal Sg. Based on these signals, the local positioning device LP performs various calculations to detect the moving or stationary position of the vehicle AM as described later. The structure and operation will be described later in detail with reference to FIGS. 2, 19, 26, and 28.

The ECU 400 includes a local position grasping device LP
And an ECU signal Se for controlling the operation of the ECU.
Receive. Based on the signal Se, the ECU controls the operation of the engine of the automobile AM. Since the ECU is a well-known technology in the automobile industry, a detailed description thereof will be omitted.
The navigation device 500 includes a local position grasping device L
Connected to P to receive the navigation signal Sn. Based on the signal Sn, the navigation device 500 performs various calculations to obtain navigation data. A navigation device commonly used in automobiles can be used.

First Embodiment FIG. 2 shows a first embodiment of the local position grasping device of the present invention. The local position grasping device LP1 includes an edge extractor 1, a threshold setting device 3, a contour extractor 5, a coordinate converter 7, a matching detector 9, a lane display contour extractor 11, an area limiter 13, and a curvature detector. Detector 17 and yaw angle detector 19
including. The edge extractor 1 is connected to the digital imaging device 100 and receives a digital image signal Si representing the forward scenery image Vi in the traveling direction. The edge extractor 1 performs a filtering process such as a Sobel filter on the signal Si to extract edge pixels whose image density changes rapidly.
At this time, in order to reduce the processing addition, as described below, the filter processing is performed only on a predetermined area, not on the entire perspective image Vi data.

FIG. 3 shows a perspective image Vi that has been subjected to the filtering process of the edge extractor 1. The perspective image Vi is divided into two parts, an upper area St and a lower area Sb, by one horizontal line Ls. Note that the horizontal line L
s coincides with the pixel arranged at the A-th horizontal position counted from the bottom of the image Vi in the vertical direction toward the upper end. As a result, the upper end area St has a Phx (PV-A) pixel area and the lower area Sb has a PhxA pixel area. This A
The third vertical position is preferably determined such that the vanishing point of the foreground coincides with the vertical pixel position appearing in the image Vi while the automobile AM is traveling on a flat road. In the embodiment of the present invention, A is the vertical position of a pixel that displays an image of a place approximately 50 meters away from the automobile AM when the digital imaging device 100 directly captures the traveling direction of the automobile AM. Set.

Immediately after the start of the local position grasping operation according to the present invention, the edge extractor 1 performs an edge extraction (filtering) process only on the lower region Sb to extract pixels and generate an extracted edge pixel signal Sx. I do. That is, the extracted edge pixel signal Sx includes only the edge pixels extracted from the Sb portion (Ph xA).

FIG. 4 shows an extracted edge image Vx represented by the extracted edge pixel signal Sx. From the lower area Sb of the image Vi, mainly the lane indications Lm1, Lm2, and Lm
3 and its surrounding pixels are extracted as edge pixels. However, although not shown, it is needless to say that edge pixels are also extracted from portions not related to the lane display.

As shown in FIG. 2, the threshold setting unit 3 is connected to the edge extractor 1 and receives the extracted edge pixel signal Sx. The threshold value setting unit 3 calculates, based on the extracted edge pixel signal Sx,
The threshold value Eth for effectively extracting the edge pixels of the contour portion of the lane display Lm is set based on the following equation.
To determine.

Eth = C · Emax + (1−C) · Emean (1)

In the first equation, Emax and Emean
Represents the maximum density and average density of pixels on a predetermined horizontal line in the lower region Sb of the perspective image Vi. Further, C is a predetermined constant larger than 0 and smaller than 1.

As shown in FIG. 2, a contour extractor 5 is connected to the edge extractor 1 and the threshold setting device 3, and receives the extracted edge pixel signal Sx and the threshold Eth, respectively. The contour extractor 5 extracts the contour of the vehicle display based on the threshold value Eth of the pixel in the lower region Sb of the extracted edge pixel signal Sx.

FIG. 5 shows an extracted edge image Vx from which a contour is to be extracted by the contour extractor 5. The lower region Sb of the image is further subdivided into left and right lower regions SbL and SbR by the vertical center line Lc. The center line Lc extends vertically from the pixel located at the center position Ph / 2 at the bottom of the extracted edge image Vx. However, the vertical line Lc is not limited to the center position Ph / 2.

The contour extractor 5 first scans along the horizontal line Ls, along the horizontal lines LhL and LhR in the horizontal direction from the center line Lc to the left and right, and determines the density of the intermediate pixels as a threshold Eth.
And extracts pixels having a density equal to or higher than the threshold value Eth as contour pixels. Then, after moving by a predetermined number of pixels toward the bottom along the center line Lc, scanning is performed from that position again in the left and right horizontal directions to obtain contour pixels.

In practice, one of the left and right lower regions is
The outline pixels are obtained by scanning. For example, horizontal line L
The left half LhL of h is sequentially moved from the upper part (position A) to the lower part along the center line Lc at a predetermined interval PLh to obtain a contour pixel in the lower left area Sb. After that, similarly, the contour extraction signal Sc is generated by obtaining the contour pixels of the lower right region Sb.

By skipping the horizontal at a predetermined interval PLh, an integer K horizontal lines Lh corresponding to the absolute value of the total (Pv / PLh) are extracted. This allows for faster extraction processing. Needless to say, PLh can be set for each pixel.

In this way, the contour pixels of the left and right lower regions SbL and SbR are obtained. If a pixel having a predetermined density is first found for each of the left and right lines LhL and LhR at each vertical position, that pixel is determined. The pixel Pe is determined as an outline pixel, and the scanning of the outline pixel is not performed on the horizontal line, and the outline pixel at the next vertical position is obtained by moving to the horizontal line at the next vertical position. That is, with reference to FIG. 5, contour pixels are detected at the inner edges of the left and right lane indications Lm1 and Lm2 with respect to the first horizontal line Lh, counting from the Ls line. However, regarding the third and fourth lines, at this vertical position, the right lane indication Lm2 is divided, so that the horizontal line passes through the right lane indication Lm2 and outlines the inner edge of the adjacent lane indication Lm3. It is detected as a pixel. Further, although not shown in FIG. 5, if there is a flaw or dirt between the vertical line Lc and each lane display, these edges are also detected as contour points. Since these contour pixels do not correspond to the currently traveling lane, they become a kind of noise component for the position determination operation according to the present invention. The removal of these noise components will be described later in detail.

FIG. 6 shows an extracted contour image Vc represented by the contour extraction signal Sc obtained in this manner. The detected contour pixels corresponding to the lane indications Lm1, Lm2, and Lm3 are dotted lines, respectively, and ScL, ScR,
And ScR '. Here, ScR ′ is the noise data described above.

As shown in FIG. 2, the coordinate converter 7 is connected to the contour extractor 5 and receives a contour extraction signal Sc having contour information as shown in FIG. The coordinate converter 7 converts the coordinate system of the contour extraction signal Sc and outputs an image V represented by the signal.
Let c be a bird's-eye view image from a perspective view. In the extracted contour image Vc, since the object is represented in a perspective state, the shape of the object becomes more distorted as the distance from the digital imaging device 100 increases. In other words,
Even with two objects of the same size,
Distant objects are represented smaller. The distortion of the shape of the object increases as the distance from the imaging device 100 or the automobile AM increases.

Referring to FIGS. 7A and 7B, the concept of coordinate conversion of the extracted contour image Vc by the coordinate converter 7 will be briefly described. FIG. 7A shows a coordinate system of the perspective images Vi, Vc, and Vcc. FIG. 7B also shows a bird's-eye view image based on the automobile AM. The x-axis of the image coordinate system (x, y) is parallel to and the same direction as the X-axis of the coordinate system of the bird's-eye view, and the optical axis Ax of the digital imaging device 100 is inclined at an angle θ with respect to the horizontal plane. It is inclined by an angle θ with respect to the axis.
Thus, the horizontal plane Pd substantially parallel to the road surface, which is the running surface of the automobile AM, is defined by the X axis and the Z axis. Y
The axis and the y axis are aligned. The Y axis is perpendicular to FIG. The origin O is the same as the origin of the coordinate system of the extracted contour image Vc, and the optical axis of the digital imaging device 100 passes through.

The coordinates of the image Vc can be converted into coordinates for the bird's-eye view image Vb by using the following equation.

X = (x / F) (Zcos θ−Ysin θ) (2)

Z = Y (Fcos θ + Y sin θ) (3)

Y = −H (4)

F is the focal length of the optical lens of the digital imaging device 100, θ is the angle between the Z axis (horizontal axis) and the optical axis Ax of the optical lens, and H is the distance from the road surface to the origin of the optical axis lens. . Both θ and H are preferably set so as to obtain a perspective image Vi in the traveling direction of the vehicle AM as described with reference to FIG. For example, θ is 6 degrees, H
Is set to 1.2 meters.

By converting the distance in the extracted contour image Vc based on the above equations (2), (3), and (4), the contour extracted signal Sc obtained by converting the coordinates of the contour extracted signal Sc is converted.
cc is obtained. In the signal Scc, the horizontal distance is accurately represented regardless of the distance from the digital imaging device 100 in the Z-axis direction. Strictly speaking,
The coordinate-transformed image Vcc is not a bird's-eye view image, but a so-called plan view of the road surface, which always reflects the road surface in a plane parallel to the road surface. Therefore, for example, the image Vcc always shows the road surface as a flat surface even if the road surface has a swelling like a small hill or has undulations. However, such errors do not impair the accuracy of localization according to the present invention. Because, as mentioned above, localization is relatively short distance, 50 meters away,
Since the control point is located at the farthest point, fluctuations such as undulations on the road surface can be ignored.

However, a complete bird's-eye view image of the road surface,
That is, if a plan view image of a road surface projected perpendicularly to a horizontal plane perpendicular to the vertical line is required, a known automatic attitude control device such as a gyro can be used without fixing the optical axis Ax of the digital imaging device 100 to the automobile AM. By using this, it is sufficient to always obtain the image Vi in a constant posture with respect to the horizontal plane.

FIG. 8 shows a contour-converted contour extraction signal Sc.
The contour represented by c is shown. Lane indications Lm1, Lm2, and
Lm3 is the corresponding edge contour line ScL, Sc
R and ScR '. As can be seen from the comparison with the extracted contour image Vc of FIG. 6, in FIG. 8, the contour lines ScL, ScR, and ScR are represented as parallel lanes on an actual road. . That is, the coordinate conversion contour extraction signal Scc includes information on the correct plane dimension of the object. FIG. 17 is a flowchart showing an operation of coordinate conversion by the coordinate converter 7.

As shown in FIG. 2, the matching detector 9 is connected to the coordinate converter 7 and outputs the coordinate conversion contour extraction signal Sc.
c, each contour ScL, ScR, and ScR ′
The equation of a straight line or an arc that conforms to is obtained as follows. First, contour lines ScL, ScR, and ScR ′ inside each lane display included in the coordinate transformation contour extraction signal Scc.
Is subjected to Hough transform separately for each corresponding contour line based on the following equation (5).

Ρ = Xcosφ + Zsinφ (5)

Ρ is the distance between the origin O and the pixel in the ZX coordinate system; φ is the angle between the line connecting the origin O and the pixel and the X axis. Then, the following equation is obtained from the fifth equation.

X = (ρ−Z sin φ) / cos φ (6)

If the coordinate transformation contour extraction signal Scc is scanned and the contour data is transformed into a parameter space based on Hough transformation, a group of curves is obtained for each contour. Based on this group of curves, whether the contour is straight,
The determination as to whether it is an arc or an arc will be described with reference to FIGS. First, FIG. 9 shows a typical pattern when the Hough-transformed line data is a straight line. That is, a group of curves intersect at one point Cp. However, as a practical problem, since the lane display Lm and the contour line are not perfect straight lines, it is very difficult to completely intersect at only one point as shown in FIG. However, since an attempt is made to intersect at approximately one point, the frequency Fc at which each curve intersects at each (pixel) point in the parameter space should be checked.
If there is a point whose frequency is larger than the predetermined threshold Fth, that point is set as the only intersection Cp. Then, the contour is regarded as a straight line, and a suitable straight line equation is obtained.

FIG. 10 shows a typical pattern when the Hough-transformed line data is an arc. That is, in this case, as shown in FIG. 10 (A), they do not intersect at all, or as shown in FIG. 10 (B), a group of curves are mutually Cp1, Cp2, Cp3, Cp4, Cp
5 and Cp6 (Cpn). n is an integer. Similarly, the intersection frequency Fc at each point
If there is no new point having a frequency higher than the set threshold Fth ′, it is determined that the group of curves does not intersect, the contour is regarded as a circular arc, and a suitable circular arc formula is determined. In other words, the dimensional characteristics of the lane in which the automobile AM is currently traveling or stationary, that is, the road, can be known from these equations.

Referring to FIG. 11, a description will be given of a method of obtaining an arc representing the curved lane Lm. Outline image V with coordinate transformation
In cc (same as in FIG. 8), all combinations of strings connecting two points are created from all points extracted from the same lane indication Lm. Now, assuming that the number of all points is Ne, the combination of all two points is (Ne-1)! It is represented by Next, a perpendicular bisector for (Ne-1)! Strings is obtained. Therefore, since the center point P is determined based on all chords extending between all edge pixels on the contour line ScR, the center of the center point P of this arc is equivalent to the XZ equivalent system. . Are represented as coordinates P (ab).

For each contour line, the distance from the calculated center of the arc to each contour point is calculated, and the most obtained distance is determined as the radius R of the arc to be adapted. In essence, the following arc relational expression is obtained for all contour lines. In addition, the fitting of the circular arc is performed by the least square method or Hough (Ho
ugh) may have a transformation.

(X−a) ＾ 2 + (Z−b) ＾ 2 = R ＾ 2 (7)

Similarly, the matching detector 9 can obtain the straight lines or arcs ScLm and ScR'm that match the other contour lines ScL and ScR. For each lane contour, data such as length and dimensions are obtained. Further, the matching detector 9 is a straight line, or
Lines ScRm, ScR'm, and ScL that fit the arc
A matching signal Sm including the dimension data of m is generated. FIG. 18 is a flowchart showing the matching operation by the matching detector 9.

As shown in FIG. 2, the lane contour extractor 11 is connected to the matching detector 9 and outputs the matching signal Sm.
Receive. As described above, the matching signal Sm includes
Contour line data ScR ′, which is a noise component, is also included. Therefore, the lane contour extractor 11 outputs the matching signal S
m is compared with principal point data such as a vehicle width, a lane width, and a division pattern of a lane separation line Lm2 in advance, and contour data S that does not conform to the target lane is compared with the characteristic data of the dimensions included in m.
cR ′ is excluded and the contour line S corresponding to the current lane is removed.
Only the data of cR and ScL are output as the matching signal Sm. If there is no data corresponding to the lane in the matching signal Sm, an error signal See is output.

Referring to FIG. 8, the filtering operation of lane contour extractor 11 will be described. The bird's eye view image Vcc now includes three contour lines ScL, ScR, and ScR ′. Up to the second pixel counted from the top, the two left contour lines S
cL and ScR form a pair that defines one lane display Lm. However, for the third to fifth pixels, since the lane table on the right side is of a split type, the coordinate transformation contour extraction signal Scc extracts the contour line ScR 'of the lane on the right. In this case, obviously, two contour lines ScL
The distance between ScR 'and ScR' is too wide for lane display. However, at this point, it is not known which contour line of ScL or ScR 'is abnormal. Regarding the eleventh pixel from the sixth request, the contour lines ScL and ScR are clearly a pair.
Further, since the left contour line ScL is matched with one arc ScLm by the matching detector 9,
The third to fifth pixels are also considered to correspond to the lane display Lm1. As a result, the next lane ScR'3
Pixels are considered erroneous and ignored. Then, only two contour lines ScL and ScR, which are a correct pair, are extracted, and the contour lines ScLm and ScRm matched with each other are selected, and together with the information, the lane extraction signal S
output as mc. In this way, it is clear that noise components that do not define one lane, such as scratches and dirt on the road, are also excluded from the lane extraction signal Smc. FIG. 12 shows an extracted lane contour image represented by the lane extraction signal Smc from which the contour noise has been removed by the lane contour extractor 11 described above.

As shown in FIG. 2, an area limiter 13 is connected to the lane contour extractor 11, and further limits the area of the perspective image Vi based on the lane extraction signal Smc. FIG.
The operation will be described with reference to FIG. FIG. 13 is basically the same as FIG. The intersections of the formulas of the left and right lane contour lines ScLm and ScRm applied in the matching process and Z = d1, d2,..., Dn are determined in advance, and the intersections are obtained by the equations (8) and (9). The points converted in the image coordinate system are connected by a straight line, and a polygonal line is created in the image coordinate system.

X = FX / (Zcos θ + Hsin θ) (8)

Z = F (Hcos θ + Z sin θ) / (Z cos θ−H sin θ) (9)

The limited regions RsL and RsR are set so as to have a width in the x direction centering on the two left and right polygonal lines. The width WR of the limited area is set in advance in consideration of the amount of movement of the lane display (white line) in the image accompanying the lateral movement of the car AM to the x method. WR is an amount that may move during one system cycle CS of the apparatus of the present invention, for example, 33 mS, assuming that the lane marks Lm1 in the perspective image Vi. Therefore, the limited width is set to an amount that increases as approaching the bottom of the image. In addition, it was set,
FIG. 14 shows the limited area. The area limiter 13 creates an area limit signal Sr. However, when the lane contour extractor 11 cannot extract the lane contour and outputs the error signal See, the above-described region limitation is not performed. The region limiter 13 is further connected to the edge extractor 1 and supplies the region signal Sr to the edge extractor 1.

The edge extractor 1 further limits the edge extraction to the limited regions RsR and RsL based on the region signal Sr, as shown in FIG. In this way, while reducing the amount of data processing and speeding up the processing,
It is possible to suppress the influence of noise components other than the target lane contour, and improve the followability to the lane.

Next, referring to FIG.
The function of No. 5 will be described. Receiving the lane extraction signal Smc from the lane contour extractor 11, the current position detector 15 applies a circle or straight line relational expression (ScL) applied to the left and right lane contours.
m and ScRm) and the intersection point HL and HR of the straight line of Z = 0. Next, the coordinates of the center of the lane, that is, the coordinates (HC, 0) of the midpoint between HL and HR are obtained. Here, since the position of the camera is (0, 0), when the digital imaging device 100 is mounted at the center of the vehicle AM, the vehicle AM
Can be obtained as a lateral variation-HC from the center of the lane.

The curvature detector 17 detects the lane curvature based on the equation (7) based on the signal Smc from the matching detector 9. If the radius of curvature of the fitted arc is, for example, 600 m or less, it is considered as a curve,
The curvature is represented by the radius of curvature of the arc. The radius of curvature of the arch line is found as the curvature of the current lane. The curvature detector 17 indicates the radius thus found,
A curvature discovery signal Scu is generated.

The yaw angle detector 19 uses the matching detector 9 to determine the angle formed by the traveling lane and the traveling method of the automobile AM based on the relational expression of a straight line or an arc. First, (.) (Installation device LP1) shown in FIG. The localization positioning device LP1 is rotated. The digital imager of the device 100 is also powered on in order to generate digital image data Si and to obtain the forward direction image Vi.

FIG. 16 shows a flowchart of the local position grasping device LP1 according to the present embodiment.

In block # 1, image data Si is received from the digital imaging device 100.

In block # 3, the edge extractor 1 extracts an edge pixel from the perspective image Vi and generates an extracted edge pixel signal Sx.

In block # 5, the threshold setting unit 3 determines a threshold Eth based on the edge pixel density in the extracted edge pixel signal Sx.

In block # 7, the contour extractor 5 extracts a contour for lane display based on the extracted edge pixel signal Sx〜 and the threshold value Eth, and generates a contour extraction signal Sc.

In block # 9, the coordinate converter 7 performs coordinate conversion, obtains a bird's-eye view image of the extracted contour image Vc, and further generates a coordinate converted contour extraction signal Scc.

In block # 11, the matching detector 9 uses the Hough transform to extract the coordinate transformation contour extraction signal S
The expression representing the inner contour line in cc is matched by a straight line or an arc. Further, the matching detector 9 generates a matching signal Sm representing a matched straight line or circular arc.

In block # 13, the lane contour extractor 11 compares the dimensional characteristics of the matched contour in the coordinate transformation contour extraction signal Scc with the dimensional characteristics of the lane. The lane contour extractor 11 generates a lane extraction signal Smc by classifying a set of lane contours that define the current lane.

In block # 15, block # 13
If an error signal See is generated in block #,
Return control to 1. Otherwise, go to the next block.

In block # 17, the area limiter 13
Defines an area around the lane indication Lm with respect to the digital image data Vi based on the matching signal Sm,
A region signal Sr is generated.

At block # 19, the current position detector 1
5 detects the current position based on the lane extraction signal Smc.
The signal Smc and the area signal Sr are transferred to the next block # 21.

In block # 21, the curvature detector 17
But. The curvature of the lane is obtained based on the lane extraction signal Smc, and the contour extraction signal Sc is generated. The signal Smc and the area signal Sr are transferred to the next block # 23. In block # 23, the yaw angle detector 19 calculates the inclination angle α based on the contour extraction signal Sc mechanism, and generates the yaw angle signal Sy. Further, the area signal Sr is transferred to the block # 1, and the system cycle Cs of the blocks # 1 to # 23 is repeated.

Second Embodiment FIG. 19 shows a local position grasping apparatus according to a second embodiment of the present invention. The local position grasping device LP2 has a configuration very similar to that of the local position grasping device LP1 of the first embodiment. However, in the present embodiment, the coordinate converter 7 of FIG. 2 is deleted, and the matching detector 9 is replaced with the inclination matching detector 9A and the second-order polynomial matching detector 9.
B has been replaced. Further, as a result, the current position detector 1
The operation of the current position detector 15 is slightly modified in its operation.
A. The other points are the same as those described in the first embodiment, and therefore, only the different parts will be described.

The inclination matching detector 9 A is the contour extractor 5
And receives the contour extraction signal Sc. The tilt matching detector 9A performs a tilt matching process on each of the pixels of the extracted contour line in the contour extraction signal Sc.

Before explaining the inclination matching processing, FIG.
With reference to FIG. 0, the appearance of the front perspective images Vi and Vc obtained while the vehicle AM is traveling on a curved lane, that is, a road, will be described. Now, as shown in the figure, if the road is curved right, the outer peripheral portion ScL appears to extend linearly from the front of the perspective image Vi in the traveling direction. On the other hand, the inner peripheral side ScR also extends linearly from the front of the perspective image Vi to the back, but it seems that the course is suddenly changed from the middle to the inner peripheral side.
This phenomenon occurs universally on curved roads. Among the pixels constituting each contour line, all the slopes of the line connecting two adjacent pixels are obtained by the horizontal method. The inclination angle at the center of the two pixels obtained in this way is represented by the inclination (SLi_
i + 1). No measurement is performed when no contour point is extracted on the adjacent scanning line, such as when the lane display contour is a broken line.

FIG. 21 shows the inclination obtained from the contour extraction signal Sc corresponding to the image of FIG. The point sequence SA at the lower left and the point sequence SD at the upper right are respectively the inner peripheral side contour S
It corresponds to cR and outer-side contour ScL. The inclination matching detector 9A applies Hough transform to each of these point sequences SA and SD, and fits a straight line represented by the following equation. y '= 2Dx + E (10)

Y ′ is the first derivative of y, and represents the slope of the following quadratic polynomial:

Y = Dx ＾ 2 + Ex + F (11)

D, E, and F are constants. The value of D represents a change in the inclination of the contour line. For example, if the extracted contour is the outer periphery, the value of D is greater than zero, and vice versa, the value of D is less than or equal to zero.

Therefore, depending on the value of D, the above two equations can be used as follows.

Equation (11) is applied when D is zero or more.

When D is equal to or less than zero, the following equation (12), which is similar to equation (10) and is derived by Hough transform, is applied.

X ′ = 2Gy + H (12)

A second-order polynomial (13) similar to equation (11) is applied.

X = 2 Gy ＾ 2 + Hy + I (13)

X ′ is the first derivative of x, and represents the change of the second-order polynomial (13). G, H, and I are constants.

Thus, the inclination matching detector 9
A converts the contour extraction signal Sc, which is a dot group in the extracted contour image Vc, into data representing the inclination of each contour line. In this manner, information indicating the bending state is obtained based on the converted inclination information data. The tilt matching detector 9A generates a tilt matching signal Sm1 indicating the information obtained in this manner.

The quadratic polynomial matching detector 9B is connected to the tilt matching detector 9A and receives the tilt matching signal Sm1. Second-order polynomial matching detector 9B
Applies a quadratic polynomial to each of the extracted contour lines ScR (ScR ′) and ScL in the signal Sc1.

If Equation (10) has already been applied and the slope matching signal Sm1 is displayed, the quadratic polynomial matching detector 9B substitutes the values of the xy coordinates for x and y in Equation (11). Then, the value of F is provided. When calculating F, the value of the most frequent one is determined as the constant F in equation (11).

Further, the slope matching signal Sm1 is given by the equation (1)
If there is a display that 3) has already been applied, the quadratic polynomial matching detector 9B continuously substitutes the value of the xy coordinate into each contour pixel in the contour extraction signal Sc according to equation (14). To determine the value of the constant I. In this way, the value of the most frequently obtained value of I is determined as the constant of equation (14).
Thus, the tilt matching detector 9A generates the tilt matching signal Sm2 indicating the information thus obtained.

If D is zero, equation (11) can be expressed as y = Ex
+ F. In this way, the curved state of the lane can be determined by a quadratic polynomial regardless of whether the lane is straight or curved.

FIG. 25 shows a flowchart of the tilt matching described with respect to the tilt matching detector 9A.

Referring to FIG. 23, the operation of the current position detector 15A for detecting the current position based on the lane extraction signal Smc which is a perspective image will be described. The current position detector 15A calculates, via the lane contour extractor 11, the quadratic polynomial applied by the quadratic polynomial matching detector 9B to the lateral displacement of the vehicle AM in the traveling lane. First, a straight line y = F / tan θ in the image coordinate system corresponding to Z = 0 and intersections PL and PR of two second-order polynomials fitted by the second-order polynomial matching detector 9B are obtained. Next, an intersection Pm between PL and PR is obtained. The coordinates (HC, 0) on the bird's-eye view image are obtained by multiplying the coordinates by a predetermined coefficient. The rest is the same as the first embodiment.

Further, referring to FIG. 22, area limiter 1
Operation 3 will be described. The region limiter 13 is set so as to have a width in the x-direction centered on the B quadratic polynomial applied by the quadratic polynomial matching processing by the quadratic polynomial matching detector 9B.

As described above, according to the present embodiment,
Since the three parameters of the quadratic polynomial are obtained in two stages, the quadratic polynomial can be applied to the lane contour, which composes the traveling lane by relatively simple calculation. The AM position can be accurately detected.

FIG. 24 shows a flowchart of the local position grasping device LP2 according to the present embodiment. This flowchart is basically the same as the flowchart according to the first embodiment in FIG. The only difference is that the coordinate transformation block # 9 in FIG. 16 is deleted, and the matching block # 11 is replaced by a gradient matching block # 10 and a quadratic polynomial matching block # 12. The basic operation has been described in relation to the tilt matching detector 9A and the quadratic polynomial matching detector 9B, respectively, and thus the description is omitted.

Third Embodiment FIG. 26 shows a third embodiment of the local position grasping device of the present invention. The local position grasping device LP3 has a configuration very similar to the local position grasping device LP2 of the second embodiment. However, in the present embodiment, the yaw angle detector 19 shown in FIG.
0, a GPS receiver 300, an ECU 400, and a navigation device 500 are added, and a local position grasping device L
From P3, a lane extraction signal Smc, a lane extraction error signal See, and a curvature signal Scu are received. Thus, an example of the localization system LPS shown in FIG. 1 is provided. The GPS receiver 300 receives a GPS signal from a navigation satellite, and determines a location, a moving speed,
Or it is obtained from global position information such as attitude. Further, from the local positioning device LP according to the present invention, it is possible to obtain the local positioning information of the automobile AM to the global positioning information by using the local positioning information, and to obtain the information from the position signal S.
output as g

The navigation device 500 includes a curve position setting device 21, a map storage device 25, and a driving safety determiner 2.
7 and an alarm 31. The map storage device 25 is provided to store road topographical features such as road maps, landmarks, and bends. The curve position setting device 21 is connected to the curvature detector 17 and the GPS receiver 300, and receives the curvature detection signal Scu and the local / global positioning signal Sg. The curve position setting device 21 outputs the curvature detection signal Sc
The radius indicated by u is compared with a predetermined curvature, and if the detected radius is larger than a predetermined value, a binary result signal that outputs a high level is generated.

The map storage device 25 is connected to the curve position setting device 21 and outputs the result signal and the curvature detection signal S.
cu, receives the localization / global positioning signal Sg. Upon receiving the high-level result signal, the map storage device 25 stores the current position (Sg) in association with the curvature (Scu) larger than a predetermined value.

The driving safety determiner 27 is provided in the map storage device 2
5. Moving speed detecting device 200 and GPS receiver 300
And the curvature of the curved road and the traveling speed signal S
v and a local / global positioning signal Sg from each. Based on these information data, the traveling safety determiner 27 determines whether or not the vehicle AM can safely travel on the current curved lane. If the determination result is “No”, the traveling safety determiner 27 outputs a warning signal. Based on the local / global position grasping signal Sg, the traveling safety determiner 27 confirms whether there is a curved portion ahead of the predetermined distance DD by referring to the record of the map storage device 25.

The predetermined distance DD corresponds to a case where the driving safety determiner 27 determines that there is a danger and issues a warning in consideration of expected driving conditions, that is, conditions such as moving speed, road conditions, and vehicle weight. In this case, the distance should be sufficient for the driver, etc., to take sufficient safety actions. In the present embodiment, 3
00m.

The traveling safety determiner 27 is performed based on the following inequality. R <V ＾ 2 (14)

R is a radius of a curved road, V is a moving speed, and Cv is a constant determined by the weight of the vehicle, kinetic performance and the like.

FIG. 24 shows a flowchart of the local position grasping device LP2 according to the present embodiment. This flowchart is basically the same as the flowchart according to the first embodiment in FIG. The difference is that the coordinate transformation block # 9 in FIG. 16 is deleted, and that the matching block # 11 is replaced by a gradient matching block # 10 and a quadratic polynomial matching block # 12. The basic features are described in relation to the tilt matching detector 9A and the quadratic polynomial matching detector 9B, respectively, and thus the description is omitted.

The alarm 31 is connected to the traveling safety determiner 27 and warns the driver based on the warning signal.

The ECU 400 is connected to the traveling safety determiner 27 and controls the drive system of the vehicle AM based on the warning signal to reduce the speed. The local position grasping device LP3 based on the third embodiment may have substantially the same structure as the local position grasping device LP1 based on the first embodiment.

FIG. 27 shows a flowchart of the local position grasping device LP3 according to the present embodiment. This flowchart is basically the same as the flowchart according to the second embodiment in FIG. The difference is that the yaw angle detection block # 23 in FIG.
25, a curve position setting block # 27, a traveling safety determination block # 29, an alarm block # 31, and an ECU operation block # 33. Since the basic operation has been described with reference to the block diagram of FIG. 26,
Further explanation is omitted.

Fourth Embodiment FIG. 28 shows a fourth embodiment of the local position grasping apparatus according to the present invention. The local position grasping device LP4 has a configuration very similar to the local position grasping device LP3 of the third embodiment. However, the moving speed detecting device 200 shown in FIG.
And the ECU 400 are deleted, while the navigation device 500 is equipped with
And the alarm device 31 are deleted, while the image capturing device 33 and the display device 34 are added to the navigation device 500R.
It has become.

[0110] The image capturing device 33 is
0 and connected to the GPS receiver 300 to receive the digital image signal Si and the local / global positioning signal Sg. The image capturing device 33 has an image buffer memory such as a frame memory, and can temporarily store image data. By the operation of the driver, the buffer memory stores the digital image signal S representing the perspective image Vi in the vehicle AM traveling direction.
i can be captured for one frame.

The map storage device 25 further includes an image capturing device 33.
To store the captured image data Si of one frame sentence in a built-in storage medium. Such recording media include SD-ROM, PD, DVD-RA
There is a large capacity rewritable optical disk such as M.

The display device 34 is connected to the image capturing device 33 to read and reproduce the stored image data,
The current perspective image Vi is displayed as it is. Display device 3
Reference numeral 4 is connected to the map storage device 25 and reproduces topographical information stored in the recording medium.

FIG. 29 shows a flowchart of the local position ascertaining device LP4 according to the present embodiment. This flowchart is basically the same as the flowchart according to the third embodiment in FIG. The difference is that the alarm block # 31 and the ECU operation block # 33 in FIG.
Only the image storage block # 35 and the display block # 37 are replaced. Since the basic operation has been described with reference to the block diagram of FIG. 28, further description will be omitted.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a local positioning system incorporating a local positioning device of the present invention.

FIG. 2 is a block diagram showing a localization device according to the first embodiment of the present invention;

FIG. 3 is a perspective image Vi according to the first embodiment of the present invention.
Explanatory diagram showing

FIG. 4 is a perspective image Vi according to the first embodiment of the present invention.
Explanatory diagram showing an edge pixel image Vx extracted from

FIG. 5 is an explanatory diagram showing a method of extracting a contour from the edge-extracted perspective image Vx according to the first embodiment of the present invention.

FIG. 6 is an explanatory diagram showing a contour extraction perspective image Vc according to the first embodiment of the present invention.

FIG. 7 is an explanatory diagram showing a difference in a coordinate system between the contour extraction perspective image Vc and the bird's-eye image according to the first embodiment of the present invention.

FIG. 8 is an explanatory diagram showing a bird's-eye view image Vcc obtained by performing coordinate conversion on the contour extraction perspective image Vc according to the first embodiment of the present invention.

FIG. 9 is a bird's-eye view image Vc according to the first embodiment of the present invention.
Explanatory diagram showing an example of a curve pattern obtained by Hough transform of c data Scc.

FIG. 10 is a bird's-eye image V according to the first embodiment of the present invention.
Explanatory drawing showing another example of a curve pattern obtained by Hough transforming cc data Scc.

FIG. 11 is an explanatory diagram illustrating arc matching detection according to the first embodiment of the present invention.

FIG. 12 is an explanatory diagram showing current position detection according to the first embodiment of the present invention.

FIG. 13 is an explanatory diagram showing a method of obtaining a reference point for limiting an area according to the first embodiment of the present invention;

FIG. 14 is an explanatory diagram showing how to determine the area width of the area limitation according to the first embodiment of the present invention.

FIG. 15 is an explanatory diagram showing a perspective image Vi having a limited area according to the first embodiment of the present invention;

FIG. 16 is a flowchart showing the operation of the local position grasping device according to the first embodiment of the present invention;

FIG. 17 is a flowchart illustrating an operation of coordinate conversion according to the first embodiment of the present invention.

FIG. 18 is a flowchart showing an operation of matching detection according to the first embodiment of the present invention.

FIG. 19 is a block diagram showing a local position grasping device according to a second embodiment of the present invention;

FIG. 20 is an explanatory view showing a perspective image Vi of a curved lane according to the second embodiment of the present invention;

FIG. 21 is an explanatory diagram showing the inclination of the contour line in FIG. 20;

FIG. 22 is an explanatory view showing a perspective image Vr in which a region is limited according to the second embodiment of the present invention;

FIG. 23 is an explanatory view showing the operation of the current position detector according to the second embodiment of the present invention.

FIG. 24 is a flowchart showing the operation of the local position grasping apparatus according to the second embodiment of the present invention;

FIG. 25 is a flowchart showing an operation of tilt matching detection according to the second embodiment of the present invention;

FIG. 26 is a block diagram showing a local location grasping apparatus according to a third embodiment of the present invention;

FIG. 27 is a flowchart showing the operation of the local location grasping apparatus according to the third embodiment of the present invention.

FIG. 28 is a block diagram showing a localization device according to a fourth embodiment of the present invention;

FIG. 29 is a flowchart showing the operation of the local position grasping device according to the fourth embodiment of the present invention;

FIG. 30 is a block diagram showing a conventional local position grasping device.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 edge extractor 3 threshold setting device 5 contour extractor 7 coordinate converter 9 matching detector 11 lane contour extractor 13 area limiter 15 current position detector 17 curvature detector 19 yaw angle detector

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-8-254432 (JP, A) JP-A-6-333192 (JP, A) JP-A-7-141599 (JP, A) JP-A-6-141599 225308 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G08G 1/16 G01C 21/00 G05D 1/02 G06F 15/62 G06F 15/70

Claims (10)

(57) [Claims] 1. A local position detecting a local position of the vehicle with respect to a lane on the local area based on a digital image signal representing a perspective view of the local area ahead or behind the vehicle. A contour extracting device for extracting contours of the lane from the digital image signal to generate contour data, and converting the contour data to represent a plan view; And a dimensional characteristic detecting means for detecting the extracted contour dimensional characteristic based on the coordinate conversion data and generating dimensional data, the dimensional characteristic detecting means comprising: The feature detecting means includes matching means for matching any one of a straight line and a circular arc with each of the extracted contours to generate matching data indicating each of the matched lines, and determining a current position based on the matching data. A local position grasping device comprising a current position detecting means for detecting. 2. A line indicating the lane among the matched lines.
A contour line extracting means for selectively extracting a book and generating dimension data indicating these two selected matched lines; and determining a current position based on the selected two matched lines. The local position grasping device according to claim 1, further comprising a current position detecting means for detecting. 3. The local position according to claim 1, further comprising: first area limiting means for designating a first area for extracting the contour in the image signal based on the dimension data. Holding device. 4. The apparatus according to claim 1, further comprising a yaw angle detection unit configured to detect a tilt in a traveling direction with respect to the lane direction based on the data of the dimension and generate a yaw angle signal. A local position grasping device according to claim 1. 5. The matching means performs a Hough transform of the coordinate transformation data, and each of the extracted contours is
The local position grasping device according to any one of claims 1 to 4, wherein it is determined whether the position matches a straight line or an arc. 6. When the plurality of lines obtained by the Hough transform intersect at a point more than a predetermined frequency, the matching means determines that the extracted contour is a straight line, and 6. The local position grasping device according to claim 1, wherein in the case, the contour is determined to be a circular arc. 7. The local position grasping device according to claim 1, further comprising a curvature detecting means for detecting a curvature of the lane at a current position based on the dimension data. apparatus. 8. A current position detecting means for detecting a current position based on the digital image data and generating a current position signal;
Based on the digital image data, the curvature of the lane at the current position is detected, a curvature detection unit that generates a curvature signal, and based on the current position signal and the curvature signal, the detected curvature and the current position, The local position grasping device according to claim 1 or 2, further comprising a navigation device that records the map data stored therein. 9. A moving speed detecting means for detecting a moving speed of a vehicle, and a safe driving judging means for judging whether or not the lane having the detected curvature can be safely driven at the detected moving speed. The local position grasping device according to claim 8, further comprising: 10. An image capture that captures the perspective image included in the digital image signal and enables the navigation device to record the captured image in the map data with respect to the detected curvature and current position. 10. The localization device according to claim 9, further comprising means.