JP2009140192A - Road white line detection method, road white line detection program and road white line detection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a road white line detection method for accurately detecting a load white line. <P>SOLUTION: An observed value m(j) at time t<SB>i</SB>, initial values of an estimated value X<SB>d</SB>(j) and error covariance C<SB>xd</SB>(j) at time t<SB>i</SB>, and Kalman gain Kd(j) at time t<SB>i</SB>are found out (steps S107, S108, S109). Then, an estimated value Xd(j) at time ti is updated based on the observed value m(j) and the Kalman gain Kd(j) (step S110), and the error covariance Cxd(j) at time ti is updated based on the Kalman gain Kd(j) (step S111). Then, the steps S109 to S111 are repeatedly executed a plurality of times by using the updated estimated value Xd(j) and error covariance Cxd(j) to lead out an estimated value Xd(j) as the positional coordinates of a road white line at time ti (step S112). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a road white line detection method, a road white line detection program, and a road white line detection device for detecting a road white line.

  Automobiles are one of the most familiar and indispensable means of transportation in modern society. Car navigation systems are widely used to support the driving of such vehicles, and the number of units sold is currently over 4 million units per year. However, with the widespread use, the number of traffic accidents caused by drivers driving too close to the navigation screen while driving and driving aside is increasing year by year. Therefore, in order to prevent the navigation screen from being watched too much, efforts are being made to improve the visibility of the navigation screen. As one of such efforts, for example, development of a direct visual navigation system (VICNAS) that performs navigation using live-action images taken with an in-vehicle camera has been actively conducted. With this VICNAS being put into practical use, the driver can intuitively associate with the real world just by looking at the navigation screen at a glance, thus reducing traffic accidents resulting from the use of car navigation. Many people expect what they can do.

  In order to construct this VICNAS, it is necessary to grasp the exact position of the host vehicle in real time and to project the virtual object on the real image on the correct position. However, in the GPS used in the current car navigation system, GPS data is output from the GPS satellite only at a cycle of 1 second. For example, GPS data is output using a 3-axis gyro sensor or a vehicle speed sensor. It is desirable to obtain the exact position and azimuth angle of the vehicle during a period when the vehicle is not in use (see Non-Patent Document 1).

Z. Hu, K. Uchimura, “Fusion of Vision, GPS and 3D Gyro Data in solving Camera Registration Problem for Direct Visual Navigation”, International Journal of ITS, vol.4, No.1, December 2006

  However, even when the position and azimuth of the host vehicle are obtained using a three-axis gyro sensor, a vehicle speed sensor, or the like, the data still contains a large error. Therefore, it is required to introduce a new measure for reducing these errors to such an extent that a virtual object can be arranged at a correct position on a real image.

  At present, as one of the promising measures for solving this problem, a measure for estimating the position and azimuth of the host vehicle by detecting a white road line is drawing attention. In the conventional detection method, the road white line is detected by using the luminance value of the live-action image, so the processing is very simple and high speed. However, in the situation where there are many edges other than the white line such as road surface display and pedestrian crossing on the road image, it is difficult to recognize the stable white line, and there is a problem in terms of accuracy. In addition, there is a problem in that the road white line may not be detected well even when another vehicle blocks the road white line or the luminance value changes due to bad weather.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a road white line detection method, a road white line detection program, and a road white line detection apparatus capable of accurately detecting a road white line.

The road white line detection method according to the present invention includes the following steps (A1) to (A8).
(A1) A step of extracting data about nodes, road links, interpolation points, and road attributes of road links from map data based on the position and azimuth of the mobile body obtained from sensor output data (A2) The road link extracted from the map data is divided into straight steps, arc steps and relaxation curve steps connected to them based on road attributes, and then divided into multiple sections, and the road horizontal curvature is obtained for each divided section. Step (A3) After extracting the edge line segment from the edge image data obtained by performing predetermined image processing on the photographed image data, the extracted edge line segment is divided into a plurality of sections, and for each divided section Step for obtaining edge horizontal curvature (A4) The road horizontal curvature is compared with the edge horizontal curvature to correspond to the road white line in the edge image data. A step of determining a search range for wedge line segments (A5) A step of selecting an edge line segment based on a predetermined search condition within the search range and acquiring coordinates corresponding to the selected edge line segment as observation values (A6) The coordinates corresponding to the virtual road white line obtained when the line segment obtained from the road horizontal curvature is projected on the edge image data are acquired as the initial value of the estimated value, the lane width, the offset width from the road white line, A step of obtaining the initial value of the error covariance obtained using the standard error of each of the deflection angle and the depression angle, and further calculating the Kalman gain using the estimated value (A7) Based on the observed value and the Kalman gain Updating the estimated value and updating the error covariance based on the Kalman gain (A8) updating the Kalman gain using the updated estimated value. Later, with further updates the estimated value based on the Kalman gain updating the observed values, the step of further updating the error covariance based on the Kalman gain updated

The road white line detection program according to the present invention causes a computer to execute the following steps (B1) to (B8).
(B1) A step of extracting data about nodes, road links, interpolation points, and road attributes of the road links from the map data based on the position and azimuth of the mobile body obtained from the sensor output data (B2) The road link extracted from the map data is divided into straight steps, arc steps and relaxation curve steps connected to them based on road attributes, and then divided into multiple sections, and the road horizontal curvature is obtained for each divided section. Step (B3) After extracting the edge line segment from the edge image data obtained by performing predetermined image processing on the photographed image data, the extracted edge line segment is divided into a plurality of sections, and for each divided section Step (B4) of obtaining edge horizontal curvature in step S4 is performed by comparing the road horizontal curvature with the edge horizontal curvature to correspond to the road white line in the edge image data. A step of determining a search range for the wedge line segment (B5) A step of selecting an edge line segment based on a predetermined search condition within the search range, and acquiring coordinates corresponding to the selected edge line segment as an observed value (B6) The coordinates corresponding to the virtual road white line obtained when the line segment obtained from the road horizontal curvature is projected onto the edge image data are acquired as initial values of the estimated value, the lane width, the offset width from the road white line, A step of obtaining the initial value of the error covariance obtained using the standard errors of the deflection angle and the depression angle, and further calculating the Kalman gain using the estimated value (B7) Based on the observed value and the Kalman gain Updating the estimated value and updating the error covariance based on the Kalman gain (B8), updating the Kalman gain using the updated estimated value. Later, with further updates the estimated value based on the Kalman gain updating the observed values, the step of further updating the error covariance based on the Kalman gain updated

The road white line detection apparatus according to the present invention has the following configurations (C1) to (C8).
(C1) Extraction means (C2) for extracting data about nodes, road links, interpolation points, and road attributes of road links in the vicinity of the moving body from map data based on the position and azimuth of the moving body obtained from the sensor output data ) After dividing the road links extracted from the map data into straight steps, arc steps and relaxation curve steps connected to them based on road attributes, it is divided into multiple sections, and the road horizontal curvature is calculated for each divided section. Obtaining road horizontal curvature derivation means (C3) After extracting edge line segments from edge image data obtained by performing predetermined image processing on real image data, the extracted edge line segments are divided into a plurality of sections. Edge horizontal curvature deriving means for obtaining edge horizontal curvature for each divided section (C4) By comparing road horizontal curvature and edge horizontal curvature, edge image data In the search range determining means (C5) for determining the search range for the edge line segment corresponding to the road white line, the edge line segment is selected based on a predetermined search condition within the search range, and the selected edge line segment is supported. Observation value acquisition means for acquiring the obtained coordinates as observation values (C6) The coordinates corresponding to the virtual road white line obtained when the line segment obtained from the road horizontal curvature is projected onto the edge image data are acquired as initial values of the estimated values. In addition, the lane width, the offset width from the road white line, the standard error of the deflection angle and the depression angle are obtained as the initial value of the error covariance, and the estimated value is used to calculate the Kalman gain. Initial value acquisition means (C7) First update means for updating the estimated value based on the observed value and the Kalman gain, and updating the error covariance based on the Kalman gain ( 8) After updating the Kalman gain using the updated estimated value, the estimated value is further updated based on the observed value and the updated Kalman gain, and the error covariance is further updated based on the updated Kalman gain. 2 Update means

  In the road white line detection method, the road white line detection program, and the road white line detection device according to the present invention, the road link extracted from the map data is divided into a straight line stage, an arc stage, and a relaxation curve stage connected thereto based on the road attributes. After that, it is divided into a plurality of sections, and the road horizontal curvature is obtained for each of the divided sections. Thereby, the road horizontal curvature can be easily extracted from the map data. Further, since the estimated edge horizontal curvature can be associated with the road horizontal curvature, it can be used to estimate the current position.

  According to the road white line detection method, the road white line detection program, and the road white line detection device according to the present invention, the road horizontal curvature can be easily extracted from the map data, and the estimated edge horizontal curvature is associated with the road horizontal curvature. Therefore, it is possible to detect the road white line with high accuracy.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic configuration of a navigation apparatus 1 according to an embodiment of the present invention. Note that the white line detection method according to the embodiment of the present invention is embodied by the operation of the navigation device 1 and will be described below.

  The navigation device 1 is mounted on a moving body (for example, an automobile) driven by a driver. For example, the sensor input unit 10, the map management unit 20, the central processing unit 30, the video display unit 40, and the like. As the main part.

  The sensor input unit 10 includes a CCD (solid-state imaging device) camera 101 (on-vehicle camera), a GPS sensor 102, a gyro sensor 103 (inertial sensor), and a vehicle speed sensor 104.

  The CCD camera 101 is fixed to a moving body so that, for example, a scene in front of the vehicle can be photographed (captured) at a camera angle substantially similar to the line of sight seen by the driver through the windshield. The CCD camera 101 is of a monocular system having a fixed focal length, for example, and captures a scene in front of the road, for example, every 1/30 seconds, and the real image data obtained by the imaging is processed by the central processing unit 30. To be transferred to.

  The GPS sensor 102 is a receiver that receives radio waves transmitted toward the ground surface from a number of GPS satellites launched into space, for example, in a satellite positioning system (GPS) for checking the current position on the earth. Various data (GPS data) included in the received radio wave is transferred to the central processing unit 30. The central processing unit 30 can calculate the position (latitude and longitude) of the moving body and the azimuth angle of the moving body from the GPS data.

  For example, radio waves including GPS data are output from the GPS satellites every second, but the radio waves are affected by the ionosphere before reaching the surface of the earth and are uncertain for 1 to 2 seconds. Reach the receiver with sexuality. Here, an inexpensive commercial GPS has a position (latitude and longitude) accuracy of about 20 meters and an azimuth angle accuracy of about 10 degrees.

  Instead of receiving radio waves transmitted from GPS satellites, DGPS (Differential GPS) may receive medium waves and terrestrial waves such as FM broadcasts transmitted from a reference station. This DGPS is a standard station where the position is known, performs GPS surveying, and transmits the difference between the actual position and the position calculated by GPS using terrestrial waves such as medium waves and FM broadcasts. It is a system that can correct the result measured by the signal, and can measure with higher accuracy than GPS.

  The gyro sensor 103 is composed of, for example, a three-dimensional vibration gyro that electrically detects vibration caused by Coriolis force, and transfers the detected vibration data to the central processing unit 30. Note that the central processing unit 30 can calculate the angular velocity of the moving body from the obtained vibration data, for example, every 1/60 seconds.

  The vehicle speed sensor 104 is a pulse sensor that outputs a pulse signal corresponding to the magnitude of the vehicle speed, for example, and transfers the output pulse signal to the central processing unit 30. The central processing unit 30 can calculate the vehicle speed of the moving body from the obtained pulse signal every 1/50 seconds, for example.

  The map management unit 20 includes a recording medium 201 on which map data of a predetermined geographical area is recorded, and a driver 202 that reads map data from the recording medium 201. The map management unit 20 reads map data (for example, data related to roads included in a predetermined area, data associated with the roads, etc.) from the recording medium 201 in accordance with a read command from the central processing unit 30, and The data is transferred to the processing unit 30.

  Here, the road-related data includes, for example, intersections, branch points, interpolation points (inflection points), interpolation points, etc. (latitude, longitude, sea level) and road link roads connecting nodes. Contains attributes. Road attributes refer to grades (numbers assigned to national roads, prefectural roads, agricultural roads, etc.), the number of lanes, lane width, maximum speed, and the like. Further, the data associated with the road includes, for example, positions, shapes, sizes, icons, and the like of landmarks, railway stations, hospitals, and gas stations.

  The central processing unit 30 includes, for example, a positioning processing module 301, a white line detection module 302, and an image processing module 303 as main parts.

  The positioning processing module 301 calculates the position, azimuth, angular velocity, and vehicle speed of the moving body from various data and pulse signals input from the sensor input unit 10, and calculates the position, azimuth, angular velocity, and vehicle speed of the moving body. This is a device that performs calculations for specifying the position and azimuth angle of the moving body based on the road-related data read from the map management unit 20. The method for specifying the position and azimuth of the moving body will be described in detail later.

  The white line detection module 302 is based on the position and azimuth angle of the moving body specified by the positioning processing module 301, the actual image data obtained from the CCD camera 101, and the road-related data read from the map management unit 20. Thus, the device performs a calculation for detecting a road white line included in the photographed image data and a calculation for correcting the position and azimuth of the moving body specified by the positioning processing module 301. A method for detecting a road white line will be described in detail later.

  Based on the position and azimuth angle of the moving body corrected by the white line detection module 302, the image processing module 303 superimposes the data associated with the road as an icon or the road white line on the real image data. It is a device that performs an operation for generating the synthesized image data.

  The video display unit 40 is composed of, for example, a liquid crystal display device, and displays the image data generated by the image processing module 302 on the screen.

  Next, a method for specifying the position and azimuth angle of the moving body will be described.

2 (1), (2), and (3) schematically show the timing at which data is output from each sensor. Figure 2 (1), (2), as shown in (3), GPS data is output from the GPS sensor 102 with a period _{T g,} gyro data is output from the gyro sensor 103 in the period _{T w,} the vehicle speed data is output from the vehicle speed sensor 104 in the period _{T v,} the photographed image data is output from the CCD camera 101 with a period _{T m.} Here, the period _{T v} of the cycle Tw and the vehicle speed data of the gyro data is shorter than the period _{T g} of the GPS data. Therefore, the positioning processing module 301 estimates the position and azimuth angle of the moving body using the gyro data and the vehicle speed data while the GPS data is not received. Also, the timing at which data is output from each sensor and camera is generally not synchronized with each other. Therefore, when the output timing from each sensor is not synchronized with each other, the positioning processing module 301 determines the position of the moving body in accordance with the output timing time of the gyro sensor 103, the vehicle speed sensor 104, or the CCD camera 101, for example. And estimate the azimuth. In the present embodiment, it is assumed that the position and azimuth angle of the moving body are estimated in accordance with the output timing of the CCD camera 101.

FIG. 3 schematically shows how the position and azimuth of the moving body change from moment to moment. FIG. 3 illustrates a state in which the displacement of the moving body at time t ₀ , t ₁ , t ₂ in FIGS. 2 (1) to 2 (4) is viewed from the height direction. The positioning processing module 301 can directly obtain the position (x _g (t ₀ ), y _g (t ₀ )) and the azimuth (ori _g (t ₀ )) of the moving object at time t ₀ from the GPS data. . On the other hand, the position (x _g (t ₁ ), y _g (t ₁ )) and azimuth (ori _g (t ₁ )) of the moving body at time t ₁ cannot be obtained directly from the GPS data. Therefore, the positioning processing module 301, Delta] t (= Time _{t 1} - time _{t 0)} on the vehicle speed of the moving body obtained from the vehicle speed data v _{(t 0)} with calculating the moving distance [Delta] d _{(t 0)} by multiplying the Then, after obtaining the displacement amount Δori (t ₀ ) of the azimuth angle by multiplying the angular velocity w (t ₀ ) of the moving body obtained from the gyro data by Δt, the movement distance Δd (t ₀ ) is calculated as (ori _g ( t ₀ ) + Δori (t ₀ )) (= ori _g (t ₁ )) to be decomposed into an x component and a y component, whereby the position (x _g (t ₁ ), y of the moving object at time t ₁ is obtained. _g (t ₁ )) and azimuth (ori _g (t ₁ )) are estimated. What generalized these is shown to the following formulas 1-3.

Thus, in the positioning processing module 301, the position (x _g (t ₀ ), y _g (t ₀ )) and azimuth (ori _g (t ₀ )) of the moving body at time t ₀ obtained directly from GPS data. ) as the starting point the position of the moving object at time _{t i} the _{((x g (t i)} , y g (t i)) and azimuth _(ori g _(t i)), using the gyro data and vehicle speed data To estimate.

Furthermore, the position ((x _g (t _i ), y _g (t _i )) and azimuth (ori _g () of the moving object at time t _i obtained using GPS data, gyro data, and vehicle speed data. by performing the map matching processing for t _i)), may be obtained the position and azimuth of the moving body on the road.

  In the map matching process, for example, the generated road shape model is compared with various information of the moving object, and the moving object is most likely to actually exist based on the comparison result. This is a process for forcibly moving the position of the moving body so that the body exists. The above road shape model searches, for example, data related to the road ahead of the moving object position based on the position and azimuth of the moving object, and determines the horizontal shape of the road based on the data related to the road obtained by the search. It can be generated by estimating the parameters of a clothoid curve model, which is a mathematical model to represent.

  Next, a method for detecting a road white line will be described.

4 and 5 show procedures for detecting a road white line. 6A schematically shows a road linear model for detecting a road white line, and FIG. 6B shows a road horizontal curvature C _hm of the road linear model of FIG. _6A . Is. FIG. 7 shows edge image data, FIG. 8 shows a road white line search range S in the edge image data of FIG. 7, and FIG. 9 shows center line segments W ₁ and W ₂ extracted from the edge image data of FIG. FIG. 10 shows virtual road white lines E ₁ and E ₂ , estimated value Xd (j) and error covariance C _xd (j) (C ₁₁ , C ₁₂ , C ₂₁ , C ₂₂ ) are each schematically represented.

First, the white line detection module 302, based on the position and azimuth of the moving object at time t _i identified by the positioning processing module 301, the node N of the moving body neighborhood, road link L, the interpolation point (inflection point) P Then, the position data of the inflection point Q (see FIG. 6) and the attribute of the road link L are extracted from the map data, and based on these data, the road level in front of the moving object (viewing range of the CCD camera 101) at the time t _i . The curvature C _hm is estimated (step S101).

Specifically, first, the road link extracted from the map data _{L (L 1, L 2)} , the linear stage on the basis of the road attribute and road structure Ordinance L _A, the arc stage L _B and the relaxation curve stage L _C Divide. Incidentally, transition curve stage L _C is one which is approximated by clothoid curve smoothly connecting the circular arc stage L _B linear segment L _A. Next, the road horizontal curvature C _hm of the straight line stage L _A , the circular arc stage L _B, and the relaxation curve stage L _C is obtained. At this time, the roads horizontal curvature _{C hm,} and 0 (zero) in the linear stage _{L A, 1 /} r i in the arc stage _{L B (r i} is the radius of curvature), and the linearly in the transition curve stage _{L C} It shall change. Next, the road link L (L ₁ , L ₂ ) is divided into fine sections, and the road horizontal curvature C _hm is obtained for each section. That is, each road link L (L ₁ , L ₂ ) is provided with a plurality of discretized road horizontal curvatures C _hm . After _obtaining the road horizontal curvature C _hm of each road link L (L ₁ , L ₂ ) in this way, the positioning processing module is based on the position of the moving body at the time t _i specified by the positioning processing module 301. The road horizontal curvature C _hm within the visual field range in the direction of the azimuth angle of the moving object at time t _i specified at 301 is extracted. At this time, the visual field range is divided into a plurality (for example, 40) at equal intervals in the direction of the azimuth angle of the moving body, and the road horizontal curvature C _hm is assigned to each _divided area. Accordingly, the road horizontal curvature C _hm is extracted as a set of road horizontal curvatures C _hmx (x = 1, 2,..., N) allocated to each area within the field of view.

  Next, R image data is generated by extracting the R component from the photographed image data at time ti (step S102), and then a filter for extracting the left edge and the right edge is applied to the R image data. Edge image data is generated by performing the conversion process (step S103). Note that it is not necessary to generate the R image data from the entire photographed image data, but only the lower region including the farthest portion of the road (the portion to fade out) in the photographed image data, that is, the region where the road can exist in the photographed image data. It is sufficient to generate from It is also possible to generate edge image data using other methods.

Next, Hough transformation and inverse transformation are performed to detect left edge line segments W _L1 and W _L2 and right edge line segments W _R1 and W _R2 from the edge image data (step S104, see FIG. 7). In FIG. 7, the left edge line segments W _L1 and W _L2 and the right edge line segments W _R1 and W _R2 are all continuous line segments. However, the road white lines are temporarily blocked in the actual image data. Left edge line segments W _L1 , W _L2, and the like when there is a moving object (other moving objects, signs, etc.) or when the edge of the road white line in the photographed image data is unclear due to bad weather or the like In the right edge line segments W _R1 and W _R2 , a portion corresponding to the portion shielded by the shielding object or the unclear portion of the edge is generated. In addition, when there are many edges other than white lines such as road surface display and pedestrian crossing on the actual image data, there are many other than the left edge line segments W _L1 and W _L2 and the right edge line segments W _R1 and W _R2 . Edge line segments are detected.

Next, the edge image data is divided into a plurality of sections S _j by a plurality of virtual line segments L _j (0 ≦ j ≦ n), and left edge line segments W _L1 and W _L2 and right edge line segments W _R1 and W _R2. The edge horizontal curvature C _hej for each section S _j is estimated in consideration of the posture (deflection angle ψ, depression angle α) of the CCD camera 101 (see step S105, FIG. 7). That is, a plurality of discretized edge horizontal curvatures C _hej are given to the left edge line segments W _L1 and W _L2 and the right edge line segments W _R1 and W _R2 . At this time, the virtual line segments _Lj are arranged at narrower intervals in the edge image data so as to be equidistant (for example, 1 m intervals) in the real space. FIG. 7 illustrates a case where ten virtual line segments L _j are arranged. From the viewpoint of ease of searching for road white lines, the number of virtual line segments L _j is larger than that ( For example, it is preferable to increase the number of sections S _j by 40).

Next, by comparing the edge horizontal curvature C _hej and the road horizontal curvature C _hm for each section S _j , a range (search range S) for searching the edge line segment corresponding to the road white line in the edge image data is determined ( Step S106, see FIG. 8). For example, the left edge line segment W _L1 , W _L2 and the right edge line segment W _R1 , W _R2 and the left edge line segment so that the overall positional shift between the road obtained from the road horizontal curvature C _hm is as small as possible. When a road obtained from the road horizontal curvature C _hm is projected onto a region including W _L1 , W _L2 and right edge line segments W _R1 , W _R2 , a region where the road R can be projected with respect to the edge image data The search range is S.

Next, in the road white line search range S, edge line segments (road white line candidates) corresponding to the road white line are predetermined from the left edge line segments W _L1 and W _L2 and the right edge line segments W _R1 and W _R2. , The center line W _{1 of} the selected left edge line segment W _L1 and the selected right edge line segment W _R1 , and the selected left edge line segment W _L2 and the selected right edge line segment. After _obtaining the central line segment W ₂ with _WR2 , the coordinates m _1j and m _2j of the intersections of the central line segments W ₁ and W ₂ and the virtual line segment L _j are observed values m (j) at time t _i . (See step S107, FIG. 9).

Since there are many edges other than white lines such as road surface display and pedestrian crossing on the actual image data, there are many other than the left edge line segments W _L1 and W _L2 and the right edge line segments W _R1 and W _R2 . If edge line segments are detected, if the above search conditions are not good, many line segments other than the left edge line segments W _L1 and W _L2 and the right edge line segments W _R1 and W _R2 are selected. Become. Therefore, it is preferable to set a search condition so that only a desired line segment can be selected well. However, if the search condition is too strict, there is a possibility that the desired line segment is also excluded from the selection symmetry. is there. Therefore, it is more preferable that the search range S is made sufficiently small so that unnecessary edge line segments are not included in the search range S before applying the search condition to the edge image data. However, in the present embodiment, the search range S is set by comparing the edge horizontal curvature C _hej and the road horizontal curvature C _hm , so the ratio of the search range S to the edge image data is sufficiently small. Unnecessary edge line segments are almost eliminated.

Next, the virtual road white lines E ₁ and E ₂ obtained when the line segment obtained from the road horizontal curvature C _hm is projected onto the two-dimensional coordinate plane (for example, edge image data) of the CCD camera 101, and the virtual line segment L _j intersection coordinates _u 1j _{with, u 2j} with give as an initial value at time _{t i} of the estimated value _X d (j), the lane width L, the deflection angle of the offset width _X 0, CCD camera 101 from the left lane marker ψ The values obtained by using the standard errors S _e (j) of the depression angle α and the road vertical curvature C _l of the CCD camera 101 are given as initial values of the error covariance C _xd (j) at time t _i (step). S108, see FIG. 10). The estimated value X _d (j), the coordinates u _1j , and the coordinates u _2j are represented by the following numbers (4) to (6), and the error covariance C _xd (j) is represented by the following number (7 ). Further, when the error covariance C _xd (j) is obtained, the road vertical curvature C _l can be omitted because the influence of the road vertical curvature C _l is small.

Here, the number 4 number 6, _{E v} is the focal length in the x direction of the CCD camera 101, _{E u} points to the focal length of the y direction of the CCD camera 101, respectively.

The basic expression of the extended Kalman filter shown in equations 5 and 6 can be obtained as follows. As shown in FIG. 11, the current road position is (x _o , z _o ), and the coordinates of the road white line points within N meters ahead of the current position are (x _i , z _i ) (i = 0, 1). , 2,..., N). At this time, if the distance between the point (x _i , z _i ) and the next point (x _{i + 1} , z _{i + 1} ) is sufficiently short (for example, 1 m), the curvature of each point on the road in this part is represented by C _i Can be approximated. Therefore, the deviations dx _i ′ and dz _i ′ from the point (x _{i + 1} , z _{i + 1} ) to the point (x _i , z _i ) can be calculated by the following equation (8).

上記のずれを、（ｘ_ｏ，ｚ_ｏ）を原点とする座標系へ数９を用いて換算する。ここで、θｉは、数１０に示した式で表される。

The above deviation is converted into a coordinate system with the origin at (x _o , z _o ) using Equation 9. Here, θi is expressed by the equation shown in Equation 10.

Thereby, each point coordinate within the viewing distance seen from the current position can be obtained by the following formulas 11 and 12, and the above formulas 5 and 6 can be obtained by projecting them to a two-dimensional plane. .

Then calculated Kalman gain Kd to (j) at time _{t i} using an estimated value Xd (j) (step S109). In this way, the observed value m (j) at time _{t i,} the estimated value _X d (j) and the error covariance _C xd and the initial value at time _{t i} of (j), the Kalman gain at time _{t i} Kd ( j), and as shown in the following formulas (13) to (15), based on the observed value m (j) and the Kalman gain Kd (j), the estimated value Xd ( j) is updated (step S110), and the error covariance Cxd (j) at time ti is updated based on the Kalman gain Kd (j) (step S111).

  Here, Hd is a value such that x (j) = Hd (j) Xd (j). x (j) is an estimated value. m (j) is an observed value.

  Next, using the updated estimated value Xd (j) and error covariance Cxd (j), the above-described steps S109 to S111 are repeatedly executed a plurality of times to obtain the optimum estimated value Xd (j) at time ti. (Step S112). Here, whether or not it is optimal is determined based on, for example, whether or not the error covariance Cxd (j) has converged. In this way, the estimated value Xd (j) as the position coordinate of the road white line at time ti can be obtained.

It should be noted that the edge of the road white line in the live-action image data is unclear due to the presence of an object that blocks the road white line in the live-action image data or bad weather, etc., so the left edge line segments W _L1 and W _L2 and the right edge If the line segments W _R1 and W _R2 are missing, the optimum estimated value Xd (j) cannot be acquired in the area corresponding to the missing, and the detection of the road white line fails. .

Therefore, in such a case, when the other road white line of the two road white lines existing in pairs on the left side and the right side when viewed from the moving body can be detected, Using the distance between the road white lines detected in the frame at time t _i−1 , the position of one road white line in the frame at time t _i is estimated. In addition, when it is not possible to detect both of the pair of two road white lines, the road white line detected in the latest frame among the past frames that have been successfully detected in the pair of two road white lines. The position is estimated as the position of the road white line in the frame at time t _i .

  It should be noted that the optimum estimated value Xd (j) and the error covariance Cxd (j) are obtained when the number of coordinates is small, or the optimum estimated value Xd (j) and the error covariance Cxd (j) are obtained. If the ratio of the obtained estimated value Xd (j) deviating from the actual road white line is large, it is preferable to relax the search condition and detect the road white line again.

As described above, the white line detection module 302 can detect the position of the road white line at time ti, but when detecting the position of the road white line at time ti + 1 and tracking the road white line, time t _i Steps S109 to S112 may be executed using the optimum estimated value Xd (j) and the error covariance Cxd (j) as the initial values at time t _{i + 1} (step S113).

  By the way, in the present embodiment, the camera parameter X (j) can be acquired using the estimated value Xd (j) as the position coordinate of the road white line obtained at the time ti as described above. Therefore, in the white line detection module 302, as shown in Equation 16, the camera parameter X (j) is obtained using the estimated value Xd (j), and the position and azimuth angle of the moving object are determined from the camera parameter X (j). The position of the moving body (especially the position in the road width direction) and the azimuth angle specified by the positioning processing module 301 are estimated and corrected.

Here, the camera parameter X (j) is the lane width L, the offset width X ₀ , the deflection angle ψ, the depression angle α, and the road vertical curvature C ₁ , and as shown in Equation 16, the initial camera parameter X (0) is calculated by considering the amount of change from the initial data of the estimated value Xd (j).

  At this time, the gain K is adjusted so as to update all the parameters when detecting both-side white lines, and only the parameters other than L and α when detecting one-side white lines, and the camera parameters are not updated when the detection of both-side white lines fails.

  The parameter obtained from the white line detection causes a slight fluctuation even when a normal white line is detected, and causes a large fluctuation of the parameter when an error is detected. Such parameter fluctuations not only prevent stable vehicle position estimation, but also lead to jumping of VICNAS display icons. Therefore, an approximate median filter (AMF) method and a probability density function (PDF) are applied in order to stably control the parameters of the lane width L and the depression angle α that are considered to have a small amount of change when traveling on the road. This makes it possible to update parameters at high speed and robustly.

  Here, the AMF method is a method of approximating a median filter (MF) by updating a median value as needed instead of obtaining an average value. When the current acquired parameter observation value is x, the median value of the previous frame is mean, and the amount of parameter update change is n, the median update by AMF is performed as shown in Equation 17 below. .

  It is not necessary to secure a large memory with AMF, and processing efficiency is better than other methods. Moreover, it can be expected to be as stable as a process that takes an average value of several past frames. The error of the acquired parameter from the median value updated by this AMF is controlled stably using PDF. In the present embodiment, parameters are corrected using a Gaussian distribution for the probability density function.

Here, Equation 18 is a parameter correction equation, and Equation 19 is a probability density function. x _new represents newly updated data. x is a parameter observation value obtained from white line detection, μ is an average (in this case, the median mean), and σ is a deviation. As a result, it is possible to stabilize the parameter in the correct detection result, and to suppress rapid fluctuation of the parameter at the time of erroneous detection.

Further, in the present embodiment, the position of the moving body specified by the positioning processing module 301 using the estimated value Xd (j) as the position coordinate of the road white line at time t _i obtained as described above. It is possible to correct (especially the position in the road extending direction).

Specifically, when the moving body has a plurality of positions in the road extending direction (front-rear direction) including the position of the moving body specified by the positioning processing module 301, it is obtained from the road horizontal curvature _Chm. Lines obtained from the virtual road white lines E ₁ and E ₂ obtained when the obtained line segments are projected onto the two-dimensional coordinate plane (for example, edge image data) of the CCD camera 101 at the respective positions, and the estimated value Xd (j). As a whole, the positional deviation with respect to the virtual road white line obtained when the minute is projected onto the two-dimensional coordinate plane (for example, edge image data) of the CCD camera 101 at the position of the moving body specified by the positioning processing module 301 is the smallest. The position of the moving body on which the virtual road white lines E ₁ and E ₂ are projected is obtained, and the position of the moving body is replaced with the position of the moving body specified by the positioning processing module 301. Make corrections.

In particular, road horizontal curvature C _hm virtual road white line E _1, E ₂ is the case in any location in the virtual road white line E _1, E ₂ is two-dimensional coordinate plane projected sufficiently larger than 0 The position of the moving body can be corrected with high accuracy in the road extending direction.

  Next, operation | movement of the navigation apparatus 1 of this Embodiment is demonstrated.

In this moving body position detection apparatus 1, the position (x _g (t ₀ ), y _g (t ₀ )) and azimuth angle (ori) at time t ₀ obtained directly from GPS data in the positioning processing module 301 _g (t ₀ )) as a starting point, the position ((x _g (t _i ), y _g (t _i )) and azimuth angle (ori _g (t _i )) of the moving object at time t _i are obtained from gyro data and is estimated by using the vehicle speed data. Further, if necessary, the position of the moving body in the GPS data, the time t _i obtained by making use of the gyro data and vehicle speed data _{((x g (t i)} , y _g (t _i )) and azimuth angle (ori _g (t _i )) are subjected to map matching processing to obtain the position and azimuth angle of the moving object on the road.

Then, the white line detection module 302, firstly, the observed value m (j) at time _{t i,} the initial value at time _{t i} of the estimated value _X d (j) and the error covariance _C xd (j), the time t A Kalman gain Kd (j) at _i is obtained. Thereafter, the observed value m (j), on the basis of the Kalman gain Kd (j), along with the estimated value Xd at time _{t i} (j) is updated based on the Kalman gain Kd (j), the time _{t i} The error covariance Cxd (j) at is updated, and as a result, an estimated value Xd (j) as the position coordinates of the road white line at time t _i is derived.

  Further, in the image processing module 303, first, the camera parameter X (j) is derived using the estimated value Xd (j) as the position coordinate of the road white line obtained at the time ti obtained by the white line detection module 302. The position and azimuth angle of the moving body are estimated from the camera parameter X (j), and the position and azimuth angle of the moving body specified by the positioning processing module 301 are corrected. Thereafter, the position and azimuth of the corrected moving body, the camera parameters specified by the white line detection module 302, the actual image data obtained from the CCD camera 101, and the road read from the map management unit 20 are attached. Based on the data, the composite image data is generated by superimposing the data associated with the road represented by the icon on the photographed image data. Then, the composite image data generated in the image processing module 303 is displayed on the video display unit 40.

  By the way, in the conventional white line detection method developed for automatic driving or driving assistance by an in-vehicle camera, the white line alignment in the field of view is approximated by a single curve (that is, one horizontal curvature). Therefore, since the error becomes large in the portion where the horizontal curvature greatly changes in the white line alignment, even if the current position is known in such a place, the horizontal curvature for detecting the road white line is determined from the design drawing of the road map ahead. The reference value cannot be extracted. Even when a clear road white line is detected, the estimated road horizontal curvature value cannot be associated with the design value of the map and cannot be used for estimation of the current position.

On the other hand, in this embodiment, the road attributes and road structure Ordinance linear segment based L _A, the arc stage L _B and the relaxation curve stage L _C road link divided into _L (L _1, L ₂₎ fine interval Since the road horizontal curvature C _hm is obtained for each divided section, if the current position is known, the road horizontal curvature C _hm can be easily extracted from the map data. Further, if a clear white road line can be detected, the estimated road horizontal curvature (edge horizontal curvature C _hej ) can be associated with a map design value (road horizontal curvature C _hm ), so that the current position can be estimated. Can be used.

In the present embodiment, the road horizontal curvature C _hm and the edge horizontal curvature C _hej are compared to determine a range (search range S) for searching for a road white line in the edge image data. There is no need to search for road white lines for all areas. Thereby, the detection speed can be increased. Furthermore, as a result of narrowing down the search range S, conditions for selecting candidate road white lines can be relaxed, so that more robust detection can be realized.

In the present embodiment, camera parameters are acquired using an estimated value Xd (j) obtained using an extended Kalman filter. As described above, since the camera parameters include at least the lane width L, the offset width X ₀ , the deflection angle ψ, and the depression angle α, the position of the moving body in the road width direction can be determined from these parameters. It can be determined accurately. Furthermore, the position of the moving body in the road extending direction (traveling direction) can be accurately determined from the edge horizontal curvature C _hej and the road horizontal curvature C _hm .

In the present embodiment, the position and azimuth of the road white line at time ti corrected by the white line detection module 302, the camera parameters specified by the white line detection module 302, and the photographed image obtained from the CCD camera 101 Based on the data and the data associated with the road read from the map management unit 20, the image data is generated by representing the data associated with the road with an icon or image data in which the road white line is superimposed. can do. This gives the driver a sense of security by overlaying the road white line on the live-action image when the front visibility is low, such as when other vehicles are blocking the front or when the front is dark due to nighttime or bad weather. be able to.

  While the present invention has been described with reference to the embodiment, its modifications, and examples, the present invention is not limited to this, and various modifications can be made.

  For example, in the above embodiment, the positioning processing module 301, the white line detection module 302, and the image processing module 303 are described as devices (hardware), but may be programs loaded in the central processing unit 30. Further, the map data recorded in the recording medium 2 may be loaded in the central processing unit 30 in advance.

1 is a schematic configuration diagram of a navigation device according to an embodiment of the present invention. It is a related figure showing the cycle and timing of each input from a sensor input part. It is a top view showing an example of the mode of displacement of a mobile. It is a flowchart for demonstrating the white line detection module of FIG. It is a flowchart following FIG. It is the schematic diagram which represented map data typically. It is the schematic diagram which represented edge image data typically. FIG. 8 is a schematic diagram schematically illustrating a road white line search range in the edge image data of FIG. 7. FIG. 8 is a schematic diagram schematically showing a center line segment and observation values extracted from the edge image data of FIG. 7. It is the schematic diagram which represented typically the virtual road white line obtained from map data, an estimated value, and an error covariance. It is a schematic diagram for demonstrating derivation | leading-out of an extended Kalman filter.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Navigation apparatus, 10 ... Sensor input part, 20 ... Map management part, 30 ... Central processing part, 40 ... Image | video display part. DESCRIPTION OF SYMBOLS 101 ... CCD camera, 102 ... GPS sensor, 103 ... Gyro sensor, 104 ... Vehicle speed sensor, 201 ... Recording medium, 202 ... Driver, 301 ... Positioning processing module, 302 ... White line detection module, 303 ... Image processing module

Claims (4)

Extracting from the map data data about road attributes of nodes, road links, interpolation points and road links in the vicinity of the mobile body based on the position and azimuth of the mobile body obtained from the sensor output data;
The road link extracted from the map data is divided into a linear step, an arc step and a relaxation curve step connected thereto based on the road attribute, and then divided into a plurality of sections, and the road horizontal curvature is divided for each divided section. A step of seeking
After extracting edge line segments from edge image data obtained by performing predetermined image processing on live-action image data, the extracted edge line segments are divided into a plurality of sections, and the edge horizontal curvature is divided for each divided section. A step of seeking
Comparing the road horizontal curvature with the edge horizontal curvature to determine a range for searching for an edge line segment corresponding to a road white line in the edge image data;
In the search range, selecting an edge line segment based on a predetermined search condition, and obtaining coordinates corresponding to the selected edge line segment as an observation value;
The coordinates corresponding to the virtual road white line obtained when the line segment obtained from the road horizontal curvature is projected on the edge image data are obtained as the initial value of the estimated value, and the lane width, the offset width from the road white line, the deflection Obtaining the initial value of the error covariance obtained using the standard error of each of the angle and the depression angle, and further calculating the Kalman gain using the estimated value;
Updating the estimated value based on the observed value and the Kalman gain, and updating the error covariance based on the Kalman gain;
After updating the Kalman gain using the updated estimated value, the estimated value is further updated based on the observed value and the updated Kalman gain, and the error covariance is further updated based on the updated Kalman gain. A method for detecting a road white line, comprising: The position and azimuth angle of the moving body obtained from the sensor output data are corrected using the line segment obtained from the estimated value obtained by the update and the line segment obtained from the road horizontal curvature. The road white line detection method according to claim 1. Extracting from the map data data about road attributes of nodes, road links, interpolation points and road links in the vicinity of the mobile body based on the position and azimuth of the mobile body obtained from the sensor output data;
The road link extracted from the map data is divided into a linear step, an arc step and a relaxation curve step connected thereto based on the road attribute, and then divided into a plurality of sections, and the road horizontal curvature is divided for each divided section. A step of seeking
After extracting edge line segments from edge image data obtained by performing predetermined image processing on live-action image data, the extracted edge line segments are divided into a plurality of sections, and the edge horizontal curvature is divided for each divided section. A step of seeking
Comparing the road horizontal curvature with the edge horizontal curvature to determine a range for searching for an edge line segment corresponding to a road white line in the edge image data;
In the search range, selecting an edge line segment based on a predetermined search condition, and obtaining coordinates corresponding to the selected edge line segment as an observation value;
The coordinates corresponding to the virtual road white line obtained when the line segment obtained from the road horizontal curvature is projected on the edge image data are obtained as the initial value of the estimated value, and the lane width, the offset width from the road white line, the deflection Obtaining the initial value of the error covariance obtained using the standard error of each of the angle and the depression angle, and further calculating the Kalman gain using the estimated value;
Updating the estimated value based on the observed value and the Kalman gain, and updating the error covariance based on the Kalman gain;
After updating the Kalman gain using the updated estimated value, the estimated value is further updated based on the observed value and the updated Kalman gain, and the error covariance is further updated based on the updated Kalman gain. A road white line detection program that causes a computer to execute steps. Extracting means for extracting data about road attributes of nodes, road links, interpolation points and road links in the vicinity of the moving body from map data based on the position and azimuth of the moving body obtained from the sensor output data;
The road link extracted from the map data is divided into a linear step, an arc step and a relaxation curve step connected thereto based on the road attribute, and then divided into a plurality of sections, and the road horizontal curvature is divided for each divided section. Road horizontal curvature derivation means for obtaining
After extracting edge line segments from edge image data obtained by performing predetermined image processing on live-action image data, the extracted edge line segments are divided into a plurality of sections, and the edge horizontal curvature is divided for each divided section. Edge horizontal curvature deriving means for obtaining
A search range determining means for comparing the road horizontal curvature with the edge horizontal curvature to determine a range for searching for an edge line segment corresponding to a road white line in the edge image data;
Within the search range, an edge line segment is selected based on a predetermined search condition, and an observed value acquisition unit that acquires coordinates corresponding to the selected edge line segment as an observed value;
The coordinates corresponding to the virtual road white line obtained when the line segment obtained from the road horizontal curvature is projected on the edge image data are obtained as the initial value of the estimated value, and the lane width, the offset width from the road white line, the deflection Initial value obtaining means for obtaining the initial value of the error covariance obtained using the standard error of each angle and depression angle, and further calculating the initial value of the Kalman gain using the initial value of the estimated value;
Updating the estimated value based on the observed value and the Kalman gain, and updating the error covariance based on the Kalman gain;
After updating the Kalman gain using the updated estimated value, the estimated value is further updated based on the observed value and the updated Kalman gain, and the error covariance is further updated based on the updated Kalman gain. A road white line detection device comprising: a second update means.

Images