JP2005148784A - Detector for traveling lane on road surface - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To stably detect a traveling lane by precisely specifying the position of a mobile body while properly detecting a traveling lane border line without being affected by the position and attitude of an imaging means. <P>SOLUTION: Straight line data are detected from images of a road surface continuously taken by the imaging means VD, and based on the linear data, the traveling lane (not shown) of the mobile body (e.g., vehicle) is detected by a traveling lane detection means LD. A projection transformation is computed based on the straight line data by a projection transformation arithmetic means PT. Based on the computing result by the projection transformation arithmetic means, the position of the imaging means VD on the road surface is specified by an imaging means position specification means ID, and the position of the imaging means VD on the road surface or the position of the mobile body (vehicle) is specified. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a road surface traveling lane detection device for a moving body, and more particularly to a road surface traveling lane detection device that detects a traveling lane of the vehicle (own vehicle) from images obtained by continuously capturing road surfaces ahead of the traveling vehicle.

  For example, in automatic driving control of automobiles, driver assistance control, and the like, it is important to appropriately and stably detect the traveling lane of the vehicle (camera-equipped vehicle) on the road surface from an image captured by a camera. Usually, on the road surface, a lane boundary line (white line or the like) for identifying the boundary of the traveling lane (lane) is painted. Therefore, it detects the driving lane by detecting the left and right lane boundary lines, or detects the left and right lane boundary lines, and adds a certain lane interval to this to identify the other lane boundary line. Thus, the traveling lane can be specified.

  In relation to such a road surface lane detection device, Patent Document 1 discloses a vanishing point (at an infinite point) from an image obtained by photographing a road including an obstacle in order to identify a three-dimensional object from a two-dimensional image. In general, a vanishing point detection device that detects a vanishing point) is disclosed. Here, when the difference between the (tilt) angle of the straight line connecting the vanishing point candidate position and the detected edge point and the edge tilt (angle) at the edge point is smaller than a predetermined threshold, the angle corresponding to the angle of the straight line The frequency is histogrammed, the number of straight lines having the highest frequency is accumulated as a straight line number accumulated image, and the position of the vanishing point is detected from this image.

  Further, Patent Document 2 and Patent Document 3 also disclose vanishing point detection devices similar to Patent Document 1. In the former Patent Document 2, after detecting vertical and horizontal edge components (horizontal / vertical linear components) irrelevant to the vanishing point from the detected edge image in advance by masking means, straight line detection means such as Hough transform is applied. I am going to do that. In the latter patent document 3, in order to eliminate noise and irrelevant edge components, a window region is set for the edge image, and this position is made variable to detect many edge points useful for vanishing point detection. An apparatus is disclosed.

  On the other hand, in Patent Document 4, a camera as an image input means mounted on a moving vehicle detects a displacement of the installation position with respect to the moving vehicle based on its own weight or vibration during traveling, and corrects the displacement of the input image. A moving body image processing apparatus including a correcting unit is disclosed. In Patent Document 4, when a reference point (specifically, a marker) is arranged on a vehicle body (specifically, a bonnet) and the position of the marker in the photographed image is shifted from the reference position, the pan head is A method of controlling and mechanically driving an image sensor (camera) to align the position of a marker in an image and a method of shifting the coordinates of image data itself by software are disclosed.

  Further, Patent Document 5 proposes a traveling path estimation device for reducing the influence of an erroneous white line candidate point (disturbance) in traveling path estimation by image processing. In Patent Document 5, at least two virtual white line candidate points are set for one white line portion behind the host vehicle by the virtual candidate point setting means, and the white line portion of the travel path estimated by the travel path estimation means is described. Since the setting condition of the virtual candidate point is changed by the condition changing means according to the estimated value of the above, the disturbance that affects the estimation without strongly affecting the white line portion of the estimated road It is described that the influence of can be reduced.

  Furthermore, in Patent Document 6, in a traveling road discrimination method for discriminating the shape of a traveling road from image data obtained by imaging a traveling road with a camera, one of the straight lines obtained by image processing is run. A line corresponding to the road edge is determined, a straight line corresponding to the other road edge is predicted from the correspondence between the horizon and the road (road) width, and the line corresponding to the other road edge is determined based on the prediction. A method for extracting a straight line is disclosed. In this patent document 6, the position of the horizontal line in the image is calculated from the installation conditions of the camera on the vehicle. Of the straight line candidates at the left and right ends corresponding to the travel road (road) width, the intersection of two straight lines is on this horizontal line. Are detected as a pair of straight lines corresponding to the ends of the traveling road.

  On the other hand, with regard to image processing technology, Hough transformation is widely known as a straight line detection method, and is described, for example, in Non-Patent Document 1 below. Such a Hough transform is known as a noise detection method that is robust to noise. In the process of converting a point in the (x, y) coordinate system to a curve on the (ρ, θ) polar coordinate system, the (x, y) coordinate system is used. In this case, the curves on the (ρ, θ) coordinate system based on the feature points on the same straight line intersect at one point. Furthermore, in recent years, RANSAC (Random Sample Consensus), which is a kind of robust method, has attracted attention in computer vision, and is described in detail in Non-Patent Document 2 below. Non-Patent Document 3 below also explains RANSAC.

  The following non-patent document 4 describes the vanishing point, the infinity and the projective space, and the projective transformation used in the present invention.

JP-A 63-106875 JP-A 63-106876 JP-A-63-191280 JP-A-1-273113 JP 7-311895 A Japanese Patent Laid-Open No. 2-90380 Supervised by Hideyuki Tamura, “Introduction to Computer Image Processing”, Soken Publishing Co., Ltd., published on March 10, 1985, the first edition, first print, pages 127 and 128 Martin A. Fischero and Robert C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography", Graphics and Image Processing, vol.24 (6), pages 381-395, published in 1981. "Multiple View Geometry in Computer Vision" by Richard Hartley and Andrew Zisserman, Cambridge University Press. August 2000, pp. 101-107 "Computer Vision" written by Satoshi Sato, Corona, October 10, 2001, first edition, 3rd edition, 17-41

  In the above-mentioned Patent Document 1, the vanishing point candidate position is established when a correct value can be confirmed in advance, but cannot be solved when it is unknown. On the other hand, when it is decided to adopt a position that is concentrated as a result without specifying the vanishing point position, the linear component that does not exist in the direction of the traveling road is strong and the linear component of the traveling road is small. Then, the intersections are concentrated at a position different from the original vanishing point position, and the correct vanishing point position cannot be detected. In particular, in an actual natural environment, it is difficult to detect the vanishing point position due to the influence of dirt on the road surface, shadows of buildings and other vehicles, and the like.

  Further, in the masking process in Patent Document 2, since the same operator as the edge detection using the 3 × 3 size operator is used, the processing time becomes long and cannot be put to practical use. Furthermore, in Patent Document 3, it is assumed that edge constituent points that should be detected as straight lines are arranged with high density, and that sparsely scattered edge points should not be detected as straight lines, and the number of edge points in the window is small Is not detected as a straight line, it is not possible to detect the original white line or the like to be detected if edge points are missing due to dirt on the road surface or white line peeling. According to Patent Document 4, when a moving vehicle gets over a road joint or unevenness while traveling, or when the occupant or the loading state changes, the positional relationship between the road surface and the camera is not determined. The purpose of can not be achieved.

  Similarly, in the apparatus described in Patent Document 5 mentioned above, an edge that is not related to the white line is mixed in the scan window due to vibration or movement of the occupant or the loading state when the moving vehicle gets over the seam or unevenness during traveling. Or the white line in the image falls out of the scan window, and the detection accuracy becomes unstable. In Patent Document 6, since the position of the horizon is determined by the installation conditions of the camera on the vehicle, as described above, the vehicle vibrates when it travels over a seam or unevenness while traveling, When the state changes, the position of the horizon changes, and a correct result cannot be obtained.

  As described above, when the vehicle is actually traveling on the road, there are slight steps in the repaired portion of the road surface, the seam of the overpass, and the like. In addition, when a certain period has elapsed since the start of service on the road, the road surface is subtle but uneven. When the vehicle passes through these places at high speed, the vibration of the vehicle body increases, the height of the camera mounted on the vehicle changes from the ground, and the direction of the optical axis also changes.

  However, according to the conventional image processing system including the above-mentioned patent documents, it is assumed that the height of the camera is constant, so that an error occurs when measuring the position and direction of the white line due to the above vibration. In particular, in a system that assumes that the direction of the optical axis of the camera is constant, the error becomes even larger. For this reason, when the width or position of the travel lane deviates from the preset range, the lane boundary line (white line) must be processed as undetected, and the travel lane is stably detected. I can't do that.

  On the other hand, in a three-dimensional driving environment, there are various sets of parallel lines including a driving lane boundary line. If the positional relationship of the imaging means with respect to these parallel lines can be specified, the imaging means is installed. It is possible to appropriately specify the position of the moving body in the travel lane.

  Furthermore, before and after the lane change (lane change), the own vehicle straddles the lane boundary line, so detection of the traveling lane becomes unstable and the position of the own vehicle tends to be inaccurate. In addition, as described above, detection of a traveling lane after a lane change may not be detected stably due to dirt on the road surface or peeling of a white line. Therefore, if it is possible to stably identify not only the traveling lane of the own vehicle but also two adjacent traveling lanes on both sides based on the traveling lane information of the own vehicle, the position of the own vehicle can be stably maintained even during a lane change. Can be specified.

  Accordingly, the present invention provides a road lane detection device that detects a travel lane on a road surface on which a mobile body travels, and is not affected by the position and orientation of an imaging means mounted on the mobile body. It is an object of the present invention to provide a road surface traveling lane detection device that can appropriately detect the boundary line and accurately identify the position of the moving body in the traveling lane and stably detect the traveling lane.

  Further, the present invention provides a road surface lane detection device that detects a travel lane on a road surface on which the mobile body travels, without being affected by the position and orientation of the imaging means mounted on the mobile body. Another object is to configure the traveling lanes adjacent to both sides of the traveling lane so as to be stably detected.

  To achieve the above object, according to the first aspect of the present invention, in the road surface traveling lane detection apparatus for detecting a traveling lane of a moving object from an image obtained by continuously capturing an image of the road surface by an imaging unit, the image Detecting the straight line data from the road, and detecting the travel lane of the moving body based on the straight line data, and extending in the travel direction of the mobile body based on the straight line data detected by the travel lane detecting means. A set of straight lines on the image corresponding to a set of straight lines on the road surface that are parallel to each other, a set of straight lines on the upper and lower ends of the image surface constituting the image, and a straight line on the image are projected on the road surface. A projective transformation calculating means for calculating a projective transformation based on the set of straight lines, and a projection of a straight line set perpendicular to the road surface existing on the road surface to the road surface based on the calculation result of the projective transformation calculating means. Is the position of an intersection obtained by extending a line of the image that was providing the imaging means position specifying means for specifying a position of the imaging means.

  In the above imaging means, internal parameters such as magnification and optical axis position and / or external parameters such as ground height and optical axis direction are unknown, or the external parameters vary from the initial setting state. It may be a thing.

  According to a second aspect of the present invention, in the road traveling lane detection device for detecting a traveling lane of a moving body from an image obtained by continuously capturing an image of a road surface by an imaging unit, the linear data is detected from the image. And a traveling lane detecting means for detecting a traveling lane of the moving body based on the straight line data, and the parallel road surface extending in the traveling direction of the moving body based on the straight line data detected by the traveling lane detecting means. A set of straight lines on the image corresponding to the set of straight lines above, a set of straight lines at the upper and lower ends of the image plane constituting the image, and a set of straight lines when the straight lines on the image are projected onto the road surface A projective transformation calculating means for calculating a projective transformation based on the calculation result, and a position of an intersection obtained by extending a straight line of the projected image on the road surface of a set of straight lines on the left and right ends of the image plane based on a calculation result of the projective transformation calculating means. Said May be that an imaging unit position specifying means for specifying a position of the image means.

  Furthermore, according to a third aspect of the present invention, in the road surface traveling lane detecting device for detecting the traveling lane of the moving body from the images obtained by continuously capturing the road surface by the imaging means, the linear data is detected from the image. And a traveling lane detecting means for detecting a traveling lane of the moving body based on the straight line data, and the parallel road surface extending in the traveling direction of the moving body based on the straight line data detected by the traveling lane detecting means. Projective transformation based on a set of straight lines on the image corresponding to the set of straight lines above, a set of straight lines at the upper and lower ends of the image plane, and a set of straight lines when the straight lines on the image are projected onto the road surface Based on the calculation result of the projective transformation calculation means, and inversely transforming straight lines that are parallel and equally spaced with respect to the set of straight lines existing on the road surface, It may be those with an adjacent lane identification means.

  4. The road surface traveling lane detection device according to claim 1, wherein the traveling lane detection means detects feature point data from a predetermined area of the image picked up this time. Based on the current straight line data detected by the straight line detecting means and the straight line detecting means for detecting the current straight line data representing the straight line most suitable for the current characteristic point data detected by the characteristic point detecting means. Lane boundary line detection means for detecting a lane boundary line, and supplies the current lane boundary line detected by the lane boundary line detection means and the current straight line data detected by the straight line detection means to the feature point detection means. The feature point detection means sets predetermined characteristics based on the current lane boundary line and the current straight line data, and detects the feature point data detected from the next image to be captured. Of the feature data satisfies said predetermined property may be configured to be detected as the next feature point data.

  The feature point detection means includes an ordering means for ordering the feature point data based on detection results of the straight line detection means and the lane detection means, as defined in claim 5, wherein the straight line detection means A data detecting means for extracting a minimum number of data necessary for adaptation of the predetermined mathematical model from the feature point data of the ordering means at predetermined intervals, and a mathematical formula based on the data detected by the data detecting means Formula model calculation means for obtaining a model, data number counting means for obtaining the number of feature point data matching the formula model obtained by the formula model calculation means, and the order of the feature point data set by the ordering means The number of feature point data that is most suitable as a result of repeating part of or all of the feature point data in order and repeating the processing of the data detecting means to the data number counting means Or equal to those with a mathematical model setting means for setting a more mathematical model as a true mathematical model.

  Furthermore, in the road surface traveling lane detection device according to claim 3, as described in claim 6, the adjacent lane specifying unit is configured such that, among the plurality of traveling lanes whose mutual lane interval is a constant distance, An adjacent lane boundary position estimating means for setting a traveling lane adjacent to the traveling lane detected by the traveling lane detecting means as an adjacent lane, and estimating a boundary line position of the adjacent lane by the projective transformation calculating means; and the adjacent lane boundary position Feature point detection area setting means for setting a detection area for detecting feature point data related to the boundary line of the adjacent lane at the position estimated by the estimation means, and feature point detection area set by the feature point detection area setting means A feature point detecting means for detecting a feature point within the feature point, an arbitrary feature point within the feature point detected within the feature point detection area as one point, and another point at the infinity point coordinate And a straight line detecting means for detecting a straight line including both points and a straight line having the most characteristic points within a predetermined range from the straight line with respect to at least one straight line detected by the straight line detecting means. Adjacent lane boundary line specifying means for specifying the boundary line of the adjacent lane may be provided.

  Since this invention is comprised as mentioned above, there exist the following effects. That is, in the apparatus according to claim 1 and 2, the straight line data is detected from the images obtained by continuously capturing the road surface by the image pickup means, the traveling lane of the moving body is detected based on the straight line data, and the straight line data is detected. The projection transformation is calculated based on the calculation result, and the position of the imaging means on the road surface is specified based on the calculation result. it can. In particular, even if the camera that is the imaging means is not calibrated, or the internal parameters and external parameters of the camera are not known in advance, the position of the camera in the driving lane and thus the position of the vehicle that is the moving object is accurately specified. Thus, the traveling lane can be detected appropriately and stably.

  By configuring as described in claim 3, it is possible to stably specify not only the traveling lane of the own vehicle but also two traveling lanes adjacent to both sides thereof. The position can be identified stably.

  Further, if configured as described in claim 4, it functions as the above-mentioned CONSAC and stably detects lanes even in the presence of disturbances such as vehicle shadows, noise barrier shadows, road surface indications, etc. be able to. Further, if the feature point detecting means is configured as described in claim 5, the feature point detection area in the image can be narrowed even when there is no prior information, so that the feature point can be detected quickly. Can be done.

  Further, by configuring as described in claim 6, it is possible to quickly and appropriately specify two traveling lanes adjacent to the traveling lane of the own vehicle based on the traveling lane information of the own vehicle.

  A specific aspect of the road surface traveling lane detection device of the present invention having the above-described configuration will be described below with reference to the drawings. FIGS. 1 to 3 show an embodiment of a road surface lane detection apparatus. In this embodiment, as shown in FIG. 1, linear data is detected from images obtained by continuously imaging a road surface by an imaging means VD. A travel lane detection means LD for detecting a travel lane (not shown) of a moving body (for example, a vehicle) based on the straight line data, and a projection for calculating projective transformation based on the straight line data detected by the travel lane detection means LD. A conversion calculation means PT and an image pickup means position specifying means ID for specifying the position of the image pickup means VD on the road surface based on the calculation result of the projective conversion calculation means PT are provided. The position of (vehicle) can be specified.

  As shown in FIG. 1, the traveling lane detecting means LD is composed of characteristic point detecting means CR, straight line detecting means SD, and lane boundary line specifying means LB, which will be described later. Further, the projective transformation calculation means PT is based on the straight line data detected by the travel lane detection means LD, and the image of the imaging means VD corresponding to a set of straight lines on the parallel road surface extending in the travel direction of the moving body (vehicle). It is configured to calculate the projective transformation based on the set of straight lines on the top, the set of straight lines on the upper and lower ends of the image plane that composes the image, and the set of straight lines when the straight lines on the image are projected on the road surface. Yes. Then, in the imaging means position specifying means ID, based on the calculation result of the projective transformation calculation means PT, the position of the intersection obtained by extending the straight line of the projected image on the road surface of the set of straight lines existing on the road surface is picked up. It is configured to specify the position of VD. These will also be described later.

  The road surface traveling lane detection apparatus of FIG. 1 has the hardware configuration shown in FIG. That is, for example, a CCD camera (hereinafter simply referred to as a camera) CM is mounted as an imaging means VD in front of a vehicle (not shown), and the field of view in front of the vehicle including the road surface is continuously imaged. The video signal of the camera CM is A / D converted through the video input buffer circuit VB and the sync separation circuit SY and stored in the frame memory FM. The image data stored in the frame memory FM is processed by the image processing unit VC. The image processing unit VC includes an image data control unit VP, a feature point detection unit CP, a straight line detection unit SP, a lane boundary line specifying unit LP, a projective transformation calculation unit PP, a camera position specifying unit IP, and an own vehicle position specifying unit JP. Has been. The feature point detection unit CP, the straight line detection unit SP, the lane boundary line specifying unit LP, the projective transformation calculation unit PP, and the camera position specifying unit IP unit are respectively the feature point detection unit CR and the straight line detection unit SD of FIG. This corresponds to the lane boundary line specifying means LB, the projective transformation calculating means PT, and the imaging means position specifying means ID.

  In the image processing unit VC, data addressed by the image data control unit VP is called from the image data in the frame memory FM and sent to the feature point detection unit CP, where a plurality of feature points are detected. . In the present embodiment, for the feature point data detected in this way, in the present embodiment, the straight line detection unit SP detects straight line data by “straight line fitting” described later, and based on this straight line data, the lane boundary line specifying unit LP Thus, the boundary line of the driving lane is specified. In addition, in the projective transformation calculation unit PP, a set of straight lines on the image of the camera CM corresponding to a set of straight lines on the parallel road surface extending in the traveling direction of the vehicle (that is, a travel lane boundary line), and the image Projection transformation is performed based on a set of straight lines at the upper and lower ends of the image plane constituting the image and a set of straight lines when the straight lines on the image are projected on the road surface. Then, the position of the intersection obtained by extending the straight line of the projected image on the road surface of a set of straight lines perpendicular to the road surface existing on the road surface is specified as the position of the camera CM by the camera position specifying unit IP unit. Thus, the position of the camera CM with respect to the boundary line of the traveling lane on the road surface is specified. In addition, as a boundary line of a driving | running lane, a guard rail etc. are included besides what is called a white line.

  Based on the position of the camera CM specified by the camera position specifying unit IP, the vehicle position specifying unit JP specifies the vehicle (vehicle) equipped with the camera CM with respect to the boundary line of the traveling lane on the road surface. In response to the detection result of the driving lane width, road curvature, posture angle, etc., it is supplied to the system controller SC (computer) and output to an external system device (not shown) via the output interface circuit OU. Is done. Note that CL, PW, and IN in FIG. 2 are a clock circuit, a power supply circuit, and an input interface circuit, respectively.

  Hereinafter, processing in each part of the image processing unit VC will be described in detail. First, a description will be given of a process in the projective transformation calculation unit PP that calculates the projection transformation according to the detection result of the lane boundary line specifying unit LP, and the camera position specifying unit IP that specifies the position of the camera CM based on the calculation result. Next, processing in the lane boundary line specifying unit LP will be described.

  In the projective transformation calculation unit PP, according to the following basic concept based on the projective transformation described in Non-Patent Document 4 described above, the camera CM is not calibrated without depending on the internal parameters and external parameters of the camera CM. Even so, it is configured so that the position of the camera CM (and consequently the position of the vehicle) can be specified. Specifically, by using the projection image in the parallel line image, the projective transformation between the image of the camera CM and the road surface (plane) is obtained, and the position of the viewpoint of the camera CM on the road surface is determined as the road surface. The horizontal position of the camera CM with respect to the travel lane can be calculated using the image obtained as an image of an infinite point on a straight line perpendicular to the lane.

In general, a point x = [x ₁ , x ₂ , x ₃ ] ^T on a plane π is changed to a point x ′ = [x ′ ₁ , x ′ ₂ , x ′ ₃ ] on a plane π ′ different from this. If projective transformation to convert to ^T and H _p, straight on _{π l = [l 1, l} 2, l 3] ' line l on' = a _{^{T π [l '1, l}} ' 2, l '3 The projective transformation to be converted into ^T can be expressed as H _l = H _p ^-T . For example, in the case where there is a road surface shown in FIG. 3 and a camera image that is substantially orthogonal to the road surface, straight lines l ₁ and l ₂ that represent two traveling lane boundaries in the image and two straight lines l that represent the upper and lower edges of the image ₃ and l ₄ a and corresponds to four linear l _'i on the road surface (i = 1,2,3,4). At this time, if the straight line l ′ ₁ and the straight line l ′ ₂ are parallel on the road surface, and l ′ ₃ and l ′ ₄ are both orthogonal to l ′ ₁ , then the homogeneous coordinates of these straight lines are Equation 1 is obtained.

The projective transformation H _l for converting the straight line l _i (i = 1, 2, 3, 4) into the straight line l ′ _i (i = 1, 2, 3, 4) is expressed by the following equation. That is, l ′ _{i to} H _l l _i (i = 1, 2, 3, 4) (where “−” represents equality except for indefiniteness of a constant multiple).

On the other hand, there are a plurality of straight lines perpendicular to the road surface in a three-dimensional traveling environment, and two projected images l ₅ and l ₆ of these appear on the screen (two-dimensional image) by the camera CM. If so, images l ′ ₅ and l ′ ₆ obtained by converting these straight lines based on the above-described projective transformation H ₁ are obtained on the road surface. Since the projected images l ₅ and l ₆ are straight lines perpendicular to the road surface in the three-dimensional space, the intersection X _C when the images l ′ ₅ and l ′ ₆ obtained by projective transformation are extended from the viewpoint C to the road surface Coincides with the intersection (position on the road surface of the camera CM) of the straight line and the road surface. This intersection can be obtained as X _C = l ′ ₅ × l ′ ₆ . Accordingly, the position of the camera CM in the travel lane can be specified based on the position of the intersection point X _C obtained in this way with respect to the straight lines l ′ ₁ and l ′ ₂ representing the travel lane boundary line.

When the above relationship is compared with the processing in the image processing unit VC described above, the following is obtained. That is, in the projective transformation calculation unit PP, a set of straight lines (l ₁ on the image of the camera CM) corresponding to a set of straight lines (l ′ ₁ and l ′ ₂ ) on the parallel road surface extending in the traveling direction of the vehicle. And l ₂ ), a set of straight lines (l ₃ and l ₄ ) at the upper and lower ends of the image plane constituting the image, and a set of straight lines (l ′ ₃ and l when the straight lines on the image are projected onto the road surface ' ₄ ) Projective transformation is performed. Then, the position (X _C ) of the intersection obtained by extending the straight lines (l ′ ₅ and l ′ ₆ ) of the projected image onto the road surface of a set of straight lines perpendicular to the road surface existing on the road surface in the camera position specifying unit IP unit Is specified as the position of the camera CM. Thus, the position (X _C ) of the camera CM with respect to the boundary line of the traveling lane on the road surface is specified. As a result, the own vehicle position specifying unit JP determines the traveling lane (that is, the road surface between the two traveling lane boundary lines). ) Can identify the position of the vehicle.

In the above aspect, at least two straight lines perpendicular to the road surface are required in the three-dimensional traveling environment. If such straight lines cannot be obtained, the following configuration is used, and the approximate shape of the driving lane is as follows. The position of the camera CM can be specified. For example, when the camera CM is arranged substantially horizontally, the image plane of the camera CM is substantially perpendicular to the road surface, and the straight lines l ₇ and l ₈ at the left and right ends of the image are substantially perpendicular to the road surface in the three-dimensional space. Accordingly, when the images l ′ ₇ and l ′ ₈ obtained by converting the straight lines l ₇ and l ₈ by the projective transformation H _l are obtained, the intersection X ′ _C of the images l ′ ₇ and l ′ ₈ is approximately the intersection X _C. It is equal to (position on the traveling lane road surface of the camera CM). In the calculation for specifying the position of the camera CM with respect to the traveling lane, only the position in the direction orthogonal to the traveling lane boundary line has a meaning. Therefore, in this approximate calculation, the straight lines l ₃ and l ₄ at the upper and lower ends of the image plane are As long as it is sufficiently parallel to the road surface, a sufficiently adequate approximation can be given even when the camera CM is inclined and a tilt angle exists to some extent.

When the above relationship is compared with the processing in the image processing unit VC described above, the following is obtained. That is, in the projective transformation calculation unit PP, a set of straight lines (l ₁ on the image of the camera CM) corresponding to a set of straight lines (l ′ ₁ and l ′ ₂ ) on the parallel road surface extending in the traveling direction of the vehicle. And l ₂ ), a set of straight lines (l ₃ and l ₄ ) at the upper and lower ends of the image plane constituting the image, and a set of straight lines (l ′ ₃ and l when the straight lines on the image are projected onto the road surface ' ₄ ) Projective transformation is performed. Then, in the camera position specifying unit IP unit, the intersection (X ′ ₇ and l ′ ₈ ) obtained by extending the straight line (l ′ ₇ and l ′ ₈ ) of the projected image onto the road surface of the set of straight lines (l ₇ and l ₈ ) at the left and right ends of the image plane. The position of ' _C ) is specified as the position of the camera CM.

  As described above, according to the present embodiment, the position of the camera CM can be specified without requiring an internal parameter or an external parameter of the camera CM. Therefore, not only does the camera CM need not be calibrated in advance, but the internal parameters of the camera CM such as the focal length change during traveling, or the mounting height and inclination of the camera CM on the vehicle fluctuate. Since it is not affected, the position of the own vehicle (camera-equipped vehicle) can be calculated and specified in a robust manner against disturbance.

  Next, detection of a traveling lane in the lane boundary line specifying unit LP, specifically, identification of lane boundary lines on both sides of the traveling lane will be described. In order to specify the lane boundary line, in addition to a general least square method, “straight line fitting” (straight line fitting) can be performed by RANSAC disclosed in Non-Patent Documents 2 and 3 described above. This RANSAC (Random Sample Consensus) is a method of finding a mathematical model that can take the most consensus with samples taken at random. With this RANSAC, it is possible to apply a mathematical model by detecting only correct data in a state where erroneous data is included in the data. As described above, since mathematical models are stably applied without being affected by disturbances such as image noise, they have recently been used for various tasks in computer vision.

  Here, the RANSAC algorithm will be described. First, the minimum number of data necessary for calculating the mathematical model to be applied is randomly extracted. For example, when applying a straight line, since the straight line is determined by two points, data of two points are randomly extracted from many feature points. Next, a mathematical model is calculated based on the extracted data. In the case of straight line fitting, as shown in FIG. 17, a straight line equation is calculated using the extracted two points of data. In FIG. 17, the broken line indicates the threshold range, and Dx indicates the distance between the straight line and the feature point. Then, the number of data applicable to the obtained mathematical model is obtained. In the case of straight line fitting, the number of feature points that apply to the obtained straight line is obtained. At this time, whether it is true or not is determined by whether or not data is included in the area near the mathematical model. In the case of straight line fitting, as shown in FIG. 17, if the distance between the obtained straight line and the specific feature point is equal to or less than a threshold value, it is determined that the feature point is applied to the obtained straight line. Finally, N sets of data are extracted from all data at random, the number of applicable data is obtained for each of them as described above, and the mathematical model having the largest number of applicable data is set as a true model. In the case of straight line fitting, a straight line having the largest number of feature points to be applied is obtained.

As described above, RANSAC can obtain a correct mathematical model with a fairly high probability by performing a subset search without performing an exhaustive search in a situation where erroneous (false) data exists. It is a big feature. However, in order to guarantee the probability of obtaining a correct solution above a certain value, it is necessary to increase the number of samples to be extracted as the proportion of incorrect data in all data increases. In general, assuming that the degree of freedom of the model is d, the proportion of wrong data is e, and the probability that a correct solution is obtained is p, in order to guarantee the probability p in RANSAC, the number of sampling times N is N = log ( 1-p) / log (1- (1-e) ^d ), which is required to be greater than or equal to the value obtained.

  In the lane detection targeted by the present invention, if the number of feature points representing the correct lane is extremely reduced due to the influence of shadows on the road surface, and the number of false feature points increases instead, the probability of obtaining a correct solution is obtained. A very large number of samplings and model fittings will be required to ensure. Conversely, if the number of sampling and model fitting is fixed to a fixed value, the probability of obtaining a correct solution varies depending on the ratio of false feature points, and the probability of false detection is extremely high when the number of false feature points is large. Get higher.

  FIG. 18 shows the relationship between the number of samplings N required to obtain a correct solution with a probability of 99.99% and the ratio e of incorrect data in the straight line detection problem (d = 2). From the figure, it can be seen that the required number of samples increases rapidly as the ratio of wrong data increases. FIG. 19 shows the relationship between the incorrect data ratio e and the probability p of obtaining a correct straight line when the number of samplings is fixed at 100. From the figure, it can be seen that the probability p that a correct straight line is obtained decreases rapidly as e increases.

  In view of the above-mentioned problems of RANSAC in lane detection, the present inventors have improved this and developed an algorithm, Connected Sample Consensus (referred to as CONSAC), that maintains a high probability of obtaining a correct solution with a small number of calculations. A patent application was filed on September 30, 2002 (Japanese Patent Application No. 2002-284678). As described above, in RANSAC, a sample is randomly extracted and the degree of fitting is checked without any information. In many tasks, data is obtained by using prior information unique to the task. In many cases, grouping or ordering can be performed in advance. CONSAC does not sample data at random by using such prior information, but samples and calculates data according to grouping and ordering to reduce the number of cases and correct solution with fewer searches. Maintains a high probability of being obtained. In this way, since CONSAC uses data succession, it is set to “Connected” instead of “Random” of RANSAC, and the algorithm is as follows.

  First, the prior information is used to group and order data so that it is meaningful to some extent. Next, a minimum number of data necessary for calculation of the mathematical model to be applied is extracted from the ordered data at predetermined intervals. A mathematical model calculation is performed based on the detected data. The number of data applicable to the mathematical model thus obtained is obtained. Then, the above processing is repeated while sampling all the data in order according to the data ordering, and the mathematical model having the largest number of applicable data is set as a true model.

  The feature points detected by the feature point detection unit CP of the present embodiment are arranged approximately in order along a straight line (lane boundary line) indicated by a solid line, as indicated by data numbers 1 to 12 in FIG. There is. In such a state, there is a very high probability that the points with similar data numbers are on the same straight line. Therefore, in this embodiment, the feature point data is ordered in the order obtained by scanning, and the straight line detection unit SP samples two points at a relatively small interval according to the feature point data order, and calculates a linear equation. The Next, the number of data that fits the straight line is counted. This process is repeated for all feature point data according to a predetermined data order, and a straight line with the maximum number of applicable data is detected as a true straight line.

  Thus, according to the above-mentioned CONSAC, the conventional Hough transform solves the problem that a correct result cannot be obtained when there are more false feature points than the number of feature points of an object to be detected. , RANSAC can solve the problem that it cannot be detected stably when the ratio of false feature points increases. Furthermore, even when there are many false feature points, a correct solution can be obtained reliably with a very small number of searches.

  Next, as another embodiment of the present invention, based on the traveling lane of the vehicle detected by the projective transformation as described above, the boundary line between the two traveling lanes adjacent thereto is specified using the vanishing point. The configuration will be described. 4 to 6 show another embodiment of the road lane detection device, and its hardware configuration is basically the same as that shown in FIG. 2 as shown in FIG. For example, as shown in FIG. 14, in the case of detecting adjacent traveling lanes adjacent to the left and right with respect to the traveling lane of the own vehicle, other vehicles are traveling in the adjacent traveling lane and the lane boundary line (white line) If the vehicle is partially hidden or the lane boundary line is difficult to identify due to the relationship between the angle of view and the sunshine condition, the detection stability is lower than the traveling lane of the vehicle. Therefore, in the present embodiment, after detecting the traveling lane of the own vehicle stably as in the above-described embodiment, the adjacent traveling lane is stably specified using this result.

  In the present embodiment, as shown in FIG. 4, straight line data is detected from images obtained by continuously imaging the road surface by the image pickup means VD, and a traveling lane (not shown) of a moving body (for example, a vehicle) based on the straight line data. ) Detecting lane detecting means LD, projective transformation calculating means PT for calculating projective transformation based on the straight line data detected by the running lane detecting means LD, and on the road surface based on the calculation result of the projective transformation calculating means PT. In addition, an adjacent lane specifying unit ND that reversely converts straight lines that are parallel to and equidistant from the set of straight lines and specifies the adjacent lanes on the image is provided.

  The traveling lane detecting means LD and the projective transformation calculating means PT are configured in the same manner as in FIG. 1, and the adjacent lane specifying means ND includes a plurality of traveling lanes whose mutual lane intervals are constant as shown in FIG. Of these, a lane adjacent to the lane detected by the lane detection means LD is set as an adjacent lane, and an adjacent lane boundary position estimation means N1 for estimating the boundary line position of this adjacent lane by the projective transformation calculation means PT; The feature point detection area setting means N2 for setting a detection area for detecting feature point data related to the boundary line of the adjacent lane at the position estimated by the adjacent lane boundary position estimation means N1, and the feature point detection area setting means Feature point detection means N3 for detecting a feature point in the feature point detection region set by N2, and an arbitrary feature point in the feature point detected in the feature point detection region A straight line detecting means N4 for calculating a point and calculating the infinity point coordinates as one other point and detecting a straight line including both points, and at least one straight line detected by the straight line detecting means N4 from the straight line. And an adjacent lane boundary line specifying means N5 for specifying, as a boundary line between adjacent lanes, a straight line having the largest number of feature points within the range.

  The road surface lane detection device of FIG. 4 has the hardware configuration shown in FIG. 5, but the same components as those in FIG. In the image processing unit VC of FIG. 5, in addition to the configuration of FIG. 2, an adjacent lane specifying unit ND is configured, and an adjacent lane boundary position estimation unit NP1, a feature point detection region setting unit NP2, corresponding to each unit of FIG. A feature point detection unit NP3, a straight line detection unit NP4, and an adjacent lane boundary line specifying unit NP5 are provided.

In the lane boundary line specifying unit LP of FIG. 5, for example, the traveling lane boundary line of the own vehicle is specified by CONSAC as described above. Next, on the premise that the distance between the driving lanes (the distance between the two lane boundary lines) is constant, based on the boundary line of the driving lane specified by the lane boundary specifying unit LP, Predicting the appearance position in the image with respect to the boundary line. As shown in FIG. 6, a set of parallel straight lines (l ′ ₁ and l ′ ₂ ) on the road surface is obtained from the straight lines l ₁ and l ₂ by the above-described projective transformation H _l , and is parallel to these. By calculating straight lines l ′ ₉ and l ′ ₁₀ located at equally spaced positions and inversely transforming them by H _l ⁻¹ to obtain images l ₉ and l ₁₀ in the image, the adjacent lane boundary position estimation unit NP1 The adjacent driving lane can be specified. The feature point detection region setting unit NP2 provides a feature point detection region in the vicinity of the predicted appearance position (images l ₉ and l ₁₀ ), so that the feature point detection unit NP3 has a boundary line between adjacent lanes. Feature points x _i (i = 1 to N) can be detected.

Next, a straight line that best fits (applies) these feature points x _i (i = 1 to N) is obtained as a boundary line between adjacent traveling lanes. In this case, since it is assumed that the boundary line of the adjacent traveling lane is parallel to the boundary line of the own vehicle's traveling lane, the boundary line of the adjacent traveling lane is the boundary line of the own vehicle's traveling lane in the image. The vanishing point V _p (infinite point coordinates) that is the intersection of (l ₁ and l ₂ ) must be passed. Therefore, in the present embodiment, among the straight lines passing through the vanishing point V _p , a straight line that best matches the feature points x _i (i = 1 to N) representing the adjacent traveling lane is obtained by RANSAC.

By using RANSAC in this way, it is possible to specify a correct lane boundary line even if a feature point representing an adjacent traveling lane includes a large deviation due to a traveling vehicle or the like. Usually, in order to obtain a straight line by RANSAC, two points are randomly selected from the feature points to obtain a straight line, and the “degree of fitting” is calculated. In this case, the straight line is always a vanishing point v. _Since it passes through _p , only the remaining one point can be selected at random from the feature points x _i (i = 1 to N) and the straight line that best fits can be obtained, which is far more than the normal line detection by RANSAC. Can be processed at high speed. For this reason, it is possible to check the total number of feature points and obtain a straight line that best fits.

  By performing the above processing on the traveling lanes adjacent to the left and right of the traveling lane of the own vehicle, three traveling lanes can be specified centering on the own vehicle. In particular, it is possible to specify adjacent driving lanes very stably and at high speed by using the information on the driving lanes of the vehicle and the vanishing point constraint that are specified relatively stably.

  Regarding the experimental results according to the embodiment of FIGS. 4 and 5 described above, the stability in detection of a plurality of travel lanes will be described with reference to FIGS. 14 and 15, and then the stability of the position of the vehicle in the travel lanes will be described. Reference is made to FIG.

FIG. 14 shows a large truck traveling in an adjacent lane, where most of the lane boundary is concealed, and a confusing shadow of the vehicle (shown in stipple in FIG. 14) falls on the road surface. It is an example of an image when it is very difficult to specify. This image is identified by applying straight lines (l ₁ and l ₂ ) based on the feature points (black dots in FIG. 14) detected by the embodiments shown in FIGS. 1 to 3 and FIGS. 4 to 6. The boundary line of the driving lane of the car is indicated by a solid line, and the boundary line of the adjacent driving lane specified by the embodiment shown in FIGS. 4 to 6 is indicated by a broken line. As apparent from FIG. 14, the lane boundary lines (l ₉ and l ₁₀ ) of the adjacent traveling lanes are largely hidden by the large truck, and conversely, the outline of the loading platform is detected as a feature point. The borderline between adjacent lanes is identified very stably. This is because the feature points are detected by predicting the positions of adjacent lanes based on the information of the vehicle's travel lanes, and the outliers in the detected feature points are converted to the aforementioned RANSAC (vanishing points). This can be defined as fixed type RANSAC).

FIG. 15 shows an example in the vicinity of the tunnel exit, which is a harsh example in which it is very difficult to detect the lane boundary because the road surface is shining, and the traveling lane must be detected with very few feature points. It is. However, according to the embodiment of FIGS. 4 to 6 described above, stable detection is performed from the traveling lane (lane boundary lines l ₁ and l ₂ ) to the adjacent traveling lane (lane boundary lines l ₉ and l ₁₀ ). be able to.

  FIG. 16 is a graph plotting the results of real-time measurement of the position of the vehicle in a plurality of lanes from the test video image regarding the detection of the traveling lane in the embodiments of FIGS. 1 to 3 and FIGS. 4 to 6. The position of the vehicle in the video image of about 10 minutes is shown. A one-dot chain line in the figure indicates a lane boundary, the horizontal axis indicates time, and the vertical axis indicates the position of the vehicle in the three lanes (the axis is omitted). In FIG. 16, the solid line indicates the position change of the own vehicle (the position of the own vehicle is obtained as a relative position when the lane width is 1), and the position of the own vehicle changes smoothly before and after the lane change. The vehicle position can be correctly identified without losing sight of the lane by lane change. Thus, in any of the above embodiments of the present application, the position of the lane boundary line can be accurately specified, and the traveling lane can be detected stably.

  The flowcharts of FIGS. 7 to 13 are a series of steps including the detection of the driving lane of the own vehicle in the embodiment shown in FIGS. 4 to 6 (this is the same as the embodiment shown in FIGS. 1 to 3) and the identification of adjacent lanes. FIG. 7 shows a main routine, and FIGS. 8 to 13 show a subroutine. These are mainly processed by the image processing unit VC of FIG. First, in step 101 of FIG. 7, feature points of the traveling lane are detected, and in step 102, a straight line is applied to the feature points. Next, in step 103, the vanishing point is calculated, and in step 104, it is determined whether or not the coordinates of the vanishing point are proper. If so, the process proceeds to step 105, and the travel lane interval (between the pair of travel lane boundaries) is determined. (Distance) is calculated, but the coordinates of the vanishing point deviate from the reference, and if it is not appropriate, the process ends.

  Then, the process proceeds to step 106, and if the distance between the travel lanes calculated in step 105 is a value between 2.6 m and 4.4 m, for example, the process proceeds to step 107 and the vehicle position relative to the travel lane is specified. When the interval is out of this range, the process ends as out of the target of travel lane detection. If the vehicle position is specified in step 107 and it is determined in step 108 that the vehicle is in the traveling lane, the process proceeds to step 109 and thereafter. finish. Thus, in step 109, the left and right outer straight lines of the travel lane are estimated, and in step 110, feature points on the left and right outer sides of the travel lane are detected, and in step 111, a straight line fit is applied to these feature points. Is performed, and the boundary line of the adjacent lane is specified. After the adjacent lanes are identified in this way, the process proceeds to step 112, where lane change (lane change) processing is performed as necessary.

  FIG. 8 shows a subroutine for detecting feature points of the driving lane in the above step 101. First, in step 201, the state of the flag Fs indicating the straight line detection state is determined, but is set (1) at the start of detection. Therefore, the process proceeds to step 203, and the feature point is detected by the differential filter. Then, in step 204, the number of feature points of the detection result is compared with a predetermined threshold value. If it exceeds this threshold value, the flag Fs is set (1) in step 205 and the process returns to the main routine. If it is equal to or less than the threshold value, the flag Fs is cleared (0) (the first time as it is) and the process returns to the main routine. In the next and subsequent calculation cycles, if it is determined in step 201 that the flag Fs is set, the process proceeds to step 202, where the feature point search range is limited to around a straight line detected in the processing described later. Therefore, it is possible to quickly detect the feature points in step 202.

  FIG. 9 shows a subroutine for straight line fitting to the feature points in step 102 in FIG. 7. First, in step 301, two arbitrary points are selected from the feature points detected as described above, and the process proceeds to step 302. . In step 302, the distance between the straight line connecting the two points and another feature point is calculated, and this distance is compared with a predetermined threshold value in step 303. If this distance is less than the threshold value, the feature point is stored in the memory by voting in step 304 and then the process proceeds to step 305. On the other hand, if the distance is greater than or equal to the threshold value, the process proceeds to step 305 as it is. It is determined whether or not the calculation is completed, and the above processing is repeated until the calculation is completed.

  In step 306, the number of votes is compared with a predetermined maximum number of votes Mv. When the number of votes exceeds the maximum number of votes Mv, the number of votes is set to the maximum number of votes Mv, and then the number of votes When the number of votes is less than Mv, the process proceeds to step 308 as it is, and the search end determination of all feature points is performed. The processes of steps 301 to 307 are repeated until it is determined that the search of all feature points is completed. As a result, the maximum number of votes obtained Mv is compared with a predetermined threshold value, and if it exceeds the threshold value, the flag Fs is set (1) in step 310 and then the process proceeds to step 312. In some cases, the process proceeds to step 312 after the flag Fs is cleared (0). Thus, in step 312, a straight line is calculated with respect to the feature points in the memory by the least square method or the above-described RANSAC or CONSAC.

  FIG. 10 shows a subroutine for calculating the travel lane interval in step 105 of FIG. 7. In step 401, the internal parameters of the camera CM are set, and in step 402, the height of the camera CM is set. In step 403, the rotation angle of the camera CM is calculated from the vanishing point. In step 404, a plane projective transformation for rotating the camera CM directly from the rotation angle is calculated, and in step 405, the left and right boundary lines of the travel lane are planar projected. Thus, in step 406, the travel lane interval is calculated from the distance between the converted straight lines. The internal parameters of the camera CM are used for the calculation of the travel lane interval. However, for example, if it is not necessary to know the absolute value of the travel lane interval, it is not necessary to set the internal parameters.

  FIG. 11 shows a subroutine for specifying the vehicle position with respect to the travel lane in step 107 in FIG. 7. A straight line group on the image surface is defined in step 501, and a straight line on the road surface in step 502. A group is defined. Similar to the above-described processing described with reference to FIG. 3, the plane projection transformation between the image plane and the road surface is calculated in step 503, and both ends of the image are planar projection transformation in step 504. At 505, the intersection of the straight lines after projective transformation is calculated. FIG. 12 shows a subroutine for estimating the left and right outer straight lines at step 108 in FIG. 7. Similar to the above-described processing described with reference to FIG. 6, from step 601 to the image plane. After the plane projective transformation is calculated, in step 602, the straight line on the road surface is planar projected. Note that the subroutine for detecting the left and right feature points in step 109 in FIG. 7 is substantially the same as that in FIG.

  FIG. 13 shows a subroutine related to straight line fitting for the left and right feature points in step 111 of FIG. 7, and compared with the subroutine related to straight line fitting of FIG. 702 (distance between a straight line connecting the selected point and the vanishing point and another feature point is calculated) and step 712 (conditional to include the vanishing point when applying the least squares method) are different. Steps 301 to 312 are the same as those in FIG.

It is a block diagram which shows the main structures of the road surface traveling lane apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the hardware constitutions of the road surface traveling lane apparatus which concerns on one Embodiment of this invention. In one Embodiment of this invention, it is a perspective view which shows an example of the projection geometry at the time of detecting a driving lane. It is a block diagram which shows the main structures of the road surface traveling lane apparatus which concerns on other embodiment of this invention. It is a block diagram which shows the hardware constitutions of the road surface traveling lane apparatus which concerns on other embodiment of this invention. In other embodiment of this invention, it is a perspective view which shows an example of the projection geometry at the time of specifying an adjacent driving | running | working lane. It is a flowchart which shows a series of processing examples including the driving | running | working lane detection and identification of an adjacent lane in this invention. It is a flowchart which shows the subroutine of the feature point detection of the driving lane in FIG. It is a flowchart which shows the subroutine of the straight line fitting with respect to the feature point in FIG. It is a flowchart which shows the subroutine of calculation of the driving | running | working lane space | interval in FIG. It is a flowchart which shows the subroutine which concerns on specification of the own vehicle position with respect to the travel lane of FIG. It is a flowchart which shows the subroutine which estimates the left-right outer straight line in FIG. It is a flowchart which shows the subroutine which concerns on the straight line fitting with respect to the feature point of the left-right outer side of FIG. It is a front view which shows an example of the image of the experimental result which detected the several driving lane by other embodiment of this invention. It is a front view which shows the other example of the image example of the experimental result which detected the several driving | running | working lane by other embodiment of this invention. It is a top view which shows the motion of the position of the own vehicle in each traveling lane when it moves between several traveling lanes regarding the experimental result by other embodiment of this invention. It is explanatory drawing which shows an example in the case of applying a straight line with respect to feature point data by the conventional RANSAC. It is a graph which shows the relationship between the frequency | count of sampling required in order to obtain a correct solution, and the ratio of wrong data, when applying a straight line with respect to feature point data by the conventional RANSAC. It is a graph which shows the relationship between the ratio of the wrong data, and the probability that a correct straight line will be obtained when the number of samplings is fixed to 100 when a straight line is applied to feature point data by conventional RANSAC. It is explanatory drawing which shows the relationship between the feature point extracted in the feature point extraction part in one Embodiment of this invention, and a straight line.

Explanation of symbols

VD imaging means LD travel lane detection means CR feature point detection means SD
SD Straight line detection means LB Lane boundary line specifying means PT Projection transformation calculating means ID Imaging means position specifying means ND Adjacent lane specifying means ND
N1 Adjacent lane boundary position estimation means N2 Feature point detection area setting means N3 Feature point detection means N4 Straight line detection means N5 Adjacent lane boundary line specifying means CM camera VB Video input buffer circuit SY Sync separation circuit FM Frame memory VC Image processing section

Claims (6)

  In a road surface traveling lane detection device that detects a traveling lane of a moving body from images obtained by continuously capturing an image of a road surface by an imaging unit, linear data is detected from the image, and the traveling lane of the moving body is detected based on the linear data. And a set of straight lines on the image corresponding to a set of straight lines on the road surface that extend in the traveling direction of the moving body based on the straight line data detected by the traveling lane detection means. A set of straight lines at the upper and lower ends of the image plane constituting the image, a projective transformation calculation means for calculating a projective transformation based on a set of straight lines when the straight lines on the image are projected onto the road surface, and the projection Based on the calculation result of the conversion calculating means, the position of the intersection of the straight line of the projected image on the road surface of a set of straight lines perpendicular to the road surface existing on the road surface is specified as the position of the imaging means Road traveling lane detecting device characterized by comprising an imaging unit position determining means that.   In a road surface traveling lane detection device that detects a traveling lane of a moving body from images obtained by continuously capturing an image of a road surface by an imaging unit, linear data is detected from the image, and the traveling lane of the moving body is detected based on the linear data. And a set of straight lines on the image corresponding to a set of straight lines on the road surface that extend in the traveling direction of the moving body based on the straight line data detected by the traveling lane detection means. A set of straight lines at the upper and lower ends of the image plane constituting the image, a projective transformation calculation means for calculating a projective transformation based on a set of straight lines when the straight lines on the image are projected onto the road surface, and the projection An imaging unit position that identifies, as the position of the imaging unit, the position of an intersection obtained by extending a straight line of a projected image on the road surface of a set of straight lines at the left and right ends of the image plane based on the calculation result of the conversion calculation unit Road traveling lane detecting device characterized by comprising a constant section.   In a road surface traveling lane detection device that detects a traveling lane of a moving body from images obtained by continuously capturing an image of a road surface by an imaging unit, linear data is detected from the image, and the traveling lane of the moving body is detected based on the linear data. And a set of straight lines on the image corresponding to a set of straight lines on the road surface that extend in the traveling direction of the moving body based on the straight line data detected by the traveling lane detection means. A projective transformation computing means for computing projective transformation based on a set of straight lines at the upper and lower ends of the image plane, and a set of straight lines when the straight line on the image is projected onto the road surface, and Based on the calculation result, it is provided with an adjacent lane specifying means for inversely converting straight lines that are parallel and equally spaced with respect to the set of straight lines existing on the road surface, and specifying the adjacent lanes on the image. Road traveling lane detection device for the butterflies.   The travel lane detection means detects feature point data from a predetermined area of the image picked up this time, and a current straight line representing a straight line that best matches the current feature point data detected by the feature point detection means A straight line detecting means for detecting data, and a lane boundary detecting means for detecting a current lane boundary line based on the current straight line data detected by the straight line detecting means, and the current lane boundary detecting means detected by the lane boundary line detecting means. The lane boundary line and the current straight line data detected by the straight line detection unit are supplied to the feature point detection unit, and the feature point detection unit has predetermined characteristics based on the current lane boundary line and the current straight line data. The feature point data satisfying the predetermined characteristic is detected as the next feature point data among the feature point data detected from the image to be captured next time. Road traveling lane detecting device according to claim 1, 2 or 3, wherein the configured into.   The feature point detection means has an ordering means for ordering the feature point data based on detection results of the straight line detection means and the lane detection means, and the straight line detection means is necessary for adaptation of a predetermined mathematical model. A data detection means for extracting a minimum number of data from the feature point data of the ordering means at a predetermined interval; a mathematical model calculation means for obtaining a mathematical model based on the data detected by the data detection means; Data number counting means for obtaining the number of feature point data that conforms to the mathematical model obtained by the mathematical model calculation means, and part or all of the feature point data according to the order of the feature point data set by the ordering means As a result of sequentially sampling and repeating the processing of the data detecting means to the data number counting means, the true number of mathematical models having the largest number of feature point data most suitable are obtained. Road traveling lane detecting apparatus according to claim 4, characterized in that a mathematical model setting means for setting as a model. The adjacent lane specifying means sets a travel lane adjacent to the travel lane detected by the travel lane detection means as an adjacent lane among a plurality of travel lanes having a constant distance between the lanes. An adjacent lane boundary position estimating means for estimating the boundary line position of the adjacent lane at the position estimated by the adjacent lane boundary position estimating means, and detecting for detecting feature point data related to the boundary line of the adjacent lane Feature point detection area setting means for setting an area, feature point detection means for detecting feature points in the feature point detection area set by the feature point detection area setting means, and feature points detected in the feature point detection area A straight line detecting means for detecting a straight line including both points by calculating an arbitrary point of the point as one point and the infinity point coordinate as another point, and a small number detected by the straight line detecting means. In addition, for both straight lines, there is provided an adjacent lane boundary line specifying means for specifying, as a boundary line of the adjacent lane, a straight line having the largest number of feature points within a predetermined range from the straight line. The road surface traveling lane detection device according to claim 3.

Images