JPH0887700A - Road environment recognition device - Google Patents

Abstract

PURPOSE: To recognize feature picture elements at high speed while reducing errors at a rectangle extracting means required for a distance up to a preceding vehicle by reducing the arithmetic amount of a lane area extracting device inside the road environment recognition device. CONSTITUTION: An image extracting the contour line of an object is prepared from the image of an image pickup means fixed in the advancing direction of the road environment recognition device, with that image, an intersection with a infinite far line such as an extracted left white line decided by the image pickup means for extracting the left side and right side sign lines of the present vehicle is calculated, and a right white line is estimated from virtual road width W. At such a time, the left white line is searched at two positions (1) and (2) decided in advance, and the spot of maximum luminance is defined as the left white line. Besides, the white line of a present frame is searched based on the position of the white line extracted at the preceding frame. When no white line can not be searched at two positions (1) and (2), the search range of that white line is changed. When the feature picture elements (edges) exist within the range of width ±α picture elements (α=1, 2) in the case of extracting a rectangle from that image, they are extracted as one continued straight edge and defined as the candidates of horizontal and vertical lines.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a road environment recognition apparatus provided with an image processing apparatus for recognizing a road environment, and more specifically, in an autonomously traveling vehicle, the vehicle is installed in front of the vehicle. From the image from the TV camera that captures the front of the traveling direction, recognize the marker line (for example, white line) provided on the road surface from the area near the vehicle to the distant area,
At the same time, it relates to a road environment recognition device capable of recognizing a vehicle traveling ahead of the host vehicle and an obstacle.

[0002]

2. Description of the Related Art In recent years, an autonomous traveling control device for recognizing a road environment by an image processing portion and controlling autonomous traveling of a vehicle based on the recognition result has been developed.

This autonomous running control device is installed in a vehicle running in the driving lane. For example, if the vehicle deviates from the marking line, the driver is automatically warned, or Based on the information, it is possible to control the own vehicle. Realizing such a device is extremely important because it can reduce the driver's driving burden and prevent accidents.

More specifically, from the image from the TV camera installed in front of the vehicle and capturing the front of the vehicle in the traveling direction, the marker line provided on the road is distant from the area near the vehicle. Recognize up to, and at the same time recognize vehicles and obstacles traveling in front of your vehicle, and in steady-state running, use the marker line such as the white line visible on the left side as a guideline and run along it to drive ahead of your vehicle. The distance to the other vehicle that is running is calculated, and the own vehicle is steered so as not to collide with the preceding vehicle.

In this way, the vehicle traveling sign line (lane) is recognized from the image of the front of the vehicle using image processing technology,
At the same time, as a system for recognizing another vehicle traveling in front of the own vehicle, the applicant of the present application has proposed a “road environment recognition device” (Japanese Patent Application No.
6-13874).

The above "road environment recognition device" (Japanese Patent Application No. 06-
No. 13874), an edge image is created by extracting only the contour of an object from an image of the front of the vehicle,
Parameters that are unique to the image capturing means are extracted by extracting edges that appear to some extent consecutively in a region that captures the region diagonally to the left (or diagonally right) of the vehicle on the image, and extend it to the distance on the image. It extends to the point at infinity calculated from (for example, depression angle, angle of view, etc.) and recognizes it as the left marking line or the right marking line of the vehicle lane.

Further, the above-mentioned "road environment recognition device" (Japanese Patent Application No.
No. 6-13874), in order to recognize the position of the vehicle in front, a plurality of image pickup means, specifically, a pair of image pickup means for picking up an image in front of the traveling direction are provided, and the rectangle extraction section obtains from the pair of image pickup means. In the rectangles extracted from the pair of front images on the respective images obtained from the pair of image pickup devices, the rectangles having the same graphic characteristics are the same from the same area, horizontal line, vertical line length, etc. Assuming that the rectangles are obtained as a result of capturing an image of an object (here, another vehicle traveling in front), the rectangles are associated with each other so that the position of the vehicle in front can be recognized. The distance to the object in front is calculated by using the principle of triangulation from the positional shift (parallax) on the attached rectangular image.

FIG. 13 is a view for explaining a conventional lane area extracting device, which is the above-mentioned "road environment recognition device" (Japanese Patent Application No.
No. 6-13874).

In general, on the road, there is an uninterrupted white line on the left,
If there are two or more lanes or the opposite lane on the right, there is a white line with a break. In the prior art, a frame image {see FIG. 13 (a)} obtained by an image pickup unit fixedly attached to a vehicle is used to change the brightness of a left white line and a right dotted white line at several positions in the image for each frame. This is a method of extracting a large portion {see FIG. 13 (b)} and linearly approximating each white line point to extract a lane area {see FIG. 13 (c)}.

[0010]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention In the means for recognizing a marking line (white line) disclosed in the above-mentioned "road environment recognition device" (Japanese Patent Application No. 06-13874), the number of vehicles near the left side, for example, Since the marker line obtained from the location and the point at infinity are linearly approximated, and the vehicle area is recognized by presuming the right marker line from the recognized actual road width in advance, There was a problem that the recognition accuracy of the car area was poor.

In the conventional method described with reference to FIG. 13, it is necessary to calculate not only the approximate straight lines of the right white line and the left white line, but also the intersections thereof. Further, since the right white line is a dotted line, not all the points on the right white line can be extracted, and it is necessary to extract the white line at many points.

In this calculation, assuming that the coordinates of two white lines are (x1, y1) and (x2, y2), the approximate straight line is (y2-y1) / (x2-X1) (x1 ) + B can be calculated, but this calculation requires a summation operation at three points and a product operation at two points. It is necessary to perform the multiply-accumulate operation 10 times or more. Furthermore, the number of white lines to be extracted increases, so that the amount of calculation also increases.

FIG. 14 is a diagram for explaining a conventional horizontal line / vertical line candidate recognizing means. FIG. 14 (a) shows an example of the whole image and FIG. 13 (b) shows an edge image. An extraction example is shown. "Road environment recognition device" (Japanese Patent Application No. 06-1
No. 3874), the rectangular extraction unit disclosed in FIG.
With respect to the image shown in FIG. 4 (a), a predetermined horizontal / vertical projection means is used to set a line having a large projection value of edge pixels and a line having a predetermined width before and after the line (for example, ± 1 line) in the horizontal direction, or A mask area is set over the entire image from the vertical direction, and the overlapping portions of the mask and the horizontal and vertical edges are extracted as candidates for horizontal and vertical lines.

However, in an actual image, for example, the pixels forming an edge are not necessarily lined up on the same straight line, and an edge exists at a position separated by two lines in the middle as shown in FIG. 14 (b). It may happen. In this case, as in the above-described conventional example, a line having a large number of projection values of edge pixels in the horizontal direction (y = n in the figure) and a line having a predetermined width before and after that (± 1 line in this case) are masked. If it is set as a region, there is a problem that a part that is one edge from a global perspective is recognized as two separate edges (candidate lines α and β).

In view of the above-mentioned conventional drawbacks, the present invention recognizes a lane area in a road environment recognition apparatus because it is a pre-processing of an unmanned vehicle technology. Therefore, the calculation amount should be reduced as much as possible, and the calculation should be performed quickly and accurately. In order to provide a lane area extraction device that is capable of extracting the horizontal and vertical lines required for rectangle recognition to determine the distance to the vehicle traveling ahead, a global extraction is performed at high speed. It is an object of the present invention to provide a rectangle extracting means that can be used.

[0016]

1 to 4 are explanatory views of the principle of the present invention. The above problems are solved by the road environment recognition device configured as follows.

(1) Image pickup means 1L, 1R for picking up an image in front of the traveling direction of the vehicle, and the marking lines on the left and right sides of the own vehicle are extracted from the images obtained by the image pickup means 1L, 1R to obtain the own lane. Lane area extracting device 2L, 2R for recognizing the area, rectangular extracting means 3L, 3R for extracting a rectangular portion from each image obtained by the image capturing means 1L, 1R, and a plurality of image capturing means IL, 1R. Then, the rectangles having the same graphic feature obtained from the plurality of image pickup means 1R and 1L are associated with each other, and the difference between the positions of the rectangles on the image and the difference between the image pickup positions of the plurality of image pickup means are detected. A lane area extracting device 2L, 2R is a road environment recognition device having a rectangular matching & distance calculating means 4 for calculating a distance to a vehicle in front of the lane area extracting device 2L, 2R.
For example, an intersection A between the infinity line defined by 1R being fixed to the vehicle and the extracted left marker line is obtained, and the intersection A and a virtual road width W recognizable on the image
Therefore, a means for inferring the marker line on the right side is provided, and the traveling lane area in front of the road environment recognition device is obtained.

(2) In the lane area extraction device (2L, 2R) provided in the road environment recognition device described in the above item (1), search for the above left marker line at two predetermined locations. Then, the marker line detecting means is provided so that the point of maximum brightness is the position of the marker line on the left side.

(3) In the lane area extraction device (2L, 2R) provided in the road environment recognition device described in the above item (1), the current frame is extracted based on the position of the marker line extracted in the previous frame. The marker line searching means for searching the position of the marker line is provided.

(4) In the lane area extraction device (2L, 2R) provided in the road environment recognition device described in the above item (1), the range for searching the position of the marking line is selectively changed (range a → The marker line detecting means in the range b) is provided.

(5) The plurality of rectangle extracting means (3L, 3R) provided in the road environment recognition device according to the above (1) have a width ± α.
When it is detected that a characteristic pixel forming a rectangle, for example, an edge pixel exists in the range of pixels (α = 1, 2), it is extracted as one straight line edge and set as a candidate for a horizontal line or a vertical line γ. To configure.

[0022]

1 to 4 are explanatory views of the principle of the present invention. In order to extract the lane area in front of the vehicle at high speed, it is essential to reduce the number of white line extraction points as much as possible, and to perform straight line approximation, intersection calculation, and white line extraction easily.

FIG. 1 shows the overall configuration of the road environment recognition apparatus of the present invention, FIGS. 2 and 3 show the principle of calculating the lane area, and FIG. 4 shows the horizontal and vertical lines in the rectangle extraction unit. The principle of line candidate extraction is shown.

1) Use of infinity line and road width: As shown in FIGS. 2 (c) and 2 (d), when the camera is fixed to the vehicle, the left and right white lines are extended to the far point and intersect (infinity). Points or vanishing points) are always on one straight line. This straight line is called the line at infinity. The horizon and the horizon are also types of infinity.

If the installation location of the camera is fixed, the infinity line can be uniquely determined in advance on the image,
Easy to calculate. The intersection point (infinity point) A between this infinity line and the extension line of the left white line obtained from the two points can be easily calculated if only an approximate straight line of the left white line can be calculated. The position of the left white line is the position where the change in brightness is the largest as the horizontal straight line at two predetermined positions is sequentially searched from the center of the image. This process is called an initial process.

Next, the initial processing of the present invention {FIGS. 2 (b) to (d))
As a reference}, a white line is searched for from the center of the image toward both ends on a certain horizontal straight line, and a virtual road width W is extracted. The road width W is assumed to be almost unchanged until the white lines on both sides disappear, and is calculated only once as an initial process. If this virtual road width W is known, the white line on the right from the position of the point at infinity described above can be accurately estimated by the principle of a triangle similar shape.

By this method, the road width is extracted once and two points are obtained for each frame (one frame of a moving image, which is 30 frames per second in the case of a video image).
The right white line can be accurately estimated and the lane area can be extracted only by extracting the position of the left white line.

Further, in the case where the point A at infinity has a large deviation in each frame, it is understood that the lane area is not correctly extracted, and the correction can be easily performed. 2) Variable type white line search width: See FIGS. 2 and 3 In the above initial processing, the virtual road width W is extracted, and the left white line positions of two points are extracted. In the next frame, instead of searching from the center of the image as in the initial processing, only the side of the position of the white line extracted in the previous frame is searched. When two positions of the left white line can be extracted in each frame, the search range {range "a" shown in FIG. 3 (a)} is narrowed as a normal operation. If a white line cannot be extracted in this search range, it is considered that the search has failed, and in the next frame, a range wider than the search range for normal operation centered on the position when the white line position could be extracted (shown in Figure 3 (b)). The search is performed in the range of "b"}. If the white line can be extracted for both two points in this search, the search returns to the normal operation. If two white lines are not found again, this search is repeated for several seconds. During this time, if neither of the two points is found, the process returns to the above-mentioned initial processing and the virtual road width W is also updated.

As described above, not only is it found using the position of the white line in the previous frame, but when it is not found,
By adopting a method of expanding the search range of the white line, the lane area can be extracted without overlooking the white line.

However, the time difference between frames is 1/30 to
It is preferably 1/10. This is because the position of the white line found in the previous frame is closer to the position in the next frame as the time difference between the frames is smaller.

3) Extraction of horizontal and vertical line candidates: Next, referring to FIG.
The operation of rectangle extraction will be described below. In the present invention, characteristic pixels forming a rectangle, for example, edge pixels are not counted in the entire image, and a line having many edge pixels is not extracted as a horizontal (or vertical) line candidate. After extracting the edge, pixels that are edges are tracked within a range of width ± α lines (for example, ± 1 line), and even if the edge pixels are not adjacent, as long as they are within the range of ± α pixels, It is designed to be extracted as the same horizontal (or vertical) line.

Therefore, since the lane area extracting means according to the present invention can perform extraction with a small amount of calculation, it can be easily applied to the visual system of a self-propelled vehicle that requires high speed such as automatic driving. In the rectangle extraction, for example, even when the edge pixels are not adjacent to each other, if they exist within the range of ± α pixels, they can be extracted as the same horizontal and vertical line candidates. Thus, even if the edge pixels that form the same contour line are not continuous, it is possible to extract them as continuous horizontal and vertical candidates.

[0033]

Embodiments of the present invention will be described in detail below with reference to the drawings. 1 to 4 described above are explanatory views of the principle of the present invention,
5 to 11 are views showing an embodiment of the present invention, FIG. 5 shows a configuration example of a lane area extracting device according to the present invention, and FIG. 6 is an input / output specification of the present invention. FIG. 7 schematically shows the road width extracting means of the present invention, FIG. 8 shows an initial process of white line detection, and FIG. 9 shows an example of a process for changing the search range in the white line detection. Is shown in FIG.
The processing flow for white line detection is shown, and FIG. 11 shows lane area calculation means. FIG. 12 is a diagram showing another embodiment of the present invention, and schematically shows the operation of the rectangle extracting section.

In the present invention, in order to extract the marker lines on the left and right sides of the vehicle from the images obtained by the image pickup means 1L, 1R fixedly provided in the traveling direction of the road environment recognition device, A means for obtaining the intersection of the infinity line determined by the image pickup means 1L, 1R and the above-extracted left white line, for example, and estimating the right white line from the virtual road width. Further, a means for searching the left white line at two predetermined locations when extracting the left white line, and setting the point of maximum brightness as the left white line. or,
A means for searching for a white line in the current frame based on the position of the white line extracted in the previous frame. At this time, the above two places
A means for changing the search range of the white line when the white line cannot be searched with. Further, when a rectangle is extracted from the image, if there are characteristic pixels forming the rectangle, such as edge pixels, within a range of width ± α pixels (for example, α = 1, 2), one continuous straight line A means for extracting the edge and using it as a candidate for a horizontal line or a vertical line is a means necessary for implementing the present invention. The same reference numerals indicate the same objects throughout the drawings.

5 to 5 with reference to FIGS.
The configuration and operation of the lane area extraction device and the rectangle extraction unit in the road environment recognition device of the present invention will be described with reference to FIGS. 11 and 12. In the present embodiment, the marker line on the road will be described as a white line in the following for convenience of description, but is not limited to the white line.

First, FIG. 5 is a diagram showing a configuration example of the lane area extracting device of the present invention. The present lane area extracting apparatus is an apparatus for extracting a lane area ahead of a vehicle to travel based on image information obtained from a camera mounted in front of the vehicle as shown in FIG. FIG. 6 (a) is an input image. The input image has a coordinate system with the Y coordinate in the vertical direction and the X coordinate in the horizontal direction, with the lower left corner as the origin. In this embodiment, the vertical and horizontal sizes of the image are Y and X, respectively. The data output by the device as a result of processing this image is shown in FIG. 6 (c). The lane area is a triangle AB as shown in FIG. 6 (b).
Since it can be expressed in C, the coordinates (x, y) of each vertex are output as a result.

The lane area extracting apparatus according to the present invention includes an image generating unit.
20, a road width extraction unit 21, a white line detection unit 23, and a lane area calculation unit 24. The image generation unit 20 converts a television (TV) signal from a camera mounted on the vehicle into an A / D signal.
It is converted and stored in the frame memory inside the image generation unit 20. The size of the frame memory is the same as the size of the image, and for example, image data of 256 gradations of X pixels × Y pixels × 8 bits can be stored. This frame memory can be randomly read from the road width extraction unit 21 and the white line detection unit 23. However, since the image (or frame) is input to the frame memory at the video rate, that is, every 1/30 second, it is a double memory for reading continuous images. That is, there are two frame memories, and the memory to be written and the memory to be read are alternately switched.

The road width extraction unit 21 starts operation when an image is stored in the frame memory. However, the operation flag from the white line detection unit 23 must be set. The operation flag is set by the white line detecting section 23. In the initial state, the operation flag is “on”.

FIG. 7 shows the operation of the road width extraction unit 21.
The image has been determined to be extracted white lines in advance which position, its position (Y coordinate only) is registered in the white line position memory 27. There are two positions, and the road width W is determined from the white line at the position of. This road width is not an actual road width, but a virtual road width W on the image shown in FIG. 7 (a).

A method of determining the road width W will be described with reference to the cross section of the image at the position, that is, the graph of the X coordinate and the brightness. The cross section of the image at is represented by the luminance bar graph shown in FIG. In the determination of the virtual road width W, pixels are scanned in the right and left directions with the center of the image as a starting point,
Select the one with the highest brightness on the bar graph. As a concrete method, the image data for one line shown in discrete representation in FIG. 7C is shown. Each cell corresponds to a pixel. Luminance is recorded in each pixel. For example, in the case of an image having 256 gradations, the value has a value of 0 to 255. In the luminance bar graph, a large value is recorded in a pixel having a high luminance value. The masks A and B are scanned in the leftward and rightward directions, respectively, starting from the pixel of the halftone line. The masks A and B are templates each having a size of two pixels, and are compared with the image while being shifted by one pixel, and the product of the number and the pixel value written in the mask and the sum of the products are calculated. The position where the maximum sum is obtained for each of the masks A and B that are scanned left and right corresponds to the position of the white line. When the two white lines are extracted, the road width W is obtained from the difference between the positions of the two white lines. The obtained road width W
Is recorded in the road width memory 22.

However, it is assumed that the position of the white line cannot be obtained unless the maximum sum is a value equal to or larger than a certain standard. If either or both of the two white lines on the left and right are not obtained, the calculation of the road width W is repeated in the next frame. Unless the road width W is extracted,
It does not move to the next processing.

In the white line detecting section 23, the road width extracting section 21
When the road width W is calculated in step S1, the operation is started. Immediately after the road width W is extracted, the initial process of white line extraction shown in FIG. 8 is performed. That is, the two Ys stored in the white line position memory 27
Only the left white line is extracted for the coordinates. For, use the position of the white line obtained in the search for the left half of the road width extraction unit 21. Then, similarly to the road width extraction, the same mask as the mask A is scanned in the left direction to find the position where the value is maximum. If this value is equal to or greater than a certain reference, it is determined that a white line has been extracted, and the white line is stored in the X coordinate of the white line position memory 27. If, on the other hand, the left white line is not extracted, the operation flag is set to the road width extraction unit 21, and this initial process is repeated in the next frame. When both white lines are extracted, the same white line extraction process is continued,
The operation flag to the road width extraction unit 21 is lowered. The operation of the white line detecting section 23 after the initial processing will be described below with reference to the processing flow shown in FIG. 9 (a), (b)
As shown in the above, in the white line detection unit 23, when two points, that is, the left white line, can be extracted in the previous frame, the search range A is centered on the position of the white line in the previous frame.
Find the white line in the next frame in (pixels). How to find
This is the same as the road width extraction unit 21 described above. The difference is that the search range is narrower than the initial processing of the road width extraction unit 21 and the white line detection unit 23. For example, if the horizontal size of the image is 320
In the case of pixels, the search range A is about 5 pixels at most. When both white lines are extracted, their positions are recorded in the white line position memory 27, and the white line detection in the search range a is continued for the next frame. {Refer to the processing steps 100 to 102 of FIG. 10}, if even one of them cannot be extracted, the operation flag for the road width extraction unit 21 is set to “ON”, and
The counter C 29 shown in is cleared and the search range is expanded to B (pixels) to perform the search {the processing step of FIG.
104, 105}. When the white line at the above position is not extracted, the position of the white line of the previous frame is used as it is, and the search range b is wider than the search range a. Is three times the search range. {Refer to the processing steps 106 to 108 in FIG. 10} In the expanded search range B, if none of the, can be extracted, the operation flag for the road width extraction unit 21 is set to “ON” and the counter C 29 is set to 1. Counts and repeats the number of times recorded in the repeat count memory 25.
During the repetition, if both can be extracted, the white line position memory 27 is updated and the left white line is extracted in the search range A. By repeating such processing, the counter C
When 29 exceeds the number of times recorded in the repetition number memory 25, the process returns to the initial process of white line detection described in FIG. {Fig. 10
Processing steps 109, 110, and 111} Next, the lane area calculation unit 24 starts the operation only when the white line detection unit 23 can extract both left white lines at the white line position.

The coordinates (X, Y coordinates) of the two points of the left white line in the white line position memory 27, the road width W in the road width memory 221, and the Y coordinate previously recorded in the infinite distance position memory 28 (here, a). ) Is used to output the vertex coordinates of the triangles A, B, and C in the lane area, which is the final result of this embodiment. The coordinates of each vertex can be calculated by a simple product-sum operation, as shown in FIGS. 11 (a) and 11 (b).

First, the X coordinate x of the point B_bTo calculate. Left white
Since the Y coordinates of the two points on the line are fixed values, the denominator value is always
It is constant. Therefore, as shown in FIG.
Then, the product is obtained twice and the sum is obtained once. That is, in FIG. 11 (a)
In the triangular image shown, B1 (x₁, y₁), B2 (x₂,
y₂) Is the white line detection unit 23 described in FIG. 9 and FIG.
Is required in. Also, the B1 (x₁, y₁), B2 (x₂, y₂) Connect
The straight line L when extended to the infinity line (y = a) side is
The intersection with the distance line is the intersection A (x_a, y_a). Also shown
The intersection B (x) of the straight line L and the straight line l with the image end (X axis)
_b, y_b), C (x_c, y _c), The straight line B1 (x₁, y₁), B2 (x₂,
y₂) Form a right triangle and the line B1 (x₁, y₁), B (x_b, y
_b) Form a right triangle, the points are similar.
X coordinate of B x_bIs easy to calculate.

Similarly, the X coordinate x _a of the point A is the above x
_If you use _b , you can calculate it by two products and two sums. Furthermore,
Since the X coordinate x _c of the point C can be calculated in advance because y2-a is a fixed value, it can be calculated by the product of 3 times and the sum of 2 times. As described above, the vertices of the triangle can be calculated by the product of 7 times and the sum of 5 times.

Further, since the frames are continuous, the position of the point A which is the point at infinity does not change extremely compared with the previous frame. Therefore, if the difference between the points A in the preceding and succeeding frames is greater than a certain standard, a correct area cannot be extracted, and the white line detection unit 23 repeats the above white line detection.

To summarize the lane area extraction processing according to the present invention, first, at the time of initialization, the operation flag for the road width extraction unit 21 is "ON". Therefore, as shown in FIG. Two points on the left white line and a virtual road width W
When the intersection point A between the extracted left white line and the infinity line is obtained, the right white line is obtained from the intersection point and the virtual road width W, and the lane area is extracted. This is called initial processing.

Then, as shown in FIGS. 2 (e) and 3, the extracted left white line is centered on the position of the white line in the preceding frame for the next frame image, Repeatedly searching the neighborhood (search range a) to find the position of the new white line, and if the search for the white line fails in this search, the search range is expanded to a → b, and the white line is extracted. If the above two white lines can be extracted, the search range is returned to b → a, and the normal white line search is repeated.

When the white line cannot be extracted even by the white line extraction processing with the expanded search range, the operation flag for the road width extraction unit 21 is set to "ON", and the process returns to the initial processing shown in FIG. By repeating the extraction of two points and the extraction of the road width W, and repeating any of the above-mentioned initial processing and white line extraction processing (normal processing and processing with an expanded search range) for each frame, a small amount of calculation is required. The lane area can be accurately extracted at high speed.

In the above embodiment, the two white lines, for example, the white lines are detected, and the white line on the left side is used as a reference to predict the white line on the right side. It goes without saying that the white line may be on the left side or the right side.

Next, the horizontal line / vertical line candidate extraction processing in the rectangle extraction units 3L, 3R will be described with reference to FIG. 12 with reference to FIG. Here, for convenience of description, the extraction of horizontal line candidates will be described as an example, but in the case of extraction of vertical line candidates, if the Y coordinate in the following description is interpreted as the X coordinate,
The same process is performed.

First, the horizontal edge extraction unit of FIG. 12 scans an image using a predetermined mask pattern to extract horizontal edges which are characteristic pixels forming a rectangle. next,
In the horizontal line candidate extraction unit of FIG. 12, as shown in FIG. 4, each line (y = 1, 2, ...) Is scanned from left to right, and the pixels on each line and the upper and lower one pixels are scanned. Check a total of 3 pixels,
As long as any one of the three pixels is an edge pixel,
As the same horizontal edge, the scanning is continued and the same horizontal edge is tracked. Here, while tracking the same edge, the edge pixels are counted for each line (that is, the line of interest, the +1 line, the −1 line).

If the edge is not present in any of the three pixels by continuing the above tracking, a rectangular horizontal line candidate is present in the line having the largest number of edge pixels as a result of the counting. As a matter of course, in the course of the tracking, the coordinates from the edge pixel first appearing (X coordinate) to the last pixel are also extracted as the horizontal line candidate γ of the rectangle. The above operation is executed up to the last line of the image.

As described above, according to the rectangle extracting unit of the present invention, in the horizontal (vertical) line candidate extracting unit, even if the edge pixels are not adjacent to each other, if they exist within the range of ± α pixels, The same horizontal (vertical) line can be extracted as a candidate. Therefore, even if the edge pixels that originally form the same contour line are not continuously adjacent to each other due to an error in capturing an image, it can be extracted as a continuous horizontal (vertical) line candidate γ.

As described above, the road environment recognition apparatus according to the present invention creates an edge image in which the contour line of the object is extracted from the image obtained by the image pickup means fixedly provided in the traveling direction of the road environment recognition apparatus. Then, using the edge image, in order to extract the marker lines on the left and right sides of the host vehicle, the intersection point between the infinity line determined by the imaging means and the extracted, for example, the left white line is obtained, and the virtual line is obtained. Guess the white line on the right from the wide road width W. When extracting the left white line,
The left white line is searched for at the defined two places, and the point having the maximum brightness is set as the left white line. Also, the white line in the current frame is searched based on the position of the white line extracted in the previous frame.
At this time, when the white line cannot be searched at the above two places, the search range of the white line is changed. Further, when a rectangle is extracted from the image, if an edge exists within a range of ± α pixels (for example, α = 1, 2), it is extracted as one continuous straight line edge, and a horizontal line or vertical line candidate is extracted. The feature is that it is set to γ.

[0056]

As described above in detail, according to the road environment recognition apparatus of the present invention, the lane area extraction processing by the lane area extraction apparatus can be performed with a small amount of calculation, so that high speed such as automatic driving can be achieved. It becomes easy to apply to the visual system of the self-propelled vehicle that needs the property.

In the horizontal (vertical) line candidate extraction unit in the rectangle extraction unit, even when the edge pixels are not adjacent to each other,
If it exists within the range of ± α pixels, it can be extracted as a candidate for the same horizontal (vertical) line. For this reason, even if the edge pixels that originally form the same contour line are not adjacent to each other due to an error when capturing an image, it is possible to extract them as continuous horizontal (vertical) line candidates.

[Brief description of drawings]

FIG. 1 is an explanatory diagram (1) of the principle of the present invention.

FIG. 2 is an explanatory diagram of the principle of the present invention (No. 2)

FIG. 3 is an explanatory diagram of the principle of the present invention (No. 3)

FIG. 4 is an explanatory diagram of the principle of the present invention (No. 4)

FIG. 5 is a diagram showing an embodiment of the present invention (No. 1).

FIG. 6 is a diagram showing an embodiment of the present invention (No. 2).

FIG. 7 is a diagram showing an embodiment of the present invention (part 3).

FIG. 8 is a diagram showing an embodiment of the present invention (No. 4).

FIG. 9 is a diagram showing an embodiment of the present invention (No. 5).

FIG. 10 is a diagram showing an embodiment of the present invention (No. 6).

FIG. 11 is a diagram showing an embodiment of the present invention (No. 7).

FIG. 12 is a diagram showing another embodiment of the present invention.

FIG. 13 is a diagram illustrating a conventional lane area extraction device.

FIG. 14 is a view for explaining a conventional horizontal line / vertical line candidate recognition means.

[Explanation of symbols]

1L, 1R Imaging means 2L, 2R Lane area extraction device 3L, 3R Rectangle extraction unit 4 Rectangle matching &
Distance calculation unit 20 Image generation unit 21 Road width extraction unit 22 Road width memory 23 White line detection unit 24 Lane area calculation unit 25 Repeat count memory 26 Search width memory 27 White line position memory 28 Infinity line position memory 29 Counter C 30 Horizontal edge extraction unit 31 Horizontal line candidate extraction unit 100-111 Processing steps, White line search position a Infinity line position W Virtual road width a, b White line search range at white line detection unit A, B, C Triangle vertices α, β, indicating lane area γ Horizontal line, vertical line candidates

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication location G06T 7/00 G08G 1/09 V H04N 7/18 J (72) Inventor Eigo Segawa Kawasaki City, Kanagawa Prefecture 1015 Kamiodanaka, Nakahara-ku, Fujitsu Limited

Claims (5)

[Claims] 1. At least an image pickup means for picking up an image in front of a vehicle in a traveling direction, and an image obtained by the image pickup means,
A road environment recognition device comprising a lane area extraction device for recognizing an area of the own lane by extracting sign lines on the left and right sides of the own vehicle, wherein the lane area extraction device includes the image pickup means for the vehicle. The infinity line determined by being fixed, and the extracted intersection of the left side or the one of the right side marker lines, the intersection, and the virtual road width that can be recognized on the image, the right side, Or equipped with a means to guess the other marking line on the left,
A road environment recognition device, characterized by obtaining a traveling lane region in front of the road environment recognition device. 2. A lane area extraction device provided in the road environment recognition device according to claim 1, wherein the one marking line is searched at two predetermined locations to find a point of maximum brightness.
A road environment recognition device comprising a marker line detecting means for determining the position of the one marker line. 3. The lane area extraction device provided in the road environment recognition device according to claim 1, wherein the position of the marker line in the current frame is searched based on the position of the marker line extracted in the previous frame. A road environment recognition device comprising a marker line search means. 4. The lane area extraction device provided in the road environment recognition device according to claim 1, further comprising marking line detection means for selectively changing a range for searching the position of the marking line. Road environment recognition device. 5. A rectangular vehicle is extracted from each image picked up by a plurality of image pickup means provided in the road environment recognition apparatus according to claim 1, and a forward vehicle is detected based on a shift of the rectangular position extracted on the image. The plurality of rectangle extracting means necessary for calculating the distance to the one is one feature when it is detected that feature pixels forming a rectangle exist within a range of width ± α pixels (α = integer). A road environment recognition device characterized by extracting as pixels and using as candidates for horizontal and vertical lines.