JP2011170596A - System, method and program for recognizing compartment line - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To recognize compartment lines on road surfaces with high accuracy. <P>SOLUTION: A system for recognizing compartment lines includes a vehicle speed detection means for detecting the speed of own vehicle; a yaw rate detection means for detecting the yaw rate of own vehicle; an image input means for inputting the original image of a road surface; a composite bird's-eye image generation means for generating a composite bird's-eye image, based on original images at a plurality of different times and associating the vehicle speed and the yaw rate at the generation of the composite bird's-eye image with the composite bird's-eye image; a compartment line candidate detection means for detecting compartment line candidates from among at least one of the original image and the bird's-eye image; and a compartment line candidate verification means for extracting sections where vehicles travel in straight lines, recognizing the compartment line candidates for each of the sections, and verifying whether the compartment line candidates recognized are suitable as compartment lines. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a lane marking recognition system, a lane marking recognition method, and a lane marking recognition program that can recognize a lane marking on a road surface with high accuracy.

  In recent years, in order to execute accurate navigation in car navigation and the like, it has become an important issue to accurately grasp the vehicle position. For example, measures for improving the accuracy of the vehicle position information are taken using GPS, peripheral image analysis, and the like. Here, for the analysis of peripheral images, there are recognition techniques using a combined bird's-eye view image combined with image recognition for each frame and images for each time series.

  On the other hand, a synthetic bird's-eye image creation unit that generates a synthesized bird's-eye image based on an original image at a plurality of different times; a lane line candidate detection unit that detects a lane line candidate from the original image; There is known a lane recognition device having lane marking candidate verification means for verifying whether or not it is suitable as a line (see Patent Document 1).

JP 2009-169510 A

  For example, in the lane recognition device shown in Patent Document 1, it is assumed that a lane marking on a road surface is recognized when a lane is changed. First, when the vehicle is traveling straight (FIG. 7 (a)), the lane markings are substantially vertical on the synthesized bird's-eye view (FIG. 7 (b)), and can be estimated by linear approximation. Can recognize lines. Further, when changing lanes during high-speed travel (FIG. 8A), it is dangerous to change lanes abruptly, so the amount of lateral movement per hour can be kept small. Therefore, as described above, the lane marking can be estimated by a substantially straight line approximation (FIG. 8B), so that the lane marking can be recognized with high accuracy.

  However, when changing lanes during low-speed travel (FIG. 9A), the amount of movement in the horizontal direction per time is large, so the lane markings do not become straight on the synthesized bird's-eye view image. Accordingly, it is difficult to approximate the lane line with a single straight line on the synthesized bird's-eye view image (FIG. 9B), and the lane line recognition accuracy may be reduced.

  One aspect of the present invention is input by vehicle speed detection means for detecting the vehicle speed of the own vehicle, yaw rate detection means for detecting the yaw rate of the own vehicle, image input means for inputting an original image of the road surface, and the image input means. The vehicle speed detected by the vehicle speed detecting means when the synthesized bird's-eye image is generated based on the original images at a plurality of different times and the synthesized bird's-eye image is generated, and the yaw rate detecting means A synthesized bird's-eye image generating means for associating the yaw rate detected by the step, a lane line candidate detecting means for detecting a lane line candidate from at least one of the original image and the synthesized bird's-eye image, the synthesized bird's-eye image, and the Based on the associated vehicle speed and yaw rate, a straight section of the vehicle is extracted, and the straight line section is detected by the lane marking candidate detection means. Wherein recognizing the lane line candidate, lane line candidates the recognition and a lane line candidate verification means for assaying whether suitable as partition lines, a division line recognition system, characterized in that.

  According to this aspect, the marking line on the road surface can be recognized with high accuracy.

  Further, according to one aspect of the present invention, the step of detecting the vehicle speed of the host vehicle, the step of detecting the yaw rate of the host vehicle, the step of inputting the original image of the road surface, and the input original images at a plurality of different times Generating a synthesized bird's-eye image, and associating the synthesized bird's-eye image with the detected vehicle speed when the synthesized bird's-eye image is generated, and the detected yaw rate, the original image, and the synthesized image Based on the step of detecting lane marking candidates from at least one of the bird's-eye images, and the combined bird's-eye image and the associated vehicle speed and yaw rate, a straight section of the vehicle is extracted, and the detection is performed for each straight section. And recognizing whether the recognized lane line candidate is suitable as a lane line, or a lane line recognition method characterized by comprising:

  Further, according to one aspect of the present invention, a process for detecting the speed of the host vehicle, a process for detecting the yaw rate of the host vehicle, a process for inputting an original image of a road surface, and the input original images at a plurality of different times And a process of associating the detected bird speed with the detected yaw rate when the synthesized bird's-eye image is generated, and the original image and the synthesized image. Based on the processing for detecting lane marking candidates from at least one of the bird's-eye images, and the combined bird's-eye image and the associated vehicle speed and yaw rate, a straight section of the vehicle is extracted, and the detection is performed for each straight section. A lane line recognition program for causing a computer to recognize a recognized lane line candidate and to test whether or not the recognized lane line candidate is suitable as a lane line. Good.

  According to the present invention, it is possible to provide a lane marking recognition system, a lane marking recognition method, and a lane marking recognition program that can recognize a lane marking on a road surface with high accuracy.

1 is a block diagram illustrating a schematic configuration of a lane marking recognition system according to an embodiment of the present invention. It is a block diagram which shows the schematic system configuration | structure of the control apparatus of the lane marking recognition system which concerns on one embodiment of this invention. (A) It is a figure which shows an example of the original image input by the image input device at the time of the time T-1. (B) It is a figure which shows an example of the original image input by the image input device at the time of the time T. (C) It is a figure which shows an example of the bird's-eye view image produced | generated from the original image of Fig.3 (a) at the time of the time T-1. (D) It is the image which moved the pixel of the bird's-eye view image of FIG.3 (c) to the vertical direction only by the distance which the own vehicle moved between the time T-1 and the time T. FIG. (E) It is a figure which shows an example of the synthetic | combination bird's-eye view image which synthesize | combined the image obtained by making FIG.3 (b) into a bird's-eye view image, and the image of FIG.3 (d). It is a flowchart which shows an example of the processing flow of the angle determination of a lane line candidate and the line type determination of a lane line candidate by the lane line candidate test part which concerns on one embodiment of this invention. It is a figure which shows an example of the search area set to the synthetic bird's-eye view image. It is a figure which shows an example of the position of the lane marking candidate in a synthetic bird's-eye view image. (A) It is a top view which shows an example of the state which the vehicle is going straight ahead. (B) It is a figure which shows the lane marking on a synthetic bird's-eye view image when a vehicle is a straight ahead state. (A) It is a top view which shows an example of a state when the vehicle is changing lanes at high speed driving | running | working. (B) It is a figure which shows the lane marking on a synthetic bird's-eye view image in the lane change at the time of high speed driving | running | working. (A) It is a top view which shows an example of a state when the vehicle is changing lanes by low-speed driving | running | working. (B) It is a figure which shows the lane marking on a synthetic bird's-eye view image in the lane change at the time of low speed driving | running | working.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a lane marking recognition system according to an embodiment of the present invention. The lane marking recognition system 10 includes a control device 1, an image input device 2, a vehicle speed sensor 3, and a yaw rate sensor 4.

  The control device 1 includes, for example, a CPU (Central Processing Unit) 1a that performs control processing, arithmetic processing, and the like, a ROM (Read Only Memory) 1b that stores a control program executed by the CPU 1a, an arithmetic program, processing data, and the like. The hardware configuration is centered on a microcomputer composed of a RAM (Random Access Memory) 1c and the like.

  The image input device 2 is configured by a photographing device such as a camera that can photograph a lane marking formed on a road surface or the like, and a captured image including the lane marking is input. Hereinafter, the captured image input from the image input device 2 is referred to as an original image. The image input device 2 is attached to a vehicle such as an automobile, for example. Hereinafter, the vehicle to which the image input device 2 is attached is referred to as the own vehicle.

  The control device 1 may be mounted on the host vehicle, and the connection between the image input device 2 and the control device 1 may be a wired connection. The connection between the image input device 2 and the control device 1 may be a wireless connection without mounting the control device 1 on the own vehicle. The image input device 2 outputs an original image taken by a camera or the like to a synthetic bird's-eye image generation unit 11 described later.

  The vehicle speed sensor 3 is a specific example of vehicle speed detection means, and can detect the vehicle speed of the host vehicle. The vehicle speed sensor 3 detects the vehicle speed using, for example, a rotation sensor that can detect the wheel speed of the host vehicle. The vehicle speed sensor 3 is connected to the control device 1 and outputs the detected vehicle speed to the control device 1.

  The yaw rate sensor 4 is a specific example of the yaw rate detection means, and can detect the yaw rate of the own vehicle using, for example, a gyro sensor attached to the own vehicle. The yaw rate sensor 4 is connected to the control device 1 and outputs the detected yaw rate of the host vehicle to the control device 1.

  FIG. 2 is a block diagram showing a schematic system configuration of the control device of the lane marking recognition system according to the present embodiment. The control device 1 includes a synthesized bird's-eye image generation unit 11, a lane line candidate detection unit 12, a lane line candidate test unit 13, a lane line position estimation unit 14, and a lane line position output unit 15. .

  The synthesized bird's-eye image generation unit 11 is a specific example of the synthesized bird's-eye image generation unit. The synthesized bird's-eye image generation unit 11 synthesizes original images photographed at a plurality of different times by the camera of the image input device 2 and the like. Is generated. Here, the synthesized bird's-eye image generation unit 11 generates a synthesized bird's-eye image as shown in FIGS. In the present embodiment, the bird's-eye view image is, for example, an image obtained by converting a landscape photographed in the original image into an image when viewed in the vertical direction in the vertical direction in the real world. Bird's-eye imaging refers to converting an original image into such an image.

  The synthesized bird's-eye image generation unit 11 uses the original images taken at a plurality of different times and connects the partial bird's-eye images obtained by converting the partial areas of the original images into bird's-eye images, thereby generating a synthesized bird's-eye image. In the example shown in FIGS. 3A to 3E, the image input device 2 is installed in the direction opposite to the traveling direction of the host vehicle and photographs the rear direction of the host vehicle.

  FIG. 3A shows an original image at time T-1, and FIG. 3B shows an original image at time T. In the lower part of FIG. 3 (b), a portion that was not photographed at time T-1 and entered the photographing range of the original image at time T is shown as a hatched portion (this is referred to as a front region 31). ). In addition, the arrow in Fig.3 (a) and (c) represents the advancing direction of the own vehicle.

  FIG. 3C is a bird's-eye view image generated from the original image in FIG. 3A at time T-1. FIG. 3D is an image in which the pixels of the bird's-eye view image of FIG. 3C are moved in the vertical direction by the distance traveled by the host vehicle from time T-1 to time T. Since the bird's-eye view image of FIG. 3C does not include a portion corresponding to the near region 31 of FIG. 3B, the portion corresponding to the near region 31 in the image of FIG. It has become.

  Here, the synthesized bird's-eye image generation unit 11 combines the image obtained by converting the bird's-eye image of FIG. 3B and the image of FIG. 3D into a blank portion of FIG. 3D. 32 (portion corresponding to the front region 31 in FIG. 3B) is filled in to generate the synthesized bird's-eye view image in FIG. In addition, the generation method of the said synthetic bird's-eye view image is disclosed by patent document 1, The generation method can be used.

  The synthetic bird's-eye image generation unit 11 generates a synthetic bird's-eye image so as to create an image having a height (vertical length) for one line every time the vehicle travels 1 m. For example, when the host vehicle travels 100 meters, the synthetic bird's-eye image generation unit 11 generates a synthetic bird's-eye image for 100 lines.

  Furthermore, the synthesized bird's-eye image generation unit 11 adds the vehicle speed detected by the vehicle speed sensor 3 when the synthesized bird's-eye image is generated to the generated synthesized bird's-eye image, and the yaw rate of the host vehicle detected by the yaw rate sensor 4. And associate. The synthesized bird's-eye image generation unit 11 outputs the generated synthesized bird's-eye image and the vehicle speed and yaw rate associated with the synthesized bird's-eye image to the lane line candidate test unit 13.

  The lane line candidate detection unit 12 is a specific example of a lane line candidate detection unit, and detects the position of a lane line candidate as a lane line candidate using an original image taken by a camera or the like of the image input device 2. To do. The lane line candidate detection unit 12 may detect the position of the lane line candidate using the synthesized bird's-eye image generated by the synthesized bird's-eye image generation unit 11.

  For example, if the number of gradations of the original image is 256 in the original image from the image input device 2, the lane marking candidate detection unit 12 uses a pixel having a luminance value of 200 or more as the foreground, and other pixels as the background. In this way, the original image is binarized into a foreground and a background. Then, the lane line candidate detection unit 12 detects straight lines by a known Hough transform based on the pixels that are the foreground in the binarized original image, and the detected straight lines are used as lane line candidates. . The lane line candidate detection unit 12 outputs the detected lane line candidates to the lane line candidate test unit 13.

  The lane line candidate verification unit 13 is a specific example of the lane line candidate verification unit, and the lane line candidates detected by the lane line candidate detection unit 12 are included in the combined bird's-eye image generated by the synthetic bird's-eye image generation unit 11. The position and line type of the lane line candidate are determined, and the lane line candidate is recognized. Then, the lane line candidate verification unit 13 verifies whether or not the recognized lane line candidate is suitable as a lane line.

  The lane line candidate verification unit 13 verifies whether the lane line candidate is suitable as a lane line based on, for example, the position of the lane line candidate recognized on the synthesized bird's-eye view image or the pixel value in the vicinity thereof. More specifically, the lane line candidate verification unit 13 verifies that the corresponding lane line candidate is suitable as a lane line when the sum of pixel values in a later-described synthesized bird's-eye image is equal to or greater than a preset threshold value. May be.

  The lane line position estimation unit 14 estimates the position of the lane line at the current time on the basis of the position of the lane line candidate verified as suitable as a lane line by the lane line candidate verification unit 13. The lane line position output unit 15 outputs the position of the lane line at the current time estimated by the lane line position estimation unit 14 to a display device such as a display, a printer, or the like.

  By the way, when changing lanes during low-speed travel (FIG. 9A), the amount of movement in the horizontal direction per time is large, so the lane markings do not become straight on the synthesized bird's-eye image. Therefore, in the conventional lane marking recognition system, it is difficult for the lane marking to be approximated by a single straight line on the synthesized bird's-eye view image (FIG. 9B), which may reduce the lane marking recognition accuracy. is there.

  Therefore, in the lane marking recognition system 10 according to the present embodiment, a straight section of the vehicle is extracted in the synthesized bird's-eye image based on the vehicle speed and yaw rate associated with the synthesized bird's-eye image, and lane line candidates are obtained for each straight segment. The lane line candidates are combined, and the lane line candidates subjected to the line approximation are combined to recognize the entire lane line candidates. Thereby, also in the lane change at the time of low-speed driving | running | working, the division line of a synthetic bird's-eye view image can be approximated and recognized as one straight line. That is, it is possible to recognize the lane markings on the road surface with high accuracy when changing lanes during low-speed driving.

  Next, a method for determining the position and line type of a lane line candidate by the lane line candidate verification unit 13 and recognizing the lane line candidate will be described in detail. FIG. 4 is a flowchart showing an example of a processing flow of lane line candidate angle determination and lane line candidate line type determination by the lane line candidate verification unit according to the present embodiment.

  First, the lane line candidate verification unit 13 is based on the synthesized bird's-eye image from the synthesized bird's-eye image generation unit 11 and the vehicle speed and yaw rate associated with the synthesized bird's-eye image. The extracted straight section is extracted, and the extracted straight section is set as search sections H1 to H3 on the synthesized bird's-eye image as shown in FIG. 5 (step S101).

  Here, the lane line candidate test unit 13 extracts a section in which the yaw rate associated with the synthesized bird's-eye image is less than a preset threshold (for example, ± 0.01 rad / sec) as a straight section, and searches the search sections H1 to H1. Set as H3. The threshold value may be changed based on vehicle characteristics (power performance, suspension performance, etc.), continuity of vehicle speed and yaw rate, and the like. For example, the lane marking candidate test unit 13 may increase the threshold when the continuity of the yaw rate is small.

  Also, the lane line candidate test unit 13 uses the vehicle speed associated with the composite bird's-eye image to obtain the travel distance of the section where the yaw rate has changed or the section where the yaw rate has not changed, and from this travel distance, Height (vertical length) can be obtained.

  As shown in FIG. 5, in the search sections H1 to H3, the host vehicle is traveling substantially straight, and the change amount of the yaw rate is small and less than the threshold value. On the other hand, at the boundary between the search sections H3 and H2 and the boundary between the search sections H2 and H1, the vehicle turns right and left, respectively, and the amount of change in the yaw rate is large.

  Next, the lane line candidate verification unit 13 determines the angles of the lane line candidates detected by the lane line candidate detection unit 12 in each of the search sections H1 to H3 set in the synthesized bird's-eye image (division line). Candidate angle determination). Here, for example, as shown in FIG. 6, the position of the lane line candidate is set as a front segment of the synthetic bird's-eye view image 61, and the angle of the lane line candidate is within one of a predetermined angle range 62. The lane marking candidate position 63 in the synthetic bird's-eye view image 61 is expressed.

  The lane line candidate verification unit 13 changes the angle of the predetermined angle range (for example, ± 30 degrees) gradually (for example, by 1 degree) within each search section of the synthetic bird's-eye view image, while determining the lane line candidate. The sum of the pixel values at the position or the periphery thereof, for example, the range of 20 cm left and right centering on the position of the lane marking candidate is calculated (step S102). Note that the lane marking candidate test unit 13 may change the range of the predetermined angle according to the value of the yaw rate detected by the yaw rate sensor 4.

  Then, when the angle search within the predetermined angle range is completed (step S103), the lane line candidate verification unit 13 determines an angle that maximizes the sum of the pixel values as the lane line candidate angle (step S104).

  Further, when the lane line candidate verification unit 13 determines the angle of the lane line candidate for all the search sections (YES in step S105), the lane line candidate determination unit 13 performs line type determination of the lane line candidate described later.

  The lane line candidate verification unit 13 calculates the lengths of white lines and blanks in each search section using the lane line candidate angles calculated for each search section of the composite bird's-eye view image in line type determination of the lane line candidates. (Step S106).

  When the lane line candidate test unit 13 calculates the lengths of the white line and the blank in all the search sections (YES in step S107), the lane line candidate test unit 13 adds up the lengths of the white line and the blank for each search section. Based on the total result, the line type of the lane line candidate is determined (step S108), and the lane line candidate is recognized.

  As described above, in the lane marking recognition system 10 according to the present embodiment, even if the lane marking is not represented by a single straight line on the synthesized bird's-eye image by performing linear approximation of the lane marking candidates for each straight section. As shown in FIG. 5, a straight line can be approximated by combining straight lines. Therefore, even in a lane change during low-speed traveling, the lane markings in the synthesized bird's-eye image can be recognized as being approximated by a single straight line. That is, it is possible to recognize the lane markings on the road surface with high accuracy when changing lanes during low-speed driving. Further, the accuracy of the vehicle position can be improved by combining the map information and the GPS information while recognizing the lane marking as described above.

  Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention.

  In the above-described embodiment, arbitrary processing can be realized by causing the CPU 1a to execute a computer program. In this case, the computer program can be provided by being recorded on a recording medium, or can be provided by being transmitted via the Internet or another communication medium.

  Further, the storage medium includes, for example, a flexible disk, a hard disk, a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD, a ROM cartridge, a battery-backed RAM memory cartridge, a flash memory cartridge, and a nonvolatile RAM cartridge. Furthermore, the communication medium includes a wired communication medium such as a telephone line, a wireless communication medium such as a microwave line, and the like.

DESCRIPTION OF SYMBOLS 1 Control apparatus 2 Image input device 3 Vehicle speed sensor 4 Yaw rate sensor 10 Completion line recognition system 11 Composite bird's-eye view image generation part 12 Completion line candidate detection part 13 Completion line candidate test part 14 Completion line position estimation part 15 Completion line position output part

Claims (7)

Vehicle speed detection means for detecting the vehicle speed of the vehicle;
A yaw rate detection means for detecting the yaw rate of the vehicle;
An image input means for inputting an original image of the road surface;
A synthesized bird's-eye image is generated based on the original images at a plurality of different times input by the image input means, and the vehicle speed detection means detected when the synthesized bird's-eye image is generated in the synthesized bird's-eye image. Synthetic bird's-eye view image generation means for associating a vehicle speed and the yaw rate detected by the yaw rate detection means;
Lane line candidate detection means for detecting a lane line candidate from at least one of the original image and the synthesized bird's-eye view image;
Based on the synthetic bird's-eye view image and the associated vehicle speed and yaw rate, a straight section of the vehicle is extracted, and the lane line candidate detected by the lane line candidate detection means is recognized for each straight section, A lane line candidate verification means for verifying whether the recognized lane line candidate is suitable as a lane line;
A lane marking recognition system comprising: The lane marking recognition system according to claim 1,
The lane line recognition system according to claim 1, wherein the lane line candidate verification unit sets a section where the yaw rate detected by the yaw rate detection unit is less than a threshold in the synthetic bird's-eye view image as the straight line section. A lane marking recognition system according to claim 2,
The lane marking candidate verification means changes the threshold according to vehicle characteristics of the vehicle, the vehicle speed detected by the vehicle speed detection means, or the continuity of the yaw rate detected by the yaw rate detection means, A lane marking recognition system characterized by that. A lane marking recognition system according to any one of claims 1 to 3,
The lane line candidate verification means recognizes the position of the lane line candidate by determining each angle of the lane line candidate when the sum of predetermined pixel values becomes maximum in each straight section of the synthetic bird's-eye view image. A lane marking recognition system characterized by that. A lane marking recognition system according to any one of claims 1 to 4,
The lane line candidate verification means totalizes the lengths of white lines or blanks for each straight section of the synthetic bird's-eye view image, determines the line type of the lane line candidates based on the total length, and A lane marking recognition system characterized by recognizing a line candidate. Detecting the speed of the vehicle,
Detecting the yaw rate of the vehicle;
Inputting the original image of the road surface;
A composite bird's-eye image is generated based on the input original images at a plurality of different times, and the detected vehicle speed when the composite bird's-eye image is generated in the composite bird's-eye image, and the detected yaw rate, And associating
Detecting a lane line candidate from at least one of the original image and the synthesized bird's-eye image;
Based on the synthetic bird's-eye view image and the associated vehicle speed and yaw rate, a straight section of the vehicle is extracted, the detected lane line candidate is recognized for each straight section, and the recognized lane line candidate is defined as a section. Testing whether it is suitable as a line;
A lane marking recognition method comprising: Processing to detect the speed of the vehicle,
Processing to detect the yaw rate of the vehicle,
Processing to input the original road image,
A composite bird's-eye image is generated based on the input original images at a plurality of different times, and the detected vehicle speed when the composite bird's-eye image is generated in the composite bird's-eye image, and the detected yaw rate, The process of associating
Processing for detecting lane marking candidates from at least one of the original image and the synthesized bird's-eye view image;
Based on the synthetic bird's-eye view image and the associated vehicle speed and yaw rate, a straight section of the vehicle is extracted, the detected lane line candidate is recognized for each straight section, and the recognized lane line candidate is defined as a section. Processing to test whether it is suitable as a line;
A lane marking recognition program that causes a computer to execute.

Images