JP2008030619A - Kinds-of-road-division-line sorting system and road-division-line recognition sytem - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a kinds-of-road-division-line soring system capable of properly judging with a simple configuration the kinds of road division lines whose images are captured by an image pickup means. <P>SOLUTION: This kinds-of-road-division-line sorting system, which is provided with the image pickup means that captures images around a vehicle, judges which of a straight road division line drawn in a straight line on a road and a dotted road division line laid or drawn on the road is the road division line whose image captured by the image pickup means. The system is also equipped with a characteristic-points-on-scanning-lines sampling means that sets up a plurality of scanning lines extending in a direction corresponding to a road width direction on a captured image of the image pick-up means and extracts characteristic points on plurality of scanning lines. It judges that the road division line on the picked-up image of the image pickup means is the straight road division line, when a prescribed number of the scanning lines or more on which a prescribed number of the characteristic points or more are extracted by the characteristic-points-on-scanning-lines sampling means exist continuously in the direction that a plurality of the scanning lines are placed side by side. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  According to the present invention, a road lane line imaged by the imaging means is a straight road lane line drawn in a straight line on the road, and a point-string road lane line laid or drawn in a point sequence on the road. The present invention relates to a road lane line type determination device that determines which one is used, and a road lane line recognition device that includes a portion thereof and recognizes road lane lines in a captured image of an imaging unit.

  Conventionally, a control system for recognizing a road marking line such as a white line on an image of a camera that captures the front or rear of the vehicle and automatically outputting an auxiliary steering force based on this recognition so that the host vehicle travels in the travel lane Are known under names such as Lane Keeping Assist (LKA) (see Non-Patent Document 1, for example). In such a control system, for example, among the image elements constituting the image data of the camera, those whose luminance change compared to the adjacent image element is equal to or greater than a threshold value are extracted as feature points, and are linear (or curved). Processing such as recognizing the aligned feature points as an outline such as a white line is performed.

  By the way, road marking lines imaged by the camera are not only straight lines such as solid lines and broken lines drawn with white lines or yellow lines (hereinafter referred to as white lines), but also botts dots, cat's eyes, and the like. There is a thing (hereinafter referred to as botsdots or the like) laid or drawn in a dotted line on the road. Therefore, when recognizing a road marking line on an image, it is desirable to switch the image analysis mode (method) between a white line or the like and a botsdot or the like. This is because the difference in brightness from the road (asphalt) and the shape of the feature points that appear are different between the two, and the fact that both are recognized uniformly using the same analysis method causes a decrease in accuracy. .

An invention for an apparatus that recognizes road marking lines by switching between a recognition process based on a luminance change for recognizing a white line or the like and a recognition process based on a pattern matching for recognizing a botsdot or the like is disclosed (for example, , See Patent Document 1). This apparatus includes an optical signal transmission / reception apparatus such as a laser radar, and switches the recognition process based on a signal received as reflected light of an optical signal transmitted to a road marking line.
JP 2004-139338 A Toyota Motor Corporation, "Crown Majesta New Model Manual (Part No. 7109100)", Toyota Motor Corporation Service Department, issued July 5, 2004, Chapter 10 Body & Electrical, p10-287-10-306

  However, an optical signal transmission / reception device such as a laser radar is generally not provided as a standard equipment in a vehicle, and installation of a new signal causes problems of installation space and cost. Laser radar also has a problem that it is difficult to detect depending on weather conditions such as heavy rain, heavy snow, and heavy fog.

  The present invention is for solving such a problem, and a road lane line type determination device capable of appropriately determining the type of road lane line imaged by an imaging means with a simple configuration, and this The main purpose is to provide a road lane marking recognition device that can appropriately recognize a road lane marking in a captured image of an imaging means.

  In order to achieve the above object, a first aspect of the present invention includes an imaging means for imaging the periphery of a vehicle, and a straight road marking line in which a road marking line imaged by the imaging means is drawn linearly on a road And a road lane line type determination device that determines whether the road lane line is a constellation road lane line that is laid or drawn in a dotted line pattern on the road, in the road width direction on the captured image of the imaging means A plurality of scanning lines extending in a corresponding direction are set side by side, and feature points on scanning lines are extracted to extract feature points on the plurality of scanning lines, and feature points greater than a predetermined number are extracted by the feature points on scanning lines. Determining that the road lane marking on the captured image of the imaging means is a straight road lane marking when there are a predetermined number or more of the scanned lines continuously in the direction in which the plurality of scanning lines are arranged. It is a feature.

  According to the first aspect of the present invention, scanning lines from which feature points of a predetermined number or more are extracted by the feature point extracting means on the scanning lines are continuously arranged in the direction in which the plurality of scanning lines are arranged (that is, in the longitudinal direction of the road). If the predetermined number or more exist, it is determined that the road lane marking on the captured image of the imaging means is a straight road lane marking. In the case of a dotted line lane marking, since many feature points do not appear continuously in the longitudinal direction of the road, it is appropriately determined whether or not the road lane marking is a straight road lane marking based on the above determination. Can be determined. In addition, since the type of the road lane marking is determined by analyzing the captured image of the camera, the space and cost problems are small. Therefore, it is possible to appropriately determine the type of road marking line imaged by the imaging means with a simple configuration.

  According to a second aspect of the present invention, there is provided imaging means for imaging the periphery of a vehicle, and road lane markings imaged by the imaging means are linear road lane markings drawn linearly on the road, and dot trains on the road A road lane line type determination device that determines whether a point lane road lane line laid or drawn on a road is a direction corresponding to a road longitudinal direction of a plurality of blocks on a captured image of an imaging means And an in-block feature point extracting means for extracting feature points in a plurality of blocks, and when there are more than a predetermined number of blocks from which feature points have been extracted by the in-block feature point extracting means, the imaging means It is determined that the road lane marking on the captured image is a straight road lane marking.

  According to a third aspect of the present invention, there is provided an imaging means for imaging the periphery of a vehicle, and a road lane line imaged by the imaging means is a straight road lane line drawn in a straight line on a road, and a point sequence in a road A road lane line type determination device that determines whether a point lane road lane line laid or drawn on a road is a direction corresponding to a road longitudinal direction of a plurality of blocks on a captured image of an imaging means And in-block feature point extracting means for extracting feature points in a plurality of blocks, and a block number storage means for storing the number of blocks from which feature points have been extracted by the in-block feature point extracting means, When the average number of blocks stored in the block number storage unit from the past to the latest is a predetermined number or more, it is determined that the road lane line on the captured image of the imaging unit is a straight road lane line Also features It is.

  According to a fourth aspect of the present invention, there is provided imaging means for imaging the periphery of a vehicle, and road lane lines captured by the imaging means are linear road lane lines drawn in a straight line on the road, and dot trains on the road A road lane line type determination device that determines whether the lane line is a line of lane lines drawn or drawn on a road, and extends in a direction corresponding to a road width direction on a captured image of an imaging unit A plurality of scanning lines are set side by side and feature points on the scanning lines are extracted, and a plurality of blocks are set side by side in a direction corresponding to the longitudinal direction of the road on the captured image of the imaging means. In-block feature point extracting means for extracting feature points in a plurality of blocks, and a direction in which a plurality of scan lines are arranged in a scan line in which feature points of a predetermined number or more are extracted by the feature point extraction means on the scan line More than a predetermined number The road lane marking on the captured image of the imaging means is the straight line if at least one of the predetermined number of blocks whose feature points have been extracted by the in-block feature point extracting means exists. It is characterized by determining that it is a road lane marking.

  According to a fifth aspect of the present invention, there is provided imaging means for imaging the periphery of a vehicle, and road lane markings imaged by the imaging means are linear road lane markings drawn linearly on the road, and dot trains on the road A road lane line type determination device that determines whether the lane line is a line of lane lines drawn or drawn on a road, and extends in a direction corresponding to a road width direction on a captured image of an imaging unit A plurality of scanning lines are set side by side and feature points on the scanning lines are extracted, and a plurality of blocks are set side by side in a direction corresponding to the longitudinal direction of the road on the captured image of the imaging means. Intra-block feature point extraction means for extracting feature points in a plurality of blocks, and block number storage means for storing the number of blocks from which feature points have been extracted by the in-block feature point extraction means, and feature point extraction on a scanning line By means There are more than a predetermined number of scanning lines from which feature points of a fixed number or more have been extracted in the direction in which the plurality of scanning lines are arranged, and the average number of blocks stored in the block number storage means is a predetermined number When at least one of the above is satisfied, the road marking line on the captured image of the imaging unit is determined to be the straight road marking line.

  In the first, fourth, or fifth aspect of the present invention, the scanning line feature point extraction unit preferably performs scanning after performing predetermined noise removal processing on an image element on the scanning line. A means for extracting feature points on a line.

  In the second, third, fourth, or fifth aspect of the present invention, the in-block feature point extraction unit preferably performs a predetermined noise removal process on the image elements in the block. It is means for extracting feature points in the processed block.

  It should be noted that the predetermined noise removal processing for the image elements on the scanning line and the predetermined noise removal processing for the image elements in the block may be the same or different. These (or any of these) predetermined noise removal processes are preferably noise removal processes based on morphological operations.

  A sixth aspect of the present invention is a road lane marking recognition apparatus that has the road lane marking type determination device according to any aspect of the present invention and recognizes a road lane marking in a captured image of an imaging means, An image for analyzing a captured image of the imaging means by switching between a first analysis mode suitable for recognizing a road lane marking and a second analysis mode suitable for recognizing a point-like road lane marking. The image analysis means includes an analysis means, and when the image analysis means performs the analysis in the second analysis mode, the road lane line on the captured image of the imaging means is a straight line by the road lane line type determination device according to any aspect of the present invention. When it is determined that the road is a road lane marking, it is switched to the first analysis mode.

  ADVANTAGE OF THE INVENTION According to this invention, the road lane line type determination apparatus which can determine appropriately the classification of the road lane line imaged by the imaging means by simple structure, and the captured image of an imaging means including this partly It is possible to provide a road lane marking recognition apparatus capable of appropriately recognizing road lane markings.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the accompanying drawings.

  The best mode for carrying out the present invention will be described below with reference to the drawings. FIG. 1 is a diagram illustrating an example of the overall configuration of a road lane marking recognition device 1 according to an embodiment of the present invention. The road lane marking recognition device 1 includes a camera 10 and an image analysis device 20 as main hardware configurations. The road lane marking recognition device 1 analyzes a captured image of the camera 10 and recognizes a road lane marking in the image.

  Road marking lines include straight lines drawn in solid colors, broken lines, double lines, etc., in any color such as white lines or yellow lines (hereinafter referred to as white lines), botts dots, Some of them are laid or drawn on a road such as a cat's eye (hereinafter referred to as botsdots). Since there are analysis methods suitable for each of white lines, botsdots, and the like, the road lane marking recognition device 1 appropriately determines in the analysis process which road lane line is captured by the camera 10 and performs analysis. A method switching process is performed.

  The botsdots are road marking lines mainly used in North America, and are formed by arranging a plurality of three-dimensional disks made of ceramic, for example, having a diameter of about 100 mm and embedded in the road surface. A cat's eye is a road marking line in which a plurality of reflectors formed in a substantially rectangular shape are arranged, and has a characteristic of reflecting incident light in the same direction. For example, in Japan, road lane markings composed only of cat's eyes are used on curved roads other than highways, and in North America, they are used not only on curved roads but also on straight roads. Both the botsdots and cat's eyes are arranged in a state of slightly protruding from the road surface.

  The camera 10 can be attached at any position where the periphery of the vehicle can be imaged (in the following description, the front side of the vehicle is imaged). As the camera 10, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) camera having a predetermined number of pixels (5 million pixels or the like) is used. The captured image of the camera 10 is transmitted to the image analysis device 20 as an image signal generated by an interlace method such as NTSC (National Television Standards Committee).

  The image analysis apparatus 20 is, for example, a computer unit in which a ROM, a RAM, and the like are connected to each other via a bus with a CPU as a center, and includes an input / output interface, a timer, a counter, and the like. The ROM stores programs and data executed by the CPU. The captured image of the camera 10 is stored in a RAM (for example, a screen frame memory). The CPU reads a captured image (for example, an image signal for one frame) from the RAM as necessary, and performs analysis processing described later.

  The image analysis apparatus 20 includes, as main functional blocks realized by using the hardware as described above, a straight road lane marking analysis processing section 30, a point sequence road lane marking analysis processing section 40, and an analysis process. And a switching unit 50.

  The straight road lane marking analysis processing unit 30 analyzes a captured image of the camera 10 by an analysis method suitable for recognizing a white line or the like, and recognizes a road lane marking in the image. Specifically, for example, a plurality of scanning lines extending in the left-right direction of the image (corresponding to the road width direction) are set in a predetermined area on the lower side of the image (corresponding to the front side of the host vehicle), and the scanning is performed. A point whose luminance change in the horizontal direction of the image on the line is equal to or greater than a first threshold is extracted as a feature point. Then, feature points arranged in a straight line by Hough transform or the like are selected, and a straight line (or a curve) connecting the selected feature points is recognized as an outline such as a white line. FIG. 2 is a diagram schematically illustrating how the straight road lane marking analysis processing unit 30 sets scanning lines and extracts feature points. The scanning lines may be set at equal intervals, or based on the perspective method, the front side of the host vehicle may be set at a wide interval, and the side far from the host vehicle may be set at a narrow interval. In addition, since the image signal is transmitted by the interlace method, actually, the image signal limited to the predetermined section corresponds to the scanning line.

  The point-string-like road lane marking analysis processing unit 40 analyzes the captured image of the camera 10 by an analysis method suitable for recognizing botsdots and the like, and recognizes the road lane marking in the image. The point-string-like road lane marking analysis processing unit 40 includes a morphological calculation unit 42, a noise removal unit 44, and a feature point registration unit 46 as further subdivided functional blocks.

  The morphological operation unit 42 performs processing based on the morphological operation on the captured image of the camera 10.

  Specifically, first, a binarized image obtained by performing binarization processing using a second threshold value on an image grayscaled by shading correction (the element value of each image element is represented by a value of 0 or 1, for example). Generated image). That is, an element value of an image element having a luminance equal to or higher than the second threshold is set to a value 1, and an element value of an image element having a luminance lower than the second threshold is set to a value 0. The second threshold value is smaller than the first threshold value described above. This is because a white line has a clear contrast with a road surface such as asphalt, and noise can be removed more positively by increasing the threshold value. On the other hand, since the contrast with a road surface is not as clear as that of a white line or the like in Botsdots, not only noise elements but also botsdots may be removed if the threshold value is excessively increased.

  The noise removal unit 44 performs noise removal processing on the binarized image element after the morphological operation unit 42 performs processing. Specifically, for example, noise smaller than that of botsdots or the like is removed by performing an opening process in the morphological operation using a structural element slightly smaller than that of botsdots or the like. The present invention is not limited to this, and a two-dimensional image may be reproduced from image elements on a scanning line, and processing such as pattern matching and Hough conversion may be performed therein.

  Here, since the bots dot is circular, when the noise removal process as described above is performed, if the scanning line is on the end, it is removed as noise. However, by setting the number of scanning lines appropriately so that several scanning lines are applied to one bots dot, some of them are recognized as bots dots by the feature point registration unit 46 as described later. (See FIG. 3).

  The feature point registration unit 46 recognizes that the image element having the element value 1 remaining as a road lane line with respect to the image element after the noise removal unit 44 performs processing, and coordinates of both ends of the image element ( For example, how many pixels are expressed from one end of the scanning line) is registered in the RAM as a feature point related to the road marking line. This registered feature point is recognized as the outline of the road marking line on the scanning line. In addition, although arbitrary 1 scanning line was demonstrated among the set several scanning lines, the same process is performed about all the scanning lines.

  The analysis processing switching unit 50 refers to each processing stage of the straight road lane marking analysis processing unit 30 and the point-string lane marking analysis processing unit 40, and the road lane line captured by the camera 10 is a white line or the like. Judge whether it is botsdots or the like. Then, based on the determination result, an operation instruction is given to either the straight road lane marking analysis processor 30 or the point sequence road lane marking analysis processor 40 to provide an image signal from the camera 10.

  Specifically, while the straight road lane marking analysis processing unit 30 is operating, it is determined whether or not the white line or the like has been switched to botsdots or the like, and the dotted line lane marking analysis processing unit 40 is operated. While is operating, it is determined whether or not the bots dots or the like has been switched to a white line or the like.

  The determination as to whether or not the road lane line has been switched from a white line or the like to botsdots or the like has become unrecognizable (lost), for example, during recognition processing of a white line or the like by the straight road lane line analysis processing unit 30. In actuality, it may be determined that the white line or the like has been switched to botsdots or the like based on the fact that a point greater than or equal to the first threshold value described above has not been detected. Since white lines have a clear contrast with road surfaces such as asphalt compared to botsdots, etc., it is possible to determine whether or not the road lane lines have been switched from white lines to botsdots, etc. with sufficient accuracy. Can be done.

  On the other hand, if the determination as to whether or not the botsdots or the like has been switched to a white line or the like is made depending on whether or not the recognition process of the dotted line road lane marking analysis processing unit 40 has become unrecognizable, the determination accuracy is sufficient. It will not be a thing. This is because, in the analysis logic for recognizing botsdots or the like, a white line or the like that is larger than bottsdots and has a clear contrast with the road surface may be erroneously recognized as bottsdots or the like. If the recognition of the road lane marking is continued with such erroneous recognition, the white lane etc. will be recognized by an analysis method that is not originally suitable for recognizing the white line etc., for example, as will be described later, the road lane marking recognition device 1 If this output is used for vehicle control, it will contribute to vehicle wobbling, which is not preferable.

  Therefore, in order to determine whether or not the botsdots or the like has been switched to a white line or the like, the analysis processing switching unit 50 includes the on-scanning feature point extraction unit 52, the in-block feature point extraction unit 54, as further subdivided functional blocks. And a switching determination unit 56.

  The feature point extraction unit 52 on the scanning line extracts the feature points registered by the feature point registration unit 46 on each scanning line set by the morphological operation unit 42, and outputs the number of feature points on each scanning line to the RAM.

  The in-block feature point extraction unit 54 reproduces the original image from the image element after the noise removal unit 44 performs processing, and sets a plurality of (for example, 30) blocks on the reproduced image. The number of blocks including feature points is output to the RAM. The block is set from both sides of the host vehicle to the front of the host vehicle (aligned in the longitudinal direction of the road) (see FIG. 4). As for the setting of the blocks, as with the setting of the scanning line, blocks having the same width (length in the longitudinal direction of the road) may be arranged, or the width of the block on the front side of the host vehicle is set to be wide. Also good.

  The switching determination unit 56 refers to the outputs of the feature point extraction unit 52 on the scanning line and the feature point extraction unit 54 in the block, and determines whether or not the switch has been made from a bots dot or the like to a white line or the like according to the flowchart shown in FIG. Therefore, this flow is started when it is determined that the white line or the like has been switched to botsdots or the like, and the dotted line road marking line analysis processing unit 40 starts to operate.

  First, the switching determination unit 56 refers to the output of the feature point extraction unit 52 on the scanning line, and scans in which there are four or more feature points (two points on both ends of the bots dot etc. x four points on both sides of the host vehicle) or more. The maximum number of lines in the vertical direction of the image (hereinafter referred to as the maximum number of continuous lines) is calculated (S100).

  Then, it is determined whether or not the maximum continuous number is a predetermined number (for example, about several to several tens, more specifically about six) (S110), and the maximum continuous number is a predetermined number or more. If there is, it is determined that the road lane line imaged by the camera 10 is a white line or the like (S120), an operation instruction is given to the straight road lane line analysis processing unit 30, and a dotted line lane line analysis process is performed. A stop instruction is transmitted to the unit 40, the provision of the image signal from the camera 10 to the straight road lane marking analysis processing unit 30 is started (S130), and one routine of this flow is terminated.

  Here, the predetermined number is an adaptation value that is predetermined as a number that is not counted when the road lane marking is botsdots or the like. In other words, feature points detected due to the presence of botsdots, etc. are not of a nature that can be detected at high density in the longitudinal direction of the road, so there is a limit to the maximum continuous number counted when the road lane marking is bottsdots, etc. Occurs. Therefore, by setting a threshold value for this and making the above determination, it is appropriately determined that the road lane marking has been switched to a white line or the like. And when a road lane line is switched to a white line, etc., it will be immediately switched to an analysis process by the straight road lane line analysis processing unit 30, so before and after the road lane line is switched from a botsdot to a white line, etc. It is possible to appropriately recognize the road marking line.

  On the other hand, when the maximum continuous number is less than the predetermined number, the output of the feature point extraction unit 54 in the block is referred to, and the average number of effective blocks on the left and right (meaning both sides of the host vehicle) is calculated ( S140). Here, the number of effective blocks is the number of blocks including feature points, and the average number of effective blocks is the past several tens of frames (the frame is the unit of the image signal transmitted at one time = 1 image). The number of effective blocks (referred to as image signals) is stored in the RAM and the average is obtained. The “average” may be any of arithmetic average, weighted average, geometric average, and harmonic average.

  Then, it is determined whether both the left and right average effective blocks are equal to or greater than a predetermined number (for example, about 10 to several tens blocks, more specifically about 25 blocks) (S150). If both numbers are greater than or equal to the predetermined number, the processing of S120 and S130 is performed, and one routine of this flow is terminated.

  Here, like the predetermined number for the maximum continuous number, the predetermined number is a suitable value that is predetermined as the number of blocks that is not counted when the road lane marking is botsdots or the like. That is, since the feature points detected due to the presence of botsdots or the like are not of a property that is detected with high density in the longitudinal direction of the road, the average effective block count counted when the road lane marking is bottsdots or the like is Limits arise. Therefore, by setting a threshold value for this and making the above determination, it is appropriately determined that the road lane marking has been switched to a white line or the like. And when a road lane line is switched to a white line, etc., it will be immediately switched to an analysis process by the straight road lane line analysis processing unit 30, so before and after the road lane line is switched from a botsdot to a white line, etc. It is possible to appropriately recognize the road marking line.

  In the present embodiment, the determination regarding the maximum continuous number and the determination regarding the average effective block number are complementary to each other. That is, the inconvenience that the road demarcation line is not determined to be a white line or the like although it is actually switched to a white line or the like due to erroneous recognition is suppressed. As a cause of this misrecognition, for example, it is conceivable that a white line or the like is partially thinned on an actual road and is recognized as a botsdot or the like. Another reason is that white lines have different shapes such as solid lines and broken lines, coloring differences, etc., and it is difficult to make uniform and accurate judgments because there are suitable discrimination parameters for each. It is done. In view of these properties, for example, it is preferable that the predetermined number for the maximum continuous number is set to the number corresponding to one broken line, and the predetermined number for the average effective block number is set to the number of blocks corresponding to a distance longer than one broken line. It is. With such a design, it is possible to make a determination that is more accurate and less leaky than a determination that is made by either one. Even if the design is not based on the length of the broken line, if it cannot be determined that one of the lines has been switched to the white line due to noise on the road, the amount of light received by the camera 10, or the like, the other is the white line. By determining that the switch has been made, it is possible to make an accurate and less leaky determination. Note that the determination order of both determinations may be reversed from the flowchart of FIG.

  When the maximum continuous number is less than the predetermined number and at least one of the left and right average effective blocks is less than the predetermined number, it is determined that the road lane line imaged by the camera 10 is botsdots or the like (S160). The provision of the image signal to the straight road lane marking analysis processing unit 30 is continued (S170).

  According to the road lane marking recognition apparatus 1 of the present embodiment described above, the predetermined number for the maximum continuous number and the predetermined number for the average effective block number are not counted when the road lane line is botsdots or the like. Is determined in advance, and it is determined whether or not the maximum number of continuous blocks and the average number of effective blocks are equal to or greater than a predetermined number or a predetermined number. And when a road lane line is switched to a white line, etc., it will be immediately switched to an analysis process by the straight road lane line analysis processing unit 30, so before and after the road lane line is switched from a botsdot to a white line, etc. It is possible to appropriately recognize the road marking line.

  Further, as described above, the determination regarding the maximum continuous number and the determination regarding the average effective block number can be complemented with each other, so that the determination can be performed more accurately and with less leakage than the determination performed by either one.

  Moreover, since these are performed only by image analysis without providing an apparatus such as a laser radar, problems of equipment cost and installation space can be suppressed.

  The road lane markings recognized by the road lane marking recognition device 1 are mapped, for example, from a camera coordinate system to a real coordinate system (an actual coordinate system viewed from above) and used for driving support of the vehicle. Note that the installation parameters (height, optical axis angle, roll, focal length, etc.) of the camera 10 are taken into consideration for the mapping. The mapping may be performed prior to the processing of the road marking line recognition device 1 described above.

  FIG. 6 is a diagram illustrating an application example in which the road lane marking recognition apparatus 1 of the present embodiment is connected to a control system that performs vehicle travel control. As shown in the figure, the image analysis device 20 of the road lane marking recognition device 1 is connected to a travel support ECU 60 that supports travel of the vehicle via a communication line. The image analysis apparatus 20 transmits information on the recognized road marking line (coordinates on the actual coordinate system of each point on the road marking line as a reference) to the travel support ECU 60 via a communication line.

  The driving support ECU 60 is connected to a main switch 61 that turns on or off the main power supply for driving support. The on / off state of the main switch 61 is transmitted to the image analysis device 20 via the communication line.

  The travel support ECU 60 is connected to the multiple communication line 63. The multiplex communication line 63 is connected to an electric power steering ECU (EMPS-ECU) 65 that controls the assist force applied to the steering. Further, an engine ECU (EFI-ECU) 67 that controls the engine in an integrated manner and a meter ECU 69 that controls display on the meter panel and the like are connected to the multiplex communication line 63. The travel support ECU 60, EMPS-ECU 65, EFI-ECU 67, and meter ECU 69 perform bidirectional data communication based on the CAN protocol or the like via the multiple communication line 63.

  The EMPS-ECU 65 is connected to a steering actuator 65a such as an electric motor that applies assist force to the steering. The EMPS-ECU 65 drives and controls the steering actuator 65a based on the torque value (current value) transmitted from the travel support ECU 60.

  The EFI-ECU 67 is connected to a vehicle speed sensor 67a that detects the vehicle speed and a steering angle sensor 67b that detects the steering angle of the steering. The vehicle speed detected by the vehicle speed sensor 67a and the steering angle detected by the steering angle sensor 67b are transmitted to the travel support ECU 60 via the EFI-ECU 67 and the multiple communication line 63.

  As described above, the driving support ECU 60 maps the road lane marking data transmitted from the road lane marking detection device 1 to a real coordinate system (an actual coordinate system viewed from the sky) as described above. Based on the data after mapping, the vehicle posture of the host vehicle with respect to the road lane line (inclination of the vehicle traveling direction with respect to the road lane line; yaw angle, and displacement of the vehicle reference position with respect to the center line of the driving lane in the road width direction; Offset).

  Then, based on the estimated vehicle attitude of the vehicle with respect to the estimated road marking line and the driving situation of the driver, a subsequent vehicle attitude is predicted, and if it is determined that the vehicle may deviate from the marking line, an alarm is issued. A signal is transmitted to the meter ECU 69 (lane departure control). When the meter ECU 69 receives the warning signal from the driving support ECU 60, the meter ECU 69 generates a warning sound in the buzzer 69a and / or turns on the warning lamp 69b.

  Further, the driving assistance ECU 60 determines that the own vehicle is an abbreviation of the lane marking based on the estimated vehicle attitude of the own vehicle, the vehicle speed detected by the vehicle speed sensor 67a, and the steering angle detected by the steering angle sensor 67b. The assist steering force (current value) required for traveling in the center is calculated and transmitted to the EMPS-ECU 65 (lane keeping control). The EMPS-ECU 65 controls the steering actuator 65a based on the assist steering force transmitted from the travel support ECU 60.

  As described above, the driving assistance ECU 60 can appropriately perform lane departure control and / or lane keeping control based on the lane line detected by the road lane line detection device 1. In other words, vehicle control can be performed while suppressing vehicle wobble due to erroneous determination or recognition.

  The best mode for carrying out the present invention has been described above with reference to the embodiments. However, the present invention is not limited to these embodiments, and various modifications can be made without departing from the scope of the present invention. And substitutions can be added.

  For example, instead of the determination regarding the average number of effective blocks (S140, 150), the determination regarding the number of effective blocks may be simply performed. That is, the process of storing the number of effective blocks for several tens of frames in the past in the RAM is omitted, and the number of effective blocks (synonymous with the example) calculated most recently is a predetermined number (similar to the predetermined number in the example) If it is equal to or higher than the value (which is expected to be numerically close), it is determined as a white line or the like.

  Further, it is possible to determine whether the road marking line is a white line or the like by performing any one of the determination regarding the maximum number of continuous, the determination regarding the average effective block number, and the determination regarding the effective block number. Absent. In this case, the mutual complementary effect described in the embodiment cannot be obtained, but the control can be simplified to reduce the processing load and the processing time.

  Further, the morphological operation is not limited to the one performed on the binarized image element, and the morphological operation can be performed on the gray scale image or the progressive image. In these cases, it is preferable to use a multidimensional structural element.

  The function of the straight road lane marking analysis processing unit 30 is not limited to that of the embodiment, and for example, analysis processing using a SOBEL filter or the like may be performed.

  The present invention can be used in the automobile manufacturing industry, the automobile parts manufacturing industry, and the like.

It is a figure which shows an example of the whole structure of the road lane marking recognition apparatus 1 which concerns on one Example of this invention. It is a figure which shows typically a mode that the analysis processing part 30 for straight road division lines sets a scanning line, and extracts a feature point. It is a figure which shows a mode that it recognizes as a botsdot etc. or is recognized as a noise based on the positional relationship of a botsdot and a scanning line. It is a figure which shows a mode that the feature point extraction part 54 in a block sets a some block on an image. 6 is a flowchart showing characteristic processing of the present invention executed by a switching determination unit 56. It is a figure which shows the application example with which the road lane marking recognition apparatus 1 of a present Example was connected to the control system which performs driving | running | working control of a vehicle.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Road division line recognition apparatus 10 Camera 20 Image analysis apparatus 30 Straight road division line analysis processing part 40 Point sequence road division line analysis processing part 42 Morphology calculation part 44 Noise removal part 46 Feature point registration part 50 Analysis process switching Unit 52 feature point extraction unit on scanning line 54 feature point extraction unit in block 56 switching determination unit 60 travel support ECU

Claims (9)

Comprising imaging means for imaging the periphery of the vehicle;
The road marking line imaged by the imaging means is either a straight road marking line drawn linearly on the road or a point-string road marking line laid or drawn on the road in a dotted line shape A road lane marking type determination device for determining whether
A plurality of scanning lines extending in a direction corresponding to the road width direction on the captured image of the imaging means, and feature points extracting means on the scanning lines for extracting feature points on the plurality of scanning lines,
When there are more than a predetermined number of scanning lines from which feature points of a predetermined number or more have been extracted by the feature points extracting means on the scanning lines in the direction in which the plurality of scanning lines are arranged, the captured image of the imaging unit It is determined that the upper road lane marking is the straight road lane marking,
Road lane marking type determination device. Comprising imaging means for imaging the periphery of the vehicle;
The road marking line imaged by the imaging means is either a straight road marking line drawn linearly on the road or a point-string road marking line laid or drawn on the road in a dotted line shape A road lane marking type determination device for determining whether
In-block feature point extracting means for setting a plurality of blocks side by side in a direction corresponding to the longitudinal direction of the road on the captured image of the imaging means and extracting feature points in the plurality of blocks,
Determining that the road lane marking on the captured image of the imaging means is the straight road lane marking when there are a predetermined number or more of blocks from which feature points have been extracted by the in-block feature point extraction means; Features
Road lane marking type determination device. Comprising imaging means for imaging the periphery of the vehicle;
The road marking line imaged by the imaging means is either a straight road marking line drawn linearly on the road or a point-string road marking line laid or drawn on the road in a dotted line shape A road lane marking type determination device for determining whether
In-block feature point extraction means for setting a plurality of blocks side by side in a direction corresponding to the longitudinal direction of the road on the captured image of the imaging means and extracting feature points in the plurality of blocks;
Block number storage means for storing the number of blocks from which feature points have been extracted by the in-block feature point extraction means,
When the average of the number of blocks stored in the block number storage unit from the past to the latest is a predetermined number or more, it is determined that the road lane line on the captured image of the imaging unit is the straight road lane line It is characterized by
Road lane marking type determination device. Comprising imaging means for imaging the periphery of the vehicle;
The road marking line imaged by the imaging means is either a straight road marking line drawn linearly on the road or a point-string road marking line laid or drawn on the road in a dotted line shape A road lane marking type determination device for determining whether
A plurality of scanning lines extending in a direction corresponding to the road width direction on the captured image of the imaging means, and a feature point extracting means on the scanning line for extracting feature points on the plurality of scanning lines;
In-block feature point extracting means for setting a plurality of blocks side by side in a direction corresponding to the longitudinal direction of the road on the captured image of the imaging means and extracting feature points in the plurality of blocks, and
There are more than a predetermined number of scanning lines from which feature points of a predetermined number or more have been extracted by the feature points extracting means on the scanning lines in the direction in which the plurality of scanning lines are arranged, and extraction of feature points in the block Determining that the road lane marking on the captured image of the imaging means is the straight road lane marking when at least one of the number of blocks from which the feature points have been extracted by the means satisfies a predetermined number or more Characterized by the
Road lane marking type determination device. Comprising imaging means for imaging the periphery of the vehicle;
The road marking line imaged by the imaging means is either a straight road marking line drawn linearly on the road or a point-string road marking line laid or drawn on the road in a dotted line shape A road lane marking type determination device for determining whether
A plurality of scanning lines extending in a direction corresponding to the road width direction on the captured image of the imaging means, and a feature point extracting means on the scanning line for extracting feature points on the plurality of scanning lines;
In-block feature point extraction means for setting a plurality of blocks side by side in a direction corresponding to the longitudinal direction of the road on the captured image of the imaging means and extracting feature points in the plurality of blocks;
Block number storage means for storing the number of blocks from which feature points have been extracted by the in-block feature point extraction means,
There are a predetermined number or more of scanning lines in which feature points of a predetermined number or more are extracted by the feature points extracting means on the scanning lines in the direction in which the plurality of scanning lines are arranged, and the block number storage means The road lane marking on the captured image of the imaging means is determined to be the straight road lane marking when at least one of the average number of stored blocks satisfies a predetermined number or more. To
Road lane marking type determination device. A road lane marking type determination device according to any one of claims 1, 4, and 5,
The feature point extracting unit on the scanning line is a unit that extracts a feature point on the scanning line subjected to the processing after performing a predetermined noise removal process on the image element on the scanning line.
Road lane marking type determination device. The road lane marking type determination device according to any one of claims 2 to 6,
The in-block feature point extracting means is means for extracting a feature point in the block subjected to the processing after performing predetermined noise removal processing on the image elements in the block.
Road lane marking type determination device. The road lane marking type determination device according to claim 6 or 7,
The predetermined noise removal process is a noise removal process based on a morphological operation.
Road lane marking type determination device. A road lane marking recognition device comprising the road lane marking type determination device according to any one of claims 1 to 8, and recognizing a road lane marking in a captured image of the imaging means,
A captured image of the imaging means by switching between a first analysis mode suitable for recognizing the straight road lane marking and a second analysis mode suitable for recognizing the point-string lane marking. Image analysis means for analyzing
The image analysis means determines that the road lane marking on the captured image of the imaging means is the straight road lane marking by the road lane marking type determination device during analysis in the second analysis mode. If it is, the first analysis mode is switched,
Road marking line recognition device.

Images