JP5090313B2 - Road marking recognition device - Google Patents

Description

  The present invention relates to a road marking recognition device for recognizing a specific road marking from an image obtained by imaging a road surface of a host vehicle traveling road.

  For example, in Patent Document 1, first and second road markings are detected from an image captured by an imaging unit, the positions of the first and second road markings are calculated, and the first and second road markings are calculated based on the mutual positional relationship. A method for improving the recognition performance of the second road marking is disclosed.

JP 2007-52645 A

  However, the above-described conventional technique has a problem that the recognition performance is difficult to improve when a plurality of road markings are not present in the vicinity. In addition, when the road marking is drowned, or when there is an adverse condition due to weather, shadows of surrounding objects, etc., there is a possibility that the expected recognition performance cannot be obtained.

  The present invention has been made in view of the above points, and an object of the present invention is to reduce the influence of disturbances that change from moment to moment such as weather and shadow, and to recognize road markings with high accuracy. It is to provide a road marking recognition device that can be used.

  The road marking recognition apparatus of the present invention recognizes a specific road marking from an image obtained by imaging the road surface of the own vehicle traveling road, detects a plurality of characteristic parts of the road marking from the image, When the detection confidence level of a high feature part is higher than a preset threshold value, it is determined that the road marking has been recognized.

  According to the present invention, if a highly specific feature part is detected with high detection confidence, it is determined that the road marking to be detected is recognized. Even when the image is captured in a situation that is difficult for recognition, the road marking can be recognized with high accuracy, and erroneous recognition can be reduced.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a functional block diagram of the present embodiment. The road marking recognition device 10 includes an image processing device 11, a calculation device 12, a main storage device 13, and a memory 14, and an image imaging device 1, a vehicle information acquisition device 2, a car navigation system 20, and an alarm device 21. It is connected.

  As shown in the figure, the image processing apparatus 11 performs image processing on the image data acquired by the image capturing apparatus 1 and processes the image data so that a plurality of characteristic parts of a specific road marking can be detected. It is means of. As image processing, for example, template matching processing, noise removal processing, edge detection processing, and luminance adjustment processing are performed.

  A characteristic part is a characteristic part that constitutes a part of the shape of a specific road marking, for example, the shape, width, length, pattern, color, brightness, or the part in the entire road marking The position, size, range, etc. are characteristic and can be identified from other road markings. The characteristic portions include those that are physically separated but can be distinguished from other road markings by combining them.

  The calculation device 12 acquires vehicle information such as the vehicle speed, yaw rate, and steering angle from the vehicle information acquisition device 2. Further, the map data such as the road width and the type of road marking present in the vicinity and the position of the own vehicle are acquired from the car navigation system 20.

  When the arithmetic device 12 detects the road marking based on the information, the arithmetic device 12 outputs the type of the road marking and the distance from the own vehicle to the car navigation system 20 and the alarm device 21 according to the result, and warns the user. Various controls such as emitting

  The main storage device 13 stores information on a plurality of characteristic parts possessed by a specific road marking (feature part information storage means). For example, a template image is stored as information on the shape of each characteristic part, and information on the uniqueness of each characteristic part is stored. The uniqueness of the feature part is set to a low value as it resembles the shape of another feature part, and is set to a higher value as it is not similar to the shape of another feature part.

  The image pickup device 1 is a device that picks up an image of the road surface of the own vehicle traveling road, and is configured by, for example, a CCD camera, and is installed at the front part or the rear part of the own vehicle. The vehicle information acquisition device 2 acquires vehicle information such as the traveling speed, yaw rate, and rudder angle of the host vehicle, and is configured in a control unit mounted on the vehicle. The car navigation system 20 is a system for guiding a vehicle by displaying a route to a destination based on own vehicle position information acquired by a GPS function and map information stored in a storage means such as a hard disk or a DVD. is there.

  Next, processing for recognizing a specific road marking from an image by the road marking recognition apparatus 10 will be described with reference to the flowchart of FIG. FIG. 2 is a flowchart showing an operation example of the road marking recognition apparatus 10.

  First, in step S1, information from the car navigation system 20 is acquired. From the car navigation system 20, information on the types of road markings present in the vicinity and the rough distance from the vehicle to the road marking is acquired.

  In step S <b> 2, it is determined from the information acquired from the car navigation system 20 whether a specific road marking exists in the vicinity of the vehicle. If there is a possibility that a specific road marking exists within the imaging range of the image capturing apparatus 1, the road marking is set as a road marking to be detected, and detection processing from step S3 is performed. If there is no obvious road marking to be detected around the host vehicle, the detection process is not performed and the process returns to step S1.

  In step S3, a process for acquiring information from the vehicle information acquisition device 2 is performed. The information acquired from the vehicle information acquisition device 2 includes the moving speed and yaw rate of the host vehicle, and steering angle information of the steering. Such vehicle information is used to improve the recognition rate of the road marking to be detected.

  In step S4, a process of acquiring image data from the image capturing apparatus 1 and transferring it to the memory 14 as a digital image is performed. At the same time, the time when the image is acquired is stored in the memory 14.

  In step S5, a feature part detection process for detecting a plurality of feature parts included in the road marking to be detected is performed (first detection means). The image processing apparatus 11 performs image processing based on a preset detection condition (first detection condition). The detection condition is stored in the main storage device 13, and is set by various parameters such as an image processing filter coefficient, a template matching threshold value, an edge detection threshold value, and a dimensional error tolerance when determining a feature shape. . In this embodiment, the feature part is detected by template matching using the template image of the feature part stored in the main storage device 13 in advance.

  FIG. 3 is a schematic diagram of an image subjected to image processing by the image processing apparatus 11, and FIG. 4 is a diagram illustrating a template image used in the present embodiment. For example, template matching using the template images 41, 42, and 43 shown in FIG. 4 is performed on the road marking 30 shown in FIG. 3, and the characteristic parts 31, 32, and 33 of the road marking are detected. The method for detecting the characteristic part of the road marking may be a method other than template matching as long as the characteristic part can be acquired from the image.

  When the road marking to be detected does not fit in one frame image depending on the installation position and the installation angle of the image pickup apparatus 1, the characteristic part is detected for a plurality of frame images. When the detection process of all the characteristic parts is completed, the process proceeds to step S6.

  When detecting a feature part, the detection confidence level of each feature part is set (first detection confidence level setting means). The degree of detection confidence indicates the degree of similarity between a feature such as the shape, color, and brightness of a feature portion assumed in advance and the feature to be detected. A shape similar to the template image shows a higher value.

  In step S6, a process of assigning a detection confidence level to each feature part is performed. The degree of detection confidence is assigned to each feature part.

  In step S7, processing is performed to determine whether or not all feature parts have been detected with a high degree of detection confidence. Here, whether or not the detection confidence level of all the feature parts assigned in step S6 is higher than a preset first threshold, in other words, out of the detection confidence levels of the feature parts given in step S6. Then, it is determined whether or not there is a value equal to or lower than a preset first threshold value (first determination means). If all the feature parts are given detection confidence levels higher than the first threshold value (YES in step S7), the process proceeds to step 8 to determine that the detected object is the road marking to be detected. Transition.

  On the other hand, if there is a feature portion having a detection confidence level lower than the first threshold (No in step S7), there is a possibility that an object that is not the road marking to be detected is erroneously detected. In order to make a determination according to the uniqueness of the part, the process proceeds to step S11 and subsequent steps.

  In step S11, a process of assigning uniqueness to all feature parts is performed. The uniqueness is determined in advance for each feature part, and the feature part that is not similar to the other feature parts has a low possibility of erroneous recognition when it can be detected. A plurality of characteristic parts with high uniqueness may exist in one road marking, or may be one important characteristic part by combining a plurality of characteristic parts.

  FIG. 5 is a diagram showing a road marking 30 to be detected, and FIGS. 6 and 7 are diagrams showing other road markings. For example, the characteristic portion 31 of the road marking 30 shown in FIG. 5 is similar to the other road markings 61, 62, and 63 that are not detection targets, as shown in FIGS. 6 (a) to 6 (c). Since 61a, 62a, 63a exist and there is a possibility of erroneous recognition, the uniqueness is set to a low value.

  Further, as shown in FIG. 7, the characteristic part 33 of the road marking 30 shown in FIG. 5 has a similar characteristic part 64a to another road marking 64 that is not a detection target, and there is a possibility of erroneous recognition. Therefore, the uniqueness is set to a low value.

  On the other hand, since there is no feature portion similar to other road markings, the feature portion 32 is likely to be accurate when such a shape is detected, and the uniqueness of the feature portion 32 has a high value. Is set.

  In step S12, combination determination processing is performed to determine the uniqueness by combining a plurality of feature parts having high detection confidence levels. For example, the characteristic portions 31 and 33 of the road marking 30 shown in FIG. 5 may be misrecognized alone as other road markings 61, 62, and 63 shown in FIG. 6 and other road marking 64 shown in FIG. Yes, the uniqueness is set to a low value.

  However, the combination of the characteristic portions 31 and 33 does not have other road markings that are erroneously recognized, and is likely to be accurate when such a combined shape is detected. Is set to a high value.

  Therefore, for example, even when the characteristic part 32 having a high degree of uniqueness is included and it is difficult to recognize its shape, it is accurately determined whether or not it is a road marking to be detected by the combination of the characteristic parts 31 and 33. Can be determined. The combination pattern of feature parts and the uniqueness thereof are stored in advance in the main storage device.

  In step S13, processing is performed to determine whether or not a characteristic part with a high degree of uniqueness has been detected with a high degree of confidence. Here, it is determined whether or not the detection confidence level of a characteristic part having a high degree of uniqueness is higher than a preset second threshold value (second determination unit).

  If it is determined that the detection confidence level of the characteristic part having a high degree of specificity is higher than the second threshold value (YES in step S13), since the possibility that the road marking is a detection target is high, the degree of specificity is low and In order to perform the re-detection process of the characteristic part having a low detection confidence level, the process proceeds to step S14 and subsequent steps. On the other hand, if the detection confidence level of the characteristic part having a high degree of uniqueness is less than or equal to the second threshold value (NO in step S13), the possibility of misrecognition cannot be denied, so the redetection process is not performed and the process returns to step S1.

  In step S14, a detection condition resetting process for resetting the detection condition (second detection condition) in the redetection process in step S15 to a milder condition than the detection condition (first detection condition) in the detection process in step S5. Is done. A characteristic part with a low degree of detection confidence is expected to have an adverse condition such as a shape change due to drowning or dirt, or a shadow of a surrounding building. In order to appropriately detect such a characteristic part having a low detection confidence level, the detection condition is basically reset in a direction that allows the influence of disturbance. In template matching, the allowable threshold of similarity is lowered.

  The parameter that is the detection condition may be dynamically changed based on the uniqueness of the characteristic part and the detection confidence level, or may be a fixed value. When the parameter is dynamically changed, for example, when the degree of detection confidence of a part with a high degree of specificity is significantly higher than the first threshold value, there is a high possibility that the road marking is a detection target. The parameter is changed in the direction to judge in (direction in which the influence of disturbance is allowed). On the other hand, if the degree of detection confidence of a part with a high degree of uniqueness is substantially the same as the first threshold, there is a risk of erroneous recognition, so the parameter is changed in a direction in which a low feature part is strictly determined.

  In step S15, a feature part re-detection process is performed to re-detect the feature part from the image using the detection condition (second detection condition) reset in step S14 (second detection unit). As a result, it is determined that the feature part of which the degree of detection confidence is not high among the plurality of feature parts detected in step S5, that is, the uniqueness is low and the degree of detection confidence is not more than the first threshold value in step S7. The characteristic part is detected based on a second detection condition that is gentler than the first detection condition. The image processing and the feature part detection processing are the same as in step S5 except that the second detection condition reset in step S14 is used, and thus detailed description thereof is omitted.

  In step S16, a process of calculating and assigning the detection confidence level of the feature part re-detected in step S15 is performed (second detection confidence level setting unit). For feature parts whose detection confidence level is not high in the detection process (first detection process) in step S5 due to the influence of disturbance or the like, in many cases, a detection confidence level higher than the first detection level is given. If the features are on completely different roads, the degree of detection confidence is hardly improved. Therefore, it is possible to reduce misrecognition due to disturbances such as drowning and shadows.

  Next, in step S17, processing is performed to determine whether or not all feature parts detected by the feature part re-detection process in step S15 have been detected with a high degree of detection confidence. Here, whether or not the detection confidence level of all the feature parts assigned in step S16 is higher than a predetermined third threshold, in other words, among the detection confidence levels of the feature parts given in step S16, It is determined whether or not there is a value equal to or smaller than a preset third threshold value (third determining means).

  Then, if the detection confidence level higher than the third threshold is given to all the characteristic portions (YES in step S17), the process proceeds to step S8 to determine that the detection from the image is the road marking to be detected. To do. On the other hand, if there is still a feature portion having a detection confidence level lower than the third threshold (NO in step S17), it is determined that an object that is not the road marking to be detected is erroneously detected, and the detection process is performed. Return to step S1 without continuing.

  In step S8, recognition determination processing is performed to determine that the road marking to be detected has been recognized (recognition determination means). And the process which calculates the relative position from the own vehicle to a road marking is performed at step S9, and the process which outputs the detection result of a road marking to the car navigation system 20 or the warning device 21 is performed at step S10.

  As in the present embodiment, if “with a pedestrian crossing” is detected as the road marking to be detected, the user can be warned that there is a pedestrian crossing ahead. If a “pedestrian crossing” or “temporary stop line” is detected from the image data ahead, a warning is given if there is no deceleration. Further, when the “maximum speed” is detected, the vehicle speed acquired from the vehicle information acquisition device 2 is compared with the effect that a warning can be given as necessary.

  According to the present embodiment, when all feature parts can be detected with high detection confidence by the feature part detection process, it is determined that the road marking to be detected can be recognized. On the other hand, when all the feature parts cannot be detected with a high degree of detection confidence due to imaging conditions such as blurring and illumination fluctuations, the recognition reliability is evaluated based on the uniqueness and detection confidence of each feature part.

  For example, if a highly specific feature part is detected with a high degree of detection, there is a high possibility that it is a specific road marking, so the detection conditions are changed to the optimal one, and re-detection processing is performed. The detection result of the entire road marking is determined from the detection result of the part. Accordingly, it is possible to reduce misrecognition due to disturbances such as drowning and shadows. On the other hand, if a characteristic part with a high degree of uniqueness is not detected with a high degree of detection confidence, there is a possibility that an object that is not a road marking to be detected is erroneously detected, so the recognition process is stopped.

  In addition, since combination determination processing is performed to determine the uniqueness by combining a plurality of feature parts having high detection confidence levels, if a combination of feature parts that can identify a road marking to be detected is detected, detection is performed. It is very likely that the target road marking is present, and it can be determined that the detection target road marking has been recognized. The road marking is composed of a plurality of characteristic portions and white lines connecting them. If it is not possible to detect a feature part to be detected with a sufficient degree of detection confidence, recognition determination is not performed.

  In addition, this invention is not limited to the content of the above-mentioned Example, A various change is possible in the range which does not deviate from the meaning of this invention.

The functional block diagram of a present Example. The flowchart which shows the operation example of a present Example. The schematic diagram of an image. The figure which shows the template image used in a present Example. Diagram showing road markings to be detected The figure which shows other road markings. The figure which shows other road markings.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up device 2 Vehicle information acquisition device 10 Road surface recognition device 11 Image processing device 12 Arithmetic device 13 Main storage device 14 Memory 30 Road surface indications 31, 32, 33 to be detected 41, 42, 43 Template images 61, 62 , 63, 64 Other road markings

Claims (9)

A road marking recognition device for recognizing a specific road marking from an image obtained by imaging a road surface of a vehicle traveling road,
First detection means for detecting a plurality of characteristic parts of the road marking from the image based on a first detection condition set in advance;
First detection confidence level setting means for setting a detection confidence level of each feature part when the plurality of feature parts are detected by the first detection means;
First determination means for determining whether or not there is a pre-set first threshold value among the detection confidence degrees of the plurality of characteristic parts set by the first detection confidence level setting means;
In the case where the first determination means determines that there is one below the first threshold, the degree of detection confidence of the characteristic part having a high uniqueness among the plurality of characteristic parts is higher than a preset second threshold Second judging means for judging whether or not,
When the second determination means determines that the detection confidence level of the characteristic part having a high degree of specificity is higher than the second threshold value, a second value set according to the detection confidence level of the characteristic part having a high degree of specificity. Second detection means for detecting a plurality of characteristic parts of the road marking from the image using the detection condition of 2;
Recognition determination means for determining that the road marking has been recognized based on the degree of detection confidence of the characteristic part detected by the second detection means ;
A road marking recognition apparatus characterized by comprising: A road marking recognition device for recognizing a specific road marking from an image obtained by imaging a road surface of a vehicle traveling road,
First detection means for detecting a plurality of characteristic parts of the road marking from the image based on a first detection condition set in advance;
First detection confidence level setting means for setting a detection confidence level of each feature part when the plurality of feature parts are detected by the first detection means;
First determination means for determining whether or not there is a pre-set first threshold value among the detection confidence degrees of the plurality of characteristic parts set by the first detection confidence level setting means;
In the case where the first determination means determines that there is one below the first threshold, the degree of detection confidence of the characteristic part having a high uniqueness among the plurality of characteristic parts is higher than a preset second threshold Second judging means for judging whether or not,
A plurality of characteristic portions of the road marking from the image based on a second detection condition that is gentler than the first detection condition when the second determination means determines that the value is higher than the second threshold. Second detecting means for detecting;
Second detection confidence level setting means for setting a detection confidence level of each feature part when the plurality of feature parts are detected by the second detection means;
Third determination means for determining whether or not there is a pre-set third threshold value or less among the detection confidence degrees of the plurality of characteristic parts set by the second detection confidence level setting means;
A recognition determination means for determining that the road marking has been recognized when it is determined by the third determination means that there is no one below the third threshold and the detection confidence level of all feature parts is higher than the third threshold; ,
A road marking recognition apparatus characterized by comprising: A feature part information storage means for storing information on the shape of each feature part for a plurality of feature parts of the road marking;
The first detection unit and the second detection unit detect the plurality of feature parts from the image using information on the shape of each feature part stored in the feature part information storage unit. The road marking recognition apparatus according to claim 2 . The characteristic part information storage means stores a template image as information on the shape of the characteristic part,
The road marking recognition apparatus according to claim 2 , wherein the first detection unit and the second detection unit detect the characteristic part by template matching using the template image. The first detection confidence level setting unit sets a higher detection confidence level as the shape of the characteristic part detected by the first detection unit is similar to the template image,
It said second detection confidence level setting means, according to claim, characterized in that the shape of the characteristic part detected by the second detection means sets the detection confidence of the higher value is in a similar shape to the template image 4 The road marking recognition device described in 1. Characteristic part information storage means for storing information on the uniqueness of each characteristic part for a plurality of characteristic parts of the road marking,
The second determination unit determines whether or not the feature part information storage unit determines whether or not the detection confidence degree of each feature part set by the first detection confidence degree setting part is higher than a second threshold value set in advance. Judging by using the stored information on the uniqueness of the characteristic part,
The third determining means determines whether or not the feature part information storing means determines whether or not the detection confidence degree of each feature part set by the second detection confidence degree setting means is higher than a preset third threshold value. The road marking recognition apparatus according to claim 2 , wherein the determination is made by using the stored information on the uniqueness of the characteristic part. The uniqueness of the characteristic part is set to be a low value according to similarity to the shape of another characteristic part, and to a higher value as it is not similar to the shape of another characteristic part. The road marking recognition apparatus according to claim 6 . The road marking according to any one of claims 2 to 7 , wherein the second detection condition is set on a side that allows an influence of disturbance more than the first detection condition. Recognition device. The road marking recognition apparatus according to any one of claims 2 to 7, wherein the second detection condition is set according to a degree of detection confidence of the characteristic part having a high degree of uniqueness.

Images