JPH05149756A - Driving road recognition device for traveling vehicle - Google Patents

Abstract

PURPOSE:To obtain a driving road recognition device for a traveling vehicle which can perform recognition processing at a higher speed. CONSTITUTION:A road image indicating an environment of an outside world of a traveling vehicle is input at a road image input part 11 and a candidate point set W of a white line of the driving road is extracted from the road image at a line candidate point extraction part 12. A driving road recognition means 20 refers to a global recognition base 30, performs operation of a driving road shape X, a vehicle position Y, and a vehicle posture Z based on the candidate point set W, and then outputs them as a recognition result of the driving road. At the same time, a combination of W, X, Y, and Z is registered in a local knowledge base 40. When a candidate point set which closely resembles the given candidate point set W is registered within the local knowledge base 40, the registered X, Y, and Z are output as the recognition output as they are without performing operation referring to the global knowledge base 30.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a traveling road recognition device for a moving vehicle, and more particularly, to input the environment of the traveling car as an image and recognize the traveling road shape, the vehicle position, the vehicle attitude, etc. based on this image For a device for doing.

[0002]

2. Description of the Related Art In an autonomous vehicle or the like, it is very important to accurately recognize the shape of the road, the position of the vehicle, the posture of the vehicle, etc., on the road currently running. When recognizing such a driving route, usually, a road image is input by a camera or the like, a feature amount such as a road lane line is extracted based on the road image, and the contents of a knowledge base prepared in advance are extracted. While referring to this, the road recognition processing is performed based on this feature amount. As the knowledge base, in addition to information about the condition settings at the time of image input such as camera height, installation angle, zoom magnification, information about the road structure such as the number of lanes and lane width of the running road, and the current vehicle speed Information such as weather and weather is prepared. By performing a geometrical calculation process between these pieces of information and the extracted feature amount, it is possible to recognize the running road shape, the vehicle position, the vehicle attitude, and the like. For example, in Japanese Patent Laid-Open No. 64-26913,
A travel route recognition device for an autonomous vehicle based on such a principle is disclosed.

[0003]

As described above, the recognition processing of the road is performed by the geometric calculation processing based on the feature amount extracted from the road image and the information in the knowledge base. Such geometric arithmetic processing generally requires a considerably long arithmetic time. Therefore, the conventional roadway recognition device has a problem that the recognition time is long.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a traveling path recognition device for a moving vehicle which can perform recognition processing at a higher speed.

[0005]

According to the present invention, in a traveling road recognition apparatus for a moving vehicle, an image showing the environment of the outside environment of the moving vehicle is inputted, and a feature amount showing the characteristics of the traveling road is extracted from this image. Quantity extracting means, a traveling road recognition means for recognizing a traveling road based on the extracted characteristic quantity, a first knowledge base holding knowledge necessary for recognizing the traveling road, and a specific characteristic quantity And a second knowledge base in which the recognition result of the traveling road corresponding to is registered is provided, and the traveling road recognizing means performs the recognition work of the traveling road, first, the second knowledge base is searched and extracted. When a feature quantity similar to the feature quantity is registered, the recognition result of the traveling path corresponding to the similar feature quantity is output as it is, and when a feature quantity similar to the extracted feature quantity is not registered, First
The recognition result is obtained by referring to the knowledge held in the knowledge base, the recognition result is output, and the recognition result is newly registered in the second knowledge base.

[0006]

[Operation] On a road with a small change in road structure, such as an expressway, a similar pattern is likely to appear in the combination of the feature amount extracted from the road image and the recognition result of the road. For example, it is assumed that the recognition process B is obtained by performing the recognition process of the traveling road based on a certain feature amount A. Subsequently, if a feature amount A ′ similar to the feature amount A is obtained, a recognition result B ′ obtained by performing recognition processing based on this feature amount A ′ is often similar to the recognition result B. .. Therefore, when the recognition result B is obtained by performing the recognition process on the feature amount A using the first knowledge base, the combination of the feature amount A-recognition result B at that time is registered in the second knowledge base. To In this way, when the feature amount A ′ is obtained later, the similar feature amount A is searched from the second knowledge base, and the recognition result B registered in combination with this feature amount A is used as the feature amount A ′. There is no problem even if it is used as the recognition result for '.

The present invention pays attention to such a point, and when it can be approximated by the past recognition result registered in the second knowledge base, the road recognition processing (complex geometry) using the first knowledge base is performed. This is because the approximate recognition result in the second knowledge base is used as it is without performing the time-consuming operation (because it requires a mathematical operation).

[0008]

EXAMPLES The following examples of the present invention will be described with reference to examples illustrating the present invention. FIG. 1 is a block diagram showing the basic configuration of a traveling road recognition apparatus for a mobile vehicle according to an embodiment of the present invention. This device is composed of a feature amount extraction means 10, a traveling road recognition means 20, a global knowledge base 30, and a local knowledge base 40. The feature amount extraction means 10 has a function of inputting an image showing the external environment of the moving vehicle and extracting a feature amount showing the feature of the traveling path from this image. In this embodiment,
It is composed of a road image input unit 11 and a line candidate point extraction unit 12. The road image input unit 11 is specifically a camera attached to a moving vehicle, and a road image in front of the traveling road is captured by this camera. The line candidate point extraction unit 12 performs a process of extracting line candidate points forming a line of a road lane based on the road image. For example, if the line of the road lane is composed of white lines, pixels having a density value corresponding to the white line are extracted as line candidate points on the road image.
Theoretically, the line connecting these line candidate points becomes the road lane. In this embodiment, this set W of line candidate points is treated as a feature amount extracted from the road image.

The traveling road recognition means 20 uses this candidate point set W.
It has a function of obtaining the traveling road shape X, the vehicle position Y, and the vehicle attitude Z based on The purpose of this device, which is to recognize the traveling path, is nothing but the task of obtaining information related to the traveling path such as the traveling path shape X, the vehicle position Y, and the vehicle attitude Z. In the device according to the present invention, two methods are prepared as a method of performing such recognition by the traveling road recognition means 20. The first method is a method with reference to the global knowledge base 30, and the second method is a method with reference to the local knowledge base 40.

The global knowledge base 30 holds the knowledge necessary to recognize the road.
In this embodiment, specifically, with respect to the camera used in the road image input unit 11, information such as mounting height, installation angle, and zoom magnification, the number of lanes on the road on which the vehicle is traveling,
Information such as lane width, and information such as current vehicle speed and weather are prepared. These pieces of information are listed as an example, and in short, any information may be prepared in the global knowledge base 30 as long as it is information that can be used for recognition of the traveling path. In the first recognition method, the road is recognized based on the feature amount extracted by the feature amount extracting means 10, that is, the information of the candidate point set W and the information prepared in the global knowledge base 30. .. As described above, since such recognition processing is geometrical arithmetic processing, it takes a long calculation time. It should be noted that various specific recognition processing methods are known, and therefore description thereof is omitted here. When such recognition processing is performed, the road shape X, the vehicle position Y, and the vehicle attitude Z as the recognition result are output. The feature of the present invention is that the recognition result obtained by referring to the global knowledge base 30 in this way is converted into the local knowledge base 40.
There is a point to register in.

In the local knowledge base 40, the recognition result of the road corresponding to a specific feature amount (that is, the candidate point set W) is registered. As described above, each time the recognition process referring to the global knowledge base 30 is performed, the recognition result is registered in the local knowledge base 40.
This registration is performed by associating a specific characteristic amount with a recognition result for the specific characteristic amount. In the example shown in the local knowledge base 40 of FIG. 1, the recognition results Xi, Yi, Zi are registered for the candidate point set Wi (i =
1, 2, 3, ...). This means that when the candidate point set Wi was obtained in the past, the recognition result Xi, Yi, Zi was obtained as a result of performing geometric calculation processing with reference to the global knowledge base 30. is doing.

The second recognition method performed by the traveling road recognition means 20 is a method of searching the local knowledge base 40 for a similar feature quantity and diverting the recognition result registered for the searched feature quantity as it is. .. For example, if the candidate point set Wk is obtained from the feature amount extraction means 10, the candidate point set similar to this candidate point set Wk is searched for in the local knowledge base 40. Then, if W3 and Wk are similar, X3, Y3, Z3 which are the recognition results registered for this W3.
Is used as it is as the recognition result for the candidate point set Wk. Such a recognition method is a task of merely searching a database, and a recognition result can be obtained in a very short time.

A feature of the traveling road recognition apparatus according to the present invention is that the first recognition method using the global knowledge base 30 and the second recognition method using the local knowledge base 40 are used together. When the candidate point set Wk is obtained from the feature amount extraction means 10, the traveling road recognition means 20 first tries the second recognition method with priority. That is, the local knowledge base 40 is searched for an item similar to the candidate point set Wk.
When a similar one is found, the recognition result registered in advance is output as it is as the recognition result of the candidate point set Wk. If a similar object is not registered, the recognition result Xk, Yk, Zk is calculated by the first recognition method, and this is output and Wk → Xk, Yk,
The correspondence relation Zk is newly registered in the local knowledge base 40.

If such a method is adopted, when a candidate point set similar to the past is obtained, the second recognition method can be applied, and the time-consuming first recognition method can be suppressed to the necessary minimum. .. As for the processing of searching for a similar candidate point set from the local knowledge base 40 for the candidate point set Wk, it is preferable to perform a similarity determination that allows some ambiguity. For example, production rules, fuzzy rules, neural nets,
It is possible to make a highly flexible judgment by making a similar judgment by applying such as. In addition, information about an error between the candidate point set Wk and a candidate point set (for example, W3) determined to be similar to the candidate point set Wk is accumulated, and the local knowledge base 40 is appropriately modified based on the information about the error. It is also possible to perform various processes. Since the information registered in the local knowledge base 40 gradually increases, if the memory capacity is limited, the oldest registered content or the least frequently searched content is sequentially deleted. You just have to go.

First Embodiment Utilizing Map Database Next, an embodiment in which the road recognition can be performed using the map database will be described with reference to the block diagram of FIG. In this embodiment, the feature amount extraction means 10 and the traveling road recognition means 20 have the same configurations as those of the above-mentioned embodiments. The knowledge base 50, as in the previous embodiment,
Global Knowledge Base 30 and Local Knowledge Base 4
It may be configured by two kinds of databases of 0, or may be configured by only the global knowledge base 30 like the conventional device. The features of this device are:
Furthermore, a map database 60 is added. Further, in order to use the map database 60, a segment counting section 61 and an environment generating section 62 are provided.

In the map database 60, a traveling road is divided into a plurality of road segments, and map data is prepared for each road segment. Any method may be adopted as the method of dividing the road segment, but in this embodiment, a white line with a broken line appears as shown in FIG.
The cycle is divided as one road segment. That is, in the example shown in FIG. 3, the region from the upper end of the broken line white line as the center line to the upper end of the next broken line white line is defined as one road segment, and road segments 1, 2,
3, 4, ... are defined. In the map database 60, a segment list as shown in FIG. 4 is prepared corresponding to such road segments. This segment list is prepared by preparing map data (for example, road width W, curvature R, slope T, etc.) for each road segment.

The operation of this device is as follows. First, similarly to the above-described embodiment, the feature amount extraction unit 10 extracts the feature amount (candidate point set W) and supplies it to the traveling road recognition unit 20. On the other hand, the road image input by the road image input unit 11 is also given to the segment counting unit 61.
The segment counting unit 61 has a function of recognizing the segment position of the own vehicle by counting the defined road segments. For example, if a road segment is defined for each dashed white line cycle as in the example shown in FIG. 3,
Based on the input road image, recognize the broken white line,
It recognizes which road segment the vehicle is currently on. Specifically, by counting the number of broken white lines that have passed based on the input road image, this count value corresponds to the road segment number that is currently passing. For example, in FIG. 3, if the count value is 4 when the vehicle starts traveling from road segment 1 as the starting point, the segment position where the vehicle is present can be identified as road segment 4. actually,
It is preferable to set a point that can be easily visually recognized, such as a confluence point of an expressway, as a starting point of a road segment.

The segment counting section 61 gives information on the recognized segment position to the map database 60. The map database 60 searches for map data corresponding to the given segment position and gives it to the environment generation unit 62. For example, when the road segment 4 is given as the segment position, map data (road width W4, curvature R4, slope T4, et
c.) and, if necessary, the map data corresponding to the road segment ahead is given to the environment generation unit 62. The environment generation unit 62 generates a predicted road environment for the road segment 4 that is currently traveling and the road segment that is ahead of it based on such map data. The traveling road recognizing means 20 recognizes the traveling road based on the candidate point set W given as the feature amount. In this recognition work, the knowledge information in the knowledge base 50 is referred to, and the environment generating unit is used. The road environment generated in 62 can be used as reference data. The road environment generated by the environment generation unit 62 is information including contents such as how wide the traveling road is, how the road is curved, and what kind of slope the road environment has. As a result, the calculation load of the traveling road recognizing process in the traveling road recognizing means 20 is significantly reduced. For example, it is possible to more finely extract the feature amount in the direction in which the road is expected to be curved, or to estimate and supplement the feature amount hidden by an obstacle such as a preceding vehicle. .

A candidate point set W given as a feature quantity,
If the reference data provided from the environment generation unit 62 is significantly different from the reference data, it is considered that an error has occurred in the count value of the segment in the segment counting unit 61. In such a case, a signal requesting map data for some road segments in the vicinity is given from the road recognition means 20 to the map database 60, and reference data suitable for the candidate point set W is obtained. It is preferable that the obtained road segment is searched for and the position of the segment counter 61 is corrected.

Second Embodiment Using Map Database Next, another embodiment in which the road recognition can be performed using the map database will be described with reference to the block diagram of FIG. The feature of this embodiment is that the vehicle position Y and the vehicle attitude Z can be accurately recognized. In this embodiment, the feature amount extraction means 10, the traveling road recognition means 20, and the knowledge base 50 have the same configurations as those of the above-mentioned embodiment. Other features of this device are a map database 70, an absolute position detection unit 71, a scene generation unit 72,
Is added. The absolute position detection unit 71 has a function of detecting the current position of the own vehicle as an absolute position using a GPS using a satellite, a geomagnetic sensor, a wheel speed sensor, or the like. However, in this device, the detection accuracy of the absolute position need not be so high. In other words,
It is sufficient that the absolute position detection unit 71 can detect the rough position of the host vehicle.

The map database 70 is necessary to generate scenes (corresponding to the external environment observed at the points and the image input by the road image input unit 11) for some points scheduled to travel. Information is prepared as a database. For example, the three-dimensional shape of the white line at a certain point, the three-dimensional coordinate value of the building existing at that point, and the like are prepared as map data. The scene generation unit 72 has a function of generating a scene at that point based on this map data. Since the absolute position of the host vehicle detected by the absolute position detection unit 71 is given to the scene generation unit 72, the map database 70 is searched and the map data for the point corresponding to this detected position is read out to detect this. A scene at a position can be generated. The scene generated by the scene generation unit 72 is given to the traveling road recognition unit 20.
The traveling road recognition unit 20 performs accurate recognition of the absolute position of the host vehicle by comparing the candidate point set W given as the feature amount from the feature amount extraction unit 10 and the scene given from the scene generation unit 72. be able to.

As a simple example, it is assumed that map data is prepared in the map database 70 for five points A to E as shown in FIG. The scene generation unit 72, based on this map data,
It is possible to generate scenes a to e, respectively.
For example, the scene a corresponds to the external environment when facing a specific direction at the point A, that is, the two-dimensional image of the scenery.

As described above, the feature of this device is that the vehicle position Y and the vehicle attitude Z can be accurately recognized. Two methods can be used to perform such accurate recognition. The first method is as follows. First,
The absolute position detector 71 detects the absolute position and orientation of the host vehicle. Since this detection accuracy is not so high, it includes an error. Now, it is assumed that it is recognized from the detection result by the absolute position detection unit 71 that the vehicle is traveling toward the point A shown in FIG. In this case, the scene generation unit 72 reads the map data for the point A from the map database 70, and creates the scene a when the point A is reached in the detected vehicle attitude based on this map data. The travel route recognition means 20 includes the created scene a and the feature amount extraction means 1
An image showing a feature amount (candidate point set W) given from 0 is compared. Then, at the moment when the two match, it is recognized that the vehicle has passed the actual point A. At this time, a position correction signal is given from the traveling road recognition means 20 to the absolute position detection unit 71, and correction is performed so that the detection value of the absolute position detection unit 71 indicates the point A. By performing such a method, it is possible to perform the position correction on the absolute position detection unit 71 every time when the vehicle passes through a point prepared in the map database 70.

On the other hand, the second method is as follows. Now, it is assumed that information that the vehicle is traveling in the vicinity of points B, D, and E shown in FIG. 6 is obtained from the detection result by the absolute position detection unit 71. Since the detection accuracy of the absolute position detection unit 71 is low, it cannot be specified which of the points B, D, E is traveling. In such cases,
The scene generation unit 72 reads the map data for the points B, D, E from the map database 70, and based on these map data, the scenes b, d, e of the points B, D, E in the detected vehicle attitude. To create. The traveling road recognition means 20 selects from the created scenes b, d, and e the one that is most approximate to the image showing the characteristic quantity (candidate point set W) given from the characteristic quantity extraction means 10. select. For example, when it is determined that the vehicle is closest to the scene b, the position of the own vehicle can be specified as the point B. After that, if the first method described above is performed, accurate position recognition is possible.

[0025]

As described above, according to the traveling road recognizing device for a mobile vehicle of the present invention, the first knowledge base for recognizing the traveling road by the geometric calculation and the past recognition result are accumulated. When the second knowledge base is prepared and the latter recognition result can be approximated, the arithmetic processing using the former is not performed, so that the recognition processing can be performed at a higher speed.

[Brief description of drawings]

FIG. 1 is a block diagram showing a basic configuration of a traveling road recognition device for a mobile vehicle according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a basic configuration of a traveling road recognition device using a map database.

FIG. 3 is a diagram showing an example of a definition of a road segment in the device shown in FIG.

FIG. 4 is a map database 60 in the apparatus shown in FIG.
It is a figure which shows an example of the content of.

FIG. 5 is a block diagram showing a basic configuration of another traveling road recognition device using a map database.

FIG. 6 is a map database 70 in the apparatus shown in FIG.
It is a figure explaining the content of.

[Explanation of symbols]

 10 ... Feature amount extraction unit 11 ... Road image input unit 12 ... Line candidate point extraction unit 20 ... Travel road recognition unit 30 ... Global knowledge base 40 ... Local knowledge base 50 ... Knowledge base 60 ... Map database 61 ... Segment counting unit 62 ... Environment generation unit 70 ... Map database 71 ... Absolute position detection unit 72 ... Scene generation unit

Claims (2)

[Claims] 1. A feature quantity extraction means for inputting an image showing an external environment of a moving vehicle and extracting a feature quantity showing a feature of a traveling road from the image, and recognition of a traveling road on the basis of the feature quantity. A traveling road recognition means to perform, a first knowledge base holding knowledge necessary for recognizing the traveling road, and a second knowledge base registering a recognition result of the traveling road corresponding to a specific feature amount, When performing the recognition work of the traveling road, the traveling road recognizing unit first searches the second knowledge base, and when a characteristic amount similar to the extracted characteristic amount is registered, The recognition result of the traveling path corresponding to the similar feature amount is output as it is, and when the feature amount similar to the extracted feature amount is not registered, the knowledge held in the first knowledge base is referred to. Ask for the recognition result, Identifies and outputs the result, the result of the recognition transport vehicle of the travel path recognizing device and performs processing for newly registering the second knowledge base. 2. The roadway recognition device according to claim 1, wherein the feature amount extraction means includes a road image input unit for inputting a road image of the roadway, and a line of a road lane based on the input road image. And a line candidate point extracting unit that extracts line candidate points that configure the line candidate point, and a set of the extracted line candidate points is used as a feature amount.