JP2020153908A - Environment recognition device - Google Patents

Abstract

To provide an environment recognition device with which it is possible to utilize an environment measuring sensor and simple map information and recognize the detailed shape of a complex road, such as an intersection, before reaching there.SOLUTION: Provided is an environment recognition device for recognizing the environment of a travel path on the basis of measured data that is outputted by an environment measuring sensor, comprising: a basic structure analysis unit for analyzing the basic structure of the travel path on the basis of map information obtained from a general-purpose map or a server map; a detail structure analysis unit for restore the detail structure of the travel path on the basis of the basic structure of the travel path; a travel path correspondence control unit for changing a recognition logic for recognizing environment information from the measured data in accordance with the basic structure of the travel path; and an external apparatus control unit for controlling the environment measuring sensor or an external apparatus on the basis of the environment information recognized by the travel path correspondence control unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to an environment recognition device that recognizes the surrounding environment by using an environment measurement sensor and map information and outputs information necessary for control and alarm in advance.

In recent years, the adoption of preventive safety technology that recognizes the surrounding environment of the own vehicle is entering a period of widespread use in automobiles, and the scene of preventive safety technology is expanding. In order to realize safer and more comfortable automatic driving, more accurate environmental recognition at intersections is an important issue, but conventional environmental recognition technology that uses only environmental measurement sensors is basically simple. Since the purpose was to recognize the environment on the road, it was difficult to properly recognize complicated road shapes such as intersections.

In response to this problem, in Patent Document 1, as described in the abstract of the same document, in order to "reduce the possibility of erroneously recognizing the start position of branching or merging", "based on the imaging result by the imaging unit". The road markings that recognize the road markings that indicate the boundaries of the lanes in front of the vehicle's course and the angle-changing sections that change at a predetermined angle with respect to the direction along the lane among the recognized road markings are used for the road. The extraction unit extracted as a candidate for the shape change part indicating the shape change of the road, the information acquisition unit for acquiring the position information indicating the position where the shape change of the road occurs in front of the course, and the extracted candidate are indicated by the position information. A narrowing section that narrows down the candidates that exist within a predetermined distance in the direction along the lane from the position where the road is marked, a marking specific section that specifies the narrowed down candidates as a shape change section in front of the course, and a branch point of the specified lane. Alternatively, we are proposing an ECU (own vehicle position estimation device) that includes a position estimation unit that specifies the position of the own vehicle on the map based on the confluence.

Then, by matching the own vehicle position estimated by the ECU (own vehicle position estimation device) of Patent Document 1 with a high-precision map created for automatic driving, the complicated road shape in the traveling direction can be grasped in advance. , Used for preventive safety and automatic driving (for example, paragraphs 0024 and 0047 of the same document).

JP-A-2018-40692

As described above, Patent Document 1 requires a high-precision map for automatic driving in which a wide variety of information is registered. High-precision maps for autonomous driving are expected to continue to be developed in limited areas such as urban areas and special autonomous driving zones, but in depopulated areas with a small population, there are cost issues such as road shape surveying. Therefore, it is considered that the maintenance will be delayed, so that the area where preventive safety and automatic driving can be realized by the technology of Patent Document 1 will be limited in the future. Further, since the usage cost of the high-precision map service is considered to be relatively high, it is desirable that the same preventive safety and automatic driving as in Patent Document 1 can be realized based on the simple map that can be used at a lower cost.

In addition, the environmental measurement sensors currently on the market as products are basically intended for environmental measurement on a single road, and it is difficult to completely realize environmental recognition that can turn left or right at an intersection by automatic driving. If the road or roadside is a single road, this can be recognized on the assumption that there are two boundaries on the left and right, but at intersections and merging, the road goes straight in the direction of travel or diagonally. This is because there are extending roads and it is difficult to recognize a complicated road shape with high accuracy without erroneous detection.

Therefore, the present invention provides an environment recognition device that can recognize the detailed shape of a complicated road such as an intersection before reaching it by using an environment measurement sensor and simple map information. The purpose.

In order to solve the above problems, the environment recognition device of the present invention recognizes the environment of the traveling path based on the measurement data output by the environment measurement sensor, and uses map information obtained from a general-purpose map or a server map. Based on the basic structure analysis unit that analyzes the basic structure of the course, the detailed structure analysis unit that restores the detailed structure of the course based on the basic structure of the course, and the basic structure of the course. Correspondingly, the traveling path corresponding control unit that changes the recognition logic that recognizes the environmental information from the measurement data and the environmental measurement sensor or the external device are controlled based on the environmental information recognized by the traveling path corresponding control unit. It is assumed to be equipped with an external device control unit.

According to the environment recognition device of the present invention, the detailed shape of a complicated road such as an intersection can be recognized before reaching the detailed shape of a complicated road by using an environment measurement sensor and simple map information.

Functional block diagram of the in-vehicle environment recognition device of one embodiment Functional block diagram of environmental measurement sensor Functional block diagram of the map information section An example of an actual road network and general-purpose map Functional block diagram of the course analysis unit An example of basic structural analysis processing An example of detailed structure restoration processing Detailed functional block diagram of road shape addition part An example of additional information priority Functional block diagram of the course correspondence control unit Functional block diagram of the recognition process selection unit Functional block diagram of the recognition process setting unit An example of the set roadside processing area An example of the set lane processing area An example of the model fitting process of the road An example of labeling Processing flowchart

Hereinafter, the in-vehicle environment recognition device 100 according to an embodiment of the present invention will be described with reference to the drawings.
<In-vehicle environment recognition device 100>
The vehicle-mounted environment recognition device 100 of the present embodiment assists the manual driving by the driver and the automatic driving by the automatic driving system by providing the detailed structure of the traveling path in advance.

As shown in the functional block diagram of FIG. 1, the in-vehicle environment recognition device 100 of this embodiment is connected to an external environment measurement sensor 1 and an external device 6, and has a map information unit 2 and a traveling path analysis unit 3 inside. , The traveling path correspondence control unit 4 and the external device control unit 5 are provided. The configuration illustrated here is just an example, and the environment measurement sensor 1 and the external device 6 may be built in the vehicle-mounted environment recognition device 100, or the map information unit 2 may be separated from the vehicle-mounted environment recognition device 100. It may be configured. In the following, each configuration will be outlined and then each configuration will be described in detail.

The in-vehicle environment recognition device 100 is actually a computer (in the automobile field, an ECU (Electronic Control Unit)) equipped with hardware such as an arithmetic unit such as a CPU, a main storage device, an auxiliary storage device, and a communication device. It is realized by (called). Then, the arithmetic unit executes the program loaded in the main storage device while referring to the database recorded in the auxiliary storage device, thereby realizing each function of the map information unit 2, the course analysis unit 3, and the like. , Hereinafter, such well-known techniques in the computer field will be described while appropriately omitting them.

The environment measurement sensor 1 is a sensor that measures the external environment of the vehicle and outputs the measurement data.

The map information unit 2 mainly holds the general-purpose map Mg and the detailed map Md, which will be described later, and acquires the position of the own vehicle by using the GNSS information. Here, the general-purpose map Mg is a simple map, for example, a map acquired from a car navigation system mounted on a vehicle. On the other hand, the detailed map Md is a map including detailed environmental information that is not included in the general-purpose map Mg or the like, and is created by adding the environmental information or the like recognized based on the measurement data of the environmental measurement sensor 1 to the general-purpose map Mg or the like. It is a map. In addition, the map information (hereinafter, referred to as server map Ms) acquired from the server via the network may be held together with the general-purpose map Mg or instead of the general-purpose map Mg. This server map Ms is a map having the same amount of information as the general-purpose map Mg, and various data are updated at any time by a map vendor or the like.

The traveling path analysis unit 3 mainly analyzes the position and traveling direction of the own vehicle V on the general-purpose map Mg, and preliminarily analyzes the basic structure (shape of the intersection, etc.) of the traveling path change point that the own vehicle V encounters next. get. The course analysis unit 3 further predicts detailed structures such as the lane width, the shoulder position, and the number of lanes of the course from the acquired basic structure and the like.

The course path correspondence control unit 4 mainly recognizes environmental information such as a road shape by using the measurement data of the environment measurement sensor 1 based on the detailed structure of the course path predicted by the course path analysis unit 3. More specifically, it is more stable by dynamically changing the recognition method and processing area of the road shape such as the road and the road edge according to the road shape and the amount of error predicted by the traveling road analysis unit 3. Recognize the environment such as road shape with high accuracy.

The external device control unit 5 mainly controls the environment measurement sensor 1 and the external device 6 based on the determination of the travel path correspondence control unit 4. When the external device 6 is a display or a speaker, the in-vehicle environment recognition device 100 displays the recognition result of the surrounding environment, displays driving support information, warns for safety support, and the like to drive the driver. to support. On the other hand, when the external device 6 is an automatic driving system, the in-vehicle environment recognition device 100 supports the realization of more appropriate automatic driving by providing the environmental information necessary for vehicle control in advance.

The details of the environment measurement sensor 1, the map information unit 2, the traveling path analysis unit 3, the traveling path corresponding control unit 4, and the external device control unit 5 outlined above will be sequentially described below.
<Environmental measurement sensor 1>
As shown in FIG. 2, the environment measurement sensor 1 includes a front sensor 11, a matching unit 12, and a 3D point cloud acquisition unit 13.

The front sensor 11 is, for example, a stereo camera installed in the vehicle that captures the front. The stereo camera is a camera in which a left camera 11L and a right camera 11R are arranged side by side with a predetermined interval. In the following, an example in which a stereo camera is used for the front sensor 11 will be described, but the front sensor 11 may be a monocular camera or a lidar as long as it can measure the front of the vehicle V.

When the matching unit 12 captures the same object in the left and right images captured by the stereo camera, the matching unit 12 identifies the image position of the same object on the left and right images by a matching process and views the same object from the left and right cameras. By identifying the difference in position (parallax) projected on the image of the case, the distance to the object is measured.

The 3D point cloud acquisition unit 13 restores the three-dimensional position of the object by triangulation by specifying the triangle whose apex is the position of the same object on the image with the left and right cameras as the bases of the triangle.
<Map Information Department 2>
As shown in FIG. 3, the map information unit 2 includes a general-purpose map storage unit 21, a GNSS unit 22, a detailed map storage unit 23, and a detailed map update unit 24, and the own vehicle on the map. Identify the position of V.

The general-purpose map storage unit 21 stores the general-purpose map Mg, which is a simple map used in the car navigation system. The general-purpose map Mg expresses a road network by a node that defines a position on a map and a link that connects the nodes. For example, when the road network around the vehicle V is the environment shown in FIG. 4A, the general-purpose map Mg corresponding to the road network is linked to a plurality of nodes (white circles) as shown in FIG. 4B. It is represented by a combination of (solid lines).

The GNSS unit 22 specifies the position of the own vehicle V on the general-purpose map Mg represented by the combination of this node and the link by using GNSS information (Global Navigation Satellite System information). The position information specified by using the GNSS information may be corrected by using another sensor (for example, the behavior of the camera, radar, gyro, and own vehicle V).

The detailed map storage unit 23 converts the environmental information measured by using the environmental measurement sensor 1 (for example, lane width, road angle, number of lanes, distance from the outermost lane to the shoulder, running road shape, etc.) into the general-purpose map Mg. Add and save as detailed map Md. By using the detailed map Md saved here, when the own vehicle V travels on the same driving path next time, more accurate and stable environment recognition is possible by using the detailed information saved during the previous driving. It becomes.

The detailed map update unit 24 saves the general-purpose map Mg map data in the detailed map storage unit 23 when the map data of the general-purpose map Mg becomes old due to new construction or abandoned roads, or when the road is temporarily closed due to construction work or the like. The detailed map Md that has been created is updated. As a result, information such as roads that do not exist in the general-purpose map Mg but can be driven and roads that exist in the general-purpose map Mg but cannot be traveled is added to the detailed map Md. However, even when environmental information such as new road construction is detected, it is not necessary to immediately update the detailed map Md with one detection, and when the same environmental information is repeatedly detected, the detailed map Md is updated. It may be a thing.
<Course analysis unit 3>
As shown in FIG. 5, the traveling path analysis unit 3 includes a basic structure analysis unit 31 and a detailed structure analysis unit 32. The basic structure analysis unit 31 analyzes the basic structure of the road on the traveling path of the own vehicle V based on the general-purpose map Mg acquired from the general-purpose map storage unit 21. Further, the detailed structure analysis unit 32 strengthens (adds) information and predicts the detailed road shape based on the basic structure analyzed by the basic structure analysis unit 31. The details of each will be described below.
<Basic structure analysis unit 31>
As shown in FIG. 5, the basic structure analysis unit 31 includes a travel path data reading unit 311, a node & link analysis unit 312, a basic road shape analysis unit 313, and a vehicle position analysis unit 314.

The traveling path data reading unit 311 reads the node and link information in the direction of the own vehicle traveling path from the general-purpose map Mg. For example, if the own vehicle V is traveling on the road network illustrated in FIG. 4 (a), the traveling path data reading unit 311 obtains the node position information and the link connection information as shown in FIG. 4 (b). Read. Since the general-purpose map Mg is generally divided into, for example, a map area of 5 km square, the travel path data reading unit 311 does not need to acquire the entire general-purpose map Mg, and the own vehicle V exists. By acquiring only the information of the 5km square node and the link, the subsequent processing load is reduced.

The node & link analysis unit 312 maps the road network around the own vehicle V as shown in FIG. 4B based on the node position information and the link connection information acquired by the travel path data reading unit 311. Express.

The own vehicle position analysis unit 314 acquires the position of the own vehicle V, and has a GNSS error analysis unit 314a, a sensor error analysis unit 314b, and a time series integrated correction unit 314c. The GNSS error analysis unit 314a performs error analysis utilizing the behavior of the own vehicle and GNSS information, and analyzes the error while comparing where the own vehicle V is traveling on the map with the map information. Correct the vehicle position with high accuracy. The sensor error analysis unit 314b analyzes the vertical position and horizontal position errors on the map by using information such as an inertial sensor (gyro), a camera, millimeter waves, and LIDAR. The time-series integrated correction unit 314c performs position correction on the map by using the error information acquired by the GNSS error analysis unit 314a and the sensor error analysis unit 314b. However, if the GNSS error and the sensor error are corrected by the instantaneous determination result, the position is likely to be updated with uncertain information or the position is unstable. Therefore, stable error correction is performed by analyzing the error in time series, and error correction is performed by the time series integrated correction unit 314c.

The basic road shape analysis unit 313 is based on the general-purpose map Mg (FIG. 4 (b)) represented by the node & link analysis unit 312 and the vehicle position acquired by the vehicle position analysis unit 314. Predict the road structure of the change point (intersection, etc.) that V is expected to encounter next. For example, when the own vehicle V is traveling at the position shown in FIG. 6 (a) on the general-purpose map Mg, the basic road shape analysis unit 313 analyzes the area surrounded by the broken line in front of the own vehicle V. .. Then, the basic road shape analysis unit 313 selects a "crossroads" corresponding to the broken line region of FIG. 6A from the plurality of basic structure candidates illustrated in FIG. 6B, and the own vehicle V selects this. It predicts the basic structure of the next change point to be encountered (Fig. 6 (c)), and also predicts the distance to the crossroads.
<Detailed structure analysis unit 32>
As shown in FIG. 5, the detailed structure analysis unit 32 includes a basic road shape reading unit 321, a detailed road shape adding unit 322, and a detailed shape updating unit 323.

The basic road shape reading unit 321 acquires information on the basic structure (for example, "crossroads") predicted by the basic structure analysis unit 31.

The detailed road shape addition unit 322 restores the detailed structure by adding the detailed shape of the road and the like to the basic structure acquired by the basic road shape reading unit 321.

FIG. 7 is a diagram specifically explaining the addition of the detailed shape by the detailed road shape addition portion 322 and the restoration of the detailed structure. As illustrated in FIG. 7A, when the basic structure predicted by the basic structure analysis unit 31 is a “crossroad”, the detailed road shape addition unit 322 has a detailed shape value of the general-purpose map Mg, and a default described later. By using the value, the previous environment recognition result, etc., detailed shapes such as the number of lanes, lane width, distance from the outer lane to the shoulder, the angle between roads at intersections, and the shape change of the road such as the angle of white lines can be obtained. , Add to the basic structure and restore the detailed structure.

FIG. 7B is an example of a detailed structure restored based on the basic structure. As shown here, the detailed road shape addition portion 322 has the basic structure “crossroads” of FIG. 7A, the number of lanes of each road, the lane width of each lane, the road shoulder position, the white line intersection angle θ ₁ , and the road shoulder intersection. The detailed structure is restored by fleshing out the angle θ _{2 and the} like.

For example, if detailed shapes such as the number of lanes, the lane width, and the angle information between the running lanes and the white lines are registered in the general-purpose map Mg, the detailed structural analysis unit 32 can use them to obtain accuracy. Restore a good detailed structure of.

On the other hand, when a part or all of the above-mentioned detailed shapes are not registered in the general-purpose map Mg, the detailed structure analysis unit 32 restores the detailed structures by using the default detailed shapes and the like. The default detailed shape may be a fixed value (for example, 2 lanes, 3 m lane width, 1.5 m shoulder, etc.), but based on information such as the type and grade of the road registered in the general-purpose map Mg. A default value may be set, or an appropriate value may be set by using indirect information if the information that can be used to restore the detailed structure is not registered in the general-purpose map Mg.

In this way, since the detailed structure analysis unit 32 can restore the detailed structure with higher accuracy, the recognition process by the traveling path correspondence control unit 4 described later, the driver or the automatic driving system via the external device 6 can be performed. Feedback information can be made more appropriate.

Here, a method of setting the detailed shape will be described with reference to Tables 1 to 8.

Table 1 shows the types of roads specified in Article 3, Paragraph 1 of the Road Structure Ordinance, and Table 2 shows the grades of roads specified in Article 3, Paragraph 2 of the Road Structure Ordinance.

From these tables, for example, national expressways in urban areas are classified as "Type 2" roads, and for example, general roads in flat areas with a planned traffic volume of 20,000 vehicles / day or more. It can be seen that it is classified as a "first-class" road.

When such type / grade information is registered in the general-purpose map Mg, the detailed road shape addition unit 322 uses the type information and the default value corresponding to the grade information to make the detailed structure highly accurate. Can be restored to.

For example, for a lane width or the like in which the Road Structure Ordinance sets a legal value according to a combination of road type and grade, the legal value is used as it is as a default value. Specifically, the statutory value of the lane width specified in Article 5, Paragraph 4 of the Road Structure Ordinance (Table 3), or the road specified in Article 8, Paragraphs 2, 4, and 7 of the Road Structure Ordinance. By using the legal value of the shoulder width (Table 4), the detailed road shape addition portion 322 can accurately restore the detailed structure such as an intersection that complies with the legal value.

However, there are cases where the combination of road type and grade is not registered in the general-purpose map Mg, and the legal value cannot be obtained from Tables 3 and 4. In such a case, the detailed road shape addition unit 322 aims to improve the restoration accuracy of the detailed structure by switching the default value based on other information registered in the general-purpose map Mg.

For example, when the road type information is registered in the general-purpose map Mg but the road grade information is not registered, a default value such as the number of lanes on one side is set using Table 5. On the contrary, when the road grade information is registered in the general-purpose map Mg but the road type information is not registered, the default value such as the number of lanes on one side is set by using Table 6. In this way, even when Table 5 or Table 6 is used, it is useful information when roughly predicting the width of the road and the number of lanes, and the detailed structure is restored as compared with the case where there is no information at all. The accuracy can be improved.

Next, when the information such as the direct lane width, the number of lanes, and the road shoulder for restoring the detailed structure is not registered in the general-purpose map Mg, and neither the road type information nor the grade information is registered (that is,). Two methods of setting the default value in (when none of Tables 3 to 6 can be used) are illustrated.

For example, if road type information and grade information are not registered in the general-purpose map Mg, but rough information on road types is registered, use Table 7 to use one-sided lane according to the road type. Set default values such as numbers. If the default values in Table 7 are partially registered in the general-purpose map Mg, the default values in the general-purpose map Mg should be prioritized and only the missing information should be quoted from Table 7. .. In addition, although rough information on the type of road is not registered in the general-purpose map Mg, if the speed limit information can be obtained from the general-purpose map Mg or the like, use Table 8 and refer to the number of lanes on each side according to the speed limit. Dynamically switch the default value of. As described above, even when the default value is dynamically set using Table 7 and Table 8, the restoration accuracy of the detailed structure can be improved as compared with the case where there is no information at all.

<Detailed shape update unit 323>
As shown in FIG. 5, the detailed shape update unit 323 includes a roadside update unit 323a, a lane update unit 323b, and an undulation update unit 323c.

The roadside updating unit 323a updates the roadside information regarding the positional relationship between the outermost lane and the roadside, and the roadside information regarding the deformation of the shape of the lane or the roadside. The lane update unit 323b updates lane information that is not described in the general-purpose map information Mg, such as the number of lanes in the lane, complex changes in lane shape at intersections and merging, lane width information, and angles between lanes. To do. The undulation update unit 323c updates the undulation information regarding the unevenness and inclination of the traveling path.

In addition to the above-mentioned update process, each of these will reuse the environmental information the next time the vehicle travels in the same location, based on the reliability of the environmental information such as roadsides, lanes, and undulations of the road. Judge whether it is good. For example, the unreliable environmental information is determined to be provisional information used only for the current road shape recognition by the travel path correspondence control unit 4, and is not transmitted to the map information unit 2. On the other hand, the highly reliable environmental information is determined to be environmental information that is not only used for the road shape recognition of this time by the traveling path correspondence control unit 4 but also reused thereafter, and is sent to the map information unit 2 together with the reliability. Send and save in detailed map Md.

To give a specific example of determining whether or not environmental information can be reused, when a convex shape orthogonal to the traveling direction is measured on a road leading to a residential area, the undulation update unit 323c has a BUMP or BUMP for speed suppression. It is determined that there is a high possibility of HUMP, and the undulation information regarding the convex shape is transmitted to the map information unit 2 as environmental information to be registered in the detailed map Md with high reliability.

It should be noted that even highly reliable environmental information does not need to be immediately transmitted to the map information unit 2 after being measured once, and may be transmitted to the map information unit 2 when the same information is repeatedly measured. This is because the environmental information measured only once may be the one that detected the provisional lane width during construction, or the one that could not detect the white line hidden by the parked vehicle, so repeat it at the same place. This is to increase the reliability of the detailed map Md by registering only the measured and reliable environmental information.

In addition, if the reliability of the environmental information obtained by this environmental recognition is extremely high and the degree of agreement with the environmental information already stored in the map information unit 2 is also high, the existing environmental information and the new environment The detailed shape on the detailed map Md may be updated by fusing the information.
<Detailed road shape addition part 322>
Next, further details of the detailed road shape addition portion 322 for which the specific processing has been described with reference to FIG. 7 will be described with reference to FIG. As shown here, the detailed road shape addition unit 322 includes a priority adjustment unit 322a, an additional information selection unit 322b, and a shape restoration unit 322c.

As described with reference to FIG. 7, the detailed road shape addition unit 322 adds detailed shapes such as lane width, number of lanes, and road shoulder information to the basic structure acquired by the basic road shape reading unit 321 to provide a detailed structure. Although it is to be restored, when there are multiple information sources with detailed shapes, it is necessary to determine which information source should be used preferentially.

Therefore, the priority adjusting unit 322a adjusts the priority of the information source used when restoring the detailed structure of the road.

As shown in the “Fixed environment information” column of FIG. 9, for the environment information with a fixed value, the information source having the highest priority (the numerical value of the priority is small) among the available information sources is used. .. For example, taking "lane width" as an example, if "lane width" for each road is set in the server map Ms acquired from the server, the "lane width" information of the server map Ms is used (priority). 1). If the "lane width" information cannot be obtained from the server map Ms and the "lane width" information can be obtained from the general-purpose map Mg, the "lane width" information of this general-purpose map Mg is used (priority 2). If "lane width" information cannot be obtained from either the server map Ms or the general-purpose map Mg, use the one with the highest priority among the legal values and default values specified in Tables 3 to 8 described above. (Priority 3 to Priority 5). Furthermore, if Tables 3 to 8 cannot be used, all fixed default values (environmental information such as the number of lanes 2, lane width 3 m, and shoulder 1.5 m in the above example) may be used. ..

Further, as described above, the recognized environmental information that exceeds a certain degree of reliability is registered in the detailed map Md of the map information unit 2 by the detailed shape updating unit 323, and is therefore registered in the detailed map Md. By reusing the provided environmental information, it is possible to grasp the "lane width" of the next change point in advance.

In FIG. 9, the superiority or inferiority of the priority is expressed by an integer, but a priority that is not an integer such as priority 1.4 may be set. For example, while setting an integer priority for fixed environment information registered in the server map Ms, general-purpose map Mg, etc., the dynamic environment information whose reliability fluctuates recognized by the measurement data of the environment measurement sensor 1 May set a priority within the range of priority 1 to priority 4, which is not limited to an integer according to the reliability, so that environment information having a small priority can be used.

Further, in the above description, the case where the detailed shape to be added to the basic structure is "lane width" has been described as an example, but even when "the number of lanes" and "road shoulder" are added, the priority is used. The detailed structure can be set. Further, since some information is missing depending on the information source, in that case, the detailed structure including the desired information may be restored by integrating the information acquired from a plurality of information sources.

In this way, the detailed road shape addition unit 322 updates the priority of the information stored in the detailed map Md according to the reliability of each environmental information. If there is no environmental information stored in the past on the detailed map Md, the environmental information with a certain degree of reliability recognized by the measurement data of the environmental measurement sensor 1 is set to priority 4 and registered. On the other hand, when the environmental information within the error range is updated, the values of the environmental information are averaged and updated, and the priority is gradually increased. The data saves the value of the environmental information each time, deletes the outliers, and lowers the priority. Raise the priority except for outliers and save the average value of the environment recognition results. Alternatively, the filter design may be such that the environment recognition result is filtered, the weight is increased to the current value, and the past value is decreased. In addition, if outliers come in continuously, the conventional value may have been wrong, or the road data may have been actually updated due to road construction, so it will take a certain amount of time or longer. Is planned, and if significantly different data comes in 5 times or more in a row, the priority is initially set to priority 4 and the value is rewritten to new data at the same time. By doing so, it is possible to update data due to actual road construction.
<Traveling path correspondence control unit 4>
As shown in FIG. 10, the traveling path correspondence control unit 4 includes a recognition process selection unit 41, a recognition process setting unit 42, a roadside recognition unit 43, a lane recognition unit 44, an exposure adjustment unit 45, a camera posture correction unit 46, and a sensor. It has an abnormality detection unit 47.

The course path correspondence control unit 4 recognizes the road shape based on the measurement data of the environment measurement sensor 1 based on the detailed structure of the change point (intersection, etc.) acquired in advance from the course path analysis unit 3, and is a single road. The recognition logic of the road shape recognition process, which is initially set on the assumption that the above is assumed, is dynamically changed according to the road shape of the traveling path, and the environmental information is recognized by a more appropriate recognition logic. Similarly, for exposure adjustment, camera attitude correction, and sensor abnormality detection, by performing processing based on the detailed structure of the traveling path, it is possible to perform processing more appropriate than the original processing. The details of each will be described below.
<Recognition processing selection unit 41>
The recognition processing selection unit 41 selects the recognition logic (number of processing areas and feature amount extraction method) according to the basic structure of the traveling path, and as shown in FIG. 11, the roadside processing area number selection unit 411, It has a roadside extraction method selection unit 412, a lane processing area number selection unit 413, and a lane extraction method selection unit 414. In the following, the description will proceed assuming that the processing data of the traveling path correspondence control unit 4 is image data captured by a stereo camera, but the processing data is 3D point cloud data or the like acquired from the environment measurement sensor 1. It may be other kinds of data.

The roadside processing area number selection unit 411 selects the number of areas for executing the roadside extraction processing according to the basic structure of the traveling path. When the front sensor 11 is a stereo camera, it measures about 100 m ahead of the vehicle V. Therefore, for example, if the traveling path within 100 m ahead is a single road, two parallel roads are parallel to the left and right of the vehicle V. Assuming that there are various roadsides, the number of roadside processing areas "2" is selected so that they can be extracted. Also, if there is a crossroads on the road within 100 m ahead, it is assumed that there are two parallel roadsides on the left and right of the vehicle V, and there are two parallel roadsides in front of the vehicle V. The number of roadside processing areas "4" is selected so that

Similarly, the lane processing area number selection unit 413 selects the number of areas for executing the lane extraction process according to the basic structure of the traveling path. For example, if the traveling road within 100 m ahead is a single road, it is assumed that there are lanes along the traveling direction of the own vehicle V, and the number of lane processing areas "1" is selected so that they can be extracted. Also, if there is a crossroads within 100 m ahead, it is assumed that there are lanes along the direction of travel of the vehicle V and lanes perpendicular to the direction of travel of the vehicle V on the front, back, left, and right of the intersection. The number of lane processing areas "4" is selected so that they can be extracted.

Further, the roadside extraction method selection unit 412 selects a roadside feature amount extraction method according to the basic structure of the traveling path. For example, if the traveling path within 100 m ahead is a single road, a roadside extraction method for laterally searching the set roadside processing area is set. If there is a crossroads on the road within 100 m ahead, a roadside extraction method that searches laterally for the roadside processing area along the traveling direction of the vehicle V is selected, and is perpendicular to the traveling direction of the vehicle V. For the roadside processing area, select the roadside extraction method for searching in the vertical direction.

Similarly, the lane extraction method selection unit 414 selects a lane feature extraction method according to the basic structure of the traveling path. For example, if the traveling road within 100 m ahead is a single road, a lane extraction method for laterally searching the set lane processing area is selected. If there is a crossroads within 100 m ahead, select a lane extraction method that searches laterally for the lane processing area along the direction of travel of the vehicle V, and lanes that are perpendicular to the direction of travel of the vehicle V. For the processing area, select the lane feature extraction method for searching in the vertical direction.
<Recognition processing setting unit 42>
As shown in FIG. 12, the recognition processing setting unit 42 includes a roadside processing area setting unit 421, a roadside area-specific extraction method setting unit 422, a lane processing area setting unit 423, and a lane area-specific extraction method setting unit 424. Each setting unit enables the measurement data of the environment measurement sensor 1 to be processed by using the number of processing areas selected by the recognition processing selection unit 41 and the feature amount extraction method.

For example, when there is a crossroad on a traveling road within 100 m ahead, the roadside processing area setting unit 421 sets the roadside processing area at the boundary between the traveling road and the sidewalk as the roadside processing area based on the selection of the roadside processing area number selection unit 411. Four regions (see FIG. 13A) having a size so as to include a road shoulder of a certain step are set in the image data. Further, under the same conditions, the lane processing area setting unit 423 is large enough to include lanes on the front, rear, left, and right roads of the intersection as the lane processing area based on the selection of the lane processing area number selection unit 413. Four areas (see FIG. 13B) are set as image data. Further, the roadside region-specific extraction method setting unit 422 and the lane-specific extraction method setting unit 424 have feature quantities selected by the roadside extraction method selection unit 412 and the lane extraction method selection unit 414 as the feature quantity extraction method for each region. Set the extraction method.

In consideration of the effects of GNSS error, self-position correction by the environment measurement sensor 1, vehicle body pitching, etc., the size of the processing area set by the roadside processing area setting unit 421 and the lane processing area setting unit 423 is shown on the image. It is desirable to have a margin in consideration of the misalignment of.
<Roadside recognition unit 43>
The roadside recognition unit 43 recognizes the roadside by using the processing area and the feature amount extraction method set by the recognition processing setting unit 42. For the roadside feature amount, a step portion that is three-dimensionally high or low with respect to the traveling road is extracted as a feature amount. Therefore, when there is a road shoulder and a wall on the side of the road, both features are extracted. In addition, various three-dimensional objects such as walls, buildings, trees, utility poles, and gutters are extracted as features, but the important step for the own vehicle V traveling on the track is first centered on the own vehicle V. The step that comes into contact is important, and it is important to drive without touching this step. For this reason, considering the vehicle V as the center, the step that may come into contact first is emphasized, and noise removal processing is performed to delete the three-dimensional object that exists at a position farther from the vehicle V. To do. After the implementation, the step where the own vehicle V may first come into contact with the own vehicle V is used as a point cloud on the roadside. Further, the roadside point cloud is constructed in chronological order by using the behavior of the own vehicle or the corresponding points in the image. The road end point group constructed in this time series and the detailed road structure are model-fitted.

FIG. 14 is an example of the model fitting process performed by the roadside recognition unit 43. Even if the detailed structure of the traveling path acquired from the traveling path analysis unit 3 by the traveling path correspondence control unit 4 is an orthogonal crossroad as illustrated in FIG. 14A, the actual shape of the crossroads is often different. .. Therefore, the roadside recognition unit 43 considers the position of the roadside point group (black circle in FIG. 14B) recognized from the measurement data of the environment measurement sensor 1, and defines the road shape as a road model control point (FIG. 14). Adjust the position of (white circle) in (b). In the example of FIG. 14B, since the road end point group of the road in the lateral direction of the paper is observed on a straight line slightly upward to the right, the road end recognition unit 43 slightly positions the position of the road model control point of the crossroads. A model fitting process is performed to move it on the upward-sloping dotted line, and this is used as a new detailed structure.

As described above, the roadside recognition unit 43 of the present embodiment acquires the detailed structure of the road in the traveling direction in advance from the traveling path analysis unit 3, and extracts the roadside processing area and the roadside feature amount according to the detailed structure. By not only using the method but also performing model fitting processing on the acquired detailed structure, accurate recognition was difficult with the conventional roadside recognition technology that basically targeted single roads, which is complicated. It becomes possible to accurately recognize the roadside of the traveling path.
<Lane recognition unit 44>
The lane recognition unit 44 recognizes a lane by using the processing area set by the recognition processing setting unit 42 and the feature amount extraction method. The same points as the roadside recognition unit 43 will not be duplicated, but if the differences between the two recognition units are listed, the lane recognition unit 44 can use the processing area and feature extraction method set by the recognition processing setting unit 42. The lane is recognized by removing noise from the feature amount extracted by using. The lane feature amount detects a feature point of a white line which is a boundary line along the traveling direction. Since the driving path is generally straight and there are few extreme curves, it is possible to roughly search for the position of the lane and the number of candidates by performing a straight line search. By performing such processing, noise removal is performed. Furthermore, by using our own behavior, we will restore lanes that are continuously connected in chronological order.
<Exposure adjustment unit 45>
The exposure adjustment unit 45 corrects the exposure of a stereo camera or the like by using information useful for determining exposure conditions such as an approximate value of the width to the shoulder of the traveling path.

In the conventional exposure adjustment, the brightness of the current image is not measured according to the driving road, but basically the brightness of the road surface is measured with the road surface where the lane does not enter as the processing area. By adjusting the exposure amount of the next frame according to the brightness of the road surface, the adjustment was made so that the parallax of the road surface would not be reduced as much as possible. Alternatively, even if there are white lines or road surface paint, the exposure amount is adjusted according to the brightness of the road surface by using the mode value. Basically, it is possible to adjust the exposure more stably by analyzing the brightness of the road surface in a wider range, but since the shape of the driving road is unknown, it is basically in the own lane or in the own lane. The exposure analysis was performed in the processing area that slightly protruded from the lane.

However, if the approximate value of the width to the shoulder can be obtained by the above-mentioned detailed shape analysis, the brightness of the traveling road for adjusting the exposure by using the result of this detailed shape analysis or the previous environment recognition result. Determine the processing area to analyze. This makes it possible to analyze the road surface brightness in the widest possible range according to the traveling road, and it is possible to realize more stable exposure adjustment.
<Camera posture correction unit 46>
If the inclination information of the traveling path is registered in the general-purpose map Mg, the camera posture correction unit 46 corrects the posture of the environment measurement sensor 1 by using the information. In addition, if there is information such as lane width and road shoulder width, it is possible to correct camera height information using this information.

Further, by finding the correct answer between the camera and the road surface, it is possible to set the processing area more appropriately.
<Sensor abnormality detection unit 47>
The sensor abnormality detection unit 47 detects an abnormality in the environment measurement sensor 1 by using map information such as a detailed map Md acquired from the map information unit 2. Since the sensor abnormality detection unit 47 can acquire detailed structures such as white lines and road shoulders at intersections in the traveling direction in advance from map information, it can detect features such as white lines and road shoulders that should be detected when passing through the intersections and the like. If not, it is determined that the environment measurement sensor 1 may have an abnormality. Then, when the abnormality is repeatedly determined within a short period of time, the abnormality of the environment measurement sensor 1 is detected, and the driver or the automatic driving system is notified of the abnormality of the environment measurement sensor 1.
<Application example of the traveling path correspondence control unit 4>
The basic control by the traveling path correspondence control unit 4 has been described above. However, when a plurality of lanes are recognized on the traveling path using the above recognition logic, or landmarks that need to be considered when performing automatic driving are recognized. In this case, the detailed structure analysis unit 32 assigns a label L indicating the type to each of the recognized lanes and landmarks so that the meaning of what kind of environment the own vehicle V is traveling in can be recognized. You can do it. For example, as shown in FIG. 15, for the lanes and landmarks recognized when heading to the gate of the expressway, the labels L1 and the ETC device indicating that the lanes and landmarks are general lanes for vehicles not equipped with the ETC device. Label L2 indicating that the vehicle is equipped with the vehicle, label L3 indicating the position of the gate, label L4 indicating that the road is on the opposite road, and the like are attached.

By attaching these labels, the own vehicle V during automatic driving can not only prevent reverse driving on the oncoming driving road, but also select an appropriate driving road according to the mounting status of the ETC device and pass through the gate. It is possible to realize automatic operation that sometimes stops or decelerates.
<Processing flowchart>
Next, an example of the processing flow executed by the vehicle-mounted environment recognition device 100 of this embodiment will be described with reference to FIG.

First, in S1, the front sensor 11 of the environment measurement sensor 1 captures left and right images in front of the vehicle V with a stereo camera.

In S2, the matching unit 12 of the environment measurement sensor 1 performs stereo matching on the left and right images captured by the stereo camera.

In S3, the 3D point cloud acquisition unit 13 of the environment measurement sensor 1 acquires the 3D point cloud based on the stereo matching result by the matching unit 12.

In S4, the basic structure analysis unit 31 of the course analysis unit 3 acquires the basic road shape consisting of the node and the link in the vicinity of the own vehicle V from the general-purpose map information Mg held by the map information unit 2.

In S5, the detailed structure analysis unit 32 of the course analysis unit 3 restores the detailed road shape based on the basic road shape acquired by the basic structure analysis unit 31. Alternatively, when the information on the result of environmental recognition is stored, it is decided based on which one to carry out the restoration of the detailed road shape according to the priority.

In S6, the recognition processing selection unit 41 of the traveling road correspondence control unit 4 selects the recognition logic for recognizing the roadside and the lane according to the road shape.

In S7, the recognition processing setting unit 42 of the traveling road correspondence control unit 4 sets the recognition logic for recognizing the roadside and the lane according to the road shape.

In S8, the roadside recognition unit 43 of the traveling path correspondence control unit 4 receives a partial change in the recognition process by the recognition process setting unit 42, a change result in the processing area or position due to parameter adjustment, and the like, and performs the roadside recognition process. To execute.

In S9, the lane recognition unit 44 of the travel path correspondence control unit 4 executes the lane recognition process in response to a partial change in the recognition process by the recognition process setting unit 42 or a change result in the processing area or position due to parameter adjustment. To do. The order of S8 and S9 may be exchanged.

In S10, the detailed structural analysis unit 32 of the traveling path analysis unit 3 first determines whether or not it is possible to reflect the environmental information related to the lane and roadside recognition results on the detailed map Md according to the reliability of the recognition results.

When it is determined that the detailed map Md can be updated, in S11, the detailed map updating unit 24, if there is information stored based on the result recognized so far, compares it with the result. , Determine whether to update the general-purpose map Mg. For example, if the result is significantly different from the conventional storage result, the final map is reflected in the general-purpose map Mg after confirming that such a result continues many times.

In S12, the external device control unit 5 uses the recognized environmental information this time to provide the driver with information on the course before arriving at an intersection or the like, or to realize automatic driving in the automatic driving system. Provide necessary information in advance.

As described above, according to the in-vehicle environment recognition device of this embodiment, the detailed shape of a complicated road such as an intersection can be obtained by using the environment measurement sensor mounted on the vehicle and simple map information. Can be recognized before reaching.

1 Environmental measurement sensor 11 Front sensor 11L Left camera 11R Right camera 12 Matching unit 13 3D point group acquisition unit 2 Map information unit 21 General-purpose map storage unit 22 GNSS unit 23 Detailed map storage unit 24 Detailed map update unit 3 Path analysis unit 31 Basic structure analysis unit 311 Path data reading unit 312 Node & link analysis unit 313 Basic road shape analysis unit 314 Vehicle position analysis unit 314a GNSS error analysis unit 314b Sensor error analysis unit 314c Time series integrated correction unit 32 Detailed structure analysis unit 321 Basic road shape reading part 322 Detailed road shape addition part 322a Additional information priority part 322b Additional information selection part 322c Shape restoration part 323 Detailed shape update part 323a Roadside update part 323b Lane update part 323c Undulation update part 4 Road shape recognition part 41 Recognition processing selection unit 411 Roadside processing area number selection unit 412 Roadside extraction method selection unit 413 Lane processing area number selection unit 414 Lane extraction method selection unit 42 Recognition processing setting unit 421 Roadside processing area setting unit 422 Roadside area extraction Law setting unit 423 Lane processing area setting unit 424 Lane-specific extraction method setting unit 43 Roadside recognition unit 44 Lane recognition unit 45 Exposure adjustment unit 46 Camera posture correction unit 47 Sensor abnormality detection unit 5 External device control unit 6 External device

Claims (10)

An environment recognition device that recognizes the environment of the course based on the measurement data output by the environment measurement sensor.
A basic structure analysis unit that analyzes the basic structure of the course based on map information obtained from a general-purpose map or a server map.
A detailed structure analysis unit that restores the detailed structure of the course based on the basic structure of the course,
A travel path correspondence control unit that changes the recognition logic that recognizes environmental information from the measurement data according to the basic structure of the travel path.
An external device control unit that controls the environment measurement sensor or the external device based on the environmental information recognized by the path corresponding control unit.
An environment recognition device characterized by being equipped with. In the environment recognition device according to claim 1,
The map information is node information and link information, and is
The detailed structure is restored by adding environmental information of any one of the number of lanes, the width of the lane, the position of the road shoulder, the white line intersection angle, and the road shoulder intersection angle to the basic structure consisting of the node information and the link information. An environment recognition device characterized by being present. In the environment recognition device according to claim 1,
The recognition logic that is changed according to the basic structure of the traveling path is an environment characterized by the number of processing areas, the location of the processing area, or the feature amount extraction method in the roadside or lane recognition process. Recognition device. In the environment recognition device according to claim 3,
The recognition logic is an environment recognition device including at least one of a feature amount extraction process, a noise removal process, and a model fitting process according to the location of the processing area. In the environment recognition device according to claim 4,
The model fitting process is a process of setting the white line intersection angle or the road shoulder intersection angle included in the detailed structure of the traveling path based on the three-dimensional arrangement of the feature amount extracted by the feature amount extraction process. An environment recognition device characterized by. In the environment recognition device according to claim 1,
The detailed structure analysis unit is an environment recognition device characterized in that the lanes or landmarks recognized by the travel path correspondence control unit are given labels indicating their types to restore the detailed structure of the travel path. .. In the environment recognition device according to claim 2.
The detailed structure analysis unit is an environment recognition device characterized in that the value of the environmental information is restored based on the type information or the grade information specified by the Road Structure Ordinance obtained from the map information. In the environment recognition device according to claim 2.
The detailed structural analysis unit is an environment recognition device characterized in that the value of the environmental information is restored based on the road type information or the speed limit information acquired from the map information. In the environment recognition device according to claim 2.
The detailed structure analysis unit is an environment recognition device characterized in that the value of the environment information is restored by using the value obtained from the map information in the past recognition process at the place. In the environment recognition device according to claim 2.
The detailed structural analysis unit
Value based on type information or grade information specified by the Road Structure Ordinance,
Value based on road type information or speed limit information, or
If more than one of the values obtained in the past recognition process at the location can be used,
An environment recognition device characterized in that the detailed structure is restored by selecting the highest priority value set for each value.

Images