JP6984379B2 - Road structure data generator - Google Patents

Description

The present invention relates to a road structure data generating equipment for generating information of a roadside area that may vehicle enters and exits.

In recent years, as a driving support device, a system for creating road information using probe data has been proposed.

Patent Document 1 describes that information from an in-vehicle sensor is collected to automatically generate a lane-level road network. In Patent Document 1, unlike the conventional navigation map composed of nodes and links, the center line is detected for each lane, and the connection relationship for each lane is automatically generated at the intersection. The autonomous driving system uses this information as a clue to design the vehicle route to the destination.

Patent Document 2 describes that a new road is detected by extracting a vehicle locus that does not exist in the existing map information for the purpose of determining a new road.

Non-Patent Document 1 describes that the concept of a potential map is used in order to make a human-like route plan. That is, Non-Patent Document 1 integrates the actual risk potential generated based on the map information and the trajectory of the preceding vehicle and the potential risk potential for the traffic participants empirically acquired, and the potential map around the own vehicle. It is described that the optimum route is generated based on the generation.

Japanese Unexamined Patent Publication No. 2015-4814 Japanese Unexamined Patent Publication No. 2014-130529

"Chunzhao Guo, et al.," Human-like Behavior Generation for Intelligent Vehicles in Urban Environment based on a Hybrid Potential Map, "Proc.of 2017 IEEE Intelligent Vehicles Symp. (IV), pp.197-203, 2017."

However, mainly on general roads, there are many stores, parking lots, etc. (hereinafter referred to as facilities) on the side of the road, and vehicles often use the facilities as destinations.

Roadside facilities are equipped with entrances and exits (including taxiways) that connect to surrounding roads. In some cases, there are multiple doorways. Vehicles slow down, stop, or turn left or right near the doorway.

For example, in Patent Document 2, since the positioning result of GPS (Global Positioning System) is used as a clue, the amount of information and control are required to accurately extract the change in the vehicle position near the entrance / exit on the roadside and create a database of the position. It lacks functionality.

INDUSTRIAL APPLICABILITY The present invention obtains a road structure data generator capable of generating a road network including facilities on the side of a road and realizing driving support including navigation and automatic driving with high accuracy and efficiency. Is the purpose.

Further, in addition to the above purpose, the purpose is to obtain a road structure database that can accurately predict how the preceding vehicle or the oncoming vehicle will move at the next moment by being used as a driving support device.

The road structure data generation device of the present invention has an acquisition means for acquiring driving information related to the driving of the vehicle from an in-vehicle sensor mounted on the vehicle, driving information acquired by the acquisition means, and road structure information known in advance. Based on the basic information generation means that generates the basic information that is the basis for recognizing the road structure, and the analysis means that analyzes the trajectory information when the vehicle is traveling based on the basic information generated by the basic information generation means. And, by the classification means that classifies the trajectory information analyzed by the analysis means for each lane of the travel path and the statistical processing of the travel trajectory of the vehicle, the end point of the road boundary, the road branch point, and the base point of the entrance / exit to the facility are determined. A lane that can be connected to the attention point is selected based on the extraction means for extracting the attention point including the attention point and the locus information classified by the classification means, and a traveling route connecting the selected lane and the attention point is generated. It has a road connection information generation means.

According to the present invention, the acquisition means acquires driving information related to the driving of the vehicle from an in-vehicle sensor mounted on the vehicle. The basic information generation means generates basic information that is the basis for recognizing the road structure based on the driving information acquired by the acquisition means and the road structure information known in advance.

Based on this basic information, the locus information when the vehicle is traveling is analyzed, and the analyzed locus information is classified for each lane of the traveling route (analysis means, classification means).

On the other hand, in the extraction means, attention points including the end point of the boundary of the road, the road junction, and the base point of the entrance / exit to the facility are extracted by statistical processing of the traveling locus of the vehicle.

The road connection information generating means selects a lane that can be connected to the attention point based on the locus information classified by the classification means, and generates a traveling route connecting the selected lane and the attention point.

As a result, a road network including facilities on the side of the road can be generated, and driving support including navigation and automatic driving can be realized with high accuracy and efficiency.

In the present invention, each of the locus information analyzed by the analysis means further includes an exceptional travel determination means for determining whether or not there is an exceptional travel including at least one of a lane change of a travel path and an obstacle avoidance action. The travel locus determined to be an exceptional travel is excluded from the classification target by the classification means.

For each of the locus information, it is determined whether or not there is an exceptional run including at least one of the lane change and the obstacle avoidance action of the runway, and the run locus determined to be the exceptional run is excluded from the classification target. This makes it possible to generate more accurate road structure data.

In the present invention, the basic information generated by the basic information generation means is based on the driving information and the road structure information, the vehicle traveling information, the own vehicle position information, the road information of the traveling route, and the peripheral information of the traveling route. It is characterized by including.

More detailed driving support is possible by generating vehicle driving information, own vehicle position information, driving road road information, and driving road peripheral information as basic information based on driving information and the road structure information. It becomes.

In the present invention, the point of interest is a facility that serves as an entrance / exit to the road and has an area in which the vehicle can travel, and the boundary between the route network in which the vehicle can travel, the parking space, and the travelable range in the facility. It is characterized by further having a spatial information generation means for generating spatial information in the facility including landmarks.

It is possible to support driving in a driving space in a facility different from the road.

In the present invention, the spatial information generation means obtains at least one piece of public information including vehicle traveling guidance information regarding the facility and data-distributed city map and aerial photograph, thereby generating the spatial information. It also has an editing means for editing including verification, update, and correction of the spatial information in the facility generated by the means.

By sequentially editing spatial information that changes frequently, such as parking spaces in the facility, optimal driving support can be provided even within the facility.

For example, when verifying, updating, or editing spatial information in a facility, it is necessary to acquire the facility information. From the viewpoint of acquiring information on this facility, open map data such as Open Sreet Map can be used. If the facility is an outdoor parking lot, the aerial photograph can be image-processed to obtain information on the parking position, travel route, and entrance / exit. That is, it is possible to create facility information from public information as well as travel guidance information distributed from the facility.

In the present invention, the moving object information detecting means for detecting the moving object information from the infrastructure sensor including the surveillance camera installed along the traveling path is further provided, and the moving object information detected by the moving object information detecting means is obtained. The feature is that the infrastructure sensor is added to the basic information passing through the traveling path in the area where the infrastructure sensor is installed.

By utilizing infrastructure sensors, the amount of information that is an element of creating a road structure database can be significantly increased.

According to the present invention, a lane capable of connecting a point of interest and the locus information is selected, road structure data including a travel route connecting the selected lane and the point of interest is stored as a road structure database, and a driving support device is used. By utilizing this, it is possible to accurately predict how the preceding vehicle and the oncoming vehicle will move in the next moment.

As described above, in the present invention, it is possible to generate a road network including facilities on the side of the road, and it is possible to realize driving support including navigation and automatic driving with high accuracy and efficiency.

Further, in addition to the above effects, when used in a driving support device, it is possible to accurately predict how the preceding vehicle and the oncoming vehicle will move at the next moment.

It is a schematic block diagram of the road structure database generation apparatus which concerns on 1st Embodiment. It is a perspective view which shows the basic information source which concerns on 1st Embodiment. It is a front view which shows the road boundary which concerns on 1st Embodiment. It is a front view which shows an example of the peripheral road image which concerns on 1st Embodiment. It is a top view which shows the target for estimating the own vehicle position which concerns on 1st Embodiment. It is a top view which shows an example of the exception data of the traveling locus which concerns on 1st Embodiment. It is a plan view which visualized the distribution of a path and a velocity which concerns on 1st Embodiment, (A) is a plan view around an intersection, (B) is a plan view of an intersection, and the distribution of a path and a velocity is visualized. It is a distribution map displayed. It is a top view which shows the merging route and the branching route which concerns on 1st Embodiment, (A) shows the road with one lane on one side, (B) shows the road with two lanes on one side, (C) the road with a median strip. It is a control flowchart which shows the road structure database generation procedure which concerns on 1st Embodiment. It is a schematic block diagram of the driving behavior database generation apparatus which concerns on 2nd Embodiment. It is a perspective view which shows the basic information source which concerns on 2nd Embodiment. (A) to (D) are perspective views showing the installation state of the camera applied as the infrastructure sensor according to the second embodiment. It is a perspective view which shows the positional relationship between the own vehicle and a camera when the position of the own vehicle is estimated by the camera based on triangulation according to the second embodiment. It is a top view of the road represented by the road structure database finally generated which concerns on the 2nd Embodiment. It is a control flowchart which shows the road structure database generation procedure which concerns on 2nd Embodiment.

In the present invention, in order to automatically additionally generate connection information to roadside facilities (stores, parking lots, etc.), that is, roadside areas other than the road on which vehicles can enter and exit, to the road network map at the lane level (lane unit). Database is automatically generated.

The present invention will be described in detail in the following first embodiment and the second embodiment, but in the first embodiment and the second embodiment, a plurality of vehicles traveling on the same road. A large amount of data collected and detected during driving is collected from the vehicle, and connection information between the road network and the roadside area (the connection relationship between the place where the roadside area is entered and exited and the lane) is automatically generated by statistical processing.

In the first embodiment and the second embodiment, data processing on the vehicle side and data processing on the center server side having functions as the basic information generation unit 14 and the roadside area information generation unit 16 described later are performed. Is not clearly distinguished. Various patterns can be considered for the distribution of data processing on the vehicle side / center server side depending on the capacity of the on-board computer and the restrictions on communication with the center server, but the functional arrangement does not greatly affect the processing content of the entire system. Because there isn't.

"First embodiment"
FIG. 1 is a schematic configuration diagram of the road structure data generation device 10 according to the first embodiment.

As shown in FIG. 1, the road structure data generation device 10 includes an in-vehicle sensor group 12, a base information generation unit 14, and a roadside area information generation unit 16. Further, as a storage medium for storing information, a road structure storage unit 18 which is a road structure database is connected to the basic information generation unit 14 and the roadside area information generation unit 16.

(In-vehicle sensor group 12)
The vehicle-mounted sensor group 12 used in the first embodiment includes an environment recognition sensor 24, a vehicle motion sensor 26, and a positioning sensor 30.

The environment recognition sensor 24 assumes a camera or a laser radar, and acquires an object existing around the own vehicle as image data or obtains distance information.

By processing the image data and distance information acquired by the environment recognition sensor 24, moving objects existing around the own vehicle such as a traveling lane, a road boundary, a preceding vehicle, and an oncoming vehicle are detected.

The observation range depends on the specifications of the environment recognition sensor 24, but here it is assumed that the own vehicle monitors the area ahead in the future.

The vehicle motion sensor 26 is composed of a gyro, a wheel speed sensor, and the like, and measures the speed and yaw rate of the vehicle at a predetermined cycle.

The positioning sensor 30 mainly assumes GPS (Global Positioning System) and measures the absolute position, moving direction, and speed of the own vehicle.

(Basic information generation unit 14)
The basic information generation unit 14 generates basic information such as road information and locus information necessary for generating a road structure database based on the detection (measurement) information received from each sensor of the vehicle-mounted sensor group 12.

In the first embodiment, as shown in FIG. 2, various data are collected from the vehicle-mounted sensor group 12 (see FIG. 1) mounted on each of the plurality of vehicles 15 from the center server CS (first). In the embodiment, it is assumed that the data is transmitted to the basic information generation unit 14 and the roadside area information generation unit 16).

As shown in FIG. 1, the basic information generation unit 14 has a road information extraction unit 32, a travel information extraction unit 34, a peripheral information extraction unit 36, and a vehicle position estimation unit 38.

The road information extraction unit 32 detects the existence of a physical road boundary by using the observation information of the environment recognition sensor 24. Detection targets are guardrails and curbs.

The road information extraction unit 32 may detect all areas where there are no obstacles such as guardrails and curbs that obstruct the passage of the vehicle, or may extract only areas having an interval longer than the length that the vehicle can pass. good. An example of a road boundary is shown in FIG.

In the road structure image 50 shown in FIG. 3A, a guardrail 56 is laid along a curb 54 provided at one end of a straight road 52, and a position where the guardrail 56 is interrupted is a crossroads 58. There is.

Further, in the road structure image 60 shown in FIG. 3B, there is no guardrail along the straight road 62, and there is a T-junction 64. As a feature of using the position of the own vehicle for estimation, as shown in FIG. 3B, road surface paint such as the lane mark 66 and the stop line 68 may be extracted.

As shown in FIG. 1, the traveling information extraction unit 34 detects the traveling locus of the own vehicle including a right turn and a left turn. Further, the traveling information extraction unit 34 extracts a place where the vehicle has stopped or a place where the vehicle has decelerated based on the speed information.

The peripheral information extraction unit 36 detects moving objects existing around the own vehicle such as a preceding vehicle and an oncoming vehicle by using the environment recognition sensor 24, and extracts a locus by tracking in the time direction while considering the position of the own vehicle. do.

The peripheral information extraction unit 36 also detects vehicles that enter the lane from the roadside region and vehicles that are stopped trying to exit (the side surface of the vehicle can be observed when viewed from the own vehicle).

FIG. 4 is an example of the peripheral information images 70A, 70B, and 70C. In the peripheral information images 70A and 70B of FIGS. 4A and 4B, the vehicle 72 stops trying to get out of the alley to the main road 74. The peripheral information image 70C of FIG. 4C is a peripheral information image showing a state in which the vehicle 72 is about to enter the alley from the main road 74.

Although not shown in FIG. 4, it is preferable to detect landmarks such as signs, signals, signboards, and cylindrical objects as a feature used when the vehicle position estimation unit 38, which will be described later, estimates the vehicle position. ..

The own vehicle position estimation unit 38 draws map data from the positioning information from GPS, and collates the landmarks registered in the map with the landmarks detected by the peripheral information extraction unit 36 and the road information extraction unit 32. Estimate the position of your vehicle in the map data. From this result, it is possible to know which lane the vehicle is traveling in and where it is currently located.

FIG. 5 shows an image of own vehicle position estimation.

As shown in FIG. 5, the position of the own vehicle 76 can be estimated from the directions of the lane marks 78, 80, the U-turn prohibition sign 82, and the speed limit sign 84, which are visible from the own vehicle 76.

(Roadside area information generation unit 16)
As shown in FIG. 1, the roadside area information generation unit 16 aggregates the basic information generated for each vehicle and generates connection information between the lane and the roadside area.

The roadside area information generation unit 16 has a data classification / statistical processing unit 40 and a road connection information generation unit 42.

The data classification / statistical processing unit 40 receives basic information from the basic information generation unit 14.

The data classification / statistical processing unit 40 classifies basic information into preset road blocks. The road block may be divided in units of links from the intersection to the intersection, or the links may be further subdivided according to a predetermined standard.

Further, the data classification / statistical processing unit 40 distributes the basic information to each lane based on the result of the own vehicle position estimation. At this time, it is desirable to remove the traveling data such as the vehicle traveling across the lane mark as exception data from the results of the own vehicle locus and the own vehicle position estimation.

FIG. 6 shows when a vehicle 92 (see solid line) is traveling in the left lane 88A in the traveling direction on a road 90 having a plurality of lanes (lanes 88A, 88B ...) partitioned by the lane mark 86. It is a top view which shows the locus (see a dotted line) when it notices the existence of a motorcycle 94 in front, and travels across a lane mark 86.

The driving data when driving in such a lane change is removed as exception data.

Further, the data classification / statistical processing unit 40 aggregates the trajectories of the own vehicle and surrounding vehicles, and generates a spatial distribution of loci that exits or joins the roadside region other than the road on which the vehicle can enter and exit.

For example, in the form of a two-dimensional occupied grid map, the distribution of trajectories is expressed in the form of probabilities and histograms so that areas where many loci overlap can be extracted. A threshold value may be set for the probability value or the frequency of the histogram, and a region exceeding a predetermined threshold value may be extracted as a highly reliable region. Furthermore, the detection results of the physical road boundary are superimposed, and it is verified whether or not there is a contradiction with the track information (reliability judgment).

FIG. 7 is an example of visualizing the distribution of the path and the velocity.

FIG. 7A is a plan view of an intersection X (within a dotted line frame) where two roads R1 and R2 intersect and a roadside region 96 existing around the intersection X. FIG. 7 (B) visualizes and displays the distribution of the route and the velocity on the plan view of the intersection X of FIG. 7 (A). The facility 98 (a parking lot or other area where a vehicle can enter) exists beyond the roadside area 96, but in the first embodiment, the inside of the facility 98 is not targeted for driving support.

In FIG. 7B, band-shaped lines L having different densities on the image of the intersection X represent the path, and the density difference of the lines L represents the velocity. The higher the concentration, the higher the speed.

According to FIG. 7B, it is possible to visualize the spatial distribution of the locus that exits to the roadside region 96 other than the roads R1 and R2 where the vehicle can enter and exit, or merges from the roadside region 96 into the roads R1 and R2. In the case of visualization, the identification is not limited to the density, and may be color-coded.

In FIG. 7, the intersection X is shown, but the locations other than the intersection X where the vehicle may make a right turn, a left turn, and a stop may be stored as a database.

As shown in FIG. 1, the road connection information generation unit 42 generates entrance / exit (position) to the roadside area and connection information with each lane. The speed distribution of the vehicle is used when determining the position of the doorway and the branch point from the lane.

Vehicles often decelerate or stop near entrances and exits and lane junctions for safety confirmation. By superimposing the velocity distribution on the locus distribution in such a case, these points can be determined from the result of statistical processing.

Regarding connection information, for example, in a single road without a median strip, the entrance / exit generally needs to be connected to two lanes facing each other. If there is a median strip, the approach is only from the lane near the roadside area. Further, if the road has a plurality of lanes, it is necessary to have connection information such that the road enters from the closest lane in both directions (see FIG. 8).

FIG. 8A is a plan view showing a merging route 100 and a branching route 102 on a road having one lane on each side.

FIG. 8B is a plan view showing a merging route 100 and a branching route 102 on a road having two lanes on each side.

FIG. 8C is a plan view showing a merging route 100 and a branching route 102 on a road having two lanes on each side in which a median strip 104 exists.

As shown in FIGS. 8A to 8C, the situation of the merging route 100 and the branching route 102 depends on the number of lanes and the road structure, but by aggregating the locus information, the connection relationship is naturally established. Can be extracted.

Finally, the connection information is defined by the branch position for each lane, the entrance / exit points of the roadside area, and the route connecting them. As shown in FIG. 8, the merging path 100 and the branching path 102 may be smoothly connected by a curve such as a spline, or may be expressed simply by indicating the connection direction. The confluence from the entrance / exit of the roadside area to the lane is also set based on the collected trajectory distribution.

As shown in FIG. 1, the road connection information generation unit 42 is connected to the road structure storage unit 18 via the roadside information addition / update unit 44.

The roadside information addition / update unit 44 adds the generated connection information (branch position, entrance / exit, and route information) to the road structure storage unit 18. If there is already generated information and it overlaps with it, it can be updated by updating to the newer one or by statistically synthesizing with the old information.

The road structure storage unit 18 adds the generated connection information (branch position, entrance / exit, route information) to the road structure storage unit 18. That is, the road structure storage unit 18 has already generated information, and when the information overlaps, the road structure storage unit 18 updates to new information or statistically synthesizes old information and new information. Can be updated by

Hereinafter, the operation of the first embodiment will be described with reference to the flowchart of FIG.

In step 150, the data detected by the vehicle-mounted sensor group 12 is acquired, and then the process proceeds to step 152 to generate basic information based on the data (sensor data) detected by the vehicle-mounted sensor group 12.

In the next step 153, the generated basic information is transmitted to the center server CS, aggregated, and the process proceeds to step 154.

In step 154, the aggregated travel data is classified. Examples of the classification items include each sensor of the in-vehicle sensor group 12, vehicle type, road type, lane unit, and the like.

In the next step 156, the reliability of the information collected by the statistical processing is determined, and the process proceeds to step 158.

In step 158, as a result of the determination in step 156, it is determined whether or not the reliability is equal to or higher than the threshold value.

If it is determined in step 158 that the reliability is less than the threshold value (reliability <threshold value), it is determined that the amount of aggregated data is insufficient, and the process proceeds to step 153.

If it is determined in step 158 that the reliability is equal to or higher than the threshold value (reliability ≥ threshold value), it is determined that the amount of aggregated data is sufficient, and the process proceeds to step 160.

In step 160, connection information between the road and the roadside area is generated, and then the process proceeds to step 162 to automatically generate spatial information in the roadside area. That is, the spatial arrangement of landmarks existing in the roadside area, the stop position of the vehicle, and the like are generated.

In the next step 164, it is determined that the road connection and the spatial information in the area are changed, and the process proceeds to step 166.

In step 166, as a result of the determination in step 164, it is determined whether or not there is a change in the road connection or the spatial information in the area.

If it is determined in step 166 that there is no change, it is determined that it is not necessary to update the road structure already stored, and the process proceeds to step 153.

If it is determined in step 166 that there is a change, it is determined that the road structure already stored needs to be updated, and the process proceeds to step 168.

In step 168, the information including the change point is stored in the road structure storage unit 18, and this routine ends.

According to the first embodiment, in addition to the lane level road network information, information on the roadside area where vehicles may enter and exit can be automatically generated.

In addition, it is possible to store as a database the places where the vehicle may make a right turn, a left turn, and a stop other than the intersection.

In addition, driving behavior can be comprehensively extracted by aggregating and analyzing a large amount of vehicle data that has traveled on the road for each lane, connection information with areas that can be entered from the lane, and appropriate entry / exit routes. Can be extracted.

Further, the road structure database aggregates the information detected by the in-vehicle sensor group 12 mounted on the vehicle traveling on the road and is generated by automatic processing on the computer, so that the data processing by the operator is suppressed and the cost is low. The system can be realized.

"Second embodiment"
The second embodiment of the present invention will be described below. In the second embodiment, the same configurations as those in the first embodiment are designated by the same reference numerals, and the description of the configurations will be omitted.

In contrast to the above-mentioned first embodiment, the features of the second embodiment include the following features.

(Characteristic a) The point of using the information of the infrastructure sensor installed near the road such as the surveillance camera (Characteristic b) The point of synthesizing the acquireable external map database information (Characteristic c) Not only the connection information with the roadside area 10 is a schematic configuration diagram of the driving behavior data generation device 10A according to the second embodiment of the present invention.

As shown in FIG. 10, the road structure data generation device 10A includes an in-vehicle sensor group 12, a base information generation unit 14, and a roadside area information generation unit 16A. Further, in the second embodiment, the infrastructure sensor 106 is used.

In the second embodiment, as shown in FIG. 11, various data are collected from the vehicle-mounted sensor group 12 (see FIG. 10) mounted on each of the plurality of vehicles 15 and the infrastructure sensor 106 from the center server. It is assumed that the data is transmitted to the CS (in the second embodiment, the system including the infrastructure information generation unit 14 and the roadside area information generation unit 16A).

The vehicle-mounted sensor group 12 used in the second embodiment includes an environment recognition sensor 24, a vehicle motion sensor 26, and a positioning sensor 30.

The basic information generation unit 14 has a road information extraction unit 32, a travel information extraction unit 34, a peripheral information extraction unit 36, and a vehicle position estimation unit 38.

Further, the roadside area information generation unit 16A has a data classification / statistical processing unit 40, a road connection information generation unit 42, a map information acquisition unit 108, and an in-region spatial information generation unit 110.

Further, as a storage medium for storing information, a road structure storage unit 18 is connected to the basic information generation unit 14 and the roadside area information generation unit 16, and an external map storage unit 112 is connected to the roadside area information generation unit 16. There is.

(Use of infrastructure sensor 106)
Here, in the above-mentioned first embodiment, the information of the vehicle-mounted sensor group 12 mounted on the vehicle is collected. On the other hand, in the second embodiment, one feature is to utilize the infrastructure sensor 106 as an information source different from the vehicle-mounted sensor group 12.

As the infrastructure sensor 106, as shown in FIGS. 12 (A) to 12 (D), cameras 106A, 106B, 106C, and 106D corresponding to the respective uses can be applied.

That is, surveillance cameras 106A and 106B (see FIGS. 12A and 12B) are often installed around the road centered on the intersection. In addition, surveillance cameras 106C (see FIG. 12C) are often installed in stores and buildings on the side of the road for the purpose of crime prevention and the like. Further, in the parking lot on the side of the road, a camera 106D (see FIG. 12D) for monitoring the parked vehicle is often installed.

With the spread of IoT (Internet of Things), there is a good possibility that detection (measurement) data by infrastructure sensors 106 represented by cameras 106A, 106B, 106C, 106D can be used in the future. The information of the fixed infrastructure sensor 106 is very useful for observing the road condition.

As shown in FIG. 10, the infrastructure sensor 106 is connected to the moving object information extraction unit 111, and the image data captured by the infrastructure sensor 60 is the moving object information extraction unit together with the position / attitude information of the infrastructure sensor 106. Sent to 111.

The cameras 106A, 106B, 106C, and 106D (see FIG. 12) applied as the infrastructure sensor 106 are basically fixed and their absolute positions do not change, so that the position information needs to be set only once at the time of installation. If the posture (imaging optical axis) of the cameras 106A, 106B, 106C, 106D can control the optical axis direction by pan / tilt / zoom or the like, a parameter including the optical axis direction information at the time of imaging is associated with the image. Is preferable.

The moving object information extraction unit 111 extracts a moving vehicle from images captured by, for example, cameras 106A, 106B, 106C, 106D by the following procedures (1) and (2), and tracks the position in the time direction. To detect the traveling locus.

(Procedure 1) First, a candidate for a moving object area is detected in an image by using background subtraction and time difference. A search area is set in the screen including the candidate area, and the vehicle is detected by using the pattern recognition process.

(Procedure 2) Assuming that the installation height and line-of-sight direction of the cameras 106A, 106B, 106C, 106D are known, the vehicle position at the lane level is estimated using means such as the principle of triangulation (see FIG. 13). do.

As shown in FIG. 13, any one of the cameras 106A, 106B, 106C, and 106D (for example, the camera 106A) is located directly above the position where the vehicle V is located on the road surface and is separated from the vehicle V by a distance d along the road surface. ) Is installed. It is assumed that the camera 106A is installed at a height h from the road surface.

The position of the vehicle V can be specified by the direction when the vehicle V is imaged by the camera 106A at an angle θ with respect to the perpendicular line of the camera 106A.

If the camera 106A has sufficient resolution, the lane mark on the road surface can be detected, so that the traveling lane can be determined.

As shown in FIG. 10, the moving object information extraction unit 111 is connected to the roadside area information generation unit 16A.

The moving object information extraction unit 111 transmits the locus information of the moving object detected in the above procedure to the roadside area information generation unit 16A, and integrates the information from the in-vehicle sensor group 12 for use.

It is also effective to set an appropriate weight for the information of the in-vehicle sensor group 12 and merge (integrate) the information of the infrastructure sensor 106. The weight may be changed according to the position of the image pickup target. By integrating the information of the infrastructure sensor 106, the accuracy and reliability of the generated information can be improved, and the change point can be detected quickly.

(Use of external map information)
In the case of large stores and public facilities, the entrance / exit to the parking lot may be described in the general map data.

In addition, information in the parking lot, such as the arrangement of parking spaces, may be provided by communication.

In the future, there may be a service in which information such as a map and empty space in the parking lot is provided to the vehicle side by wireless communication when passing through the gate at the entrance of the parking lot. If such public information (external maps and aerial photographs) can be obtained, it is possible to process and integrate the information to make up for the lack of spatial information in the region and to verify the generated structure.

That is, it is possible to obtain information on the parking position, the traveling route, and the entrance / exit by performing image processing on open map data such as Open Street Map or, if the facility is an outdoor parking lot, an aerial photograph. That is, it is possible to create facility information from public information as well as travel guidance information distributed from the facility.

Therefore, in the second embodiment, the map information acquisition unit 108 is added to the roadside area information generation unit 16A to perform data conversion for integrating the information provided from the external map storage unit 112.

For example, the location of entrances and exits in the roadside area, routes that can be traveled within the area, parking spaces, placement of characteristic structures (concrete pillars, road surface paint, signs / signs for guidance and regulations, artificial markers, etc.), etc. Is read, converted into a desired format, and sent to the road connection information generation unit 42 and the intra-regional space information generation unit 110.

(Generation of spatial information in the roadside area)
The in-region spatial information generation unit 110 automatically generates spatial information necessary for route guidance in the roadside region, in addition to the connection between the road network in the road connection information generation unit 42 and the entrance / exit of the roadside region.

Technically, it uses a method called SLAM (Simultaneous Localization and Mapping) as a base.

Absolute position information is also given if GPS positioning is possible outdoors.

For example, in Patent Document 1, a road surface image is generated in order to extract a lane mark, but the track in the roadside region does not always have an appropriate mark.

For example, when a camera is used as an in-vehicle sensor group 12, in a method called V-SLAM (Visual SLAM), prominent feature points are extracted from an image, and the same feature points are searched and collated before and after movement to perform triangulation. The three-dimensional position of the feature point can be estimated by the principle of.

If the three-dimensional positions of many feature points are stored, the positions of the own vehicle can be estimated even at different times by collating them with the current observation results.

However, a lot of memory is required to store many feature points. In addition, since the extraction of feature points is affected by lighting changes, robustness to time and weather is an issue.

Therefore, the feature points should be kept to a minimum and only those that are spatially and temporally stable. Instead, as described in the above-mentioned "Use of external map information", characteristic structures are extracted and their spatial arrangements are made into a database.

For example, the boundary line of a runnable track in the area, road surface paint such as parking space and temporary stop, concrete pillars and poles, signs / signs such as guidance and regulation, artificial markers, and the like.

These objects can be automatically extracted from the camera image using image processing technology such as pattern recognition. Information collected from multiple vehicles is collated in the neighborhood area, and the reliability of observation is calculated by statistical processing. The three-dimensional position of the object that satisfies the predetermined reliability is registered as spatial information.

According to the second embodiment, in the finally generated road structure database (see FIG. 7B) shown in FIG. 7 in the first embodiment described above, the roadside region 48 and the road network 47 Connection-related routes 49 can be generated in more detail.

An example of the finally generated road structure database is shown in FIG.

FIG. 14A is a general road map 46, and FIG. 14B is a detailed map generated specifically for a specific area from the road map 46 by the road structure data generation device 10. It is 46A.

The roadside area 48 is set in addition to the lane-level road network 47 (indicated by the dotted line in FIG. 14B), and the connection-related line 49 between the roadside area 48 and the road network 47 (solid line in FIG. 14B). (Displayed in) is added. Further, in the roadside area 48, the landmark 48A, the stop position 48B, and the travelable route 48C are automatically generated.

Hereinafter, the operation of the second embodiment will be described with reference to the flowchart of FIG. For the same processing step as the predetermined procedure (see FIG. 9) in the first embodiment described above, "A" is added to the end of the reference numeral.

In step 150A, the data detected by the vehicle-mounted sensor group 12 is acquired, and then the process proceeds to step 152A to generate basic information based on the data (sensor data) detected by the vehicle-mounted sensor group 12.

In the next step 153A, the generated basic information is transmitted to the center server CS, aggregated, and the process proceeds to step 154A.

In step 154A, the aggregated travel data is classified. Examples of the classification items include each sensor of the in-vehicle sensor group 12, vehicle type, road type, lane unit, and the like.

Here, in the second embodiment, the data detected by the infrastructure sensor 106 is added as the basic information to be classified.

That is, in step 170, the data detected by the infrastructure sensor 106 is acquired, and then the process proceeds to step 172 to generate basic information including the trajectory of the moving object.

In the next step 174, the generated basic information is transmitted to the center server CS, aggregated, and the process proceeds to step 154A.

As a result, the infrastructure information based on the data detected by the in-vehicle sensor group 12 and the infrastructure information based on the data detected by the infrastructure sensor 106 are aggregated in the center server CS.

In the next step 156A, the reliability of the information collected by the statistical processing is determined, and the process proceeds to step 158A.

In step 158A, as a result of the determination in step 156A, it is determined whether or not the reliability is equal to or higher than the threshold value.

If it is determined in step 158A that the reliability is less than the threshold value (reliability <threshold value), it is determined that the amount of aggregated data is insufficient, and the process proceeds to step 153A.

If it is determined in step 158A that the reliability is equal to or higher than the threshold value (reliability ≥ threshold value), it is determined that the amount of aggregated data is sufficient, and the process proceeds to step 160A.

Here, in the second embodiment, when the connection information between the road and the roadside area is generated, the related information obtained from the existing map data is added. The related information obtained from the map data includes road connection information and parking position information.

That is, in step 176, related information is acquired from the existing map data, and the process proceeds to step 160A.

In step 160A, connection information between the road and the roadside area is generated. At this time, in the second embodiment, in addition to the basic information about the road, the road connection information, the parking position information, and the like are acquired from the existing map data, so that the connection information between the road and the roadside area is generated. The accuracy can be improved.

In the next step 162A, spatial information in the roadside region is automatically generated. That is, the spatial arrangement of landmarks existing in the roadside area, the stop position of the vehicle, and the like are generated.

In the next step 164A, it is determined that the road connection and the spatial information in the area are changed, and the process proceeds to step 166A.

In step 166A, as a result of the determination in step 164A, it is determined whether or not there is a change in the road connection or the spatial information in the area.

If it is determined in step 166A that there is no change, it is determined that it is not necessary to update the road structure already stored, and the process proceeds to step 153A.

If it is determined in step 166A that there is a change, it is determined that the road structure already stored needs to be updated, and the process proceeds to step 168A.

In step 168A, the information including the change point is stored in the road structure storage unit 18, and this routine ends.

According to the second embodiment, the road structure database aggregates the information of the in-vehicle sensor group 12 mounted on the vehicle traveling on the road and the infrastructure sensors 106 such as the cameras 106A to 106D shown in FIG. 12, and is a computer. Generated by the above automatic processing. It is possible to realize a system at low cost by suppressing data processing by the operator.

In addition, in the first embodiment and the second embodiment, when there is a change in the road connection or the spatial information in the area, it is stored in the road structure storage unit 18 (step 166 in FIG. 9 →). 168 and the process of step 166A → 168A in FIG. 15), when the data of the road structure storage unit 18 is old (when a certain amount of time has passed since the last update), the reliability is higher than the predetermined value regardless of the passage of time. If it is determined that the information has been changed, the road connection or the spatial information in the area may be updated even if there is no change. Further, it may be updated periodically in combination with the case where there is a change, or it may be updated by a manual instruction operation.

10 Road structure data generator 12 In-vehicle sensor group 14 Infrastructure information generator 16 Roadside area information generator 18 Road structure storage unit 24 Environment recognition sensor 26 Vehicle motion sensor 30 Positioning sensor 32 Road information extraction unit 34 Driving information extraction unit 36 Peripheral information Extraction unit 38 Own vehicle position estimation unit 40 Data classification / statistical processing unit 42 Road connection information generation unit 44 Roadside information addition / update unit 46 Road map 46A Detailed map 48A Landmark 48B Stop position 48C Travelable route 50 Road structure image 52 Straight Road 54 Edge stone 56 Guard rail 60 Road structure image 62 Straight road 64 T-shaped road 66 Lane mark 68 Stop line 70A, 70B, 70C Area information image 72 Vehicle 74 Main road 76 Own vehicle 78, 80 Lane mark 82 U-turn prohibition sign 84 Speed Restriction sign 86 Lane mark 88A, 88B Lane 90 Road 92 Vehicle 94 Two-wheeled vehicle 96 Roadside area 98 Facility 100 Confluence route 102 Branch route 104 Central separation zone R1, R2 Road X intersection (second embodiment)
10A Road structure data generation device 16A Roadside area information generation unit 106 Infrastructure sensor 106A, 106B, 106C, 106D Camera 108 Map information acquisition unit 110 Area spatial information generation unit 111 Moving object information extraction unit 112 External map storage unit

Claims (6)

An acquisition means for acquiring driving information related to the driving of the vehicle from an in-vehicle sensor mounted on the vehicle, and
Based on the driving information acquired by the acquisition means and the road structure information known in advance, the basic information generation means for generating the basic information which is the basis for recognizing the road structure, and the basic information generation means.
An analysis means for analyzing trajectory information when the vehicle is traveling based on the basic information generated by the basic information generation means, and an analysis means.
A classification means for classifying the locus information analyzed by the analysis means for each lane of the travel path, and a classification means.
Extraction means for extracting points of interest including end points of road boundaries, road junctions, and base points of entrances and exits to facilities by statistical processing of vehicle travel loci.
A road connection information generating means that selects a lane that can be connected to the attention point based on the locus information classified by the classification means and generates a traveling route connecting the selected lane and the attention point.
Road structure data generator with. For each of the locus information analyzed by the analysis means, the exception running determination means for determining whether or not there is an exceptional running including at least one of a lane change of a running path and an obstacle avoidance action is further provided.
The travel locus determined to be an exceptional travel is excluded from the classification target by the classification means.
The road structure data generation device according to claim 1. The basic information generated by the basic information generation means is
Based on the driving information and the road structure information, the vehicle includes traveling information of the vehicle, own vehicle position information, road information of the traveling route, and peripheral information of the traveling route.
The road structure data generation device according to claim 1 or 2, wherein the road structure data generation device is characterized by the above. The point of interest is a facility that serves as an entrance / exit to the road and has an area in which the vehicle can travel. The road structure data generation device according to any one of claims 1 to 3, further comprising a spatial information generation means for generating spatial information in the facility including the facility. The spatial information generation means
By acquiring at least one piece of public information including vehicle travel guidance information related to the facility and data-distributed city map and aerial photograph, the spatial information in the facility generated by the spatial information generation means is verified. The road structure data generation device according to claim 4, further comprising an editing means for editing including updating and correction. It also has a moving object information detection means that detects moving object information from infrastructure sensors including surveillance cameras installed along the road.
Any of claims 1 to 5, wherein the moving object information detected by the moving object information detecting means is added to the basic information passing through the traveling path in the area where the infrastructure sensor is installed. The road structure data generation device according to item 1 .

Images