JP6171612B2 - Virtual lane generation apparatus and program - Google Patents

Description

  The present invention relates to a virtual lane generation device and a program.

  2. Description of the Related Art Conventionally, a white line detection device that extracts edges constituting a lane mark from an in-vehicle camera and detects a white line based on their distribution is known (Patent Document 1). In addition, a boundary line detection apparatus is known that calculates road parameters after a white line is detected, determines whether the white line is a main line, and calculates a route on which the host vehicle should travel (Patent Document 2).

Japanese Patent Laid-Open No. 2004-240636 JP 2005-346382 A

  The technologies described in Patent Documents 1 and 2 can prevent lane departure based on a road white line, and are effective on highways and the like. However, there are cases where the white line is unclear or does not exist on general roads, and it is difficult to apply to such cases.

  The present invention has been made in view of the above circumstances, and provides a virtual lane generation device and a program capable of generating a virtual traveling lane even when a white line is unclear or does not exist. Objective.

  In order to achieve the above object, a virtual lane generation device of the present invention is obtained based on a stereo image that is mounted on a vehicle and that is generated by an imaging unit that images the periphery of the vehicle to generate a stereo image. Based on the parallax information of the stereo image, road detection means for detecting a travelable area of the vehicle, absolute position of the vehicle detected by the position detection means for detecting the absolute position of the vehicle, and road information of each point Initial value setting means for setting an initial value of a lane parameter representing a travel lane on which the vehicle is traveling based on road map data prepared in advance, and the travel possible detected by the road detection means Based on the region and the initial value of the lane parameter set by the initial value setting means, a virtual travel lane in which the vehicle is traveling Of the lane parameter estimation, based on the estimated lane parameter, the vehicle is configured to include a virtual lane generating means for generating data representing a virtual traveling lane running.

  The program according to the present invention includes a computer mounted on parallax information of the stereo image obtained on the basis of a stereo image generated by an imaging unit that is mounted on a vehicle and generates a stereo image by imaging the periphery of the vehicle. Based on road detection means for detecting a travelable area of the vehicle, a road map prepared in advance representing the absolute position of the vehicle detected by the position detection means for detecting the absolute position of the vehicle, and road information of each point Initial value setting means for setting an initial value of a lane parameter representing a travel lane in which the vehicle is traveling based on the data, the travelable area detected by the road detection means, and the initial value setting means Based on the initial value of the lane parameter set by the lane parameter of the virtual driving lane in which the vehicle is driving. Estimating the over data, based on the estimated lane parameters, a program for functioning as a virtual lane generating means for generating data representing a virtual lane on which the vehicle is traveling.

  According to the present invention, the imaging unit captures the periphery of the vehicle to generate a stereo image, and the road detection unit, based on the stereo image parallax information obtained based on the stereo image generated by the imaging unit. Thus, the travelable area of the vehicle is detected.

  Then, based on the absolute position of the vehicle detected by the position detection means for detecting the absolute position of the vehicle by the initial value setting means, and road map data prepared in advance representing road information of each point, An initial value of a lane parameter indicating a traveling lane in which the vehicle is traveling is set. The virtual lane generation means is a virtual vehicle in which the vehicle is running based on the travelable area detected by the road detection means and the initial value of the lane parameter set by the initial value setting means. A lane parameter of the traveling lane is estimated, and data representing a virtual traveling lane in which the vehicle is traveling is generated based on the estimated lane parameter.

  As described above, based on the parallax information of the stereo image, the vehicle travelable area is detected, and based on the absolute position of the vehicle and the road map data, the initial lane parameter indicating the travel lane in which the vehicle is traveling When a white line is unclear or does not exist by setting a value and estimating the lane parameter of a virtual driving lane in which the vehicle is driving based on the detected travelable area and the initial value of the lane parameter Even so, a virtual travel lane can be generated.

  The virtual lane generation apparatus according to the present invention is based on vehicle detection means for detecting a vehicle candidate based on at least one of the stereo images generated by the imaging means, and on the stereo image generated by the imaging means. Driving based on the relationship between the travel lane in which the vehicle is traveling detected or the virtual travel lane generated by the virtual lane generation means and the vehicle candidate detected by the vehicle detection means And a target vehicle detecting means for detecting a target vehicle used for support.

  The road detection unit according to the present invention detects the travelable area based on parallax information of the stereo image obtained based on the stereo image generated by the imaging unit for each time, and the virtual lane generation unit Generates a plurality of particles representing the initial value of the lane parameter set by the initial value setting means, and for each time, based on the travelable area detected by the road detection means, from the plurality of particles By sampling each of the particles and repeatedly updating the lane parameter represented by each of the sampled particles according to a predetermined state transition model, the lane parameter representing the virtual traveling lane for each time is obtained. Based on estimated lane parameters Te, it is possible to generate the data representing the virtual lane.

  Further, the virtual lane generation unit can update the lane parameter represented by each of the sampled particles according to the state transition model determined in advance based on a clothoid curve.

  The virtual lane generation device further includes a vehicle detection unit that detects a vehicle based on at least one of the stereo images generated by the imaging unit, and the virtual lane generation unit The particles representing the lane parameters of the traveling lane that overlaps the region of the other traveling lane in which the vehicle detected by the vehicle detecting unit is traveling out of the travelable region detected by the road detecting unit. The particles are sampled each of the particles with a smaller weight so as to overlap the area of the driving lane in which the vehicle is traveling, and the lane parameters represented by the sampled particles are represented by the state transition model. Can be updated accordingly.

  The target vehicle detection means may detect the vehicle candidate existing in the virtual traveling lane as a tracking target vehicle.

  The storage medium for storing the program of the present invention is not particularly limited, and may be a hard disk or a ROM. Further, it may be a CD-ROM, a DVD disk, a magneto-optical disk or an IC card. Furthermore, the program may be downloaded from a server or the like connected to the network.

  As described above, according to the virtual lane generation device and the program of the present invention, the travelable area of the vehicle is detected based on the parallax information of the stereo image, and based on the absolute position of the vehicle and the road map data. An initial value of a lane parameter representing a travel lane in which the vehicle is traveling is set, and a virtual travel lane in which the vehicle is traveling based on the detected travelable area and the initial value of the lane parameter. By estimating the parameters, there is an effect that a virtual travel lane can be generated even when the white line is not clear or does not exist.

1 is a block diagram showing a driving support control device according to a first embodiment of the present invention. It is a figure which shows the example of a driving assistance system. (A) The figure which shows the example of a parallax map, (b) The figure which shows the gradient of the perpendicular direction of a parallax map, (c) The figure which shows the detection result of a driving | running | working area | region. (A) The figure for demonstrating a clothoid road model, (b) The figure which shows the result of having estimated the road structure, (c) The figure which shows the result of having estimated the road structure. (D) The figure which shows the example of the road energy map corresponding to the said FIG. 4-1 (b), (e) The figure which shows the example of the road energy map corresponding to the said FIG. 4-1 (c). It is a figure which shows the result of having produced | generated the virtual travel lane of each time. (A) The figure which shows a driveable area | region, (b) The figure which shows the orthographic projection image which shows a driveable area | region, (c) It is an enlarged view of a bounding box part. It is a flowchart which shows the content of the virtual target production | generation routine in the computer of the driving assistance control apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the virtual lane production | generation process in the computer of the driving assistance control apparatus which concerns on the 1st Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the first embodiment, the present invention is applied to a driving support control device that detects a driving lane when performing lane keeping control and a tracking target vehicle when performing tracking control and outputs the detected vehicle to a driving support unit. A case will be described as an example.

<Overview>
This embodiment extends the use of advanced driver assistance systems (ADAS), which are mainly used on systematic roads such as arterial roads, to ordinary city streets, especially roads without marks. It relates to the technique to do. Specifically, this embodiment can generate a virtual target for satisfying the requirements of the ADAS system without changing the system. In the present embodiment, generation of a virtual lane mark for an LKA (lane keep assistance) system, and identification of a preceding vehicle in the own vehicle lane for an ACC (adaptive cruise control) system (Or generation of a virtual vehicle).

  Many unresolved problems still remain in ADAS technology, particularly those that use affordable sensors such as cameras. In order to implement these systems for daily traffic, there are mainly the following two types of difficulties.

(1) Structural information according to the function of the system is required. For example, the LKA system needs to detect a lane mark in order to run the vehicle without protruding from the inside of the roadway. The ACC system requires detection of forward / preceding vehicles in the vehicle lane, and the PCS system requires detection of such vehicles to avoid collisions. However, many normal city streets do not have a lane mark that defines a driving lane or displays a lane curvature. If the driving lane and the lane curvature are not understood, the LKA system and the ACC system, as shown in FIG. 2, use a roadside vehicle (such as a parked vehicle) or another driving lane (a vehicle that changes course on a crossing / curved road). Etc.) may not be distinguished from real targets on the vehicle lane, and false alarms and erroneous control commands may be generated. Furthermore, PCS (pre-crash system) requires free space information on the road in order to avoid hitting other objects during the collision avoidance operation. Some systems use GPS along with other prior information such as RNDF (Route Network Definition File) and GIS (Geographic Information System) to obtain such information. However, GPS has limited spatial and temporal resolution, and map data is not up-to-date and may be inaccurate, especially at the lane level. Furthermore, such a map is static and cannot accommodate a dynamic vehicle in real time.

(2) It is difficult for computer vision to recognize targets such as lanes and other vehicles at traffic sites. This is because the appearance of the target is often not easily detected and affects several factors in time and space, such as road material, vehicle type and orientation, lighting, weather conditions, etc. Because it receives. These factors can have a significant impact on the visibility of the target and thus also provide for system accuracy and reliability.

  In this embodiment, a virtual lane is generated to reflect structural information in a normal city street, and the context relationship among the own vehicle, the traveling lane, and another vehicle satisfies the ADAS requirement, and the object detection. Used to improve performance.

  Today, high-quality cameras are commercially available at very low prices, so this embodiment aims at a stereoscopic-based solution to the above-mentioned problem. In order to execute LKA, ACC and PCS on a road without a lane mark, it is necessary to detect a driving lane, nearby vehicles, and estimate their relationship.

  In this embodiment, first, a travel lane is recognized from an input stereo image based on a navigation map and a clothoid road model. In the traveling lane recognition process, a nearby vehicle is detected from the input image. Subsequently, the recognition result of the traveling lane and the detection result of the vehicle are used for context correlation for the following two purposes. One objective is to increase detection performance and reliability for lanes and vehicles, and the other is to generate virtual targets that meet the requirements of the ADAS system.

  When an image input is given, lane mark detection is first executed. When the lane mark is detected, ADAS such as LKA uses the detection result to guide the driver / own vehicle. When there is no lane mark on the current road, a virtual lane that satisfies the requirements of the LKA system is generated from the boundary of the travelable area as follows.

  First, based on the stereo image, a parallax map is calculated using SGM (Semi-global Matching Stereo), and a cost map that describes the shape characteristics of the road in the image is generated. Subsequently, based on the cost map, the boundaries of the travelable area are detected in an MRF (Markov Random Field) based method using a belief propagation algorithm. Further, the vehicle is positioned in the navigation map using GPS positioning, and an initial value of the lane parameter representing the traveling lane is provided based on the road information of the vehicle position obtained from the navigation map. By using the resulting initial value, a virtual lane is generated in a particle filter based manner. At this time, the particle filter is updated based on the clothoid road model. This is because clothoid road models are widely used in road design to build lane marks and curbs in the real world.

  Also, most of the existing ADAS systems focused on detecting objects of interest or creating scene segmentation (different categories such as roads, vehicles, etc.) The level of information provided by the method is too limited to implement ADAS systems such as LKA, ACC, and PCS on normal city streets as described above. Understanding the road structure itself, such as nearby vehicles and road lanes in the road context, is the key to successful ADAS application in urban environments. This is extremely difficult especially in a dynamic urban scene where many clutters occur or in a scenario where there is no lane mark to display the road structure. In the present embodiment, an object of interest is detected by correlating the detected or generated travel lane with the vehicle.

<System configuration>
As shown in FIG. 1, the driving support control device 10 according to the first embodiment of the present invention captures a GPS receiver 12 that receives radio waves from GPS satellites and the front of the host vehicle to obtain a stereo image. The driving lane is detected or generated based on the imaging device 14 that generates the image, the received signal from the GPS satellite received by the GPS receiver 12, and the stereo image captured by the imaging device 14, and the driving support unit 18. And a computer 16 for generating a virtual target used in the above.

  The driving support unit 18 performs lane keeping control of the host vehicle based on the detected or generated travel lane, and also follows the driving of the host vehicle based on, for example, a preceding vehicle that is generated as a virtual target. Take control.

  The GPS receiver 12 receives radio waves from a plurality of GPS satellites at each time and outputs received signals from all the received GPS satellites to the computer 16.

  The imaging device 14 repeatedly captures the front of the host vehicle at each time, repeatedly generates a stereo image, and outputs the stereo image to the computer 16.

  When the computer 16 is represented by functional blocks, as shown in FIG. 1 above, GPS satellite information is obtained from the GPS receiver 12 for all GPS satellites that have received radio waves, and the position of the vehicle at each time is determined. When the position detection unit 20 to calculate, the lane mark detection unit 22 that detects the lane mark of the traveling lane of the host vehicle from the stereo image generated by the imaging device 14, and when no lane mark is detected for each time A road detection unit 24 that detects the travelable area of the host vehicle based on the stereo image generated by the imaging device 14 and a vehicle that detects other vehicle candidates based on the stereo image generated by the imaging device 14. A detection unit 26, a map database 28 for storing navigation map data storing road information at each point, and a road model Based on the road model storage unit 30 that stores data to be represented, the detected position of the own vehicle, the detected travelable area, the detected other vehicle candidates, the navigation map data, and the data representing the road model, A lane recognition unit 32 for recognizing a travel lane or virtual travel lane, a context correlation calculation unit 34 for calculating a context correlation between the recognized travel lane or virtual travel lane and the detected vehicle, and a context model. A context model storage unit 36 that stores data, a target vehicle detection unit 37 that detects a vehicle based on the context correlation, and a virtual target generation unit 38 that generates a virtual target based on the detected vehicle. Yes. The lane recognition unit 32 is an example of an initial value setting unit and a virtual lane generation unit.

  The position detection unit 20 acquires GPS satellite information for all GPS satellites that have received radio waves from the GPS reception unit 12, and calculates the absolute position of the host vehicle based on the acquired GPS information.

  The lane mark detection unit 22 detects a lane mark, for example, by pattern matching, for example, from the left image of the stereo image generated by the imaging device 14 at each time.

  When no lane mark is detected by the lane mark detection unit 22, the road detection unit 24 calculates a parallax map from the stereo image generated by the imaging device 14 (see FIG. 3A), and calculates the calculated parallax map. From this, a cost map based on the vertical parallax gradient is generated (see FIG. 3B). The road detection unit 24 detects the boundary of the travelable area based on the generated cost map according to the MRF-based method (see FIG. 3C), and detects the travelable area.

  The vehicle detection unit 26 uses, for example, a left-right image of the stereo image generated by the imaging device 14 in a region of interest (ROI: region-of-interest) and a gradient direction histogram (HOG) feature. Generated by the imaging device 14 by detecting vehicle candidates based on a deformable part-based model (DPF) and determining a threshold value of the score obtained in the detection process for the vehicle candidates. The vehicle is detected from the left image of the stereo image.

  The map database 28 stores navigation map data storing road information (lane width, curvature, change in curvature) at each point.

  The road model storage unit 30 stores parameters of a state transition model based on the clothoid road model.

  When the lane mark detection unit 22 detects a lane mark, the lane recognition unit 32 recognizes a traveling lane based on the detected lane mark. When there are a plurality of traveling lanes, the plurality of traveling lanes are recognized.

  When no lane mark is detected by the lane mark detection unit 22, the lane recognition unit 32 detects the detected vehicle position, the detected travelable area, the detected other vehicle, the navigation map data, and the state transition model. Based on these parameters, a virtual travel lane in which the host vehicle is traveling is generated for each time as described below.

  First, in the clothoid road model, the lane boundary position is expressed by the following equation.

However, x _{k, t} (z) is the lateral position of the lane boundary at the distance z and time t. k = {-1, 1} indicates the lane boundary side, that is, -1 indicates the left side and 1 indicates the right side. As shown in FIG. 4A, the road and the position of the vehicle relative to the road at time t are the lane width w _t , the lateral offset x _{0, t of} the vehicle relative to the center line of the road, and the vehicle relative to the road. Given by the yaw angle θ _t , the curvature c _{0, t} , and the change in curvature c _{1, t} . These parameters form a state vector S _t to be estimated, where S _t = (w _t , x _{0, t} , θ _t , c _{0, t} , c _{1, t} ), which is an example of a lane parameter is there. The state transition model between time t and t + Delta] t for the S _t is defined as the following equation on the basis of the clothoid road model.

  In the present embodiment, a road structure in which no lane mark is detected is estimated using a particle filter in a tracking process in a tracking process (see FIGS. 4-1 (b) and (c)). In each estimation period, 1500 particles are generated, each particle has a lane parameter indicating a lane boundary position based on the clothoid road model, and the weight associated with these particles is shown in FIG. , (E) is calculated based on the likelihood measurement on the road energy map. The likelihood score is calculated by the following formula.

However, P _L is a region between the virtual left and right lane marks. p is _{P L} inner _{pixels, N PL} is the size of the _{P L.} ρ is a positive constant representing the energy of the road points α, β, γ, and α, β, γ are positive constants for adjusting the energy of a specific point. In the experiment, in this embodiment, ρ = 1, α = 10, β = 5, and γ = 50 were set. Actually, the likelihood score defined by the above equation (3) is the average value of each point in the area between the virtual left and right lane marks, and the estimated virtual driving lane is More road points are included while excluding non-road points.

  As shown in the above equation (4), each point p is a free running lane Pblue (see the thin hatched area in FIG. 4-2 (d) above) in which the host vehicle can travel, and a travelable area. Among the driving lanes Pred (see the thin hatched area in FIG. 4-2 (e) above) and other driving possible areas Proad (see FIGS. 4-2 (d) and (e) above). A white area), and a non-travelable area Pnon-road (see black areas in FIGS. 4-2 (d) and (e) above).

The road energy map representing the free travel lane Pblue in which the host vehicle can travel in the travelable region, the region travel lane Pred in which the host vehicle cannot travel in the travelable region, and the other travelable region Proad is traveled. Based on the correlation between the detection result of the possible region and the detection result of the other vehicle, it is obtained as shown in FIGS. 4D and 4E. For example, when the detected other vehicle is outside the safety ROI, the traveling lane in which the other vehicle is traveling can be used for traveling the own vehicle, so that the own vehicle can travel. Is required as a free-running lane Pblue. Further, when the detected other vehicle is inside the safety ROI, the other traveling lane in which the other vehicle is traveling is occupied by the other vehicle, and the own vehicle is traveling. Since it cannot be used, it is calculated | required as driving lane Pred in which the own vehicle cannot drive | work. The above-described free travel lane Pblue and the travel lane Pred incapable of travel are defined by the same parameters as the parameters other than x0, _t in the above state vector.

  The lane recognition unit 32 acquires road information (lane width, curvature, etc.) at the current own vehicle position from the navigation map data based on the detected own vehicle position, and uses the own vehicle position and road information, An initial value of the state vector S is set. A plurality of particles are generated using the initial value of the set state vector. For each generated particle, the weight is calculated according to the above equation (3) for the virtual traveling lane represented by the state vector of the particle.

  In accordance with the calculated weight, the particles are sampled from the generated particles, and the state vectors of the sampled particles are updated according to the above equation (2). Based on the road energy map based on the travelable area detected at each time, the lane parameters at the vehicle position at each time are estimated and estimated by repeating the calculation of the weight of each particle and the sampling of the particle at each time. Based on the lane parameter at each time, a virtual travel lane at each time the host vehicle is traveling is recognized (see FIG. 5).

  Next, the principle of generating a virtual target based on the context correlation between the recognized travel lane or virtual travel lane and the detected vehicle will be described.

  In the real world, there is a strong relationship between the environment and the objects found in the environment, for example between the road and the vehicle. We present five relationship classes between object settings and objects that can characterize the composition of the object to the real world. These are: 1) intervening (object cuts into its background), 2) support (object tends to rest on the surface), 3) possibility (found in one context but not found in another context) 4) location (when an object is expected in one scene, the object is often found in one location and not found elsewhere), 5) a well-known size (with object The set of size relationships with other objects is limited). Such context information can play a beneficial role in object detection in at least two ways. First, this context information can simplify object identification by reducing the number of positions and scales that need to be considered. Second, this context information is used to improve object recognition performance when local appearance support is inadequate due to several factors such as clutter, noise, occlusion, pose and lighting variations. The detection hypothesis can be weighted again.

  In addition, false detections returned by only the appearance model can be divided into the following three main categories.

1) Incorrect vehicle detection at an unrealistic position: In the detection by an appearance model, an unrealistic position in an image, such as in the air or in a non-running area, which is often detected with high reliability by stereoscopic vision. A vehicle located at is detected. However, in the real world, a vehicle is more likely to be at a certain location, such as on or near a road.

2) False vehicle detection with an unreasonable size: Appearance model detection often looks close to the camera, but returns very small detection or is far away from the camera but returns very large detection . However, according to the pinhole camera model, the vehicle size in the image should be inversely proportional to the distance between the vehicle and the camera.

3) False vehicle not detected due to weak appearance support: In real traffic, the vehicle usually travels along the lane. A vehicle in its own lane may be very small in the image when it is far away from the camera. The vehicles in the adjacent lanes may only be partially visible due to shielding by the vehicles ahead. In these cases, the appearance support is too weak to be detected by the appearance model alone.

  To contribute to improved vehicle detection in the above situation, the present embodiment is provided by the boundaries of detected travelable areas for both low level detection improvement and high level road understanding. Combine spatial contexts. Such an approach can greatly improve the performance and reliability of the ADAS system.

  In the present embodiment, the vehicle detection unit 26 detects vehicle candidates within the region of interest ROI. In the context correlation calculation unit 34, the detected vehicle candidate is used in the following two methods. In one method, rough threshold processing is performed in order to extract vehicle candidates having relatively high appearance evidence. In the other method, vehicle candidates are correlated with the road context to search for spatial support including spatial location and size. Both results obtained are then cross-validated using AND logic for the vehicle recognition output, thereby retrieving the most reasonable vehicle detection results for both appearance and spatial support.

In the present embodiment, the vehicle space configuration between the road and the vehicle candidates (see the rectangular frame in FIG. 6A) displayed in the bounding box b = {p _u , p _v , w, h}. Is defined as a four-dimensional spatial context descriptor β = {d _L , d _R , O, H}. (P _u , p _v ) are the image coordinates of the upper left point of the bounding box, and w and h are the width and height of the bounding box, respectively. d _L is the distance of the vehicle candidate to the center line of the own vehicle lane (see the dodd portion in FIG. 6C). As described above, a vehicle having a weak appearance support usually appears in a driving lane, and therefore, in the present embodiment, a high weight is given to a candidate vehicle located in a parallel driving lane, and a position where there is no possibility. A low weight is assigned to the vehicle candidate located at. d _R is the vertical distance of the vehicle candidate to the road area. A vehicle traveling on the road should be near the detected travelable area, in which case a higher weight is assigned. Vehicle candidates that are far away from the road area in the vertical direction usually do not belong to the ground. In the present embodiment, the horizontal direction of distance measurement is omitted. This is because the relative horizontal position does not have any identifying information in a traffic scenario. As shown in FIG. 6B, all the distances of the vehicle candidates are measured in an orthographic image obtained by three-dimensional reconstruction from the parallax map. O is the overlap rate of the bounding box of the vehicle candidate with respect to the travelable area. That is, the overlap rate O is calculated by the following equation.

O = (number of pixels in the travelable area in the bounding box) / (total number of pixels in the bounding box)

A large overlap rate corresponds to false detection. This is because the reliability of detection of the travelable area by the road detection unit 24 is very high. H is the vehicle height in the real world, which is estimated from the bounding box, ie H = h · d _c / f. However, d _c is the distance between the vehicle candidate and the camera, f is the focal length of the camera. The actual vehicle should have a predetermined height according to the vehicle type of the vehicle.

  All features give high weights to the specific location and well-known size of the vehicle being detected, so in this embodiment they are Gaussian to calculate a context score for re-weighting the vehicle candidates. Model as a distribution. In particular, in the present embodiment, the score Scon calculated by the following equation (5) is added to the detection score for the vehicle candidate, and the score is calculated again.

However, (μ, σ) is an average value and a standard deviation of features, and is obtained from actual data. i represents the number of traveling lanes. Here, in this embodiment, a case where there is a traveling lane in which the host vehicle travels and an adjacent traveling lane is considered. Therefore, in the present embodiment, two sets (μ, σ) for d _L corresponding to vehicles located in the own vehicle traveling lane and the adjacent traveling lane are obtained. Similarly, in the present embodiment, two sets (μ, σ) of H corresponding to a normal size vehicle such as a passenger car and a large vehicle such as a truck are also obtained. λ is a weight parameter for adjusting the weight of the context score. In this embodiment, λ = 0.5 is set so that a value between 0 and 2 is detected within the range of [−1, 1] according to the size and position of the detected vehicle. It will be added to the score.

  As described above, in the driving support control apparatus 10 according to the present embodiment, the driving lane recognized by the lane recognition unit 32 for each of the vehicle candidates detected by the vehicle detection unit 26 by the context correlation calculation unit 34 or Calculate the context correlation with the virtual lane.

  In the context model storage unit 36, various parameters in the equation (5) are stored as data representing the context model.

  The target vehicle detection unit 37, for each of the vehicle candidates detected by the vehicle detection unit 26 based on the context correlation calculated by the context correlation calculation unit 34 and the data representing the context model, according to the above equation (5). The score Scon is calculated, added to the detection score, the score is calculated again, and the vehicle is detected based on the calculated score. Further, the target vehicle detection unit 37 detects a vehicle by rough threshold processing for each of the vehicle candidates detected by the vehicle detection unit 26. The target vehicle detection unit 37 detects the vehicle by AND logic with respect to the detection result of the vehicle by the rough threshold process and the detection result of the vehicle based on the score calculated again.

  The virtual target generation unit 38 is a vehicle in a travel lane in which the host vehicle is traveling based on the vehicle detected by the target vehicle detection unit 37 and the travel lane or virtual travel lane recognized by the lane recognition unit 32. Is generated as a virtual target for the follow-up driving control and output to the driving support unit 18. Further, the virtual target generation unit 38 outputs the travel lane or virtual travel lane recognized by the lane recognition unit 32 to the driving support unit 18 as a virtual target for lane keeping control.

  In addition, you may obtain | require the average value and standard deviation of the characteristic regarding a spatial context not by defining in advance but by learning. In this case, a general and robust description of the spatial interaction between the vehicle and the road is provided. For example, using a data set (eg, a 750 frame data set) recorded from an experimental vehicle equipped with a synchronized pair of cameras that produces approximately 15 frames per second at a resolution of 640 × 480 pixels, the average value and Perform standard deviation parameter learning. For the data set, all vehicle instances were manually labeled as long as more than ¼ of the vehicle body was visible in the image.

<Operation of the driving support control device>
Next, the operation of the present embodiment will be described.

  When the vehicle equipped with the driving support control device 10 is running, the imaging device 14 captures the front of the host vehicle, sequentially generates a stereo image, and the GPS receiver 12 generates a signal from a GPS satellite. Are sequentially received, the driving support control device 10 executes a virtual target generation processing routine shown in FIG.

  In step 100, the stereo image produced | generated by the imaging device 14 is acquired, and in step 102, the GPS signal received by the GPS receiving part 12 is acquired, and the absolute position of the own vehicle is detected.

  In the next step 104, a vehicle candidate is detected from the left image of the stereo image acquired in step 100, and the vehicle is detected by threshold processing for the detection score of the vehicle candidate obtained in the vehicle candidate detection process. In step 106, a lane mark is detected from the left image of the stereo image acquired in step 100. In step 108, it is determined whether or not a lane mark is detected in step 106. When the lane mark is detected, in step 110, the traveling lane is recognized based on the lane mark detected in step 106.

  On the other hand, if no lane mark is detected, in step 112, based on the stereo image acquired in step 100, the absolute position of the host vehicle detected in step 102, and the vehicle detected in step 104. To generate a virtual driving lane.

  In step 114, for each vehicle candidate detected in step 104, a context correlation with the travel lane recognized in step 110 or the virtual travel lane generated in step 112 is calculated.

  In step 116, for each of the vehicle candidates detected in step 104, based on the context correlation calculated in step 114, the data representing the context model, the position and size of the vehicle candidate, (5) A score related to the context correlation is calculated according to the equation, the calculated score is added to the detection score of the vehicle candidate, and the score of the vehicle candidate is calculated again.

  In step 118, for each of the vehicle candidates detected in step 104, the result of performing rough threshold processing on the detected score and the result of performing threshold processing on the score recalculated in step 116, The vehicle is detected by AND logic.

  In step 120, a virtual target is generated based on the vehicle detected in step 118 and the driving lane recognized in step 110 or the virtual driving lane generated in step 112, and driving is performed. The data is output to the support unit 18 and the process returns to step 100.

  By repeatedly executing the virtual target generation processing routine described above, the sequentially generated virtual targets are output to the driving support unit 18, and the driving support unit 18 performs driving support based on the virtual targets.

  Step 112 is realized by the virtual lane generation processing routine shown in FIG.

  In step 130, a parallax map is calculated based on the acquired stereo image. In step 132, a cost map is generated based on the parallax map calculated in step 130. In step 134, the boundary of the travelable area is detected based on the cost map generated in step 132.

  In step 135, it is determined whether or not the initial value of the lane parameter representing the virtual traveling lane has already been set in step 136 described later. When the initial value of the lane parameter has been set, the process proceeds to step 137 described later. On the other hand, if the initial value of the lane parameter is not set, in step 136, road information at the position of the own vehicle is acquired based on the absolute position of the own vehicle detected in step 102 and the map data. Then, based on the acquired road information, an initial value of a lane parameter representing a virtual traveling lane is set.

  In step 138, the initial value of the lane parameter set in step 136 or the state vector of the particle updated last time, the boundary of the travelable area detected in step 134, and the step 104 detected. Based on the vehicle, a weight is calculated for each particle, the particle is sampled according to the calculated weight, and the state vector of each sampled particle is updated according to the state transition model.

  In step 138, a lane parameter representing a virtual traveling lane at the current position of the host vehicle is estimated based on the state vectors of the particles updated in step 137.

  In step 140, a virtual travel lane is generated based on the lane parameter estimated in step 138, and the virtual lane generation processing routine is terminated.

  As described above, according to the driving support control apparatus according to the first embodiment of the present invention, based on the parallax map obtained from the stereo image generated by imaging the front of the vehicle, the travelable region of the vehicle Based on the absolute position of the vehicle detected from the GPS reception signal and the road map data, the initial value of the lane parameter representing the traveling lane in which the vehicle is traveling is set, and the detected travelable area And the lane parameter of the virtual driving lane in which the vehicle is driving based on the initial value of the lane parameter, the virtual driving lane can be determined even if the white line is not clear or does not exist. Can be generated.

  Further, a travelable area on a road without a white line is detected based on a parallax map obtained by stereo matching with respect to a stereo image. Although it is difficult to determine the route on which the vehicle should travel only with the travelable area, a virtual travel lane can be generated by using the road shape information obtained from the navigation map and applying a curve to the travelable area. This represents the route that the host vehicle should travel.

  Furthermore, based on the positional relationship between the virtual travel lane and the vehicle, it is possible to determine whether the vehicle on the road should be followed or avoided by simple processing.

  Next, a second embodiment will be described. In addition, since the structure of the driving assistance control apparatus which concerns on 2nd Embodiment becomes a structure similar to 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

  In the second embodiment, not the travel lane in which the host vehicle is traveling, but a virtual travel lane that represents an emergency travel lane for avoiding a collision is generated. Is different.

  In the second embodiment, the lane recognition unit 32 estimates a lane level road structure in a tracking process using a particle filter for a road without a lane mark. In each estimation period, 1500 particles are generated and the weights associated with these particles are updated based on the likelihood measurement on the road energy map. The likelihood score is calculated by the following formula.

However, P _L is a region between the virtual left and right lane marks. p is _{P L} inner _{pixels, N PL} is the size of the _{P L.} ρ is a positive constant representing the energy of the road point γ, and γ is a positive constant for adjusting the energy of a specific point.

  As shown in the above equation (7), each point p is classified into either a travelable region Proad or a region Pnon-road other than the travelable region.

  Thus, the emergency lane is estimated in the same manner as the virtual lane generation except that a different energy map is used. This energy map represents a travelable region Proad and a region Pnon-road other than the travelable region.

  In the above equation (7), a very large weight parameter γ is used instead of negative infinity because the parameter is surely non-except unless it is unavoidable due to noise or the like. This is because the road points are not included in the emergency lane.

  The lane recognition unit 32 acquires road information (lane width, curvature, etc.) at the current own vehicle position from the navigation map data based on the detected own vehicle position, and uses the own vehicle position and road information, Set the initial value of the state vector. A plurality of particles are generated using the initial value of the set state vector. For each generated particle, a weight is calculated according to the above equation (6) for the virtual traveling lane represented by the state vector of the particle.

  In accordance with the calculated weight, the particles are sampled from the generated particles, and the state vectors of the sampled particles are updated according to the above equation (2). By calculating the weight of each particle and sampling the particle at each time, the lane parameters at the vehicle position at each time are estimated, and based on the estimated lane parameters at each time, a virtual emergency at each time Recognize the driving lane.

  The virtual target generation unit 38 detects the vehicle detected by the target vehicle detection unit 37 as an obstacle candidate, and generates the virtual emergency travel lane recognized by the lane recognition unit 32 as a virtual target for rear-end collision avoidance operation control. And output to the driving support unit 18.

  In addition, about the other structure and effect | action of the driving assistance control apparatus which concern on 2nd Embodiment, since it is the same as that of 1st Embodiment, description is abbreviate | omitted.

  In this way, the vehicle travelable area is detected based on the parallax map obtained from the stereo image generated by imaging the front of the vehicle, and the lane of the emergency travel lane is determined according to the cost map based on the detected travelable area. By estimating the parameters, a virtual emergency lane can be generated even when the white line is not clear or does not exist.

DESCRIPTION OF SYMBOLS 10 Driving assistance control apparatus 12 GPS receiving part 14 Imaging device 16 Computer 18 Driving assistance part 20 Position detection part 22 Lane mark detection part 24 Road detection part 26 Vehicle detection part 28 Map database 30 Road model memory | storage part 32 Lane recognition part 34 Context correlation Calculation unit 36 Context model storage unit 37 Target vehicle detection unit 38 Virtual target generation unit

Claims (6)

A vehicle travelable area is detected based on parallax information of the stereo image obtained on the basis of a stereo image that is mounted on a vehicle and that captures the periphery of the vehicle and generates a stereo image. Road detection means to
A traveling lane in which the vehicle is traveling based on the absolute position of the vehicle detected by a position detecting unit that detects the absolute position of the vehicle and road map data prepared in advance representing road information at each point Initial value setting means for setting an initial value of a lane parameter representing
Based on the travelable area detected by the road detection unit and the initial value of the lane parameter set by the initial value setting unit, the lane parameter of a virtual travel lane in which the vehicle is traveling is determined. Virtual lane generation means for generating data representing a virtual traveling lane in which the vehicle is traveling based on the estimated lane parameters,
Only including,
The road detection means detects the travelable area based on parallax information of the stereo image obtained based on the stereo image generated by the imaging means for each time,
The virtual lane generation unit generates a plurality of particles representing initial values of lane parameters set by the initial value setting unit,
For each time, based on the travelable area detected by the road detection means, each of the particles is sampled from the plurality of particles, and a lane parameter represented by each of the sampled particles is determined in advance. By repeatedly updating according to the state transition model, the lane parameter representing the virtual traveling lane is estimated at each time, and data representing the virtual traveling lane is generated based on the estimated lane parameter. Virtual lane generator. Vehicle detection means for detecting a vehicle candidate based on at least one of the stereo images generated by the imaging means;
A travel lane in which the vehicle is detected detected based on the stereo image generated by the imaging means, or the virtual travel lane generated by the virtual lane generation means, and detected by the vehicle detection means. A target vehicle detecting means for detecting a target vehicle used for driving support based on the relationship with the vehicle candidate,
The virtual lane generation device according to claim 1, further comprising: 3. The virtual lane generation device according to claim 1, wherein the virtual lane generation unit updates lane parameters represented by each of the sampled particles according to the state transition model determined in advance based on a clothoid curve. Vehicle detection means for detecting a vehicle based on at least one of the stereo images generated by the imaging means;
The virtual lane generation unit overlaps with other travel lane regions where the vehicle detected by the vehicle detection unit is traveling among the travelable regions detected by the road detection unit for each time. The particles representing the lane parameters of the traveling lane are sampled each of the particles by applying a smaller weight so as to overlap the region of the traveling lane in which the vehicle is traveling, and each of the sampled particles lane parameters representing the virtual lane generating apparatus of any one of claims 1 to 3 for updating in accordance with the state transition model.   The virtual lane generation apparatus according to claim 2, wherein the target vehicle detection unit detects the vehicle candidate existing in the virtual travel lane as a tracking target vehicle. Computer
A vehicle travelable area is detected based on parallax information of the stereo image obtained on the basis of a stereo image that is mounted on a vehicle and that captures the periphery of the vehicle and generates a stereo image. Road detection means,
A traveling lane in which the vehicle is traveling based on the absolute position of the vehicle detected by a position detecting unit that detects the absolute position of the vehicle and road map data prepared in advance representing road information at each point Based on the initial value setting means for setting the initial value of the lane parameter that represents, the travelable area detected by the road detection means, and the initial value of the lane parameter set by the initial value setting means, Virtual lane generation for estimating lane parameters of a virtual traveling lane in which the vehicle is traveling and generating data representing the virtual traveling lane in which the vehicle is traveling based on the estimated lane parameters A program for functioning as a means ,
The road detection means detects the travelable area based on parallax information of the stereo image obtained based on the stereo image generated by the imaging means for each time,
The virtual lane generation unit generates a plurality of particles representing initial values of lane parameters set by the initial value setting unit,
For each time, based on the travelable area detected by the road detection means, each of the particles is sampled from the plurality of particles, and a lane parameter represented by each of the sampled particles is determined in advance. By repeatedly updating according to the state transition model, the lane parameter representing the virtual traveling lane is estimated at each time, and data representing the virtual traveling lane is generated based on the estimated lane parameter. Program .