Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Definitions

• the present invention relates to a method for obtaining drivable road area for assisting a driver of a vehicle when said driver driving said vehicle running on said road.
• the invention also relates to a computer system and a computer program for obtaining drivable road area for assisting a driver of a vehicle when said driver driving said vehicle running on said road, suitable for carrying out such a method.
• the invention relates to a system for obtaining drivable road area for assisting a driver of a vehicle when said driver driving said vehicle running on said road.
• Vision-based road detection is an important research topic in different areas of computer vision such as autonomous driving, car collision warning and pedestrian crossing detection.
• detecting the free-road surface using a vision system is very challenging since the road is an outdoor scenario imaged from a mobile platform.
• the detection algorithm must be able to deal with continuously changing background, the presence of different objects (vehicles, pedestrian), different environments (urban, highways, off-road), different road types (shape, colour), and different imaging conditions (varying illumination, different viewpoints and weather conditions).
• the US patent application US 2009/01 18994 A1 discloses a lane recognizing device that comprises an image processing means which performs a process of estimating a lane of a road by processing an image of the road and outputs a result of the process as first lane information; a lane estimating means which performs a process of estimating the lane using a map data of the road and the current position information of a vehicle and outputs a result of the process as second lane information; and an actual lane recognizing means which recognizes an actual lane of the road on the basis of the first lane information and the second lane information.
• Said lane recognizing device uses prior road knowledge obtained from map data of the road and the current position information of the vehicle for finally estimating a left-hand line and a right-hand line.
• an image of the road is obtained (through a vehicle on-board camera) and processed to identify an existing left-hand line and a right-hand line on the road.
• said estimated left-hand line and right-hand line are compared with said identified left-hand line and right-hand line (from captured image) to recognize the actual lane of the road. That is to say, the actual lane of the road recognition is based on comparison between estimated and identified lines.
• This approach has two main drawbacks. First, the method requires the existence of specific lane markings. This is an important limitation since these markings are not always present especially in urban environments. Second, this approach is based on simple line representations, thus, error tolerance within the comparison between lines is very little. That is, the possibilities of producing an acceptable result are reduced when errors are present in any of the lane estimation processes: lane estimation from map data and vehicle situation, lane estimation from road images. Hence, this solution has low versatility and robustness.
• Another approach exploiting prior road knowledge for road detection consists in using general knowledge of the road by learning off-line some aspects of typical road images and applying this knowledge to process new images.
• This approach consists in using a scene classifier to provide the probability that a road image contains certain road geometry (left turn, straight, t-like junction). These road geometries are learned off-line using training images.
• the algorithm relies on visual information and is limited to a finite number of classes. That is, the current road shape to be detected is limited to the shapes in the training images.
• the drawback of low versatility and low robustness still remains in this offline learning solution.
• the object of the present invention is to fulfil such a need. Said object is achieved with a method for obtaining drivable road area according to claim 1 , a computer program product comprising program instructions for causing a computer to perform the method for obtaining drivable road area according to claim 16, a computer system for obtaining drivable road area according to claim 19, and a system for obtaining drivable road area according to claim 20.
• the present invention provides a method for obtaining drivable road area for assisting a driver of a vehicle, comprising:
• obtaining at least one current pose of the vehicle through a localization system obtaining a map model of the road from a repository of road maps according to the at least one current pose of the vehicle;
• a road image is obtained through some on-board image capturing device (e.g., a camera).
• This captured road image refers to the driver's view of the road.
• a bird's view representation of the road is obtained from a repository of road maps (e.g. maps server or geographic information system) according to the current pose (position and orientation) of the vehicle.
• the pose of the vehicle is obtained using a localization system (e.g., global positioning system).
• a computer vision method based on image properties such as colour or texture is applied to the road image to obtain a probabilistic map image where each pixel value represents the probability of a pixel being road.
• the road model is projected according to the pose of the vehicle.
• the bird's view of the road is transferred to the driver's view of the road.
• this projection of the road model incorporates obtaining a probabilistic map where the value of each map point represents the probability of that point being road.
• two probabilistic driver's view maps i.e., comparable
• a probabilistic analysis method e.g.
• Bayesian framework is applied to the two obtained comparable probabilistic maps for obtaining a final probabilistic map representing the drivable road area or free road surface.
• the method of the present invention increases the error tolerance because this method is based on much richer data, since the image of the road and the projected model of the road are modelled for obtaining probabilistic maps, which is much more than two simple lines defining a lane.
• said lines do not exist in all the types of roads, for example, in urban environments wherein the assumption of specific lane markings on the road is a big source of errors. Consequently, the method of the present invention has the advantage of not requiring any kind of marks on the road and improving versatility and robustness of the known methods in the state of the art.
• the driver's view is in fact the view of the image capturing device in order to be able to compare the probabilistic map coming from the road map and the pose of the vehicle to the probabilistic map coming from the image capturing device.
• the image capturing device can be positioned on the rear part of the vehicle car looking backwards. That is, this method can perform in images of the road ahead or behind the vehicle. Hence, the pose of the image capturing device respect to the car is necessary to obtain the projected road model.
• the at least one image capturing device may be two or more image capturing devices. If two or more image capturing devices are used, each device is dedicated to obtain specific data that is taken into account in the method. If more than one image is obtained simultaneously from different points in the vehicle (e.g., different cameras placed at different positions), said images can be merged to obtain more accurate data or extra information to generate the probabilistic map associated to the road image.
• a stereo acquisition system two or more image capturing devices
• This depth map is an extra image wherein each pixel contains the distance to the object in the scene.
• Examples of specific data provided by each camera are: monochromatic information (with grey and texture levels), colour information (with colour and texture), depth information (with geometric features), temperature (for MIR and FIR cameras), etc.
• different specific computer vision algorithms may be used.
• the main strength of this new approach is the introduction of road priors estimated on-line to improve the performance of vision-based algorithms, through the combination of road priors and vision-based results using probabilistic analysis.
• This probabilistic analysis yields a simple formulation of prior influence on road detection.
• the probabilistic algorithm exploits the inherent diversity of road priors and low-level based algorithms. The former provides a rough detection of the road despite acquisition conditions whereas the latter provides the required accuracy.
• Either road likelihood and road priors are built on-line to improve the adaptability of the algorithm to current lighting conditions and the presence of other vehicles in the scene.
• obtaining drivable road area by applying the probabilistic analysis method to the at least one projected model of the road and the at least one probabilistic map of the image of the road comprises: obtaining an averaged projected model of the road by averaging the at least one projected model of the road;
• obtaining the at least one current pose of the vehicle through a localization system comprises:
• obtaining a localization pose of the vehicle through the localization system obtaining the at least one current pose of the vehicle by applying a random sampling based method to the localization pose of the vehicle.
• Random sampling based methods refer to iteratively evaluating a deterministic model using sets of random numbers as inputs. In this way, a set of values within a range are randomly generated from probabilistic distributions to simulate the process of sampling the complete range. Finally, different probabilistic road maps are obtained from said simulation and are averaged to obtain the road prior (i.e., road probability map) for the corresponding image.
• obtaining a projected model of the road by projecting the map model of the road according to the current pose of the vehicle and to the predetermined device parameters also comprises applying the random sampling based method to the pose of the image capturing device, said pose being comprised in the predetermined device parameters.
• different possible poses of the image capturing device are simulated in order to use the uncertainty in the pose of the image capturing device to finally infer the projected model of the road. Therefore, different probabilistic projected models of the road are obtained from said simulation and are averaged to obtain the projected model of the road.
• This approach of modelling the uncertainty increases the error tolerance of the overall method, since no hard restrictions are required by the method. No particular marks on the road are required and it is assumed that the pose of the vehicle provided by the localization system may comprise some error.
• the method tries to identify points in the image that "seem" to belong to the road and assuming that the real pose of the vehicle is around (not exactly) the pose obtained through the localization system. That is to say, it is assumed that data elements used in the method contain some errors, which are probabilistically modelled and allowed by the method.
• Some examples of data elements used in the method are: probabilistic map of the image of the road wherein each point of the image is labelled as belonging to the road or not, pose of the vehicle from the localization system, map from the repository of maps, etc.
• the localization pose of the vehicle comprises a localization position of the vehicle and a localization orientation of the vehicle; and obtaining the localization pose of the vehicle through the localization system comprises:
• the method of the invention may obtain the position and the orientation of the vehicle directly from a localization system able to provide both parameters.
• the localization pose of the vehicle comprises a localization position of the vehicle and a localization orientation of the vehicle; and obtaining the localization pose of the vehicle through the localization system comprises:
• the method of the invention can obtain the position and orientation of the vehicle even though the localization system is only able to provide the position, because the method is able to obtain the orientation from consecutive positions provided by the localization system.
• the orientation may be inferred, for example, by joining with a straight line at least two consecutive positions of the vehicle.
• the localization pose of the vehicle comprises a localization position of the vehicle and a localization orientation of the vehicle; and obtaining the localization pose of the vehicle through the localization system comprises: obtaining the localization position of the vehicle through the localization system; verifying if the localization system is able to provide the localization orientation of the vehicle;
• the method is able to determine if the localization system is able to provide position and orientation, in which case, both parameters are directly obtained from the localization device.
• the method can also determine if the localization device is only able to obtain position, in such a case, said parameter is directly obtained from the localization system and the orientation is inferred from consecutive points provided by the localization system.
• obtaining the map model of the road from the repository of road maps according to the at least one current pose of the vehicle comprises:
• the method also takes into consideration if the repository of road maps is able to provide complex data representing the geometry of the road, in which case, the map model of the road can be directly obtained from said geometry of the road.
• obtaining the map model of the road from the repository of road maps according to the at least one current pose of the vehicle comprises:
• the method also takes into consideration if the repository of road maps is only able to provide simple data representing points and type of the road, in which case, the map model of the road is obtained by inferring an skeleton of the road from the points produced by the maps repository and modelling said skeleton according to a standard model of the road related to its type.
• this standard model may comprise the width of the lane and the width of roadsides depending on the road being highway, primary, secondary or residential, etc.
• obtaining the map model of the road from the repository of road maps according to the at least one current pose of the vehicle comprises:
• the method is able to determine if the repository of maps is able to provide geometry of the road, in which case, the map model of the road is directly obtained from said geometry.
• the method can also determine if the repository of road maps is only able to obtain points and type of the road, in such a case, the map model of the road is obtained by inferring an skeleton of the road from the points produced by the maps repository and modelling said skeleton according to a standard model of the road related to its type.
• this standard model may comprise the width of the lane and the width of roadsides depending on the road being highway, primary, secondary or residential.
• each current pose of the vehicle comprises a current position of the vehicle and a current orientation of the vehicle; and obtaining the at least one current pose of the vehicle by applying the random sampling based method to the localization pose of the vehicle comprises:
• the current pose of the vehicle is defined in terms of position and orientation of the vehicle.
• the random sampling based method is applied to both parameters (position and orientation) of the vehicle previously obtained through the localization system. That is, the uncertainty is modelled in terms of the position and orientation of the vehicle by generating a "cloud" of possible positions and orientations around the position and orientation provided by the localization system.
• the method further comprises storing relevant data of the obtained drivable road area in a repository for off-line error analysis and correction.
• the method further comprises sending, through a communications network, relevant data of the obtained drivable road area to a server for on-line error analysis and correction.
• Relevant data of the obtained drivable road area may be the final binary map representing the drivable road area and/or discrepancies between the probabilistic map coming from the image of the road and the probabilistic map coming from the projected map of the road.
• Said final binary map can be compared with all the maps comprised in the "cloud" of projected models of the road, so that the projected model having lowest discrepancy with the final binary map can be considered the best model (corresponding to the best parameters) and the reference for inferring and correcting errors.
• the method of the invention may also comprise reducing the uncertainty in the next iteration of the method by using the best parameters related to the considered best model.
• the main advantage of this feature is acceleration of the process of drivable road area generation without decreasing the accuracy of the method.
• the random sampling based method comprises a Monte Carlo method.
• An alternative to Monte Carlo may be a full scanning of the possibilities derived from the variables producing uncertainty for obtaining an uniform distribution of the possible cases, but this approach is very time consuming.
• Another alternative is obtaining an analytic expression of the problem according to the variables producing uncertainty and obtaining the probability formula to be applied.
• Monte Carlo methods are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used in simulating physical and mathematical systems. Because of their reliance on repeated computation of random or pseudo-random numbers, these methods are most suited to calculation by a computer and tend to be used when it is unfeasible or impossible to compute an exact result with a deterministic algorithm.
• the probabilistic analysis method comprises a Bayesian Framework.
• the method of the invention is based on an approach consisting in combining the on-line priors and any low-level based algorithm using a Bayesian framework.
• the result is a pixel-wise confidence map depicting the probability of an image pixel being a road pixel.
• a classifier assigns a road or background label to each pixel depending on its probability value.
• An alternative to the Bayesian Framework may be a simple intersection of probability masks. Both the Bayesian Framework and/or the intersection of probability masks may be complemented by adding extra sources of data, for example an accelerometer, said extra sources not being directly joined to provide road probability.
• the computer vision method comprises computer vision algorithms based on at least one of among the following approaches: colour analysis, texture analysis, distribution analysis, neighbourhood analysis, monochromatic analysis (with grey and texture levels), depth analysis (with geometric features), temperature analysis (for MIR and FIR cameras), etc.
• the present invention relates to a computer program product comprising program instructions for causing a computer to perform the method for obtaining drivable road area for assisting a driver of a vehicle.
• the invention also relates to such a computer program product embodied on a storage medium (for example, a CD-ROM, a DVD, a USB drive, on a computer memory or on a readonly memory) or carried on a carrier signal (for example, on an electrical or optical carrier signal).
• a storage medium for example, a CD-ROM, a DVD, a USB drive, on a computer memory or on a readonly memory
• a carrier signal for example, on an electrical or optical carrier signal
• a computer system for obtaining drivable road area for assisting a driver comprising:
• the present invention relates to a system for obtaining drivable road area for assisting a driver, comprising:
• At least one vehicle on-board image capturing device for providing at least one image of the road to the computer system for obtaining drivable road area;
• a localization system for providing at least one current pose of the vehicle to the computer system for obtaining drivable road area;
• a repository of road maps for providing road map data to the computer system for obtaining drivable road area, said road map data being for obtaining a map model of the road according to the at least one current pose of the vehicle.
• the image capturing device is a camera.
• the camera may be a matrix camera, for example: CMOS or CCD camera depending on the type of sensors; B/W (grey level), colour camera, or a mixed model (some intensity pixels and some colour pixels) depending on the intensity or colour information; linear or non-linear depending on the dynamic range.
• the camera can also be a time-of-flight camera (TOF), which is a camera that illuminate the scene with an active illumination and are able to measure the distance to each point of the scene apart from acquiring an intensity image. Or an stereo camera that obtains similar information (intensity/colour and depth) by means of two or more cameras.
• TOF time-of-flight camera
• the camera may be a camera capturing information out of the visible spectrum (infrared, ultraviolet).
• visible spectrum infrared, ultraviolet
• NIR Near Infrared
• MIR Medium
• FIR FIR
• TIR TIR
• the camera may comprise some devices able to generate depth maps similar to previous examples of stereo and TOF camera, with less resolution but with higher accuracy, as radar and lidar sensors.
• the localization system is a Global Positioning System.
• the Global Positioning System is an Inertial Global Positioning System.
• GPS Global Positioning System
• the related method for obtaining drivable road area may comprise obtaining only position from the GPS, or obtaining position and orientation from the GPS, or even on-line detection of the type of GPS and obtaining only position or obtaining position and orientation depending on the type of GPS.
• DGPS Differential GPS
• RTKGPS Real Time Kinetic GPS
• GPS Global System for Mobile Communications
• Some of said sensors that may be combined with a GPS are: compass (electronic), accelerometer, gyroscope, tachometer (the own tachometer of the vehicle in charge of measuring the speed could be used), etc.
• Any current vehicle normally comprises a plurality of accelerometers in its configuration, so said accelerometers could be adapted to combine their measurements with the GPS information for achieving the mentioned goals.
• the repository of road maps is a maps server.
• the repository of road maps is a Geographic Information System.
• the maps repository may be installed on the vehicle, in which case no access to a communications network is needed. That is to say, the maps may be accessible through Internet, in which case it is possible to connect and download the required maps in each moment.
• the maps may be accessible through Internet, in which case it is possible to connect and download the required maps in each moment.
• Other alternatives may be based on accessing to specific traffic networks instead of accessing to public networks (e.g. Internet).
• traffic networks the information could be obtained from elements of the road (e.g. traffic signs, sidelights, traffic infrastructures in general) or even from the closer vehicles on the road.
• Figure 1 is a graphical representation of the main system elements and data elements taking part in obtaining projected models of the road, according to an embodiment of the invention
• Figure 2 is a graphical representation of a bird's view of a road map with superimposed points defining the road skeleton obtained smoothing the road trajectory using non-linear interpolation, according to an embodiment of the invention
• Figure 3 is a graphical representation of a map model of the road according to an embodiment of the invention.
• Figure 4 is a graphical representation of the four coordinate systems and relations between them for projecting the map model of the road onto the image plane of the camera, according to an embodiment of the invention
• Figure 5 is a graphical representation of a road confidence map (projected model of the road) obtained modelling the uncertainty in camera pose and car position using a Monte Carlo approach, according to an embodiment of the invention
• Figure 6 shows some results of projecting the map model of the road according to the current pose of the vehicle and to the predetermined device parameters, according to an embodiment of the invention
• Figure 7 shows some examples of on-line prior results being obtained using a system and applying a method according to an embodiment of the invention.
• Figure 1 shows a preferred embodiment of the system for obtaining drivable road area of the invention, said system comprising a computer system (1 12) for obtaining drivable road area; at least one vehicle onboard camera (103) for providing at least one image (104) of the road to the computer system (1 12) for obtaining drivable road area; a localization system (102) for providing at least one current pose (101 , 107) of the vehicle to the computer system (1 12) for obtaining drivable road area; a repository of road maps (1 12) for providing road map data (1 13, 108, 109) to the computer system (1 12) for obtaining drivable road area, said road map data (1 13, 108, 109) being for obtaining a map model of the road (1 10, 1 1 1 ) according to the at least one current pose (101 , 107) of the vehicle.
• the projected model of the road (105) or road prior is a probability map comprising a bi-dimensional array where for each cell there is the probability of the corresponding input image pixel depicting road surface. It is also preferred the projected model of the road (105) being estimated online using information available in Geographical Information Systems (GIS) acting as the repository of road maps (1 12).
• GIS Geographical Information Systems
• the system comprises means (1 12) for connecting to Internet and accessing to the GIS.
• GIS are database systems that capture, store and manage geographically referenced information. That is, data which is linked to physical locations. This information describes the world in geographic terms representing objects like rivers, lakes or roads) using simple geometries such as points, poly-lines or polygons. The proper combination of these geometries creates a bird's view map (1 13) of a specific region of the Earth.
• road information is usually associated with additional attributes such as road name, road type, construction level, vehicle driving direction and number of lanes in each direction.
• the computer system (1 12) is able to store and execute a computer program comprising instructions to perform a method for obtaining drivable road area.
• the Figure 1 also shows the data elements taking part in said method and the relations between said data elements.
• the method starts obtaining localization information (101 ) provided by a GPS antenna (102).
• a map model (1 10, 1 1 1 ) of the road is obtained from the GIS according to the localization information (101 ) defining the position of the vehicle.
• at least one image (104) of the road is obtained through the on-board camera (103), said camera having predetermined parameters associated (pose of the camera on the vehicle, focal length, resolution, etc.).
• a "cloud” of possible positions (107) of the vehicle is obtained by applying a Monte Carlo approach to the position of the vehicle.
• the "cloud” of possible positions (107) also takes into consideration diverse poses of the onboard camera (103) in order to model the uncertainty in the pose of the camera (including pose of the vehicle).
• each map model (1 10, 1 1 1 ) of the road is projected according to the associated current pose (possible pose) of the vehicle and the predetermined parameters of the camera (including possible pose of the camera), said projection producing a projected model (105, 106) of the road for each map model (1 10, 1 1 1 ) of the road.
• projecting the road shape at the current vehicle position onto the 2D driver's view i.e., the image plane of the on- board camera.
• Each one of the obtained projected models (105, 106) of the road comprises a bi- dimensional array where for each cell of the projected model (105, 106) there is the probability of the corresponding input image pixel depicting road surface.
• the diverse projected models (105, 106) of the road are averaged to obtain an averaged projected model (105) of the road.
• a computer vision method is applied to each of the obtained images (104) of the road to obtain a probabilistic map of the image (104) of the road associated to said image (104) of the road.
• Each one of the probabilistic maps of the image (104) of the road comprises a bi-dimensional array where for each cell of the image of the road (104) there is the probability of the corresponding input image pixel depicting road surface.
• the averaged projected model (105) of the road and the probabilistic map of the image (104) of the road are inputted to a Bayesian Framework, wherein said two inputs are probabilistically compared (or analyzed) for producing the drivable road area for assisting the driver of the vehicle.
• the localization information (101 ) provided by the GPS antenna (102) only comprises data defining the position of the vehicle, so the orientation is inferred from a set of consecutive positions (or localization positions) of the vehicle.
• a skeleton (21 ) of the road is obtained, said skeleton consisting in a set of junctions (points defining the consecutive positions of the vehicle) (22) connected by piecewise continuous lines (24).
• These line segments (24) provide a rough description of the road trajectory.
• non-linear interpolation (23) is applied to improve the accuracy of the road trajectory, finally obtaining a realistic shape of the road by modelling the road trajectory (25).
• the map model of the road is obtained from the skeleton (31 ) of the road according to a predetermined road model related to the type of the road, said type of the road being comprised in the localization information (101 ) provided by the GPS antenna (102).
• a road model ( Figure 3) consisting of a drivable area (33) and two roadsides (32) is used.
• the width of these parts, road lane W RL (34) and roadside W RS (35) may be estimated according to the road attributes comprised in the road map data (1 13, 108, 109) and additional country's national road legislation according to for example Table 1 .
• the Figure 4 shows four different coordinate systems involved in the projection of the map models of the road: World Coordinate System (WCS), Vehicle Coordinate System (VCS), Camera Coordinate System (CCS) and Image Coordinate System (ICS).
• the projection of the map model of the road consists in transferring the map model of the road from VCS to the ICS, using WCS as reference. This process can be decomposed in a rigid body translation (t) and rotation (roll, pitch and yaw) between VCS and CCS, and a perspective projection from CCS to ICS.
• a set of N points p [po, p,,...,p/v] defining the road shape (map model of the road) is mapped (projected) from WCS to ICS as follows,
• K is a matrix characterizing the optical, geometric and digital characteristics of the camera. It is used to link the pixel coordinates of an image point with the corresponding coordinates in the camera reference frame (i.e., image plane of the camera). These parameters are fixed for the particular camera being used and can be estimated through calibration.
• R roll, pitch and yaw
• t define the orientation (pose) and location of the camera reference frame with respect to the known vehicle position (VCS).
• VCS vehicle position
• Figure 6 shows some results of projecting the map model of the road according to the current pose of the vehicle and to the predetermined device parameters, according to an embodiment of the invention.
• the road shape information is properly recovered for each frame at different daytime and in different situations such as soft and hard turns.
• the column 61 comprises original images from the on-board camera
• the column 62 comprises bird's view of the road map and vehicle location
• column 63 comprises projected models of the road.
• Figure 5 is a graphical representation of a road confidence map (projected model of the road) obtained modelling the uncertainty in camera pose and car position using a Monte Carlo approach, according to an embodiment of the invention.
• Several possible camera poses and car positions (52) are simulated according to the map model (51 ) of the road for obtaining projected models (53, 54, 55) of the road.
• road detection consists in combining the on-line priors and any low-level based algorithm using a Bayesian framework (for probabilistic analysis).
• the result is a pixel-wise confidence map depicting the probability of an image pixel being a road pixel.
• a classifier assigns a road or background label to each pixel depending on its probability value.
• the problem of road detection is formulated as a road likelihood function as follows: where P(road ⁇ 0) is a probabilistic description of the current estimate of the road (i.e., road probability map) given a set of observations O. These observations are low-level features such as pixel intensity values or colour distributions. ⁇ o(0 ⁇ road) is the road likelihood function for the same set of observations. That is, a probability density function of a pixel being road with respect to that measurement set. Finally, P(road) is the probability of the /-th pixel in the image to belong to the road surface. That is, the road prior.
• a classifier assigns a road or background label to each pixel according to its probability of being road.
• the label assignment is performed by direct thresholding the road probability map using a fixed threshold ⁇ .
• ⁇ a threshold
• a pixel belongs to the road class if P(road ⁇ 0) ⁇ X. Otherwise, the pixel belongs to the background class.
• the remaining is estimating the road likelihood function £o(0 ⁇ road).
• the likelihood is computed online for every image using small regions on the bottom part of each frame.
• the algorithm is highly adaptive and can cope with sudden changes (i.e., lighting variations and shadows).
• a normalized histogram is used as an empirical form of probability distribution for a random variable.
• Quantitative evaluations are provided using pixel-based measures, from which the following error measures are computed: quality 9, detection accuracy DA, detection rate DR and effectiveness F (Table 2). Each of these measures provides a different insight in the performance of a method. Quality takes into account the completeness of the extracted data as well as its correctness. Detection accuracy, also known as precision, is the probability that the result is valid. Detection rate, or recall, is the probability that the ground-truth data is detected. Effectiveness is a single measure that trad es-off the detection accuracy versus detection rate.
• Each acquired image is geo-referenced (synchronized) with GPS information (latitude, longitude and altitude) provided by a standard GPS antenna (Woxter Slim II) using NMEA protocol.
• This protocol defines a set of self-contained sentences to interface between different electronic equipment. For instance, a GGA sentence from GPS devices provide the 3D location of the device. Each of these locations (latitude and longitude) is converted to Cartesian coordinates (X, Y, Z) using the datum 84 (WGS-84). This geodesic datum is a reference used to describe the localization of points on the Earth's surface.
• the road database (repository of road maps) is obtained from OpenStreetMap, which is an opensource database containing geographical information and road attributes in XML format. These attributes comprise features such as type (motorway, path, trunk, primary road, secondary road), name, maximum speed or one or two ways among others.
• the resolution of the road trajectory is improved interpolating consecutive road segments using cubic interpolation.
• the road model is parameterized using equivalences in Table 1 (previously shown) for obtaining map models of the road.
• the on-line road prior is estimated using the previously described algorithm based on projecting the map models of the road according to the pose of the vehicle and camera. Example road prior results for different road geometries and different lighting conditions are shown in Fig.
• row 71 comprises road images from the on-board camera
• row 72 comprises projected models of the road (road prior)
• row 73 comprises superimposition of obtained road prior onto road images from camera. As shown, the road layout is properly recovered despite lighting and weather conditions since the prior does not rely on current observations.
• the improvement in performance when road prior is taken into account is evaluated using different measurement sets.
• the Hue-Saturation and Intensity (HSI) colour space and a physics-based illuminant invariant colour space are considered.
• the former is a transformed colour space which is, to a certain degree, sensitive to lighting variations such as shadows and highlights.
• This colour space has been widely used for road detection and to process generic outdoor scenes under varying illumination.
• the latter refers to the feature space introduced by Finalyson.
• This is a gray-level feature space derived from the Lambertian reflection model to minimize the influence of shadows and lighting variations under Planckian light source assumption.
• This illuminant-invariant colour space has been successfully used for road detection and in surveillance applications.
• road likelihoods in the probabilistic map of the image of the road
• on-line prior that is, assuming that all the pixels in the image have the same probability of being road pixels.
• Table 3 A summary of the results is listed in Table 3 for urban scenarios and in Table 4 for highways.
• a higher performance is achieved when low-level properties include the road prior independently of the measurement set being used.
• !J exhibits the higher performance in both scenarios. This is an expected result since images in the database contain strong shadows. From these results it can be concluded that considering on-line priors improves the performance of current colour based algorithms enabling reliable road segmentation despite lighting conditions and road shape.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Traffic Control Systems (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (1)

Publications (1)

Family

ID=43490099

Family Applications (1)

Country Status (2)

Families Citing this family (15)

Family Cites Families (1)

• 2010
  • 2010-06-21 EP EP10727715.4A patent/EP2583217A1/de not_active Withdrawn
  • 2010-06-21 WO PCT/EP2010/058733 patent/WO2011160672A1/en active Application Filing

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events