Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Definitions

• the present invention relates to a method and an on-board device for driving assistance with generation of the outline of an exceeded track.
• a driving history map is constructed from measurements of the roughness of the road along the journeys made, this characteristic representing the quality of the road then serving to anticipate the choice of the profile of the road. suspension to be applied to the suspension system.
• One of the aims of the invention is to remedy at least part of the
• the invention provides an on-board method of assisting in driving a vehicle comprising:
• the detected lane contour makes it possible to constitute the route of the road markings exceeded without having a rear camera or any other rear sensor detecting lane contours.
• the track contour is a
• the representation of said contour is a spiral, or a polynomial of order 3, or a series of polynomials of order 3 or a spline of spirals.
• the projection step comprises at least one iterative sub-step of searching for the point of projection of the reference point of the vehicle on the representation, in particular by using a Newton optimization method. -Raphson, which optimizes the projection time.
• the projection step comprises a sub-step of initializing the at least one iterative search sub-step from a first identified point of the representation of said contour.
• the representation of said contour is a spiral of
• the projection is orthogonal, within a tolerance of approximately 5 °, which is easy to identify trigonometrically.
• the plotting step comprises a sub-step for calculating intermediate points, the discretization step of which is a function of the speed of the vehicle, which allows an optimized plot without a point with low added value.
• an output of the tracing step of said exceeded contour is a point cloud, which subsequently allows storage and simple memory management.
• the step of generating a representation of said detected contour comprises a sub-step of recalculating the length of the representation of said contour.
• the advantage of including a step of storing at least part of the trace of said exceeded contour allows display of the past of the journey.
• the projection step comprises a sub-step of validation of said reference point of the vehicle projected on the representation, which makes it possible to robustify the projection.
• the plotting step comprises a sub-step aimed at geometrically ordering said intermediate points, which makes it possible to keep in memory only the most relevant points.
• the method comprises a sub-step aimed at evaluating an error associated with the projected point by calculation of the variance and error propagation, which makes it possible to robustify the method.
• the reference point of the vehicle corresponds to the middle of the segment constituted by a rear axle of the vehicle, which allows better represent the vehicle, in particular when changing lanes, to avoid false lane change detections.
• the method comprises a step of displaying on a screen the trace of said exceeded contour
• the method comprises a step of associating an object with a path delimited at least partially by said trace of said exceeded contour, which makes it possible to display the situation of the objects detected at the rear. of the vehicle in relation to the plotted lanes displayed.
• the invention also relates to an on-board driving assistance device comprising:
• a means of detecting a lane contour located in front of a reference point of the vehicle in particular a sensor, and more particularly a front camera,
• this device has advantages similar to those of the method.
• the invention also relates to a motor vehicle which includes such a device, said vehicle being in particular devoid of contour detection means located behind said reference point.
• the invention also relates to a computer program comprising instructions for implementing the steps of such a method when said program is executed on a computer, the advantages being the same as those of the method.
• Figure 1a illustrates the principle of the invention at the time of
• FIG.1b Figure 1b illustrates the principle of the invention at the next instant taking place after a movement.
• Figure 2 shows a functional flowchart of the method according to the invention.
• FIG.3 shows a block diagram of the method according to the invention.
• Figure 4 shows a flowchart of the projection step of the
• Figure 5 shows an illustration of the plotting step of the method according to the invention.
• Figure 6a shows an illustration of a sub-step of the plotting step of the method according to the invention for a first use case.
• Figure 6b shows an illustration of a substep of the plotting step of the method according to the invention for a second use case.
• Figure 7 shows a schematic view of a display using the method according to the invention.
• Figure 8 shows a schematic view of a use case of driving assistance using the method according to the invention.
• XYZ direct orthonormal reference XYZ conventionally used in automotive design
• X denotes the longitudinal direction of the vehicle, directed towards the front
• Y is the transverse direction to the vehicle, directed to the left
• Z is the vertical direction directed upwards.
• the angle Q being the vehicle's yaw angle, by convention positive when it turns to the left and negative when it turns to the right.
• the concepts "front” and “rear” are indicated with reference to the normal direction of travel towards the front of the vehicle.
• substantially means that a slight deviation can be allowed with respect to a determined nominal position or orientation, for example “substantially perpendicular” means that a deviation of the order of 5 ° per in relation to a strictly perpendicular orientation is allowed within the framework of the invention.
• substantially perpendicular means that a deviation of the order of 5 ° per in relation to a strictly perpendicular orientation is allowed within the framework of the invention.
• the invention aims to optimize the use of data from vision sensors, ie sensors such as cameras, lidar or radar. More particularly, the invention, on the basis of the information on the contours of lanes obtained from these sensors, derives therefrom from the plots of the contours of exceeded lanes.
• the lane contours can be markings on the ground, in particular detected by cameras, or else lane edges detected by radars.
• the invention aims to obtain from this information a representation of the lane contours, that is to say in particular the markings on the ground or the edges of the road, overtaken and left behind the ego vehicle in order to improve the assistance services. conduct.
• the information of the contours of the past lane will be used for driving assistance methods, in particular via the association of objects located at the rear of the vehicle ego with one lane, which makes it possible to apply to said objects rules of conduct and not only kinematic rules.
• These objects are for example detected by a side and / or rear radar and / or 360 ° lidar located on the roof of the vehicle.
• a coherent association of the objects with the lanes will make it possible to predict their potential behavior (changing lanes, keeping in the lane, etc.) and then forecasting more efficiently. the trajectory and probable collisions of the ego vehicle with the various surrounding objects.
• the past of the lane contours is also useful when the vehicle is backing up, especially if the vehicle does not have a rear camera to provide information on the lines located at the rear.
• the past of the lane contours may allow interpolation for a predetermined maximum distance.
• Figure 1a illustrates the starting point, preferably identified by the time of starting the vehicle ego VEH, the moment from which the detection of the lane contour in front of the vehicle ego VEH begins. More precisely, it involves the detection of a lane contour situated in front of a reference point O of the vehicle ego to which the geometric coordinates (0,0) are assigned in the reference point of the vehicle ego VEH.
• the reference point O associated with the vehicle ego is preferably chosen as being the middle of the segment formed by the rear axle of the vehicle ego VEH and is represented in FIG. 1a by a non-solid cross. This reference point O thus chosen is more stable with respect to the calculations of rotations and vibrations of the vehicle and avoids, with respect to a reference point which would be chosen at the front of the vehicle, false detections linked to changing lanes in particular. .
• the sensor preferably a camera located at the front of the ego VEH vehicle and oriented towards the front, detects lane contours via the markings on the ground and in particular in this example the line located to the right of the ego VEH vehicle.
• Raw data from a sensor can be in the form of points or third degree polynomials and in the case of using multiple sensors their data is merged when they match the same item, the track contour data is then transformed to generate a spiral C describing the detected line shape.
• the spiral also called the Cornu spiral or Euler’s spiral in English, is a curve of affine space of dimension two. It is known to use spirals in the road field since the geometric design of roads itself uses these plan profiles. Road layouts are therefore commonly designed and defined by spirals. Spirals allow you to represent straight lines, turns to the right, to the left and to make them follow each other easily.
• a spiral is defined by three elements:
• the spiral C is described by:
• FIG. 1b illustrates a time step after a first displacement of the vehicle ego VEH, after the plotting step (T).
• the ego vehicle (VEH) has therefore advanced along the initial spiral C, part of this initial spiral C has been passed, this part is represented by dotted lines between the point Mo of origin of the initial spiral C and the projection orthogonal of the reference point O of the vehicle ego on the initial spiral C.
• This projected point M ' represented by a solid cross in figure 1b, has for coordinates (x'o, y'o) and divides the spiral into a length The not yet exceeded by the point of reference O of the initial spiral C and an exceeded length L-L '
• the updated spiral C' is therefore of length L 'not yet exceeded and the protruded part of the initial spiral C is represented by dotted lines.
• a straight line path being shown, the yaw angle was not mentioned as it remained zero.
• Figure 2 illustrates the steps of the method according to the invention.
• the method begins with a step D of detecting the track contour (marking on the ground, such as a line, or track edge), the track contour detection data corresponding for example to raw data such as as third degree polynomials, segments and / or points associated with a marking on the ground and coming from a sensor, such as a camera.
• this step D of detection by the sensors takes place continuously at an execution frequency specific to each sensor.
• Each polynomial associated with the contour of a track can be associated with fields of characteristics via a data structure which describes it by associating it for example with the quality of the signal received, the nature (continuous, discontinuous) of the marking on the ground detected if the track outline corresponds to a marking on the ground.
• Step R of receiving raw track contour data from the sensors is performed with a determined time step, in particular greater than or equal to the response time of the slowest sensor, that is to say to the duration necessary for the slowest time sensor to realize and deliver the result of its detection, for example a few milliseconds.
• This reception step R preferably comprises a sub-step (not shown) of chronological ordering of the data received from the various sensors for each track contour and a sub-step of merging these data and associating an identifier with each contour detected.
• the format of the data output from this receiving step R remains the same as the input format for the raw data.
• a step G of generating a representation of the input data more particularly a step G of generating or updating a
• each track contour detected, and preferably identified by its identifier is therefore transmitted preferentially in the form of a polynomial and then transformed in step G into a geometric shape representing the marking on the ground detected, preferably in a spiral C or in a spline of spirals, or for example in a polynomial of order 3 or in a sequence of polynomials of order 3.
• each track contour is represented by a single spiral.
• a spiral spline i.e. a curve that includes several spirals connected in series. This modeling approach allows a better representation for elongated track contours.
• an entire spiral may correspond to that segment.
• the invention is described below with reference to the case of a single spiral, which is more fundamental since a spiral spline is formed only by spiral spirals, and searching for the past from a spiral spline amounts to find the past tense of a track contour represented by a single spiral.
• the invention can also be applied to track contours modeled by polynomials by replacing the equations related specifically to the spiral model with the reciprocal equations for the polynomial model.
• each spiral C associated with an identified track contour polynomial keeps the identifier of said detected contour as well as its characteristic fields.
• This spiral generation step G also comprises, when it comes to updating a spiral C, a sub-step (not shown) of compensation for the displacement made by the vehicle ego VEH and determined at the displacement definition step D making it possible to update the spiral C.
• the characteristics M 0 (x 0 , y 0 ), ip 0 , L of the spiral C are updated by taking into account the prediction of the displacement of the vehicle ego during the calculation step: from dx along the x axis, dy along the y axis and d0 for the yaw angle by application of the rules of trigonometry, xo becoming (xo-dx). cos (0) + (yo-dy).
• an intermediate sub-step (not shown) of aggregation of the input data from the camera and / or other sensors and / or with the points in memory is added between the sub-step (not shown) displacement compensation and projection step P, this aggregation sub-step provides for choosing the values having the lowest error rate and if necessary, for example in the event of momentary failure of the sensors or 'invalidity of the values received by the sensors (for example when entering a tunnel when the lights are not yet lit) to choose the spiral in memory by substituting it for the faulty or invalid data of the sensor (s).
• the point projection can be slaved to the speed of the vehicle ego VEH so as to have less frequent sampling at high speed.
• a projection every meter at 90 km / h is of no descriptive interest and will fill the memory unnecessarily.
• sampling associated with the projection step P is therefore preferably a function of the speed so as not to calculate unnecessary points.
• step P If the projection in step P could not be achieved (for example: the vehicle has not moved, the vehicle has advanced too far and the entire representation of the lane contour is exceeded without continuity with the present , change of trajectory related to an unknown track contour for the moment ...), then no addition cannot be added to this representation of the track contour, a new representation of the track contour, preferably with a new identifier, will be started at the next iteration.
• a step T of generating a plot of the representation of the track contour exceeded as a function of the stored points Preferably, the plot is a cloud of points, comprising the stored projected points.
• the first calculation step including a movement of the ego VEH vehicle
• a step M of storing and updating the memory then takes place either consecutively or in parallel with the tracing step T.
• this first new projected point is recorded in the memory of the computer that hosts the process, for example the ECU-ADAS (from English Electronic Control Unit - Advanced Driver Assistance System) driving assistance computer.
• lane contours for example the right lane limit and the left lane limit of the ego VEH vehicle, are detected.
• step T completed in step T and the memory in step M, knowing that the memory is for example limited to 50 projection points with FIFO (acronym for First in First Out) first-in-first-out logic.
• FIFO first-in-first-out logic
• each loop lasts a predetermined maximum duration, for example 40ms. and that the plot is generated or updated at 40ms.
• This execution step allows the process to be able to interact with other driving assistance processes and in particular to provide the layout of the lane contour representations to other driving assistance services.
• FIG. 3 represents an overall flowchart of use of the method and more specifically of communication between the output data of the method. described above and another method of driving assistance, in particular of representation on a man-machine interface with association of the objects detected at the rear of the vehicle with the lanes traced in the tracing step T.
• step E-0 of data acquisition by at least one sensor more particularly of a camera which films towards the front of the vehicle.
• the data received is used in step D of detecting the track contour
• Steps D to M of the method as described above are carried out and symbolized by the dotted arrow between step D and step M for storing.
• the data from the sensors (camera, radar, etc.) received in step E-0 may contain data associated with objects, and give rise to a step E-1 of object detection.
• the sensors used for object detection may not be the same as those used for lane contour detection and may even be located elsewhere on the ego vehicle.
• step T a plot of the exceeded ground markings makes it possible, in combination with the detection of objects, to proceed to a step E-2 of object-channel association for each object detected behind the point reference of the ego vehicle and belonging to one of the lanes delimited by the traced exceeded lane contour representations.
• FIG. 4 represents a flowchart of step P of projecting the
• This projection step P is relatively long and limited in time by a stopwatch which exits the step after a duration of 7 milliseconds for example.
• this projection step P comprises an iterative step of
• the search for the projection point then consists, either of the real length I: I e R, the spiral C (l) e R 2 , in searching for L * e R such that C (L *) or the orthogonal projection of the reference point O associated with the ego VEH vehicle on the representation, i.e. here on the spiral C.
• the search sub-step P_ff ’therefore corresponds to the bricks of the
• the chosen exit condition can be replaced by another, by
• step P_ds in which we calculate a displacement ds on the curve of the spiral C according to the Newton-Raphson method:
• the starting point of the spiral is updated, for this the spiral C is evaluated at the length s: C (s) according to the Math equation. 4, that is: (x (s), y (s), Y (e), c (s), ci) and if this point is different from the previous start of the spiral (x 0 , y 0 , (/ 0 , c 0 , ⁇ 3 ⁇ 4) it will replace it as a new start
• next validation sub-step P_val it is checked whether the segment formed by the connection of the reference point O of the ego VEH and of the point M 'corresponding to the length s on the spiral forms a right angle with the spiral at this point. To do this,
• the trace step T consumes the output of P_C0, that is to say that once the projection of the reference point O of the vehicle ego on the initial spiral C obtained, this last projected point M 'existing and belonging to the spiral, we define this projected point M' with coordinates (x'0, y'0) as being both the new point of origin of the spiral and the last point, historically, from the past part of the ground marking.
• the step T of tracing the contour exceeded beyond the point M projected M 'and of length dL L- L', cf.
• Figure 1b comprises a sub-step of calculation of intermediate points in which one carries out, as represented in FIG.
• This discretization step dl can be fixed at a predetermined value of 2 m for example, but it can also be a function of the speed of the vehicle ego so as to have less frequent sampling at high speed, with for example dl going up to 5 m above 90 km / h, as in the projection stage P.
• the tracing step T comprises a sub-step aimed at geometrically ordering the new set of projection points obtained, whether for the same track contour identifier or without having such an identifier, including the series of intermediate points, a geometric comparison is made for this to put them in decreasing order.
• this new set of projection points already includes points from the past, ie behind (xo, yo)
• the points of G "old past” are represented connected to each other by a solid line and the points of the "new" past are represented connected to each other by a dotted line.
• the plot has the form of a cloud of points.
• Another form of trace can be chosen, for example in the form of a line, by connecting the recorded points.
• a memorized characteristic can be the nature of the marking on the ground (continuous or discontinuous for example) represented by the spiral at the projection point since this characteristic can be part of the characteristics detected by the sensors, and associated with the spiral.
• the nature of the line may depend on the characteristic of the nature of the marking on the ground assigned to each projection point. This example is not limiting.
• the plot can be virtual, in the sense that it is not displayed on a human machine interface (HMI) but serves as input data for driving assistance algorithms, but it can also be displayed on an HMI for the information of the driver, in particular when backing up and the vehicle does not have a rear camera and in this case it is preferentially chosen not to project again onto the past but to keep the track and the memory as what.
• HMI human machine interface
• step M of storing and updating the memory which follows, or which takes place in parallel with the plotting step T the new point s as well as the series of intermediate points are then stored knowing that 'a maximum number of memorized points is fixed for example at 50.
• the maximum number of memorized points is fixed for example at 50.
• this last projected point M 'existing and belonging to with the spiral one defined in the sub-step P_C0 this projected point M 'of coordinates (x'o, y'o) as being both the new point of origin of the spiral and the last point, historically, of the past part of the floor marking.
• the new length L ′ is then preferably calculated in step G of updating the spiral, as indicated in the description of FIG. 1b. This sub-step of calculating the new length L ′ could also be carried out at the end of the projection step (P) for example.
• a sub-step aimed at evaluating the error associated with the new point of origin of the spiral, which corresponds to the projected point M ′, can also be carried out.
• This sub-step can of course be carried out in another step once the projected point M ’has been obtained.
• the evaluation of the error corresponds to the calculation of the variance ⁇ (l), of the new point of origin of the updated spiral C.
• This step involves the error propagation of the previous initial point whose variance is ⁇ (0), which gives:
• the memory is emptied when the vehicle falls asleep.
• the method is advantageously used in driving assistance applications.
• FIG. 7 illustrates a use in which the display of the trace of the past is offered to the driver.
• the driver of the ego VEH vehicle has at his disposal displayed on the HMI screen the vehicles behind him, each one assigned to a lane which has been previously identified during the execution of the process.
• three objects have been detected behind the vehicle ego VEH, by means of a radar for example, they are vehicles VEH1, VEH2 and VEH3.
• Information such as the position and / or the relative distance of each object of the vehicle ego, its type, kinematic data such as speed and acceleration, the presence of the indicator (here represented by a gray area in the figure), etc. are part of the characteristics detected and associated with each object.
• an association step followed by a step of displaying on a screen the tracing of said exceeded contour are added to the method, they use the stored data which are already ordered and these steps can be executed during the process (for example at the end of the storage step M) or outside the process by just calling the stored data.
• a detected channel is associated with each object if it exists. For example, for each object belonging to the zone delimited by the outer contour of the lanes delimited by the route of their marking on the exceeded ground, a step takes place of associating the object in question with a lane of said lanes.
• the objects were then associated with the lane in which they were detected, thus the first object VEH1 is associated with the adjacent lane on the right with respect to the lane in which the ego vehicle is traveling.
• the second object VEH2 is associated with the same lane as the ego VEH vehicle and the third object VEH3 is associated with the adjacent left lane with respect to the lane in which the ego VEH vehicle is traveling.
• FIG. 8 illustrates a use in which an improvement is proposed in the on-board function acting automatically and allowing a vehicle to follow another at an appropriate distance (ACC for Adaptive Cruise Control in English).
• an association step is added to the method in which a detected path is associated with each object if it exists, as explained in the previous paragraphs. This step
• association especially when it is performed for the sole need of ACC, that is to say without HMI display, can for example be executed in parallel in the ACC computer which then calls the stored process data.
• the lateral and longitudinal control needs to know not only the C_ACC target to be followed but also information on the objects VEH1, VEH2, VEH3 surrounding the vehicle ego VEH, allocated on the critical channels that are in particular the path of the ego vehicle and the two adjacent paths.
• Information such as the position of each object, its type, kinematic data, the presence of the indicator (here represented by a gray area in the figure), etc., will be used by the function to adapt the trajectory and speed of the vehicle ego VEH according to the dynamic behaviors of these objects.
• the ACC function can change the target to follow if it considers that the level of risk associated with the first target exceeds a risk threshold for the ego VEH vehicle, for example if the target C_ACC is traveling slowly compared to the vehicle.
• VEH2 vehicle following in the ego vehicle lane It is therefore useful to know in which lane the other vehicles are in order to better anticipate their behavior as a function not only of kinematic information but also of semantics (in relation to FIG. 8 to VEH2 AND C_ACC in particular).
• This method can be applied to the automobile to use and / or visualize outdated markings on the ground, but can for example also be applied to the field of robotics.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Traffic Control Systems (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=68581942

Family Applications (1)

Country Status (4)

Families Citing this family (1)

Family Cites Families (3)

• 2019
  • 2019-07-16 FR FR1908021A patent/FR3098778B1/fr active Active
• 2020
  • 2020-06-08 CN CN202080051344.9A patent/CN114762011A/zh active Pending
  • 2020-06-08 WO PCT/EP2020/065802 patent/WO2021008775A1/fr unknown
  • 2020-06-08 EP EP20730427.0A patent/EP3999997A1/de active Pending

Also Published As

Similar Documents

Legal Events