Classifications

• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
  • B60W30/10—Path keeping
  • B60W30/12—Lane keeping
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
  • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
  • B60W40/06—Road conditions
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/044—Recurrent networks, e.g. Hopfield networks
  • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/048—Activation functions
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/084—Backpropagation, e.g. using gradient descent
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
  • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
  • B60W2420/403—Image sensing, e.g. optical camera
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W2540/00—Input parameters relating to occupants
  • B60W2540/18—Steering angle
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W2552/00—Input parameters relating to infrastructure
  • B60W2552/10—Number of lanes
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W2552/00—Input parameters relating to infrastructure
  • B60W2552/53—Road markings, e.g. lane marker or crosswalk
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W2710/00—Output or target parameters relating to a particular sub-units
  • B60W2710/20—Steering systems
  • B60W2710/207—Steering angle of wheels

Definitions

• the present invention generally relates to the field of driver assistance systems.
• Some motor vehicles are currently equipped with an image capture device designed to detect the position of lines marked on the ground and delimiting traffic lanes.
• the position of the lines, around and in front of the vehicle makes it possible to implement guidance systems to aid driving.
• These guidance systems can, for example, be centering systems maintaining the vehicle equidistant between a straight line and a left line (we speak of an LCA system - from the English "Lane Centering Assist").
• the guidance systems can also be misled by poor detection of the position of the lines. For example, if the vehicle mistakes a shadow or reflection for a line, the guidance system can cause the vehicle to follow that shadow or reflection and therefore guide the vehicle out of the lane.
• the widenings and narrowings of the track can deceive the guidance systems and/or lead them to follow non-optimal trajectories in terms of safety, distance traveled or energy consumed.
• document LR3109920 discloses a method comprising steps of: - detection, on the basis of the variation of the difference in distance between the right and left lines, of a widening of said lane, then, when a widening is detected,
• the present invention proposes a solution allowing better detection of road widenings and narrowings.
• the invention proposes to rely on a very specific type of neural network, namely recurrent neural networks, to detect singular areas.
• At least two parameters are calculated, including a parameter for detecting an increase in the number of traffic lanes, and a parameter for detecting a reduction in the number of traffic lanes;
• the recurrent neural network comprises an input layer, an output layer, and at least a first hidden layer comprising a recurrence mechanism
• the recurrence mechanism is of the LSTM type
• said first hidden layer comprises several tens of neurons
• the output layer is an activation layer and at least one other hidden densification layer is provided, which is located after said first hidden layer and whose number of neurons is equal to the number of neurons of the output layer.
• the invention also relates to a method for assisting in maintaining a motor vehicle on a traffic lane of a road, comprising:
• the LCA function is designed to calculate the steering setpoint differently, depending on whether the parameter indicates that the vehicle is in a singular zone or not. It is also designed here to calculate this setpoint differently depending on whether the parameter indicates that the vehicle is in a road widening or road narrowing zone.
• a selection step from two lines of separation of right and left lanes located on either side of the motor vehicle, of a first line which deviates the least from the motor vehicle and / or its trajectory , and in which,
• the steering angle setpoint is determined according to the position of said first line, and independently of the position of the other of the two right and left lane separation lines. [0021]Preferably, during said determination step, the steering angle setpoint is determined by considering that the width of the traffic lane measured before the singular zone does not vary within the singular zone.
• the steering angle setpoint is determined according to the positions of the right and left lane separation lines located after the singular zone.
• the invention also relates to a method for configuring a computer to help maintain a motor vehicle on a traffic lane of a road, comprising:
• a step of learning a recurrent neural network making it possible to determine at least one detection parameter of a singular road zone within which the number of traffic lanes increases or decreases, said recurrent neural network receiving said base of data as training data, and
• FIG-1 is a schematic view of a motor vehicle carrying a computer adapted to implement a driving assistance method according to the invention, and which is driving on a portion of road in front of which finds a singular area;
• FIG.2 is a graph illustrating variations in vehicle position data automobile on a road
• FIG-3 is a table illustrating a training database of a neural network
• FIG.4 is a table illustrating in another form the values of the learning database of [Fig.3];
• FIG.5 is a graph illustrating the variations over time of a first detection parameter of a singular road zone
• FIG.6 is a graph illustrating the variations over time of a second detection parameter of a singular road zone
• FIG.7 is a graph illustrating the variations over time of a third detection parameter of a singular road zone.
• FIG.l there is shown a vehicle 100, here a motor vehicle, driving on a road 10.
• This vehicle 100 can be of any type, for example car, truck, etc.
• a steering system which makes it possible to act on the orientation of the steered wheels of the vehicle and which is controlled by a steering actuator.
• the vehicle 100 is also equipped with an image capture device (not shown).
• This image capture device is located at the front of the vehicle 100 to capture at regular intervals an image on which appears the road 100 taken by the vehicle, and in particular the markings printed on the road.
• the image capture device may comprise a camera equipped with a wide-angle lens or a series of several cameras.
• the vehicle 100 also includes a data processing computer unit, hereinafter referred to as a “computer”.
• This computer comprises at least one processor, one memory and input and output interfaces. In practice, it may consist of several separate units each comprising a processor.
• the computer is suitable for receiving the images captured by the camera(s). Thanks to its output interfaces, the computer is suitable for controlling F steering actuator.
• the computer stores a computer application, consisting of computer programs comprising instructions whose execution by the processor(s) allows the computer to implement the method described below.
• the computer is thus in particular able to implement a method for maintaining the vehicle in the center of its traffic lane.
• a method is better known by the English acronym LCA (for “Lane Centering Assist”).
• LCA for “Lane Centering Assist”.
• this process is to detect the position of the center of the traffic lane based on the marking lines on the ground, and to control the steering actuator to maintain the vehicle in the central position.
• the computer may include separate entities, one of which implements the driving assistance methods (including the LCA function) and in which the method described below afterwards will be programmed.
• the method could be programmed in a separate entity, for example in an entity integrated into the camera.
• FIG.1 there is shown an example of road 10 (country road, street, highway, etc.) on which vehicle 100 travels.
• road 10 country road, street, highway, etc.
• a road has one or more traffic lanes, allowing vehicles to overtake or cross each other. In the remainder of this presentation, only the traffic lanes of the road whose traffic directions are the same will be considered.
• lane separation lines are generally separated from each other, from the roadside and from the lanes in the opposite direction by marking lines on the ground, hereinafter called “lane separation lines”.
• the vehicle 100 travels on a road 10 which, on the left of the figure, comprises a single traffic lane 17 and, on the right, comprises two lanes of circulation 18, 19.
• These different lanes are delimited (with respect to the aisles) by two lane separation lines 11, 13, namely a left aisle line 11 and a right aisle line 13. These lines are here represented as being continuous but they could alternatively be discontinuous.
• the two traffic lanes 18, 19 are separated by a center line 12 shown in dotted lines but which could be continuous.
• Such an area is defined as a portion of the road along which the number of traffic lanes varies, either increasing or decreasing.
• this number increases so much that we will speak in the following of road widening. If it was reduced, we would rather speak of a narrowing of the road.
• the number of traffic lanes increases from one to two.
• the present invention proposes to calculate the steering setpoint at send to the steering actuator so that it is not deceived by a widening of the track 20.
• the proposed method comprises two successive operations, including a singular zone detection operation then, when such a zone is detected, an operation for calculating the steering setpoint taking into account the presence of this singular zone.
• the detection operation is implemented by the computer in several steps, repeated in a loop at regular time steps.
• the first step consists in recording positioning data of the motor vehicle 100 on the road 10.
• the processor is programmed in particular to detect, from the captured images, the position of the central lines 12 and of the left 11 and right 13 sides with respect to the vehicle 100, that is to say with respect to a repository linked to the vehicle 100.
• the processor is programmed to implement known image processing algorithms.
• the vehicle 100 rolls in a straight line along a main trajectory DI.
• the main trajectory DI therefore corresponds to the direction of the vehicle 100 when the latter is traveling in a straight line. When turning, this trajectory would be curved.
• the computer is then able to calculate, at a sight distance d v determined in front of the vehicle, the distance between the main trajectory DI and each marking line on the ground.
• the sighting distance d v can be predetermined and fixed (for example equal to 100 meters) or it can vary according to the speed of the vehicle (it can for example be a distance that the vehicle travels in a determined period of time, for example equal to one second).
• the computer determines at the sighting distance d v :
• the computer determines, at each time step:
• the computer calculates, by means of a recurrent neural network, at least one parameter Ps, Pm, Po for detection of singular zone 15.
• these parameters can be used to calculate a steering instruction for the vehicle which allows the latter to stay well in the center of its traffic lane and to cross without difficulty and without danger. singular areas.
• the neural network used here is of the recurrent type (better known by the English acronym RNN). It thus makes it possible to provide results which depend on the inputs provided to the network at the current time step, but also according to the inputs which have been provided at the previous time steps.
• the recurrence mechanism chosen is the LSTM type (from the English “Long shortterm memory”, which can be translated as “short and long term memory”). Such a mechanism allows efficient training of a neural network by employing long-duration training sequences.
• this neural network comprises an input layer, an output layer and two hidden layers, one of which includes the recurrence mechanism.
• the input layer comprises at least four neurons in order to receive the values of the distances pos_L and pos_R and of the differences Diff_Nxt_L and Diff_Nxt_R. She in here comprises eight (see [Fig.4]) in order to receive:
• the first hidden layer which receives as input the values transmitted by the neurons of the input layer and which comprises the LSTM recurrence mechanism, here comprises several tens of neurons. This number is preferably between 30 and 70, and here equal to 50, in order to obtain reliable results without overloading the calculations.
• the second hidden layer is here a densification layer which receives as input the outputs of the fifty neurons of the first hidden layer and which comprises a number of neurons equal to the number of parameters to be calculated (here equal to three).
• the exit layer could be formed by this densification layer.
• this output layer is an activation layer with three neurons, located behind the densification layer.
• This activation layer operates a “softmax” type function. More precisely, its neurons are able to calculate a 3-dimensional output vector, whose values Y correspond to the values of the parameters Ps, Pm, Po. Its neurons receive as input a 3-dimensional input vector X, and we can write :
• This learning of the neural network is done by backpropagation of the error gradient, on a learning set.
• the learning set corresponds to available data values.
• the learning set obtained then comprises several sequences corresponding each rolling upstream, through and downstream of a unique area.
• these distance and difference values are normalized so as to all take values between 0 and 1.
• MinMaxScaler a tool known as MinMaxScaler can be used.
• This tool makes it possible, when the table of the database 200 is filled, to perform successively, for each of the four columns (here for the pos_L column), a calculation of the following type:
• the value 0 is assigned to the Ps and Pm parameters and the value 1 is assigned to the Po parameter.
• the value 0 is assigned to the Po and Pm parameters and the value 1 is assigned to the Ps parameter.
• the value assigned to the parameter Ps increases linearly so as to reach the value 1 before the end of the singular zone.
• the value assigned to the parameter Po decreases linearly so as to reach the value 0 before the end of the singular zone.
• the value assigned to the parameter Ps suddenly drops to 0 while that assigned to the parameter Po suddenly increases to be equal to 1.
• the value 0 is assigned to the parameters Po and Ps and the value 1 is assigned to the parameter Pm.
• the value assigned to the Pm parameter increases linearly so as to reach the value 1 before the end of the singular zone.
• the value assigned to the parameter Po decreases linearly so as to reach the value 0 before the end of the singular zone.
• the value assigned to the Pm parameter suddenly drops to 0 while the one assigned to the Po parameter suddenly increases to be equal to 1.
• This new database 201 comprises eleven fields, namely:
• This shifting step makes it possible to provide the network with its recurrence capacity. Indeed, the values of the first eight columns acquired at time f are then found associated with the values of the parameters that can be calculated at time t i+ i.
• the optimization algorithm chosen for this purpose is here known as the Adam optimization algorithm (which is an extension of the stochastic gradient method). This method has indeed demonstrated acceptable results both in terms of speed and convergence.
• This algorithm uses a cost function Le, which is here equal to the average of the errors in absolute value (better known by the acronym MAE - from the English “Minimum Absolute Error”), which can be written:
• -y is the expected value, as given by one of the last three fields of the new database 201, and
• -y. is the value predicted by the neural network being trained.
• This learning is done in stages, in a loop. To know when to stop learning, it is planned to regularly evaluate the neural network using another database (obtained like the new 201 database, but based on other rolling sequences of the test vehicle).
• the values of the first eight fields of this other database are provided as input to the neural network, at different time steps, and the difference between the results obtained (the predicted values y'.) is observed. and the expected results (y'i) provided by the last three fields of this other database.
• the neural network is trained and ready to be used. It can therefore be exported and stored in the memory of the computer of the motor vehicle 100.
• the vehicle is facing a widening of the road if the parameter Ps takes a value greater than 0.9, or facing a narrowing of the road if the parameter Pm takes a value greater than 0.9.
• FIGS. 5 to 7 illustrate the variations of the parameters obtained when the motor vehicle 100 progresses on a road such as that illustrated in [Fig.l].
• the neural network makes it possible to detect the widening of the road without difficulty, at a time when the two other parameters Pm, Po take substantially zero values.
• the parameters Ps, Po vary substantially linearly when the vehicle 100 arrives at near road widening. This is normal since the neural network has been trained by submitting values to it which vary in this way when approaching each singular zone 15.
• This trained neural network is then recorded in each of the motor vehicles of the same model as the test vehicle, and in particular in the motor vehicle 100.
• the motor vehicle 100 is then able to implement two successive steps, including:
• the computer is able to detect a singular zone 15.
• this process is well known and consists in maintaining the center of gravity of the vehicle at an equal distance from the right and left lane separation lines located immediately on either side of the motor vehicle 100.
• this process is modified so that the motor vehicle 100 does not end up straddling two lanes.
• the selection step thus simply consists in observing whether the distance between the trajectory and the left line is less than the distance between the trajectory and the right line at the sighting distance d v .
• the first line selected is that on the left, which makes it possible to keep the vehicle in the left lane. Otherwise, the first line selected is the one on the right, which allows to keep the vehicle in the right lane.
• the guidance of the vehicle 100 is carried out according to the position of this first line, without taking into account the position of the other line 13 at the sighting distance d v .
• the guidance may for example consist of following the first line 11 at a determined distance.
• This determined distance can for example be the distance at which the vehicle 100 was located from the first line 11 when the widening was detected.
• the LCA function makes it possible to determine a steering setpoint for the steering actuator of the vehicle which is suitable for crossing a road widening zone.
• the computer When the computer detects such a narrowing, it determines whether the traffic lane taken upstream of the narrowing continues beyond this narrowing. If this is the case, the computer determines the steering instruction so as to keep the vehicle in its lane, taking into account the positions of the two lane separation lines located beyond the narrowing. If this is not the case, the vehicle can be steered so as to place itself between the two marking lines of the nearest traffic lane which continues beyond the narrowing of the lane, and this as soon as it arrives in the singular area.
• the widening or narrowing of the route can be caused by a line detection error: this is then referred to as virtual widening or narrowing. Indeed, it can happen that a shadow or a reflection is confused with a marking line on the ground. In such a situation, the calculator will allow the zone to be passed without difficulty, in the same way as if an enlargement or contraction had actually taken place.
• singular zone detection may be used for purposes other than to keep the vehicle in the center of its lane.
• this detection could provide useful data to other vehicle driving assistance functions, in particular to the obstacle avoidance function (the indication of a singular zone which can help to find a trajectory of obstacle avoidance that stays on the road).

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• Evolutionary Computation (AREA)
• General Physics & Mathematics (AREA)
• Computing Systems (AREA)
• Health & Medical Sciences (AREA)
• Artificial Intelligence (AREA)
• General Health & Medical Sciences (AREA)
• Software Systems (AREA)
• Multimedia (AREA)
• Mathematical Physics (AREA)
• Transportation (AREA)
• Mechanical Engineering (AREA)
• Automation & Control Theory (AREA)
• Life Sciences & Earth Sciences (AREA)
• Biomedical Technology (AREA)
• Biophysics (AREA)
• Computational Linguistics (AREA)
• Data Mining & Analysis (AREA)
• Molecular Biology (AREA)
• General Engineering & Computer Science (AREA)
• Medical Informatics (AREA)
• Databases & Information Systems (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Traffic Control Systems (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=80225595

Family Applications (1)

Country Status (4)

Families Citing this family (1)

Family Cites Families (15)

• 2021
  • 2021-12-10 FR FR2113268A patent/FR3130230B1/fr active Active
• 2022
  • 2022-11-24 US US18/716,849 patent/US20250026342A1/en active Pending
  • 2022-11-24 EP EP22822059.6A patent/EP4445346A1/de active Pending
  • 2022-11-24 WO PCT/EP2022/083129 patent/WO2023104532A1/fr not_active Ceased

Also Published As

Similar Documents

Legal Events