EP4569493A1 - Verfahren zur bestimmung einer fahrspurmarkierung einer fahrspur für ein fahrzeug - Google Patents
Verfahren zur bestimmung einer fahrspurmarkierung einer fahrspur für ein fahrzeugInfo
- Publication number
- EP4569493A1 EP4569493A1 EP23750957.5A EP23750957A EP4569493A1 EP 4569493 A1 EP4569493 A1 EP 4569493A1 EP 23750957 A EP23750957 A EP 23750957A EP 4569493 A1 EP4569493 A1 EP 4569493A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- lane
- lane marking
- course
- width
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/766—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using regression, e.g. by projecting features on hyperplanes
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30172—Centreline of tubular or elongated structure
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
Definitions
- the present invention relates to a method for determining a lane marking for a vehicle.
- the present invention further relates to a method for training at least one machine learning model.
- the detection of a lane marking of a lane for a vehicle is an important aspect, particularly in the area of autonomous driving.
- the detection of a lane marking of a lane has so far been pursued by using different technical approaches.
- a first approach is based on a classic gradient method. Gradients are extracted from an image of the road that was recorded using sensors and/or cameras located in a vehicle. From this, a lane marking of the lane in which the vehicle is located is determined. The inner and outer edges of the lane marking are determined and put together in order to calculate the width of a lane marking.
- a second approach uses segment-based methods based on deep learning scenarios.
- the outer and inner edges of a lane marking of a lane are also determined by means of an estimate of individual segment masks of the lane marking.
- the accuracy of this method depends significantly on the resolution of the segment masking used.
- Another approach to determining the lane marking of a lane is pursued by so-called anchor-based or anchor-less approaches.
- the aim is to use a direct or immediate representation of a line of the lane marking to be detected around a center of the lane marking to be detected to model. This approach does enable greater accuracy in the exact determination or determination of the position of the lane marking.
- no direct conclusions can be drawn about the inner and outer edges of the lane marking to be detected.
- the disclosure relates to a method for determining a lane marking of a first lane for a vehicle with the following steps:
- the first step involves providing measurement data from monitoring the vehicle's surroundings.
- the measurement data is fed to at least one machine learning model.
- a course of the lane marking is evaluated using the at least one machine learning model.
- a width of the lane marking is evaluated using the at least one machine learning model.
- the present invention therefore offers the advantage that not only is a center of a lane marking to be detected in a lane of a vehicle modeled, but also that a width of the lane marking to be detected in the lane of the vehicle is also recorded or estimated at any time. This is done by adding or solving a corresponding regression problem.
- the approach pursued by the present invention achieves the advantage of a higher accuracy in determining the lane marking than with known approaches, since the center of the lane marking to be detected of a lane is still determined.
- a further advantage is that all lane markings can be represented or determined using the approach according to the invention.
- Another advantage of the solution according to the invention is that solving a regression problem for a width of the lane marking to be detected of a lane is much easier to solve than a continuous estimate of an inner and outer edge of a lane marking. This is because the width of the lane marking to be determined can be determined relative to a center line or reference line of the lane, which makes the method according to the invention independent of a position or location of the lane marking in a recorded image.
- a possible embodiment of the method provides that the course indicates a center line of the lane marking in discrete or continuous form. This provides the advantage of greater accuracy when determining the lane marking.
- a possible embodiment of the method provides that the width of the lane marking is determined by regression. This has the advantage that the width of the lane marking can be determined efficiently.
- a possible embodiment of the method provides that a portion of the lane marking closest to the vehicle is selected for the regression. This has the advantage that the width of the lane marking can be determined efficiently.
- a possible embodiment of the method provides that the regression is carried out at several points along the lane marking and the determined widths result in a final result for the width of the lane marking be aggregated. This further reduces the error in determining the width of the lane marking
- a possible embodiment of the method provides that the course is specified in the form of distances to a reference line through the monitored area of the vehicle's surroundings. This has the advantage that the width of the lane marking can be determined efficiently and with high accuracy.
- a possible embodiment of the method provides that the image data and/or video data are selected as measurement data. These are the most important measurement modalities for detecting lane markings.
- the first step is to select a position along the course of the lane marking.
- the search takes place in a predetermined search direction relative to the course of the lane marking for two boundary points between the lane marking on the one hand and the lane surface on the other.
- the width of the lane marking is determined from a distance between the two border points.
- a possible embodiment of the method provides that camera calibration data and / or information about the road condition of the first lane are taken into account when determining the width of the lane marking of the first lane. This can further increase the accuracy.
- the disclosure relates to a method for training at least one machine learning model for use in the method described above, with the following steps:
- the first step involves providing training examples of measurement data that were recorded from the perspective of an ego vehicle and indicate the presence of one or more lane markings.
- a target course and a target width are provided for at least one lane marking that delimits the lane currently traveled by the ego vehicle
- the training examples are fed to the machine learning model to be trained, so that this machine learning model, together with the previous one, determines a course and a width of the lane marking.
- a deviation between this course and this width on the one hand, and the target course or the target width on the other hand, is evaluated with a predetermined cost function.
- parameters that characterize the behavior of the machine learning model are optimized with the aim that the evaluation by the cost function is likely to improve with further processing of training examples.
- the width of a lane marking that delimits the lane currently traveled by the ego vehicle is the target width that can be determined most precisely within the framework of the label, because this lane marking is closest to the ego vehicle. It is therefore advantageous to only use this target width.
- a possible embodiment of the method provides that at least one target course is provided and a course is determined for at least one further road marking that does not limit the lane currently traveled by the ego vehicle, and the cost function also determines the deviation of this course from its target -Progress evaluated. This achieves the advantage that the method can also be used or transmitted efficiently to determine additional road markings that are not in the lane currently being traveled.
- a possible embodiment of the method envisages ignoring at least one lane marking that is apparent from the measurement data by setting its contribution to the cost function to zero. This allows this lane marking to be hidden more efficiently than by removing it from the measurement data of the training example.
- the disclosure relates to a computer program containing machine-readable instructions which, when executed on one or more computers and/or computer instances, cause the computer or computer instances to carry out the method according to the invention.
- the disclosure relates to a machine-readable data carrier and/or download product with the computer program.
- the disclosure relates to one or more computers and/or computer instances with the computer program and/or with the machine-readable data carrier and/or download product.
- Figure 1 Schematic flow diagram of the method for determining a lane marking of a first lane for a vehicle
- FIG. 2 Schematic flowchart of the method for training at least one machine learning model for use in the method according to Figure 1;
- Figure 3 exemplary representation of a method known from the prior art for determining a center of a lane marking for a lane of a vehicle
- Figure 4 exemplary representation of a method according to the invention for determining a width of a lane marking for a lane of a vehicle
- Figure 5 exemplary representation of the provision of a target position and a target width of a lane marking for a lane of a vehicle.
- FIG. 3 shows a schematic flow diagram of the method 100 for determining a lane marking 2 of a first lane 1 for a vehicle 10 with the following steps:
- a first step 102 measurement data from monitoring the surroundings of the vehicle 10 is provided.
- a second step 104 the measurement data is fed to at least one machine learning model.
- a course of the lane marking is evaluated using the at least one machine learning model.
- a width 3 of the lane marking 2 is evaluated using the at least one machine learning model.
- the evaluation of the width 3 of the lane marking 2 of the first lane 1 in the fourth step 108 is preferably carried out with the following steps:
- a position along the course of the lane marking 2 of the lane 1 is selected.
- a search is made in a predetermined search direction relative to the course of the lane marking 2 for two boundary points - see the line segment 9 in Figure 4 - between the lane marking 2 on the one hand and the lane surface on the other.
- a third step 124 the width 3 of the lane marking 2 of the lane 1 or the first lane 1 is determined from a distance between the two border points.
- Figure 2 shows a schematic flowchart of the method 200 for training at least one machine learning model for use in the method 100 according to Figure 1 with the following steps:
- a first step 202 training examples of measurement data are provided, which were recorded from the perspective of an ego vehicle 10 and indicate the presence of one or more lane markings.
- a target course and a target width are provided for at least one lane marking that delimits the lane currently traveled by the ego vehicle 10.
- a third step 206 the training examples are fed to the machine learning model to be trained, so that this machine learning model Model with the method according to one of claims 1 to 9 determines a course and a width 3 of the lane marking 2.
- a deviation between this course and this width 3 on the one hand, and the target course or the target width on the other hand, is evaluated with a predetermined cost function.
- a fifth step 210 parameters that characterize the behavior of the machine learning model are optimized, with the aim that the evaluation by the cost function is likely to improve with further processing of training examples.
- Figure 3 shows an exemplary representation of an image recorded by an (ego) vehicle 10 for using a method known from the prior art for determining a center of a lane marking 2 for a lane 1 of the vehicle 10 using an anchor-based approach. Between the starting point 6 and the end point 7 of a reference line 5, the individual distances of the individual line sections or the individual line segments 8 to the respective center 4 of the lane marking 2 of the lane 1 are determined for each point in the horizontal direction via regression.
- FIG. 4 shows an example of how the method 100 proposed here for determining a width 3 of a lane marking 2 for a lane 1 of a vehicle 10 can be carried out on the same recorded image.
- this method described in simple words, starting from the reference line 5, which runs between a starting point 6 and an end point 7, the respective width 3 of the lane marking 2 of the lane 1 is determined in the horizontal direction in accordance with the following steps of the method 100 as follows :
- a first step 102 measurement data from monitoring the surroundings of the vehicle 10 is provided, here in the form of the recorded image.
- the measurement data can generally be made available as image data and/or video data from corresponding sensors or other technical detection devices (not shown) of the vehicle 10.
- the measurement data is fed to at least one machine learning model.
- Several models can also be used, for example, a first model can be provided for determining the course of the lane marking 2 of the lane 1 and a second model to be used to determine the width 3 of the lane marking 2 of lane 1.
- a course of the lane marking 2 is evaluated using the at least one machine learning model.
- the course can be, for example, a center line 4, but also an edge.
- the course of the center line 4 can in particular be specified, for example, in the form of many discrete positions.
- the course of the center line 4 can also be specified in a continuous form.
- the course of the center line 4 can preferably - and as shown in Figure 4 - be specified in the form of distances of the individual line segments 9 of the lane marking 2 from the reference line 5 through the monitored area of the surroundings of the vehicle 10.
- a width 3 of the lane marking 2 is evaluated with the at least one machine learning model.
- the width 3 of the lane marking 2 of the lane 1 is preferably determined by regression.
- a so-called scalar regression can be used here, since it can be carried out at a higher speed and provides more precise results, since the lane marking 2 appears larger for a given pixel resolution of a vehicle camera.
- camera calibration data and/or information about the road condition of the first lane 1 can preferably also be taken into account.
- the regression can preferably be carried out at several points along the lane marking 2, with the determined widths being aggregated into a final result for the width 3 of the lane marking 2.
- a corresponding weighting can be carried out with the distance to the vehicle 10 in order to take into account a different degree of accuracy.
- the evaluation of the width 3 of the lane marking 2 of the first lane 1 in the fourth step 108 is preferably carried out with the following steps:
- a position along the course of the lane marking 2 of the lane 1 is selected.
- a search is made in a predetermined search direction relative to the course of the lane marking 2 for two boundary points - see the line segment 9 in Figure 4 - between the lane marking 2 on the one hand and the lane surface on the other.
- a third step 124 the width 3 of the lane marking 2 of the lane 1 or the first lane 1 is determined from a distance between the two border points.
- FIG. 5 shows an example of how a target position and a target width of a lane marking 2 for a lane 1 of a vehicle 10 can be obtained.
- Figure 5 shows how the middle 4 of the lane marking of the lane 1 of the ego vehicle 1 - dashed line - and the inner edge 11 of the lane marking 2 are marked accordingly.
- a target width of a lane marking can be determined, which limits the lane currently traveled by the ego vehicle 10. This target width can be used to label training examples for the machine learning model.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102022208429.2A DE102022208429A1 (de) | 2022-08-12 | 2022-08-12 | Verfahren zur Bestimmung einer Fahrspurmarkierung einer Fahrspur für ein Fahrzeug |
| PCT/EP2023/070893 WO2024033103A1 (de) | 2022-08-12 | 2023-07-27 | Verfahren zur bestimmung einer fahrspurmarkierung einer fahrspur für ein fahrzeug |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4569493A1 true EP4569493A1 (de) | 2025-06-18 |
Family
ID=87556285
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP23750957.5A Pending EP4569493A1 (de) | 2022-08-12 | 2023-07-27 | Verfahren zur bestimmung einer fahrspurmarkierung einer fahrspur für ein fahrzeug |
Country Status (6)
| Country | Link |
|---|---|
| US (1) | US20260011157A1 (de) |
| EP (1) | EP4569493A1 (de) |
| JP (1) | JP2025526670A (de) |
| CN (1) | CN119698643A (de) |
| DE (1) | DE102022208429A1 (de) |
| WO (1) | WO2024033103A1 (de) |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4616068B2 (ja) * | 2005-04-28 | 2011-01-19 | 本田技研工業株式会社 | 車両、画像処理システム、画像処理方法及び画像処理プログラム |
| JP2007304965A (ja) * | 2006-05-12 | 2007-11-22 | Mitsubishi Electric Corp | 画像処理装置 |
| JP6115429B2 (ja) * | 2013-09-27 | 2017-04-19 | 日産自動車株式会社 | 自車位置認識装置 |
| US9286524B1 (en) | 2015-04-15 | 2016-03-15 | Toyota Motor Engineering & Manufacturing North America, Inc. | Multi-task deep convolutional neural networks for efficient and robust traffic lane detection |
| EP3576008B1 (de) | 2018-05-30 | 2023-12-27 | Aptiv Technologies Limited | Bildbasierte fahrbahnmarkierungsklassifizierung |
| DE102018113559B4 (de) | 2018-06-07 | 2025-10-30 | Connaught Electronics Ltd. | Verfahren zum Erkennen einer Fahrbahnmarkierung durch Validieren anhand der Linienbreite; Steuereinrichtung; Stellplatzerkennungssystem; sowie Fahrerassistenzsystem |
| DE102019209634B4 (de) | 2019-07-02 | 2021-08-26 | Zf Friedrichshafen Ag | Computerimplementiertes Verfahren zum maschinellen Lernen von Fahrbahnmarkierungen mittels Audiosignalen, Steuergerät für automatisierte Fahrfunktionen, Verfahren und Computerprogramm zum Erkennen von Fahrbahnmarkierungen |
| US10867190B1 (en) | 2019-11-27 | 2020-12-15 | Aimotive Kft. | Method and system for lane detection |
| DE102020215324A1 (de) | 2020-12-03 | 2022-06-09 | Robert Bosch Gesellschaft mit beschränkter Haftung | Auswahl von Fahrmanövern für zumindest teilweise automatisiert fahrende Fahrzeuge |
-
2022
- 2022-08-12 DE DE102022208429.2A patent/DE102022208429A1/de active Pending
-
2023
- 2023-07-27 JP JP2025507274A patent/JP2025526670A/ja active Pending
- 2023-07-27 EP EP23750957.5A patent/EP4569493A1/de active Pending
- 2023-07-27 WO PCT/EP2023/070893 patent/WO2024033103A1/de not_active Ceased
- 2023-07-27 US US18/993,889 patent/US20260011157A1/en active Pending
- 2023-07-27 CN CN202380059399.8A patent/CN119698643A/zh active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| US20260011157A1 (en) | 2026-01-08 |
| WO2024033103A1 (de) | 2024-02-15 |
| DE102022208429A1 (de) | 2024-02-15 |
| JP2025526670A (ja) | 2025-08-15 |
| CN119698643A (zh) | 2025-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| DE102016113699B4 (de) | Spurhalte-Steuerverfahren und Vorrichtung hierfür | |
| DE102019131702A1 (de) | Anhängerwinkelerfassung unter verwendung von ende-zu-ende-lernen | |
| DE112016002910T5 (de) | Spurhaltevorrichtung | |
| DE102018128289A1 (de) | Verfahren und vorrichtung für eine autonome systemleistung und zur einstufung | |
| EP3393875B1 (de) | Verfahren zum verbesserten erkennen von objekten durch ein fahrerassistenzsystem | |
| EP3090922A1 (de) | Verfahren zur lenkung eines landwirtschaftlichen anhängers und landwirtschaftlicher zugverbund | |
| DE112018005265B4 (de) | Lenksteuerungsvorrichtung, lenksteuerungsverfahren und aufzeichnungsmedium | |
| EP2464992A1 (de) | Kollisionsüberwachung für ein kraftfahrzeug | |
| EP3482975A1 (de) | Raupenfahrzeug | |
| EP3850536A1 (de) | Analyse dynamisscher räumlicher szenarien | |
| DE102014210752A1 (de) | Verfahren und Vorrichtung zum Überwachen einer Solltrajektorie eines Fahrzeugs | |
| DE112009002415T5 (de) | Verfahren zum Messen eines zylinderförmigen Elements in einer Faserbahnmaschine und eine Anordnung und eine Messvorrichtung für das Verfahren | |
| EP4569493A1 (de) | Verfahren zur bestimmung einer fahrspurmarkierung einer fahrspur für ein fahrzeug | |
| DE102013200724A1 (de) | Vorhersage eines Fahrmanövers eines Fahrzeugs | |
| DE102010032462B4 (de) | Verfahren und Vorrichtung zum Einparken eines Kraftfahrzeugs | |
| EP3576013A1 (de) | Abschätzen eines verlaufs eines schienenpfads | |
| DE102018128535A1 (de) | Trainieren eines künstlichen neuronalen Netzwerks mit durch Sensoren eines Fahrzeugs erfassten Daten | |
| DE102021207830A1 (de) | Verfahren zur Steuerungsassistenz eines Fahrzeugs | |
| DE102019110018B4 (de) | Verfahren zur Kollisionsvermeidung, System, fahrzeuginternes System, Zugfahrzeug und Signal | |
| WO2014202365A1 (de) | Verfahren und vorrichtung zur ermittlung eines fahrzustands eines externen kraftfahrzeugs | |
| DE102019209813A1 (de) | Vorrichtung zum Navigieren eines Ist-Fahrzeugs durch zumindest einen Abschnitt einer Soll-Trajektorie, Verfahren und Verwendung | |
| DE112015006698B4 (de) | Servolenkungs-Einstellvorrichtung | |
| EP4350660A1 (de) | System und verfahren zur prädiktion einer zukünftigen position eines verkehrsteilnehmers | |
| DE102018129485A1 (de) | Verfahren sowie System zur Bestimmung eines Fahrkorridors | |
| DE102018219829A1 (de) | Verfahren, Steuergerät, System und Computerprogramm zur Bestimmung von charakteristi-schen Objektpunkten eines Zielobjektes sowie Verfahren zum kameragestützten Positionieren eines motorbetriebenen Fahrzeugs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20250312 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |