EP4062370A1 - Kamerabasierte detektion von kippbewegungen - Google Patents
Kamerabasierte detektion von kippbewegungenInfo
- Publication number
- EP4062370A1 EP4062370A1 EP20803815.8A EP20803815A EP4062370A1 EP 4062370 A1 EP4062370 A1 EP 4062370A1 EP 20803815 A EP20803815 A EP 20803815A EP 4062370 A1 EP4062370 A1 EP 4062370A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- camera
- optical flow
- vehicle
- movement
- expected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/269—Analysis of motion using gradient-based methods
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
- H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Definitions
- the invention relates to a method according to claim 1 and a vehicle according to the independent device claim.
- the invention is based on the object of improving the stability of vehicles from tipping over at low speeds. This object is achieved by a method according to claim 1 and a vehicle according to the independent device claim.
- a tilting movement is a rolling movement, i.e. a turning movement around the longitudinal axis of the vehicle, or a pitching movement, i.e. a turning movement around the transverse axis of the vehicle, which causes the vehicle to become unbalanced and fall over.
- the vehicle is, for example, a wheel loader, a forklift or a dumper.
- the vehicle has a camera.
- a camera is a means of recording images of the environment on an imaging medium. It is preferably an optical camera in which visible light is imaging. The camera is fixed to the vehicle.
- the method provides that an image sequence is recorded with the camera.
- An image sequence is a sequence of at least two images.
- An optical flow is determined in the recorded image sequence. This is a vector field of the speed projected into the image plane of the camera of visible points of at least one object recorded by the camera.
- an optical flow to be expected is determined. The determination is based on a driving movement of the vehicle and a topography of the ground. A movement of the vehicle relative to the ground is referred to as driving movement.
- the topography of the subsurface is its surface texture.
- the movement of the vehicle and the topography of the ground clearly define a movement of the camera relative to objects that are stationary relative to the ground. This results in the expected optical flow of the object recorded by the camera.
- the step of determining the expected optical flow is subdivided into several sub-steps.
- the driving movement of the vehicle is determined.
- the driving movement is preferably performed while the image sequence is being recorded. averages. If necessary, the topography of the subsurface is also determined. This can be omitted if the surface is level.
- a movement of the camera relative to one or more objects recorded by the camera is clearly defined. This movement is determined in a further sub-step.
- the objects are objects that are stationary, i.e. immovable relative to the subsurface.
- the background itself can be such an object.
- the optical flow is determined which is to be expected on the basis of the determined movement of the camera.
- the sub-step of determining an optical flow to be expected due to the movement of the camera is further subdivided into sub-steps in a further preferred development.
- the object recorded by the camera is recognized in the image sequence.
- Corresponding automated or computer-implemented methods for image recognition are known to those skilled in the art from the prior art.
- neural networks or stochastic algorithms can be used for image recognition.
- an area in the image sequence is identified in which the object is imaged.
- the optical flow of this area is determined. This means that not the optical flow of the entire image area, but only the optical flow of the object in the image sequence is determined. This makes the determination more efficient and interference, for example caused by moving objects or people, is avoided.
- a corresponding optical flow of the object to be expected based on the movement of the camera is determined in the image sequence.
- the optical flow to be expected due to the movement of the camera is determined in the above-mentioned area in which the object is imaged.
- a vehicle according to the invention has at least one camera and at least one evaluation unit which implements the method according to the invention or a preferred development, ie which is designed to carry out such a method.
- FIG. 1 A preferred embodiment of the invention is shown in FIG. 1 in detail shows:
- the vehicle 101 shown in FIG. 1 has a camera 103 and an evaluation unit 105.
- the camera 103 is connected to the evaluation unit 105 in a signal-conducting manner.
- An object 107 is located in the image area of the camera 103, i.e. in an area recorded by the camera 103.
- the evaluation unit 105 determines an optical flow of the object 107 in an image sequence recorded by the camera 103.
- the evaluation unit 105 also detects a travel movement of the vehicle 101. This results in an expected optical flow of the object 107, which the evaluation unit 105 compares with the actual optical flow. If there are deviations, this suggests a tilting movement of the vehicle 101. In this case, the evaluation unit 105 initiates countermeasures to stabilize the vehicle 101.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
- Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102019217988.6A DE102019217988A1 (de) | 2019-11-21 | 2019-11-21 | Kamerabasierte Detektion von Kippbewegungen |
| PCT/EP2020/081404 WO2021099149A1 (de) | 2019-11-21 | 2020-11-09 | Kamerabasierte detektion von kippbewegungen |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4062370A1 true EP4062370A1 (de) | 2022-09-28 |
Family
ID=73198317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP20803815.8A Withdrawn EP4062370A1 (de) | 2019-11-21 | 2020-11-09 | Kamerabasierte detektion von kippbewegungen |
Country Status (6)
| Country | Link |
|---|---|
| US (1) | US20220414893A1 (de) |
| EP (1) | EP4062370A1 (de) |
| JP (1) | JP2023503915A (de) |
| CN (1) | CN114651278A (de) |
| DE (1) | DE102019217988A1 (de) |
| WO (1) | WO2021099149A1 (de) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102023127324A1 (de) * | 2023-10-06 | 2025-04-10 | Jungheinrich Aktiengesellschaft | Flurförderzeug mit einer Auswerteeinheit und wenigstens einer Kamera und Verfahren zur Erhöhung der Fahrsicherheit eines Flurförderzeugs |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102014007900A1 (de) * | 2014-05-27 | 2015-12-03 | Man Truck & Bus Ag | Verfahren und Fahrerassistenzsystem zur Ermittlung fahrdynamischer Zustände eines Nutzfahrzeugs |
Family Cites Families (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE19918597C2 (de) * | 1999-04-23 | 2001-03-08 | Deutsch Zentr Luft & Raumfahrt | Verfahren zur Reduktion der Kippgefahr von Straßenfahrzeugen |
| DE10326190A1 (de) * | 2003-06-06 | 2004-12-30 | Daimlerchrysler Ag | Vorrichtung und Verfahren zur Ermittlung einer räumlichen Ausrichtung eines Auflegers oder Anhängers |
| US7493202B2 (en) * | 2004-11-12 | 2009-02-17 | Takata Corporation | Vehicle safety control system by image processing |
| JP2009060499A (ja) * | 2007-09-03 | 2009-03-19 | Sanyo Electric Co Ltd | 運転支援システム及び連結車両 |
| WO2010134502A1 (ja) * | 2009-05-18 | 2010-11-25 | 小平アソシエイツ株式会社 | 画像情報出力方法 |
| DE102010010906A1 (de) * | 2010-03-10 | 2011-01-05 | Daimler Ag | Verfahren zur Erzeugung einer Darstellung eines Fahrzeuges und einer aktuellen Fahrzeugumgebung des Fahrzeuges |
| JP2012046081A (ja) * | 2010-08-27 | 2012-03-08 | Daihatsu Motor Co Ltd | 走行状態判定装置 |
| US9251708B2 (en) * | 2010-12-07 | 2016-02-02 | Mobileye Vision Technologies Ltd. | Forward collision warning trap and pedestrian advanced warning system |
| DE102011081354B4 (de) * | 2011-08-23 | 2025-05-15 | Robert Bosch Gmbh | Verfahren zur Bestimmung einer Fahrbahnunebenheit eines von zumindest einem Scheinwerfer eines Fahrzeugs ausgeleuchteten Fahrbahnabschnitts und Verfahren zur Steuerung einer Lichtaussendung zumindest eines Scheinwerfers eines Fahrzeugs |
| DE102011055795A1 (de) * | 2011-11-29 | 2013-05-29 | Continental Automotive Gmbh | Verfahren zur Ermittlung eines drohenden Überschlags eines Fahrzeugs |
| DE102012110082A1 (de) * | 2012-10-23 | 2014-04-24 | Continental Automotive Gmbh | Verfahren zum Steuern von Sicherheitseinrichtungen eines Kraftfahrzeugs im Falle einer Böschungsfahrt |
| DE102013222268A1 (de) * | 2013-10-31 | 2015-04-30 | Continental Automotive Gmbh | Automatische Korrektur der Gierausrichtung eines Kamerasystems für ein Fahrzeug |
| DE102014109062A1 (de) * | 2014-06-27 | 2015-12-31 | Connaught Electronics Ltd. | Verfahren zum Verfolgen eines sich einem Kraftfahrzeug nähernden Zielfahrzeugs mittels eines Kamerasystems des Kraftfahrzeugs, Kamerasystem und Kraftfahrzeug |
| JP6462557B2 (ja) * | 2015-11-06 | 2019-01-30 | 株式会社Soken | 車両ピッチ角推定装置 |
| CN107031523A (zh) * | 2015-11-30 | 2017-08-11 | 法乐第未来公司 | 利用已知目标进行基于摄像头的车辆位置确定 |
| US10699136B2 (en) * | 2016-01-29 | 2020-06-30 | Faraday & Future Inc. | System and method for camera-based detection of object heights proximate to a vehicle |
| DE102016209900B4 (de) * | 2016-06-06 | 2024-10-31 | Robert Bosch Gmbh | Verfahren und Vorrichtung zum Erkennen eines seitlichen Kippens eines Fahrzeugs und Fahrzeug |
| DE102017200278A1 (de) * | 2017-01-10 | 2018-07-12 | Volkswagen Aktiengesellschaft | Verfahren zum Bestimmen einer Bewegungsrichtung einer Kamera |
| WO2019089853A1 (en) * | 2017-10-31 | 2019-05-09 | Agjunction Llc | Three-dimensional terrain mapping |
| CN109878532A (zh) * | 2017-12-01 | 2019-06-14 | 福特全球技术公司 | 具有挂车的机动车辆的控制 |
| US10657654B2 (en) * | 2017-12-28 | 2020-05-19 | Denso Ten Limited | Abnormality detection device and abnormality detection method |
| DE102018102287A1 (de) * | 2018-02-01 | 2019-08-01 | Sick Ag | Optoelektronische Sensorvorrichtung |
| CN109300143B (zh) * | 2018-09-07 | 2021-07-27 | 百度在线网络技术(北京)有限公司 | 运动向量场的确定方法、装置、设备、存储介质和车辆 |
| WO2020079927A1 (ja) * | 2018-10-18 | 2020-04-23 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | 情報処理装置、プログラム及び情報処理方法 |
-
2019
- 2019-11-21 DE DE102019217988.6A patent/DE102019217988A1/de active Pending
-
2020
- 2020-11-09 EP EP20803815.8A patent/EP4062370A1/de not_active Withdrawn
- 2020-11-09 WO PCT/EP2020/081404 patent/WO2021099149A1/de not_active Ceased
- 2020-11-09 CN CN202080078080.6A patent/CN114651278A/zh active Pending
- 2020-11-09 JP JP2022529749A patent/JP2023503915A/ja active Pending
- 2020-11-09 US US17/778,186 patent/US20220414893A1/en not_active Abandoned
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102014007900A1 (de) * | 2014-05-27 | 2015-12-03 | Man Truck & Bus Ag | Verfahren und Fahrerassistenzsystem zur Ermittlung fahrdynamischer Zustände eines Nutzfahrzeugs |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021099149A1 (de) | 2021-05-27 |
| CN114651278A (zh) | 2022-06-21 |
| JP2023503915A (ja) | 2023-02-01 |
| US20220414893A1 (en) | 2022-12-29 |
| DE102019217988A1 (de) | 2021-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| DE102009015142B4 (de) | Fahrzeugumgebungs-Erkennungs-Vorrichtung und Steuerungssystem zum Verfolgen eines vorausfahrenden Fahrzeugs | |
| EP1332334B2 (de) | Messeinrichtung zur berührungslosen messung von reifen | |
| DE102016100030B4 (de) | Gerät zum sicheren führen eines fahrzeugs | |
| DE102015109238B4 (de) | Verfahren zum ermitteln eines zustands eines nassen strassenbelags für ein fahrzeug, das auf einer strasse fährt | |
| DE102014221144A1 (de) | Zielerfassungsvorrichtung | |
| DE102004022113A1 (de) | Überwachung eines PKW-Anhängers mit einer Rückfahrkamera | |
| EP3727936B1 (de) | Verfahren zum erkennen von zumindest einem an einem kraftfahrzeug anliegenden objekt sowie steuervorrichtung und kraftfahrzeug | |
| EP1505367A2 (de) | Verfahren zur Bestimmung der Achsgeometrie und Sensor zu dessen Durchführung | |
| WO2021151412A1 (de) | Vorrichtung und verfahren zur automatischen erkennung von beschädigungen an fahrzeugen | |
| EP2949531A2 (de) | Verfahren und fahrerassistenzsystem zur ermittlung fahrdynamischer zustände eines nutzfahrzeugs | |
| DE102021116703A1 (de) | Verfahren und System zur Vorhersage einer Trajektorie eines Zielfahrzeugs in einer Umgebung eines Fahrzeugs | |
| WO2002048670A2 (de) | Vorrichtung und verfahren zur qualitätsüberprüfung eines körpers | |
| WO2014108561A1 (de) | Verfahren und vorrichtung zum überwachen eines umfelds eines fahrzeugs und verfahren zum durchführen einer notbremsung | |
| EP3193276B1 (de) | Erfassungsvorrichtung und verfahren zum erkennen von fahrzeugachsen | |
| DE102019115330A1 (de) | Echtzeit-Sicherheitspfaderzeugung für hochautomatisiertes Fahrzeugrückfallmanöver | |
| DE102023123012B3 (de) | Verfahren und Vorrichtung zur schnellen Kollisionserkennung bei hochautomatisiertem Testen von Fahrvorgängen | |
| EP4062370A1 (de) | Kamerabasierte detektion von kippbewegungen | |
| WO2020201217A1 (de) | Verfahren und vorrichtung zur bestimmung von verformungen an einem objekt | |
| DE102019209463A1 (de) | Verfahren zur Bestimmung eines Vertrauenswertes eines Objektes einer Klasse | |
| DE102021203671B4 (de) | Verfahren zur Zustandsüberwachung eines Hubwagens eines Staplerfahrzeugs | |
| DE102022209403B4 (de) | Verfahren zum Überprüfen der Durchführung einer Prädiktionsaufgabe durch ein neuronales Netzwerk | |
| DE102022204406B4 (de) | Verfahren zum Klassifizieren und/oder zur Regression von Eingangssignalen unter Zuhilfenahme einer Gram-Matrix-Variante | |
| EP3871006B1 (de) | Regenerkennung mit einem umgebungssensor zur punktweisen erfassung einer umgebung eines fahrzeugs, insbesondere mit einem lidar-basierten umgebungssensor | |
| DE102022207295A1 (de) | Verfahren und Vorrichtung zur Überwachung eines Sichtfeldes eines stationären Sensors | |
| DE102022211241A1 (de) | Erkennen von Fahrbahnauflage auf einer Fahrbahn |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20220520 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| 17Q | First examination report despatched |
Effective date: 20230615 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
| 18D | Application deemed to be withdrawn |
Effective date: 20250603 |