Classifications

• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
  • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
  • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
  • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
  • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
  • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
  • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
  • B60R1/28—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with an adjustable field of view
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
  • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
  • B60W50/08—Interaction between the driver and the control system
  • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
  • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
  • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
  • B60R2300/101—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using cameras with adjustable capturing direction
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
  • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
  • B60R2300/301—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
  • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
  • B60R2300/306—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using a re-scaling of images
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
  • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
  • B60R2300/307—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing virtually distinguishing relevant parts of a scene from the background of the scene
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
  • B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
  • B60R2300/602—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective with an adjustable viewpoint
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
  • B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
  • B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
  • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
  • B60W2420/403—Image sensing, e.g. optical camera
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
  • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
  • B60W2420/408—Radar; Laser, e.g. lidar

Definitions

• the invention relates to a method and apparatus for distortion-free viewing a vehicle surroundings of a traveling ⁇ tool, in particular a road vehicle which has a camera surround-view system.
• Driver assistance systems that assist the driver in making driving maneuvers.
• These advanced driver assistance systems include partially surround camera-view systems which allow it to the driving ⁇ imaging environment of the vehicle to the driver of the vehicle ⁇ show.
• Such a camera surround-view systems include one or more vehicle cameras which provide camera images sammenge Stahlgt by a data processing unit of the camera surround view system for a vehicle environment to concernsstruckder ⁇ . The image of the vehicle environment is displayed on a display unit.
• Conventional inevitablyba ⁇ catalyzed driver assistance systems project texture information of the camera system on a static screen, for example, on a static two-dimensional basic ⁇ surface or on a static three-dimensional shells ⁇ surface.
• the invention accordingly provides, according to a first aspect, a camera surround view system for a vehicle, wherein the camera surround view system comprises at least one vehicle camera which supplies camera images which are transmitted by a data processing unit to the vehicle Generation of a surround view image or an environmental image displayed on a display screen, the data processing unit re-projecting textures captured by the vehicle cameras on an adaptive reprojection surface similar to the vehicle environment calculated on the basis of sensor data provided by vehicle sensors of the vehicle, wherein the data processing unit adjusts the re-projection area depending on a position and / or orientation of a virtual camera.
• the camera surround view system comprises at least one vehicle camera which supplies camera images which are transmitted by a data processing unit to the vehicle Generation of a surround view image or an environmental image displayed on a display screen, the data processing unit re-projecting textures captured by the vehicle cameras on an adaptive reprojection surface similar to the vehicle environment calculated on the basis of sensor data provided by vehicle sensors of the vehicle, wherein the data processing unit adjusts the re-projection area depending on a position and / or
• the sensor data provided by the vehicle sensors accurately represent the vehicle surroundings of the vehicle.
• the sensor data comprise Parking distance data, radar data, lidar data, camera data, laser scan data and / or movement data.
• the adaptive Re ⁇ projection surface on a dynamically changeable grid is provided.
• the grid of the re Proj edictionsional is dynamically varied as a function of the provided Sen ⁇ sorstein.
• the grating of the re-projection surface is a three-dimensional grating.
• the display unit is a touch screen and the position and / or orientation of the virtual camera is on the touch screen by a user adjustable ⁇ bar.
• the invention further provides a driver assistance system having the features specified in claim 7.
• the invention accordingly provides, in a second aspect, a driver assistance system with a Aarin integrated camera surround-view system, said at least one vehicle ⁇ camera that provides camera images, which are processed by a data processing unit for generating image of a surround-view, which is displayed on a display unit ⁇ , wherein the data processing unit textures that are detected by the vehicle cameras of the vehicle, on an adaptive vehicle environment-like feedback
• the invention further provides a method for distortion-free display of a vehicle environment of a vehicle with the features specified in claim 8.
• the invention accordingly provides a method for distortion-free display of a vehicle environment of a vehicle with the following steps:
• FIG. 2 shows a flow chart for illustrating an exemplary embodiment of the method according to the invention for the distortion-free display of a vehicle environment of a vehicle; a schematic representation for explaining the operation of the method according to the invention and the camera surround Vi ew system according to the invention.
• the camera surround view system 1 has several components in the illustrated example.
• the camera surround-view system 1 comprises in the illustrated embodiment, at least one Anlagenka.- ra 2, the camera images provides that by Da ⁇ ten kausaku 3 of the camera surround-view system 1 to a surround-view Bi Id or an environmental image of the vehicle are processed.
• the surround view images or vehicle environment images generated by the data processing unit 3 are displayed on a display unit 4.
• the data processing unit 3 calculated on the basis of sensor data which are penetratege from vehicle sensors 5 of the vehicle ⁇ represents, etechnischsflache an adaptive re-Proj.
• Textures that are detected by the vehicle cameras 2 of the camera surround-view system 1 are similar to that of the vehicle environment calculated adaptive re-Proj etechnischsflache re ⁇ projected, thereby nimiert distortions or artifacts distorted mi ⁇ or eliminated.
• the sensors 5 shown in FIG. 1 are, for example, sensors of a parking space control or parking space control.
• the sensors of the vehicle may be radar sensors, lidar sensors.
• the Sensorda- be of th other vehicle cameras 2, in particular a Ste ⁇ reoarra or mono camera, for the calculation of the adaptive re-Proj etechnischsflache delivered.
• the sensor data is provided by a laser scanning system of the vehicle.
• the processing unit 3 uses the sensor data or structural data for calculating the re-Proj be emiesflache .vera.rbei by the processing unit 3 used.
• the sensor data provided by the vehicle sensors 5 reproduce the vehicle environment or objects in the vehicle environment with high accuracy. These objects are at spielswerse to other vehicles in the immediacy ⁇ cash vicinity of the vehicle, for example. To a ⁇ circle of up to five meters are located. Furthermore, these objects may also be passers-by who pass by the vehicle in close proximity at a distance of up to five meters. The objects may also be other obstacles, such as bars for limiting a parking area.
• the re-projection area calculated by the data processing unit 3 on the basis of the sensor data preferably has a dynamically changeable grid or mesh.
• This grid of re-Proj etechnischsflache is dynamically changed in a possible exporting ⁇ approximate shape as a function of sensor data provided.
• the grid of the re-projecting surface is. preferably a three-dimensional grid. Is calculated by the As ⁇ ten kausaku 3 Re-Proj etechnischsION not, statically but dynamically and adaptively adaptable to the instantaneous sensor data, which are supplied by the vehicle sensors 5.
• This vehicle sensors 5 may comprise a single camera or a front Ste ⁇ reoarra in a possible embodiment. Furthermore, the sensor units.
• the data processing unit 3 may include one or more microprocessors, which processes the sensor data and calculates a re-projection area in real time. Textures that are detected by the vehicle cameras 2 are ied to this ⁇ be expected of the vehicle environment similar Re-P.rojetechnischsflache proj or re-proj ied.
• the display of the vehicle cameras 2 may vary. In one possible embodiment, the vehicle has four vehicle cameras 2 at four un ⁇ ter Kunststofflichen sides of the vehicle. In the vehicle han ⁇ delt it is preferably a road vehicle, insbesonde ⁇ re a truck or a passenger car.
• the inventive Käme- ra surround-view system 1 With the inventive Käme- ra surround-view system 1, the textures captured by the camera 2 of the camera system of the environment by the adap ⁇ tive re-P.roj re-proj ied etechnischsflache to reduce the above genann ⁇ th artifacts or eliminate.
• the inventive camera surround view system 1 thus greatly improves the quality of the vehicle environment shown. Objects in the vehicle environment, for example, other parked near vehicles or located near Perso ⁇ nen appear less distorted than in systems that use a static re- Pan10 ⁇ 3Ober Structure.
• the data processing unit 3 controls a virtual camera 6 as shown in FIG.
• the virtual camera 6 provided by the Data processing unit 3 is controlled, camera images on the vehicle F from a bird's eye view.
• the virtual camera 6 is arranged perpendicular with virtually ei ⁇ nem angle of 90 ° and a height H above the body of the vehicle F.
• the camera image of the virtuel ⁇ len camera 6 may be made of camera images of surround-view cameras which are provided on the vehicle F, are calculated by the data processing unit.
• the virtual Ka ⁇ ra 6 has a camera orientation relative to the driving ⁇ generating F and a relative position to the vehicle F.
• the positi on ⁇ and orientation of the virtual camera 6 is preferably adjustable.
• the virtual camera can be 6, starting from its vertical positi on ⁇ with an angle of 90 ° to the vehicle body tends ⁇ ge, where it assumes an inclination angle ⁇ of, for example 45th
• the distance relationship e the height of the vehicle camera 6 with respect to the vehicle F remains constant in the example shown in FIG.
• the data processing unit 3 reads out the current Po ⁇ sition and orientation of the virtual camera 6 relative to the vehicle F from a parameter memory of the virtual ⁇ Ka ra. 6 Depending is of the read-out parameters of the virtual camera 6 then the adaptive re-Proj etechnischsflache set by the data processing unit 3 relationship ⁇ adapted manner such that on the display unit 4 are as much texture and camera information distortion Darge ⁇ represents while obstructions in the immediate Environment of the vehicle F for the driver of the vehicle F easily recognizable.
• the display unit 4 is a touch screen.
• a driver or user of the vehicle F can touch the touchscreen and adjust or adjust the position and / or orientation of the virtual camera 6 in order to avoid obstacles in the immediate vehicle environment, such as bars, which mark a limited parking area clearly visible.
• An obstacle may be any object, which prevents aas vehicle F at umherfah ⁇ ren on the road surface, for example a pile of snow or rod for limiting a parking area.
• FIG. 2 shows a flow chart for illustrating an embodiment of the method according to the invention for the distortion-free display of a vehicle environment of a vehicle.
• a first step Sl camera images of the vehicle surroundings are generated by vehicle cameras 2 of the vehicle F.
• the camera images are formed by a plurality of vehicle cameras 2, which are mounted on different sides of the driving ⁇ tool ..
• the camera images produced are processed including in step S2 to ge ⁇ nerieren a surrounding image of the vehicle environment.
• the processing of camera images generated by a ve ⁇ rarbeitungsaku 3 takes place as shown in FIG. 1
• the Camera images are preferably processed in real time to generate a corresponding environmental image.
• a Re ⁇ P.roj emiesflache is first ⁇ net calculated and then textures that are detected by the vehicle cameras, re-projected on this adaptive calculated Re Monsterionsflache on the basis of provided sensor data.
• the adaptive re-projection surface has a dynamically changeable grid that is dynamically changed depending on the sensor data provided. This grid is preferably a three-dimensional grid.
• Step S4 the re-projection surface by the data processing unit 3 in response to a position and / or orientation of a virtual camera 6, which provides a bird's eye view camera image of the vehicle F from above, adjusted.
• the method illustrated in Figure 2 may be implemented by a Computerprogratrun at a Moegli ⁇ chen embodiment containing computer instructions that can be executed by a microprocessor.
• This program is stored in a possible embodiment on a data carrier or in a program memory.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Mechanical Engineering (AREA)
• Signal Processing (AREA)
• Automation & Control Theory (AREA)
• Transportation (AREA)
• Physics & Mathematics (AREA)
• Mathematical Physics (AREA)
• Human Computer Interaction (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Closed-Circuit Television Systems (AREA)
• Traffic Control Systems (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=55661011

Family Applications (1)

Country Status (7)

Families Citing this family (7)

Family Cites Families (15)

• 2015
  • 2015-02-17 DE DE102015202863.1A patent/DE102015202863A1/de not_active Withdrawn
• 2016
  • 2016-02-04 KR KR1020177022638A patent/KR20170118077A/ko not_active Application Discontinuation
  • 2016-02-04 CN CN201680010084.4A patent/CN107249934B/zh active Active
  • 2016-02-04 JP JP2017538401A patent/JP2018509799A/ja active Pending
  • 2016-02-04 DE DE112016000188.6T patent/DE112016000188A5/de active Pending
  • 2016-02-04 EP EP16714224.9A patent/EP3259907A1/de active Pending
  • 2016-02-04 WO PCT/DE2016/200074 patent/WO2016131452A1/de active Application Filing
• 2017
  • 2017-08-17 US US15/679,603 patent/US20170341582A1/en not_active Abandoned

Also Published As

Similar Documents

Legal Events