DE102012219735A1 - Method for image processing of data of images detected by cameras arranged at e.g. outer side of vehicle, involves adapting image data to be signaled such that hidden object is perceptible on image to be signaled - Google Patents

Method for image processing of data of images detected by cameras arranged at e.g. outer side of vehicle, involves adapting image data to be signaled such that hidden object is perceptible on image to be signaled

Info

Publication number
DE102012219735A1
DE102012219735A1 DE201210219735 DE102012219735A DE102012219735A1 DE 102012219735 A1 DE102012219735 A1 DE 102012219735A1 DE 201210219735 DE201210219735 DE 201210219735 DE 102012219735 A DE102012219735 A DE 102012219735A DE 102012219735 A1 DE102012219735 A1 DE 102012219735A1
Authority
DE
Germany
Prior art keywords
image
signaled
image data
environment
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
DE201210219735
Other languages
German (de)
Inventor
Thomas Reicher
Guenter Bauer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bayerische Motoren Werke AG
Original Assignee
Bayerische Motoren Werke AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bayerische Motoren Werke AG filed Critical Bayerische Motoren Werke AG
Priority to DE201210219735 priority Critical patent/DE102012219735A1/en
Publication of DE102012219735A1 publication Critical patent/DE102012219735A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08GTRAFFIC CONTROL SYSTEMS
    • G08G1/00Traffic control systems for road vehicles
    • G08G1/16Anti-collision systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation

Abstract

Image data (DAT) of respective images (P_i) are detected by a plurality of cameras arranged on a vehicle. Depending on the image data (DAT), environment model image data (PU) of an environment model image are determined. Depending on the environment model image data (PU), image data (PS) to be signaled of an image to be signaled are determined from the perspective of a virtual camera on the environment model image. In the image to be signaled, an imaged obstacle is detected which, from the point of view of the virtual camera, obscures an object that is depicted in the environment model image. The image data (PS) to be signalized are adjusted so that the hidden object is perceptible on the image to be signaled.

Description

  • The invention relates to a method and a device for image processing of image data of respective images, which are detected by a plurality of cameras, which are arranged on a vehicle.
  • Vehicles are increasingly being equipped with several cameras, which are mounted on the outside, ie at the periphery of the vehicle. For example, such a camera can be arranged on an exterior mirror of a vehicle. Such a camera can also be arranged, for example, on a vehicle front or at the rear of the vehicle. Such cameras may assist the driver in sensing the exterior of the vehicle.
  • The object on which the invention is based is to provide a method and a device which contributes to the fact that an image to be signaled is determined from a view that is catchy for a vehicle driver.
  • The object is solved by the features of the independent claims. Advantageous embodiments are characterized in the subclaims.
  • The invention is characterized by a method and a corresponding device for image processing of image data of respective images captured by a plurality of cameras arranged on a vehicle. Depending on the image data of the captured images, environmental model image data of an environment model image is determined. Depending on the environment model image data to be signaled image data of a signal to be signaled from the perspective of a virtual camera on the environment model image is determined. An obstacle imaged in the image to be signalized is detected, which, from the view of the virtual camera, obscures an object which is imaged in the environment model image. The image data to be signaled are adjusted so that the hidden object is perceptible on the image to be signaled.
  • The view of the virtual camera is preferably a view helpful to a driver.
  • By checking whether there is an obstacle in the image to be signaled, which, as seen by the virtual camera, obscures one or more objects imaged in the environment model image, this can be taken into account when creating the image to be signaled. For example, obstacles obscuring objects can be displayed transparently so that a driver of the vehicle perceives the objects from a helpful point of view. Thus, the driver receives additional help in the image to be signaled. In this way, for example, a cross traffic can be perceived from a driver's point of view, although the cross traffic is actually obscured by an obstacle.
  • According to an advantageous embodiment, a geometry of a three-dimensional environment model is provided. The environment model image data is obtained by projecting the image data of the respective images of the various cameras onto the surface of the three-dimensional environment model. By projection onto a three-dimensional environment model, it may be possible with little computational effort to determine the environment model image data. From these three-dimensional environment model image data, if necessary, the obstacle can be easily determined.
  • According to a further advantageous embodiment, the geometry of the three-dimensional environment model is determined depending on a measurement signal of a distance sensor of the vehicle. As a result, a three-dimensional environment model can optionally be determined simply and inexpensively. Such a distance sensor is for example an ultrasonic sensor.
  • According to a further advantageous embodiment, the geometry of the three-dimensional environment model is determined depending on the image data of the captured images. In this way it may be possible to determine the environment model without additional sensors.
  • According to a further advantageous embodiment, the image data to be signaled are adjusted by means of alpha blending. This makes it possible, if necessary, simple and with little computational effort feasible to display the hidden objects.
  • According to a further advantageous embodiment, an additional information to be signaled is additionally added to the image data to be signaled. As a result, for example, the hidden objects can be highlighted and / or marked and / or additional information about the hidden objects can be displayed.
  • Embodiments of the invention are explained in more detail below with reference to the schematic drawings. Show it:
  • 1 , a vehicle with a control device and
  • 2 , A flow chart of a program that is executed in the control device.
  • Elements of the same construction or function are identified across the figures with the same reference numerals.
  • A vehicle 1 has several cameras KAM1, KAM2, KAM3, KAM4. The respective camera KAM1, KAM2, KAM3, KAM4 is designed and arranged at least part of an environment of the vehicle 1 capture. A first camera KAM1 is, for example, at the front of the vehicle 1 arranged, for example in the area of the radiator grille. A second and third camera KAM2, KAM3 are, for example, on the exterior mirrors 3 . 4 of the vehicle 1 arranged. A fourth camera KAM4 is, for example, in the rear area of the vehicle 1 arranged. By a suitable arrangement of the cameras, ie in particular the first to fourth camera KAM1, KAM2, KAM3, KAM4, the entire environment of the vehicle 1 , So in particular a panoramic view to be detected.
  • In the environment of the vehicle 1 There are one or more obstacles H. The first camera KAM1 has a first opening angle W1, which limits the image area of the first camera KAM1. The obstacle H covers a hidden area VB from the point of view of a virtual camera VKAM. The hidden area VB is detected by the first camera KAM1 because it is within the first opening angle W1.
  • It is additionally within a virtual opening angle WV of the virtual camera VKAM, which limits the image area of the virtual camera VKAM. In the hidden area VB is an object OBJ.
  • The vehicle 1 further comprises a control device 11 which is signal-coupled with the various cameras KAM1, KAM2, KAM3, KAM4 in such a way that it can receive image data DAT from images P_i acquired by the respective camera KAM1, KAM2, KAM3, KAM4. In addition, the control device 11 even with other sensors that the vehicle 1 are assigned to be signal technically coupled. Such sensors may for example be a distance sensor, which is for example ultrasound-based.
  • The control device 11 has a data and program memory and further comprises a computing unit, in the programs during operation of the vehicle 1 be processed, which are stored in particular in the data and program memory. In addition, the control device 11 preferably has an output interface via which it is signal-technically coupled, for example with an optical output unit. In this way, an image to be signaled can be signaled by means of image data PS to be signaled, specifically to a vehicle driver.
  • The control device 11 may also be referred to as an image processing device.
  • A program for image processing, in the control device 11 is processed, in a step S1 ( 2 ), in which variables can be initialized if necessary.
  • In a step S3, image data DAT of the respective images P_i detected by the various cameras KAM1, KAM2, KAM3, KAM4 are provided. Thus, the image data DAT of an image P1 acquired by the first camera KAM1, an image P_i detected by the second camera KAM2, an image P_i detected by the third camera KAM3, and an image P_i are those of the fourth Camera KAM4 was detected, provided for further processing in the subsequent steps.
  • In a step S5, a geometry of a three-dimensional environment model UM is provided. The three-dimensional environment model UM adjoins an ambient space UR in three dimensions, which also includes the vehicle 1 includes.
  • The three-dimensional environment model UM, for example, depends on a measurement signal of a distance sensor of the vehicle 1 determined and / or by image processing.
  • In a step S7, environment model image data PU are determined by projecting the image data DAT of the respective images P_i of the different cameras KAM1, KAM2, KAM3, KAM4 onto the surface of the three-dimensional environment model UM.
  • In a step S9, a position POS of the virtual camera VKAM and a viewing direction of the virtual camera VKAM are provided. These can be predetermined, for example, by the vehicle driver in the sense of a selection of various offered positions POS and / or of another unit within the control device 11 be determined.
  • In a step S11, image data PS to be signaled of an image to be signaled are determined from the perspective of the virtual camera VKAM on the environment model image depending on the environment model image data PU.
  • In a step S13, an obstacle H depicted in the image to be signalized is determined that, from the perspective of the virtual camera VKAM, obscures an object OBJ mapped in the environment model image.
  • Since the image to be signaled images the environment model image from the perspective of the virtual camera VKAM, the obstacle H is displayed on the image to be signaled and not the hidden area VB and the object OBJ which are behind the obstacle H from the point of view of the virtual camera VKAM. Thus, the imaged area of the obstacle H may possibly be easily detected, since this is the image area of the image to be signaled, on which the hidden area VB and the object OBJ would be imaged.
  • In a step S15, the image data PS to be signaled are adapted such that the hidden object OBJ is perceptible on the image to be signaled. This can be realized for example by means of alpha blending. For example, the image data PS to be signaled can be adapted in such a way that the part of the imaged obstacle H lying within the virtual opening angle WV is superimposed on the hidden area VB by means of alpha blending. Alternatively or additionally, the part of the imaged obstacle H that lies within the virtual opening angle WV can also be hidden.
  • In this way, it is possible that an object OBJ, which is actually obscured by the obstacle H from the viewpoint of the virtual camera VKAM, is displayed on the image to be signaled, so that a driver can perceive it.
  • In a step S17, the program is ended and may optionally be started again in step S1.
  • LIST OF REFERENCE NUMBERS
  • 1
    vehicle
    3, 4
    Mirrors
    11
    control device
    DAT
    image data
    H
    obstacle
    CAMERA 1-KAM4
    cameras
    OBJ
    object
    Pi
    image
    POS
    Position of the virtual camera
    PS
    to be signaled image data
    PU
    Environment Model image data
    AROUND
    environment model
    UR
    surrounding space
    VB
    covered area
    VKAM
    virtual camera
    W1
    first opening angle
    WV
    virtual opening angle

Claims (7)

  1. Method for image data processing (DAT) of respective images (P_i) acquired by a plurality of cameras (KAM1, KAM2, KAM3, KAM4) mounted on a vehicle ( 1 ), in which - depending on the image data (DAT) of the captured images (P_i) environment model image data (PU) of an environment model image are determined, - depending on the environment model image data (PU) to be signalized image data (PS) of an image to be signaled from a point of view a virtual camera (VKAM) can be determined on the environment model image, an obstacle (H), which is depicted in the image to be signalized and which, from the viewpoint of the virtual camera (VKAM), obscures an object (OBJ) which is depicted in the environment model image, and the image data (PS) to be signaled are adapted such that the hidden object (OBJ) is perceptible on the image to be signaled.
  2. The method of claim 1, wherein - a geometry of a three-dimensional environment model (UM) is provided and - The environment model image data (PU) by projecting the image data (DAT) of the respective images (P_i) of the different cameras (KAM1, KAM2, KAM3, KAM4) are determined on the surface of the three-dimensional environment model (UM).
  3. Method according to Claim 2, in which the geometry of the three-dimensional environment model (UM) is determined as a function of a measurement signal of a distance sensor of the vehicle ( 1 ).
  4. The method of claim 2 or 3, wherein the geometry of the three-dimensional environment model (UM) is determined depending on the image data (DAT) of the captured images (P_i).
  5. Method according to one of the preceding claims, in which the image data (PS) to be signaled are adapted by means of alpha blending.
  6. Method according to one of the preceding claims, in which additional information to be signaled is added to the image data (PS) to be signaled.
  7. Device for image data processing (DAT) of respective images (P_i) acquired by a plurality of cameras (KAM1, KAM2, KAM3, KAM4) mounted on a vehicle ( 1 ), the device being designed to determine ambient model image data (PU) of an environment model image as a function of the image data (DAT), image data (PS) to be signaled of an image to be signaled from the perspective of a virtual camera (VKAM) on the environment model image to determine from the environment model image data (PU), to determine an obstacle (H) imaged in the image to be signaled, which from the point of view of Virtual camera (VKAM) obscured an object (OBJ), which is mapped in the environment model image and - to adjust the image data (PS) to be signaled so that the hidden object (OBJ) on the image to be signaled is perceptible.
DE201210219735 2012-10-29 2012-10-29 Method for image processing of data of images detected by cameras arranged at e.g. outer side of vehicle, involves adapting image data to be signaled such that hidden object is perceptible on image to be signaled Pending DE102012219735A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
DE201210219735 DE102012219735A1 (en) 2012-10-29 2012-10-29 Method for image processing of data of images detected by cameras arranged at e.g. outer side of vehicle, involves adapting image data to be signaled such that hidden object is perceptible on image to be signaled

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE201210219735 DE102012219735A1 (en) 2012-10-29 2012-10-29 Method for image processing of data of images detected by cameras arranged at e.g. outer side of vehicle, involves adapting image data to be signaled such that hidden object is perceptible on image to be signaled

Publications (1)

Publication Number Publication Date
DE102012219735A1 true DE102012219735A1 (en) 2014-04-30

Family

ID=50479666

Family Applications (1)

Application Number Title Priority Date Filing Date
DE201210219735 Pending DE102012219735A1 (en) 2012-10-29 2012-10-29 Method for image processing of data of images detected by cameras arranged at e.g. outer side of vehicle, involves adapting image data to be signaled such that hidden object is perceptible on image to be signaled

Country Status (1)

Country Link
DE (1) DE102012219735A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102014008687A1 (en) * 2014-06-12 2015-12-17 GM Global Technology Operations LLC (n. d. Ges. d. Staates Delaware) Method for displaying vehicle surroundings information of a motor vehicle
WO2016198059A1 (en) 2015-06-11 2016-12-15 Conti Temic Microelectronic Gmbh Method for generating a virtual image of vehicle surroundings
DE102016220894A1 (en) * 2016-10-25 2018-04-26 Conti Temic Microelectronic Gmbh A method and apparatus for generating a vehicle environment view in a vehicle
DE102017205194A1 (en) * 2017-03-28 2018-10-04 Bayerische Motoren Werke Aktiengesellschaft Method and device for providing 3D image information in a driver assistance system of a motor vehicle

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19947766A1 (en) * 1999-10-02 2001-05-10 Bosch Gmbh Robert Device for monitoring the surroundings of a parking vehicle
DE102006061230A1 (en) * 2006-12-22 2008-06-26 Robert Bosch Gmbh A method of displaying a map detail in a navigation system and navigation system therefor
DE102009007395A1 (en) * 2008-03-25 2009-10-01 Volkswagen Ag Motor vehicle i.e. car, environment map based representation providing method, involves modifying determined evidence information based on contradiction to actually determined evidence information before fusion into modified information
DE102009020300A1 (en) * 2009-05-07 2010-11-11 Bayerische Motoren Werke Aktiengesellschaft Method for displaying partially automatically determined surrounding of vehicle for user of vehicle from outlook of passenger, involves detecting automatically surrounding of vehicle with one or multiple object recognition devices
DE102009027755A1 (en) * 2009-07-16 2011-01-20 Robert Bosch Gmbh Method for assisting driving of vehicle e.g. cab vehicle, involves indicating instantaneous position of obstacle to driver of vehicle, and encrusting obstacle by analog contact display unit in vision field of driver
DE102011014699A1 (en) * 2011-03-22 2012-09-27 Audi Ag Method for operating driver assistance system for protection of motor car, involves determining recording locations of object from different positions of motor car based on three-dimensional positional data and endangerment value

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19947766A1 (en) * 1999-10-02 2001-05-10 Bosch Gmbh Robert Device for monitoring the surroundings of a parking vehicle
DE102006061230A1 (en) * 2006-12-22 2008-06-26 Robert Bosch Gmbh A method of displaying a map detail in a navigation system and navigation system therefor
DE102009007395A1 (en) * 2008-03-25 2009-10-01 Volkswagen Ag Motor vehicle i.e. car, environment map based representation providing method, involves modifying determined evidence information based on contradiction to actually determined evidence information before fusion into modified information
DE102009020300A1 (en) * 2009-05-07 2010-11-11 Bayerische Motoren Werke Aktiengesellschaft Method for displaying partially automatically determined surrounding of vehicle for user of vehicle from outlook of passenger, involves detecting automatically surrounding of vehicle with one or multiple object recognition devices
DE102009027755A1 (en) * 2009-07-16 2011-01-20 Robert Bosch Gmbh Method for assisting driving of vehicle e.g. cab vehicle, involves indicating instantaneous position of obstacle to driver of vehicle, and encrusting obstacle by analog contact display unit in vision field of driver
DE102011014699A1 (en) * 2011-03-22 2012-09-27 Audi Ag Method for operating driver assistance system for protection of motor car, involves determining recording locations of object from different positions of motor car based on three-dimensional positional data and endangerment value

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102014008687A1 (en) * 2014-06-12 2015-12-17 GM Global Technology Operations LLC (n. d. Ges. d. Staates Delaware) Method for displaying vehicle surroundings information of a motor vehicle
US9802540B2 (en) 2014-06-12 2017-10-31 GM Global Technology Operations LLC Process for representing vehicle surroundings information of a motor vehicle
WO2016198059A1 (en) 2015-06-11 2016-12-15 Conti Temic Microelectronic Gmbh Method for generating a virtual image of vehicle surroundings
DE102016220894A1 (en) * 2016-10-25 2018-04-26 Conti Temic Microelectronic Gmbh A method and apparatus for generating a vehicle environment view in a vehicle
DE102017205194A1 (en) * 2017-03-28 2018-10-04 Bayerische Motoren Werke Aktiengesellschaft Method and device for providing 3D image information in a driver assistance system of a motor vehicle

Similar Documents

Publication Publication Date Title
US9499018B2 (en) Trailer coupling assistance system with vehicle video camera
EP2974909B1 (en) Periphery surveillance apparatus and program
CN104210424B (en) Enhanced front curb viewing system
EP2763407B1 (en) Vehicle surroundings monitoring device
US20200148114A1 (en) Vehicular vision system with reduced distortion display
EP2511137B1 (en) Vehicle Surround View System
EP2660104B1 (en) Apparatus and method for displaying a blind spot
US6366221B1 (en) Rendering device
US9886636B2 (en) Enhanced top-down view generation in a front curb viewing system
US8058980B2 (en) Vehicle periphery monitoring apparatus and image displaying method
JP4636346B2 (en) Car camera calibration apparatus, method, and program
US20170193338A1 (en) Systems and methods for estimating future paths
CN103448634B (en) The dynamic reference superposed with image cropping
CN101404122B (en) Driving support device, driving support method, and computer program
DE102012102508B4 (en) Adjustment method and system of a smart vehicle imaging device
EP1972496B1 (en) Vehicle outside display system and display control apparatus
EP2058762B1 (en) Method and apparatus for generating bird's-eye image
JP4583883B2 (en) Ambient condition display device for vehicles
JP6311020B2 (en) Image composition system, image composition apparatus and image composition method therefor
EP3140813B1 (en) Method and device for the distortion-free display of an area surrounding a vehicle
JP4751939B2 (en) Car camera calibration system
US20140114534A1 (en) Dynamic rearview mirror display features
DE102009005505B4 (en) Method and device for generating an image of the surroundings of a motor vehicle
US8233045B2 (en) Method and apparatus for distortion correction and image enhancing of a vehicle rear viewing system
JP3778849B2 (en) Vehicle periphery image processing apparatus and recording medium

Legal Events

Date Code Title Description
R163 Identified publications notified
R012 Request for examination validly filed