EP2719173A1 - Vehicle camera system and method for providing a continuous image of the vehicle surroundings - Google Patents

Vehicle camera system and method for providing a continuous image of the vehicle surroundings

Info

Publication number
EP2719173A1
EP2719173A1 EP12727346.4A EP12727346A EP2719173A1 EP 2719173 A1 EP2719173 A1 EP 2719173A1 EP 12727346 A EP12727346 A EP 12727346A EP 2719173 A1 EP2719173 A1 EP 2719173A1
Authority
EP
European Patent Office
Prior art keywords
image
vehicle
camera
camera system
processing unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP12727346.4A
Other languages
German (de)
French (fr)
Inventor
Wolfgang Niehsen
Thomas Engelberg
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Robert Bosch GmbH
Original Assignee
Robert Bosch GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch GmbH filed Critical Robert Bosch GmbH
Publication of EP2719173A1 publication Critical patent/EP2719173A1/en
Withdrawn legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture

Definitions

  • the present invention relates to a vehicle camera system and a method for providing images of the vehicle surroundings by means of at least one camera.
  • the invention relates to a vehicle camera system and a
  • the invention also relates to a vehicle with such a vehicle camera system.
  • multi-camera systems for all-round or top view representation (surround-view or top-view representation) are known which comprise imaging systems based on four wide-angle camera modules and a central processing unit for image synthesis.
  • FIG. 1 shows a vehicle 10 with such a multi-camera system 20 with four cameras 30, 40, 50, 60 and a central processing unit
  • Control unit 90 for surround view display In general, a front camera 30 in the grille, in each case a side camera 40, 50 integrated in the two exterior mirrors and a rear camera 60 at the rear of the vehicle in the grip of the trunk lid.
  • the camera modules 30, 40, 50, 60 are each equipped with an optical system which allows a horizontal opening angle of more than 180 ° (fish-eye optics), so that with the four cameras 30, 40, 50, 60 a gapless
  • the camera signals are analog, such as via NTSC (Digital Television Systems Committee) digital lossy encoding such as MJPEG (Moving Joint Photographic Experts Group) or MPEG4 (Moving Picture Experts or H.264 Group) or digitally without data compression , as for example by LVDS (Low Voltage Differential Signaling), transmitted.
  • NTSC Digital Television Systems Committee
  • digital lossy encoding such as MJPEG (Moving Joint Photographic Experts Group) or MPEG4 (Moving Picture Experts or H.264 Group) or digitally without data compression , as for example by LVDS (Low Voltage Differential Signaling), transmitted.
  • Analog surround-view systems (surround view systems) have one
  • dynamic artifacts contain coding artifacts (block artifacts, time-varying image resolution, etc.).
  • Image data compensation the components for connecting the digital camera modules to the central processing unit are very expensive.
  • Camera modules including the basic characteristic in the form of an intelligent reversing camera detectable.
  • Preprocessing also includes identifying overlapping portions of the video images and non-intersecting ones
  • Image display unit is brought. To avoid the e.g. In road traffic necessary requirements for availability and speed of presentation of the video images to be fair, the signal transmission between the individual components must be made via a high-speed bus.
  • the disadvantage here is that the image processing units must communicate with each other in the identification of overlapping image areas and exchange and preprocess very large amounts of data in a very short time. To have a fast data exchange of the big ones
  • the image processing units To enable data volumes between the image processing units, the image processing units must communicate with each other by means of a
  • the disadvantage here is that the image processing units must exchange a very large amount of data with the graphics unit in a very short time, so they the data with respect to the overlapping image areas for the correlation and
  • the vehicle camera system according to the invention for providing a complete image of the vehicle environment comprises at least one camera which is designed to provide images of the vehicle surroundings. Furthermore, the vehicle camera system according to the invention comprises a local
  • Image processing unit which is associated with the at least one camera and adapted to perform a non-linear transformation of at least one captured by the camera image of the vehicle environment in at least one corresponding field corresponding to a predefined image section of the image of the vehicle environment.
  • vehicle camera system according to the invention comprises a central
  • Image processing unit which is designed to take over the partial image generated by the at least one local image processing unit by means of the nonlinear transformation and to insert it into an image of the vehicle environment to produce a complete overall image of at least one region of the vehicle surroundings.
  • a method is further provided for providing a seamless image of the vehicle environment, which by means of a
  • Car camera system is performed.
  • at least one image of the vehicle surroundings is recorded by means of at least one camera.
  • the at least one recorded image by means of a non-linear transformation in at least one
  • Partial image that corresponds to a predefined image section of the image
  • Vehicle environment corresponds, transformed. Furthermore, the at least one partial image generated by means of the nonlinear transformation is converted into an image of the
  • Vehicle environment inserted to generate a gapless overall picture of at least a portion of the vehicle environment.
  • the non-linear transformation serves in particular to generate the partial image of the image displayed on a display, e.g. one
  • Vehicle display to be displayed overall image, ie a reduction of the recorded data on the displayed.
  • the invention makes the scalability of an inventive
  • Vehicle camera system of, for example, a reversing camera up to an inventive intelligent vehicle camera system with preferably four or more individual cameras allows.
  • the at least one local area is a particularly advantageous embodiment of the vehicle camera system according to the invention.
  • Image processing device designed to perform an image interpretation of at least one of the associated camera captured image of the vehicle environment by means of an object detection.
  • Data reduction by at least an order of magnitude, is the expected network system load of the entire system also by about one
  • the invention enables an efficient realization of a vehicle camera system according to the invention (such as a surround view system) with high image quality.
  • the invention further provides the modular expandability of a
  • vehicle multi-camera system such as a surround view system
  • the at least one local area network In a particularly advantageous embodiment of the vehicle multi-camera system according to the invention, the at least one local area network
  • Image processing unit connected to the central image processing unit via a network system, which is a bus system and / or a
  • Ethernetsystem with a transmission data rate of in particular not more than 100 Mbit / s covers.
  • Ethernet connections for local wired networks Since the data reduction achieved by the invention is considerable, it is also possible to use very cost-effective crosslinking systems which have a
  • the invention makes it possible to use simple 100 Mbit / s Ethernet bus systems in motor vehicles without the otherwise usual data compensation or data compression, which represents a considerable advantage in terms of cost and feasibility of functions with image interpretation.
  • Vehicle multi-camera system equipped with an optics that allows a horizontal opening angle of more than 180 °.
  • Vehicle camera system designed to, the
  • each one camera associated image processing units by means of nonlinear transformation generated fields and another field with the top view of the vehicle to a complete overall picture of the vehicle and its entire
  • a high-quality image of the vehicle environment can be provided in a simple manner from the perspective of a virtual camera above the vehicle (bird's-eye view).
  • According to the invention is further a vehicle that with a
  • the equipped vehicle camera system according to the invention is provided.
  • the driver of such a vehicle can be warned early due to the vehicle camera system according to the invention when detecting an obstacle.
  • the vehicle camera system according to the invention can be designed to intervene in the vehicle dynamics when detecting an obstacle. As a result, the risk of collision of the vehicle is significantly reduced, especially when parking.
  • FIG. 1 shows a vehicle with a vehicle multi-camera system according to the
  • Figure 2 is an overall view of the vehicle with its entire
  • Vehicle environment from the bird's eye view which was generated by means of a vehicle camera system according to the invention according to a first embodiment of the invention
  • FIG. 3 is a block diagram of an inventive
  • FIG. 4 is a block diagram of a vehicle system according to the invention according to a second embodiment of the invention.
  • FIG. 2 shows an overall picture 71 of the vehicle with its entire
  • the invention has been produced and brought to display on a display 70.
  • Embodiment of the invention comprises a plurality of intelligent camera modules, each comprising a highly integrated computing unit for decentralized image signal processing and image interpretation.
  • the arithmetic unit is either integrated directly on the imaging Imager module or is integrated as a separate component on the imager board or the imager PCB (Printed Circuit Board) or on one or more additional PCBs in the camera module.
  • Arithmetic unit can be used, for example, as a DSP (Digital Signal Processor), Field Programmable Gate Array (FPGA), Application Specific Standard Product (ASSP), Application Specific Integrated Circuit (ASIC) or SoC (System-on-Chip) ) consisting of two or more DSP (Digital Signal Processor), Field Programmable Gate Array (FPGA), Application Specific Standard Product (ASSP), Application Specific Integrated Circuit (ASIC) or SoC (System-on-Chip) ) consisting of two or more DSP (Digital Signal Processor), Field Programmable Gate Array (FPGA), Application Specific Standard Product (ASSP), Application Specific Integrated Circuit (ASIC) or SoC (System-on-Chip) ) consisting of two or more DSP (Digital Signal Processor), Field Programmable Gate Array (FPGA), Application Specific Standard Product (ASSP), Application Specific Integrated Circuit (ASIC) or SoC (System-on-Chip) ) consisting of two or more
  • the arithmetic unit makes in addition to the image interpretation in particular a non-linear transformation of the input image to generate the
  • FIG. 2 shows the display representation (overall picture) 71 of the surround view system (vehicle camera system) according to the invention with superimposition of the
  • the image signal to be displayed is formed of five approximately equal components, i. from the static plan view (top view) 1 1 of the own vehicle as well as from the image sections
  • 800x600 pixels are followed by a maximum frame size of only 96,000 pixels, at a frame rate of 25 fps and ⁇
  • FIG. 3 shows a block diagram of an intelligent inventive device
  • the vehicle camera system 20 includes four smart ones
  • Camera modules 30, 40, 50, 60 each having a (not shown) integrated computing unit (image processing unit).
  • image processing unit image processing unit
  • Camera modules 30, 40, 50, 60 only two camera modules 30, 60 are shown in FIG.
  • the camera modules 30 ... 60 are connected via Ethernet 81.
  • the channel 81 can also be used to transmit vehicle data or other sensor data from the central processing unit
  • Image processing device 90 to the individual camera modules 30 ... 60 are used. This opens, among other things, the opportunity to
  • Data synchronization for example, synchronization of local timers
  • the display 70 can optionally be controlled via the connection 82 analog (NTSC) or digital (LVDS), wherein the displayed image signal in the central processing unit (image processing unit) 90 on the basis of the individual image sections (patches) composed and optionally with other information (text , Data, symbols, etc).
  • the vehicle camera system 20 according to the invention also includes a plurality of ultrasonic sensors, of which only two ultrasonic sensors 31, 61 are shown.
  • an intelligent vehicle camera system 20 for the efficient realization of a
  • the arithmetic unit 90 has access via the CAN bus (Car Area Network Bus) 83 to vehicle data such as steering angle, wheel speeds and inertial sensors.
  • CAN bus Car Area Network Bus
  • Figure 4 shows a block diagram of a vehicle system according to the invention according to a second embodiment of the invention.
  • the display 70 of the vehicle camera system 20 according to the invention according to the second embodiment of the invention can also be connected via Ethernet 81 to the central processing unit (image processing unit) 90.

Abstract

The invention relates to a vehicle camera system for providing a continuous image of the vehicle surroundings by means of at least one camera which is designed to produce images of the vehicle surroundings. The claimed vehicle camera system comprises at least one local image processing unit, which is associated with the at least one camera and designed to carry out a non-linear transformation of at least one image of the vehicle surroundings, captured by the camera, in at least one partial image which corresponds to a predefined image section of image of the vehicle surroundings. Said claimed vehicle camera system also comprises a central image processing unit which is designed to take over the partial image generated by the at least one local image processing unit by means of the non-linear transformation and to insert it into an image of the vehicle surroundings for producing a continuous overall image of at least one area.

Description

_  _
Beschreibung description
Titel title
Fahrzeugkamerasystem und Verfahren zur Bereitstellung eines lückenlosen Bildes der Fahrzeugumgebung  A vehicle camera system and method for providing a seamless image of the vehicle environment
Stand der Technik State of the art
Die vorliegende Erfindung betrifft ein Fahrzeugkamerasystem und ein Verfahren zur Bereitstellung von Bildern der Fahrzeugumgebung mittels mindestens einer Kamera. Insbesondere betrifft die Erfindung eine Fahrzeugkamerasystem und einThe present invention relates to a vehicle camera system and a method for providing images of the vehicle surroundings by means of at least one camera. In particular, the invention relates to a vehicle camera system and a
Verfahren, bei denen ein zusammengesetztes, lückenloses Bild zumindest eines Bereichs der Fahrzeugumgebung bereitgestellt wird. Auch betrifft die Erfindung ein Fahrzeug mit einem solchen Fahrzeugkamerasystem. Aus dem Stand der Technik sind Multikamerasysteme zur Rundumsicht- beziehungsweise Draufsichtdarstellung (Surround-View- beziehungsweise Top- View-Darstellung) bekannt, die bildgebende Systeme auf der Basis von vier Weitwinkelkameramodulen und einer zentralen Recheneinheit zur Bildsynthese umfassen. Figur 1 zeigt ein Fahrzeug 10 mit einem solchen Multikamerasystem 20 mit vier Kameras 30, 40, 50, 60 und einer zentralen Recheneinheit A method of providing a composite, seamless image of at least a portion of the vehicle environment. The invention also relates to a vehicle with such a vehicle camera system. From the state of the art, multi-camera systems for all-round or top view representation (surround-view or top-view representation) are known which comprise imaging systems based on four wide-angle camera modules and a central processing unit for image synthesis. FIG. 1 shows a vehicle 10 with such a multi-camera system 20 with four cameras 30, 40, 50, 60 and a central processing unit
(Steuergerät) 90 zur Surround-View-Darstellung. Im Allgemeinen sind dazu eine Frontkamera 30 im Kühlergrill, jeweils eine Seitenkamera 40, 50 in den beiden Außenspiegeln sowie eine Heckkamera 60 am Fahrzeugheck im Bereich der Griffmulde des Kofferraumdeckels integriert.  (Control unit) 90 for surround view display. In general, a front camera 30 in the grille, in each case a side camera 40, 50 integrated in the two exterior mirrors and a rear camera 60 at the rear of the vehicle in the grip of the trunk lid.
Zur Synthese der Abbildung des Fahrzeugumfelds aus der Perspektive einer virtuellen Kamera über dem Fahrzeug (Vogelperspektive) werden die einzelnen Kamerabilder der vier verschiedenen Teile 35, 45, 55, 65 der To synthesize the image of the vehicle environment from the perspective of a virtual camera above the vehicle (bird's eye view), the individual camera images of the four different parts 35, 45, 55, 65 of the
Fahrzeugumgebung zu der zentralen Recheneinheit (Bildverarbeitungseinheit) 90 übertragen, mit einem geeigneten Abbildungsmodel transformiert und zu einemVehicle environment to the central processing unit (image processing unit) 90, transformed with a suitable imaging model and to a
Gesamtbild, welches das eigene Fahrzeug 10 als zentrale Bildelement enthält, zusammengesetzt und zur Anzeige auf einem Display 70 gebracht. Die Kameras 30, 40, 50, 60 und die zentrale Recheneinheit 90 sind mittels Videoleitungen 80 miteinander verbunden. Overall image, which contains the own vehicle 10 as a central pixel, assembled and brought to display on a display 70. The cameras 30, 40, 50, 60 and the central processing unit 90 are interconnected by means of video lines 80.
Die Kameramodule 30, 40, 50, 60 sind jeweils mit einer Optik ausgestattet, welche einen horizontalen Öffnungswinkel von mehr als 180° ermöglicht (Fish- Eye-Optik), sodass mit den vier Kameras 30, 40, 50, 60 eine lückenlose The camera modules 30, 40, 50, 60 are each equipped with an optical system which allows a horizontal opening angle of more than 180 ° (fish-eye optics), so that with the four cameras 30, 40, 50, 60 a gapless
Abdeckung des gesamten Fahrzeugumfelds erreicht werden kann. Cover the entire vehicle environment can be achieved.
Die Kamerasignale werden analog, wie zum Beispiel über NTSC (National Television Systems Committee), digital mit verlustbehafteter Codierung, wie zum Beispiel mittels MJPEG (Moving Joint Photographic Experts Group) oder MPEG4 (Moving Picture Experts oder H.264 Group) oder digital ohne Datenkompression, wie zum Beispiel mittels LVDS (Low Voltage Differential Signaling), übertragen. The camera signals are analog, such as via NTSC (Digital Television Systems Committee) digital lossy encoding such as MJPEG (Moving Joint Photographic Experts Group) or MPEG4 (Moving Picture Experts or H.264 Group) or digitally without data compression , as for example by LVDS (Low Voltage Differential Signaling), transmitted.
Analoge Rundumsichtsysteme (Surround-View-Systeme) haben eine Analog surround-view systems (surround view systems) have one
beschränkte Auflösung und Signaldynamik und bieten deswegen nur eine begrenzte Bildqualität. limited resolution and signal dynamics and therefore offer only a limited image quality.
Im Fall von digitalen Rundumsichtsystemen mit verlustbehafteter Bildcodierung treten bei dynamischen Szeneninhalten Codierartefakte (Blockartefakte, zeitvariable Bildauflösung, usw.) auf. In the case of digital all-round vision systems with lossy image coding, dynamic artifacts contain coding artifacts (block artifacts, time-varying image resolution, etc.).
Im Falle eines Rundumsichtsystems (Surround-View-System) ohne In the case of a surround view system without
Bilddatenkompensation sind die Komponenten zur Anbindung der digitalen Kameramodule an die zentrale Recheneinheit sehr kostenintensiv. Image data compensation, the components for connecting the digital camera modules to the central processing unit are very expensive.
Es ist ferner eine fehlende bzw. unzureichende Erweiterbarkeit eines bekannten, anzeigenden Rundumsichtsystems zu einem intelligenten It is also a lack of expandability of a known, displaying all-round visibility system to an intelligent one
Umfelderfassungssystem mit Bildinterpretation zur Objekterkennung feststellbar. Surround detection system with image interpretation for object detection detectable.
Es ist weiterhin eine fehlende bzw. unzureichende Skalierbarkeit eines bekannten Rundumsichtsystems bezüglich der Anzahl der verwendeten There is also a lack of or insufficient scalability of a known all-round visibility system with respect to the number of used
Kameramodule einschließlich der Basisausprägung in Form einer intelligenten Rückfahrkamera feststellbar. Camera modules including the basic characteristic in the form of an intelligent reversing camera detectable.
Aus dem Dokument DE 101 64 516 A1 ist ein Fahrzeugrückblicksystem bekannt, bei dem eine rückwärtige Panoramasicht über mindestens zwei Kameras erzeugt „ From the document DE 101 64 516 A1 a vehicle rear view system is known, in which a rear panoramic view is generated by at least two cameras "
wird, deren Videobilder mittels einer zentralen digitalen Grafikeinheit their video images by means of a central digital graphics unit
zusammengefügt werden. Die Videobilder werden mittels separater be joined together. The video images are separated by means of
Bildverarbeitungseinheiten für verschiedene Kameras zugleich digital Image processing units for different cameras simultaneously digital
vorverarbeitet. Die Vorverarbeitung beinhaltet auch eine Identifizierung von sich überschneidenden Teilen der Videobilder und sich nicht überschneidendenpreprocessed. Preprocessing also includes identifying overlapping portions of the video images and non-intersecting ones
Teilen der Videobilder. Um die sich überschneidenden Teile der Videobilder identifizieren zu können, ist es erforderlich, dass die Bildverarbeitungseinheiten miteinander kommunizieren. Die für jede Kamera separat erfassten, nicht- überlappenden Bereiche der Videobilder werden dann durch eine Festkorrektur jeweils mittels der entsprechenden Bildverarbeitungseinheit in ihrer Bildposition verschoben und nach einer Verschiebung in einem Speicher zur Anzeige bereitgestellt. Die überlappenden Bildbereiche werden dann mittels eines von der zentralen Grafikeinheit bereitgestellten, rechenintensiven Prozesses korreliert, durch Triangulation interpoliert, zugeordnet und versatzkorrigiert und Share the video images. In order to identify the overlapping parts of the video images, it is necessary that the image processing units communicate with each other. The non-overlapping regions of the video images acquired separately for each camera are then shifted in their image position by a fixed correction in each case by means of the corresponding image processing unit and provided for display after a shift in a memory. The overlapping image areas are then correlated by means of a computationally intensive process provided by the central graphics unit, interpolated by triangulation, assigned and offset corrected, and
zusammengefügt, wonach das entstandene Gesamtbild zur Darstellung auf dermerged, after which the resulting overall picture for presentation on the
Bildanzeigeeinheit gebracht wird. Um den z.B. im Straßenverkehr notwendigen Anforderungen an Verfügbarkeit und Darstellungsgeschwindigkeit der Videobilder gerecht werden zu können, muss die Signalübermittlung zwischen den einzelnen Komponenten über einen Hochgeschwindigkeitsbus erfolgen. Image display unit is brought. To avoid the e.g. In road traffic necessary requirements for availability and speed of presentation of the video images to be fair, the signal transmission between the individual components must be made via a high-speed bus.
Nachteilig dabei ist, dass die Bildverarbeitungseinheiten bei der Identifizierung von sich überlappenden Bildbereichen untereinander kommunizieren müssen und sehr große Datenmengen in kürzester Zeit untereinander austauschen und vorverarbeiten müssen. Um einen schnellen Datenaustausch der großen The disadvantage here is that the image processing units must communicate with each other in the identification of overlapping image areas and exchange and preprocess very large amounts of data in a very short time. To have a fast data exchange of the big ones
Datenmengen zwischen den Bildverarbeitungseinheiten ermöglichen zu können, müssen die Bildverarbeitungseinheiten miteinander mittels eines To enable data volumes between the image processing units, the image processing units must communicate with each other by means of a
kostenintensiven Hochgeschwindigkeitsbusses verbunden werden. Nachteilig dabei ist auch, dass die Bildverarbeitungseinheiten in kürzester Zeit auch eine sehr große Datenmenge mit der Grafikeinheit austauschen müssen, damit sie die Daten bezüglich der sich überlappenden Bildbereiche für die Korrelation undcostly high-speed buses are connected. The disadvantage here is that the image processing units must exchange a very large amount of data with the graphics unit in a very short time, so they the data with respect to the overlapping image areas for the correlation and
Weiterverarbeitung an die Grafikeinheit übermitteln. Die mittels der Grafikeinheit durchgeführten Korrelation und Weiterbearbeitung der sich überlappenden Bildbereiche ist mit sehr viel Rechenaufwand verbunden. Um einen schnellen Datenaustausch der großen Datenmengen zwischen den Forward processing to the graphics unit. The correlation and further processing of the overlapping image areas carried out by means of the graphics unit requires a great deal of computation. To a fast data exchange of the large amounts of data between the
Bildverarbeitungseinheiten und der Grafikeinheit ermöglichen zu können, müssen die Bildverarbeitungseinheiten auch mittels eines kostenintensiven To enable image processing units and the graphics unit to have the image processing units by means of a costly
Hochgeschwindigkeitsbusses mit der Grafikeinheit verbunden werden. Offenbarung der Erfindung High-speed bus to be connected to the graphics unit. Disclosure of the invention
Das erfindungsgemäße Fahrzeugkamerasystem zur Bereitstellung eines lückenloses Bildes der Fahrzeugumgebung umfasst mindestens eine Kamera, die dazu ausgebildet ist, Bilder der Fahrzeugumgebung bereitzustellen. Ferner umfasst das erfindungsgemäße Fahrzeugkamerasystem eine lokale The vehicle camera system according to the invention for providing a complete image of the vehicle environment comprises at least one camera which is designed to provide images of the vehicle surroundings. Furthermore, the vehicle camera system according to the invention comprises a local
Bildverarbeitungseinheit, die der mindestens einen Kamera zugeordnet und dazu ausgebildet ist, eine nichtlineare Transformation mindestens eines von der Kamera aufgenommenen Bildes der Fahrzeugumgebung in mindestens ein entsprechendes Teilbild, das einem vordefinierten Bildausschnitt des Bildes der Fahrzeugumgebung entspricht, durchzuführen. Weiterhin umfasst das erfindungsgemäße Fahrzeugkamerasystem eine zentrale Image processing unit, which is associated with the at least one camera and adapted to perform a non-linear transformation of at least one captured by the camera image of the vehicle environment in at least one corresponding field corresponding to a predefined image section of the image of the vehicle environment. Furthermore, the vehicle camera system according to the invention comprises a central
Bildverarbeitungseinheit, die dazu ausgebildet ist, das von der mindestens einen lokalen Bildverarbeitungseinheit mittels der nichtlinearen Transformation erzeugten Teilbild zu übernehmen und in ein Bild der Fahrzeugumgebung einzufügen zur Erzeugung eines lückenlosen Gesamtbildes zumindest eines Bereichs der Fahrzeugumgebung. Erfindungsgemäß wird weiterhin ein Verfahren geschaffen zur Bereitstellung eines lückenlosen Bildes der Fahrzeugumgebung, das mittels eines Image processing unit, which is designed to take over the partial image generated by the at least one local image processing unit by means of the nonlinear transformation and to insert it into an image of the vehicle environment to produce a complete overall image of at least one region of the vehicle surroundings. According to the invention, a method is further provided for providing a seamless image of the vehicle environment, which by means of a
erfindungsgemäßen Fahrzeugkamerasystems durchgeführt wird. Bei dem erfindungsgemäßen Verfahren wird mindestens ein Bild der Fahrzeugumgebung mittels mindestens einer Kamera aufgenommen. Dabei wird das mindestens eine aufgenommene Bild mittels einer nichtlinearen Transformation in mindestens einCar camera system according to the invention is performed. In the method according to the invention, at least one image of the vehicle surroundings is recorded by means of at least one camera. In this case, the at least one recorded image by means of a non-linear transformation in at least one
Teilbild, das einem vordefinierten Bildausschnitt des Bildes der Partial image that corresponds to a predefined image section of the image
Fahrzeugumgebung entspricht, transformiert. Weiterhin wird das mindestens eine mittels der nichtlinearen Transformation erzeugte Teilbild in ein Bild der Vehicle environment corresponds, transformed. Furthermore, the at least one partial image generated by means of the nonlinear transformation is converted into an image of the
Fahrzeugumgebung eingefügt zur Erzeugung eines lückenlosen Gesamtbildes zumindest eines Bereichs der Fahrzeugumgebung. Vehicle environment inserted to generate a gapless overall picture of at least a portion of the vehicle environment.
Die Unteransprüche zeigen bevorzugte Weiterbildungen der Erfindung. The dependent claims show preferred developments of the invention.
Bei dem erfindungsgemäßen Fahrzeugkamerasystem werden neben der zentralen Recheneinheit (Bildverarbeitungseinheit) weitere den Kameras zugeordnete Recheneinheiten verwendet. Insbesondere werden intelligente Kameramodule mit integrierter Recheneinheit verwendet. Das Konzept der dezentralen Intelligenz ermöglicht eine signifikante Datenreduktion ohne In the vehicle camera system according to the invention, further arithmetic units assigned to the cameras are used in addition to the central processing unit (image processing unit). In particular, intelligent camera modules with integrated computing unit are used. The concept of decentralized intelligence allows significant data reduction without
Informationsverlust, da die erforderliche nichtlineare Transformation des partiellen Bildsignals bereits in dem jeweiligen Kameramodul erfolgen kann, denn ein jeweiliges Kameramodul ist für einen bestimmten vordefinierten Information loss, since the required non-linear transformation of the partial image signal can already take place in the respective camera module, because a respective camera module is for a specific predefined
Bildausschnitt vorgesehen. Die nichtlineare Transformation dient insbesondere der Generierung des Teilbildes des auf einer Anzeige, z.B. einem Image section provided. The non-linear transformation serves in particular to generate the partial image of the image displayed on a display, e.g. one
Fahrzeugdisplay, anzuzeigenden Gesamtbildes, also einer Reduktion der aufgenommenen Daten auf die darzustellenden. Vehicle display, to be displayed overall image, ie a reduction of the recorded data on the displayed.
Durch die Erfindung wird die Skalierbarkeit eines erfindungsgemäßen The invention makes the scalability of an inventive
Fahrzeugkamerasystems von beispielsweise einer Rückfahrkamera bis zu einem erfindungsgemäßen, intelligenten Fahrzeugkamerasystem mit vorzugsweise vier oder mehreren Einzelkameras ermöglicht. Vehicle camera system of, for example, a reversing camera up to an inventive intelligent vehicle camera system with preferably four or more individual cameras allows.
Bei einer besonders vorteilhaften Ausführungsform des erfindungsgemäßen Fahrzeugkamerasystems ist die mindestens eine lokale In a particularly advantageous embodiment of the vehicle camera system according to the invention, the at least one local
Bildverarbeitungsvorrichtung dazu ausgebildet, eine Bildinterpretation mindestens eines von der zugeordneten Kamera aufgenommenen Bildes der Fahrzeugumgebung mittels einer Objekterkennung durchzuführen. Image processing device designed to perform an image interpretation of at least one of the associated camera captured image of the vehicle environment by means of an object detection.
Da auch die Interpretation des Bildsignals im Fall eines erfindungsgemäßen, intelligenten Fahrzeugkamerasystems mit Objekterkennung zu einer Since the interpretation of the image signal in the case of an inventive, intelligent vehicle camera system with object recognition to a
Datenreduktion um mindestens eine Größenordnung führt, ist die zu erwartenden Vernetzungssystemlast des Gesamtsystems ebenfalls um etwa eine Data reduction by at least an order of magnitude, is the expected network system load of the entire system also by about one
Größenordnung geringer als in dem Fall eines aus dem Stand der Technik bekannten Multikamerasystems mit zentraler Recheneinheit und uncodierter Datenübertragung. Order of magnitude lower than in the case of a known from the prior art multi-camera system with central processing unit and uncoded data transmission.
Durch die Erfindung wird eine effiziente Realisierung eines erfindungsgemäßen Fahrzeugkamerasystems (wie z.B. eines Surround-View-Systems) mit hoher Bildqualität ermöglicht. The invention enables an efficient realization of a vehicle camera system according to the invention (such as a surround view system) with high image quality.
Durch die Erfindung wird ferner die modulare Erweiterbarkeit eines The invention further provides the modular expandability of a
erfindungsgemäßen, anzeigenden Fahrzeugmultikamerasystems (wie z.B. eines Surround-View-Systems) bezüglich der Objekterkennung ermöglicht. „ according to the invention, displaying vehicle multi-camera system (such as a surround view system) with respect to the object recognition allows. "
In einer besonders vorteilhaften Ausführungsform des erfindungsgemäßen Fahrzeugmultikamerasystems ist die mindestens eine lokale In a particularly advantageous embodiment of the vehicle multi-camera system according to the invention, the at least one local
Bildverarbeitungseinheit mit der zentralen Bildverarbeitungseinheit über ein Vernetzungssystem verbunden, das ein Bussystem und/oder eine Image processing unit connected to the central image processing unit via a network system, which is a bus system and / or a
Ethernetsystem mit einer Übertragungsdatenrate von insbesondere nicht mehr als 100 Mbit/s umfasst. Ethernetsystem with a transmission data rate of in particular not more than 100 Mbit / s covers.
Da durch die Erfindung eine signifikante Datenreduktion ohne Informationsverlust erreicht wird, wird der Einsatz von kostenoptimalen Vernetzungssystemen, wie zum Beispiel von üblichen Multimedia-Bussystemen oder üblichen Since a significant data reduction without loss of information is achieved by the invention, the use of cost-optimal networking systems, such as conventional multimedia bus systems or conventional
Ethernetverbindungen für lokale, drahtgebundene Netzwerke ermöglicht. Da die durch die Erfindung erreichte Datenreduktion erheblich ist, können auch sehr kostengünstige Vernetzungssysteme verwendet werden, die eine Ethernet connections for local wired networks. Since the data reduction achieved by the invention is considerable, it is also possible to use very cost-effective crosslinking systems which have a
Übertragungsdatenrate von weit unter 100 Mbit/s aufweisen. Transmission data rate of well below 100 Mbit / s have.
Durch die Erfindung wird eine Nutzung von einfachen 100 Mbit/s Ethernet- Bussystemen in Kraftfahrzeugen ohne die sonst übliche Datenkompensation bzw. Datenkompression ermöglicht, was bezüglich Kosten und Realisierbarkeit von Funktionen mit Bildinterpretation einen erheblichen Vorteil darstellt. The invention makes it possible to use simple 100 Mbit / s Ethernet bus systems in motor vehicles without the otherwise usual data compensation or data compression, which represents a considerable advantage in terms of cost and feasibility of functions with image interpretation.
Insbesondere ist die mindestens eine Kamera des erfindungsgemäßen In particular, the at least one camera of the invention
Fahrzeugmultikamerasystem mit einer Optik ausgestattet, die einen horizontalen Öffnungswinkel von mehr als 180° ermöglicht. Vehicle multi-camera system equipped with an optics that allows a horizontal opening angle of more than 180 °.
Durch die Verwendung von Kameras mit Fish-Eye-Optik, welche einen horizontalen Öffnungswinkel von mehr als 180° aufweisen, ist es mit vier Kameras möglich, eine lückenlose Abdeckung des Fahrzeugumfelds zu erreichen. By using cameras with fish-eye optics, which have a horizontal opening angle of more than 180 °, it is possible with four cameras to achieve a complete coverage of the vehicle environment.
Vorteilhafterweise ist die zentrale Bildverarbeitungseinheit des Advantageously, the central image processing unit of
erfindungsgemäßen Fahrzeugkamerasystems dazu ausgebildet, die Vehicle camera system according to the invention designed to, the
verschiedenen von mindestens vier lokalen, jeweils einer Kamera zugeordneten Bildverarbeitungseinheiten mittels der nichtlinearen Transformation erzeugten Teilbilder und ein weiteres Teilbild mit der Draufsicht des Fahrzeuges zu einem lückenlosen Gesamtbild des Fahrzeuges und seiner gesamten various of at least four local, each one camera associated image processing units by means of nonlinear transformation generated fields and another field with the top view of the vehicle to a complete overall picture of the vehicle and its entire
Fahrzeugumgebung aus der Vogelperspektive zusammenzufügen. Mit dem erfindungsgemäßen Fahrzeugkamerasystem kann in einfacher Weise eine hochwertige Abbildung des Fahrzeugumfelds aus der Perspektive einer virtuellen Kamera über dem Fahrzeug (Vogelperspektive) bereitgestellt werden. To combine vehicle surroundings from a bird's-eye view. With the vehicle camera system according to the invention, a high-quality image of the vehicle environment can be provided in a simple manner from the perspective of a virtual camera above the vehicle (bird's-eye view).
Erfindungsgemäß wird weiterhin ein Fahrzeug, das mit einem According to the invention is further a vehicle that with a
erfindungsgemäßen Fahrzeugkamerasystem ausgestattet ist, bereitgestellt. Der Fahrer eines solchen Fahrzeuges kann aufgrund des erfindungsgemäßen Fahrzeugkamerasystems beim Erkennen eines Hindernisses frühzeitig gewarnt werden. Insbesondere kann das erfindungsgemäße Fahrzeugkamerasystem zum Eingreifen in die Fahrzeugdynamik beim Erkennen eines Hindernisses ausgebildet sein. Dadurch wird die Kollisionsgefahr des Fahrzeugs insbesondere beim Einparken deutlich reduziert. equipped vehicle camera system according to the invention is provided. The driver of such a vehicle can be warned early due to the vehicle camera system according to the invention when detecting an obstacle. In particular, the vehicle camera system according to the invention can be designed to intervene in the vehicle dynamics when detecting an obstacle. As a result, the risk of collision of the vehicle is significantly reduced, especially when parking.
Kurze Beschreibung der Zeichnungen Brief description of the drawings
Nachfolgend werden Ausführungsbeispiele der Erfindung unter Bezugnahme auf die beigefügten Figuren im Detail beschrieben. Dabei zeigen: Hereinafter, embodiments of the invention will be described in detail with reference to the accompanying drawings. Showing:
Figur 1 ein Fahrzeug mit einem Fahrzeugmultikamerasystem nach dem 1 shows a vehicle with a vehicle multi-camera system according to the
Stand der Technik,  State of the art,
Figur 2 ein Gesamtbild des Fahrzeuges mit seinem gesamten Figure 2 is an overall view of the vehicle with its entire
Fahrzeugumfeld aus der Vogelperspektive, das mittels eines erfindungsgemäßen Fahrzeugkamerasystems nach einer ersten Ausführungsform der Erfindung erzeugt wurde,  Vehicle environment from the bird's eye view, which was generated by means of a vehicle camera system according to the invention according to a first embodiment of the invention,
Figur 3 ein Blockschaltbild eines erfindungsgemäßen Figure 3 is a block diagram of an inventive
Fahrzeugkamerasystems nach der ersten Ausführungsform der Erfindung, und  Vehicle camera system according to the first embodiment of the invention, and
Figur 4 ein Blockschaltbild eines erfindungsgemäßen Fahrzeugsystems nach einer zweiten Ausführungsform der Erfindung. Figure 4 is a block diagram of a vehicle system according to the invention according to a second embodiment of the invention.
Figur 2 zeigt ein Gesamtbild 71 des Fahrzeuges mit seinem gesamten Figure 2 shows an overall picture 71 of the vehicle with its entire
Fahrzeugumfeld aus der Vogelperspektive, das mittels eines erfindungsgemäßen Fahrzeugkamerasystems (nicht dargestellt) nach einer ersten Ausführungsform „ Vehicle environment from a bird's eye view, by means of a vehicle camera system according to the invention (not shown) according to a first embodiment "
der Erfindung erzeugt wurde und zur Anzeige auf einem Display 70 gebracht wurde. The invention has been produced and brought to display on a display 70.
Das erfindungsgemäße Fahrzeugkamerasystem nach der ersten The vehicle camera system according to the invention after the first
Ausführungsform der Erfindung umfasst mehrere intelligente Kameramodule, die jeweils eine hochintegrierte Recheneinheit zur dezentralen Bildsignalverarbeitung und Bildinterpretation umfassen. Die Recheneinheit ist entweder direkt auf dem bildgebenden Imagermodul integriert oder wird als separate Komponente auf der Imager-Platine bzw. dem Imager-PCB (Printed Circuit Board) oder auch auf einer oder mehreren zusätzlichen PCBs in das Kameramodul integriert. Die Embodiment of the invention comprises a plurality of intelligent camera modules, each comprising a highly integrated computing unit for decentralized image signal processing and image interpretation. The arithmetic unit is either integrated directly on the imaging Imager module or is integrated as a separate component on the imager board or the imager PCB (Printed Circuit Board) or on one or more additional PCBs in the camera module. The
Recheneinheit kann zum Beispiel als DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), C (MikroController), ASSP (Application Specific Standard Product), ASIC (Application Specific Integrated Circuit) oder auch als SoC (System-on-Chip) bestehend aus zwei oder mehr  Arithmetic unit can be used, for example, as a DSP (Digital Signal Processor), Field Programmable Gate Array (FPGA), Application Specific Standard Product (ASSP), Application Specific Integrated Circuit (ASIC) or SoC (System-on-Chip) ) consisting of two or more
Signalverarbeitungseinheiten ausgeführt werden. Signal processing units are executed.
Die Recheneinheit leistet neben der Bildinterpretation insbesondere eine nichtlineare Transformation des Eingangsbildes zur Generierung des The arithmetic unit makes in addition to the image interpretation in particular a non-linear transformation of the input image to generate the
entsprechendes Teilbildes beziehungsweise Ausschnittes 36, 46, 56, 66 der auf dem Fahrzeugdisplay 70 anzuzeigenden Surround-View-Darstellung corresponding partial image or detail 36, 46, 56, 66 of the display on the vehicle display 70 surround view display
(Gesamtbild) 71 .  (Overall picture) 71.
Hierbei erfolgt eine signifikante Datenreduktion ohne Informationsverlust, sodass zur Vernetzung der einzelnen intelligenten Kameramodule mit der zentralen Recheneinheit auf entsprechend einfache und kostengünstige Lösungen zurückgegriffen werden kann. This results in a significant data reduction without loss of information, so that can be used for networking the individual intelligent camera modules with the central processing unit on correspondingly simple and cost-effective solutions.
Figur 2 zeigt die Displaydarstellung (Gesamtbild) 71 des erfindungsgemäßen Surround-View-Systems (Fahrzeugkamerasystem) mit Überlagerung der FIG. 2 shows the display representation (overall picture) 71 of the surround view system (vehicle camera system) according to the invention with superimposition of the
Erfassungsbereiche und Objektdetektionen eines Ultraschall basierten Park-Pilot-Detection areas and object detections of an ultrasound-based park pilot
Systems. Wie in der Figur 2 gezeigt wird, wird das darzustellende Bildsignal aus fünf etwa gleich großen Komponenten gebildet, d.h. aus der statischen Draufsicht (Top-View) 1 1 des eigenen Fahrzeuges sowie aus den Bildausschnitten System. As shown in Figure 2, the image signal to be displayed is formed of five approximately equal components, i. from the static plan view (top view) 1 1 of the own vehicle as well as from the image sections
(Patches) 36, 46, 56, 66 der in diesem Fall vier Kameramodule. Unter der Annahme einer vergleichsweise hohen Auflösung des Displays 70 von zum(Patches) 36, 46, 56, 66 of the four camera modules in this case. Assuming a comparatively high resolution of the display 70 of FIG
Beispiel 800 x 600 Bildpunkten folgt hieraus eine maximale Bildausschnittsgröße von lediglich 96000 Bildpunkten, was bei einer Bildwiederholrate von 25 fps und Λ For example, 800x600 pixels are followed by a maximum frame size of only 96,000 pixels, at a frame rate of 25 fps and Λ
einer Farbtiefe von 16 pp zu einer Übertragungsrate von 38,4 Mbit s führt. Diese Datenrate ist kompatibel mit heutigen Multimedia-Bussystemen (MOST50, MOST150) im Kraftfahrzeug und auch mit den weit verbreiteten und daher sehr kostengünstigen 100 Mbit/s Ethernet und Giga-Ethernet für lokale, a color depth of 16 pp leads to a transmission rate of 38.4 Mbit s. This data rate is compatible with today's motor vehicle multimedia bus systems (MOST50, MOST150) and also with the widespread and therefore very cost-effective 100 Mbit / s Ethernet and Giga-Ethernet for local,
drahtgebundene Netzwerke. Die Generierung des Bildausschnittes (Bildpatches)wired networks. The generation of the image section (image patches)
36, 46, 56, 66 kann hingegen auf einem hochaufgelösten Bildsignal erfolgen, da dieses nicht zur zentralen Recheneinheit (Bildverarbeitungseinheit) übertragen werden muss. Dies ist ein wesentliches Unterscheidungsmerkmal der 36, 46, 56, 66, however, can take place on a high-resolution image signal, since this does not have to be transmitted to the central processing unit (image processing unit). This is an essential distinguishing feature of
vorliegenden Erfindung im Vergleich zu konventionellen, aus dem Stand der Technik bekannten Multikamerasystemen zur Rundumsichtdarstellung. present invention in comparison with conventional, known from the prior art multi-camera systems for all-round view.
Figur 3 zeigt ein Blockschaltbild eines intelligenten erfindungsgemäßen FIG. 3 shows a block diagram of an intelligent inventive device
Fahrzeugkamerasystems (Multikamerasystem) 20 mit einem einfachen, kostenoptimalen Vernetzungssystem 80 nach der ersten Ausführungsform der Erfindung. Das Fahrzeugkamerasystem 20 umfasst vier intelligente Vehicle camera system (multi-camera system) 20 with a simple, cost-optimal networking system 80 according to the first embodiment of the invention. The vehicle camera system 20 includes four smart ones
Kameramodule 30, 40, 50, 60, die jeweils eine (nicht dargestellte) integrierte Recheneinheit (Bildverarbeitungseinheit) aufweisen. Von den vier Camera modules 30, 40, 50, 60, each having a (not shown) integrated computing unit (image processing unit). Of the four
Kameramodulen 30, 40, 50, 60 sind in der Figur 3 nur zwei Kameramodule 30, 60 dargestellt. Bei dem erfindungsgemäßen Fahrzeugkamerasystem 20 nach der ersten Ausführungsform der Erfindung sind die Kameramodule 30...60 über Ethernet 81 verbunden. Neben der bereits erläuterten verlustlosen Übertragung der Bildausschnitte (Patches) zur Surround-View-Darstellung (Gesamtbild) auf dem Display 70 kann der Kanal 81 auch zur Übertragung von Fahrzeugdaten oder weitere Sensordaten von der zentralen Recheneinheit Camera modules 30, 40, 50, 60, only two camera modules 30, 60 are shown in FIG. In the vehicle camera system 20 according to the invention according to the first embodiment of the invention, the camera modules 30 ... 60 are connected via Ethernet 81. In addition to the already described lossless transmission of the image sections (patches) for surround view presentation (overall picture) on the display 70, the channel 81 can also be used to transmit vehicle data or other sensor data from the central processing unit
(Bildverarbeitungseinrichtung) 90 zu den einzelnen Kameramodulen 30...60 genutzt werden. Dies eröffnet unter anderem die Möglichkeit zur (Image processing device) 90 to the individual camera modules 30 ... 60 are used. This opens, among other things, the opportunity to
Datensynchronisation (zum Beispiel Synchronisation lokaler Timer, Data synchronization (for example, synchronization of local timers,
Kamerasynchronisation), Sensordatenfusion (zum Beispiel videobasierte Camera synchronization), sensor data fusion (for example, video-based
Verifikation von Ultraschall-Objekthypothesen), Diagnose (zum Beispiel Verification of ultrasonic object hypotheses), diagnosis (for example
Ausfallerkennung) und Kalibrierung (geometrisch, radiometrisch). Failure detection) and calibration (geometric, radiometric).
Das Display 70 kann wahlweise über die Verbindung 82 analog (NTSC) oder digital (LVDS) angesteuert werden, wobei das angezeigte Bildsignal in der zentralen Recheneinheit (Bildverarbeitungseinheit) 90 auf der Basis der einzelnen Bildausschnitte (Patches) zusammengesetzt und gegebenenfalls mit weiteren Informationen (Text, Daten, Symbole, u.s.w.) angereichert wird. Das erfindungsgemäße Fahrzeugkamerasystem 20 umfasst auch mehrere Ultraschallsensoren, von denen nur zwei Ultraschallsensoren 31 , 61 dargestellt sind. The display 70 can optionally be controlled via the connection 82 analog (NTSC) or digital (LVDS), wherein the displayed image signal in the central processing unit (image processing unit) 90 on the basis of the individual image sections (patches) composed and optionally with other information (text , Data, symbols, etc). The vehicle camera system 20 according to the invention also includes a plurality of ultrasonic sensors, of which only two ultrasonic sensors 31, 61 are shown.
Erfindungsgemäß wird insbesondere ein intelligentes Fahrzeugkamerasystem 20 (intelligentes Multikamerasystem) zur effizienten Realisierung einer According to the invention, in particular an intelligent vehicle camera system 20 (intelligent multi-camera system) for the efficient realization of a
Rundumsichtdarstellung mit Umfeldinterpretation auf der Basis der All-round view with environment interpretation based on the
Sensordatenfusion von Ultraschallsensorik 31...61 (USS) und Videosensorik 30...60 (sCam) bereitgestellt. Die Recheneinheit 90 hat über den CAN-Bus (Car Area Network Bus) 83 Zugriff auf Fahrzeugdaten wie Lenkwinkel, Raddrehzahlen und Inertialsensorik. Die Anbindung der intelligenten Kameramodule 30...60 erfolgt per Ethernet 81 , beispielsweise über UDP (User Data Protokoll). Sensor data fusion of ultrasonic sensors 31 ... 61 (USS) and video sensors 30 ... 60 (sCam) provided. The arithmetic unit 90 has access via the CAN bus (Car Area Network Bus) 83 to vehicle data such as steering angle, wheel speeds and inertial sensors. The connection of the intelligent camera modules 30 ... 60 via Ethernet 81, for example via UDP (User Data Protocol).
Figur 4 zeigt ein Blockschaltbild eines erfindungsgemäßen Fahrzeugsystems nach einer zweiten Ausführungsform der Erfindung. Figure 4 shows a block diagram of a vehicle system according to the invention according to a second embodiment of the invention.
Anders als bei dem erfindungsgemäßen Fahrzeugkamerasystem nach der ersten Ausführungsform der Erfindung kann das Display 70 des erfindungsgemäßen Fahrzeugkamerasystem 20 nach der zweiten Ausführungsform der Erfindung auch per Ethernet 81 an die zentrale Recheneinheit (Bildverarbeitungseinheit) 90 angebunden werden. Unlike the vehicle camera system according to the invention according to the first embodiment of the invention, the display 70 of the vehicle camera system 20 according to the invention according to the second embodiment of the invention can also be connected via Ethernet 81 to the central processing unit (image processing unit) 90.

Claims

Ansprüche claims
1 . Fahrzeugkamerasystem (20) zur Bereitstellung von Bildern der 1 . Vehicle camera system (20) for providing images of
Fahrzeugumgebung mittels mindestens einer Kamera (30, 40, 50, 60), gekennzeichnet durch  Vehicle environment by means of at least one camera (30, 40, 50, 60), characterized by
mindestens eine lokale Bildverarbeitungseinheit, die der mindestens einen Kamera (30, 40, 50, 60) zugeordnet ist und dazu ausgebildet ist, eine nichtlineare Transformation von mindestens einem von der Kamera (30, 40, 50, 60) aufgenommenen Bild der  at least one local image processing unit associated with the at least one camera (30, 40, 50, 60) and configured to perform a non-linear transformation of at least one image captured by the camera (30, 40, 50, 60)
Fahrzeugumgebung in mindestens ein Teilbild (36, 46, 56, 56), das einem vordefinierten Bildausschnitt eines Gesamtbildes (71 ) entspricht, durchzuführen, und  Vehicle environment in at least one partial image (36, 46, 56, 56), which corresponds to a predefined image section of an overall image (71), perform, and
eine zentrale Bildverarbeitungseinheit (90), die dazu ausgebildet ist, das von der mindestens einen lokalen Bildverarbeitungseinheit mittels der nichtlinearen Transformation erzeugte Teilbild (36, 46, 56, 56) zu übernehmen und in das Gesamtbild (71 ) einzufügen zur Erzeugung eines lückenlosen Bildes (71 ) zumindest eines Bereichs (35, 45, 55, 65) der Fahrzeugumgebung.  a central image processing unit (90) which is adapted to take over the partial image (36, 46, 56, 56) generated by the at least one local image processing unit by means of the nonlinear transformation and to insert it into the overall image (71) in order to produce a gapless image ( 71) at least one region (35, 45, 55, 65) of the vehicle environment.
2. Fahrzeugkamerasystem (20) nach Anspruch 1 , dadurch gekennzeichnet, dass die mindestens eine lokale Bildverarbeitungsvorrichtung dazu ausgebildet ist, eine Bildinterpretation mindestens eines von der zugeordneten Kamera (30, 40, 50, 60) aufgenommenen Bildes der Fahrzeugumgebung mittels einer Objekterkennung durchzuführen. 2. Vehicle camera system (20) according to claim 1, characterized in that the at least one local image processing device is adapted to perform an image interpretation of at least one of the associated camera (30, 40, 50, 60) recorded image of the vehicle environment by means of an object detection.
3. Fahrzeugkamerasystem (20) nach einem der vorangehenden Ansprüche, dadurch gekennzeichnet, dass die mindestens eine lokale 3. Vehicle camera system (20) according to any one of the preceding claims, characterized in that the at least one local
Bildverarbeitungseinheit mit der zentralen Bildverarbeitungseinheit (90) über ein Vernetzungssystem (80) mit einer Übertragungsdatenrate von insbesondere nicht mehr 100 Mbit/s verbunden ist, das ein Bussystem (83) und/oder ein Ethernetsystem (81 ) umfasst. Fahrzeugmultikamerasystem (20) nach einem der vorangehenden Image processing unit is connected to the central image processing unit (90) via a network system (80) with a transmission data rate of in particular not more 100 Mbit / s, comprising a bus system (83) and / or an Ethernet system (81). Vehicle multi-camera system (20) according to one of the preceding
Ansprüche, dadurch gekennzeichnet, dass die mindestens eine Kamera (30, 40, 50, 60) mit einer Optik ausgestattet ist, die einen horizontalen Öffnungswinkel von mehr als 180° ermöglicht. Claims, characterized in that the at least one camera (30, 40, 50, 60) is equipped with an optics, which allows a horizontal opening angle of more than 180 °.
Fahrzeugkamerasystem (20) nach einem der vorangehenden Ansprüche, dadurch gekennzeichnet, dass die zentrale Bildverarbeitungseinheit (90) dazu ausgebildet ist, die verschiedenen von mindestens vier lokalen, jeweils einer Kamera (30, 40, 50, 60) zugeordneten Vehicle camera system (20) according to one of the preceding claims, characterized in that the central image processing unit (90) is adapted to the various of at least four local, each one camera (30, 40, 50, 60) associated
Bildverarbeitungseinheiten mittels der nichtlinearen Transformation erzeugten Teilbilder (36, 46, 56, 66) und ein weiteres Teilbild (1 1 ) mit der Draufsicht des Fahrzeuges (10) zu einem lückenlosen Gesamtbild (71 ) des Fahrzeuges (10) und seiner gesamten Fahrzeugumgebung aus der Vogelperspektive zusammenzufügen. Image processing units by means of the non-linear transformation generated partial images (36, 46, 56, 66) and another partial image (1 1) with the top view of the vehicle (10) to a complete overall image (71) of the vehicle (10) and its entire vehicle environment from the To put together bird's eye view.
Verfahren zur Bereitstellung von Bildern der Fahrzeugumgebung, das mittels eines Fahrzeugkamerasystems (20) nach mindestens einem der vorangehenden Ansprüche durchgeführt wird, dadurch gekennzeichnet, dass bei dem Verfahren mindestens ein Bild der Fahrzeugumgebung mittels mindestens einer Kamera (30, 40, 50, 60) aufgenommen wird, dass das mindestens eine aufgenommene Bild mittels einer nichtlinearen Transformation in mindestens ein Teilbild (36, 46, 56, 66), das einem vordefinierten Bildausschnitt eines Gesamtbildes (71 ) zumindest eines Teils der Fahrzeugumgebung entspricht, transformiert wird und dass das mindestens eine mittels der nichtlinearen Transformation erzeugte Teilbild (36, 46, 56, 66) in das Gesamtbild (71 ) der Fahrzeugumgebung eingefügt wird zur Erzeugung eines lückenlosen Bildes (71 ) zumindest eines Bereichs (35, 45, 55, 65) der Fahrzeugumgebung. A method for providing images of the vehicle environment, which is carried out by means of a vehicle camera system (20) according to at least one of the preceding claims, characterized in that the method at least one image of the vehicle environment by means of at least one camera (30, 40, 50, 60) was taken in that the at least one recorded image is transformed by means of a nonlinear transformation into at least one subimage (36, 46, 56, 66) which corresponds to a predefined image section of an overall image (71) of at least part of the vehicle environment, and the at least one The non-linear transformation generated partial image (36, 46, 56, 66) in the overall image (71) of the vehicle environment is inserted to produce a seamless image (71) of at least a portion (35, 45, 55, 65) of the vehicle environment.
Verfahren nach Anspruch 6, dadurch gekennzeichnet, dass das mindestens eine mittels mindestens einer Kamera (30, 40, 50, 60) aufgenommene Bild der Fahrzeugumgebung vor seiner Einfügung in das lückenlose Gesamtbild (71 ) des Fahrzeugumgebungsbereichs (36, 46, 56, 66) mittels einer Bildinterpretation anhand einer Objekterkennung vorverarbeitet wird. Verfahren nach einem der Ansprüche 6 oder 7, dadurch gekennzeichnet, dass mindestens vier verschiedene Bilder der Fahrzeugumgebung mittels von mindestens vier Kameras (30, 40, 50, 60) aufgenommen werden und die aus den vier verschiedenen Bildern der Fahrzeugumgebung mittels der nichtlinearen Transformation erzeugten Teilbilder (36, 46, 56, 66) und ein weiteres Teilbild (1 1 ) mit der Draufsicht des Fahrzeuges (10) zu einem lückenlosen Gesamtbild (71 ) des Fahrzeuges (10) und seiner gesamten Fahrzeugumgebung aus der Vogelperspektive zusammengefügt werden. Method according to Claim 6, characterized in that the at least one image of the vehicle environment recorded by means of at least one camera (30, 40, 50, 60), prior to its insertion into the complete overall image (71) of the vehicle surroundings region (36, 46, 56, 66). is pre-processed by means of an image interpretation based on an object recognition. Method according to one of claims 6 or 7, characterized in that at least four different images of the vehicle surroundings are recorded by means of at least four cameras (30, 40, 50, 60) and the partial images generated from the four different images of the vehicle environment by means of the non-linear transformation (36, 46, 56, 66) and another partial image (1 1) with the top view of the vehicle (10) to a complete overall image (71) of the vehicle (10) and its entire vehicle environment are assembled from a bird's eye view.
Fahrzeug (10) mit einem Fahrzeugkamerasystem (20) nach mindestens einem der Ansprüche 1 bis 5. Vehicle (10) with a vehicle camera system (20) according to at least one of claims 1 to 5.
EP12727346.4A 2011-06-07 2012-06-04 Vehicle camera system and method for providing a continuous image of the vehicle surroundings Withdrawn EP2719173A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102011077143A DE102011077143A1 (en) 2011-06-07 2011-06-07 A vehicle camera system and method for providing a seamless image of the vehicle environment
PCT/EP2012/060467 WO2012168170A1 (en) 2011-06-07 2012-06-04 Vehicle camera system and method for providing a continuous image of the vehicle surroundings

Publications (1)

Publication Number Publication Date
EP2719173A1 true EP2719173A1 (en) 2014-04-16

Family

ID=46275811

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12727346.4A Withdrawn EP2719173A1 (en) 2011-06-07 2012-06-04 Vehicle camera system and method for providing a continuous image of the vehicle surroundings

Country Status (4)

Country Link
EP (1) EP2719173A1 (en)
CN (1) CN103583041A (en)
DE (1) DE102011077143A1 (en)
WO (1) WO2012168170A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105187690A (en) * 2015-09-17 2015-12-23 北京华航无线电测量研究所 Image processing method for guidance, and image processing apparatus for guidance

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102013200427B4 (en) * 2013-01-14 2021-02-04 Robert Bosch Gmbh Method and device for generating an all-round view image of a vehicle environment of a vehicle, method for providing at least one driver assistance function for a vehicle, all-round view system for a vehicle
DE112014002065B4 (en) * 2013-06-26 2023-06-01 Continental Autonomous Mobility Germany GmbH mirror replacement device and vehicle
DE102013011927B4 (en) * 2013-07-17 2015-03-05 Diehl Bgt Defence Gmbh & Co. Kg Method for controlling a monitoring device
DE102013214368A1 (en) 2013-07-23 2015-01-29 Application Solutions (Electronics and Vision) Ltd. Method and device for reproducing a lateral and / or rear surrounding area of a vehicle
DE102014108684B4 (en) 2014-06-20 2024-02-22 Knorr-Bremse Systeme für Nutzfahrzeuge GmbH Vehicle with an environmental monitoring device and method for operating such a monitoring device
CN105721789B (en) * 2014-12-01 2019-09-10 中国航空工业集团公司第六三一研究所 A kind of low delay omnirange video multi-mode display control method
CN107003389A (en) * 2014-12-05 2017-08-01 奥迪股份公司 For vehicle driver assistance system and aid in vehicle driver method
DE102015208343B4 (en) * 2015-05-06 2023-09-07 Robert Bosch Gmbh Method for generating an overall image of a vehicle environment of a vehicle and corresponding device
DE102015010009A1 (en) 2015-08-05 2017-02-09 Wirtgen Gmbh Self-propelled construction machine and method for displaying the environment of a self-propelled construction machine
DE102015010011B4 (en) 2015-08-05 2020-03-19 Wirtgen Gmbh Self-propelled construction machine and method for displaying the environment of a self-propelled construction machine
DE102015119871A1 (en) * 2015-11-17 2017-05-18 Connaught Electronics Ltd. Method for operating a camera system with a plurality of image sensor elements and motor vehicle
DE102015223175A1 (en) * 2015-11-24 2017-05-24 Conti Temic Microelectronic Gmbh Driver assistance system with adaptive environment image data processing
DE102016116859A1 (en) 2016-09-08 2018-03-08 Knorr-Bremse Systeme für Nutzfahrzeuge GmbH Sensor arrangement for an autonomously operated commercial vehicle and a method for round imaging
DE102016223391A1 (en) * 2016-11-25 2018-05-30 Conti Temic Microelectronic Gmbh METHOD AND DEVICE FOR PRODUCING A VEHICLE ENVIRONMENT VIEW ON A VEHICLE
WO2018103795A1 (en) * 2016-12-06 2018-06-14 Conti Temic Microelectronic Gmbh Camera device and method for capturing a surrounding region of a vehicle in a situation-adapted manner
JP6855331B2 (en) * 2017-06-09 2021-04-07 アイシン精機株式会社 Image transfer device, diagnostic imaging device, and diagnostic imaging system
DE102017216822A1 (en) * 2017-09-22 2019-03-28 Continental Automotive Gmbh An apparatus and method for adjusting image processing based on a shape of a display device for a motor vehicle
DE102018108751B4 (en) 2018-04-12 2023-05-04 Motherson Innovations Company Limited Method, system and device for obtaining 3D information from objects
CN108839616A (en) * 2018-06-08 2018-11-20 衡阳师范学院 A kind of common auto-panorama image providing service system

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1408693A1 (en) * 1998-04-07 2004-04-14 Matsushita Electric Industrial Co., Ltd. On-vehicle image display apparatus, image transmission system, image transmission apparatus, and image capture apparatus
DE10164516A1 (en) 2001-12-19 2003-07-10 Magna Auteca Ag Weiz Electronic vehicle rearview system
DE102006003538B3 (en) * 2006-01-24 2007-07-19 Daimlerchrysler Ag Image acquisitions merging method for bird`s eye perspective, involves selecting image sections such that shadowing effects produced by moved objects are projected during transition in overall image from image section to other section
JP4321543B2 (en) * 2006-04-12 2009-08-26 トヨタ自動車株式会社 Vehicle periphery monitoring device
JP4656005B2 (en) * 2006-06-02 2011-03-23 日産自動車株式会社 Image processing system and image processing method
DE112008004279A5 (en) * 2007-02-16 2015-04-23 Autonetworks Technologies, Ltd. Vehicle-mounted video communication system and vehicle-mounted image acquisition system
EP2431226B1 (en) * 2010-09-17 2016-07-27 SMR Patents S.à.r.l. Rear view device for a motor vehicle

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2012168170A1 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105187690A (en) * 2015-09-17 2015-12-23 北京华航无线电测量研究所 Image processing method for guidance, and image processing apparatus for guidance
CN105187690B (en) * 2015-09-17 2019-08-09 北京华航无线电测量研究所 Image processing method and device applied to guidance

Also Published As

Publication number Publication date
WO2012168170A1 (en) 2012-12-13
DE102011077143A1 (en) 2012-12-13
CN103583041A (en) 2014-02-12

Similar Documents

Publication Publication Date Title
WO2012168170A1 (en) Vehicle camera system and method for providing a continuous image of the vehicle surroundings
EP2431226B1 (en) Rear view device for a motor vehicle
DE102006003538B3 (en) Image acquisitions merging method for bird`s eye perspective, involves selecting image sections such that shadowing effects produced by moved objects are projected during transition in overall image from image section to other section
EP2623374B1 (en) Vision system for commercial vehicles for the display of the statutory fields of view of a main mirror and a wide-angle mirror
EP2484558B1 (en) Display device for fields of vision of a commercial vehicle
EP1211132B1 (en) Device and method for monitoring the environment of a vehicle
DE102006007493B4 (en) Device for viewing the periphery of a vehicle
DE102011077398B4 (en) Vehicle camera system for providing a complete image of the vehicle environment and the corresponding method
DE102012025322B4 (en) Motor vehicle with camera monitor system
DE102017130566A1 (en) Visual system for detecting a vehicle environment
DE102017100004A1 (en) Method for providing at least one information from a surrounding area of a motor vehicle, display system for a motor vehicle, driver assistance system for a motor vehicle and motor vehicle
DE102008046214A1 (en) Environment monitoring method for vehicle e.g. commercial motor vehicle, utilized for transporting goods, involves determining and displaying relevant surrounding regions based on distance between vehicle and obstacle
DE102013209418A1 (en) Vehicle collision warning system and method
DE112008000089T5 (en) Apparatus and method for monitoring the environment of a vehicle
DE102008046544A1 (en) Vehicle e.g. passenger car, surrounding monitoring method, utilized for passenger transportation, involves displaying section of complete image for positioning of vehicle, and selecting section as function of vehicle movement direction
EP2791895A1 (en) Method for improving the object detection in multicamera systems
DE102013220477B4 (en) Method and device for correcting a pictorial representation of a vehicle environment
DE102008030104A1 (en) Vehicle i.e. tractor-trailer, surrounding monitoring method, involves assigning positions of image capturing units and weighing factor to each pixel, and determining overall picture between vehicle elements by taking into inclination angle
DE102013214369A1 (en) Method and device for reproducing a surrounding area of a vehicle
EP3106349B1 (en) Vision system for a commercial vehicle for the display of the statutory fields of view of a main mirror and a wide-angle mirror
EP1922871B1 (en) Transmission of useful data in the picture signal of a camera system
WO2014202496A1 (en) Method for displaying environment information in a vehicle
EP2312525A2 (en) Method and camera system for generating images for transfer to an external control unit
DE102017100669A1 (en) Method for detecting an environmental region of a motor vehicle with adaptation of a region of interest as a function of a trailer, computing device, camera system and motor vehicle
DE102013102207A1 (en) Camera system for a vehicle

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140107

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20150410

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20150811