DE102013215408A1 - A vehicle display system and method for displaying a vehicle environment - Google Patents

Abstract

The invention relates to a display system for a vehicle (5), comprising: - at least one display (51) for displaying at least one representation (60) of a vehicle environment; - At least one detection device (30), in particular a lidar detection device and / or a radar detection device, for detecting at least one object geometry of a real object in the vehicle environment; A memory device (43) for storing a plurality of object classes; A selection device (45) for selecting an object class as a function of the object geometry; - A visualization device (45) for creating at least one object image (61, 63) in dependence on the selected object class and for displaying the object image (61, 63) on the display (51).

Description

The present invention relates to a display system for a vehicle and a method for displaying a vehicle environment.

From the DE 10 2011 082 881 A1 a display system for a vehicle is known, which makes it possible to present a top view of the own vehicle in conjunction with environmental views. The system of DE 10 2011 082 881 A1 transforms images taken by side cameras so as to give a top view.

A major problem with this transformation is that the image planes of the top view and the cameras differ essentially by 90 degrees, so that only a small amount of information is available for the generation of a top view. This leads to the top view being incomplete. For example, the display system of a parked adjacent vehicle still provide relatively much information regarding the side facing the own vehicle. Concrete information regarding the design of the roof or the opposite side are not available. This means that corresponding areas can not be filled when displaying.

For the user of a corresponding display system, this results in the problem that he can not easily bring his own perception of the real world into line with the virtual world displayed in the display. In this respect, it proves to be difficult for the user to process the information displayed to him.

Furthermore, radar, ultrasound and laser systems are known which make it possible to scan the surroundings of a vehicle and present the information gained to the user or driver. For example, a distance to an object located at the rear of the vehicle can be acoustically signaled. There are also display systems that display objects relative to their own vehicle. Since the display systems only the object geometry of the obstacle is known, the objects are colored, for example, with the colors of a traffic light depending on their distance. Even with these systems, it proves to be difficult for the driver to assign individual objects displayed on a display to real objects. For example, it is difficult to distinguish a post from a lantern, so that it can easily lead to a misallocation, which may lead to an accident.

From the DE 10 2010 012 954 A1 and the DE 10 2005 024 716 A1 Driver assistance systems are known, which detect a vehicle environment while driving and try to categorize individual detected objects. This categorization of the objects should serve to better predict the behavior of the objects. A corresponding classification of objects in the area of vehicle environment visualization is not known.

The DE 10 2010 042 063 A1 describes a system in which multiple cameras are used to create a top view image (bird's eye view) of one's own vehicle and the surrounding area. Objects recognized in the environment, in particular vehicles, are classified, measured and, if necessary, replaced in color with the recognized object matching standard symbols. The use of the parking system is highlighted.

From the EP 2 107 503 A1 is a system for modeling the vehicle environment in real time known.

Starting from the DE 10 2010 042 063 A1 Therefore, the object of the present invention is to provide an improved display system. In particular, the quality of the image display should be increased. Also, the intuitive understanding of appropriate ads should be improved.

This object is achieved by a display system according to claim 1 and by a method according to claim 13.

• • mindestens eine Anzeige zum Anzeigen mindestens einer Darstellung einer Fahrzeugumgebung;
• • mindestens eine Erfassungseinrichtung, insbesondere eine Lidar-Erfassungseinrichtung und/oder eine Radar-Erfassungseinrichtung und/oder eine Ultraschall-Erfassungseinrichtung, zur Erfassung mindestens einer Objektgeometrie eines realen Objekts in der Fahrzeugumgebung;
• • eine Speichereinrichtung zum Speichern einer Vielzahl von komplexen Objektklassen für komplexe Objekte und Basisobjektklassen für Basisobjekte, wobei ein komplexes Objekt aus einer Vielzahl von Basisobjekten und/oder komplexen Objekten besteht;
• • eine Auswahleinrichtung zur Auswahl mindestens einer Basisobjektklasse und mindestens einer komplexen Objektklasse in Abhängigkeit von der mindestens einen Objektgeometrie;
• • eine Visualisierungseinrichtung zur Erstellung mindestens eines Objektabbilds in Abhängigkeit von den ausgewählten Objektklassen und zur Anzeige des Objektabbilds auf der Anzeige.

In particular, the object is achieved by a display system for a vehicle, which comprises:

• • at least one display for displaying at least one representation of a vehicle environment;
• At least one detection device, in particular a lidar detection device and / or a radar detection device and / or an ultrasound detection device, for detecting at least one object geometry of a real object in the vehicle environment;
• Memory means for storing a plurality of complex object classes for complex objects and base object classes for base objects, wherein a complex object consists of a plurality of base objects and / or complex objects;
• A selection device for selecting at least one base object class and at least one complex object class as a function of the at least one object geometry;
• • A visualization device for creating at least one object image depending on the selected object classes and for displaying the object image on the display.

One idea of the present invention is therefore to be replaced by sensors such as radar and / or or Lidar and / or stereo cameras to detect and recognize the objects around the own vehicle. Then, a classification is to take place in which at least the detected object geometries are compared with known object geometries stored in a memory device. An organization of the known objects in the storage device can take place on the basis of object classes. Once the selector has selected one or more object classes depending on the detected object geometries, the visualizer may read or compute object images from memory and display them on the display. In this respect, it is conceivable, starting from the own vehicle not detectable areas of an object. This representation is possible because the detectable information of individual real objects is sufficient to classify them. After the classification, stored knowledge about the nature of the object can be used to create an appropriate representation of the environment.

In the classification, certain elements, for example base objects and / or parts of a complex or basic object, can be detected by the sensor system. This may be a tail edge, a trunk, a rear window, a roof, a windshield, a hood and / or a front end act. Based on the design of the elements can be closed on the type or a specific feature (hatchback, notchback, ...). For example, over the width of the trunk projected on the silhouette on the type of tail can be closed (hatchback, notchback, ...). On the basis of the determined features, the appropriate object can then be selected and instantiated from the database. Preferably, the virtual vehicle is composed of several objects, complex objects and / or base objects.

For example, a 3D map of the vehicle environment can be set up. In this map, in addition to the own vehicle especially the neighboring vehicles, but also stationary objects are drawn.

In one embodiment, the visualization device instantiates a virtual object according to the object class to create the object image. In the end, a virtual world of the environment can be created. This virtual world / scene can be viewed from different perspectives by means of the display.

The display system may have appropriate input means to select or set a particular perspective.

The stored object classes may have at least one attribute that indicates, for example, the color of the virtual object and / or its contour and / or its size.

The visualization device can set the at least one attribute as a function of at least one signal supplied by at least one detection device. For example, it is conceivable to detect the object geometry of a neighboring vehicle by means of a lidar system. After classifying this vehicle in the class of vehicles, it may be desirable to color the vehicle in a suitable color. For this purpose, for example, a further detection device in the form of a camera can be provided, which supplies a color tone for coloring the virtual object. The chosen approach is to make the displayed image easier to understand. Furthermore, this approach makes it easier to identify objects on the display in reality. Conversely, objects visible to the driver can be easily associated with displayed objects.

At least one detection device can be designed to detect at least one color and / or texture of a surface and / or a partial surface of the object. This is preferably a camera. As already explained, it is possible to color a specific virtual object in a certain color. Preferably, textures are used to ensure the most realistic possible surface of the virtual objects. For example, individual objects are dyed with texture portions of the camera image or a color determination on another sensor, eg. B. Color of the doors.

The detection device may be configured to capture a video image of the real object, and the visualization device may project at least a portion of the video image onto the virtual objects. For example, scenes filmed in the near field area of the cameras can be projected onto the corresponding objects of the surroundings. The virtual objects can superimpose the filmed scenes. Theoretically, it is conceivable that the virtual objects serve as a kind of screen for filmed scenes or sections of the video images.

The memory device can be designed to store base object classes for base objects and / or complex object classes for complex objects. A complex object may consist of a plurality of base objects and / or at least one further complex object. Preferably, a corresponding relation is stored in the memory device, so that it is the Display system is possible to process appropriate information in the detection of the real environment. For example, the display system can be designed to detect certain basic objects that can be easily recognized by their object geometry and / or position, and to "assemble" complex objects based on these base objects.

For example, the display system may attempt to detect detectable tires from its own vehicle and then, upon detection of those tires, determine what kind of vehicle it is. In the end, therefore, the detected base object (eg = tire) is used to close on the overlying complex object (eg = vehicle). This functionality can be implemented, for example, in the selection device already described.

Theoretically, it is conceivable to use the number of base objects in order to draw conclusions about the object class of the complex objects.

Modeling, d. H. the selection of objects may be based on multiple images being captured.

Alternatively or additionally, the display system may attempt to determine a distance or position of the base objects relative to each other to infer a particular complex object class (eg, truck or car). In addition, extracted information can be processed by the detection device via the complex object.

As already explained, the visualization device may be configured to create a virtual world using a plurality of virtual objects and to display a view of the virtual world on the display. Particular advantages of the present invention arise when this view is essentially a bird's-eye view, since in this perspective areas not detectable by the detection devices can and must be supplemented by the stored object information.

In connection with the stored complex objects and basic objects described, the detection device can be designed to check predetermined conditions (the wheelbase of a passenger car is at least 1.50 m). For example, one or more base objects can be detected by the detection device for the selection of a complex object class, and the predetermined conditions can be checked after their detection. In this respect, the display system makes it possible to determine complex objects with a high accuracy.

At least one detection device can be designed to detect position data and / or alignment data of the at least one real object. This position data and / or alignment data can be used to position the individual objects very accurately, for example in the virtual world, and therefore to produce very realistic virtual images of reality.

At least one detection device can be designed to detect at least the position of a base object in order to detect the position and / or orientation of a complex object. For example, the positions of wheels indicate the orientation of a vehicle relatively accurately.

The stated object is further achieved by a vehicle having a display system, wherein the display system is designed in the same way as has already been described.

• • Erfassen mindestens eines Objekts in der Fahrzeugumgebung, vorzugsweise mittels mindestens einer Kamera und/oder mindestens einer Lidar-Einrichtung und/oder einer Radar-Einrichtung;
• • Vergleichen von Merkmalen des erfassen Objekts mit Merkmalen von gespeicherten Objektklassen;
• • Auswahl einer Vielzahl von Objektklasse basierend auf dem Vergleich;
• • Instanziierung der ausgewählten Objektklassen zur Erzeugung eines Objektabbilds, vorzugsweise aus einer Vogelperspektive;
• • Anzeigen des Objektabbilds als Teil einer Darstellung der Fahrzeugumgebung.

Furthermore, the object is achieved by a method for displaying a vehicle environment, the method comprising the steps:

• • detecting at least one object in the vehicle environment, preferably by means of at least one camera and / or at least one lidar device and / or a radar device;
• • comparing characteristics of the detected object with features of stored object classes;
• Selecting a plurality of object classes based on the comparison;
• Instantiating the selected object classes to produce an object image, preferably from a bird's-eye view;
• • Display the object image as part of a representation of the vehicle environment.

In one embodiment, one of the display systems already described is used to perform the method.

There are similar or identical advantages as have already been explained in connection with the display system. Preferably, in the method, complex objects are composed of a plurality of "simpler" objects.

According to the method, a plurality of objects can be detected, wherein for at least some of these objects, an object class is selected and corresponding object images are generated. Preferably, a relative position of the objects to each other can be detected and taken into account in the representation of the vehicle environment.

The instantiation of the object class / object classes may involve coloring the virtual objects in a captured color and / or projecting them include at least a portion of a video image. In this respect, the generated virtual objects are very similar to real objects.

The stated object may be further solved by a computer-readable storage medium storing executable machine instructions which cause a computer to implement one of the described methods when the machine instructions are executed.

Further advantageous embodiments will be apparent from the dependent claims.

The invention will be described by means of several embodiments, which are explained in more detail with reference to figures. Hereby show:

1 a plan view of a vehicle with multiple cameras;

2 a display system in which a display device is communicatively connected to a plurality of detectors;

3 individual components of the display device 2 ;

4 a flowchart of a method for displaying a vehicle environment;

5 a schematic representation of an image captured by a side camera; and

6 a virtual representation of a vehicle environment from a bird's eye view.

In the following description, the same reference numerals are used for the same and like parts.

1 shows a schematic plan view of a vehicle 10 with a vehicle front 12 , a passenger compartment and a vehicle rear 16 , Also shown is a driver 5 , The vehicle 10 is essentially so with detection devices 30 equipped so that the entire environment of the vehicle can be detected at a 360 degree angle.

Specifically, this is a mounted on the sides of the vehicle left side camera 31 and a right side camera 33 , which can be configured, for example, as monocameras or stereo cameras, in particular as mono wide-angle cameras. Furthermore, at the rear of the vehicle 16 another camera, namely a rear camera 35 attached, which detects the rear of the vehicle. Also this rear camera 35 can be a monocamera or a stereo camera. Furthermore, at the front of the vehicle 12 a front camera 38 arranged.

The front camera 38 and the rear camera 35 each capture a field of view of about 90 degrees, this being a front camera field of view 38 or a rear camera field of view 36 is. Alternatively, the front camera 38 and the rear camera 35 have a field of view of about 180 degrees horizontally and 145 degrees vertically. Lateral fields include a left field of view 32 the left side camera 31 and a right field of view 34 the right side camera 33 , The side cameras 31 . 33 are preferably on the outside mirrors 20 of the vehicle 10 arranged and capture the entire page area. In the exemplary embodiment, they have a very large viewing angle (greater than 180 degrees), for example 195 degrees. According to the 1 the rear camera field of view overlap 36 both with the left field of view 32 as well as with the right field of view 34 , Likewise, the front camera field of view overlaps 38 with the left field of view 32 and the right field of view 34 , Overall, it is at the in 1 shown vehicle 10 possible to obtain a video image of the entire vehicle environment. Depending on the version of the cameras, this may be a stereo or mono image. The cameras can have fish eye lenses.

As another detection device 30 includes the vehicle 10 of the 1 a radar device 39 , which by means of electromagnetic waves, the right side of the vehicle 10 scans and provides distance information to individual objects there.

Purely exemplary is the vehicle 10 of the 1 only with a radar device 39 equipped on the right side. However, it is easily possible, even on the left side of the vehicle 5 at the front of the vehicle 12 and / or at the rear of the vehicle 16 corresponding radar devices 39 provided. According to the invention, lidar devices and / or ultrasound devices and / or (existing or additional) stereo cameras can also function as described radar devices 39 take.

The vehicle 10 has a display device according to the invention 40 in the described embodiment in the cockpit of the vehicle 10 is arranged.

The display device 40 is communicative with the recording devices 30 connected (cf. 2 ). In particular, there is a system bus 50 that the display device 40 , the left side camera 31 , the right side camera 33 , the rear camera 35 , the front camera 37 and the radar facility 39 connects communicatively.

Like in the 3 shows an embodiment of the display device 40 a screen 41 , a storage device 43 for storing a plurality of object classes, a selector 45 for selecting an object class and a visualization device 47 for creating a representation of the vehicle environment and for displaying this representation on the screen 41 ,

An exemplary environment representation 60 for the screen 41 goes out of the 6 out.

An inventive method for generating a corresponding environmental representation 60 can the step 100 of image capture include ( 4 ). This is done for example by means of the right side camera 33 a video picture 70 the right side of the vehicle 10 detected. An equalized video image 70 is exemplary in the 5 shown. The video image 70 contains illustrations of a first object, namely a tree (image 71 a tree) and a second object, namely a vehicle (Figure 73 of a vehicle). The image 73 of the vehicle can be pictures 75 . 75 ' of wheels.

After the image capture step 100 may be the step of distance detection 110 follow, for example by means of the radar device 39 the right side area of the vehicle 10 is scanned. The radar device 39 can provide a distance image of the environment. This distance image can be used to measure the distances of the objects (for example, the tree and the vehicle 5 ) capture. In the distance image, similar to a video image, objects can be identified. Furthermore, geometries of an object can be derived.

In the step of detection (step 120 ) become corresponding objects in the distance image of the radar device 39 detected. In a subsequent step, a selection step 125 , the detected object geometry is determined by the selector 45 compared with stored object geometries of individual object classes. In this respect, the selection device 45 recognize the object tree and the class tree from the storage device 43 read. The generic description of the class tree in the storage device 43 may serve in a step of instantiation (step 130 ) to instantiate the object tree in a virtual space. Preferably, this step is by the visualization device 47 guaranteed. During the instantiation of the object, for example, a measured height and / or trunk width of the tree can be taken into account.

In a step of positioning 140 becomes the object in the virtual world relative to the vehicle 10 , which preferably also exists as a virtual object, positioned. For this purpose, it may be necessary for the radar device 39 relative to the vehicle 10 and / or relative to the side camera 33 calibrated. Preferably, all detection devices 30 relative to the vehicle 10 calibrated. In the described embodiment, the steps become 120 . 125 . 130 and 140 for all detected objects, including the detected vehicle (see Fig 73 out 5 ), carried out.

Optionally, in a subsequent step, namely in a step of specifying a view 150 A user can pretend a perspective in which he wants to view the virtual world. For this purpose, the display device 40 be equipped with appropriate input and output means. On the other hand, it is also conceivable that the display device 40 sets certain views by default.

The visualization device 47 calculated taking into account the given perspective in a step of outputting a representation (step 160 ) and display it on the screen 41 at. The environment representation 60 thus shows an object image 61 of the tree and an object image 63 of the detected vehicle, these object images 61 . 63 relative to an object image 65 of the vehicle 10 are positioned.

In one embodiment of the present invention, the individual virtual objects are the environment representation 60 according to sensor signals of the cameras, in particular the right side camera 33 , colored. In another embodiment, portions of the video image become 70 aufprojiziert onto the objects, with non-detectable areas of the objects are provided with correspondingly adapted image excerpts from detectable areas.

In the described embodiments, the radar device was 39 used to specify object geometries of individual objects. It is possible that the right side camera 33 - Also used as a monocamera - to determine appropriate object geometries. Furthermore, other detection devices 30 are used to specify the object geometries for selecting the object classes from the storage device 43 pretend. In this context, the radar device 39 are only used to determine the positions of the real objects and to position the corresponding virtual objects accordingly. Theoretically, it would be conceivable on the radar device 39 completely renounce.

On the other hand, it is also possible to implement the method according to the invention without the use of cameras. So the radar Facility 39 sufficient to specify an object geometry for selecting the corresponding object class. An instantiation of this object class can then take place without the presence of any color information. For the resulting display system, there is still the advantage that non-detectable surfaces taking into account the information from the memory device 43 Realistic can be displayed. So it is unlikely that the roof of a detected vehicle, for example, from the side already ends after a few centimeters. Usually, a corresponding vehicle roof has a width of at least one meter. Corresponding information may be obtained after the identification of the object class from the storage device 43 be derived. According to the invention, the virtual object is supplemented by image components which further increase the quality of the representation.

Overall, there is a much better environment representation, the driver 5 Helps in controlling the vehicle 10 To avoid mistakes.

In another or improved selection process (step 120 and or 125 ) are in the video image 70 first detected base objects, such as the wheels of the detected vehicle. For this purpose, conventional image processing operators can be used. In the end, it is possible the images 75 . 75 ' the wheels in the video image 70 and assign corresponding base object classes. These base object classes may be stored in the memory device 43 be deposited, that these are usually used by complex object classes - for example, a vehicle. The display device 40 can therefore try to use the basis objects to infer conclusions about the potentially existing complex object vehicle. In this case, for example, determining the wheelbase can be helpful. Likewise, the basis objects wheels are particularly well suited to determine the orientation of the complex virtual object vehicle depending on their position. As already explained, corresponding conditions or additional information in the memory device 43 be deposited to the environment representation 60 continue to improve or accelerate its creation.

The individual embodiments described can be combined in various ways according to the invention.

LIST OF REFERENCE NUMBERS

5

driver

10

vehicle

12

vehicle front

16

vehicle rear

30

detection devices

31

Left side camera

32

Left field of view

33

Right side camera

34

Right field of view

35

rear view camera

36

Rear camera field of view

37

front camera

38

Front camera field of view

39

Radar equipment

40

display

41

screen

43

memory device

45

selector

47

visualiser

50

system

60

environment representation

61

Object image of a tree

63

Object image of a vehicle

65

Object image of the vehicle 10

70

video image

71

Image of a tree

73

Image of a vehicle

75, 75 '

Image of a wheel

100

Step of image capture

110

Step of distance detection

120

Step of detection of objects

125

Step of selecting object classes

130

Step of instantiating objects

140

Step of positioning objects

150

Step of specifying a view

160

Step of outputting an environment representation

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102011082881 A1 [0002, 0002]
• DE 102010012954 A1 [0006]
• DE 102005024716 A1 [0006]
• DE 102010042063 A1 [0007, 0009]
• EP 2107503 A1 [0008]

Claims (15)

Display system for a vehicle ( 5 ), comprising: - at least one advertisement ( 51 ) for displaying at least one representation ( 60 ) a vehicle environment; At least one detection device ( 30 ), in particular a lidar detection device and / or a radar detection device, for detecting at least one object geometry of a real object in the vehicle environment; A memory device ( 43 ) for storing a plurality of base object classes for base objects and complex object classes for complex objects, wherein a complex object consists of a plurality of base objects and / or complex objects; - a selection device ( 45 ) for selecting at least one base object class and at least one complex object class as a function of the at least one object geometry; A visualization device ( 45 ) for creating at least one object image ( 61 . 63 ) using the selected object class and displaying the object image ( 61 . 63 ) on the display ( 51 ). Display system according to claim 1, characterized in that the visualization device ( 45 ) for creating the object image ( 61 . 63 ) instances at least one virtual object according to the object classes. Display system according to one of the preceding claims, characterized in that at least one object class has at least one attribute, in particular a color attribute, contour attribute and / or size attribute, and the visualization device ( 45 ) the at least one attribute in dependence on at least one of the detection device ( 30 ) signal is received. Display system according to one of the preceding claims, characterized in that at least one detection device ( 30 ) is adapted to detect at least one color and / or texture of a surface and / or a partial surface of the object. Display system according to one of the preceding claims, characterized in that at least one detection device ( 30 ) is adapted to capture a video image of the real object and the visualization device ( 45 ) projects at least a portion of the video image onto the virtual object. Display system according to one of the preceding claims, characterized in that the memory device ( 43 ) stores data on a plurality of the object classes, which at least partially describe the three-dimensional body of the object and the visualization device ( 47 ) the object class for creating the object image ( 61 . 63 instantiated). Display system according to one of the preceding claims, characterized in that the memory device ( 43 ) Stores base object classes for base objects and complex object classes for complex objects. Display system according to one of the preceding claims, characterized in that the memory device ( 43 ) stores conditions on the base object classes and / or complex object classes that preferably indicate a relation between the base objects and the complex objects. Display system according to one of the preceding claims, characterized in that the visualization device ( 47 ) is adapted to create a virtual world using a plurality of virtual objects and a view on the virtual world, in particular from a bird's-eye view, on the display ( 51 ). Display system according to one of the preceding claims, characterized in that for the selection of a complex object class, for example a complex object class that describes a vehicle, base objects detected by a detection device and preferably for the complex object class predetermined conditions are checked. Display system according to one of the preceding claims, characterized in that at least one detection device ( 30 ) is adapted to capture position data and / or orientation data of the at least one real object. Display system according to one of the preceding claims, characterized in that at least one detection device ( 30 ) is adapted, for detecting the position and / or the orientation of a complex object, to detect at least the position of a base object. Method for displaying a vehicle environment, in particular by means of a display system ( 40 ) according to one of claims 1 to 12, comprising the steps: - detecting at least one object in the vehicle environment, preferably by means of at least one camera ( 31 . 33 . 35 . 37 ) and / or at least one lidar device and / or one radar device; Comparing features of the detected object with features of stored object classes; Selection of a plurality of object classes based on the comparison; Instantiating the selected object classes to create an object image ( 61 . 63 ), preferably from a bird's-eye view; - Display the object image ( 61 . 63 ) as part of a representation ( 60 ) of the vehicle environment. A method according to claim 13, characterized in that a plurality of objects detected, a plurality of object classes selected and a plurality of object images ( 61 . 63 ), wherein preferably at least one relative position of the objects is detected relative to one another and in the representation ( 60 ) of the vehicle environment is displayed. Method according to claim 13 or 14, characterized in that the instantiation of the object classes comprises coloring the objects in a detected color and / or projecting at least one section of a video image ( 70 ).

Images