DE102016223389A1 - IMAGE PROCESSING SYSTEM AND IMAGE PROCESSING METHOD - Google Patents

Abstract

The present invention discloses an image processing system (100, 200) for a vehicle (300, 400) having an image capture device (101, 201) and at least one display device (102, 202) having a communication interface (103, 203) which is formed To retrieve vehicle parameters (104, 204) indicative of a current state of the vehicle (300, 400) from the vehicle (300, 400) with a model memory (105, 205) including a vehicle model (106, 206) and a vehicle model Having a plurality of geometry objects (107-109, 207-209) for the vehicle model (106, 206), and with an image generating device (110, 210), which is designed based on the queried vehicle parameters (104, 204) corresponding geometry objects (107 - 109, 207 - 209) and to associate with the vehicle model (106, 206) to form an overall model (215) and a representation of the overall model (215) with one of the image acquisition devices (101, 201) Overlay image (114) and output on the display device (102, 202). Further, the present invention discloses a corresponding image processing method.

Description

Field of the invention

The present invention relates to an image processing system and a corresponding image processing method.

State of the art

Although the present invention will be described below mainly in connection with passenger cars, it is not limited thereto, but may be used with any type of vehicle.

In modern vehicles, a variety of assistance systems assist the driver in controlling the vehicle. For example, camera systems can display to the driver on screens within the vehicle areas of the vehicle environment that the driver otherwise can not see.

Such camera systems can not only show the driver the area outside the vehicle. Modern so-called surround-view systems can be used by the driver e.g. also display parts of the vehicle itself together with the respective sections of the vehicle environment.

Such surround view systems use the images of a large number of cameras and calculate the respective representation from these. The vehicle is inserted into the image based on a static model of the vehicle.

This type of presentation makes it difficult for the driver to assess the condition of the vehicle.

Disclosure of the invention

It is therefore an object of the present invention to enable an improved detection of the vehicle condition.

This object is achieved by an image processing system having the features of independent patent claim 1 and an image processing method having the features of independent patent claim 7.

Accordingly, it is provided:

An image processing system for a vehicle having a number, ie one or more, of image capture devices and having at least one display device with a communication interface configured to retrieve vehicle parameters indicative of a current state of the vehicle from the vehicle, comprising a model memory a vehicle model and a plurality of geometry objects for the vehicle model, and with an image generating device, which is configured based on the queried vehicle parameters corresponding geometry objects to select and associate with the vehicle model to an overall model and a representation of the overall model with an image captured by the image capture devices to overlay and output on the display device.

It is also provided:

An image processing method for a vehicle having a number, one or more, of image capture devices and having at least one display device, the method comprising querying vehicle parameters indicative of a current state of the vehicle from the vehicle, e.g. via a vehicle bus, providing a vehicle model and a plurality of geometry objects for the vehicle model, e.g. in a data memory, selecting corresponding geometry objects based on the queried vehicle parameters and associating the selected geometry objects with the vehicle model into an overall model, generating a representation of the overall model and overlaying the representation of the overall model with an image captured by the image capture devices, and outputting the with the Representation of the overall model superimposed image on the display device.

Advantages of the invention

In contrast to the static approaches mentioned above, the present invention enables dynamic generation of a representation of the vehicle.

For this purpose, a vehicle model is used as the basis for generating a vehicle representation. This base is supplemented by corresponding geometry objects that depend on the vehicle parameters of the vehicle. Vehicle parameters may be any types of parameters that occur in the vehicle, e.g. can be transmitted via a vehicle bus to the image processing system. The vehicle parameters thereby identify the current state of the vehicle and thus allow the selection of suitable geometry objects that represent the current state.

The image generation device can consequently generate an overall model from the vehicle model and the corresponding geometry objects which depicts the current state of the vehicle. Under the term vehicle model or In this context, the overall model is to be understood as a 3D model, that is, for example, as a description of the three-dimensional structure that enables rendering or raytracing of a corresponding representation.

From this virtual model, the imaging device may then be e.g. Based on rendering or raytracing techniques, create a graphical representation of the model that can be inserted into the image of a surround view system at the appropriate location. The image of the surround view system with the inserted representation of the vehicle can then be displayed to the driver of the vehicle.

The present invention thus enables a detailed representation of the vehicle, e.g. through a surround view system. A driver can thereby easily get an overview of the vehicle and its current condition.

Advantageous embodiments and further developments emerge from the dependent claims and from the description with reference to the figures.

In one embodiment, the vehicle parameters may relate to components of the vehicle that may occupy different states during operation of the vehicle. Such ingredients may e.g. be the wheels, which may have different rotational speeds and steering angle dependent steering angle. Further, such ingredients may also be used e.g. the porters, the windscreen wipers, the doors or engine or tailgates or the seats. The vehicle model may thus be e.g. the rigid parts of the body and interior of the vehicle include while movable components of the vehicle may be provided as separate geometry objects. It is understood that also movable components can be statically integrated in the vehicle model. Which components are provided as separate geometry objects and which as parts of the static vehicle model may vary from application to application.

The vehicle parameters may thus be e.g. be the steering angle or the speed of the vehicle. The vehicle parameters may also include the status and height adjustment of the piggery, e.g. on / off, mark. Other vehicle parameters may relate to the opening angle of the doors, the rear or engine door and the like. In general, vehicle parameters describe the state of the respective component of the vehicle so that the respective geometry object can be selected.

In one embodiment, at least one geometry object may be stored for each component of the vehicle. If a separate geometry object is provided for each variable component of the vehicle, the vehicle model and the geometry objects can be assembled very easily.

The geometry objects can thereby describe the geometry of the respective component of the vehicle, so that these e.g. can be drawn or rendered as a three-dimensional image. The geometry objects may also contain additional information, e.g. have textures, colors, light sources, axes of rotation, origin of coordinates, and the like.

Information about light sources may e.g. describe the intensity and orientation of a light source. Information about a rotation axis can be found in components that can rotate about one or more axes, e.g. Wheels, doors, flaps, seats, describe the axis of rotation. This allows the exact angle insertion of the geometry object into the vehicle model. Coordinate origins may be e.g. identify the point of the geometry object at which it is to be coupled to the vehicle model.

In one embodiment, a plurality, that is two or more, of geometry objects can be stored for at least one component of the vehicle. This allows a simple selection of the respective geometry object that corresponds to the specific state of a component. For example, two geometry objects can be provided for a headlight of a vehicle, wherein one of the geometry objects identifies the headlight switched off, while the other geometry object identifies the headlight switched on. Likewise, individual geometry objects may e.g. be provided for different Einlenkwinkel the wheels. Thus, the appropriate representation can be generated by a simple selection. An elaborate calculation of a rotation of a single geometry object to the wheel can then be omitted.

For example, a geometry object for a steering wheel in a left-hand drive vehicle and a geometry object for a steering wheel in a right-hand drive vehicle can also be provided. Based on the vehicle parameters then the appropriate geometry object can be selected.

In an embodiment, the image generator may be configured to perform geometric transformations of the geometry objects based on the vehicle parameters and to transform the transformed geometry objects with the geometry object Vehicle model to link to an overall model. If the individual geometry objects can be transformed dynamically, it becomes possible to describe with a single geometry object, for example, several states of the respective component of the vehicle. For example, only a single geometry object can be provided for the wheels of the vehicle. Through transformations, such as rotations and reflections, the one geometry object can form the basis for all four wheels of the vehicle. Furthermore, different states of individual wheels can also be mapped. Thus, the geometry object for the representation of a front wheel, for example, according to the current steering angle can be rotated before it is inserted into the vehicle model.

In an embodiment, the image generation device may include a state machine that may be configured to receive the vehicle parameters as inputs and to select the corresponding geometry objects. The use of a state machine allows a very simple selection of the necessary geometry objects. Thus, values of individual vehicle parameters, e.g. be translated directly into names of the respective geometry objects.

For example, a geometry object "headlight_an" can be provided for a headlight when it is switched on. Another geometry object "headlight_out" may e.g. be provided for the headlight in the off state. The state machine may then select the corresponding geometry object based on the vehicle parameter for the state of the headlamp.

It is understood that the individual components of the image processing system, e.g. the communication interface, the model memory and the imaging device, as hardware and / or software, e.g. be designed as a processor, ASIC, FPGA, CPLD, and / or as a program code, function, method or a combination of said options.

The above embodiments and developments can, if appropriate, combine with each other as desired. Further possible refinements, developments and implementations of the invention also include combinations of features of the invention which have not been explicitly mentioned above or described below with regard to the exemplary embodiments. In particular, the person skilled in the art will also add individual aspects as improvements or supplements to the respective basic form of the present invention.

list of figures

• 1 ein Blockschaltbild einer Ausführungsform eines erfindungsgemäßen Bildverarbeitungssystems;
• 2 ein Blockschaltbild einer weiteren Ausführungsform eines erfindungsgemäßen Bildverarbeitungssystems;
• 3 ein Ablaufdiagramm einer Ausführungsform eines erfindungsgemäßen Bildverarbeitungsverfahrens;
• 4 eine Ausführungsform einer erfindungsgemäßen Darstellung eines Fahrzeugs; und
• 5 eine Ausführungsform einer weiteren erfindungsgemäßen Darstellung eines Fahrzeugs.

The present invention will be explained in more detail with reference to the exemplary embodiments indicated in the schematic figures of the drawings. It shows:

• 1 a block diagram of an embodiment of an image processing system according to the invention;
• 2 a block diagram of another embodiment of an image processing system according to the invention;
• 3 a flowchart of an embodiment of an image processing method according to the invention;
• 4 an embodiment of a representation of a vehicle according to the invention; and
• 5 an embodiment of another illustration of a vehicle according to the invention.

In all figures, identical or functionally identical elements and devices have been provided with the same reference numerals, unless stated otherwise.

Embodiments of the invention

1 shows a block diagram of an image processing system according to the invention 100 for a vehicle with an image capture device 101 and with a display device 102. It will be understood that in other example applications, both multiple image capture devices and multiple display devices may be provided. The image capture device 101 captures a picture 114 which after processing by the image processing system 100 on the display device 102 can be issued. The term "image" is not just a single image to understand, it may, for example, a sequence of images or a video to be understood.

The image processing system 100 has a communication interface 103 and a model store 105 on, both with an imaging device 110 are coupled.

The communication interface 103 allows the image processing system 100 eg via a vehicle bus or other data connection vehicle parameters 104 of the vehicle. Such vehicle parameters 104 For example, the speed or the steering angle, ie physical driving parameters, may apply. The vehicle parameters 104 but may also have information about whether the vehicle is a left-hand drive or a right-hand drive vehicle, a door is open or the like, for example. The vehicle parameters 104 may relate to any component of the vehicle that may occupy different states during operation of the vehicle.

The model memory 105 has a vehicle model 106 on. Furthermore, the model memory points 105 Geometry objects 107, 108 and 109. The geometry objects 107 and 108 relate to the same component of the vehicle, which is indicated by a dashed frame shown. This means that the geometry objects 107 and 108, for example, both describe a headlight, one of which describes the headlight in the on state and one describes the headlight in the off state. In contrast, the geometry object 109 can, for example, describe a wheel of the vehicle.

The image forming device 110 selects based on the vehicle parameters 104 the corresponding geometry objects that make up the vehicle model 106 to complete.

If the vehicle parameters 104 For example, indicate that the headlight of the vehicle is turned off, the image forming device 110 select the geometry object 108. At the same time, the image forming device 110 For example, the geometry object 109 of the wheel according to a specified steering angle rotated on the vehicle model 109 attach and so create the overall model.

The image forming device 110 can then use the overall model to create a pictorial representation of the image 114 can be superimposed. For this purpose, the image forming device 110 For example, generate suitable projections of the virtual three-dimensional space with the overall model on the respective projection surface.

The driver can then in the display device 102 The vehicle not only locally correct, but also physically correct, see, for example, with wheels and turned on / off headlights.

2 shows a block diagram of another image processing system 200 , The image processing system 200 based on the image processing system 100 and in particular complements the image forming device 210 for more elements.

The image forming device 210 has a state machine 211 based on the vehicle parameters 204 select the appropriate geometry objects 207, 208, 209.

The selected geometry objects 207, 208, 209 then become a transformation device 212 provided the selected geometry objects 207, 208, 209 according to the vehicle parameters 204 transformed. It can thus, for example, screw in a geometry object 209 for a wheel corresponding to the current steering angle of the vehicle. Furthermore, the transformation device 212 also perform geometric transformations that allow or represent a projection of the virtual three-dimensional model on a projection plane. This projection level can correspond, for example, to the image 214 or its perspective.

The results of the transformation device 212 are then the image processing device 213 in a corresponding overall model 215 transmitted. This can on the one hand from the 3D data of the overall model 215 a pictorial representation of the overall model 215 and on the other hand this pictorial representation of the picture 214 overlap.

Then the picture can 216 be output via a display device.

It is understood that the elements described above may be embodied as hardware or software or as a corresponding combination. For example, the image processing system may be 200 be arranged as a separate control unit in the vehicle and have its own memory and processor. The image forming device 210 may be, for example, a software that runs on the processor. Alternatively, this may be the image processing system 200 but also be integrated as a software component in an already existing in the vehicle system. The same applies to the image processing system 100 ,

3 shows a flowchart of an image processing method for a vehicle 300 . 400 with a number of image capture devices 101 . 201 and at least one display device 102 . 202 ,

The method provides for polling S11 of vehicle parameters 104, 204, which identify a current state of the vehicle 300, 400. Further, a vehicle model 106 , 206 and a variety of geometry objects 107-109 , 207-209 for the vehicle model 106 . 206 provided S2. Based on the queried vehicle parameters 104 . 204 become corresponding geometry objects 107-109 . 207-209 selected and with the vehicle model 106 . 206 to an overall model 215 connected. The selection can be done eg with a state machine 211 done by the vehicle parameters 104 , 204 receives as input variables and the corresponding geometry objects 107-109 . 207-209 selects.

Finally, a representation of the overall model 215 generated and one of the image capture devices 101 , 201 captured image 114 superimposed. That with the presentation of the overall model 215 overlaid image 114 is then on the display device 102 . 202 output.

The vehicle parameters 104 . 204 can be components of the vehicle 300 . 400 concern in the operation of the vehicle 300 , 400 different states. It can be for every component of the vehicle 300 . 400 at least one geometry object 107-109 . 207-209 to be provided. Furthermore, for at least one component of the vehicle 300 . 400 a variety of geometry objects 107-109 , 207-209.

To create the representation of the overall model 215 can geometric transformations of the geometry objects 107-109 , 207-209 based on the vehicle parameters 104 . 204 be performed. When linking, in this case, the transformed geometry objects 107-109 . 207-209 with the vehicle model 106 . 206 to an overall model 215 connected.

4 shows an embodiment of a representation of a vehicle according to the invention 300 or a graphical representation of the corresponding overall model. It can be seen that the front right wheel 301 is strongly deflected to the right. At the moment, the driver has hit the steering wheel in a clockwise direction. This was transmitted by the vehicle parameters to the image processing system and the geometry object of the wheel was adjusted accordingly and inserted into the overall model.

5 shows an embodiment of a representation of a vehicle according to the invention 400 or a graphical representation of the corresponding overall model.

Unlike the vehicle 300 is at the vehicle 400 The front right wheel is strongly turned to the left. The driver has therefore hit the steering wheel strongly counterclockwise.

With the aid of the present invention, the two vehicles 300 . 400 or the two states of the same vehicle are very easily prepared and mapped in the overall model.

Although the present invention has been described above with reference to preferred embodiments, it is not limited thereto, but modifiable in a variety of ways. In particular, the invention can be varied or modified in many ways without deviating from the gist of the invention.

LIST OF REFERENCE NUMBERS

100, 200

Image processing system

101, 201

Images detector

102, 202

display

103, 203

Communication Interface

104, 204

vehicle parameters

105, 205

model memory

106, 206

vehicle model

107-109, 207-209

geometry object

110, 210

Imaging device

211

state machine

212

transformation means

213

Image processing means

114, 214

image

215

overall model

216

image

300, 400

karoserie

301, 401

wheel

Claims (12)

Image processing system (100, 200) for a vehicle (300, 400) having an image acquisition device (101, 201) and at least one display device (102, 202), with a communication interface (103, 203), which is designed to generate vehicle parameters (104, 204) identifying a current state of the vehicle (300, 400) from the vehicle (300, 400), having a model memory (105, 205) containing a vehicle model (106, 206) and a plurality of geometry objects (107 - 109, 207 - 209) for the vehicle model (106, 206), and with an image generating device (110, 210), which is designed based on the queried vehicle parameters (104, 204) corresponding geometry objects (107 - 109, 207 - 209) and associate with the vehicle model (106, 206) to form an overall model (215) and a representation of the overall model (215) with an image captured by the image capture devices (101, 201) (114) and output on the display device (102, 202). Image processing system (100, 200) after Claim 1 wherein the vehicle parameters (104, 204) relate to components of the vehicle (300, 400) that can assume different states during operation of the vehicle (300, 400). Image processing system (100, 200) after Claim 2 , wherein at least one geometry object (107 - 109, 207 - 209) is stored for at least one component of the vehicle (300, 400). Image processing system (100, 200) after Claim 3 wherein for at least one component of the vehicle (300, 400) a plurality of geometry objects (107 - 109, 207 - 209) is stored. An image processing system (100, 200) according to any one of the preceding claims, wherein the image generating means (110, 210) is adapted to perform geometric transformations of the geometry objects (107-109, 207-209) and the transformed geometry objects based on the vehicle parameters (104, 204) (107-109, 207-209) with the vehicle model (106, 206) to a total model (215) to link. An image processing system (100, 200) according to one of the preceding claims, wherein the image generation device (110, 210) has a state machine (211) which is designed to receive the vehicle parameters (104, 204) as input variables and to output the corresponding geometry objects (107 - 109, 207 - 209). An image processing method for a vehicle (300, 400) having an image capture device (101, 201) and at least one display device (102, 202), comprising: Querying (S1) vehicle parameters (104, 204) identifying a current state of the vehicle (300, 400) from the vehicle (300, 400), Providing (S2) a vehicle model (106, 206) and a plurality of geometry objects (107 - 109, 207 - 209) for the vehicle model (106, 206), and Selecting (S3) corresponding geometry objects (107-109, 207-209) based on the queried vehicle parameters (104, 204) and associating the selected geometry objects (107-109, 207-209) with the vehicle model (106, 206) into an overall model (215) Generating (S4) a representation of the overall model (215) and overlaying the representation of the overall model (215) with an image (114) acquired by the image acquisition device (101, 201), and Outputting (S5) the image (114) overlaid with the representation of the overall model (215) on the display device (102, 202). Image processing method after Claim 7 wherein the vehicle parameters (104, 204) relate to components of the vehicle (300, 400) that can assume different states during operation of the vehicle (300, 400). Image processing method after Claim 8 wherein at least one geometry object (107-109, 207-209) is provided for at least one component of the vehicle (300, 400). Image processing method after Claim 9 wherein a plurality of geometry objects (107-109, 207-209) are provided for at least one component of the vehicle (300, 400). Image processing method according to one of the previous Claims 7 to 10 , further comprising performing geometric transformations of the geometry objects (107-109, 207-209) based on the vehicle parameters (104, 204), wherein when linking, the transformed geometry objects (107-109, 207-209) are coupled with the vehicle model (106, 106). 206) to an overall model (215). Image processing method according to one of the previous Claims 7 to 11 wherein, when selected, a state machine (211) picks up the vehicle parameters (104, 204) as inputs and selects the corresponding geometry objects (107 - 109, 207 - 209).

Images