DE102009035422B4 - Method for geometrical image transformation - Google Patents

Abstract

A method for the geometric transformation of the images of an image sequence generated by an imaging device of a motor vehicle, comprising the steps that are repeatedly carried out in each case: the imaging device generates a source image representing at least parts of the vehicle environment, detection of an object in the vicinity of the motor vehicle by distance sensors of the motor vehicle - Determination of a distance of the motor vehicle to the object, - Determination of a desired virtual image recording perspective as a function of the distance, the image recording perspective being shifted as a function of the distance along a spatial path defined relative to the motor vehicle, - Geometric transformation of the source image into a virtual one, Result image apparently recorded from the virtual image recording perspective.

Description

The invention relates to a method for geometrically transforming the images of an image sequence generated by an imaging device of a motor vehicle.

A wide variety of methods and technologies for geometrically transforming image sequences or video images are known from the prior art, in particular cutting out a specific area from a video image (windowing), electronically zooming or stretching a video image and / or shifting the real or video image. physical picture taking perspective into a virtual picture taking perspective.

So-called top view or bird view representations of the surroundings of motor vehicles as the result of geometric image transformations have already been described several times in the literature, in particular in connection with parking aids.

Proof of the practical feasibility of such image transformations in so-called virtual perspectives is, for example, the so-called top view or bird view function, which is already available on the market for selected vehicle models of the applicant. As part of this function, the driver is shown a virtual image on a display unit inside the vehicle. The source data for this image are recorded by cameras in the side mirrors of the motor vehicle. However, the virtual image that is presented to the driver after a corresponding geometric transformation is apparently taken from a virtual image recording perspective a few meters (approx. 3 meters) above the vehicle.

A virtual image recording perspective is generally realized in that pixel values of the virtual result image are determined from pixel values of the original or source image in such a way that the result image appears essentially as if it were recorded from another image recording perspective. In the simplest case, the position of pixels in an image can be shifted using a mathematical function, e.g. affine or non-affine image transformation, or using a look-up table. The geometric relationships related to the real or physical position of the imaging device (e.g. camera) are taken into account and new geometric relationships (such as would exist in the case of a recording from the virtual image recording perspective) are determined.

Known methods for using virtual perspectives in monitoring the surroundings of a motor vehicle and / or for using virtual perspectives in the display of images of the vehicle environment in the vehicle interior often have disadvantages, in particular when displaying complex three-dimensional object constellations. These disadvantages can mean that parts of objects or entire objects in the vehicle environment are not included in the virtual result image. This in turn can cause the vehicle driver to overlook such object parts or objects and increase the risk of accidents.

The document US 2008/0 231 702 A1 discloses an exterior vehicle display system for a vehicle. The system includes a camera that photographs a photographed area outside the vehicle and outputs a photographed image as a photograph result. Furthermore, the system has a first obstacle sensor that detects an obstacle in a first detection area that is included in the photographed area. And the system has a second obstacle sensor that detects an obstacle in a second detection area included in the photographed area and different from the first detection area. An image display device displays an image, and a display control device applies processing to the photographed image output from the camera to thereby generate a processed image after processing and display the processed image on the image display device.

It is an object of the invention to specify an improved method for the geometric transformation of the images of an image sequence generated by an imaging device of a motor vehicle.

This object is achieved by a method according to claim 1. Advantageous embodiments and developments of the invention emerge from the dependent claims.

• Zunächst wird durch die bildgebende Einrichtung, insbesondere eine Kamera, ein Quellbild erzeugt bzw. aufgenommen, das zumindest Teile der Fahrzeugumgebung darstellt. In der Umgebung des Kraftfahrzeugs wird zudem zumindest ein Objekt durch Abstandssensoren des Kraftfahrzeugs detektiert und es wird ein Abstand des Kraftfahrzeugs zu dem Objekt bestimmt. In Abhängigkeit von diesem Abstand wird eine gewünschte virtuelle Bildaufnahmeperspektive bestimmt und das Quellbild wird geometrisch transformiert in ein virtuelles Ergebnisbild, welches scheinbar aus der virtuellen Bildaufnahmeperspektive aufgenommen ist.

The method according to the invention for geometrically transforming the images of an image sequence generated by an imaging device of a motor vehicle comprises the following steps, each of which is carried out repeatedly:

• First, a source image is generated or recorded by the imaging device, in particular a camera, which represents at least parts of the vehicle environment. In the vicinity of the motor vehicle, at least one object is also detected by distance sensors of the motor vehicle and a distance between the motor vehicle and the object is determined. Depending on this distance, a desired virtual one becomes Determined image recording perspective and the source image is geometrically transformed into a virtual result image, which is apparently recorded from the virtual image recording perspective.

Each of the process steps mentioned is preferably carried out once for each image of the image sequence.

With the invention, the virtual image recording perspective can be established in such a way that the detected object, which can in particular be an obstacle or another road user, is contained in the resulting image.

As an alternative or in addition, the invention can define the virtual image recording perspective in such a way that the detected object can be identified or analyzed particularly well in the resulting image.

Alternatively or additionally, it can be achieved by the invention that the viewer of the image, in particular the driver of the motor vehicle, is better assisted in controlling his vehicle relative to objects in the vicinity of the vehicle.

An image recording perspective within the meaning of the invention is not only to be understood as the image recording position (e.g. the position of the camera; can be described in three dimensions, e.g. x, y, z), but the image recording position together with an orientation in space (e.g. position and orientation of the camera ; writable in six dimensions, e.g. x, y, z, phi, theta, psi).

Images from virtual perspectives can be realized through the use of perspective transformation methods which are known per se to the person skilled in the art. The perspective transformation takes into account the change in the position of certain image areas relative to one another, which corresponds to a change in the observation point, which in particular goes beyond linear scaling, shear and rotation. In particular, the perspective transformation usually comprises non-linear and / or numerical mathematical operations that are applied directly to an image area or, for example, to a changeable assignment table with the aid of which the image area is transformed.

The invention is based on the knowledge that the distance between the motor vehicle and the object has a decisive influence on the achievement, in particular, of the relatively abstract goal mentioned above (the object should be contained in the resulting image). This knowledge enables the definition of a particularly simple and efficient method for solving the stated problem.

1 demonstrates the problem of distance and shows schematically: a motor vehicle only partially shown in the figure 1 with a camera 2 for recording the vehicle environment, an obstacle 3 in the vehicle environment as well as two different virtual image recording perspectives 4th and 5 .

In the source image of the camera 2 is the obstacle 3 not included due to its geometric shape and arrangement in space, nevertheless it represents a threatening obstacle for the motor vehicle 1 represent.

The source image is now transformed geometrically into a virtual one, apparently from the virtual image recording perspective 5 transformed the recorded result image, this result image represents the obstacle 3 due to the principle not represented (it is only based on from the camera 2 supplied image data). One (hypothetically) actually physically from the picture taking perspective 5 captured image would - like 1 easy to see - naturally contain the obstacle. The driver of the motor vehicle 1 becomes through the virtual, only apparently from the virtual image recording perspective 5 The recorded result image thus conveys a deceptive, actually not given security.

To give a further example, the source image is converted geometrically into a virtual one, apparently from the virtual image recording perspective 4th If the recorded result image is transformed, this result image contains the obstacle 3 also not due to the principle. One (hypothetically) actually physically from the picture taking perspective 4th captured image would - like 1 can be recognized - at least partially contain the obstacle. In this case, too, the driver of the motor vehicle 1 thus through the virtual, only apparently from the virtual image recording perspective 4th The recorded result image conveys a deceptive, actually not given security.

That based on 1 The safety risk shown by way of example is particularly pronounced when the distance between the motor vehicle and the obstacle is very small. This is where the invention comes into play. An intelligent process is intended to allow a pronounced change of perspective, for example, when the distance is large and the security risk associated with the change of perspective is low. On the other hand, a change of perspective should be prevented or at least reduced in its extent if the distance is small and the change in perspective would be associated with a security risk.

The distance between the motor vehicle and the object can be understood to mean the distance from at least part of the motor vehicle to at least part of the object. In particular, the at least one part of the motor vehicle that is included in the evaluation can optionally be specified. In the case of more complex implementations, several distances or distance values - in particular from different parts of the motor vehicle to different parts of the object - can optionally be evaluated.

The imaging device can in particular be a camera. However, the method according to the invention can also be applied to other imaging devices or their image sequences (e.g. image sequences from an infrared camera or sequences / sequences of artificially generated model images from a lidar-based and / or radar-based and / or otherwise sensor-based device).

According to a preferred embodiment of the present invention, objects in the vicinity of the motor vehicle are detected on the basis of distance sensors, in particular ultrasound-based distance sensors.

According to an alternatively or additionally applicable preferred embodiment of the present invention, objects in the vicinity of the motor vehicle are detected on the basis of at least one source image from the imaging device. This can in particular be the last source image recorded. Alternatively or additionally, however, earlier source images or entire sequences of source images can also be included in the environmental monitoring. Source images can also be recorded specifically for this purpose (and not used for the geometric transformation).

According to another alternatively or additionally applicable preferred embodiment of the present invention, objects in the vicinity of the motor vehicle are detected on the basis of at least one result image. This can in particular be the last image generated. Alternatively or additionally, however, earlier result images or entire sequences of result images can also be included in the environmental monitoring. Result images can also be created specifically for this purpose (and not used for optical output or other utilization).

According to the invention, the image recording perspective is shifted as a function of the distance along a spatial path established relative to the motor vehicle.

The virtual image recording perspective is preferably shifted in the direction of a “flatter” perspective with decreasing distance (compared to the image recording perspective of preceding result images that are generated on the basis of the same source image sequence). This means that the amount of the elevation angle is reduced as the distance decreases or there is a tendency to switch from a bird's eye view to a horizontal forecast.

Alternatively or additionally, it can be advantageous if the virtual image recording perspective (compared to the image recording perspective of preceding result images that are generated on the basis of the same source image sequence) is changed with decreasing distance such that the virtual image recording position is shifted towards the physical image recording position.

A special case that advantageously combines the two last-mentioned approaches is that the virtual image recording perspective approaches the physical image recording perspective with decreasing distance - both with regard to the image recording position and with regard to the orientation in space.

Before moving on to the representation of a preferred exemplary embodiment, two scenarios are intended to demonstrate the advantages of the invention in the following. Further details, preferred embodiments and developments of the invention emerge from the representation of the scenarios.

• Gemäß einem ersten Szenario wird bei einem erfindungsgemäßen Verfahren bzw. bei einem Kraftfahrzeug mit erfindungsgemäß betriebenen Einrichtungen die virtuelle Bildaufnahmeperspektive selbsttätig derart verändert, dass eine Darstellung eines hindernisfreien Teils der Fahrzeugumgebung im Ergebnisbild größtenteils vermieden wird, wenn sich in der direkten Blickrichtung von der virtuellen Bildaufnahmeperspektive zu diesem hindernisfreien Teil der Umgebung ein Hindernis befindet. Mit anderen Worten: Es darf kein hindernisfreies virtuelles Ergebnisbild erzeugt werden, wenn ein tatsächlich physikalisch aus der entsprechenden Bildaufnahmeperspektive aufgenommenes Bild nicht ebenfalls hindernisfrei wäre.

scenario 1 :

• According to a first scenario, in a method according to the invention or in a motor vehicle with devices operated according to the invention, the virtual image recording perspective is automatically changed in such a way that a representation of an obstacle-free part of the vehicle environment in the result image is largely avoided if the direct line of sight is from the virtual image recording perspective this unobstructed part of the environment is an obstacle. In other words: no obstacle-free virtual result image may be generated if an image actually physically recorded from the corresponding image recording perspective was not also free of obstacles.

The automatic control of the virtual image recording perspective ideally behaves in such a way that the advantages that can be achieved through the formation of virtual perspectives are exploited to the greatest possible extent, but those based on them 1 The problem outlined here is specifically avoided. This can be done without loss of generality for example mean that the road ahead is shown to the driver from a convenient virtual perspective, for example vertically from above, until a situation occurs or is approaching that is comparable to the situation 1 . If this is the case, the virtual perspective is changed (preferably continuously) in such a way that the problem is avoided, but the best possible or comfortable view of the surroundings of the vehicle is still possible.

Further conditions can preferably be taken into account, in particular the area of the roadway contained in the result image can be maximized. Thus, the advantages of a virtual perspective are built up adaptively as much as possible without any disadvantages becoming too great.

It can also be advantageous to differentiate obstacles according to their height, their arrangement in space and / or their danger, etc., in order to achieve optimal behavior of the automatic control of the virtual perspective.

• Gemäß einem zweiten Szenario wird bei einem erfindungsgemäßen Verfahren bzw. bei einem Kraftfahrzeug mit erfindungsgemäß betriebenen Einrichtungen die virtuelle Bildaufnahmeperspektive selbsttätig derart verändert, dass ein detektiertes Hindernis stets zumindest teilweise im Ergebnisbild erscheint. Vorzugsweise wird die Anpassung der Bildaufnahmeperspektive dabei so ausgeführt, dass der im Ergebnisbild enthaltene Teil des Hindernisses stets kleiner ist als eine vordefinierte Bezugsgröße.

scenario 2 :

• According to a second scenario, in a method according to the invention or in a motor vehicle with devices operated according to the invention, the virtual image recording perspective is automatically changed in such a way that a detected obstacle always appears at least partially in the resulting image. The adaptation of the image recording perspective is preferably carried out in such a way that the part of the obstacle contained in the resulting image is always smaller than a predefined reference variable.

In practice, this means that the virtual image recording perspective is always selected in such a way that the resulting field of view or the resulting cone of view is only pivoted so far into the preceding direction or the direction that is considered ideal (for supporting the driving task as a rule) that the obstacle does not completely disappear from this field of vision. In this way, the user can always recognize and understand why the perspective of the result image presented to him has changed.

If an obstacle is detected (by the distance sensors and / or with the aid of image processing on the basis of a camera image) which could be overlooked if the virtual image recording perspective were unfavorably defined (cf. 1 ), the virtual image recording perspective is therefore preferably adapted in such a way that at least a part of the obstacle that is smaller than a predefined reference variable can be seen in the result image. The probability of the driver misinterpreting the presented view of the surroundings is thus reduced. In the situation according to 1 the virtual image recording perspective would have to be shifted for this purpose, preferably continuously, in the direction back to the vehicle and / or downwards.

In other cases - depending on the geometric constellation of obstacles - it can be advantageous to shift the image recording perspective alternatively or additionally to the right or left, up or down, in order to achieve the above-mentioned goal.

• 2 die räumlichen Verhältnisse zwischen einem Kraftfahrzeug, einem Hindernis und verschiedenen physikalischen und virtuellen Bildaufnahmeperspektiven und
• 3 eine Bildschirmaufteilung bei einem beispielhaften Verfahren zur Darstellung einer Umgebungsszene aus verschiedenen Bildaufnahmeperspektiven.

A preferred exemplary embodiment of the invention is described below with reference to the additional accompanying drawings. This results in further details, preferred embodiments and developments of the invention. In detail show schematically

• 2 the spatial relationships between a motor vehicle, an obstacle and various physical and virtual image recording perspectives and
• 3 a screen division in an exemplary method for displaying an ambient scene from different image recording perspectives.

In order to use the 1 In the example system presented below, the geometric transformation of the source image of a camera arranged on a motor vehicle is to avoid the indicated safety risks of a change of perspective 2 configured as a function of an object recognition related to the vehicle environment.

In particular, the virtual image recording perspective into which the source images of the imaging device are geometrically transformed is automatically set or controlled or determined in the example system on the basis of the results of the environment recognition.

The environment recognition can basically be designed in different ways. It can be an obstacle detection or, for example, a detection of lane boundaries or lane markings, but also a detection of drivable or non-drivable road sections.

As a result of the automatic control or definition of the virtual image recording perspective, the virtual image recording perspective can be defined differently for different images of an image sequence without the driver of the motor vehicle or another operator having to intervene. So an obstacle - for example when approaching it dangerously - can hit the driver are shown from a different perspective than when the distance between the motor vehicle and the obstacle is greater. That is to say, the top view angle “felt” by the driver or the virtual viewing angle set by defining the virtual image recording perspective is set automatically or automatically as a function of the distance from the obstacle.

Distance sensors, in particular ultrasound-based distance sensors, serve as sensors for the detection of objects in the vicinity of the motor vehicle. Such distance sensors are already present in many modern motor vehicles and do not have to be provided specifically for the purposes of the invention.

2 shows the components of a corresponding example system and the spatial relationships between motor vehicle, obstacle, physical image recording perspective and virtual image recording perspective in an example scenario.

Initially comparable to the scene 1 , shows 2 schematically: a motor vehicle only partially shown in the figure 1 with a camera 2 for recording the vehicle environment, an obstacle 9 in the vehicle environment as well as two different virtual image recording perspectives 4th and 5 .

The distance sensor 7th of the motor vehicle 1 (as well as a downstream evaluation unit) monitors the vehicle environment and detects the obstacle 9 in the vehicle environment. Based on the signals from the distance sensor 7th becomes in particular the distance of the motor vehicle 1 from the obstacle 9 certainly.

Depending on the results of the environmental monitoring, a virtual image recording perspective can now be used, in which the camera 2 Recorded images can be transformed, set in space.

In the in 2 The illustrated - still relatively simple - example case is the image recording perspective as a function of the distance between the motor vehicle 1 and obstacle 9 along the indicated by the dashed arrow 6th indicated space orbit shifted.

The automatic shift is preferably carried out in such a way that the virtual image recording perspective is shifted in the direction of a “flatter” perspective with decreasing distance, while at a greater distance a plan view or a bird's eye view is more likely to be presented. So the above is based on 1 effectively avoided the problem presented.

Alternatively, however, the automatic shift can also - for example, to provide a solution in special cases - be carried out in such a way that the virtual image recording perspective is shifted with decreasing distance in the direction of a top view or a view from the bird's eye view, while with greater distance it tends to be a "flatter one “Perspective is used. In this way, it can be ensured that the driver is given an improved view of the rest of the vehicle at a greater distance, while at a small distance from an obstacle he can better estimate the shape of the obstacle on the basis of the top view.

More complex design variants of the invention provide for the environment recognition, the object detection and / or possibly a scene interpretation at least partially based on the camera 2 to base captured source images.

Again, more complex design variants of the invention provide for the environment recognition, the object detection and / or possibly a scene interpretation to be based at least partially on virtual result images that are generated by geometrical transformation of source images from the camera 2 have been generated in a virtual image recording perspective.

If necessary, additional images can also be provided specifically by the camera for the purposes of environment recognition, object detection and / or possibly scene interpretation 2 be included. These additional images are not intended for presentation to the driver and do not necessarily have to be subjected to a geometric transformation - at least for this purpose.

In special cases, additional images can also be provided specifically by the camera for the purpose of environment recognition, object detection and / or possibly scene interpretation 2 and specifically for the purposes of environment recognition, object detection and / or possibly scene interpretation are transformed into a suitable virtual image recording perspective.

If camera images are used both for environment sensing / object recognition / obstacle detection and for optical output to the driver (possibly in a transformed form), then the environment sensing, object recognition, obstacle detection is preferably carried out on the basis of a much larger detection angle than the optical output on the driver.

By applying the environment recognition to the already transformed image (which in turn is based on the results of the environment recognition depends), a control loop can be formed, which can bring about an automatic maximization of the detection probability.

With a clever design of a method based on these techniques, an obstacle present in the vehicle environment can be localized on the basis of the environment sensing and the corresponding image area can be selected from the entire source image of the imaging device and, for example, shown enlarged, on the basis of this localization. This has the advantage that the driver - thanks to the enlarged display - is better exposed to obstacles and can analyze them more precisely.

In addition, the image section of the imaging device can also be determined as a function of the results of an environment monitoring or obstacle detection.

The corresponding image processing procedures can basically be carried out in any order. However, it appears to be advantageous to carry out the windowing or zoom functionality (if included) first and only then to apply the virtual perspective transformation to the image. This procedure enables the computing power to be used for the entire process to be reduced.

If a virtual result image is generated on the basis of the invention, further advantages can result from a clever or advantageous optical output of the same. In the following, the optical output on a display unit in the vehicle interior is assumed without loss of generality.

For example, a composite image can be generated and displayed. It can be generated as a combination of at least one perspective transformed result image and at least one non-perspective transformed source image.

In particular, both images, i.e. the at least one result image and the at least one source image, can originate from one and the same imaging device or are based on image data from one and the same imaging device.

The presentation of the at least one result image and the at least one source image can in principle take place simultaneously or alternately.

A result image displayed simultaneously with a source image is preferably based on image data of this source image. However, it can also be based on image data from another source image that was recorded in close temporal proximity to the displayed source image.

By also displaying the source image (in addition to at least one virtual result image), the driver does not lose reference to the physical image recording perspective of a camera attached to the vehicle.

A possible automatic alternation between a display of the at least one result image and a display of the at least one source image can take place in a manner comparable to switching the channels of a video surveillance system. An automatic alternation can be carried out in a time-controlled manner, or - which should generally be preferable - as a function of a route traveled by the vehicle since the last switchover. In this way, the speed of the vehicle could be taken into account and the probability minimized that changed environmental conditions are overlooked by the switching processes. Optionally, the speed can also be included directly in the alternation control.

It can also be advantageous to integrate or insert a result image into a source image for display purposes or - conversely - to integrate or insert a source image into a result image for display technology. In particular, the image that is inserted into the other image can be reduced in size. By limiting the reduced image to a relevant area (recognized with the aid of image processing), it can be avoided that the representation of this relevant area turns out to be too small for the viewer.

In particular, the result image and the source image can be displayed together in the context of a picture-in-picture overlay.

For example, enlarged sections can be faded in in a “normally” interpretable source image of the camera, which show obstacles in the vehicle environment and are each generated according to the invention as virtual, perspective transformed result images. The respective virtual image recording perspective of these images is preferably determined adaptively as a function of the distance to the respective obstacle.

3 shows a possible screen view, generated by picture-in-picture overlay.

The largest field of view 11 shows a source image that comes directly from the recording camera and has not been transformed in perspective. The smaller fields of view 12th , 13th and 14th contain views of an or several obstacles from one or more virtual perspectives.

It is referred to as an alternative to the constellation 3 It is also conceivable to generate and display a source image together with a multiplicity of smaller, virtually transformed, zoomed result images of a selected object that are arranged around the source image.

According to an advantageous embodiment of the display scheme according to 3 will be in the image field 13th , that is in the center of the overall image, a result image with essentially virtual perspective "from above" (ie above the physical camera position) is displayed in the image field 12th top left a result image with essentially virtual perspective “from top left” and in the image field 14th At the top right a result image with essentially a virtual perspective “from the top right”. The arrangement of the individual result images is based on the virtual image recording perspectives and thus supports the interpretability of the image content.

In terms of orienting the arrangement of the individual result images to the respective virtual image recording perspectives, it can also be advantageous to move them within the larger source image or along the edge area of the larger source image depending on a changing image recording perspective of the respective result image.

The relative position or arrangement of the image fields to one another preferably corresponds to the relative position or arrangement of the respective image recording perspectives to one another. The result images are therefore preferably arranged in relation to the source image in accordance with the arrangement of the respective virtual image recording perspectives in relation to the physical position of the imaging device.

If a virtual result image is generated on the basis of the invention, further advantages can result from a clever or advantageous design of the associated man-machine interface and the operating options opened up by this man-machine interface.

According to a preferred embodiment of such a man-machine interface, the driver or operator is given the opportunity to switch between two different picture-in-picture views. In the first of these two picture-in-picture views, at least one (smaller) virtual result image is inserted into a (larger) “normal”, non-perspective transformed image of the surroundings, in particular the source image belonging to the result image. The second of these two picture-in-picture views, to which the driver or operator can switch, has the reverse structure: the (smaller) “normal”, non-perspective transformed image of the surroundings is inserted into the (larger) virtual result image.

The second picture-in-picture view represents, so to speak, the “advanced” view mode. The switchover option is particularly advantageous and useful because a result image generated according to the invention shows a surrounding scene or an obstacle in the vehicle environment from a virtual perspective shows that for many operators it is often difficult to interpret at the beginning, ie it takes some getting used to. If a mode switch is available, such an operator can initially use the more easily interpretable representation (large picture from a physical perspective, small picture (s) from a virtual perspective) and only switch to the reverse display, which may be useful for some driving tasks, after a certain period of getting used to it .

According to another preferred embodiment of a man-machine interface that can be combined with the aspects mentioned above, the driver or operator is given the option of changing at least one parameter of the virtual perspective and / or the image section by means of an operating element. Thus, the user can change an automatically preset perspective (e.g. pilot function) and / or navigate between the view of several highlighted objects / obstacles using a rotary knob, joystick and / or using a gesture recognition control element, etc., if necessary also by voice commands.

According to yet another preferred embodiment of a man-machine interface, which can also be combined with the above-mentioned aspects, a particularly suitable or suitable display type is automatically selected from a set of predefined possible display types depending on the number of obstacles and their speed.

Although large parts of the present description relate to the optical output to the driver of a motor vehicle, the method according to the invention can be used not only for such optical output, but also for the purpose of automatic further processing, in particular object recognition.

The example system or example method described below is also related to the invention: In this example system or example method, the choice of the virtual image recording perspective is primarily driven by downstream image processing that selects the most advantageous view according to the detection or classification task. This automatically enables a virtual focus on an algorithmically determined region, which can be kept stable over the course of a sequence of images. The image stability obtained in this way enables a human observer to quickly distinguish relevant information from irrelevant information.

This should first be illustrated on a vehicle with a reversing camera, the trailer coupling of which is to be docked on a trailer. Using the invention, the target, i.e. the trailer or its counterpart to the trailer coupling, can initially be detected with the aid of image processing and an image perspective can be determined on the basis of the detection in such a way that the trailer is shown in a frontal view. This view would not have any perspective distortions and would keep the trailer in the center of the image for the entire docking maneuver. Steering instructions to the driver can be displayed, for example, by overlaying them, i.e. using an overlay. In contrast to pure image stabilization processes, this is a combination of stabilization and targeted correction of perspective. Any relevant image area can thus also be prepared for better perceptibility.

This should also be illustrated on a vehicle with a wide-angled front camera, the driver of which would like to carry out a left-turn process. The virtual image recording perspective is ideally selected in such a way that the driver in the vehicle interior - possibly in addition to other representations - is shown a result image by which the driver is shown an (apparently) frontal approach of another vehicle. This has the advantage of a targeted control of the driver's attention.

Claims (9)

Method for the geometric transformation of the images of an image sequence generated by an imaging device of a motor vehicle, comprising the steps that are carried out repeatedly in each case: - Generation of a source image representing at least parts of the vehicle environment by the imaging device, - Detection of an object in the vicinity of the motor vehicle by distance sensors of the motor vehicle, - determining a distance of the motor vehicle to the object, - Determination of a desired virtual image recording perspective as a function of the distance, the image recording perspective being shifted as a function of the distance along a spatial path defined relative to the motor vehicle, - Geometric transformation of the source image into a virtual result image, apparently recorded from the virtual image recording perspective. Procedure according to Claim 1 , characterized in that distance sensors are used to detect the object in the vicinity of the motor vehicle, in particular ultrasound-based distance sensors. Method according to one of the Claims 1 or 2 , characterized in that objects in the vicinity of the motor vehicle are detected on the basis of at least one source image from the imaging device. Procedure according to Claim 3 , characterized in that at least one source image, on the basis of which the detection takes place, is generated specifically for this purpose by the imaging device. Method according to one of the Claims 1 to 4th , characterized in that objects in the vicinity of the motor vehicle are detected on the basis of at least one virtual result image. Procedure according to Claim 5 , characterized in that at least one result image, on the basis of which the detection takes place, is generated specifically for this purpose. Method according to one of the Claims 1 to 6th , characterized in that the virtual image recording perspective is changed with decreasing distance in the direction of a flatter orientation in space. Method according to one of the Claims 1 to 7th , characterized in that the virtual image recording perspective is changed with decreasing distance in such a way that the virtual image recording position is shifted towards the physical image recording position. Method according to one of the Claims 1 to 6th , characterized in that the virtual image recording perspective is approximated to the physical image recording perspective with decreasing distance.

Images