EP3570063A1 - Method for quick estimation of a perspective concealment in a simulation of an imaging sensor - Google Patents

Abstract

Method for simulating a sensor system with an imaging sensor and an image processing unit. A virtual test environment includes the sensor and a plurality of virtual objects, a number of which are recognizable by the image processing unit. When a first virtual object and a second virtual object recognizable by the image processing unit are detected by a sensor field of view of the sensor, a projection plane is spanned. A first geometric body resembling the first virtual object, the dimensions of which at least approximately coincide with the dimensions of the first virtual object, and a second geometric body having the same size as the second virtual object whose dimensions at least approximately coincide with the dimensions of the second virtual object projected in a central projection with the spatial coordinates of the sensor as an eye point on the projection plane. If an intersection of the images of the first geometric body and the second geometric body exists, the Euclidean distance between the sensor and the first virtual object is less than the Euclidean distance between the sensor and the second virtual object and the size of the intersection exceeds a predefined threshold, the second virtual object assumed to be hidden in perspective. Otherwise, the second virtual object is assumed to be unobstructed in perspective.

Description

The invention relates to sensor simulation and virtual testing.

In the development of safety-critical embedded systems, it is known to check these systems for correct functioning in a virtual test environment. In a method known as Hardware in the Loop (HIL), an embedded system prototype is connected to a simulation computer that provides a largely virtual test environment and provides the embedded system with hard real-time input data generated by the virtual test environment. Conversely, the simulation computer can read control signals of the embedded system and take them into account in the calculation of the state of the virtual test environment in the respective next simulation time step. In earlier stages of development, it is also known to integrate only its software or its program logic in a virtual test environment instead of a prototypical embedded system. Such fully virtual test methods are known as Software in the Loop (SIL) or Model in the Loop (MIL) and do not necessarily have to be performed in real time. Also known in the automotive industry is a method called Vehicle in the Loop (VIL) in which a prototype of the embedded system is tested in a physical test vehicle mounted on a test track or chassis dynamometer, the embedded system in hard real-time using a virtual test environment supplied synthetic sensor data is supplied.

Some embedded systems evaluate sensor data that is supplied by sensor systems with imaging sensors. Of particular importance are such embedded systems in the automotive industry in the context of Advanced Driving Assistance Systems (ADAS) and Highly Automated Driving (HAF). By imaging sensors, all sensors are understood from the Sensor data an image of the environment of the sensor is reproducible, in particular RADAR, LIDAR and ultrasonic sensors and also the optical sensors of camera systems for passive image acquisition in the visible or non-visible light spectrum. Such sensor systems may include not only the sensor itself but also an image processing unit for evaluating the image data supplied by the sensor. The image processing unit can be designed as an autonomous embedded system or integrated into the embedded system to be tested.

If a virtual test environment is envisioned for testing an embedded system that expects input data from a sensor system with an imaging sensor, the virtual test environment must include a simulation of the sensor system. In some applications, for example, if the image processing unit is not a test piece or no part of the test object, it makes sense to save on computational effort to dispense with a detailed simulation of the sensor. Instead of computing image data from the sensor from the virtual test environment, a sensor field of view is associated with the simulated sensor, such as a limited range conical field of view from the sensor, and a plurality of virtual objects included in the virtual test environment become a number of virtual objects than by the sensor system identifiable virtual objects. If the sensor simulation is intended, for example, for the simulation of a traffic sign recognition, all the virtual traffic signs that can be added to the virtual test environment can be determined as virtual objects recognizable by the sensor system. In one simulation run in the virtual test environment, in one example application, all virtual road signs captured by the sensor field of view would then be included in a preliminary list of detected virtual objects, while all other virtual objects captured by the sensor field of view, such as persons, trees, buildings , Vehicles and traffic lights are basically not recognized by the simulated sensor system, that is basically not included in the preliminary list of detected virtual objects.

In the present description of the invention, a simulation of a sensor system therefore preferably means a simulation based on a simple sensor system model which only superficially imitates the behavior of the sensor system, but at most simulates to a limited extent physical or technical processes of a physical sensor system and in particular no calculation of synthetic image data which, for example, emulate the image data generated by a real imaging sensor. Such simple sensor system models are also known as ground truth sensors. One advantage of this is the low computational effort that is necessary for their execution. This makes ground-truth sensors particularly suitable for use in virtual tests that are performed in hard real-time, such as hardware-in-the-loop techniques.

A fundamental problem with the implementation of ground truth sensors is the consideration of a perspective concealment of virtual objects detectable by the sensor system. As previously described, a ground truth sensor first evaluates as all detected virtual objects detected by the sensor field of view and determined as objects recognizable by the sensor system. In order to realistically simulate the sensor system, however, it must subsequently be checked whether a virtual object deemed to be recognized is actually visible to the sensor or whether the virtual object is at least partially concealed by another virtual object from the perspective of the sensor. If this is the case and the perspective occlusion exceeds a certain extent, the virtual object must be removed from the preliminary list of detected virtual objects.

From the field of computer graphics, numerous methods for detecting perspective hidden surfaces are known. For example, a selection is in the presentation " Visibility and Hidden Surface Removal "by Torsten Möller (part of the lecture" CMPT 361: Introduction to Computer Graphics "by Simon Fraser University, Canada , available on the internet). Some of them also involve the projection of a three-dimensional scene onto a projection plane. The aim of this known method is but the calculation of a realistic image, and their high computational complexity makes them problematic for real-time applications, especially after the calculation of a sensor image according to one of the known methods would have to be evaluated whether a determined perspective occlusion would still allow detection of the hidden object by the simulated sensor system ,

The item " Fast Collision Detection Based on Projection Parallel Algorithm "by Xue-li Shen and Tao Li (2nd International Conference on Future Computer and Communication, IEEE, 2010 ) describes a method for collision detection of two bodies in a three-dimensional space by projection on three orthogonal projection planes and performing collision detection on each projection plane.

The EP 0 620 409 B1 discloses a simulation environment for the development and testing of target seeking missiles. The simulation environment computes in hard real time a synthetic image for the imaging sensors of a missile, and the disclosure includes a method for detecting perspective hidden surfaces.

Against this background, the object of the invention is to accelerate the detection of a perspective occlusion in a simulation of a sensor system with an imaging sensor in a virtual test environment.

To solve the problem, a method described below for simulating a sensor system is proposed. The simulated sensor system comprises an imaging sensor and an image processing unit for detecting objects in the sensor image output by the sensor. The simulation of the sensor system itself is preferably a ground truth sensor and includes the imaging sensor and the image processing unit only implicitly.

For the application of the method, a virtual test environment is used, which comprises a plurality of virtual objects and in which the sensor and each virtual object are each assigned location coordinates.

The sensor is assigned a sensor field of view in the virtual test environment, and a number of objects of the plurality of virtual objects are determined as virtual objects detectable by the sensor system.

In the virtual test environment, a simulation run is started, in the course of which the sensor or at least one virtual object is cyclically assigned new location coordinates in the virtual test environment, and it is cyclically checked whether at least one virtual object is detected by the sensor field of view.

When at least a first virtual object and a second virtual object are detected by the sensor field of view and the second virtual object is recognizable by the sensor system, an estimation of the perspective obscuration of the second object by the first object is performed in the manner described below. For this purpose, in the virtual test environment, first a projection plane is spanned whose spatial orientation is selected such that it is suitable for a central projection of the first object and the second object if the spatial coordinates of the sensor are used as the eye point of the central projection. A central projection is to be understood as a projection with a fan-shaped beam path emanating from the eye point, which generates an image of the projected points on the projection plane, as would be perceived by an observer standing at the point of view, specifically the sensor.

An at least partial central projection of a geometrical body which is the same as the first object with the spatial coordinates of the sensor as eye point is performed on the projection plane, the dimensions of the unscaled first geometric body completely or at least approximately coinciding with the dimensions of the first object. It does not matter if the first geometric body is natively present in the virtual test environment or is generated to perform central projection as needed. When the dimensions of the first geometric body and the first object are in complete agreement, the first geometric body can also be aligned with the first Object to be identical, ie only implicitly exist in the virtual test environment. The first geometric body is preferably an enveloping body or a scaled enveloping body of the first object. An enveloping body of an object is to be understood as meaning a geometrical body completely encompassing the object with a simplified geometry relative to the encompassed object, in particular a polyhedron with few vertices or a body which can be analytically described in a simple manner, for example a sphere or a cylinder.

How many vertices an enveloping body should at most include, or how simple its geometry should be, depends on the application and is hardly universally determinable. If the method is provided for real-time execution, the geometry of the geometric bodies should in principle be selected such that a complete run of a software implementation of the method within a period of time specified by the application, in particular within a simulation time step, is so assured that within one complete simulation run, no violation of the predetermined time span by the software implementation is to be expected. Which geometries of the geometric bodies or which number of vertices are acceptable depends in particular on the implementation of the method, the complexity of the virtual test environment and the simulation, and on the hardware configuration of the computer system performing the method.

A partial central projection or a partially performed central projection of an object means that only an incomplete selection of points of the object in a central projection is projected onto the projection plane. In some embodiments, from the projected points, an amount is formed on the projection plane that approximates an image resulting from a complete central projection of the object.

An analogous manner for carrying out the central projection of the first geometric body becomes an at least partial central projection of a with the second object the same second geometrical body is performed on the projection plane with the spatial coordinates of the sensor as an eye point, wherein the dimensions of the unscaled second geometric body completely or at least approximately coincide with the dimensions of the second object.

The projection of the first geometric body produces a first image on the projection plane, the projection of the second geometric body produces a second image on the projection plane. Subsequently, it is checked whether an intersection of the first image and the second image exists. Secondly, in the case where there exists an intersection, secondly the Euclidean distance between sensor and first object is less than the Euclidean distance between sensor and second object and thirdly the size of the intersection in relation to the size of the second image exceeds a predefined threshold value second object scored as geometrically obscured, ie it is assumed that the image processing unit does not recognize the second object, even if the second object is partially visible from the sensor's point of view, because the degree of perspective occlusion is too high for recognition of the second object by the image processing unit. A determination of the Euclidean distances is preferably based on the location coordinates of the sensor, the first object and the second object.

A Euclidean distance between the sensor and an object can be determined on the basis of the spatial coordinates of the sensor and a defined point of the object, for example based on the point of the object closest to the sensor or the spatial coordinates of the object.

In the event that at least one of the aforementioned conditions is not met, that is either no intersection exists or the Euclidean distance between the sensor and the first object is greater than or equal to the Euclidean distance between the sensor and the second object or the size of the intersection in relation to the size of the second image does not exceed the predefined threshold value, the second object is considered not to be hidden in perspective, ie it is assumed that that the image processing unit recognizes the second object, even if the second object is partially hidden in perspective by the first object, because the degree of perspective occlusion is sufficiently low for recognition of the second object by the image processing unit.

If necessary, the previously described estimation of a perspective occlusion can be performed multiple times on respectively different virtual objects if, in addition to the first object and the second object, further virtual objects are detected by the sensor field of view. For evaluation by the device under test or by the virtual test environment operating computer system, a signal of the sensor system is generated to which the information is removable, that all detected by the sensor field of view and detectable by the sensor system virtual objects that are not considered to be hidden from the perspective of the Image processing unit have been detected and all detected by the sensor field of view and recognizable by the sensor system virtual objects that are considered to be hidden in perspective, were not recognized by the image processing unit.

In one possible application, the signal simulates a possible signal of a real sensor system, in particular a series-produced or prototype sensor system. In another possible application, the signal does not simulate a possible signal from a real sensor system and is used, for example, to drive a test bench that is set up to simulate the existence of objects within its sensor field of view to a real imaging sensor.

An advantage of the invention is that it enables a fast and reliable estimation of the visibility of a virtual object for an image processing sensor system, without resorting to computationally expensive methods from computer graphics, for example ray tracing or Z-buffering. As a result, the method also reduces the hardware expenditure for a simulation of a corresponding sensor system, because it can be performed without the use of a graphics processor (GPU, Graphics Processing Unit). Investigations by the Applicant have shown that a software implementation of a method according to the invention can reliably run through within a millisecond. This means that the invention is suitable for application to a hardware-in-the-loop method and the method according to the invention is real-time capable in the sense of a hardware-in-the-loop method.

Advantageously, the sensor is suspended in the virtual test environment on a virtual machine, wherein the signal of the sensor system is monitored by a control system set up to control the virtual machine or by a control software set up to control the virtual machine. In particular, the control system may be a device under test, for example a prototype embedded system for controlling a physical machine, which makes decisions about the control of the machine on the basis of a sensor image. Accordingly, the control software may be, in particular, a prototypical control software intended for installation on a control system for controlling a physical machine or a program logic under development intended for later preparation for installation on a corresponding control system. The virtual vending machine is particularly preferably a virtual automated vehicle, wherein a vehicle is to be understood as meaning any technical system set up to move on its own power. A suspension of the sensor on a virtual machine is to be understood in particular that the location coordinates of the sensor and the spatial orientation of the sensor field of view by the location coordinates and / or the spatial orientation and / or state of the virtual machine are clearly specified.

To further reduce the computational effort, it is advantageous not to perform a complete projection of a geometric body on the projection plane, but to project only a selection of points of the geometric body on the projection plane and from the of the images The point cloud formed the selected point cloud to form a two-dimensional amount on the projection plane, which corresponds approximately to the image of the fully projected geometric body and preferably includes all points of the point cloud, for example by forming the convex hull of the point cloud. For example, to project a polyhedral geometric body, it would be possible to project all vertices or a selection of vertices of the geometric body. Particularly advantageous only points of a geometric body are projected, whose images are elements of the convex hull of resulting from a complete projection of the geometric body image. A numerical method for determining such a selection of points is in the description of FIG FIGS. 4 to 6 disclosed.

The convex hull of a set of points is the smallest convex set that includes all points from the set of points. A set is called convex if every connecting line drawn between two points of the set is completely contained in the set. A body is called convex if a set of spatial points congruent with the body is convex.

In order to test the existence of an intersection and to determine the size of the intersection in relation to the size of the second image, a known numerical method for the measurement of areas can be used in one embodiment of the invention. For example, the projection plane can be subdivided into sufficiently small fields, for example in squares, triangles or hexagons. The detection of the existence of an intersection and the size comparison to the second image can then take place by counting the fields that lie at least partially both within the first image and within the second image or at least partially within the second image, and by comparing the determined fields numbers.

Advantageously, however, a dimensioning of surfaces is dispensed with by setting the threshold value for the size of the intersection to zero and the examination of the existence of an intersection is carried out on the basis of a collision detection of the first image and of the second image on the projection plane. Thus, in this latter embodiment, the mere proof of the existence of an intersection of the first image and the second image to make the second virtual body concealed is sufficient; the Euclidean distance of the first virtual object to the sensor is less than the Euclidean distance of the second virtual object to the sensor. The size of the intersection is insignificant. The method is considerably accelerated because the mere verification of the existence or nonexistence of an intersection with correspondingly clever implementation of the method requires fewer calculation steps than the measurement of an area.

For collision detection of the first image and the second image, the use of a Gilbert-Johnson-Keerthi algorithm, hereinafter GJK algorithm, advantageous, as it is known, for example, in the technical article " The Gilbert-Johnson-Keerthi Distance Algorithm "by Patrick Lindemann (written for the Media Informatics Proseminar on" Algorithms on Media Informatics ", 2009 ) is described.

The first and the second image collide if and only if their Minkowski difference comprises the origin of a Cartesian coordinate system defined on the projection plane. The GJK algorithm proposes to construct the Minkowski difference of the first image and the second image, and to check whether the Minkowski is repeated by defining a triangle in the Minkowski difference, with the triangle approximating the origin in each iteration step Difference includes the origin. The GJK algorithm is applicable only to convex sets.

For a fast pass of the GJK algorithm, it is advantageous if the first image and the second image are convex closed polygons with few vertices, so that the first image and the second image - and consequently their Minkowski difference - each into a small Number of triangles are divisible. This can be ensured by in that the geometric bodies are designed as convex polyhedra having few vertices, wherein, in particular, before performing the projection of a geometric body, a selection of vertices of the geometric body whose images are elements of the convex hull of the image of the geometric body is determined. Only the selection of vertices is projected.

By means of collision detection, for example of the GJK algorithm, the existence or non-existence of an intersection of the first image and the second image can be detected. However, collision detection generally does not provide information about the size of an existing intersection. If, as in the previously described embodiments of the invention, the threshold for the size of the intersection is set equal to zero and collision detection is used for the existence of an intersection check, each perspective obscuration of the second object by the first object results in an evaluation of the second Object as perspective concealed, regardless of the degree of perspective occlusion.

In some cases this does not match the behavior of real image processing sensor systems. In general, it is to be expected that a real sensor system, which is set up for the identification of objects in a sensor image, detects an object that is recognizable but perspectively concealed for the sensor system, provided that the degree of perspective concealment does not exceed a certain tolerance value, and that Sensor system does not recognize the perspective hidden object when the degree of perspective occlusion exceeds the tolerance value.

In order to simulate this behavior when using a collision detection, it is provided in one embodiment of the invention to determine a tolerance value for a perspective masking of an object recognizable for the sensor system, to calculate a scaling factor from the tolerance value, and at least the first geometric body before the projection the second geometric body to scale the scaling factor smaller or to scale down at least the first image or the second image by the scale factor before performing the collision detection. In the description of Figure 7, an exemplary algorithm for calculating scaling factors is disclosed.

In general, especially with appropriately skilled programming of the projection process or the collision detection, it is not always necessary to complete the projection process. For a further acceleration of the method according to the invention, it is provided in a particularly advantageous embodiment, already during a current projection operation of the first geometric body or the second geometric body, in particular wherein the projections of the first geometric body and the second geometric body are performed in parallel, a collision detection perform on the basis of the images of the already projected on the projection plane points of the first geometric body and the second geometric body and abort the incomplete projections when based on the images of the already projected points a collision is detected or a collision is excluded.

The claimed invention also encompasses a computer system configured to carry out a method according to the invention and a computer program product for setting up a computer system for carrying out a method according to the invention.

Figur 1

ein zur Durchführung eines erfindungsgemäßen Verfahrens eingerichtetes Computersystem;

Figur 2

eine auf dem Computersystem hinterlegte virtuelle Testumgebung;

Figur 3

einen beispielhaften Projektionsvorgang eines ersten geometrischen Körpers und eines zweiten geometrischen Körpers;

Figur 4

ein Verfahren zur Auswahl eines zur Projektion vorgesehenen Punktes eines geometrischen Körpers;

Figur 5

ein erstes Beispiel für einen Abbruch eines Projektionsvorgangs nach Ausschluss einer Kollision;

Figur 6

ein zweites Beispiel für einen Abbruch eines Projektionsvorgangs nach Erkennung einer Kollision; und

Figur 7

beispielhafte Skalierungen des ersten oder des zweiten geometrischen Körpers.

Hereinafter, an advantageous embodiment will be described with reference to the figures, which explains the invention in more detail. Show it

FIG. 1

a computer system configured to carry out a method according to the invention;

FIG. 2

a virtual test environment stored on the computer system;

FIG. 3

an exemplary projection process of a first geometric body and a second geometric body;

FIG. 4

a method for selecting a point of projection of a geometric body;

FIG. 5

a first example of a termination of a projection process after exclusion of a collision;

FIG. 6

a second example of a termination of a projection process after detection of a collision; and

FIG. 7

exemplary scalings of the first or the second geometric body.

The picture of the FIG. 1 shows a set up as a test computer system TB for a virtual test designed as a driver assistance system radar-based control system UUT. The computer system TB comprises a simulation computer SI, a radar test bench RD and a host computer H. The simulation computer SI comprises a processor unit C, on which a real-time operating system (not shown) and a virtual test environment VT are stored, and an interface unit IN for exchanging data between the virtual test environment VT and the periphery of the simulation computer SI. Deposited on the host computer H is a computer program product for setting up the virtual test environment VT and for storing the virtual test environment VT on the processor unit C, as well as operating software for monitoring and activating the virtual test environment VT on the processor unit C during a simulation run of the virtual test environment VT.

The computer system TB includes as a DUT the radar-based control system UUT with a transceiver unit TR, a first controller E1 and a second controller E2. The transceiver unit TR is set up to emit a radar pulse RS, to receive it by means of an imaging radar sensor radar echo RE of the radar pulse RS and to create a sensor image of an environment of the transceiver unit TR on the basis of the radar returns RE. The first control device E1 comprises an image processing unit for detecting objects in the sensor image and is also set up to use the detected objects to create a description of the environment of the transceiver unit. The second control unit E2 is set up to read in the description of the environment, to evaluate it and, based on the description of the environment, to control a virtual automated vehicle VE in the virtual test environment VT.

The radar test rig RD comprises a number of movable radar transmitters T arranged on circular arcs which are set up to emit radar pulses for the simulation of radar echoes RE. The simulation computer SI is set up to control the radar test stand RD. The simulation computer positions the radar transmitter T and adjusts the simulated radar returns RE in time such that the radar test rig RD emulates radar returns RE of virtual objects in the virtual test environment to a radar pulse emitted by the transceiver unit TR.

The picture of the FIG. 2 shows an exemplary snapshot from the virtual test environment VT. The virtual test environment comprises the virtual automated vehicle VE driven by the second control device E2 on a virtual road R and a plurality of further virtual objects O3,..., O10. The virtual objects include, for example, vehicles O3, O4, O9, O10, building O5, trees O6, O7 and traffic signs O8. Each virtual object O3,..., O10 is assigned static or variable location coordinates in the virtual test environment VT.

A sensor S (hidden) is suspended on the front of the virtual automated vehicle VE. The sensor S is part of a simulation of a sensor system, which in addition to the sensor S also includes an image processing unit for the identification of objects. The sensor S is assigned a pyramid-shaped sensor field FV emanating from the sensor S, the spatial orientation of which is predetermined by the location coordinates and the spatial orientation of the virtual automated vehicle VE. At least a subset of the objects present in the virtual test environment VT is determined as objects recognizable by the sensor system. All objects O1,..., O8 recognizable by the sensor system, detected by the sensor field of view FV are included in a preliminary list of detected virtual objects. For all objects listed in the preliminary list, a check for perspective occlusion is performed, and all objects considered to be perspectively obscured are removed from the preliminary list. The objects still listed after completion of the checks for perspective occlusion form a list of detected objects, and the virtual test environment VT generates a signal of the sensor system, from which the objects listed in the list of detected objects are removable.

The exact configuration of the signal of the sensor system may vary depending on the application. The signal may carry further information about the listed objects, for example their absolute or relative to the sensor S location coordinates. By way of example, the signal can simulate a possible signal of a real sensor system and be led out to the outside via an interface unit IN for evaluation by a control system UUT. In the in the FIGS. 1 and 2 illustrated embodiment, the signal does not detect a signal of a real sensor system and is not led to the outside, but evaluated by the simulation computer SI for controlling the radar test bench RD.

In the exemplary embodiment described here, all virtual objects O3,..., O10 in the virtual test environment VT can be recognized by the sensor system. In other embodiments, some of the virtual objects present in the virtual test environment VT are not recognizable by the sensor system. In principle, only objects recognizable by the sensor system are included in the preliminary list of recognizable objects. Advantageously, however, objects that can not be detected by the sensor system are also able to cover objects that can be recognized by the sensor system in a perspective manner.

Each object O1,..., O10 in the virtual test environment VT is assigned an envelope cuboid B1, B2 as an enveloping body, referred to in the art as a bounding box.

The radar test stand RD should emulate only radar echoes RE of objects O3,..., O10 in the virtual test environment VT, which are not concealed in perspective from the perspective of the sensor S or whose perspective concealment falls below a tolerance value, so that the first control device E1 has a corresponding perspective view would recognize a hidden physical object as a stand-alone object. In the snapshot taken in the FIG. 2 is illustrated, three vehicles O3, O9, O10, a building O5, a tree O7 and a road sign O8 are detected by the sensor field of view FV and therefore included in the preliminary list of detected objects. However, the vehicle O10 is hidden in perspective by the vehicle O9, the traffic sign O8 is partially hidden in perspective by the vehicle O9, and the vehicle O3 is partially hidden in perspective by the building O5.

The figures described below represent a method according to the invention for estimating a perspective occlusion.

The picture of the FIG. 3 shows a three-dimensional scene in the virtual test environment VT with the sensor S, a first object O1 in a first envelope cuboid B1 and a second object O2 in a second envelope cuboid B2. The first object O1 and the second object O2 are both detected by the sensor field of view FV and included in the preliminary list of detected objects. First, the Euclidean distance of the first object O1 to the sensor is determined based on the location coordinates of the first object O1 and the location coordinates of the sensor S1, and based on the location coordinates of the second object O2 and the location coordinates of the sensor S, the Euclidean distance of the second object O2 becomes the sensor determined. The Euclidean distances are compared, and for the object farther from the sensor S, a perspective occlusion is checked by the object closer to the sensor S as viewed by the sensor S. By way of example, it is checked whether the second object O2 is hidden in perspective by the first object O1 from the perspective of the sensor S.

For this purpose, a projection plane P is spanned, on which a biaxial local coordinate system CS is defined, and the first envelope block B1 and the second envelope cuboid B2 are at least partially projected onto the projection plane P, with a central projection with the spatial coordinates of the sensor S being used as an eye point for the projection. Since the envelope cubes B1, B2 are convex bodies, their images I1, I2 on the projection plane are also convex. The projection of the first envelope cuboid B1 takes place in such a way that, before carrying out the projection, vertices of the first envelope quadrant B1 are determined whose images on the projection plane P are the vertices of the first image I1 of the first envelope quadrant B1. A numerical method for determining the corresponding vertices is described below in the description of FIG FIG. 4 disclosed. The projection of all six vertices visible from the perspective of the sensor S of the first envelope squiggle B1 and the formation of the convex envelope of the images of the projected vertices would result in an exact image I1 of the first envelope squared circle B1.

In the same way, a second image I2 of the second envelope square B2 is created on the projection plane P. Instead of Hüllquadern other enveloping bodies with different geometry can be used, the images of which, for example, have greater conformity with the images of the enveloped by the enveloping virtual objects.

also die Menge, die durch Subtraktion jedes Elements von B von jedem Element von A entsteht. Eine Schnittmenge von A und B existiert genau dann, wenn der Ursprung ein Element von Σ ist. Der GJK-Algorithmus sieht vor, durch eine Zerlegung der Minkowski-Differenz Σ in Dreiecke zu überprüfen, ob der Ursprung des lokalen Koordinatensystem CS ein Element der Minkowski-Differenz Σ des ersten Abbilds I1 und des zweiten Abbilds I2 ist. Durch die Verwendung polyedrischer Hüllkörper für das erste Objekt O1 und das zweite Objekt O2 mit nur wenigen Vertices sind das erste Abbild I1 und das zweite Abbild I2, und in der Folge auch deren Minkowski-Differenz Σ, Polygonzüge mit wenigen Vertices, sind also in eine geringe Anzahl von Dreiecken (Simplexe) zerlegbar. Dadurch ist ein schneller Durchlauf des GJK-Algorithmus begünstigt.A collision detection of the first image I1 and the second image I1 is performed by means of a GJK algorithm. A collision exists if and only if an intersection of the first image I1 and the second image I2 exists, and the existence of an intersection is evaluated as perspective concealment of the second object O2 by the first object O1. The GJK algorithm proposes to first form the Minkowski difference of the first image I1 and the second image I2 in the local coordinate system CS. The Minkowski difference Σ of two sets A, B is the set $$\Sigma \left(\mathrm{A},\mathrm{B}\right)=\left\{a-b|a\in A.b\in B\right\}.$$

that is, the set produced by subtracting each element of B from each element of A. An intersection of A and B exists if and only if the origin is an element of Σ. The GJK algorithm sees to check whether the origin of the local coordinate system CS is an element of the Minkowski difference Σ of the first image I1 and the second image I2 by a decomposition of the Minkowski difference Σ into triangles. By using polyhedral enveloping bodies for the first object O1 and the second object O2 with only a few vertices, the first image I1 and the second image I2, and consequently also their Minkowski difference Σ, are polygons with few vertices, ie they are in one small number of triangles (simplexes) can be dismantled. This promotes a faster run of the GJK algorithm.

The picture of the FIG. 3 For illustration of the projection process alone, a complete projection of the first envelope squares B1 and B2 is shown in each case. In fact, it is provided to project both envelope cuboids B1, B2 in parallel, already during the projection process, to form or partially form the Minkowski difference of the already projected point clouds of the vertices of both enveloping bodies B1, B2 and to cancel the projection process, as soon as the images of the already projected Vertices or points a collision of the first image I1 and the second image I2 is detected or excluded. This aspect of the invention will be explained in more detail in the following descriptions.

The picture of the FIG. 4 outlines a search algorithm for determining a projection of the selection of points of an enveloping body using the example of the second envelope quad B2.

• Die Suchebene SP liegt orthogonal zur Projektionsebene P, und
• die Suchebene SP liegt parallel zur Suchrichtung SD1.

First, an initial search direction SD1 lying parallel to the projection plane P is determined (rules for selecting the initial search direction are described in the descriptions of FIG Figures 5 and 6 introduced) and based on the Initialsuchrichtung SD1 spanned a search plane SP, which meets the following conditions:

• The search plane SP is orthogonal to the projection plane P, and
• the search plane SP is parallel to the search direction SD1.

1. 1) Suche einen ersten Punkt P1 des zweiten Hüllquaders, der bezogen auf die Suchrichtung SD1 am weitesten außen liegt.
   Der erste Punkt P1 lässt sich gleichbedeutend auch als der Punkt definieren, auf dem eine Fläche, deren Normalenvektor parallel zur Suchrichtung SD1 liegt, aufliegen würde, sodass die Fläche nur einen einzigen Schnittpunkt mit dem zweiten Hüllquader B2 umfasst und nach einer beliebig kleinen Verschiebung in Suchrichtung SD1 keinen Schnittpunkt mit dem zweiten Hüllquader B2 mehr aufweisen würde.
2. 2) Ermittle den Verbindungsvektor V vom Augpunkt, d.h. den Ortskoordinaten des Sensors S, zum ersten Punkt P1.
3. 3) Bilde eine neue Suchrichtung SD2, die parallel zur Suchebene SP und orthogonal zum Verbindungsvektor V liegt und deren Skalarprodukt mit der alten Suchrichtung SD1 positiv ist, um einen zweiten Punkt P2 zu suchen, der bezogen auf die neue Suchrichtung SD2 am weitesten außen liegt.
4. 4) Wiederhole die Schritte 1) bis 3), bis der Abstand zwischen dem im aktuellen Iterationsschritt gefundenen Punkt und dem im vorhergehenden Iterationsschritt gefundenen Punkt einen Toleranzwert unterschreitet.

The search algorithm includes the following steps:

1. 1) Find a first point P1 of the second envelope squared closest to the search direction SD1.
   The first point P1 can be synonymously defined as the point at which a surface whose normal vector is parallel to the search direction SD1 would rest, so that the surface comprises only a single intersection with the second envelope cuboid B2 and after any small shift in the search direction SD1 would have no intersection with the second envelope cuboid B2 more.
2. 2) Determine the connection vector V from the eye point, ie the location coordinates of the sensor S, to the first point P1.
3. 3) Create a new search direction SD2 that is parallel to the search plane SP and orthogonal to the connection vector V and whose scalar product is positive with the old search direction SD1 to search for a second point P2 that is outermost with respect to the new search direction SD2.
4. 4) Repeat steps 1) to 3) until the distance between the point found in the current iteration step and the point found in the previous iteration step falls below a tolerance value.

In the in the FIG. 4 In the example shown, the second point P2 would be determined as the point of the second envelope square B2 provided for projection on the basis of the described search algorithm. A subsequent search with an initial search direction lying in the opposite direction to the search direction SD1 would determine a third point P3 as the point to be projected. In principle, the search algorithm can be performed as often as desired with changing search directions. Each search leads to the determination of an edge point of the second envelope square B2, which is far outward from the perspective of the sensor S and would belong to the convex envelope of the image of the second envelope square B2 in a complete central projection of the second envelope cuboid B2 with the location coordinates of the sensor S ,

By multiple execution of the search algorithm with different initial search directions, a selection of points P2, P3 of the second envelope square B2 is determined, and the projection of the second envelope square is performed by selecting points P2, P3 in a central projection with the location coordinates of the sensor S as an eye point is projected onto the projection plane P.

If the selection of points P2, P3 is sufficiently extensive, the image I2 corresponds exactly to the actual image of the second envelope, as would result from a complete central projection. The search algorithm can be applied to any enveloping body, including non-polyhedral enveloping bodies, for example cylindrical enveloping bodies, wherein the enveloping bodies are advantageously convex enveloping bodies. By determining a selection of points of a convex envelope on the basis of the search algorithm and forming the convex set of the images of the selection of points, it is always possible to create an image of the enveloping body which approximately corresponds to the image resulting from a complete projection of the enveloping body.

The determination of a selection of points for projection is performed in parallel for the first envelope cuboid B1 and for the second envelope cuboid B2, and the points determined using the search algorithm are each projected onto the projection plane P immediately after their determination. On the projection plane P, a collision detection on the images of the already projected points is already performed during the ongoing projection process of the first Hüllquaders B1 and the second Hüllquaders B2, and the projection is aborted when based on the images of the already projected points a collision is detected or a collision is excluded.

The picture of the FIG. 5 illustrates a first example of premature termination of a projection of a first envelope B1 and a second envelope B2. In the example, the first enveloping body B1 is shown as a cube and the second enveloping body B2 is shown as a sphere. First, a first initial search direction SD1-A is set to use the in the FIG. 4 illustrated Search algorithm to find from the perspective of the sensor S furthest in the direction of the first Initialsuchrichtung SD1-A first point P1-A of the first envelope body B1.

In order to determine the first initial search direction SD1-A, it is basically provided in the exemplary embodiment described to subtract the spatial coordinates of the first envelope B1 from the position coordinates of the second envelope B2 and to project the connection vector resulting from the subtraction onto the projection plane P.

The first point P1-A is projected onto the projection plane P immediately after its detection to produce an image I1-A of the first point. Subsequently, a second initial search direction SD1-B opposite the first initial search direction SD1-A is determined in order to find the second point P1-B of the second envelope B2 lying farthest in the direction of the sensor S in the direction of the second initial search direction SD1-B. The second point P1-B is projected onto the projection plane P immediately after its detection to produce an image I1-B of the second point.

Immediately after projection of the second point P1-B, the difference D of the image of the first point I1-A and the image of the second point I1-B is formed. The difference D is a point of the Minkowski difference Σ of the images of the first enveloping body B1 and the second enveloping body B2. From the negative scalar product of the first initial search direction SD1-A and the difference D, since both the first enveloping body B1 and the second enveloping body B2 are convex bodies, the image I1-A of the first point and the image I1-B of the second point are the nearest points of the images of the first envelope B1 and the second envelope B2, and hence the difference D is the point of the Minkowski difference Σ nearest the origin of the local coordinate CS. The Minkowski difference Σ can therefore not include the origin of the local coordinate system CS. The projection is therefore aborted after the projection of the first point P1-A and the second point P1-B, and the second object enveloped by the second envelope B2 O2 is not considered to be hidden in perspective, as long as the second enveloping body B2 is not hidden in perspective by another enveloping body other than the first enveloping body B1.

The picture of the FIG. 6 illustrates a second example of premature termination of a projection and at the same time an advantageous embodiment of the projection process. Basically, the following listed method steps are provided for the detection of a perspective occlusion of the second envelope B2 by the first envelope B1. The selection of a point provided for the projection happens basically as in the description of FIG. 4 described using the currently valid Initialsuchrichtung. In the following it is assumed that the Euclidean distance of the second virtual object O 2 to the sensor S is greater than the Euclidean distance of the first virtual object O 1 to the sensor S.

• 1.1) Definiere eine erste Initialsuchrichtung SD1-A durch Projektion des Vektors, der sich durch Subtraktion der Ortskoordinaten des ersten virtuellen Objekts O1 von den Ortskoordinaten des zweiten virtuellen Objekts O2 ergibt, auf die Projektionsebene P, und definiere eine der ersten Initialsuchrichtung SD1-A entgegengesetzte Initialsuchrichtung SD1-B.
• 1.2) Wähle anhand der ersten Initialsuchrichtung SD1-A einen ersten Punkt P1-A des ersten Hüllkörpers B1 aus und projiziere ihn auf die Projektionsebene P, um ein Abbild I1-A des ersten Punktes P1-A des ersten Hüllkörpers B1 zu erzeugen. Wähle anhand der entgegengesetzten Initialsuchrichtung SD1-B einen ersten Punkt P1-B des zweiten Hüllkörpers B2 aus und projiziere ihn auf die Projektionsebene P, um ein Abbild I1-B des ersten Punktes P1-B des zweiten Hüllkörpers B2 zu erzeugen
• 1.3) Bilde eine erste Differenz D1 der Abbilder I1-A und I1-B in dem lokalen Koordinatensystem CS.
• 2.1) Definiere eine zweite Initialsuchrichtung SD2-A, die ausgehend von der ersten Differenz D1 in Richtung des Ursprungs des lokalen Koordinatensystems CS zeigt, wähle anhand der zweiten Initialsuchrichtung SD2-A einen zweiten Punkt P2-A des ersten Hüllkörpers B1 aus und projiziere ihn auf die Projektionsebene P, um ein Abbild I2-A des Punktes P2-A zu erzeugen.
• 2.2) Wähle anhand einer der zweiten Initialsuchrichtung SD2-A entgegengesetzten Initialsuchrichtung SD2-B einen zweiten Punkt P2-B des zweiten Hüllkörpers B2 aus und projiziere ihn auf die Projektionsebene P, um ein Abbild I2-B des Punktes P2-B zu erzeugen.
• 2.3) Bilde eine zweite Differenz D2 der zuletzt erzeugten Abbilder I2-A und I2-B im lokalen Koordinatensystem CS.
• 3.1) Definiere eine dritte Initialsuchrichtung SD3-A, die senkrecht zur Verbindungslinie der ersten Differenz D1 und der zweiten Differenz liegt und in Richtung des Ursprungs des lokalen Koordinatensystems CS zeigt, wähle anhand der dritten Initialsuchrichtung SD3-A einen dritten Punkt P3-A des ersten Hüllkörpers B1 aus und projiziere ihn auf die Projektionsebene P, um ein Abbild I3-A des dritten Punktes P3-A des ersten Hüllkörpers B1 zu erzeugen.
• 3.2) Wähle anhand einer der dritten Suchrichtung SD3-A entgegengesetzten Suchrichtung SD3-B einen dritten Punkt P3-B des zweiten Hüllkörpers B2 aus und projiziere ihn auf die Projektionsebene P, um ein Abbild I3-B des Punktes P3-B zu erzeugen.
• 3.3) Bilde eine dritte Differenz D3 der zuletzt erzeugten Abbilder I3-A und I3-B im lokalen Koordinatensystem CS.
• 4) Wenn das aus der ersten Differenz D1, der zweiten Differenz D2 und der dritten Differenz D3 aufgespannte Simplex der Minkowski-Differenz Σ den Ursprung des lokalen Koordinatensystems CS umfasst, brich den Projektionsvorgang ab und werte das zweite virtuelle Objekt O2 als perspektivisch verdeckt.
• 5.1) Wenn das Simplex den Ursprung des lokalen Koordinatensystems CS nicht umfasst, verwirf den am weitesten vom Ursprung entfernten Eckpunkt des Simplex und definiere eine neue Initialsuchrichtung, die senkrecht zur Verbindungslinie der beiden verbleibenden Eckpunkte liegt und in Richtung des Ursprungs des lokalen Koordinatensystems CS zeigt. Wähle anhand der neuen Suchrichtung einen Punkt des ersten Hüllkörpers B1 aus und projiziere ihn auf die Projektionsebene P, um ein Abbild des Punktes zu erzeugen.
• 5.2) Wähle anhand einer der neuen Initialsuchrichtung entgegengesetzten Suchrichtung einen Punkt des zweiten Hüllkörpers B2 aus und projiziere ihn auf die Projektionsebene P, um ein Abbild des Punktes zu erzeugen.
• 5.3) Bilde die Differenz der zwei zuletzt erzeugten Abbilder im lokalen Koordinatensystem CS und verwende die Differenz als neuen Eckpunkt, um zusammen mit den zwei verbleibenden Eckpunkten ein neues Simplex aufzuspannen.
• 6) Wiederhole die Schritte 5.1 bis 5.3, bis entweder das aktuelle Simplex den Ursprung des lokalen Koordinatensystems CS umfasst (Abbruchkriterium A) oder das Skalarprodukt der aktuellen für die Auswahl eines Punktes des ersten Hüllkörpers B1 verwendeten Initialsuchrichtung und des anhand der aktuellen Initialsuchrichtung gesetzten Eckpunkts negativ ist (Abbruchkriterium B).

The process steps are as follows:

• 1.1) Define a first initial search direction SD1-A by projecting the vector obtained by subtracting the location coordinates of the first virtual object O1 from the location coordinates of the second virtual object O2 to the projection plane P, and defining one of the first initial search direction SD1-A Initial search direction SD1-B.
• 1.2) Select on the basis of the first initial search direction SD1-A a first point P1-A of the first envelope B1 and project it onto the projection plane P in order to produce an image I1-A of the first point P1-A of the first envelope B1. Using the opposite initial search direction SD1-B, select a first point P1-B of the second envelope B2 and project it onto the projection plane P to produce an image I1-B of the first point P1-B of the second envelope B2
• 1.3) Make a first difference D1 of the images I1-A and I1-B in the local coordinate system CS.
• 2.1) Define a second initial search direction SD2-A, which points from the first difference D1 towards the origin of the local coordinate system CS, select a second point P2-A of the first envelope B1 from the second initial search direction SD2-A and project it the projection plane P to produce an image I2-A of the point P2-A.
• 2.2) Select on the basis of one of the second initial search direction SD2-A opposite Initialsuchrichtung SD2-B from a second point P2-B of the second envelope B2 and project it to the projection plane P to produce an image I2-B of the point P2-B.
• 2.3) Create a second difference D2 of the most recently generated images I2-A and I2-B in the local coordinate system CS.
• 3.1) Define a third initial search direction SD3-A which is perpendicular to the connecting line of the first difference D1 and the second difference and points towards the origin of the local coordinate system CS, select a third point P3-A of the first from the third initial search direction SD3-A Boundary body B1 and projecting it on the projection plane P to produce an image I3-A of the third point P3-A of the first envelope body B1.
• 3.2) Select a third point P3-B of the second envelope B2 from a search direction SD3-B opposite to the third search direction SD3-A and project it onto the projection plane P to produce an image I3-B of the point P3-B.
• 3.3) Form a third difference D3 of the last generated images I3-A and I3-B in the local coordinate system CS.
• 4) If the simplex of the Minkowski difference Σ spanned from the first difference D1, the second difference D2 and the third difference D3 comprises the origin of the local coordinate system CS, abort the projection process and evaluate the second virtual object O2 as being perspectively hidden.
• 5.1) If the simplex does not include the origin of the local coordinate system CS, discard the most distant vertex of the simplex and define a new initial search direction that is perpendicular to the connecting line of the two remaining vertices pointing towards the origin of the local coordinate system CS. Using the new search direction, select a point of the first envelope B1 and project it onto the projection plane P to create an image of the point.
• 5.2) Using a search direction opposite to the new initial search direction, select a point of the second envelope B2 and project it onto the projection plane P to produce an image of the point.
• 5.3) Compute the difference of the two most recently generated images in the local coordinate system CS and use the difference as a new vertex to span a new simplex together with the two remaining vertices.
• 6) Repeat steps 5.1 to 5.3 until either the current simplex comprises the origin of the local coordinate system CS (abort criterion A) or the scalar product of the current initial search direction used for selecting a point of the first envelope B1 and the vertex set using the current initial search direction is (termination criterion B).

If abort criterion A occurs, the second virtual object O2 is considered to be hidden in perspective. If abort criterion B occurs, the second virtual object O2 is considered to be not hidden in perspective. The testing and application of the termination criterion A is of course only possible if at least three vertices D1, D2, D3 were determined. Termination criterion B is already applicable after the determination of a single vertex. In the in the FIG. 6 In the example shown, after the third vertex D3 has been determined, the projection process is aborted and the second virtual object O2 is regarded as being hidden in perspective, because the simplex spanned by the three corner points D1, D2, D3 Minkowski difference Σ includes the origin of the local coordinate system CS and thus a partial perspective occlusion of the second envelope B2 is detected by the first envelope body B1.

Although the existence of an intersection of two sets is detectable by the GJK algorithm, it is not possible to measure the size of the intersection. Consequently, an arbitrarily small perspective occlusion of the second enveloping body B2 by the first enveloping body leads to the fact that the second enveloping body B2 is concealed as perspectively and is therefore considered not to be visible to the sensor system. From a real sensor system with an imaging sensor S and an image processing unit, however, it is to be expected that the image processing unit also recognizes partially occluded objects as such, provided that the perspective occlusion does not exceed a tolerance value.

In order to emulate this realistic behavior, a tolerance value T can be defined for a perspective occlusion of a virtual object O1,..., O8. A tolerance value of T = 0.25 means, for example, that a virtual object located in the sensor field of view FV and recognizable by the sensor system is to be recognized by the sensor system if, from the perspective of the sensor S, no more than approximately one quarter of the surface of the virtual object is in perspective is covered. Depending on the selected tolerance value, at least the first enveloping body B1 or the second enveloping body B2 is scaled smaller. The second virtual object O2 is considered to be hidden in perspective if an intersection of the images I1, I2 of the scaled envelopes exists.

werden die Skalierungsfaktoren wie folgt festgelegt:
s ₁ = 1 $$s_2=2\left(0,5-T\right)$$

For the first enveloping body B1, a first scaling factor s _{1 is} introduced. For the second enveloping body B2, a second scaling factor s _{2 is} introduced. For tolerance values in the range of $$0\le T\le 0.5$$

the scaling factors are set as follows:
s ₁ = 1 $$s_2=2\left(0.5-T\right)$$

werden die Skalierungsfaktoren wie folgt festgelegt: $$\begin{array}{c}{s}_1=1-2\left(\sqrt{\frac{A_2}{A_1}}\right)\left(T-0,5\right)\\ s_2=0\end{array}$$

A₁ ist ein exakter oder abgeschätzter Wert für die Fläche des ersten Abbilds I1 des ersten Hüllkörpers B1 unter der Annahme einer vollständig durchgeführten Projektion, und A₂ ist ein exakter oder abgeschätzter Wert für die Fläche des zweiten Abbilds I2 des zweiten Hüllkörpers B2. Die Fläche A₁ des Abbilds eines quaderförmigen Hüllkörpers wird anhand folgender Formel abgeschätzt: $$A_1=\frac{1}{L^2}{\displaystyle \sum _{i=1}^3}{D}_i\cdot e_s\cdot e_{n,i}$$

For tolerance values in the range of $$0.5<T\le 1$$

the scaling factors are set as follows: $$\begin{array}{c}{s}_1=1-2\left(\sqrt{\frac{A_2}{A_1}}\right)\left(T-0.5\right)\\ s_2=0\end{array}$$

A ₁ is an exact or estimated value for the area of the first image I1 of the first envelope B1 assuming full projection, and A ₂ is an exact or estimated value for the area of the second image I2 of the second envelope B2. The area A _{1 of} the image of a cuboid envelope is estimated using the following formula: $$A_1=\frac{1}{L^2}{\displaystyle \underset{i=1}{\overset{3}{\Sigma }}}{D}_i\cdot e_s\cdot e_{n.i}$$

In this case, L is the Euclidean distance between projection plane P and the spatial coordinates of the virtual object enveloped by the enveloping body, D ₁ , D ₂ and D _{3 are} three mutually orthogonal faces of the cuboid, e _{s is} the difference vector of the spatial coordinates of the virtual object enveloped by the enveloping body and of Sensors S, and e _{n, 1} , e _{n, 2} , and e _{n, 3} are the surface normals.

The picture of the FIG. 7 shows applications of this scaling rule for three different tolerance values. The first enveloping body B1 and the second enveloping body B2 are shown in the figure for the sake of clarity as two-dimensional rectangles of the edge lengths 8cm x 6cm and 6cm x 4cm. Dashed lines represent the unscaled envelope, while solid lines represent the scaled envelope.

With a tolerance value T = 0.25, the first enveloping body B1 remains unscaled and the second enveloping body has a scaling factor of 0.5. The first enveloping body B1 must cover approximately one quarter of the visible area of the unscaled second enveloping body B2 before a perspective concealment of the scaled second enveloping body B2 occurs.

With a tolerance value of T = 0.5, the first enveloping body B1 remains unscaled, and the second enveloping body B2 is scaled to point size. The first enveloping body B1 must cover the visible area of the unscaled second enveloping body B2 approximately halfway before a perspective concealment of the scaled second enveloping body B2 occurs.

With a tolerance value of T = 1, a scaling factor of s ₁ = 0.5 results for the first enveloping body B1, and the second enveloping body B2 is scaled to point size. The unscaled first enveloping body B1 must cover the unscaled second enveloping body B2 approximately completely before a perspective concealment of the scaled second enveloping body B2 occurs through the scaled first enveloping body B1.

That in the FIG. 7 illustrated scaling methods is universally applicable if all virtual objects O1, ..., O8 in the virtual test environment VT have similar dimensions. Otherwise, a universal application can be problematic. Since the potentially perspective concealed second enveloping body B2 can be scaled down to point size, a perspective occlusion can result from an arbitrarily small first enveloping body B1, which is not a realistic behavior of a sensor system according to the invention. For example, a method according to the invention in a situation as described in the FIG. 3 is shown to conclude that the vehicle O1 is not visible to the radar-based control system UUT because it is hidden by the child O2.

In the case of strongly divergent dimensions of the virtual objects O1,..., O8 in the virtual test environment VT, it is therefore advantageous to use the previously described estimates of the surfaces of the images I1, I2 to pre-evaluate the plausibility of a perspective view Occlusion. For example, the method may be extended such that the check for perspective occlusion is performed only if the estimated area of the first image I1 exceeds a predetermined value in relation to the estimated area of the second image I2.

Claims (14)

Method for simulating a sensor system comprising an imaging sensor (S) and an image processing unit for detecting objects (O1, ..., O10) in the sensor image output by the sensor (S),
wherein in a virtual test environment (VT) comprising a multiplicity of virtual objects (O1, ..., O10), spatial coordinates are assigned to the sensor (S) and spatial coordinates are assigned to each virtual object (O1, ..., O10);
the sensor (S) is assigned a sensor field of view (FV) in the virtual test environment (VT) and a number of objects of the plurality of virtual objects (O1, ..., O10) are determined as virtual objects detectable by the sensor system;
a simulation run is started in the course of which the sensor (S) or at least one virtual object (O1,..., O10) is cyclically assigned new location coordinates in the virtual test environment (VT),
and cyclically checking whether at least one virtual object (O1, ..., O10) is detected by the sensor field of view (FV);
characterized in that in the case that a first virtual object (O1) and a second virtual object (O2) recognizable by the sensor system are detected by the sensor field of view (FV), - A suitable for a central projection of the first virtual object (O1) and the second virtual object (O2) with the spatial coordinates of the sensor (S) as an eye point projection plane (P) is spanned; a first image (I1) is formed on the projection plane (P) in that a first geometric body (B1) which is the same as the first virtual object (O1) and whose dimensions at least approximately coincide with the dimensions of the first virtual object (O1), is projected in a central projection with the spatial coordinates of the sensor (S) as an eye point at least partially on the projection plane (P); a second image (I2) is formed on the projection plane (P) by a second geometric body (B2) having the same location as the second virtual object (O2) whose dimensions at least approximately coincide with the dimensions of the second virtual object (O2), is projected in a central projection with the spatial coordinates of the sensor (S) as an eye point at least partially on the projection plane (P); it is checked whether an intersection of the first image (I1) and the second image (I2) exists; - in the case that i. an intersection exists, ii. the Euclidean distance between the sensor (S) and the first virtual object (O1) is less than the Euclidean distance between the sensor and the second virtual object (O2) and iii. the size of the intersection, in particular in relation to the size of the second image (I2), exceeds a predefined threshold value, the second virtual object (O2) is considered to be hidden in perspective; - in the event that at least one of the conditions i., ii. or iii. is not satisfied, the second virtual object (O2) is considered as not being hidden in perspective; and - A signal of the sensor system is generated, which can be deduced that all detected by the sensor field of view (FV) and recognizable by the sensor system virtual objects (O1, ..., O10), which are considered not to be hidden in perspective, recognized by the image processing unit and all detected by the sensor field of view (FV) and recognizable by the sensor system virtual objects (O1, ..., O10), which are considered to be hidden in perspective, were not recognized by the image processing unit. The method of claim 1, wherein the sensor (S) to a virtual machine, in particular a virtual automated vehicle (VE), suspended in the virtual test environment (VT) and the signal of the sensor system from one set up for driving the virtual machine Control system (UUT) or monitored by a set up to control the virtual machine control software. The method of claim 1, wherein the first geometric body (B1) is a first envelope body of the first object (O1) and the second geometric body (B2) is a second envelope body of the second object (O2). Method according to one of claims 1 to 3, wherein a selection of points (P1-A, P2-A, P1-B, ..., P3-B) of a geometric body (B1, B2) is projected and
the image (I1, I2) of the geometric body (B1, B2) is formed by taking the point cloud formed by the images (I1-A, I1-B, I2-A, ..., I2-B) of the selected points a two-dimensional set is formed on the projection plane (P). The method of claim 4, wherein the two-dimensional set is the convex hull of the point cloud. Method according to claim 4 or 5, wherein the selection of points (P1-A, P2-A, P1-B,..., P3-B) contains exclusively points whose images are elements of the convex hull one of a complete projection of the geometric one Body (B1, B2) resulting image of the geometric body (B1, B2) are. Method according to one of the preceding claims, wherein for checking the existence of an intersection, a collision detection for the first image (I1) and the second image (I2) is performed and the threshold value for the size of the intersection is equal to zero. The method of claim 7, wherein for the collision detection a Gilbert-Johnson-Keerthi algorithm is applied. Method according to claim 7, wherein a tolerance value for a perspective concealment of a virtual object (O1,..., O10) recognizable for the sensor system is determined, from the tolerance value a scaling factor is calculated and the first geometric body (B1) or the second geometric Body (B2) is scaled smaller by the scaling factor before performing the projection. Method according to claim 7, wherein a tolerance value for a perspective masking of a virtual object (O1,..., O10) recognizable for the sensor system is determined, from the tolerance value a scaling factor is calculated and the first image (I1) or the second image ( I2) is scaled down before collision detection is performed. Method according to one of claims 7 to 9, wherein during the execution of the projection of the first geometric body (B1) or of the second geometric body (B2) a collision detection on the basis of the images (I1-A, I2-A, I1-B, .. ., I3-B) of the already projected to the projection plane (P) points (P1-A, P2-A, P1-B, ..., P3-B) of the first geometric body (B1) and the second geometric body ( B2) is performed
and the incomplete projections are aborted when using the images (I1-A, I2-A, I1-B, ..., I3-B) of the already projected points (P1-A, P2-A, P1-B , ..., P3-B) a collision is detected or a collision is excluded. Method according to one of claims 1 to 11, wherein the Euclidean distances of the first geometric body (B1) and the second geometric body (B2) to the sensor (S) are determined, in particular based on the location coordinates of the first virtual object (O1), the second virtual object (O2) and the sensor (S). Computer system (TB) set up to carry out a method according to one of Claims 1 to 12. Computer program product for setting up a computer system (TB) for carrying out a method according to one of Claims 1 to 12.

Images