DE102022125794A1 - Method for remotely carrying out a driving maneuver using vehicle-external sensor information for a remote controller, and electronic remote control system - Google Patents

Abstract

The invention relates to a method for remotely carrying out a driving maneuver of a vehicle (1), comprising the following steps:- detecting the environment (10) with at least one detection device (13) of the vehicle (11) to be remotely controlled and providing this environmental information detected by the vehicle itself;- detecting the environment (10) in an environmental area (16) of the vehicle (11) with at least one infrastructure detection device (18) arranged externally to the vehicle (11), which is arranged on a stationary infrastructure unit (16), and providing this environmental information detected externally to the vehicle;- transmitting the environmental information detected by the vehicle itself and the environmental information detected externally to the vehicle to a remote control center (1);- remotely carrying out the driving maneuver of the vehicle (1) by a remote controller (3) in the remote control center (1) depending on the environmental information detected by the vehicle and the environmental information detected externally.

Description

One aspect of the invention relates to a method for remotely carrying out a driving maneuver with a vehicle. Another aspect of the invention relates to an electronic remote control system for a vehicle.

Methods are known in which a vehicle, in particular a motor vehicle, is controlled by a human remote controller, in particular a teleoperator, positioned externally to the vehicle. The teleoperator is usually located in a fixed remote control center and thus in a building in which one or more display units, in particular screens, are also arranged. On these screens, the remote controller can recognize the vehicle to be remotely controlled and/or the surroundings of this remotely controlled vehicle, or this is displayed on the screens.

Such methods differ significantly from those in which a driver is positioned outside of his own vehicle in the immediate vicinity and must be there in order to, for example, carry out driving maneuvers with the vehicle he has left using a portable remote control that he is holding in one hand. For example, parking maneuvers are carried out here. However, such methods require, and are only permitted in this way, that the driver is positioned in close proximity to the vehicle with such a portable remote control and must also see the vehicle in order to control such driving maneuvers.

This is not the case with significantly different procedures in which the above scenario is carried out with a central remote controller located far from the vehicle. The remote controller in the remote control center is positioned in such a way that it cannot see the vehicle directly, nor can it see the vehicle's surroundings directly. With such a procedure, it is therefore not necessary and may not even be possible for the remote controller to be located within a specified close range, for example a maximum of 5 meters, around the vehicle to be remotely controlled.

Therefore, such procedures make it more difficult for a remote control operator in a remote control center to be able to maneuver the vehicle safely during corresponding movements.

The object of the present invention is to more safely carry out a method for remotely carrying out a driving maneuver with a remote controller who cannot directly see the vehicle to be remotely controlled. Furthermore, the object is to create an electronic remote control system.

This object is achieved by a method and an electronic remote control system according to the independent claims.

• - Erfassen der Umgebung mit zumindest einer Erfassungsvorrichtung des fernzusteuernden Fahrzeugs und Bereitstellen dieser fahrzeugseitig selbsterfassten Umgebungsinformationen;
• - Erfassen der Umgebung in einem Umfeldbereich des Fahrzeugs mit zumindest einer extern zum Fahrzeug angeordneten Infrastrukturerfassungsvorrichtung, die an einer, insbesondere dauerhaft, ortsfest angeordneten Infrastruktureinheit angeordnet ist und Bereitstellen dieser fahrzeugextern erfassten Umgebungsinformationen;
• - Übermitteln der fahrzeugseitig selbsterfassten Umgebungsinformationen und der fahrzeugextern erfassten Umgebungsinformationen an eine Fernsteuerzentrale;
• - ferngesteuertes Durchführen des Fahrmanövers des Fahrzeugs durch einen Fernsteuerer in der Fernsteuerzentrale abhängig von den fahrzeugseitig erfassten Umgebungsinformationen und den fahrzeugextern erfassten Umgebungsinformationen.

One aspect of the invention relates to a method for remotely carrying out a driving maneuver of a vehicle. The method comprises the following steps:

• - Detecting the environment with at least one detection device of the vehicle to be remotely controlled and providing this vehicle-side self-detected environmental information;
• - detecting the environment in an area surrounding the vehicle with at least one infrastructure detection device arranged externally to the vehicle, which is arranged on an infrastructure unit that is arranged in a fixed location, in particular permanently, and providing this environmental information recorded externally to the vehicle;
• - Transmitting the environmental information recorded by the vehicle itself and the environmental information recorded outside the vehicle to a remote control center;
• - remote-controlled execution of the driving maneuver of the vehicle by a remote controller in the remote control center depending on the environmental information recorded on the vehicle side and the environmental information recorded externally to the vehicle.

Such a method improves the teleoperated performance of a driving maneuver of a vehicle. This is because additional auxiliary information is generated and transmitted to the remote controller or teleoperator, who cannot see the vehicle and its surroundings directly and immediately when the driving maneuver is carried out remotely. This means that not only the information generated by the detection device of the remotely controlled vehicle itself, in particular from the surroundings, is available, but also additional information from very specific other vehicle-external detection devices. These are formed by the infrastructure detection device. Since such infrastructure detection devices also detect very specific areas of the surroundings, additional supplementary information can be generated from the surroundings that cannot, or cannot currently, be detected by the detection device of the vehicle itself when viewed from the current position of the vehicle. This makes it possible for the Remote controllers generate an additional information base that makes this teleoperated driving maneuver safer. In this context, safety-critical driving maneuvers can be avoided when controlling the vehicle teleoperated.

Particularly advantageous in this context is the vehicle-external environmental information that is generated using an infrastructure detection device that is fixed to an infrastructure unit. This is because this infrastructure detection device is always permanently and reliably present in the same position. This infrastructure detection device therefore always provides information from the same area. This means that detection information from this infrastructure detection device, which can also be generated for a variety of other aspects, can now also be specifically used for such a remote control process of a driving maneuver. In one embodiment, it can therefore also be said that an infrastructure detection device that is already present is now also used for such a remote-controlled driving maneuver with regard to its information collection. This means that such an infrastructure detection device that is already present is now used in a more multifunctional way.

The environmental information external to the vehicle is at least proportionally different from the environmental information recorded by the vehicle itself. In particular, if the recording is carried out simultaneously with the vehicle's recording device and with the infrastructure recording device, this environmental information is at least proportionally different. In particular, the recording areas of these recording devices only overlap in some areas at most. They can also be arranged without overlapping. In particular, this can also depend on the respective position of the vehicle, especially when the vehicle is moving.

In one embodiment, the infrastructure detection device detects at least some of the surrounding areas than the detection device. This means in particular that the detection areas of the infrastructure detection device and the detection device of the remote-controlled vehicle do not overlap or overlap at most in some areas. In this context, the added value of this information recorded by the infrastructure detection device is generated precisely by the fact that additional information is available that cannot be generated with the detection device at the current location of the vehicle. This is particularly advantageous when, in traffic situations, at least from the current position of the vehicle, sections of road or the like that are not visible cannot be detected with the detection device, but road users, for example, are coming or could come from this area of road that is not visible in the direction of the path of the remote-controlled vehicle. By detecting these road users with the infrastructure detection device, an early indication can be sent to the remote controller so that he is prepared for this when the driving maneuver is carried out remotely.

In one embodiment, the infrastructure detection device detects at least one area of the environment that is not visible from the vehicle. For example, these can also be traffic areas that are hidden from the vehicle's detection device by another vehicle or a building or the like, at least in the current position of the remote-controlled vehicle.

A traffic area is recorded interactively as an environmental area using the infrastructure recording device. Since road users such as other vehicles, for example a motor vehicle or a truck or a motorcycle or a bicycle or the like, can also be moved in a traffic area, the recording of such traffic areas is particularly advantageous. This is because these road users in particular can then also move in the direction of the path of the remote-controlled vehicle and lead to critical traffic situations. Therefore, the recording of such traffic areas is more advantageous than other environmental areas.

In one embodiment, the transmission of the environmental information recorded outside the vehicle to the remote control center is only carried out when the vehicle approaches the surrounding area. This is a very intelligent use of the infrastructure detection device. Information is not transmitted to the remote control center permanently, but only in these very specific situations. This can save computing and transmission costs. Furthermore, a remote controller in the remote control center is not overloaded with information, especially if it is not needed or helpful for him in the current situation. Since this additional information from the infrastructure detection device is particularly advantageous in situations where traffic-critical situations can arise when the vehicle is being remotely controlled to perform a driving maneuver, this above-mentioned embodiment. Therefore, the link between the position of the vehicle and the surrounding area that can be detected by the infrastructure detection device is an advantageous trigger criterion for this transmission. With regard to approaching, this can define the reference of the orientation of the vehicle and/or its direction of travel and/or its distance to the surrounding area. The orientation of the vehicle to this surrounding area can also be taken into account in this context.

In one embodiment, the position and/or direction of travel of the vehicle is determined and, depending on this, it is checked whether and/or where an infrastructure detection device is arranged in a surrounding zone for the position of the vehicle and/or in the direction of travel of the vehicle. If such an infrastructure detection device is present in such a surrounding zone around the vehicle and is arranged in particular to detect an area of the environment that cannot currently be detected by the detection device of the vehicle, the environmental information detected outside the vehicle is then transmitted from this infrastructure detection device to the remote control center. In particular, such a transmission is requested beforehand in this context. The remote control center can send a signal to this infrastructure detection device for this purpose. Depending on the receipt of this request signal by the infrastructure detection device, it then transmits its detected information to the remote control center. This is also advantageous in that the infrastructure detection device then also knows where to send its environmental information detected outside the vehicle. This also enables a very needs-based and intelligent operation of the infrastructure recording device with regard to the coupling with a remote control center.

In one embodiment, the environmental information recorded on the vehicle side and the environmental information recorded outside the vehicle are displayed on at least one display unit in the remote control center for the remote controller. In one embodiment, the remote controller can therefore also recognize this information visually. It is also possible for at least some environmental information recorded outside the vehicle to be communicated acoustically in addition to or instead of a visual representation. The remote controller can then also gain knowledge of this accordingly.

In one embodiment, the environmental information recorded outside the vehicle is displayed as a real image on a first display unit in the remote control center. In particular, this information is displayed as a real image from a perspective from the point of view of a driver in the vehicle to be remotely controlled. This gives the remote controller in his remote control center, when he looks at the at least one display unit, the impression that he is sitting directly in the vehicle to be remotely controlled. This enables a more realistic scenario for the remote controller outside the vehicle.

In one embodiment, environmental information recorded outside the vehicle is displayed symbolically in the real image. In one embodiment, this makes it possible to visually recognize the representation of this environmental information outside the vehicle very easily and intuitively. A remote controller therefore knows immediately which information was recorded by the vehicle's recording device and which comes from the infrastructure recording device.

In this context, symbolic elements can be simple auxiliary elements such as lines or frames or circles or the like. However, symbolic elements can also characterize an individual object. For example, a symbol can represent a motor vehicle or a bicycle or a motorcycle or the like. This presents the remote control operator with more detailed information, which then also makes it possible to immediately identify which object is located in the area surrounding the device that cannot yet be detected.

In a very advantageous embodiment, the environmental information recorded outside the vehicle is also displayed dynamically. This means that any moving objects in the surrounding area are then also displayed in the real image on the display unit in a correspondingly moving manner. This enables the remote control operator to recognize and easily understand the dynamics of such objects in the surrounding area.

In one embodiment, from the current perspective of the vehicle, locally obscured environmental information recorded outside the vehicle and/or environmental information recorded outside the vehicle that is not recognizable due to weather conditions and/or time of day is displayed in the real image by auxiliary symbols, in particular those displayed in a location that is accurate to reality. This means in particular that these environmental areas that are currently not recorded by the vehicle's detection device, and in particular objects located therein, are displayed to the remote controller on the at least one display unit in such a way that the remote controller can also recognize from the perspective displayed to him where these objects are in the unrecognizable and invisible area. visible surrounding area compared to the vehicle. In this context, it is also possible that objects that could currently be detected by the vehicle's detection device in different weather or other weather conditions and/or at a different time of day, but which cannot be recognized and detected due to the current weather conditions and/or time of day, are detected by the infrastructure detection device and are then declared as environmental information recorded outside the vehicle. Such constellations can then also be mapped. An object that is currently in the detection area of the vehicle's detection device, but cannot be detected by the vehicle's detection device due to the weather conditions and/or time of day, can still be detected by the infrastructure detection device. Objects detected in this way can then also be displayed symbolically in the real image. Symbolic elements can also be displayed here, as already explained above. In addition to this, or instead of this, it is also possible for these symbolic elements to be displayed partially transparently in the real image. This means that they can be recognized by a remote control operator, particularly with regard to their current location, especially relative to the vehicle being remotely controlled, and yet other objects behind them, viewed in three dimensions, can still be recognized in the display field of the display unit. The partially transparent representation of these objects is particularly helpful for such constellations.

In one embodiment, the environmental information recorded outside the vehicle is shown on a second display unit of the remote control center that is different from the first display unit. In such an embodiment, the different environmental information recorded is separated into two different display units. This means that in individual situations, information overload for the remote controller on a single display unit can be avoided. In addition, for specific situations, a remote controller can individually better recognize and/or better understand the environmental information recorded in each case.

In one embodiment, the environmental information recorded outside the vehicle is analyzed to determine whether and which object was detected. Depending on this, an object type and/or an object speed and/or an object movement direction of the object is determined. This allows a more precise analysis to be carried out of how a specific scenario appears in the surrounding area. In this context, detailed information can then also be transmitted to the remote controller. This can also be done visually and/or acoustically. This means that the remote-controlled driving maneuver can be further improved and also carried out more safely.

In one embodiment, a warning signal can be issued depending on the specific type of object and/or depending on the specific object speed and/or depending on the specific object movement direction. This can also provide a remote controller with additional auxiliary information in order to be able to better avoid safety-critical conditions during remote-controlled driving maneuvers. In particular, it can be provided that at least two different warning signals are present. An analysis can be carried out to classify a traffic situation in the surrounding area that is detected by the infrastructure detection device. In this context, a classification into one of at least two different classes can be carried out. In this context, a warning signal can be assigned to each of the classes. Depending on which class the traffic situation in a surrounding area is classified into, the associated warning signal is then issued. This enables an even more finely structured and detailed analysis to be carried out and appropriate information to be provided to a remote controller.

In one embodiment, a maximum speed for the continued movement of the vehicle during the remote-controlled driving maneuver is determined based on the environmental information recorded outside the vehicle and/or on weather information or information on the weather conditions in the area surrounding the vehicle. This can be carried out in particular by an electronic determination unit. This can be part of the remote control center, for example. This also takes into account the high safety requirements when carrying out a remote-controlled driving maneuver. A maximum speed determined by the system for the remote-controlled vehicle also gives the remote controller more safety, and any risks for the remote-controlled driving maneuver that the remote controller may not have recognized can be minimized.

Such a method now makes it possible to carry out safer manoeuvring with a remote control that is arranged decentrally to the vehicle and cannot directly see the vehicle and its immediate surroundings when remotely controlling it. This is because additional information from the infrastructure detection device on the at least one display unit in the remote control center for the remote controller, such a decentrally performed and remotely controlled driving maneuver by the remote controller is also possible with greater precision. Areas in the environment that cannot be recognized by the remote controller on its at least one display unit or cannot be recognized with sufficient accuracy and/or areas of the vehicle that cannot be recognized accurately or at all by the remote controller on the display unit are therefore of less importance for the remote controller when carrying out the driving maneuver. In this context, he no longer has to primarily consider how to carry out the driving maneuver based on the information shown on the display unit, but is offered essential electronic auxiliary information based on the additional information on the vehicle's surroundings provided by the system. In addition to the information shown on the display unit, in particular the pictorial representation of the vehicle's surroundings and/or at least parts of the vehicle, he can now essentially use this information from the specific additional information generated outside the vehicle to be able to carry out the remote-controlled driving maneuver. This means that the vehicle can be moved safely, especially in remote control processes where the remote controller is located in a remote control center that is very far away from the vehicle. This is particularly advantageous in situations where the remote controller cannot directly observe the vehicle and/or its surroundings from his or her position in the remote control center due to being in the remote control center, for example in a building.

The remote controller therefore has his position in the remote control center, for example a building, and from this position he cannot directly see the vehicle he is remotely controlling and/or its surroundings when remotely controlling it, and in particular cannot see it completely.

In one embodiment, the orientation of the vehicle in the environment and/or a distance of the vehicle to at least one object in the environment is determined to determine the position of the vehicle. This information is advantageous in order to be able to determine the possible potential direction of movement of the vehicle. The orientation in particular is essential in order to recognize whether the vehicle will or must travel in a forward direction or in a reverse direction, for example. Other information that characterizes dynamic and/or static objects in the immediate vicinity of the vehicle in the environment is therefore also advantageous.

In one embodiment, the surroundings of the vehicle are recorded with a front camera of the vehicle and/or with a rear camera of the vehicle. In addition or instead of this, the surroundings can also be recorded with at least one side camera. In particular, the images from the front camera and/or the images from the rear camera are shown on the display unit. This makes it possible to achieve a representation on the display unit, particularly in the key directions, in particular to the front and/or to the rear, which shows the remote controller a particularly realistic scenario. In this context, a representation is conveyed in which the remote controller is given the perspective as if he were sitting in the vehicle to be remotely controlled, in particular as the driver. This enables the remote-controlled driving maneuver to be carried out in a particularly intuitive and realistic manner.

In one embodiment, the driving maneuver is carried out in a parking area as the environment of the vehicle, so that parking or exiting a parking zone of the parking area is carried out as a driving maneuver. In this context, a parking zone is an area in the parking area that is intended for parking a single vehicle. This parking zone can be identified, for example, by markings, in particular floor markings. In this context, it is possible for a single vehicle to be a passenger car or a truck or a bus. A motor vehicle with a trailer can also be considered to be such a single vehicle. Several such parking zones are preferably formed in the parking area. A parking zone is therefore not a complete parking garage or an outdoor parking unit intended for parking motor vehicles, but is only intended for the locally defined parking of a single such motor vehicle. A parking area can, for example, be a parking garage with several such parking zones. A parking area can also be another area, even one that is not enclosed, on which parking in such marked parking zones is permitted. For example, these can be perpendicular parking spaces or parallel parking spaces adjacent to a roadway. The invention is particularly advantageous for driving maneuvers that involve parking and/or reversing using a remote control in a remote control center. This is because, especially when parking or reversing, there can be cramped situations or areas that are difficult to see, which can lead to critical driving maneuvers and/or collisions. Therefore, it is particularly advantageous to display such additional information on the remote control display unit in such situations during the remote-controlled driving maneuvers described above. Even difficult parking maneuvers, for example due to tight spaces, can therefore be carried out particularly advantageously.

In one embodiment, the information on the position of the vehicle is transmitted to the evaluation unit located in the remote control center. The advantages already mentioned above for the determination unit also apply in this regard.

It is also possible that the evaluation unit and the determination unit are combined in a common unit. However, they can also be separate units in a corresponding system.

In one embodiment, at least one camera of the vehicle to be remotely controlled captures its surroundings. In particular, images from the camera are provided to the evaluation unit as information for determining the position of the vehicle. The advantages that can be achieved in this way have already been explained above.

A further aspect of the invention relates to an electronic remote control system for remotely controlling a vehicle. The electronic remote control system has the vehicle to be remotely controlled. It has at least one evaluation unit and at least one display unit. The display unit is arranged in a remote control center that is arranged at a fixed location away from the vehicle. The remote control system is designed to carry out a method according to the above-mentioned aspect or an advantageous embodiment thereof. In particular, the method is carried out using the electronic remote control system.

• 1 eine schematische Darstellung eines Ausführungsbeispiels eines elektronischen Fernsteuerungssystems;
• 2 eine Draufsicht auf ein Verkehrsszenario, in welchem ein ferngesteuertes Fahrzeug und ein von einer Erfassungsvorrichtung des Fahrzeugs aktuelle nicht erfassbarer Umfeldbereich gezeigt sind; und
• 3 ein Ausführungsbeispiel einer Anzeige auf einer Anzeigeeinheit in der Fernsteuerzentrale.

Embodiments of the invention are explained in more detail below using schematic drawings. They show:

• 1 a schematic representation of an embodiment of an electronic remote control system;
• 2 a plan view of a traffic scenario in which a remote-controlled vehicle and an area of the surroundings that cannot currently be detected by a detection device of the vehicle are shown; and
• 3 an embodiment of a display on a display unit in the remote control center.

In the figures, identical or functionally equivalent elements are provided with the same reference symbols.

In 1 a remote control center 1 is shown in a schematic representation. The remote control center 1 is, for example, in a stationary building. The remote control center 1 has at least one remote control station 2. A human remote controller 3 is arranged at the remote control station 2. The person in the form of the remote controller 3 sits or stands there in a fixed place. In the exemplary embodiment, remote control elements 4 are arranged at this remote control station 2. Such remote control elements 4 can be, for example, a keyboard and/or at least one operating unit, such as a joystick or the like. At least one display unit 5 is arranged at the remote control station 2. Several display units 5, 6 and 7 can also be arranged.

In the exemplary embodiment, a determination unit 8 is also arranged in the remote control center 1. In one exemplary embodiment, an evaluation unit 9 can also be arranged in the remote control center 1. This determination unit 8 and the evaluation unit 9 are in particular electronic units. They can also be implemented as software modules.

As further stated in 1 can also be seen, an environment 10 is shown externally and in particular further away from the remote control center 1. The environment 10 is arranged in particular in such a way that it cannot be directly seen by the remote controller 3. The remote controller 3 can therefore not directly recognize vehicles, in particular a vehicle to be remotely controlled, from his remote control station 2 and thus cannot directly inspect them.

In 1 It is also shown that the environment 10 represents an individual traffic scenario. For example, this can be in an intersection area. However, a variety of other traffic scenarios are also possible.

In 1 In this context, a vehicle 11 is shown. The vehicle 11 is the one which is controlled by a corresponding remote control system. In particular, remote control elements 4 are operated by the remote controller 3. The remote control system then wirelessly transmits signals via a communication interface 12, in particular to the vehicle 11 to be remotely controlled via The signals received in this regard are then processed in the vehicle 11 and the functional units to be operated are then activated.

The vehicle 11 has at least one detection device 13 for detecting the surroundings 10. This detection device 13 has the detection area 14. For example, the detection device 13 can be a front camera of the vehicle 11. However, it is also possible for the detection device 13 to have a rear camera of the vehicle 11 in addition to or instead of this.

In the environment 10 there is also an environmental area or a surrounding area 15. This environmental area 15 cannot be detected from the current position of the vehicle 11 with the detection device 13. This can be the case, for example, because the environmental area 15 is hidden from the detection device 13 by a building 16 or the like.

For example, at least one object 17 can be arranged in this surrounding area 15. This object 17 can also be a dynamically movable object. It can be, for example, a vehicle or a cyclist or a scooter or the like.

In the exemplary embodiment shown, the overall system also has at least one infrastructure detection device 18. This can be arranged on an infrastructure unit 19. The infrastructure unit 19 is arranged in a fixed location. It can be a building, for example. However, it can also be a stationary traffic control element, for example. In this context, it can also be a traffic light, for example. It is also possible for the infrastructure device 18 to be arranged on a lamppost, for example. Other positions for such an infrastructure detection device 18 can also be provided. The infrastructure detection device 18 can be a camera, for example. It has a detection area 19. With this detection area 19, the surrounding area can be detected at least in areas 15. Therefore, in the exemplary embodiment, the object 17 in the surrounding area 15 can be detected by the infrastructure detection device 18. In the scenario shown as an example, it is therefore provided that the detection device 13 of the vehicle 11 is used to generate self-detected environmental information on the vehicle side. With the different infrastructure detection device 18 arranged outside the vehicle, environmental information detected outside the vehicle is generated and provided.

In one embodiment, it is provided that the environmental information recorded on the vehicle side and the environmental information recorded outside the vehicle are transmitted to the remote control center 1. For this purpose, this infrastructure detection device 18 is also designed for wireless communication with the remote control center 1. This can be done via a direct communication interface. However, it is also possible for this environmental information recorded outside the vehicle to be transmitted to the remote control center 1 via a further communication unit 21, which is not arranged in the remote control center 1.

It is also possible that this transmission or the transmission of the environmental information recorded outside the vehicle to the remote control center 1 is only carried out when the vehicle 11 approaches the surrounding area 15. In an advantageous embodiment, the position of the vehicle 11 is determined and/or the direction of travel of the vehicle 11 is determined. Depending on this position and/or the direction of travel, it is checked whether at least one infrastructure detection device 18 is located in a surrounding zone 22 of the vehicle 11. In this case, it is checked in particular depending on the position of the vehicle 11 and/or the direction of travel of the vehicle 11 whether an infrastructure device 18 is located in a surrounding zone 22. If this is the case, in one embodiment the transmission of the environmental information recorded outside the vehicle is carried out from this infrastructure detection device 18, which is then arranged in a surrounding area 15, to the remote control center 1. In particular, in this context, a request for this transmission is made depending on the criteria mentioned above. The surrounding area 15 can be at least partially, in particular completely, part of the surrounding zone 22.

In particular, the infrastructure detection device 18 is used to detect at least some different environmental areas with the detection device 13, each depending on the current position of the vehicle 11. The infrastructure detection device 18 detects at least one environmental area which, in one embodiment, is not visible from the vehicle 11 from its current position. In addition to this, or instead of this, the infrastructure detection device 18 can detect at least one environmental area which is currently also detected by the detection device 13, but in which objects located therein cannot be detected or cannot be detected sufficiently by the detection device 13 due to, for example, weather conditions and/or the time of day. sufficiently accurate, but this is made possible with the infrastructure detection device 18.

In 2 is a schematic plan view of the environment 10 according to the embodiment in 1 shown.

In one embodiment, it is provided that the environmental information recorded on the vehicle side and the environmental information recorded outside the vehicle are displayed on at least one display unit 5, 6, 7 in the remote control center 1 for the remote controller. In one embodiment, they can therefore be displayed on a common display unit, for example the display unit 5, in particular displayed simultaneously.

In one embodiment, it can be provided that the environmental information detected on the vehicle side is displayed on a first display unit, for example the display unit 5, as a real image, in particular a perspective from the point of view of a driver in the vehicle 11 to be remotely controlled is displayed as a real image.

In one embodiment, it is possible for the environmental information recorded outside the vehicle to be displayed symbolically in this real image. In this context, it can preferably be displayed symbolically on this real image with the exact location as it is in reality.

It can be provided that from the current perspective of the vehicle 11, locally concealed environmental information recorded outside the vehicle is displayed in the real image by means of location-specific auxiliary symbols. For this purpose, for example, in 3 a schematic representation of the environment 10 is shown on, for example, a display unit 5. Here, the perspective from the viewpoint of a driver in the remote-controlled vehicle 11 is shown as a real image on this display unit 5.

In this context, for example, lines, here in particular a frame 24 ( 3 ), the object 17 can be represented symbolically. In the exemplary embodiment, this frame 24 is shown locally at the location in the real image where the object 17 is also located in reality in relation to the vehicle 11 and/or other elements in the environment 10. As can be seen here, this object 17 is behind the building 16 viewed from the current perspective of the vehicle 11.

As already mentioned in 1 and 2 As shown, the vehicle 11 travels on the roadway 23 along the path with the direction of travel P.

It is also possible for the environmental information recorded outside the vehicle to be analyzed. In particular, this can be done to determine which object 17 was recorded and thus which type of object is present. In addition to or instead of this, an object movement direction P can also be determined. In addition to or instead of this, an object speed of the object 17 can also be determined. With this more precise analysis, it can then also preferably be determined whether the vehicle 17 would collide with the vehicle 11, for example at a potential collision point K. In particular, if such a potential collision could occur, in one embodiment a warning signal is issued depending on the specific object type and/or depending on an object speed and/or an object movement direction of the object 17. This is output in particular to the remote controller 3. It can be output acoustically and/or visually.

In one embodiment, the overall system can also have at least one sensor 25 that is arranged in the environment 10. The sensor 10 can be, for example, a temperature sensor and/or a humidity sensor. It is then possible that, depending on the environmental information recorded outside the vehicle and/or depending on weather information that is recorded in particular on the basis of the at least one sensor 25 and that relates to the weather in the environment 10 of the vehicle 11, for example, a maximum speed for the further remote-controlled movement of the vehicle 11 is determined. This can be done by a determination unit. In particular, this can be done by the determination unit 8. For this purpose, the at least one sensor 25 can communicate directly with the remote control center 1 via the communication interface 12 and/or via the communication unit 21.

Overall, 1 also an electronic remote control system 26 is shown, which is designed to remotely control the vehicle 11.

As further stated in 3 can also be recognized, for example, further objects 27 and/or 28 can be present in the environment 10. These objects 27 and 28 can be arranged, for example, at the current position of the vehicle 11 in the detection range 14 of the detection device 13. However, due to weather conditions and/or the time of day, the detection of these objects 27 and 28, which here are dynamic objects, in particular vehicles, for example, may not be carried out or may not be carried out with sufficient accuracy. This means that, for example, in the images that are taken by the detection device 13, these vehicles or objects 27 and 28 are not recognized as such. In particular, if an infrastructure detection device 18 is arranged in such a way that these objects 27 and 28 can be detected with the infrastructure detection device 18 at this time and are then also recognized, this can be displayed on the display unit 5. This is exemplified in 3 shown. In such a situation, these objects 27 and 28, which then also represent environmental information recorded outside the vehicle, can be identified. They can also be represented symbolically, for example. In this context, they can be shown in the real image on the display unit 5 with precise location in relation to reality. It is also possible that their orientation and/or their direction of movement are also shown, as is the case in 3 shown by way of example. To clarify that this information is not information that was detected by the detection device 13 itself, it can again be visually indicated by, for example, a frame 29 and 30 that this is environmental information detected outside the vehicle.

Generally speaking, an infrastructure detection device can have a camera that is sensitive in the spectral range visible to humans. It is also possible for an infrastructure detection device to have a lidar sensor. A radar sensor or an ultrasonic sensor is also possible. Other sensors, such as a rain sensor or a wind sensor or a temperature sensor, can also be part of the infrastructure detection device. It is also possible for several infrastructure detection devices to be present to detect at least the surrounding area 15. In particular, these are then present in the surrounding zone 22.

It is also possible that an individual warning signal or an individual warning notice is generated with regard to the type of object analyzed. For example, pedestrians or cyclists, where people are not protected by the body of a vehicle, can be categorized differently than, for example, people in another vehicle. Since such unprotected people in the surrounding area 15 could potentially be more seriously harmed in the event of a potential collision, an individual warning signal is generated here. For example, a corresponding window can also be displayed on the display unit 5 as additional visual warning information in this context. It is also possible that a specific warning light is displayed for the remote control operator. It is also possible that different warning information is generated as to whether a potential collision between an object that is currently not visible from the detection device 13 and the remote-controlled vehicle 11 could occur or not. Two different warning signals or warning notices can also be generated and individually output for these two different scenarios. In an embodiment in which an infrastructure detection device has a camera that is sensitive in the visible spectral range, the image of this camera of the infrastructure detection device can be displayed on a display unit, for example the display unit 6 and/or 7, for the remote controller 3. In such an embodiment, this real image is then displayed on a separate display unit on which the real image of the detection device 13 is not displayed. In another embodiment, it is also possible for the symbolically displayed environmental information recorded outside the vehicle to be shown partially transparently as an overlay on the real image that is recorded by the detection device 13 and is displayed on at least one of the display units 5, 6, 7.

The above-mentioned maximum speed, which is determined by the system for the remote-controlled vehicle 11, can be determined in particular depending on weather conditions and/or depending on the traffic density in the surrounding area 10. The traffic density 10 particularly affects road users who are currently in the surrounding area 10.

With regard to the information transmission, for example, the information from the at least one infrastructure detection device 18 and/or the at least one sensor 25 can be transmitted directly to the remote control center 1. Transmission can also take place via the communication unit 21. It is also possible for the corresponding information from the infrastructure detection device 18 and/or the at least one sensor 25 to be transmitted first to the vehicle 11 to be remotely controlled. From there, in one embodiment, it can then be transmitted to the remote control center 1. The information transmission or the data transmission can be transmitted via a data network. This can be a mobile network. Compressed data transmission is also possible, so that the bandwidth is reduced. It is also possible for time stamps to be used for the units or sensors with regard to data synchronization. In particular, the infrastructure detection device 18 and/or the at least one sensor 25 and the vehicle to be remotely controlled 11 synchronized. In particular, this can be through a GPS (Global Positioning System) time.

Claims (15)

Method for remotely carrying out a driving maneuver of a vehicle (1), comprising the following steps: - detecting the environment (10) with at least one detection device (13) of the vehicle (11) to be remotely controlled and providing this environmental information detected by the vehicle itself; - detecting the environment (10) in an environmental area (16) of the vehicle (11) with at least one infrastructure detection device (18) arranged externally to the vehicle (11), which is arranged on a stationary infrastructure unit (16), and providing this environmental information detected externally to the vehicle; - transmitting the environmental information detected by the vehicle itself and the environmental information detected externally to the vehicle to a remote control center (1); - remotely carrying out the driving maneuver of the vehicle (1) by a remote controller (3) in the remote control center (1) depending on the environmental information detected by the vehicle and the environmental information detected externally. Procedure according to Claim 1 , wherein the infrastructure detection device (18) detects at least partially different environmental areas (16) than those detected by the detection device (13). Procedure according to Claim 2 , wherein the infrastructure detection device (18) detects at least one environmental area (16) which is not visible from the vehicle (11). Procedure according to Claim 2 or 3 , wherein a traffic area is detected as an environmental area (16) with the infrastructure detection device (18). Method according to one of the preceding claims, wherein the transmission of the environmental information acquired externally of the vehicle to the remote control center (1) is only carried out when the vehicle (11) approaches the surrounding area (16). Method according to one of the preceding claims, wherein the position and/or direction of travel of the vehicle (11) is determined and, depending on this, it is checked whether and/or where an infrastructure detection device (18) is arranged in an environment zone (22) in the environment (10) with respect to the position of the vehicle (11) and/or in the direction of travel of the vehicle (11), and if this is the case, a transmission of the environmental information acquired outside the vehicle from this infrastructure detection device (18), which is arranged in the environment zone (22), to a remote control center (1) is requested. Method according to one of the preceding claims, wherein the environmental information acquired on the vehicle side and the environmental information acquired externally of the vehicle are displayed on at least one display unit (5, 6, 7) in the remote control center (1) for the remote controller (3). Procedure according to Claim 7 , wherein the environmental information detected on the vehicle side is displayed on a first display unit (5, 6, 7) as a real image, in particular a perspective from the point of view of a driver in the vehicle to be remotely controlled (11) is displayed as a real image. Procedure according to Claim 8 , whereby environmental information recorded outside the vehicle is displayed symbolically in the real image. Procedure according to Claim 9 , wherein from the current perspective of the remote-controlled vehicle (11), locally concealed environmental information recorded externally of the vehicle and/or environmental information recorded externally of the vehicle that is not recognizable due to weather conditions and/or time of day are displayed in the real image by means of auxiliary symbols, in particular those displayed precisely in relation to reality, in particular as frames and/or transparently displayed objects. Procedure according to Claim 7 , wherein the environmental information acquired outside the vehicle is displayed on a second display unit (5, 6, 7). Method according to one of the preceding claims, wherein the environmental information detected outside the vehicle is analyzed to determine whether and which object (17) was detected in the surrounding area (15), and depending thereon an object type and/or an object speed and/or an object movement direction of the object (17) is determined. Procedure according to Claim 12 , wherein one of at least two different warning signals is issued depending on the specific object type and/or an object speed and/or an object movement direction. Method according to one of the preceding claims, wherein depending on the environmental information recorded outside the vehicle and/or depending on weather information concerning the weather in the surroundings of the vehicle (11), a maximum speed for further movement of the vehicle (11) during remote control is determined. Electronic remote control system (26) for remotely controlling a vehicle (11), with the vehicle (11), an evaluation unit (9) and a display unit (5, 6, 7) which is arranged in a remote control center (1) arranged at a distance from the vehicle (11), wherein the remote control system (26) is designed to carry out a method according to one of the preceding claims.

Images