DE102019219663B4 - Device and method for processing video signals and controlling cameras in an autonomous or semi-autonomous vehicle - Google Patents

Abstract

The present invention relates to a device (12, 121, 122) for processing video signals and controlling cameras in an autonomous or semi-autonomous vehicle (70), having: a video input interface (22) for receiving video signals from a camera (18, 181, 182) ) the vehicle (70); a bidirectional control interface (20, 201, 202, 203) for receiving and outputting camera control signals for the camera (18, 181, 182); a processing unit (32) for processing the video signals and the camera control signals and for generating control signals for a control unit (14, 141, 142, 143) of the vehicle (70); and a bidirectional device interface (36) for transmitting the control signals to the control unit (14, 141, 142, 143) and for receiving control signals from the control unit (14, 141, 142, 143); wherein the control signals include the video signals and the camera control signals. The present invention further relates to a system (10) and a method for processing video signals and controlling cameras in a vehicle.

Description

The present invention relates to a device, a system and a method for processing video signals and controlling cameras in an autonomous or partially autonomous vehicle.

Modern vehicles (cars, vans, trucks, motorcycles, etc.) include a large number of systems that provide the driver with information and control individual functions of the vehicle in a partially or fully automated manner. The surroundings of the vehicle and other road users are recorded via optical sensors. Based on the recorded data, a model of the vehicle environment can be generated and changes in this vehicle environment can be reacted to. Due to the progressive development in the field of autonomous and semi-autonomous vehicles, the influence and scope of such driver assistance systems (Advanced Driver Assistance Systems, ADAS) are increasing. The development of ever more precise sensors makes it possible to record the environment and the traffic and to control individual functions of the vehicle completely or partially without the intervention of the driver. Driver assistance systems, including autonomous driving systems, can contribute to increasing safety in traffic and improving driving comfort.

In addition to static objects in the vicinity of one's own vehicle, moving objects and other road users in the vehicle environment can also be recorded and, based on this, the behavior of an autonomous or semi-autonomous vehicle can be adapted to a current situation. For this purpose, environmental sensors of different sensor types are installed in order to draw a more precise picture of the vehicle environment and to be able to react more quickly to influences in the area of action of the driver assistance systems. In particular, cameras are used to get an accurate picture of the environment. Their number is steadily increasing, as is the quality of the information received. At the same time, the resolution of the cameras is increasing, which leads to a growing amount of data that has to be processed in the vehicle.

Processing devices, video units, control units and control devices which have interfaces to the cameras or camera sensors are used for data processing. The control units and devices have processors and computing units in order to process video signals and control and control signals from the cameras.

In systems for autonomous driving in vehicles, the connection between a camera and a control unit or a control unit is implemented via a video unit. This video unit is also known as a serializer / deserializer system. Unidirectional camera signals are transmitted from the camera to the video unit. The camera signals include video signals and bidirectional control data or camera control data. The camera signals of the camera received by the video unit are split into video signals and camera control signals. The video signals are transmitted from the video unit to the control unit or the control unit via a CSI interface (Camera Serial Interface) or an extended CSI-2 interface. The control control data or camera control signals are transmitted as I2C signals via corresponding I2C interfaces (Inter-Integrated Circuit interfaces). The prerequisite for this is that the control unit, which is often designed as a system-on-chip, has the corresponding specific interfaces.

Several cameras are connected to typical video units (deserializers). Typically a video unit is connected to two or four cameras. For this purpose, the video units provide two CSI-2 video outputs and two I2C interfaces for camera control signals. These interfaces are sufficient to control up to four cameras at the same time and to further process the received data and to transmit them to one or more control units via the interfaces.

In the prior art, it is common to connect a video unit via a CSI-2 video interface, for example via a MIPI interface (Mobile Industry Processor Interface), and in parallel to this via an I2C interface for the camera control signals directly to the control unit, for example in the form of a system-on-chip. Since the video unit usually comprises two equal I2C interfaces as control ports, the video unit can be connected to two control devices or control units and signals can be transmitted to and controlled by them at the same time. Any prioritization of processing is carried out using application software that is included in the control unit.

If two video units are to be connected to one control unit, the control unit must have a corresponding number of video inputs and control interfaces. Alternatively, it is known to place a processor unit between the control unit and the video units in order to combine the video signals and to connect them to the control unit via a fast interface. PCIe connections, which the control units or control devices usually have, are often used here. In this case, there is no need to have a video input interface on the control unit being. However, the camera control signals are still transmitted to the control units via direct connections.

Due to the large number of cameras installed in an autonomously or partially autonomously driving vehicle, video processing systems in the vehicles have to process more and more camera data, i.e. video signals and camera control signals. Numerous control units, especially if they are implemented as system-on-chip, have no or insufficient number of camera control interfaces to receive the camera control signals. If there are only a few control interfaces in the individual control units, the video signals must be distributed to a large number of control units or control units. This increases the cost. At the same time, the known systems cannot cover a fault in a control unit without foregoing the video information it processes.

DE 10 2018 112 085 A1 describes devices, systems and methods in (partially) autonomous vehicles. Depending on the driving mode, the data streams from various sensors (including cameras) are connected to various sub-control modules via switches.

US 2018/0 191 928 A1 teaches a device for the synchronized connection of several cameras with a control unit in a vehicle, wherein the device can convert the camera data.

The object of the present invention is therefore to create a device and a system which make it possible to process a large number of video signals from a plurality of cameras and to have access to all camera data in the event of a control unit failure.

• einer Videoeingangsschnittstelle zum Empfangen von Videosignalen einer Kamera des Fahrzeugs;
• einer bidirektionalen Steuerschnittstelle zum Empfangen und zum Ausgeben von Kamerasteuersignalen für die Kamera;
• einer Verarbeitungseinheit zum Verarbeiten der Videosignale und der Kamerasteuersignale und zum Erzeugen von Steuersignalen für eine Steuereinheit des Fahrzeugs; und
• einer bidirektionale Geräteschnittstelle zum Übertragen der Steuersignale zu der Steuereinheit und zum Empfangen von Steuersignalen von der Steuereinheit; wobei die Steuersignale die Videosignale und die Kamerasteuersignale umfassen.

To achieve this object, the invention relates in a first aspect to a device for processing video signals and controlling cameras in an autonomous or semi-autonomous vehicle, with:

• a video input interface for receiving video signals from a camera of the vehicle;
• a bidirectional control interface for receiving and outputting camera control signals for the camera;
• a processing unit for processing the video signals and the camera control signals and for generating control signals for a control unit of the vehicle; and
• a bidirectional device interface for transmitting the control signals to the control unit and for receiving control signals from the control unit; wherein the control signals include the video signals and the camera control signals.

In a further aspect, the present invention relates to a system for processing video signals and for controlling cameras in an autonomous or partially autonomous vehicle with a device as described above, with a video unit with a video output port and a control port and with a control unit.

In a further aspect, the present invention relates to a vehicle with a system as described above.

• Empfangen von Videosignalen einer Kamera des Fahrzeugs mittels einer Video-Eingangsschnittstelle;
• Empfangen von Kamerasteuersignalen;
• Verarbeiten der Videosignale und der Kamerasteuersignale mittels einer Verarbeitungseinheit;
• Erzeugen von Steuersignalen, die die Videosignale und die Kamerasteuersignale umfassen, für eine Steuereinheit des Fahrzeugs;
• Übertragung der Steuersignale zu der Steuereinheit.

The present invention also relates to a method for processing video signals and controlling cameras in an autonomous or semi-autonomous vehicle, with the following steps:

• Receiving video signals from a camera of the vehicle by means of a video input interface;
• Receiving camera control signals;
• Processing the video signals and the camera control signals by means of a processing unit;
• Generating control signals comprising the video signals and the camera control signals for a control unit of the vehicle;
• Transmission of the control signals to the control unit.

Another aspect relates to a computer program product with program code for performing the steps of the method when the program code is executed on a computer, as well as a storage medium on which a computer program is stored which, when executed on a computer, executes the one described herein Procedure causes.

Preferred embodiments of the invention are described in the dependent claims. It goes without saying that the features mentioned above and those yet to be explained below can be used not only in the respectively specified combination, but also in other combinations or on their own, without departing from the scope of the present invention. In particular, the method and the computer program product for the control device and the system for processing video signals and for controlling cameras and the vehicle can be implemented in accordance with the configurations described in the dependent claims.

The device according to the invention for processing video signals and for controlling cameras in an autonomous vehicle has the advantage that the connection between the device and the control unit or a control device takes place via the bidirectional device interface. The device interface can be a known standard interface. It preferably has high transmission rates with a bandwidth of at least 10 Gbit / s, preferably of at least 30 Gbit / s. Control signals are transmitted to the control unit. The control signals include the video signals and the camera control signals which the device receives from a video unit via the video input interface and the control interface. The control unit itself does not require a video input interface or a control interface that is specifically designed for camera control signals. Since many control units and control devices do not have such interfaces, an adaptation can take place by means of the device according to the invention, so that video signals and camera control signals can still be received and further processed by cameras.

Since the control signals include the camera control signals and the video signals, the video signals are transmitted from the device to the control unit in a practically tunneled manner. The video signals are integrated into the control signals and can be separated again from the camera control signals in the control unit. It is therefore possible to extract the video signals from the transmitted control signals.

In a preferred embodiment, the device interface of the device is a PCIe interface. Accordingly, the interface on the control unit is also a PCIe interface. Such interfaces are typical standard interfaces and are often provided in control units in the vehicle in the prior art. They enable very fast transmission of data and signals with a data rate of 30 gigabits per second (Gbit / s) and more. Since the PCIe interface is available as standard in most control units and control units, a large number of control units and control units are available for processing the video signals.

In a preferred embodiment of the device according to the invention, the processing unit is designed to extract camera control signals from the received control signals. Since the device interface of the device is a bidirectional interface, not only control signals that were generated in the device are sent to the control unit via this interface. The device also receives control signals from the control unit by means of the bidirectional device interface. These received control signals are preferably processed in the processing unit and the camera control signals integrated therein are extracted. This can be done by filtering or similar suitable measures. The extracted camera control signals are preferably output via the control interface, which is also bidirectional, for example to a video unit. The camera control signals can be transmitted directly or indirectly (via video unit) from the device to the corresponding cameras, for example in order to be able to make settings on the camera or to control the camera.

In a preferred embodiment, the device has a plurality of video input interfaces. The device thus receives multiple video signals from different cameras. The video signals that are received via the video input interfaces are then preferably processed jointly in the processing unit of the device. For example, several video signals can be combined. These (combined) video signals are then available for further processing. Together with the corresponding camera control signals, they are processed into control signals for the control unit. Thus, multiple video signals are tunneled in the control signal.

In a preferred embodiment of the device according to the invention, the device has several control interfaces. The processing unit is preferably designed to combine the camera control signals that are received via the plurality of control interfaces. They are processed together with the video signals to form the control signals and transmitted to the control unit.

The device preferably has a multiplexer by means of which the video signals or the camera control signals are combined if several video input interfaces or several control interfaces are present. The use of a multiplexer is an effective means of combining and mixing several signals with one another. A plurality of multiplexers can advantageously be present; a multiplexer for the video signals and a multiplexer for the camera control signals. In a preferred embodiment, the multiplexer or multiplexers are integrated in the processing unit.

In a preferred embodiment, the device according to the invention has a communication interface. This makes it possible to communicate with other devices of the same type. The communication interface is used to Exchange video signals and / or camera control signals. The communication interface is bidirectional.

When the device comprises a video multiplexer and a camera control signal multiplexer, there may be a plurality of communication interfaces. A communication interface is preferably connected directly to the video multiplexer. Another communication interface is also preferably connected to the camera control signal multiplexer. In this way it is possible for the multiplexer to exchange further video signals with other devices directly. The video signals are exchanged between the respective multiplexers of the devices. This can be useful and advantageous, for example, in the event of a fault in a control unit. The same applies to the camera control signals, which can be exchanged directly from the camera control signal multiplexers of various devices via a corresponding communication interface.

The device is preferably an FPGA module, that is to say a field programmable gate array. This integrated circuit is used as standard in digital technology and in particular in vehicles with autonomous driving systems and is available in a large number of configurations. As a result of the programmability, the device can thus be adapted simply and inexpensively to the corresponding requirements and the system architecture.

According to the invention, the system for processing video signals and for controlling cameras in a vehicle comprises a device as described above, a control unit and a video unit which has a video output port and a control port. The video unit is connected to one or more cameras, preferably two to four cameras, and receives the sensor signals from the camera. The video unit of the system has the function of a serializer / deserializer and converts the serial signals and sensor data of the camera into (parallel) video signals and camera control signals. The video signals are transmitted to the device by means of the video output port of the video unit and received there by means of the video input interface. The camera control signals are transmitted from the video unit to the device via the control port and are received there at the control interface. The connections between the control port and the control interface are typically transmitted via 12C (Inter-Integrated Circuit) connections. The control interface and the control port are correspondingly 12C interfaces. The transmission takes place via the corresponding protocol.

The video input interface and the video output port of the video unit are CSI (Camera-Serial-Interface) interfaces. The video signals are transmitted using an appropriate protocol.

In a preferred embodiment, the device has multiple video input interfaces and multiple control interfaces so that multiple video units can be connected to the device. In this way, it is possible to process the information from a large number of cameras and to combine their signals in control signals in order to transmit them to the control unit or the control unit, in which the camera signals are further processed and corresponding driving functions are initiated and control data for the actuators are generated in the vehicle.

In a preferred embodiment of the system, the control unit has an input interface which is connected to the device interface of the device. The input interface of the control unit and the device interface of the device are preferably designed as PCIe interfaces. The control signals are transmitted between the control unit and the device using the PCIe protocol. Since high data rates can be achieved with these interfaces, a large amount of camera information and camera data from several cameras can be transmitted to the control unit or the control device. In this way, it is possible to carry out fast and quasi real-time processing of the optical data even with a large number of data in order to enable autonomous driving in a vehicle.

In a preferred embodiment, the system has a safety processor which is designed to process the data from the camera and assign the data to one or more devices in the system and / or to one or more control units in the system. The safety processor is designed to control the processing of the camera control signals and the video data. These data are assigned to various devices or control units.

For example, in a system in which there are several devices and several control units, if a control unit fails, such a failure can be detected by the safety processor. The safety processor can then instruct the device connected to the failed control unit to transmit the video signals and camera control signals directly to the second device connected to the intact control unit. In this case, the intact control unit takes over the processing of all data stored in the Both devices for processing the video signals and for controlling the cameras are available.

• 1 eine schematische Darstellung eines Systems gemäß einem Aspekt der vorliegenden Erfindung;
• 2 eine schematische Darstellung einer Anordnung mit zwei Systemen, die jeweils zu einer eigenen Stromversorgungsdomain gehören;
• 3 eine schematische Darstellung der prinzipiellen Anordnung aus 2 im Fehlerfall einer Steuereinheit;
• 4 eine schematische Darstellung einer Anordnung mit zwei Systemen gemäß einem weiteren Aspekt der vorliegenden Erfindung;
• 5 ein Fahrzeug mit dem erfindungsgemäßen System; und
• 6 ein Ablaufdiagramm des Verfahrens gemäß der Erfindung.

The invention is described and explained in more detail below with the aid of a few selected exemplary embodiments in connection with the accompanying drawings. Show it:

• 1 a schematic representation of a system according to an aspect of the present invention;
• 2 a schematic representation of an arrangement with two systems, each belonging to its own power supply domain;
• 3 a schematic representation of the basic arrangement 2 in the event of a fault in a control unit;
• 4th a schematic representation of an arrangement with two systems according to a further aspect of the present invention;
• 5 a vehicle with the system according to the invention; and
• 6th a flow chart of the method according to the invention.

1 shows a system according to the invention 10 with a device according to the invention 12th , a control unit 14th and two video units 16 . To the video units 16 are multiple cameras 18th connected. In the example shown here, there are video units on both 16 three cameras each 18th connected. Typically connected to a video unit 16 two to four cameras 18th connected. The cameras can be, for example, CCD cameras or other cameras, some of which are very high-resolution.

The device according to the invention 12th comprises in the embodiment shown here a bidirectional control interface 20th for receiving and outputting camera control signals for the cameras 18th and two video input interfaces 22nd for receiving video signals from the cameras 18th of a vehicle by the video units 16 be transmitted.

The video units 16 each have a control port 24 and a video output port 26th on. The control interface 20th the device 10 is via a control line 28 with the two control ports 24 of the video units 16 connected. The control line 28 is divided so that both control ports 24 the two video units can be controlled. The two video input interfaces 22nd the device 12th are each with a video output port 26th of the video units 16 with one video line each 30th connected. The video lines 30th as CSI or CSI-2 (camera serial interface) lines and the video input interface 22nd or the video output port 26th are designed as corresponding interfaces. The video signals are transmitted in accordance with the relevant standard. This also corresponds to the MIPI standard (Mobile Industry Processor Interface). The control line 26th corresponds to the I2C standard (Inter Integrated Circuit). The control port 24 and the control interface 20th are designed according to this standard.

The device 12th further comprises a processing unit 32 , in which the video signals and via the control interface 20th received camera control signals are processed. In the processing unit 32 control signals are generated which include both the camera control signals and the video signals, thus tunneling the video signals. In the embodiment shown here, the processing unit comprises 32 preferably a multiplexer 34 , which is the video signals transmitted by means of the video input interfaces 22nd are received, before they are further processed together with the camera control signals to form the control signal.

A device interface 36 the device 12th is designed as a PCIe interface. Via a PCIe data line 38 are the control signals from the processing unit 32 to the control unit 14th transmitted, with the data line 38 with an input interface 40 the control unit 14th connected is.

The control unit 14th , which is preferably designed as a system-on-chip, comprises an evaluation unit for dividing the control signals into video signals and camera control signals 42 . It is preferably a digital signal processor (DSP). The control data are divided into camera control data and video signals, which are transmitted via suitable drivers 44 and processors are further processed. For example, by means of the processor 46 an evaluation of the camera signals takes place, as well as a first preprocessing for an application. In the processor 46 Application software can be implemented that does the preprocessing. The processed video signals and camera control signals are then made available to other control units and processor units for autonomous driving.

As the device 12th via the specific control interfaces 20th and video input interfaces 22nd and the video signals as well as camera control signals are processed into control signals and via a standardized data line 38 outputs, the control unit needs 14th just a standardized input interface 40 . The bandwidth of this PCIe interface provides data rates of approx. 32 to 40 Gbit / s. So that everyone can use the device 12th incoming camera data be transmitted sufficiently quickly, so that a quasi real-time processing of the data in the control unit 14th , which preferably comprises a high-performance microprocessor, takes place.

The video units 16 are often referred to as a video board or as a serializer / deserializer because the cameras are serial 18th transmitted data in the video unit 16 are separated so that camera control signals and video signals via the corresponding ports in parallel 24 , 26th can be issued.

2 shows an arrangement 48 for video processing with two separate systems 10 , each with its own power supply domain 50 assigned. The separation of the two power supply domains 50 is by the dashed line 52 indicated. Data lines or control lines from a domain 50 to the other domain 50 lead, are via capacitors or lead-through capacitors 54 passed through.

2 it can be seen that the control lines 28 from the video units 16 to the corresponding devices 12th the same power domain 50 to lead. At the same time, lines lead into the other power supply domain, with the control lines 28 by acting as a capacitor 54 trained implementation capacities. In this way, the camera control signals of the video units 16 the left power domain 50 in 2 to the device 12th the right power supply domain 50 be transmitted. The camera control signals of the video units can also 16 the right power domain of the device 12th the left power supply domain. In this way it is possible for all control units 14th about the device 12th of your own system 10 and by means of the video units 16 the overall arrangement of all cameras 18th be able to control and control the entire system. To control the cameras, control signals are transmitted to the cameras, which, for example, cause the cameras to be switched on and off or contain certain configurations that can be set using configuration parameters. In this way it is possible to receive camera control signals from one system 10 to the other system 10 exchange, so that a control of the cameras 18th can also take place if, for example, the control unit 14th of the corresponding system 10 has failed.

2 it can also be seen that the two devices 12th of the two systems 10 via transceiver lines 56 are interconnected and able to communicate. The transceiver lines 56 are lines with a high data rate, preferably with data rates of 32 to 40 Gbit / s. The data rates correspond to those of the PCIe lines between the devices 12th and the control units 14th . By means of the direct connections of the devices 12th over the transceiver lines 56 can also use the video input interfaces 22nd received video signals can be exchanged. In this way it is possible if a control unit fails 14th of a system 10 access the video data of the camera of this system. Consequently, in the event of a fault in a control unit 14th both on the cameras 18th of the system 10 accessed and controlled, at the same time there is access to the video data of the cameras 18th . In this way, autonomous driving functions can be carried out even if a control unit fails 14th maintain. Usually the control units are 14th so performant that it can be used in spite of the failure of control units in the neighboring system 10 can process the additional video signals so that real-time processing can be maintained.

3 shows another illustration of the arrangement 48 the end 2 with one system 101 , on the left side of the line 52 and a second system 102 , to the right of the line 52 shown. The system 101 belongs to a power supply domain 501 while the system 102 to a power supply domain 502 heard. Both power domains 501 , 502 are independent of each other.

The system 101 comprises the device 121 as well as two video units 161 to the multiple cameras 181 are connected, each of which shows a camera.

The system 102 includes a device 122 and two video units 162 , to the respective cameras 182 are connected, each of which shows a camera. The system 102 further comprises two control units 142 and 143 , each via a data line 38 with the device 122 are connected. In contrast, the system has 101 just a control unit 141 on that over the data line 38 with the device 121 connected is.

In contrast to 2 are in 3 only the control lines 28 for the camera control signals, but not the video lines and the corresponding interfaces. With the video units 161 , 162 are only the control ports 24 shown at the devices 121 , 122 only the control interfaces 20th .

In the event of a failure of the two control units 142 , 143 of the system 102 can the cameras 182 nevertheless from the control unit 141 are controlled and the camera control signals from the cameras 182 be received. Access can be from the control unit on the one hand 141 about the device 121 and the control line 28 take place, the from the control interface 202 to the control ports 24 of the video units 162 runs. In addition or as an alternative, but also preferred, control of the cameras is preferred 182 from the control unit 141 about the device 121 and the transceiver lines 56 as well as the device 122 , the control interface 203 and the corresponding control line 28 possible. The cameras can of course also 181 from the control unit 141 are controlled by means of the device 121 and the video units 161 .

Through this arrangement 48 So even the failure of two control units can be avoided 142 , 143 balance. Even with an additional failure of the device 122 access to the cameras would remain 182 of the system 102 possible. A control of the cameras 182 could still be done. The video signals could also be accessed.

In 3 are parts of the described processing streams through the lines 58 shown.

4th shows an expanded arrangement 48 with two systems 101 and 102 . Any of the systems 101 and 102 has in addition to the control units 14th and fixtures 12th a safety processor 60 on which the respective systems 101 , 102 supervised. The safety processor not only controls the overall system and monitors the individual components including the control units 141 , 142 , 143 . It also controls the data flow of the camera control signals and, if necessary, the video signals of the individual systems 101 , 102 .

The safety processor 601 of the system 101 is via a control line 281 with the video units 161 of the system 101 connected. Via a control line 282 the safety processor is connected 601 with the device 122 via their control interface 204 . The two video units are at the same time 161 via the control line 282 also with the device 122 of the system 102 connected, the connection using the same control interface 204 he follows.

In addition, there is preferably a connection to the safety processor 601 directly to the device 121 via a security line 62 . The safety processor 601 directly to a camera multiplexer 64 access. In the camera multiplexer 64 the device 121 are made via the control interfaces 201 and 202 received camera control signals. The device 121 comprises another video multiplexer 66 representing the video signals passing through the video input interfaces 22nd from the video units 161 are received, brings together. The output signals of the two multiplexers 64 , 66 are then in the processing unit 32 processed and via the device interface 36 and the PCIe data line 38 to the control unit 141 transmitted.

The same configuration also applies to the system 102 in which the safety processor 602 the device 122 and their camera multiplexer 64 controls accordingly. Can be seen in 4th also that the device 122 a video multiplexer 66 and the video signals as well as the camera control signals in the processing units 32 are processed into control signals in which the video signals are tunneled. The control signals are sent via the PCIe data lines 38 to the control units 142 , 143 transmitted.

In the event of a fault in a control unit 14th can the safety processor 60 use the remaining resources accordingly and redirect the data and data streams and switch off defective parts or elements of the system, for example the defective control unit. With the appropriate application software, this control and monitoring can also be carried out via the connection between the safety processor 60 and the control unit 141 or the control units 142 and 143 indirectly via the PCle data line 38 respectively.

5 shows a vehicle 70 with the system according to the invention 10 and a variety of cameras 18th connected to the video units via serial lines 16 of the system 10 are connected. Of course the system can 10 also part of an arrangement 48 be like her in the 2 until 4th are shown. A vehicle preferably comprises a plurality of such arrangements 48 with multiple systems 10 , with failure of individual control units 14th that of the control unit 14th assigned cameras 18th can be controlled and controlled by other control units and their data can be processed. In this way, even if individual control units fail 14th autonomous driving guaranteed. This arrangement makes it possible to increase the overall robustness of the autonomous driving system. The vehicle is not only robust, but also very reliable.

6th shows schematically the method according to the invention for processing video signals and for controlling cameras in an autonomous or partially autonomous vehicle 70 . The method comprises several steps that can be processed in the order shown here or in a different order. In a receiving step S10, video signals are sent from a camera of the vehicle 70 received, preferably by means of a video input interface 22nd the device 12th . The method comprises a further step of receiving S12 camera control signals from a camera 18th that preferably has a video unit 16 get to the device. In a step of processing S14, the video signals and the camera control signals are processed by means of a processing unit 32 processed. In a generation step S16, control signals are generated which include the video signals and the camera control signals. The control signals can be from a control unit of the vehicle 70 are further processed. The method further comprises a step of transmission S18, in which control signals are sent to the control unit 14th of the vehicle 70 be transmitted.

Optionally, the method can include further steps that are described in 6th are shown in dashed lines. The expanded method comprises a step of receiving S20, in which control signals from the control unit 14th be received. In another step of the processing S22, the control signals from the control unit 14th in the device 12th processed. Optionally, the method includes the further step of extracting S24, in which from the control signals from the control unit 14th Camera control signals for the camera 18th extracted in another step of outputting S26 to a video unit 16 of the vehicle 70 can be output so that the camera control signals for the camera 18th by means of the video unit 16 can be forwarded.

The invention has been comprehensively described and explained with reference to the drawings and the description. The description and explanation are to be understood as an example and not restrictive. The invention is not limited to the disclosed embodiments. Other embodiments or variations will become apparent to those skilled in the art using the present invention and a careful analysis of the drawings, the disclosure, and the following claims.

In the claims, the words “comprising” and “having” do not exclude the presence of further elements or steps. The undefined article “a” or “an” does not exclude the presence of a plural. A single element or a single unit can perform the functions of several of the units mentioned in the patent claims. An element, a unit, an interface, a device and a system can be implemented partially or completely in hardware and / or in software. The mere mention of some measures in several different dependent patent claims should not be understood to mean that a combination of these measures cannot also be used advantageously. A computer program can be stored / distributed on a non-volatile data carrier, for example on an optical memory or on a semiconductor drive (SSD). A computer program can be distributed together with hardware and / or as part of hardware, for example by means of the Internet or by means of wired or wireless communication systems. Reference signs in the patent claims are not to be understood as restrictive.

List of reference symbols

10,101,102

system

12, 121, 122

contraption

14, 141, 142, 143

Control unit

16,161,162

Video unit

18,181,182

camera

20, 201, 202, 203, 204

Control interface

22nd

Video input interface

24

Control port

26th

Video output port

28,281,282

Control line

30th

Video line

32

Processing unit

34

multiplexer

36

Device interface

38

PCIe data line

40

Input interface

42

Evaluation unit

44

driver

46

processor

48

arrangement

50, 501, 502

Power domain

52

line

54

capacitor

56

Transceiver line

58

line

60, 601, 602

Safety processor

62

Security line

64

Camera multiplexer

66

Video multiplexer

70

vehicle

Claims (15)

Device (12, 121, 122) for processing video signals and controlling cameras in an autonomous or semi-autonomous vehicle (70), with: a video input interface (22) for receiving video signals from a camera (18, 181, 182) of the vehicle (70); a bidirectional control interface (20, 201, 202, 203) for receiving and outputting camera control signals for the camera (18, 181, 182); a processing unit (32) for processing the video signals and the camera control signals and for generating control signals for a control unit (14, 141, 142, 143) of the vehicle (70); and a bidirectional device interface (36) for transmitting the control signals to the control unit (14, 141, 142, 143) and for receiving control signals from the control unit (14, 141, 142, 143); wherein the control signals include the video signals and the camera control signals. Device (12) after Claim 1 wherein the device interface (36) is a PCIe interface. Device (12, 121, 122) according to one of the preceding claims, wherein the processing unit (32) is designed to extract camera control signals from the received control signals. Device (12, 121, 122) according to one of the preceding claims, wherein the device has a plurality of video input interfaces (22) and the processing unit processes the video signals of the video input interfaces (22) jointly. Device (12, 121, 122) according to one of the preceding claims, wherein the device has several control interfaces (20, 201, 202, 203) and the processing unit (32) combines the camera control signals of the control interfaces (20, 201, 202, 203). Device (12, 121, 122) according to Claim 4 or 5 , wherein the device (12, 121, 122) has a multiplexer (34) by means of which the video signals or the camera control signals are combined. Device (12, 121, 122) according to one of the preceding claims, wherein the device (12, 121, 122) has a communication interface in order to communicate directly with a further device (12, 121, 122) and to exchange video signals and / or camera control signals . Device (12, 121, 122) according to one of the preceding claims, wherein the device (12, 121, 122) is an FPGA module. System (10, 101, 102) for processing video signals and controlling cameras (18, 181, 182) in an autonomous or semi-autonomous vehicle (70), having a device (12, 121, 122) according to one of the preceding claims a video unit (16, 161, 162) with a video output port (26) and a control port (24), and with a control unit (14, 141, 142). System (10, 101, 102) according to Claim 9 , wherein the device has multiple video input interfaces (22) and the system (10, 101, 102) comprises multiple video units (16, 161, 162) and the control ports (24) of the video units (16, 161, 162) with the control interface (20 , 201, 202, 203) of the device (12, 121, 122) are connected and the video output ports (26) are each connected to a video input interface (22) of the device (12, 121, 122). System (10, 101, 102) according to one of the preceding Claims 9 until 10 , wherein the system (10, 101, 102) has a safety processor (60) which is designed to control the processing of the camera control signals and the video data and various devices (12, 121, 122) or control units (14, 141, 142, 143). Vehicle (70) with a system (10, 101, 102) according to one of the Claims 9 until 11 . Method for processing video signals and controlling cameras (18, 181, 182) in an autonomous or semi-autonomous vehicle (70), comprising the following steps: Receiving video signals from a camera (18, 181, 182) of the vehicle (70) by means of a video input interface (22); Receiving camera control signals; Processing the video signals and the camera control signals by means of a processing unit (32); Generating control signals comprising the video signals and the camera control signals for a control unit (14, 141, 142, 143) of the vehicle (70); Transmission of the control signals to the control unit (14, 141, 142, 143). Procedure according to Claim 13 , with the further steps of: receiving control signals from the control unit (14, 141, 142, 143); Processing the control signals from the control unit (14, 141, 142, 143); Extracting camera control signals for the camera (18, 181, 182) from the control signals from the control unit (14, 141, 142, 143); and outputting camera control signals for the camera (18, 181, 182) to a video unit (16, 161, 162) of the vehicle (70). Computer program product with program code for performing the steps of the method to Claim 13 or 14th when the program code is executed on a computer.

Images