WO2023139140A1

WO2023139140A1 - Error management for an automotive camera

Info

Publication number: WO2023139140A1
Application number: PCT/EP2023/051160
Authority: WO
Inventors: Michael Buckley; Gerard Conway; Patrick Mcdaid; Enda Peter Ward; Jose Alberto De La Fuente Gamiz
Original assignee: Connaught Electronics Ltd.
Priority date: 2022-01-21
Filing date: 2023-01-19
Publication date: 2023-07-27
Also published as: DE102022101358A1

Abstract

An automotive camera (3) comprises an imager (5), wherein the imager (5) comprises an array of optical detectors, which is configured to generate pixel data depending on light impinging on the array of optical detectors. The imager (5) is configured to provide image data depending on the pixel data at a data output of the imager (5). The imager (5) comprises an error monitoring unit, which is configured to detect an internal error of the imager (5). The error monitoring unit is configured to determine a bit-pattern, which is assigned to the detected internal error, and to generate an error signal (9) representing the bit pattern (8) at an error output (7) of the imager (5).

Description

Error management for an automotive camera

The present invention is directed to an automotive camera comprising an imager, wherein the imager comprises an array of optical detectors, which is configured to generate pixel data depending on light impinging on the array of optical detectors, the imager is configured to provide image data depending on the pixel data at a data output of the imager and the imager comprises an error monitoring unit, which is configured to detect an internal error of the imager. The invention is further directed to a camera system with such an automotive camera and to a corresponding method for error management.

Cameras, which are designed to be arranged on motor vehicles, may be denoted as automotive cameras. Automotive cameras depicting an exterior environment of the motor vehicle are used for various driver assistance functions or other functions for partially or fully automatic driving of the motor vehicle. However, automotive cameras may also be employed for the capturing an interior of the vehicle and may be arranged correspondingly. Thus, automotive cameras may be exposed to significant temperature fluctuations and/or other environmental conditions. Moreover, automotive cameras are also exposed to mechanical stress due to the vibrations while the vehicle is moving. Automotive cameras for motor vehicles should therefore be designed to still operate as desired under said conditions and to achieve sufficiently good image quality. The images may then be provided to driver assistance systems or other electronic vehicle guidance systems, which may use them as an input for driver assistance or at least partly automatically guiding the vehicle.

In an automotive camera, in particular in the imager, a variety of errors or malfunctions may occur. To fulfill certain standards, for example ratings according to the automotive safety integrity level, ASIL, scheme, safety mechanisms have to be provided for the automotive camera. In known automotive cameras, the imager may output a binary alert to indicate whether or not an error is present. An external electronic control unit, ECU, of the motor vehicle detects the alert and reads a plurality of registers in the imager to find out what type of error is actually present. If necessary, the ECU may then initiate certain actions. Since the ECU must actively interrogate the registers of the imager, there is a time delay between providing the alert and the initiation of the action.

It is an objective of the present invention to reduce the reaction time in error management of an automotive camera. This objective is achieved by the respective subject matter of the independent claims.

Further implementations and preferred embodiments are subject matter of the dependent claims.

The invention is based on the idea to generate an error signal by the imager, which represents a characteristic bit pattern, which allows to directly identify the actual error type of a detected error.

According to an aspect of the invention, an automotive camera comprising an imager is provided. The imager comprises an array of optical detectors, wherein the array is configured to generate pixel data depending on light impinging on the array of optical detectors. The imager is configured to provide, in particular to generate and provide, image data, in particular raw image data, depending on the pixel data at a data output of the imager. The imager comprises an error monitoring unit, which is configured to detect an internal error of the imager. The error monitoring unit is configured to determine a bit pattern, in particular a predefined bit pattern, which is assigned to the detected internal error. In particular, the assignment of the bit pattern to the detected internal error is predetermined. The error monitoring unit is configured to generate an error signal, in particular a digital error signal, representing the bit pattern at an error output, in particular an error output pin, of the imager.

Herein, light may be understood as electromagnetic radiation including visible light, in particular light with a wavelength between 380 nm and 750 nm, infrared light, in particular near or mid infrared light with a wavelength between 750 nm and 8000 nm, as well as ultraviolet light, in particular light with a wavelength between 100 nm and 380 nm. The expression “optical” can be understood to refer to light according to this meaning.

In particular, the automotive camera is specifically adapted to be used as a sensor system for a motor vehicle, in particular for an advanced driver assistance system, ADAS, or another electronic vehicle guidance system for partially or fully autonomous driving of the motor vehicle. In particular, the automotive camera is designed as a so-called satellite camera, as used for example for automotive surround view systems, SVS. Satellite cameras provide raw image data that is unprocessed or only partially processed pixel data of the array of optical detectors to an electronic control unit, ECU, of the motor vehicle. The ECU may receive the respective sets of raw image data from a plurality of satellite cameras and carry out a centralized processing of the various sets of raw image data from different satellite cameras to produce images or videos, which may be displayed by means of a display device, in contrast to the raw image data. Therefore, satellite cameras may be distinguished from so-called smart cameras, which directly provide fully processed images or videos, which may directly be displayed by means of a display device.

An imager may be understood as an electronic device comprising an image sensors with an array of optical detectors. The image sensor may be designed as a CCD image sensor, a CMOS image sensor, in particular an active pixel sensor or a passive pixel sensor, or another image sensor, which is sensitive to light. The imager may also comprise further electronic components in addition to the image sensor, for example for reading out the image sensor and/or or preprocessing purposes et cetera. The imager may be implemented on a single semiconductor chip and may therefore also be denoted as imager chip. Alternatively, the image sensor and the further electronic components may be implemented on different semiconductor chips, which may for example be mounted on a common circuit board.

The internal error of the imager may for example correspond to a malfunction of an electronic component of the imager or to a parameter of the imager laying outside of a predefined specification. Some exemplary non-limiting examples include a frame counter error, a checksum error, in particular cyclic redundancy check, CRC, checksum error, a communication error, such as an I2C communication error, an error of a storage device, for example of an error correction code random access memory, ECC-RAM, an OTP error et cetera. Other examples correspond to errors due to an internal voltage of the imager, for example a supply voltage, or an internal temperature of the imager being out of a predefined specification.

Consequently, for each of a plurality of different potential errors, a corresponding bit pattern that is a sequence of two or more bits is predefined. The corresponding assignments may be stored on the imager, for example a lookup table. The error monitoring unit may then for example read the stored assignment of the detected internal error and thereby determine the corresponding bit pattern.

The error monitoring unit may be implemented in software and/or in hardware and may, in particular, comprise an error monitoring software module and/or an error monitoring circuit. In some implementations, two or more error monitoring units may be provided, which may run in parallel. By means of the invention, an external system, in particular the ECU of the motor vehicle, may read the error signal at the error output of the imager and determine the bit pattern from the error signal. Therefore, the external system or ECU has immediately the respective information about which type of error is present and does not need to interrogate the imager in detail or for example read the corresponding registers in the imager to find out what type of error is present. The ECU can therefore initiate the corresponding action immediately with a reduced delay time. In this way, the reaction time between identification of the internal error and initiation of the corresponding action may be reduced. While not per se limited to satellite cameras, the invention is particularly suitable for those, since for known camera systems based on satellite cameras, the ECU needs to interrogate the registers of the imagers.

In some implementations, the bit pattern is given by a sequence of at least three bits, preferably of at least four bits, for example of at least eight bits. In preferred embodiments, the length of the bit pattern is equal to eight bits.

In this way, a large number of different internal errors may be distinguished. In particular, also other components of the automotive camera, for example a serializer, may provide a corresponding further error signal as well. In this case, it may be beneficial to provide unique assignments of the bit patterns to the corresponding errors also amongst the different components of the automotive camera.

According to several implementations, the imager, in particular the error monitoring unit, comprises a storage device storing a rule, which assigns the detected internal error to the bit pattern. The error monitoring unit is configured to read the rule in order to determine the bit pattern.

The rule corresponds to the assignment of the internal error to the bit pattern. This may for example be stored in terms of a lookup table.

In alternative implementations, the error monitoring unit may determine the bit pattern in a predefined manner without explicitly reading a corresponding assignment, which is stored on a storage device. For example, the error monitoring unit could carry out a corresponding monitoring routine and individual digits of the bit pattern may result from individual steps of the monitoring scheme in a predefined manner. In both cases, the error signal representing the bit pattern may be provided in a particularly fast way.

In alternative implementations, the assignment of the bit-pattern to the detected internal error is hard-coded in the imager.

According to several implementations, the automotive camera comprises a serializer with an error input, in particular an error input pin, which is connected to the error output of the imager to receive the error signal. The serializer comprises a serializer output, in particular a serializer output pin, for connecting the automotive camera, in particular the serializer, to the ECU of the motor vehicle. The serializer is configured to provide the error signal to the ECU at the serializer output. The error input may for example be given by a general purpose input-output pin of the serializer.

In such implementations, the ECU is not necessarily directly connected to the imager but rather to the serializer, to which it may anyway be connected in order to receive the serialized image data.

According to several implementations, the serializer comprises a data input, in particular a data input pin, which is connected to the data output of the imager for receiving the image data. The serializer is configured to serialize the image data and to provide the serialized image data to the ECU at the serializer output or at a further serializer output, in particular a further serializer output pin, of the serializer.

According to several implementations, the serializer comprises a further error monitoring unit, which is configured to detect a further internal error of the serializer. The further error monitoring unit is configured to determine a further bit pattern, which is assigned to the detected further internal error. The further error monitoring unit is configured to generate a further error signal representing the further bit pattern at a further error output, in particular a further error output pin, of the serializer or at the further serializer output.

The further error output of the serializer is connectable or connected to the ECU such that the ECU may receive the further error signal.

In such implementations, the ECU may immediately receive not only the information regarding the type of error detected internally in the imager but also in of an error detected in the serializer. This increases the overall level of safety and/or decreases the response time.

According to a further aspect of the invention, a camera system for a motor vehicle is provided. The camera system comprises an automotive camera according to the invention and the electronic control unit for the motor vehicle. The ECU is connected to the automotive camera for receiving the error signal from the automotive camera.

In particular, the ECU is connected to the serializer output of the serializer to receive the error signal. Optionally, the ECU may be connected to the serializer output or the further serializer output to receive the serialized image data. Optionally, the ECU may be connected to the further error output of the serializer output to receive the further error signal.

According to several implementations, the ECU is configured to initiate an action, in particular a predefined action, which is assigned to the bit pattern depending on the error signal.

For example, the ECU may read a further storage device of the ECU or of the motor vehicle, which stores the rule assigning the bit pattern to the detected internal error and consequently initiate the action.

Initiating the action may for example include generating a warning or information signal for an electronic vehicle guidance system of the motor vehicle, which informs the electronic vehicle guidance system that the automotive camera may be delivering faulty, unreliable or impaired data. For example, the image data received from the automotive camera may then not be used for any safety relevant purposes. Alternatively or in addition, an error code may be stored as a part of the initiated action, which may be read out at a service inspection. In particular, the individual actions initiated by the ECU differ amongst each other depending on which bit pattern and correspondingly which internal error has been detected.

According to several implementations of the camera system, the automotive camera is implemented to comprise the serializer as described above. The ECU is connected to the serializer output to receive the error signal. According to several implementations, the ECU comprises a deserializer with a deserializer input, in particular a deserializer input pin, which is connected to the serializer output or the further serializer output to receive the serialized image data. The deserializer is configured to reconstruct the image data by deserializing the serialized image data.

According to several implementations, the ECU is configured to process the reconstructed image data to generate image data suitable for being displayed on a display device of the motor vehicle.

According to a further aspect of the invention, a motor vehicle comprising a camera system according to the invention is provided.

According to a further aspect of the invention, a method for error management for an automotive camera is provided. Therein, pixel data depending on light impinging on an array of optical detectors of an imager of the automotive camera is generated, in particular by the array of optical detectors. Image data depending on the pixel data is provided, in particular by the imager, at a data output of the imager. An internal error of the imager is detected by the imager, in particular by an error monitoring unit of the imager. A bit pattern, which is assigned to the detected internal error, is determined by the imager, in particular the error monitoring unit. An error signal representing the bit pattern is generated by the imager, in particular by the error monitoring unit, at an error output of the imager.

According to several implementations of the method, the error signal is transmitted from the automotive camera to an electronic control unit of a motor vehicle.

According to several implementations, an action, which is assigned to the bit pattern, is initiated, in particular by the electronic control unit, depending on the error signal.

Further implementations of the method for error management according to the invention follow directly from the various implementations of the automotive camera and the camera system according to the invention and vice versa, respectively. In particular, individual features and corresponding explanations regarding the various embodiments to the automotive camera or the camera system may be transferred analogously to corresponding implementations of the method for error management. In particular, an automotive camera or camera system according to the invention is designed or programmed to perform a method according to the invention or carries out such a method.

Further features of the invention are apparent from the claims, the figures and the figure description. The features and combinations of features mentioned above in the description as well as the features and combinations of features mentioned below in the description of figures and/or shown in the figures may be comprised by the invention not only in the respective combination stated, but also in other combinations. In particular, embodiments and combinations of features, which do not have all the features of an originally formulated claim, are also comprised by the invention. Moreover, embodiments and combinations of features which go beyond or deviate from the combinations of features set forth in the recitations of the claims are comprised by the invention.

In the following, the invention will be explained in detail with reference to specific exemplary implementations and respective schematic drawings. In the drawings, identical or functionally identical elements may be denoted by the same reference signs. The description of identical or functionally identical elements is not necessarily repeated with respect to different figures.

In the figures,

Fig.1 shows schematically a motor vehicle with an exemplary implementation of a camera system according to the invention;

Fig. 2 shows a block diagram of a further exemplary implementation of a camera system according to the invention; and

Fig. 3 shows an exemplary representation of an error signal and a corresponding bit pattern.

Fig. 1 shows schematically a top view of a motor vehicle 1 , which comprises a camera system 2 according to the invention. The camera system 2 comprises an electronic control unit, ECU, 4 of the motor vehicle 1 and one or more automotive cameras 3 connected to the ECU 4. The ECU 4 may receive respective data from each of the automotive cameras 3 and process them data in a centralized manner to generate images or videos, which are displayable on a display device (not shown) of the motor vehicle 1 . The automotive cameras 3 may for example correspond to different satellite cameras of a surround view system of the motor vehicle 1 .

Fig. 2 shows a schematic block diagram of the camera system 2, wherein only a single automotive camera 3 is shown. However, in case the camera system 2 comprises more than one automotive cameras 3, the explanations with respect to the shown automotive camera 3 of Fig. 2 may be carried over to the remaining automotive cameras 3 accordingly.

The automotive camera 3 comprises an imager 5 and a serializer 6, wherein a data output of the imager 5 is connected to a data input of the serializer 6. The imager 5 comprises an array of optical detectors (not shown), which may generate pixel data depending on light impinging on the array of optical detectors. The imager 5 provides image data, in particular raw image data, depending on the pixel data at the data output to the serializer 6.

The serializer 6 serializes the received image data and provides the serialized image data to the ECU 4. The ECU 4 may comprise a deserializer (not shown) to deserialize the serialized image data in order to reconstruct the image data. The ECU 4 may then process the reconstructed image data to obtain a suitable data format for being displayed.

In addition, the imager 5 comprises an error monitoring unit (not shown), which is configured to detect an internal error of the imager 5 and to determine a bit pattern 8, which is assigned to the detected internal error. The error monitoring unit is configured to generate an error signal 9, as indicated exemplarily in Fig. 3, at an error output pin 7 of the imager 5.

The error output pin 7 may for example be connected to an error input pin of the serializer 6, which then provides the error signal to the ECU 4. Alternatively, the ECU 4 may be directly connected to the error output pin 7 of the imager to receive the error signal. 9

The ECU 4 obtains the error signal 9 and determines the bit pattern 8 based on the error signal 9. From the bit pattern 8, the ECU 4 may then identify the underlying error of the imager 5 and initiate a corresponding action, which is associated to the particular type of error identified. Consequently, as described, the invention allows for a reduced response time between the detection of the error and the initiation of the action, since the ECU 4 does not have to interrogate individual registers of the imager 5 to determine the type of the error.

The invention may for example implement an error code system, which use one or two wires. Instead of the safety mechanism giving back a single binary value, that is 0 or 1 value, it may provide a unique bit-pattern, for example an 8-bit pattern allowing for 256 unique values. This is very beneficial as it makes it obsolete for the ECU to read multiple registers of the imager to find out the error type. The reaction time is shortened, which makes the system overall more efficient and safer as the user may be notified faster.

The advantages of therefore may include a reaction time improvement, an optimization of the overall system, for example by making better use of the available error pins, and/or an ASIL safety feature improvement, which results in particular from the reaction time improvement.

Claims

1 . Automotive camera (3) comprising an imager (5), wherein the imager (5) comprises an array of optical detectors, which is configured to generate pixel data depending on light impinging on the array of optical detectors; the imager (5) is configured to provide image data depending on the pixel data at a data output of the imager (5); the imager (5) comprises an error monitoring unit, which is configured to detect an internal error of the imager (5); characterized in that the error monitoring unit is configured to determine a bit-pattern, which is assigned to the detected internal error; and the error monitoring unit is configured to generate an error signal (9) representing the bit pattern (8) at an error output (7) of the imager (5).

2. Automotive camera (3) according to claim 1 , characterized in that the bit pattern (8) is given by a sequence of at least two bits, preferably of at least three bits, for example of at least four bits.

3. Automotive camera (3) according to one of the preceding claims, characterized in that the imager (5) comprises a storage device storing a rule, which assigns the detected internal error to the bit-pattern and the error monitoring unit is configured to read the rule in order to determine the bit-pattern; or the assignment of the bit-pattern to the detected internal error is hard-coded in the imager (5).

4. Automotive camera (3) according to one of the preceding claims, characterized in that the automotive camera (3) comprises a serializer with an error input, which is connected to the error output (7) of the imager (5) to receive the error signal (9); and the serializer comprises a serializer output for connecting the automotive camera (3) to an electronic control unit (4) of a motor vehicle (1); and the serializer is configured to provide the error signal (9) at the serializer output.

5. Automotive camera (3) according to claim 4, characterized in that the serializer comprises a data input, which is connected to the data output of the imager (5) for receiving the image data; and the serializer is configured to serialize the image data and to provide the serialized image data at the serializer output or at a further serializer output of the serializer.

6. Automotive camera (3) according to one of claims 4 or 5, characterized in that the serializer comprises a further error monitoring unit, which is configured to detect a further internal error of the serializer; the further error monitoring unit is configured to determine a further bit-pattern, which is assigned to the detected further internal error; and the further error monitoring unit is configured to generate a further error signal (9) representing the further bit-pattern at a further error output (7) of the serializer or at the serializer output.

7. Camera system (2) for a motor vehicle (1 ), the camera system (2) comprising an automotive camera (3) according to one of the preceding claims and an electronic control unit (4) connected to the automotive camera (3) for receiving the error signal (9).

8. Camera system (2) according to claim 7, characterized in that the electronic control unit (4) is configured to initiate an action, which is assigned to the bit-pattern, depending on the error signal (9).

9. Camera system (2) according to one of claims 7 or 8, characterized in that the automotive camera (3) is implemented according to one of claims 4 or 5; and the electronic control unit (4) is connected to the serializer output to receive the error signal (9).

10. Camera system (2) according to claim 9, characterized in that the automotive camera (3) is implemented according to claim 5; the electronic control unit (4) comprises a deserializer with a deserializer input, which is connected to the serializer output or the further serializer output to receive the serialized image data; and the deserializer is configured to reconstruct the image data by deserializing the serialized image data.

11 . Camera system (2) according to claim 10, characterized in that the electronic control unit (4) is configured to process the reconstructed image data to generate image data suitable for being displayed on a display device of the motor vehicle (1).

12. Motor vehicle (1) comprising a camera system (2) according to one of claims 7 to 11 .

13. Method for error management for an automotive camera (3), wherein pixel data depending on light impinging on an array of optical detectors of an imager (5) of the automotive camera (3) is generated; image data depending on the pixel data is provided at a data output of the imager (5); an internal error of the imager (5) is detected by the imager (5); characterized in that a bit-pattern, which is assigned to the detected internal error, is determined; and an error signal (9) representing the bit pattern (8) is generated at an error output (7) of the imager (5).

14. Method according to claim 13, characterized in that the error signal (9) is transmitted from the automotive camera (3) to an electronic control unit (4) of a motor vehicle (1).

15. Method according to claim 14, characterized in that an action, which is assigned to the bit-pattern, is initiated depending on the error signal (9).