EP4721015A1 - Method for anonymization of image data for a telematics device of a vehicle, in particular utility vehicle, telematics device, vehicle, computer program - Google Patents

Abstract

Method (100) for anonymization of image data (250) for a telematics device (210) of a vehicle (200a), in particular utility vehicle (200b), wherein the method (100) comprises: receiving (110), from a control unit (225) of an advanced driver assistance system (220), a presence tag (260) indicating the presence or absence of a representation of a vulnerable road user (290) and/or of a further vehicle (280) in sensor data (270) being acquired by a sensor device (226) of the advanced driver assistance system (220); obtaining (120) the image data (250) from a camera device (215) being adapted to be connected to the telematics device (210); and performing (130) anonymization of the image data (250), depending on the presence tag (260).

Description

Method for anonymization of image data for a telematics device of a vehicle, in particular utility vehicle, telematics device, vehicle, computer program

The invention relates to a method for anonymization of image data for a telematics device of a vehicle, in particular utility vehicle. The invention also relates to a telematics device for a vehicle, in particular utility vehicle, to a vehicle, in particular utility vehicle, and to a computer program.

A vehicle comprises a plurality of sensors which may be adapted to acquire data which may contain personal data. Such personal data may comprise image data which may relate to a person and/or a feature of a person. Personal data may also comprise image data that may enable an identification of vehicle-specific information which may be indicative of a person, such as a license plate and/or a number plate.

WO 2019/048034 A1 discloses an electronic logging and track detection system for mobile telematics devices and method thereof. In particular, an electronic logging and track detection system and system for mobile telematics devices, as smart phones and/or mobile cellular phones is disclosed, which tend to change their sensing and measuring orientation and direction in respect to the main direction of movement/motion or moving sense, as for example given by a person with proper motion holding a mobile phone within a moving vehicle. Instantaneous movement telematics data are measured by and logged from sensors of the mobile telematics devices and trips and/or trip-segments based on the instantaneous movement sensory telematics data are automatically identified and detected at least by the telematics sensors comprising an accelerometer sensor and a gyroscope sensor and a Global Positioning System sensor. The telematics devices comprise one or more wireless connections acting as a wireless node within a corresponding data transmission network by means of antenna connections of the telematics device. Therein, exteroceptive sensors or measuring devices can, for example, comprise at least radar devices for monitoring surrounding of a motor vehicle and/or LIDAR devices for monitoring surrounding of the motor vehicle, vehicle tracking devices and/or computer vision devices or video cameras for monitoring the surrounding of the motor vehicle. Therein, the surrounding may comprise vehicles and pedestrians.

However, an anonymization of images captured through a road facing camera as an example of a sensor for further applications, such as connected advanced driver assistance systems, ADAS, may be advisable and/or mandatory due to requirements relating to the compliance with a data protection regulation, e.g., of contractual form and/or as regulated by law, e.g., Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation), GDPR.

The sensor data that is captured for performing an automatic driving function, a so- called event, and/or for recording a backup, typically comprises image data and/or video data with a comparatively large resolution and is thereby of a large size. Due to data protection regulations, any personally identifiable information in the data may be anonymized before it is shared, e.g., with a cloud server and/or viewed by a human operator.

Image and/or video anonymization is usually performed using machine learning and, specifically, by deep learning algorithms. Such algorithms are usually processing and/or memory intensive. A vehicle telematics device, however, is usually constrained with respect to memory storage and processing capabilities. Hence realtime or even near real-time anonymization of any acquired data would allocate a majority of the processing power and/or memory storage, which may result in other applications on the device being be hampered and/or even in an insufficient performance of the anonymization algorithm.

It is an object of the present invention to provide a technological contribution and to improve at least one aspect of the prior art. In particular, an object of the invention may be to provide an effective anonymization of image data which may be shared by a telematics device of a vehicle. The object is solved by the subject-matter according to independent claim 1 and according to the remaining independent claims. Dependent claims relate to preferred embodiments.

According to an aspect of the invention, a method for anonymization of image data for a telematics device of a vehicle, in particular utility vehicle, is provided. Therein the method comprises: receiving, from a control unit of an advanced driver assistance system, a presence tag indicating the presence or absence of a representation of a vulnerable road user and/or of a further vehicle in sensor data being acquired by a sensor device of the advanced driver assistance system; obtaining the image data from a camera device being adapted to be connected to the telematics device; and performing anonymization of the image data, depending on the presence tag.

Therein, data may be acquired by different sensors: the sensor data may be acquired by the sensor device and the image data may be acquired by the camera device. The sensor device typically acquires the sensor data for further processing for performing an automated driving function by the advanced driver assistance system. The camera device is connected to the telematics device and the image data may be communicated by the telematics device.

The invention has realized that it is not necessary to perform an anonymization of all the image data that is acquired. It is further realized that the sensor data is, for the purpose of performing the automated driving function, to be processed anyway so that the presence tag may be derived while processing the sensor data. The presence tag indicates the presence or absence of a representation of a vulnerable road user and/or of a further vehicle and may thus be indicative of the necessity to anonymize the image data. Therein, the presence or absence of a representation of a vulnerable road user and/or of a further vehicle may relate to a pictorial description of the vulnerable road user and/or of the further vehicle. The vulnerable road user may be a person. The further vehicle may be a vehicle different from the ego vehicle which comprises the telematics device. The presence tag may enable a selective anonymization of image data. This may reduce the amount of image data which is to be anonymized may thus reduce memory consumption and the allocation of computing capabilities. The above approach helps reduce the number of frames, i.e., images, that would be processed by the anonymizing algorithm.

Optionally, the anonymization of the image data is performed if the presence tag indicates the presence of a representation of a vulnerable road user and/or a further vehicle in sensor data. The presence tag may then be indicative of potentially sensible personal data. Thus, the presence tag may serve as a trigger for the telematics device to perform the anonymization of the image data. This achieves an effective protection of personal data.

Optionally, the anonymization of the image data is dispensed with if the presence tag indicates the absence of a representation of a vulnerable road user and/or a further vehicle in sensor data. Therein, the presence tag may indicate that there is no anonymizable information and/or no information that requires anonymization comprised by the image data. Thus, performing the anonymization may be dispensed with. This achieves an effective processing of data without allocating computational resources for potentially unnecessary image processing operations.

Optionally, the presence tag is received via a vehicle data bus [ TMMSCT1 ] . Transmitting the presence tag via a vehicle data bus, e.g., a CAN bus, Ethernet and/or any suitable high speed data communication network may be an effective implementation for providing and obtaining the presence tag. Therein, the control unit and the telematics device may be interconnected by the vehicle data bus [ TMMSCT2 ] .

Optionally, the sensor data comprises second image data, and the control unit comprises an image processing module to process the second image data to obtain the presence tag. There, the sensor device may comprise an ADAS camera device to obtain the second image data. The content of the second image data relates to the content of the image data. However, the second image data may be processed, by the image processing module, for performing the automated driving function and for obtaining the presence tag. Optionally, method comprises: receiving, from the control unit, a sensor timestamp relating to a data acquisition time of the sensor data by the sensor device, and performing anonymization of the image data is dependent on the sensor timestamp and on an image timestamp relating to an image acquisition time of the image data by the camera device. The timestamps may relate the data with the time at which the data is acquired. I.e., the sensor timestamps relate to the data acquisition time at which the sensor data is acquired, and the image timestamp relates to the image acquisition time at which the image data is acquired. The sensor timestamp and the image timestamp enable a direct comparison between the acquisition times. Therein, the comparison of the timestamps and the presence tag may trigger or avoid performing the anonymization of the image data. E.g., if the vulnerable road user and/or a further vehicle is present in the sensor data and a difference between the timestamps undercuts a threshold value, then the anonymization is performed, and otherwise the anonymization may be dispensed with.

The above may, in other words, be summarized as follows: the approach helps to mitigate some of the compute requirements for performing anonymization. The control unit broadcasts on the vehicle data bus when it detects vulnerable road users (VRUs) as well as vehicles in the field of view of the sensor device. If the telematics device is made aware of this information by the presence tag, the anonymization algorithm could be tweaked to only consider the frames and/or images that have the same and/or a similar timestamp as the information of VRUs or vehicles on the road. For such a system to be enabled, the telematics device is enabled to access the CAN information broadcasted by the control unit.

According to an aspect of the present invention, a telematics device for a vehicle, in particular utility vehicle, is provided. The telematics device is configured to performing the method steps of the method as described above. Optionally, the telematics device is configured to performing optional steps and/or realize optional features of the method as described above to achieve a technical effect corresponding thereto.

Optionally, the telematics device is adapted to be connected to a control unit of an advanced driver assistance system via a vehicle data bus [ TMMSCT3 ] . According to an aspect of the present invention, a vehicle, in particular utility vehicle, is provided. The vehicle, in particular utility vehicle, comprises the telematics device as described above, a camera device being adapted to be connected to the telematics device, an advanced driver assistance system with a control unit and a sensor device. Optionally, the telematics device is configured to performing optional steps and/or realize optional features of the method as described above to achieve a technical effect corresponding thereto.

According to an aspect of the present invention, a computer program is provided. The computer program comprises instructions which, when the program is executed by a processor, causes the processor to carry out the method as described above.

Optionally, the computer program comprises instructions to realize optional and/or preferred features and/or steps of the method as described above to achieve a technical effect corresponding thereto.

Further advantages and technical features and their technical effects are disclosed in the figures and the description thereof.

The figures show preferred embodiments as follows:

Fig. 1 a schematic of a vehicle, in particular utility vehicle, according to an embodiment of the invention; and

Fig. 2 a schematic of a method according to an embodiment of the invention.

Figure 1 shows a schematic of a vehicle 200a, in particular utility vehicle 200b, according to an embodiment of the invention. In the following, the vehicle 200a, in particular utility vehicle 200b, is referred to as vehicle 200a, 200b. The vehicle 200a, 200b is a land vehicle 200a, 200b.

The vehicle 200a, 200b comprises a telematics device 210 and an advanced driver assistance system 220. Each of the telematics device 210 and the vehicle 200a, 200b is adapted to perform the method 100 as described with reference to Figure 2. The telematics device 210 of Figure 1 comprises a communication module 211 [ TMMSCT4 ] to communicate with a vehicle external device, e.g., with a server and/or a cloud service provider (not shown). The communication module 211 is adapted to communicate over one more vehicle-to-everything (V2X) interfaces, e.g., via a cellular network and/or a wireless local area network.

The vehicle 200a, 200b comprises an imaging device 215, e.g. a video camera. The imaging device 215 is adapted to acquire image data 250 relating to a surrounding of the vehicle 200a, 200b. The image data 250 may comprise pixel graphics and/or a sequence thereof. The image data 250 is acquired at an image acquisition time t’ which may be processed and/or stored as an image timestamp 251 . The image timestamp 251 is defined by the imaging device 215 to indicate the image acquisition time t’.

In Figure 1 , the image data 250 as illustrated by a box with a solid line comprises a plurality of images, each of which is indicated by a box with a dashed line. The plurality of images may be a sequence of images, wherein each of the images is acquired at a specific image acquisition time t’. The course of image acquisition time t’ is schematically indicated by an arrow. Therein, in the surrounding of the vehicle, a vulnerable road user 290 and/or a further vehicle 280 [TMMscTsimay be present as indicated schematically with boxes with solid lines within the boxes with dashed lines. The vulnerable road user 290 and/or further vehicle 280 [TMMscT6]may be present in one or more of the images and thus in the image data 250 or be absent in the image data 250.

The imaging device 215 is connected to the telematics device 210. The telematics device 210 is adapted to obtain the image data 250 and the image acquisition time t’ from the imaging device 215. The telematics device 210 may process and/or transmit the image data 250 via the communication module 211 .

The vehicle 200a, 200b comprises an advanced driver assistance system 220 and a sensor device 226. The advanced driver assistance system 220 (ADAS) is adapted to perform automated driving functions and/or driver assistance functions. To perform such functions, the advanced driver assistance system 220 comprises a control unit

225 which is connected to the sensor device 226.

The sensor device 226 is adapted to acquire sensor data 270 relating to the surrounding of the vehicle 200a, 200b. The sensor device 226 may comprise a plurality of sensors. The sensor data 270 may comprise pixel graphics and/or a sequence thereof, i.e., second image data 271 . Thus, the sensor device 226 may comprise a second imaging device (not indicated). Furthermore, the sensor device

226 may comprise one or more sensors for acquiring distance-related information, e.g., the sensor device 226 may comprise a radar sensor and/or a lidar sensor and/or a camera sensor. The sensor data 270 is acquired at a sensor acquisition time t which may be processed and/or stored as sensor timestamp 261 . The sensor timestamp 261 is defined by the sensor device 226 to indicate the sensor acquisition time t.

Similar to the image data 250, the sensor data 270 as illustrated by a box with a solid line comprises a plurality of pieces of data, each of which is indicated by a box with a dashed line. The plurality of pieces of data may be a sequence of images, wherein each of the images is acquired at a specific sensor acquisition time t. The course of sensor acquisition time t is schematically indicated by an arrow. Therein, in the surrounding of the vehicle, the vulnerable road user 290 and/or the further vehicle 280 [TMMscT7] may be present as indicated schematically with boxes with solid lines within the boxes with dashed lines. The vulnerable road user 290 and/or further vehicle 280 [TMMscTsi may be present in one or more of the pieces of data and thus in the sensor data 270 or be absent in the sensor data 270.

The imaging device 215 and the sensor device 226 comprise an overlapping field of view, detection area and/or detection volume. Le., the vulnerable road user 290 and/or further vehicle 280 [TMMscT9] may be, at the same time, absent or present in both, the sensor data 270 and the image data 250. Le., the sensor data 270 may comprise a representation of the vulnerable road user 290 and/or further vehicle 280 at the sensor acquisition time t and the image data 250 may comprise a representation of the vulnerable road user 290 and/or further vehicle 280 [TMMSCT10] at the image acquisition time t', wherein the sensor acquisition time t and the image acquisition time t' are comparable. The sensor acquisition time t and the image acquisition time t' are comparable, if a difference between the sensor acquisition time t and the image acquisition time t' undercuts a threshold, e.g., of a second or a fraction thereof and/or a time that relates to an acquisition frequency, i.e., to the difference between two consecutive image acquisition time t' and/or sensor acquisition times t.

The control unit 225 comprises an image processing module 227 to process the second image data 271 and to obtain a presence tag 260. The control unit 225 and in particular the image processing module 227 is adapted to derive and/or compute, based on the sensor data 270, the presence tag 260 indicating the presence or absence of a representation of the vulnerable road user 290 and/or of the further vehicle 280 in the sensor data 270.

The vehicle 200a, 200b comprises a vehicle data bus 205. The telematics device 210 and the control unit 225 are interconnected with each other via the vehicle data bus 205. The telematics device 210 is adapted to receive the presence tag 260 and the sensor timestamp 261 via the vehicle data bus 205. The sensor data 270 may not be available to the telematics device 210, e.g., is not broadcasted over the vehicle data bus 205.

The telematics device 210 is adapted to obtain the presence tag 260 and to perform an anonymization of the image data 250, depending on the presence tag 260. The presence tag 260 indicates to presence or absence of the vulnerable road user 290 and/or of the further vehicle 280 in the sensor data 270 and thus also indicates, due to the overlapping field of views, detection areas and/or detection volumes, the presence or absence of the vulnerable road user 290 and/or of the further vehicle 280 in the image data 250. The presence tag 260 comprises an information relating to the presence or to the absence or is void to indicate presence or absence. The anonymization of the image data 250 is performed if the presence tag 260 indicates the presence of a representation of a vulnerable road user 290 and/or a further vehicle 280 in sensor data 270. The anonymization of the image data 250 is dispensed with if the presence tag 260 indicates the absence of a representation of a vulnerable road user 290 and/or a further vehicle 280 in sensor data 270. The telematics device 210 comprises a processor and a memory (not shown) and is adapted to perform a machine learning algorithm to perform an anonymization of the image data 250, e.g., by object detection for detecting potentially sensible pieces of information within the image data 250 and applying, for example, a blurring filter to the detected object.

Figure 2 shows a schematic of a method 100 according to an embodiment of the invention. The method 100 is a method 100 for anonymization of image data 250 for a telematics device 210 of a vehicle 200a, in particular utility vehicle 200b. Such a vehicle 200a, 200b is described with reference to Figure 1 .

The method 100 of Figure 2 comprises: receiving 110, from a control unit 225 of an advanced driver assistance system 220, a presence tag 260 indicating the presence or absence of a representation of a vulnerable road user 290 and/or of a further vehicle 280 in sensor data 270 being acquired by a sensor device 226 of the advanced driver assistance system 220. The presence tag 260 is received via a vehicle data bus 205. The sensor data 270 comprises second image data 271 , and the control unit 225 comprises an image processing module 227 to process the second image data 271 to obtain the presence tag 260.

The method 100 comprises: receiving 110’, from the control unit 225, a sensor timestamp 261 relating to a data acquisition time t of the sensor data 270 by the sensor device 226.

The method 100 comprises: obtaining 120 the image data 250 from a camera device 215 being adapted to be connected to the telematics device 210.

The method 100 comprises: performing 130 anonymization of the image data 250, depending on the presence tag 260. The anonymization of the image data 250 is performed if the presence tag 260 indicates the presence of a representation of a vulnerable road user 290 and/or a further vehicle 280 in sensor data 270. The anonymization of the image data 250 is dispensed with if the presence tag 260 indicates the absence of a representation of a vulnerable road user 290 and/or a further vehicle 280 in sensor data 270. Performing 130 anonymization of the image data 250 is dependent on the sensor timestamp 261 and on an image timestamp 251 relating to an image acquisition time t’ of the image data 250 by the camera device 215.

List of reference signs (Part of the description)

100 method

110 receiving

110’ receiving

120 obtaining

130 performing

200a vehicle

200b utility vehicle

205 vehicle data bus

210 telematics device

211 communication module

215 camera device

220 advanced driver assistance system

225 control unit

226 sensor device

227 image processing module

250 image data

251 image timestamp

260 presence tag

261 sensor timestamp

270 sensor data

271 second image data

280 further vehicle

290 vulnerable road user t data acquisition time t’ image acquisition time

Claims

Claims

1 . Method (100) for anonymization of image data (250) for a telematics device (210) of a vehicle (200a), in particular utility vehicle (200b), wherein the method (100) comprises:

- receiving (110), from a control unit (225) of an advanced driver assistance system (220), a presence tag (260) indicating the presence or absence of a representation of a vulnerable road user (290) and/or of a further vehicle (280) in sensor data (270) being acquired by a sensor device (226) of the advanced driver assistance system (220);

- obtaining (120) the image data (250) from a camera device (215) being adapted to be connected to the telematics device (210); and

- performing (130) anonymization of the image data (250), depending on the presence tag (260).

2. Method (100) as claimed in claim 1 , wherein the anonymization of the image data (250) is performed if the presence tag (260) indicates the presence of a representation of a vulnerable road user (290) and/or a further vehicle (280) in sensor data (270).

3. Method (100) as claimed in claim 1 or 2, wherein the anonymization of the image data (250) is dispensed with if the presence tag (260) indicates the absence of a representation of a vulnerable road user (290) and/or a further vehicle (280) in sensor data (270).

4. Method (100) as claimed in any one of the preceding claims, wherein the presence tag (260) is received via a vehicle data bus (205) [TMMSCTl l ] .

5. Method (100) as claimed in any one of the preceding claims, wherein the sensor data (270) comprises second image data (271), and the control unit (225) comprises an image processing module (227) to process the second image data (271 ) to obtain the presence tag (260).

6. Method (100) as claimed in any one of the preceding claims, wherein method (100) comprises: - receiving (110’), from the control unit (225), a sensor timestamp (261 ) relating to a data acquisition time (t) of the sensor data (270) by the sensor device (226), and

- performing (130) anonymization of the image data (250) is dependent on the sensor timestamp (261 ) and on an image timestamp (251 ) relating to an image acquisition time (f) of the image data (250) by the camera device (215).

7. A telematics device (210) for a vehicle (200a), in particular utility vehicle (200b), configured to performing the method steps of the method (100) as claimed in any one of the preceding claims.

8. The telematics device (210) as claimed in claim 7, wherein the telematics device (210) is adapted to be connected to a control unit (225) of an advanced driver assistance system (220) via a vehicle data bus (205) [TMMSCT12 ] .

9. A vehicle (200a), in particular utility vehicle (200b), comprising the telematics device (210) as claimed in claim 7 or 8, a camera device (215) being adapted to be connected to the telematics device (210), an advanced driver assistance system (220) with a control unit (225) and a sensor device (226).

10. Computer program, comprising instructions which, when the program is executed by a processor, causes the processor to carry out the method (100) of any of claims