WO2021122339A1

WO2021122339A1 - Method and device for making a neural network more robust in relation to adversarial disruptions

Info

Publication number: WO2021122339A1
Application number: PCT/EP2020/085651
Authority: WO
Inventors: Peter Schlicht; Fabian HÜGER
Original assignee: Volkswagen Aktiengesellschaft
Priority date: 2019-12-17
Filing date: 2020-12-10
Publication date: 2021-06-24
Also published as: CN114787650A; DE102019219925A1; EP4078239A1

Abstract

The invention relates to a method for making a neural network (15) more robust in relation to adversarial disruptions, wherein detected sensor data (20) of at least one sensor (51) is supplied the neural network (15) as input data, wherein output data (18) of at least one layer (16-x,17-x) of the neural network (15) is replaced part by part by means of quilting, and wherein that output data (28) that has be replaced part by part is supplied to at least one following layer (16-x,17-x) of the neural network (15) as input data. The invention also relates to a device (1) for making a neural network (15) more robust in relation to adversarial disruptions, a method for operating an assistance system (200) for a vehicle (60) and an assistance system (200) for a vehicle (50), as well as a vehicle (50), a computer program and a data carrier signal.

Description

description

Method and device for robustizing a neural network against adversarial ones

Disruptions

The invention relates to a method and a device for robustizing a neural network against adversarial disturbances. The invention also relates to a method for operating an assistance system for a vehicle and an assistance system for a vehicle as well as a vehicle, a computer program and a data carrier signal.

Machine learning, for example based on neural networks, has great potential for use in modern driver assistance systems and automated vehicles. Functions based on deep neural networks process sensor data (e.g. from cameras, radar or lidar sensors) in order to derive relevant information from it. This information includes, for example, a type and a position of objects in the surroundings of the motor vehicle, a behavior of the objects or a road geometry or topology.

Among the neural networks, in particular convolutional neural networks (CNN) have proven to be particularly suitable for applications in image processing. Convolution networks gradually extract various high-quality features from input data (e.g. image data) in an unsupervised form. During a training phase, the convolution network independently develops feature maps based on filter channels that process the input data locally in order to derive local properties. These feature cards are then processed again by further filter channels, which derive more valuable feature cards from them. On the basis of this information compressed from the input data, the deep neural network finally derives its decision and makes it available as output data.

While convolution meshes outperform classic approaches in terms of functional accuracy, they also have disadvantages. For example, attacks based on adversarial disturbances in the sensor data / input data can result in a misclassification or an incorrect semantic segmentation or object detection taking place in spite of a semantically unchanged content in the recorded sensor data. From Chuan Guo et al. , Countering Adversarial Images Using Input Transformations, arXiv: 1711.00117v3 [cs.CV], Jan. 25, 2018, https://arxiv.org/pdf/1711.00117.pdf, is a quilting

Method for eliminating adversarial disturbances in image data is known.

The invention is based on the object of creating a method and a device for robustizing a neural network against adversarial disturbances.

The object is achieved according to the invention by a method with the features of claim 1 and a device with the features of claim 9. Advantageous refinements of the invention emerge from the subclaims.

In particular, a method for robustizing a neural network against adversarial disturbances is provided, with sensor data recorded from at least one sensor being fed to the neural network as input data, with output data of at least one layer of the neural network being replaced piece by piece by means of quilting, and the piece-by-piece replaced output data are fed as input data to at least one subsequent layer of the neural network.

Furthermore, in particular a device for robustizing a neural network against adversarial disturbances is created, comprising a computing device, wherein the computing device is set up to provide a neural network or to access a provided neural network, to receive detected sensor data of at least one sensor and to the neural network as To supply input data, to replace output data at least one layer of the neural network piece by piece by means of quilting, and to supply the piece-by-piece replaced output data as input data to at least one subsequent layer of the neural network.

The method and the device make it possible to eliminate or at least reduce an effect of adverse disturbances potentially contained in the acquired sensor data on a final output of the neural network. The neural network or a function of the neural network is thereby robustized against adversarial disturbances. For this purpose, the output data of at least one layer of the neural network is replaced piece by piece by means of quilting. The layer is, in particular, an internal layer of the neural network. The output data, which are replaced piece by piece, are fed to at least one subsequent layer of the neural network. An effect of Adversarial disturbances on a data flow within the neural network can hereby be eliminated or at least reduced, so that an effect on the final output of the neural network is eliminated or at least reduced.

An advantage of the method and the device is that the robustification is in particular independent of a specific embodiment of an adverse disturbance contained in the sensor data, since the piece-wise replacement by means of quilting is independent of the presence of an adverse disturbance and, in particular, independent of a type of adverse disturbance . Another advantage is that the method and the device can be easily integrated into existing neural networks or KI functions provided with them. A structure of the neural network does not have to be adapted, only the output data of the at least one layer of the neural network have to be replaced or exchanged piece by piece by means of quilting. Furthermore, a training phase of the neural network in particular does not have to be adapted or repeated. In this way, effort and costs can be saved despite the robustification achieved.

The output data include, in particular, activations of the at least one layer of the neural network. The activations are in particular in the form of an activation map of the layer. The activation map can also be a feature map of a layer of the neural network designed as a convolution layer.

Quilting includes, in particular, the piece-wise replacement of output data, which can also be referred to as piece-wise reconstruction of the output data. For this purpose, the output data are divided into several sections. In the case of two-dimensional output data (for example in the form of activation cards), small, in particular rectangular sections (also referred to as patches) are defined for this purpose. The individual subsections are compared with subsections, hereinafter referred to as output data patches, which are stored in a database, for example. The comparison takes place on the basis of a distance measure which is defined, for example, via a Euclidean distance on picture element vectors. For this purpose, a partial section is linearized as a vector. A distance is then determined using a vector standard, for example using the L2 standard. For quilting, the partial sections are replaced by the closest or most similar output data patch from the database. It can be provided here that a minimum distance must be maintained or that at least no identity exists between the partial excerpt from the output data and the output data patch may. If the output data have a different form or a different format, the piece-by-piece replacement takes place in an analogous manner.

The at least one layer can in principle be any layer in the neural network. It can also be provided that output data from several layers of the neural network are replaced piece by piece by means of quilting. For the output data of each of the layers, in particular output data patches generated and provided individually for each layer are used. The at least one layer is in particular an internal layer of the neural network.

The sensor data of the at least one sensor can in principle be one-dimensional or multidimensional, in particular two-dimensional. For example, the sensor data can be two-dimensional camera images from a camera and / or two-dimensional or three-dimensional lidar data from a lidar or radar sensor. A sensor can in particular be a camera, a lidar sensor, a radar sensor, an ultrasonic sensor or some other sensor suitable for detecting the surroundings.

An adversarial perturbation is, in particular, a deliberately made disruption of the input data of a neural network, for example provided in the form of sensor data, in which semantic content in the input data is not changed, but the disruption leads to the neuronal Network inferred an incorrect result, that is, for example, a misclassification or an incorrect semantic segmentation of the input data or an incorrect detection or localization of objects in it.

A neural network is in particular a deep neural network, in particular a convolutional neural network (CNN). The neural network is trained for a specific perception function, for example, for the perception of pedestrians or other objects in captured camera images.

The method and the device are applied in particular to a (fully) trained neural network.

The method can be carried out as a computer-implemented method. In particular, the method can be carried out by means of a data processing device. The Data processing device comprises in particular at least one computing device and at least one storage device.

In particular, a computer program is also created, comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method steps of the disclosed method in accordance with any of the described embodiments.

In addition, a data carrier signal is also created that transmits the aforementioned computer program.

It can be provided that the method includes acquiring the sensor data by means of the at least one sensor.

It can also be provided that a final output of the neural network is fed to at least one control device, in particular a vehicle. The control device can, for example, provide a function for automated driving of a vehicle and / or for driver assistance of the vehicle and / or for environment detection and / or environment perception. The control device can, for example, control or regulate longitudinal and lateral guidance of the vehicle. In principle, however, the method and the device can also be used in other areas of application, for example in industrial production or in medical robots.

A vehicle is in particular a motor vehicle. In principle, however, a vehicle can also be another land, rail, water, air or space vehicle, for example a drone or an air taxi.

Parts of the device, in particular the computing device, can be designed individually or collectively as a combination of hardware and software, for example as program code that is executed on a microcontroller or microprocessor. However, it can also be provided that parts are designed individually or combined as an application-specific integrated circuit (ASIC).

In one embodiment it is provided that the neural network provides at least one function for automated or partially automated driving of a vehicle and / or for perception of the surroundings, with a final output of the neural network at least is fed to a control device of the vehicle. As a result, the control device of the vehicle is supplied with more reliable and safer input data, so that a function provided by the control device can also be provided more reliably and safely. The control device can, for example, provide an actuator control for an actuator of the vehicle or carry out (higher-value) further processing of the final output of the neural network.

In one embodiment it is provided that a database with output data patches is provided for quilting, the output data patches being or having been generated on the basis of output data of the at least one layer of the neural network which were obtained with the aid of undisturbed input data. As a result, adversarial disturbances in the output data of the at least one layer of the neural network can be safely removed without any content or semantic information contained in the output data being lost. In particular, the undisturbed input data are selected and compiled in such a way that they have no adverse disturbances with certainty or at least with an increased probability. The output data inferred from the at least one layer of the neural network on the basis of the undisturbed input data are then broken down into sub-sections, each sub-section forming an output data patch. A size of the partial sections is selected depending on a specific application scenario of the method and the device. The output data patches generated in this way are stored in the database. The database is provided, for example, by means of a storage device that can be accessed by the computing device. The output data patches can be stored in the database, for example, linearized as vectors. When the method is used or when quilting, output data that are provided by the at least one layer of the neural network are then also broken down into partial sections and linearized to form a vector. The respective vectors of the partial sections can then be compared with the vectors of the output data patches stored in the database by means of a vector standard, for example the L2 standard. A partial section of the output data is then replaced by the most similar or closest output data patch, based on the specific distances.

In one embodiment it is provided that during quilting the output data are processed in the form of two-dimensional image data, with image sections of the two-dimensional image data being replaced piece by piece by means of the quilting. In particular, activation maps (“activation maps”) are used, which represent the at least one layer of the neural network generated as output data, treated as two-dimensional image data or images. The piece-by-piece replacement carried out for quilting takes place via, in particular rectangular, image sections that are replaced. These image details can, for example, have a size of 8 × 8 picture elements (pixels). In this example, the output data patches also have a size of 8 × 8 picture elements each.

In one embodiment it is provided that at least one item of identification information is obtained, the piece-wise replacement during quilting additionally taking into account the at least one identification information item received. Identification information can also be referred to as a tag or label. In this way, for example, the entries in the database, that is to say output data patches stored therein, can be marked with additional information so that they can be found more quickly later. In particular, it can be provided that the database is indexed with the help of a hash function so that a search in the database can be accelerated, since a number of entries in the database is already reduced via a preselection before a comparison with the output data of the at least one layer of the neural network can be.

In a further developing embodiment, it is provided that the identification information obtained is or is derived from context information of an environment in which the sensor data of the at least one sensor is or was recorded. Context information can, for example, be a geographical coordinate (e.g. GPS coordinate), a time of day and / or season, a month, a weekday, weather (sun, rain, fog, snow, etc.) and / or a traffic context (city, country , Motorway, pedestrian zone, country road, main road, secondary road etc.). In this way, on the one hand, the quality of the output data that is replaced piece-by-piece can be increased, since in the case of piece-by-piece replacement, a context in which the output data were generated by the at least one layer of the neural network can be taken into account. In particular, output data patches can be marked (“tagged”) with at least one context information stored in the database. On the other hand, depending on the at least one received identification information or the at least one context information, a preselection can be made before the search in the database, so that only entries or output data patches that partially or completely match the at least one identification information are considered during the search or who have at least one context information. This means that the piece-by-piece replacement can be accelerated. If many feature cards with several channels are output in one layer of the neural network, the channels can be replaced piece by piece, depending on one another. All channels can either be treated together as one datum and replaced piece by piece, whereby this replacement can take into account the spatial correlation between the channels, or the channels can be replaced independently of one another, with each channel being broken down individually and reconstructed piece by piece using output data patches from its own database becomes.

In one embodiment it is provided that the neural network provides a function for the automated driving of a vehicle and / or for a driver assistance of the vehicle and / or for a surrounding area detection and / or surrounding area perception. A vehicle is in particular a motor vehicle. In principle, however, the vehicle can also be another land, air, water, rail or space vehicle. In principle, however, the method can also be used in other areas, for example in industrial production, e.g. in production robots that have to process sensor data, or in medical robots.

Features for the configuration of the device emerge from the description of configurations of the method. The advantages of the device are in each case the same as in the embodiments of the method.

A method for operating an assistance system for a vehicle is also provided, with at least one function for automated or partially automated driving of the vehicle and / or for perception of the surroundings being provided by means of the assistance system, with sensor data being recorded by means of at least one sensor, with one method is carried out according to one of the described embodiments, and wherein a final output of the neural network is fed to at least one control device of the vehicle. The control device can, for example, provide an actuator control for an actuator of the vehicle or carry out further processing of the final output of the neural network.

Furthermore, an assistance system for a vehicle is created, comprising at least one sensor set up to acquire sensor data, and a device according to one of the described embodiments, the assistance system being set up to perform at least one function for automated or partially automated driving of the vehicle and / or for Provide environment perception, the detected sensor data of the device and to feed a final output of the neural network provided by means of the device to at least one control device of the vehicle. The assistance system can also include the control device.

In addition, a vehicle is also created, in particular, comprising at least one device according to one of the described embodiments or at least one assistance system according to one of the described embodiments.

The invention is explained in more detail below on the basis of preferred exemplary embodiments with reference to the figures. Here show:

1 shows a schematic representation of an embodiment of the device for robustizing sensor data against adversarial disturbances and an embodiment of an assistance system;

2 shows a schematic representation to illustrate quilting (prior art);

3 shows a schematic representation to illustrate an embodiment of the

Method for the robustification of a neural network against adversarial disturbances.

1 shows a schematic representation of an embodiment of the device 1 for robustizing a neural network 15 against adversarial disturbances. The device 1 comprises a computing device 2 and a storage device 3. The device 1 carries out the method described in this disclosure for robustizing the neural network 15 against adversarial disturbances.

The device 1 can in particular be used in a vehicle 50 in order to robustize a neural network 15 or a function provided by the neural network 15 against adversarial disturbances. The vehicle 50 is in particular a motor vehicle. In particular, the vehicle 50 can include an assistance system 200, the assistance system 200 including the device 1. The assistance system 200 further comprises at least one sensor 51 and a control device 52. The embodiment is shown by way of example in connection with a motor vehicle. In principle, however, the method and the device 1 as well as the assistance system 200 can also be used in other vehicles 50.

The neural network 15 in particular provides at least one function for automated or partially automated driving of the vehicle 50 and / or for perception of the surroundings. In particular, the assistance system 200, in particular at least partially by means of the neural network 15, provides at least one function for automated or partially automated driving of the vehicle 50 and / or for perception of the surroundings.

Parts of the device 1, in particular the computing device 2, can be designed individually or collectively as a combination of hardware and software, for example as program code that is executed on a microcontroller or microprocessor.

The computing device 2 provides the neural network 15, that is to say it provides a functionality of the neural network 15 and in particular carries out the necessary arithmetic operations for this purpose. A structure and parameters of the neural network 15 are stored in the memory device 3, for example. Alternatively, the computing device 2 can only access a neural network 15 provided in some other way. Acquired sensor data 20 of sensor 51, for example a camera or a lidar sensor of vehicle 50, are fed to computing device 2.

The computing device 2 receives the acquired sensor data 20 and feeds them to the neural network 15 as input data. Furthermore, the computing device 2 replaces output data of at least one layer of the neural network 15 piece by piece by means of quilting. The output data replaced piece by piece are then fed as input data to at least one subsequent layer of the neural network 15.

A final output 30 of the neural network 15 is output by the computing device 2, for example in the form of a digital data packet. The final output 30 is fed, for example, to the control device 52 of the assistance system 200 or of the vehicle 50, which, for example, starting from the final output 30, controls or regulates a longitudinal and / or lateral guidance of the vehicle 50. After the quilting or after the piece-wise replacement, the output data replaced piece by piece have the same format as the (original) output data, so that it is possible to insert the method or the device 1 into already existing applications of neural networks 15 and use them.

In particular, it is provided that a database 40 with output data patches 60 is provided for quilting, the output data patches 60 being or having been generated on the basis of output data of the at least one layer of the neural network 15, which were obtained with the aid of undisturbed input data. The undisturbed input data are based, for example, on trustworthy training data with which the neural network 15 was trained.

Provision can be made for at least one item of identification information 10 to be obtained, the piece-wise replacement during quilting additionally taking into account the at least one identification information item 10 received. The identification information 10 is fed to the computing device 2, for example, and can be used to preselect output data patches 60 of the database 40 used in quilting, so that the quilting can be accelerated.

In a further development, it can be provided that the identification information 10 obtained is or is derived from context information 11 of an environment in which the sensor data 20 of the sensor 51 are or have been recorded. Context information can, for example, be a geographical coordinate (e.g. GPS coordinate), a time of day and / or season, a month, a weekday, weather (sun, rain, fog, snow, etc.) and / or a traffic context (city, country , Motorway, pedestrian zone, country road, main road, secondary road etc.).

The context information 11 can, for example, be recorded by means of a context sensor system (not shown) of the vehicle 50 set up for this purpose (e.g. rain sensor, temperature sensor, etc.), but can also be provided by a controller of the vehicle 50 and, for example, via a controller area network (CAN) Bus can be called up and / or provided (e.g. vehicle speed, vehicle orientation, etc.).

FIG. 2 shows a schematic representation to clarify quilting from the prior art using the example of a camera image 22. Sensor data 20, in the present case a camera image 22, are divided into partial sections 23. For each of the sub-sections 23 (ie Image section) of the camera image 22 is searched for a sensor data patch 61 in a database 40 as part of a quilting step 100, which has the smallest distance to the respective partial section 23 in terms of a distance. In the present case, a sensor data patch 61 is an image section which has the size of the partial sections 23, that is to say the same number of picture elements (pixels). The distance measure is, for example, the L2 standard, which is applied to vectors that have been generated by linearizing the partial sections 23 or image sections. In the quilting step 100, each partial section 23 or image section is then replaced by the respective sensor data patch 61 with the smallest distance therefrom. It can be provided here that a minimum distance must be maintained. In this way, all partial sections 23 or image sections are replaced by sensor data patches 61 from database 40. Replaced partial sections 24 are created which, taken together, form the replaced sensor data 21 or a replaced camera image 25.

FIG. 3 shows a schematic illustration to clarify an embodiment of the method for robustizing a neural network 15 against adversarial disturbances.

In the example shown, the neural network 15 comprises several convolutional layers 16-x and fully connected layers 17-x.

Sensor data 20 recorded as input data, for example a camera image 22, are fed to the neural network 15.

The last convolution layer 16-4 supplies an activation map 19 as output data 18, which can also be referred to as a feature map and in particular can be understood as images with many image channels (i.e. filter channels). The output data 18 or the activation card 19 are replaced piece by piece in a quilting step 100 by means of quilting.

The quilting takes place here in principle in the same way as was described in connection with FIG. 2, with the difference that it is not the sensor data 20 that is replaced piece by piece by means of quilting, but rather the output data 18. There is therefore an intervention in a data flow within the neural network 15. In particular, the quilting takes place in that partial sections 23 of the output data 18, that is to say image sections of the images of the Activation card 19 can be replaced by output data patches 60 which, with regard to a distance measure, have the smallest distance from the respective partial sections 23 or image sections. The output data patches 60 are stored in a database 40 and are retrieved from this. Since, in particular, several filter channels are included in the activation card 19, the quilting step 100 can also be referred to as multi-channel quilting.

After the piece-by-piece replacement by means of quilting, replaced partial sections 24 or replaced image sections are available as replaced output data 28 or replaced activation card 29. The replaced output data 28 or the replaced activation card 29 are fed to a subsequent layer 17-1 as input data. In the replaced output data 28 or the replaced activation card 29, an influence or an effect of an adverse disturbance is eliminated or at least reduced compared to the original output data 18 or the original activation card 19.

The neural network 15 provides a final output 30 which can contain, for example, object recognition or semantic segmentation on the acquired sensor data 20. In particular, it is provided that the neural network 15 provides a function for automated driving of a vehicle and / or for driver assistance of the vehicle and / or for environment detection and / or environment perception.

The final output 30 can, for example, be fed to a control device of the vehicle which, for example, controls or regulates a longitudinal and / or lateral guidance of the vehicle, plans a trajectory or carries out a higher-quality evaluation of the final output 30.

The method is shown by way of example for the output data 18 of a single layer 16-4 of the neural network 15. However, it can be provided that the method is carried out for further layers 16-x, 17-x. If the output data of several layers 16-x, 17-x of the neural network 15 are replaced piece by piece by means of quilting, output data patches 60 generated or provided individually for each of the layers 16-x, 17-x are used. For this purpose, the database 40 includes, in particular, output data patches 60 for each individual layer.

The method can be embedded in a method for operating an assistance system. LIST OF REFERENCE NUMERALS Device Computing device Storage device Identification information Context information Neural network -x Convolution layer -x Completely connected layer Output data Activation card Sensor data Replaced sensor data Camera image Partial section Replaced partial section Replaced camera image Replaced output data Replaced activation card Final output database Vehicle sensor control device Output data patches Sensor data patch 0 Quilting step 0 Assistance system

Claims

1. A method for robustizing a neural network (15) against adversarial disturbances, wherein the neural network (15) recorded sensor data (20) of at least one sensor (51) is supplied as input data, with output data (18) at least one layer (16-x , 17-x) of the neural network (15) are replaced piece by piece by means of quilting, and the output data (28) replaced piece by piece are fed as input data to at least one subsequent layer (16-x, 17-x) of the neural network (15).

2. The method according to claim 1, characterized in that the neural network (15) provides at least one function for automated or partially automated driving of a vehicle (50) and / or for perception of the surroundings, with a final output (30) of the neural network (15) at least a control device (52) of the vehicle (50) is supplied.

3. The method according to claim 1 or 2, characterized in that a database (40) with output data patches (60) is provided for quilting, the output data patches (60) based on output data (18) of the at least one layer (16-x, 17-x) of the neural network (15) are generated or were generated, which were obtained with the aid of undisturbed input data.

4. The method according to any one of the preceding claims, characterized in that during quilting the output data (18) are processed in the form of two-dimensional image data, with image sections of the two-dimensional image data being replaced piece by piece by means of the quilting.

5. The method according to any one of the preceding claims, characterized in that at least one piece of identification information (10) is obtained, the piece-wise replacement during quilting additionally taking into account the at least one received identification information (10).

6. The method according to claim 5, characterized in that the identification information (10) obtained is derived or derived from context information (11) of an environment in which the sensor data (20) of the at least one sensor (51) are or have been recorded is.

7. The method according to any one of the preceding claims, characterized in that the neural network (15) provides a function for automated driving of a vehicle and / or for driver assistance of the vehicle and / or for environment detection and / or environment perception.

8. A method for operating an assistance system (200) for a vehicle (50), with at least one function for automated or partially automated driving of the vehicle (50) and / or for perception of the surroundings being provided by means of the assistance system (200), with sensor data (20) by means of at least one sensor (51), wherein a method according to one of claims 1 to 7 is carried out, and wherein a final output (30) of the neural network (15) is fed to at least one control device (52) of the vehicle (50).

9. Device (1) for robustizing a neural network (15) against adversarial disturbances, comprising a computing device (2), wherein the computing device (2) is set up to provide a neural network (15) or to a provided neural network (15 ) to receive detected sensor data (20) of at least one sensor (51) and to supply the neural network (15) as input data, output data (18) of at least one layer (16-x, 17-x) of the neural network (15) by means of To replace quilting piece by piece, and to feed the piece-by-piece replaced output data (28) as input data to at least one subsequent layer (16-x, 17-x) of the neural network (15).

10. Assistance system (200) for a vehicle (50), comprising: at least one sensor (51), set up to acquire sensor data (20), and a device (1) according to claim 9, wherein the assistance system (200) is set up for this purpose to provide at least one function for automated or partially automated driving of the vehicle (50) and / or for perception of the surroundings, to supply the recorded sensor data (20) to the device (1), and a final output (30) of the data provided by means of the device (1) To feed the neural network (15) to at least one control device (52) of the vehicle (50).

11. The vehicle (50) comprising at least one device (1) according to claim 9 or at least one assistance system according to claim 10.

12. Computer program, comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method steps of the method according to any one of claims 1 to 8.

13. Data carrier signal which the computer program according to claim 12 transmits.