EP4256475A1

EP4256475A1 - Comparing a first artificial neural network with a second artificial neural network

Info

Publication number: EP4256475A1
Application number: EP21815132.2A
Authority: EP
Inventors: Fabian Woitschek; Georg Schneider
Original assignee: ZF Friedrichshafen AG
Current assignee: ZF Friedrichshafen AG
Priority date: 2020-12-07
Filing date: 2021-11-15
Publication date: 2023-10-11
Also published as: DE102020215430A1; WO2022122307A1

Abstract

The invention relates to a method for comparing a first artificial neural network, hereinafter referred to as ANN, with a second ANN, the first ANN and the second ANN forming a first and a second mapping, respectively, of an input data space to a result data space, comprising the following steps: - specifying (S1) a first input data space for the first ANN; - specifying or estimating (S2) a first partition from subsets of the first input data space with multiple data points, wherein each data point is allocated a mapping result of the first ANN from the result data space, wherein a data point from the input data space together with an associated mapping result of the first ANN forms a first mapping pair in each case; - determining (S3) second mapping pairs of the second ANN, wherein each mapping pair contains a data point from the input data space and an associated mapping result from the result data space; - comparing (S4) the second mapping pairs with the first mapping pairs in order to assess the second ANN in respect of the similarity or difference thereof to or from the first ANN.

Description

Comparing a first ANN to a second ANN

FIELD OF THE INVENTION

The present invention relates to a method for comparing a first artificial intelligence with a second artificial intelligence.

TECHNICAL BACKGROUND

It is known to deceive an artificial intelligence by means of so-called adversarial attacks. In this case, input data for an artificial intelligence is determined, which, to put it simply, forms a borderline case, which means that a slight change in the input data is sufficient for the artificial intelligence to produce an incorrect result.

SUMMARY OF THE INVENTION

Against this background, the invention is based on the object of creating a comparison option for artificial intelligences or artificial neural networks.

Accordingly, it is provided:

- A method for comparing a first artificial neural network, hereinafter referred to as ANN, with a second ANN, the first ANN and the second ANN forming a first and a second mapping from an input data space to a result data space, with the following steps : specifying a first input data space to the first ANN; Predetermining or estimating a first partition from subsets of the first input data space with a plurality of data points, with each data point being assigned a mapping result of the first ANN from the result data space, with each subset together with an associated mapping result of the first ANN forming a first mapping pair; determining second mapping pairs of the second ANN, each mapping pair containing a data point from the input data space and an associated mapping result from the result data space; Comparing the second pairs of images with the first pairs of images in order to assess the second ANN with regard to its similarity or difference to the first ANN.

An artificial neural network (ANN) is in particular a network of networked artificial neurons that is simulated in a computer program. The artificial neurons are typically arranged on different layers. Usually, the artificial neural network comprises an input layer and an output layer (output layer), whose neuron output is the only one of the artificial neural network that is visible. Layers lying between the input layer and the output layer are typically referred to as hidden layers. Typically, an architecture or topology of an artificial neural network is first initiated and then trained in a training phase for a specific task or for a plurality of tasks in a training phase.

The term "topology of an ANN" includes all aspects relating to the structure of an ANN. This includes, for example, the number of neurons in the ANN, the allocation of the neurons to the individual layers of the ANN, the number of layers in an ANN, the networking of the neurons and the weighting of the networking.

The term "architecture of an ANN" means its initial topology, i.e. its topology before the ANN was trained.

The training of the artificial neural network typically includes changing a weight of a connection between two artificial neurons of the artificial neural network. The weight contains information about the strength of the consideration of an input of a neuron. The training of the artificial neural network can also develop new connections between artificial neurons, deleting existing connections between artificial neurons, adjusting threshold values of the artificial neurons and/or adding or deleting artificial neurons.

A partition (also decomposition) of a set M is a set P whose elements are nonempty subsets of M such that every element of M is contained in exactly one element of P. A is a subset of B if every element of A is also in B. An edge of a set is its boundary. The border may or may not be included in the set. A vicinity of a subset edge lies within the subset and is adjacent to the edge. The close range can be determined by a predetermined maximum distance from the edge. The subset boundary of a subset can also be referred to as the decision boundary of the associated ANN.

A mapping is a relationship between two sets that assigns each element of one set, in this patent application an input data space, to exactly one element of the other set, in this patent application a result data space. A mapping result is the result of a mapping for a data point from the input data space. A mapping pair is the pair of a data point from the input data space with its associated mapping result.

An input data space is a set of data that contains all conceivable or well-defined input data for an ANN. A result data space is a set that contains all conceivable result data of an ANN. This patent application is based on the assumption that an input data space can be partitioned into subsets or data points, with each element, ie a subset or a data point, of the partition being mapped to a different mapping result from the result data space. Accordingly, estimating a partition of the input data space corresponds to estimating decision boundaries.

An image sensor is, for example, a camera, a radar sensor or a lidar sensor that captures real image data. Image data generated by software-supported methods are artificial image data, such as frequency images, that contain a collection of pixels with specific properties. The properties can relate to the pixels themselves, such as their gray value, or to a relationship between the pixels, such as gradient or frequency.

Computer program products typically include a sequence of instructions that, when the program is loaded, cause the hardware to perform a specific method that leads to a specific result.

The basic idea of the invention is to examine the behavior of a first ANN and a second ANN for certain input data and to characterize the ANN comparatively based on a different or identical behavior of the first or second ANN with regard to the input data.

In the following, the aspects of the invention are explained clearly using a two-dimensional input data space. It goes without saying, however, that this example is primarily used for explanation and may have rather little relevance for practice.

An ANN defines a mapping from an (example two-dimensional) input data space to a result data space. The mapping depends significantly on the purpose of the artificial intelligence, for example it is conceivable to classify data points of the input data space with regard to their properties. In this case, a classification result, such as "tree" or "house", from the result data space is assigned to the data points from the input data space.

Alternatively, it is also conceivable that the result data space does not contain any discrete elements such as "house" or "tree", but is continuous. For example, it is conceivable that an ANN maps input data to a rational number.

Based on this, it can be seen that an input data space of an ANN can be broken down into a partition, so that each element of the partition can be assigned an element from the result data space. Based on these pairs of images, i.e. pairs from the input data space and the result data space, ANNs can be compared by comparing pairs of images of the second ANN with pairs of images or the image of the first ANN.

It goes without saying that the invention is applicable for the comparison of black box as well as gray box or white box ANN routines.

White box means that the entire ANN is known, i.e. its entire topology, and can be precisely observed during operation. It is thus possible for the user to understand the internal calculations and to be able to monitor them at any time. This applies, for example, to a specially developed ANN. However, this is not possible with black-box systems. The user can only observe the input data and the output of the ANN on this data. It is not possible for him to understand the calculation in the ANN and he has no information about the topology of the ANN. This applies, for example, to a purchased ANN.

Accordingly, the following scenarios are conceivable: a second ANN with an unknown partition of the input data space that is to be compared with a first ANN, the first ANN being able to have both a known and an unknown partition of the input data space. If the first partition of the first ANN is unknown, mapping pairs can be determined both for the first and for the second ANN and the mapping pairs can be compared with one another. If, on the other hand, the partition of the first ANN is known, i.e. it is a matter of white-box artificial intelligence, then it is sufficient to compare the mapping pairs of the second ANN with the first partition in connection with the mapping of the first ANN.

If an ANN has an unknown partition of the input data space, this means that there is input data for which the associated mapping result of the ANN can only be determined by the ANN. Advantageous refinements and developments result from the further dependent claims and from the description with reference to the figures of the drawing.

According to a preferred development of the invention, the comparison is used to assess the similarity between the first training data, which was used to train the first ANN, and the second training data, which was used to train the second ANN.

Training data significantly influence the behavior of an ANN. For example, the identity of the first and second training data can only be assumed if the comparison of the pairs of images of the second ANN with the pairs of images of the first ANN produces no or only minor differences.

According to a preferred development of the invention, the comparison is used to assess the similarity between a first architecture of the first ANN and a second architecture of the second ANN.

A similarity of the first and second architecture can be considered if the comparison of the mapping pairs of the second ANN with the mapping pairs of the first ANN produces no differences or only minor differences or if the result of the comparison shows certain similarities and differences between the mapping pairs of the second ANN and the mapping pairs of the first ANN produces, which are to be defined application-specifically with regard to the ANN to be compared.

According to a preferred development of the invention, pairs of images with data points from the input data space, which are located within a predetermined vicinity of a subset edge of a subset of the first partition, are specified.

For a two-dimensional input data space, this aspect can be illustrated by adding a region close to the partition to each subset of the partition Subset edge is set. In the following, the behavior of the ANN on data points contained in the close range is examined. i.e. it is examined whether the first ANN supplies different result data for input data from the specified close range than the second artificial intelligence.

It is also expedient if a minimum disturbance is determined, which is to be linked to the data point from the input data space, so that the mapping result from the result data space associated with the disturbed data point from the input data space differs from the associated mapping result of the undisturbed data point. It goes without saying that it is advantageous to determine minimum disturbances for a large number of data points.

This means that for at least two data points in the input data space, each located in a different area close to a subset edge of the first partition, it is examined whether the result of the first ANN differs from the result of the second ANN. The fact that the input data lie in a different vicinity of a subset edge in each case implies that the first ANN determines a different result for the input data with and without interference. In order to characterize the first ANN as precisely as possible, it is advantageous to minimize the interference between the undisturbed and disturbed input data, so that the first ANN (just about) determines a different result for the disturbed and undisturbed input data. In the following, the first ANN can be compared with the second ANN to determine whether the second ANN also determines a different result for the input data with and without interference. In this case, it could be concluded that the first partition is locally identical or at least locally similar to the second partition.

According to a preferred development of the invention, the input data space contains data that was captured by an image sensor. This can be image data, radar data and/or lidar data, for example, which serve as input data for an ANN. Alternatively, it is also conceivable that the input data space contains image data that was generated using software-supported methods, ie was not recorded by a sensor. These include, for example, pixel values with specific predetermined properties, for example in terms of a frequency or a gradient.

A computer program product according to a method of an embodiment of the invention performs the steps of a method as described above when the computer program product runs on a computer, in particular an in-vehicle computer. When the program in question is used on a computer, the computer program product produces an effect, namely a characterizing comparison of two ANNs.

It goes without saying that a processing unit that is set up to be loaded with the first or second ANN and/or to be coupled to a processing unit that executes the first or second ANN, the processing unit being set up to carry out the method as above has been described to carry out is advantageous.

CONTENTS OF THE DRAWINGS

The present invention is explained in more detail below with reference to the exemplary embodiments given in the schematic figures of the drawings. They show:

Figure 1 is a schematic block diagram of an embodiment of the invention.

FIG. 2 shows a basic sketch for an exemplary explanation of an embodiment of the invention;

FIG. 3 shows a basic sketch for an exemplary explanation of an embodiment of the invention. The accompanying drawings are provided to provide a further understanding of embodiments of the invention. They illustrate embodiments and, together with the description, serve to explain principles and concepts of the invention. Other embodiments and many of the foregoing advantages will become apparent by reference to the drawings. The elements of the drawings are not necessarily shown to scale with respect to one another.

In the figures of the drawings, elements, features and components that are identical, have the same function and have the same effect--unless stated otherwise--are each provided with the same reference symbols.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

FIG. 1 shows a method for comparing a first ANN with a second ANN, the first ANN and the second ANN each forming a first and a second mapping from an input data space to a result data space, with steps S1 to S4.

In step S1, a first input data space is specified for the first ANN.

In step S2, a partition is specified or estimated from subsets of the first input data space, in particular calculated, with each subset of the first partition being assigned a mapping result of the first ANN from the result data space. If a partition is calculated, this means that it is determined by an estimate.

In step S3, mapping pairs of the second ANN are determined, each mapping pair containing a data point from the input data space and an associated mapping result from the result data space.

In step S4, the mapping pairs of the second ANN are compared with the first partition to the second ANN in terms of their similarity or to assess the difference to the first ANN.

FIG. 2 shows a basic sketch to illustrate an embodiment of the invention. An input data space 19 which is divided into a partition 13 is shown in FIG. The partition 13 comprises three subsets 15, 16 and 17. The subsets 15, 16 and 17 are delimited from one another by a subset edge 11. A region 12 close to the edge 11 of the subset is shown in an illustrated manner within the subset 15 . Although the subset edge 12 is only shown for the subset 15, it goes without saying that the subsets 16 and 17 also have a comparable subset edge within these subsets. The subset 15 contains a plurality of data points 1 that are located within the subset 15 at a distance from the edge 12 of the subset. Furthermore, the subset 15 includes data points 2 that are located within the subset 15 and within the subset boundary 12 .

Subset 16 includes multiple data points 3. Subset 17 includes multiple data points 4.

An ANN that processes data from the input data space 19 can, for example, be in the form of a classifying ANN. Accordingly, an ANN would assign an object “house” to data points 1 and 2 of subset 15, an object “tree” to data points 3 of subset 16 and an object “vehicle” to data points 4 of subset 17. Typically, it can be assumed that in practical application, the ANN is generally fed with input data 1 , 3 or 4 that clearly lie within a subset of the partition 13 . In order to characterize the ANN, however, borderline cases, ie data points in a close range 12 at the subset edge 11, are also important. It can therefore be provided that data points 2 that are in the vicinity of a subset edge 11 are determined in a targeted manner and that the artificial neural network is characterized on the basis of these data points 2 .

It is conceivable that a user or developer of an ANN will ask about specific differences between two similar ANNs. So exist to one Input data space 19 generally implements a large number of ANNs or at least a large number of options for an ANN which is fed with data from the input data space 19 .

In practice, it is often the case for two similar ANNs that the ANNs for data points 3 and 4, which are clearly within a subset of a partition 13 of the input data space, i.e. have a sufficient distance from a subset edge 11, determine an identical result.

If two ANNs also determine identical results for data points 2 that are in the vicinity of a subset edge 11, it can be assumed that the ANNs are almost identical and were trained with almost identical training data.

However, if the ANN are different or were trained with different training data, it can be assumed that the ANN will not deliver an identical result for all input data 2 that is within a close range 12 of a subset limit 11 .

Figure 3 schematically illustrates another embodiment of the invention. FIG. 3 shows a detailed view of the partition 13 according to FIG. 2. In FIG. A disturbance 5 was determined for data point 2, which can be added to data point 2, as a result of which data point 2 is shifted from subset 15 to subset 16. Accordingly, the disturbance 5 transfers the data point 2 into the data point 3 within the subset 16. It can be provided that the disturbance 5 is minimized, i.e. that the disturbance 5 is just large enough that the data point 2 in the subset 15 is just in the subset 16 is shifted. Accordingly, the data point 3 in FIG. 3 would also be located very close to the edge 11 of the subset. It is conceivable to determine such faults 5 using a software-supported method. It can be provided that a disruption shifts a data point within one subset to another specific or undefined subset. The knowledge of data points 2 within a close range 12 and of disturbed data points 3 is helpful for the characterizing comparison of ANN. If several data points 2 or a large number of data points 2, which are located in different subsets within a subset boundary, are known, then a partition 13 of an input data space 19 can be estimated from this.

If at least the partition of an input data space of an ANN is known, then the ANN can be fully characterized by the partition if it is also known which subsets are mapped onto which mapping result. In practice, however, the partition should at best be known for the first ANN and not for the second ANN. The unknown of the partition of an ANN can, for example, be due to the fact that the ANN is a statistical black-box ANN, to the fact that another provider of the ANN does not disclose any information about the structure of the ANN, or to the fact that it is not possible to to determine the exact partition. In reality, it is difficult to calculate the partition exactly, even for a known system. However, it may be necessary to compare two ANNs with an unknown partition.

Reference symbol -S4 method steps data points disturbance subset border partition -17 subsets input data space

Claims

patent claims

1 . Method for comparing a first artificial neural network, hereinafter referred to as ANN, with a second ANN, the first ANN and the second ANN respectively forming a first and a second mapping from an input data space (19) onto a result data space, with the following steps:

- Specification (S1) of a first input data space for the first ANN;

- Predetermining or estimating (S2) a first partition (13) from subsets (15-17) of the first input data space (19) with a plurality of data points, with each subset being assigned a mapping result of the first ANN from the result data space, with a subset in each case from the input data space together with an associated mapping result of the first ANN form a first mapping pair;

- determining (S3) second pairs of images of the second ANN, each pair of images containing a data point from the input data space (19) and an associated image result from the result data space;

- Comparing (S4) the second pairs of images with the first pairs of images in order to assess the second ANN with regard to its similarity or difference to the first ANN.

2. The method as claimed in claim 1, wherein the comparison is used to assess the similarity of the first training data, which was used to train the first ANN, and the second training data, which was used to train the second ANN.

3. The method according to claim 1, wherein the comparison is used to assess the similarity between a first architecture of the first ANN and a second architecture of the second ANN.

4. The method according to any one of the preceding claims, wherein first pairs of images with a data point from the input data space (19) which are within a predetermined vicinity of a subset boundary (1 1) of a subset (15; 16; 17) of the first partition (13). , to be specified.

5. The method according to claim 4, wherein a minimum disturbance (5) is determined, which is to be linked to the data point from the input data space, so that the mapping result associated with the disturbed data point from the input data space (19) differs from the associated mapping result of the undisturbed Data point from the input data space (19) differs.

6. The method according to any one of the preceding claims, wherein the input data space contains data that was captured by an image sensor and/or contains image data that were generated using software-supported methods.

7. Computer program product with program code means to carry out the method according to any one of the preceding claims.

8. Arithmetic logic unit that is set up to be loaded with the first and/or second ANN and/or to be coupled to a computing unit that executes the first or second ANN, wherein the arithmetic logic unit is set up to perform the method according to any one of the preceding claims 1 -6 to perform.