DE102022107010A1 - System and method for identifying objects - Google Patents

Abstract

The invention relates to a system for identifying objects. The system has at least one first neural network, in particular an autoencoder or transformer, preferably a sparse autoencoder or sparse transformer, for converting a query into at least one typed representation, preferably a plurality of typed representations of the query. The first neural network is trained in such a way that it converts a query into more abstract, typed representations of the query and, on the other hand, of the data sets representing the properties of the objects. In addition, the system has a second neural network for converting a query into at least one typed representation, preferably a plurality of typed representations, of the data sets with object properties. The second neural network is trained in such a way that it converts a respective data set with object properties into a more abstract, typified representation of the respective data set representing the properties of the objects.

Description

The invention relates to a system and method for identifying objects.

Objects, for example items, but also people are typically characterized by a large number of properties, at least some of which are physical in nature and measurable. Such characteristics may be size, color, age, weight, composition, or the like. Other properties that are not readily measurable, but can also belong to the attributes of an object, are, for example, the history of the object. An example of this can be the manufacturing history of a complex component that is successively manufactured in different stages from several basic elements and basic components to form a more complex component. Such a production history is also an attribute of an object and can be relevant, for example, in connection with quality control. For example, batch numbers for a product are regularly recorded in the chemical industry in order to be able to trace how the batch to which the product belongs was manufactured in the event of problems. One problem is that the large number of properties and attributes of different objects has typically not been recorded completely or in a uniform manner. For example, used automotive spare parts can be described both by the type of spare part, e.g. "tailgate" or by the manufacturer's part number. If only "tailgate" is specified, typically the vehicle to which the tailgate is to belong must be described in more detail. With regard to the chosen example, the color also often plays a role. This can, for example, be indicated roughly as light green or with the manufacturer's specification "spring green" or with a color code from the manufacturer. A light green tailgate may be a spring green tailgate. However, the latter is not certain if there are several light green shades for vehicles from the same manufacturer. Other properties of such an object are, for example, the equipment such as a closing sensor, rear window cleaning or a drive for opening and closing the tailgate.

If many objects of a similar or the same type are recorded in a database with their properties, the problem arises that the data recorded for a single object can be incomplete or inaccurate. Therefore, it is difficult to use an automated query to identify those objects that best match the query. A precisely formulated query does not necessarily lead to the identification of the objects that best match the query, because the description of the objects in the database may differ from the query with regard to the format of the information or the required accuracy. If, for example, the target objects that potentially best match a query are to be identified from a thousand objects without subsequently re-entering the object properties, a problem usually arises.

A query is typically composed of a plurality of attributes or object properties which objects to be selected should have if possible.

Ideally, the objects are described exactly by such attributes or values of such object properties as are also named in the query. However, the values of the recorded object properties are often incomplete (i.e. the object properties are incompletely recorded in the sense of the query) or have an incorrect format, e.g. because they are recorded by values in different units or on different scales or in different languages or terms.

Accordingly, it is desirable to capture the properties and attributes of objects as accurately and completely as possible. This requires precise specifications and, for example, precise measurement methods. The latter is always difficult to implement when the information on the properties of the objects comes from different sources - for example from different institutions or people or using different devices. After capturing the properties and attributes of objects, they must be represented in some form. It is usual to describe the properties and attributes of objects with words and numbers or other codes and to combine these representations of the properties (words, number codes, etc.) of a respective object into a data set. Also the data sets as a kind of container for these representations of the properties of a respective object can have different formats, just like the representations of the properties contained in them. For example, when it comes to properties that can be measured and expressed in numbers and associated units, such as size, weight or volume, the measurement can vary in accuracy, the representation in numbers and units can vary (number of decimal places, unit used) and their representation and formatting in a file can differ widely for two identical objects, even if the measurements and the representations are free of errors. When describing Eigen When dealing with words in natural language, the variety of possibilities for naming a specific property and representing it in data sets becomes even greater.

Recorded properties and attributes of objects can thus be converted into machine-readable descriptions. Usually these must be unique for a deterministic query. However, this is often not the case in practice.

The object of the invention is to offer a solution for the latter problem.

According to the invention, a system is proposed to solve this problem, which has at least a first neural network, in particular an autoencoder or transformer, preferably a sparse autoencoder or sparse transformer for converting a query into at least one typed representation, preferably a plurality of typed representations of the query. The first neural network is trained to transform a query into more abstract, typed representations of the query. In addition, the system has a second neural network for converting the data sets with object properties into at least one typed representation, preferably a plurality of typed representations, of the data sets with object properties. The second neural network is trained in such a way that it converts a respective data set with object properties into a more abstract, typified representation of the respective data set representing the properties of the objects.

It is particularly preferred if the respective neural network, ie in particular the (sparse) autoencoder or (sparse) transformer, is configured in such a way that it generates a plurality of typified representations of a query or of a respective data set with object properties. In the case of an autoencoder, this can be done, for example, by means of a varying drop-out in the individual hidden layers, or in the case of a transformer by decoupling the outputs of the individual decoders (see below).

In addition, the system has a comparison unit that is configured to compare the typed representation or the typed representations of a query (or multiple typed representations of a query) with the typed representations of the properties of the objects and to generate corresponding similarity values. A selection and output unit connected to the comparison unit is designed to identify those objects whose typed representations of their properties are most similar to the typed representation of the query. The comparison unit is preferably designed to determine the similarity of typified representations by determining the Euclidean distance. The respective Euclidean distance between one of the typified representations of the query and a typified representation of a respective data set with object properties is then a similarity value formed by the comparison unit. The typed representations of the object property datasets and the typed representations of the query may be in the form of tensors generated by the second and first neural networks, respectively. To this end, the first and the second neural network preferably have one or more output layers with an identical number of nodes.

Alternatively, the query's typed representations can be converted to queries in a query format that is directly applicable to the object property records. The comparison unit then determines the similarity between such transformed queries and the original object property records - rather than with typed representations of the object property records.

For such a transformation of the queries, in a first step they are converted into typed representations of the query as described here. The typed representations of the query obtained in this way are then - in a second step - mapped to an interpretable data representation of the typed query by means of a further decoder or by deterministic mapping and thus converted into a query format that can be applied directly to the data sets with object properties. Such an interpretable data representation can be implemented as a JSON file, for example. In this JSON file, the key terms (e.g. the names of the parameters such as "year of construction" etc.) are named in natural language.

The interpretable data representation of the typed query can then be applied directly to the object property records. By "interpretable" here is meant that the data representation is understandable to humans and can be read to the underlying query or data to be selected - and not in abstract form.

A first approach described here consists in converting both the queries and the data sets with object properties into typed representations, each of which has an abstract format. The typified representations tations of the queries can then be compared to the typed representations of the object property records to determine, through this comparison, those object property records that best match the particular query.

The second approach described here is to first convert only the queries into typed representations of the queries and then to convert the typed representations of the queries into interpretable data representations that are both understandable to humans and applied directly to the datasets with object properties can become. To determine the object property records that best match the query at hand, the interpretable data representations of the queries can then be compared to the object property records to identify the object property records that best match the query.

The quality of the mapping of the typed representations to interpretable data representations (the representation mapping) is preferably provided with a ranking in order to determine those interpretable data representations from among all the possible data representations that best match the respective typed representation. Since both the typified representations and the interpretable data representations can be represented as vectors composed of values for a large number of parameters, it is possible to determine a similarity measure between the vectors, e.g. their Euclidean distance, and to carry out the ranking based on the similarity measure .

A comparison unit similar to that described above can be provided for this ranking. Accordingly, a typified representation of a query can be compared with different interpretable data representations of queries, and the most suitable interpretable data representation can be determined and selected in each case.

If the parameters (e.g. age, size, year of construction, etc.) in the queries are not defined by a single value, but by a range of values, each query spans a multidimensional vector space. to which a center can be assigned, for example. In this case, the positions of the centers in the multidimensional vector space can be compared with each other. The ranking is then the higher, the closer the mean values derived from the typified representation of the query and from the interpretable data representation are in the multidimensional vector space.

The comparisons carried out on the basis of the typed representations of the queries or the corresponding interpretable data representations with the data sets with object properties or their typed representations can also be evaluated by means of a corresponding ranking. The same method as for ranking the representation mappings can be used for this. For this purpose, data sets with object properties are assigned to the respective query on the basis of vector spaces that represent a respective query. The more central a data set with object properties is in the vector space defined by the interpretable representation of the query, the higher the ranking.

In a respective vector space representing a query, each part of the query (e.g. each parameter defined in the query) represents a vector dimension. For example, material density, date of manufacture and condition each represent a vector dimension. This results in a three-dimensional parameter or vector space (the terms parameter space and vector space are used synonymously here). The values of the individual dimensions must be converted to a uniform scale. If each value is to have the same value for ranking, the highest ranking value is assigned to the middle of the vector space. However, weightings can also be assigned for the individual dimensions. In this case, the point of the highest ranking value is no longer in the middle of the vector space, but is shifted in the vector space.

The representations - be it the typified representations or the interpretable data representations - are stored in a structured data standard. This can be a JSON format, for example. A format according to the FHIR standard is also suitable for medical data. In addition to the representations of the parameters that define the query or the object properties, the representations can also contain values that represent the quality of the mapping of the resource

If a query or also a respective data set with object properties is converted into a plurality of typed representations, each typed representation of a data set with object properties can be compared with the various typed representations of the query. For example, with n different representations of a query and m representations of the Data sets with object properties result in a matrix of relative similarity values, with the matrix having the dimension nxm.

• - eine Eingabeschnittstelle zum Eingeben oder Empfangen einer Abfrage z.B. in Form eines Abfragedatensatzes,
• - ein mit der Eingabeschnittstelle verbundenes erstes neuronales Netz, das zum Umwandeln einer Abfrage in wenigstens eine typisierte Repräsentation, vorzugsweise mehrere typisierte Repräsentationen der Abfrage konfiguriert und trainiert ist,
• - einen Zugang zu einer Datenbank, die Datensätze mit Objekteigenschaften enthält,
• - ein mit dem Zugang zur Datenbank verbundenes zweites neuronales Netz, das zum Umwandeln eines jeweiligen Datensatzes mit Objekteigenschaften in wenigstens eine typisierte Repräsentation des Datensatzes mit Objekteigenschaften, vorzugsweise mehrere typisierte Repräsentationen des Datensatzes mit Objekteigenschaften konfiguriert und trainiert ist, wobei die Anzahl der Knoten der Ausgangsschicht oder der Ausgangsschichten des zweiten neuronalen Netzes der Anzahl der Knoten der Ausgangsschicht oder der Ausgangsschichten des ersten neuronalen Netzes entspricht, so dass die typisierten Repräsentationen der Anfrage und des jeweiligen Datensatzes mit Objekteigenschaften die gleiche Dimension haben,
• - eine Vergleichseinheit zum Bestimmen derÄhnlichkeiten von mittels des ersten neuronalen Netzes generierten typisierten Repräsentationen der Abfrage mit mittels des zweiten neuronalen Netzes generierten typisierten Repräsentationen der Datensätze mit Objekteigenschaften, sowie
• - eine Auswahl- und Ausgabeeinheit zum Identifizieren zu einer Abfrage passender Objekte auf Basis der von der Vergleichseinheit bestimmten Ähnlichkeiten und zum Anzeigen identifizierter passender Objekte.

A system according to the invention thus has the following components:

• - an input interface for entering or receiving a query, e.g. in the form of a query data record,
• - a first neural network connected to the input interface, which is configured and trained to convert a query into at least one typed representation, preferably a plurality of typed representations of the query,
• - access to a database containing data sets with object properties,
• - a second neural network connected to the access to the database, which is configured and trained to convert a respective data set with object properties into at least one typed representation of the data set with object properties, preferably several typed representations of the data set with object properties, the number of nodes of the output layer or the output layers of the second neural network corresponds to the number of nodes in the output layer or layers of the first neural network, so that the typified representations of the query and the respective data set with object properties have the same dimension,
• - a comparison unit for determining the similarities of typified representations of the query generated by means of the first neural network with typified representations of the data sets with object properties generated by means of the second neural network, and
• - a selection and output unit for identifying matching objects to a query on the basis of the similarities determined by the comparison unit and for displaying identified matching objects.

• - Erfassen einer Mehrzahl physikalisch messbarer Eigenschaften einer Anzahl M von Objekten oder Individuen, wobei jedes Objekt oder Individuum durch Werte der für dieses Objekt oder Individuum erfassten Mehrzahl physikalisch messbarer Eigenschaften individuell charakterisiert ist
• - Bilden einer der Anzahl m entsprechenden Anzahl von Datensätzen mit Objekteigenschaften, von denen jeder erfasste Werte für wenigstens einen Teil Mehrzahl physikalisch messbarer Eigenschaften zu dem jeweiligen Objekt oder Individuum enthält
• - Bilden einer Abfrage zum Auswählen einer Teilmenge der Objekte oder Individuen, wobei die Abfrage Werte für wenigstens einen Teil der Mehrzahl physikalisch messbarer Eigenschaften enthält
• - Transformieren der Abfrage mittels eines Autoencoders oder eines Transformers in eine Anzahl n typisierter Repräsentationen der Abfrage,
• - Transformieren eines jeden der m Datensätze mit Objekteigenschaften mittels eines Autoencoders oder eines Transformers in eine Anzahl m typisierter Repräsentationen der Datensätze mit Objekteigenschaften, und
• - Bestimmen eines Ähnlichkeitswerts für jeden der m typisierten Repräsentationen der Datensätze mit Objekteigenschaften durch Ermitteln der Ähnlichkeit zwischen einer typisierten Repräsentation der Abfrage mit einer jeweiligen der m typisierten Repräsentationen der Datensätze mit Objekteigenschaften.

A corresponding method according to the invention comprises the following steps:

• - detecting a plurality of physically measurable properties of a number M of objects or individuals, each object or individual being individually characterized by values of the plurality of physically measurable properties detected for this object or individual
• - Forming a number m corresponding to the number of data sets with object properties, each of which contains detected values for at least a part of a plurality of physically measurable properties of the respective object or individual
• - forming a query to select a subset of the objects or individuals, the query including values for at least a portion of the plurality of physically measurable properties
• - Transforming the query using an autoencoder or a transformer into a number n of typed representations of the query,
• - transforming each of the m object property data sets by means of an autoencoder or a transformer into a number m of typed representations of the object property data sets, and
• - determining a similarity value for each of the m typed representations of the object property data sets by determining the similarity between a typed representation of the query and a respective one of the m typed representations of the object property data sets.

This solution includes the idea that the problem mentioned can not only be solved by more precisely detecting the properties and attributes of the objects, but that it is also possible to achieve good results on the basis of imprecisely or incompletely detected properties and attributes in that the the data sets representing properties and attributes are converted into a more abstract, typed representation by means of an autoencoder or transformer and the queries are also first converted into a more abstract, typed representation, and then based on a comparison of the typed representation of the query with the typed representations of the objects to be able to identify one or more objects matching the query.

The system preferably has an autoencoder or a transformer as the first neural network, the input layer of which is supplied with a representation of a query.

The system preferably has an autoencoder or a transformer as the second neural network, to the input layer of which data sets representing properties of the objects are supplied.

The autoencoder or transformer forming the first neural network is trained with training data to convert a specific query into a plurality of typified representations of the query. The autoencoder or transformer forming the second neural network is trained with training data to convert a data set representing respective properties of the objects into a typified representation of the data set. This then allows the query's typed representations to be compared with each of the typed representations of the various object property datasets to identify, among the plurality of objects, those objects whose typed representations are most similar in properties to the query's typed representation.

Both the typed representation of the query and the typed representations of the properties of the objects can be represented as tensors generated by the autoencoder or transformer and provided to their respective output layer. The similarity of tensors representing the typed representation of the datasets with object properties to a tensor representing a typed representation of the query can be determined, for example, by determining the Euclidean distance between the tensor representing the typed query and the various tensors representing the typed properties of the objects. A desired number of objects can then be identified for which the Euclidean distance between the tensor describing the respective object and the tensor describing the typed query is the smallest.

Autoencoders typically have an encoder-decoder architecture with an input layer and an output layer and a plurality of hidden layers in between. As is known, each layer is composed of a large number of nodes, whereby the number of nodes of the input layer must match the representation of the query data set. A property of the autoencoder is that the number of nodes, starting from the input layer, first decreases layer by layer and finally increases again layer by layer, so that a bottleneck forms between the input layer and the output layer.

The presented method and the presented system solve the problem of identifying "suitable" objects/persons in relation to a query criterion, i.e. finding objects/persons that match a requirement profile, even if the query criterion or the data for the queried attributes do not match are uniform or incomplete because, for example, data records for the individual objects/persons from which the selection is to be made do not contain the requested attributes in full or in a uniform form.

Areas of application are, for example, finding suitable recycling material or finding suitable used spare parts, e.g. car spare parts, batteries or circuit boards with electronic components.

In these use cases, there is often the problem of incomplete or inaccurate data sets. For example, the color of a used spare part can be generally "black" or manufacturer-specific "night black" or a special color code "AB201"b. There is also often inaccurate or missing information about the time of manufacture (model year), inaccurate or incomplete information about the condition (e.g. current battery capacity) etc.

• 1: ein erfindungsgemäßes System zum Identifizieren von Objekten, die in einer Abfrage festgehaltene Objekteigenschaften haben;
• 2: einen Teil des Systems aus 1, bei dem mithilfe eines Autoencoders eine Abfrage mit Objekteigenschaften in eine typisierte Repräsentation der Abfrage überführt wird;
• 3: einen Teil des Systems aus 1, bei dem mithilfe eines Autoencoders Datensätze mit Objekteigenschaften in typisierte Repräsentationen dieser Objekteigenschaften überführt werden können;
• 4: ein Beispiel, bei dem der Autoencoder zum Überführen einer Abfrage oder von Datensätzen mit Objekteigenschaften in entsprechende typisierte Repräsentationen ein Sparse Autoencoder ist;
• 5: ein Beispiel, bei dem der Autoencoder zum Überführen von Abfragen oder Datensätzen mit Objekteigenschaften in entsprechende typisierte Repräsentationen nur der Encoder-Teil eines Autoencoders ist;
• 6: ein Beispiel, bei dem der Autoencoder zum Überführen einer Abfrage oder von Datensätzen mit Objekteigenschaften in entsprechende typisierte Repräsentationen einen Multiheadautoencoder ist, bei dem ein Encoder eines Autoencoders mit verschiedenen Decodern verknüpft ist;
• 7: ein Beispiel für einen Autoencoder zum Überführen von Datensätzen mit Objekteigenschaften in entsprechende typisierte Repräsentationen der Objekteigenschaften mittels dem Encoder-Teil eines Autoencoders;
• 8: eine schematische Skizze einer Vergleichseinheit und einer Auswahl- und Ausgabeeinheit des Systems aus 1;
• 9: eine Skizze eines dreidimensionalen Parameterraums - aufgespannt durch die Parameterwerte einer Abfrage - und darin auf Basis der Objekteigenschaften verortete Objekte; und
• 10: ein Beispiel für einen Transformer zum Überführen von Abfragen oder von Datensätzen mit Objekteigenschaften in mehrere typisierte Repräsentationen der Abfrage oder eines jeweiligen Datensatzes mit Objekteigenschaften; und
• 11: eine andere Repräsentation eines Transformers.

The invention will now be explained in more detail using exemplary embodiments with reference to the figures. From the figures shows

• 1 : a system according to the invention for identifying objects that have object properties recorded in a query;
• 2 : part of the system off 1 , in which a query with object properties is converted into a typed representation of the query using an autoencoder;
• 3 : part of the system off 1 , in which an autoencoder can be used to convert data sets with object properties into typed representations of these object properties;
• 4 : an example where the autoencoder for transforming a query or records with object properties into corresponding typed representations is a sparse autoencoder;
• 5 : an example where the autoencoder for transforming queries or records with object properties into corresponding typed representations is just the encoder part of an autoencoder;
• 6 : an example where the autoencoder for converting a query or records with object properties into corresponding typed representations is a multi-head autoencoder, where an encoder of an autoencoder is associated with different decoders;
• 7 : an example of an autoencoder for converting data sets with object properties into corresponding typed representations of the object properties using the encoder part of an autoencoder;
• 8th : a schematic sketch of a comparison unit and a selection and output unit of the system 1 ;
• 9 : a sketch of a three-dimensional parameter space - spanned by the parameter values of a query - and objects located therein on the basis of the object properties; and
• 10 : an example of a transformer for transforming queries or object property datasets into multiple typed representations of the query or a respective object property dataset; and
• 11 : another representation of a Transformer.

A system for identifying objects that match a query includes an input interface 12 for a query, a first autoencoder or transformer 14 for converting the query into a typed representation 16 of the query and, in parallel, a database 18 with data records that represent object properties, a second Autoencoder or transformer 20 for converting the data sets with object properties into typed representations 22 of the data sets with object properties and a comparison unit 24 for determining the similarity of typed representations 16 of the query with typed representations 22 of the object properties, and a selection and output unit 26 for displaying identified matching ones objects; please refer 1 .

An autoencoder suitable for converting a query into a typed representation of the query typically has an encoder-decoder structure formed by nodes 30 organized in layers. These layers include an input layer 32, several hidden layers 34, and an output layer 36; please refer 2 . The nodes 30 of successive layers are connected to one another, so that a node 30 of a subsequent layer is supplied with the output values of some or all of the nodes 30 of a preceding layer. The output values of the nodes of a previous layer thus form the input values for a node in a subsequent layer. A node is designed to process the input values in a weighted manner, for example to add them together in order to form an output value in this way, which is then forwarded to the nodes of the subsequent layer.

An autoencoder has the property that the number of nodes in the layers that follow the input layer initially decreases. This portion of the autoencoder forms an encoder 38. At least a middle layer of the autoencoder 14 is a layer with the fewest number of nodes and forms a bottleneck 40 Nodes of the input layer 32 of the autoencoder 14 are given. In a decoder part 42 the number of nodes per layer increases again successively up to the output layer 36 .

An autoencoder is a neural network whose properties are determined by training. During training, the autoencoder is supplied with a large number of training data. The training data is first converted into more abstract presentations by the encoder of the autoencoder and then reconstructed as precisely as possible by the decoder part of the autoencoder. A reconstruction loss between the data that is given to the input layer of the autoencoder and the data that the autoencoder outputs to the output layer should be as small as possible. Due to the training with a large number of data sets, which, for example, represent different queries, weights arise in the nodes due to the requirement to keep the reconstruction loss as low as possible, which lead to heavily scattering parts of the training data sets given on the input layer are weighted less than those components of the training data sets that scatter less. This gives the autoencoder the ability to convert any query into a typed representation of the query during later operation. This typed representation of the query can be completely abstract and hardly understandable for a user. However, the decoder typically generates a typed representation of a query that is understandable to a user and has been cleaned of random inaccuracies, for example.

In order to avoid overfitting of the trained autoencoder 14, the autoencoder 14 is varied during training, e.g. by randomly deactivating (dropping out) some nodes of the hidden layers 34 during training.

Instead of a conventional autoencoder 14 or 20, a sparse autoencoder 50 can be used to generate the typed representations of either the query or the data sets with object properties, as exemplified in 3 is shown. Compared to the autoencoder 14 or 20, the sparse autoencoder 50 lacks the charac teristic bottleneck between encoder 38 and decoder 42. Rather, with a sparse autoencoder 50, all layers (layer) contain the same number of nodes, ie both the input layer 32 and the output layer 36 and the hidden layers (hidden layer) 34 lying in between have the same number of nodes 30. In order to nevertheless achieve the desired effect, for example for a specific query given to the input layer 32 of the sparse autoencoder 50 to generate a typified representation in which arbitrary features are weighted less than central features, some nodes 30 ' of the hidden layers 34' disabled. To this end, the sparse autoencoder 50 can have additional bias nodes for each layer. It should be noted that the structure (topology) of the sparse autoencoder 50 can be similar to that of the autoencoder 14. That is, even with the sparse autoencoder 50, hidden layers 34' can have fewer nodes 30 than the input layer 32 and the output layer 36.

It should be noted that by disabling individual nodes in a sparse autoencoder, a query can be transformed into different typed representations, as is the case in 3 is shown.

Instead of providing a full autoencoder 14 or 20 or a sparse autoencoder 50 to generate the typed representations of the query and/or the object property records, only the encoder 38 of an autoencoder including the bottleneck 40 can be used. The nodes of the bottle neck 40 then already represent the output layer. Such an example is shown in 4 shown. With such a neural network, a query or data sets with object properties can be converted into feature vectors 56, which likewise represent typified representations, for example of a query or a respective one with object properties. The example in 4 shows the conversion of a query into a plurality of feature vectors by means of the encoder 38. Similarly, data sets with object properties can also be converted into corresponding feature vectors. This is not shown.

In 5 It is illustrated that the autoencoder 18′ can also have a multihead architecture in which an encoder 38 is connected to a large number of decoders 42 which are each trained differently. In this way it is also possible, for example, to convert a query or also data sets with object properties into a plurality of typed representations.

6 and 7 illustrate how each object property record is selected from a plurality of object property records using an autoencoder 20 ( 6 ) can be converted into typed representations of these datasets with object properties. 7 illustrated analogously to 4 that each data set with object properties from a plurality of data sets with object properties can be converted by means of the encoder part 38 of an autoencoder into feature vectors which represent a typed representation of the respective data set with object properties.

After, on the one hand, a query has been converted into one or more typed representations and, on the other hand, the data sets with object properties have each been converted into one or more typed representations, the similarities between the typed representation(s) of the query and the typed representation(s) of the respective data sets with object properties to be determined. This is done using the in 1 comparison unit 24 shown. 8th shows the comparison unit 24 in a more detailed representation.

After determining the similarities between, for example, a representation of the query and the representations of the object property records, those associated objects that are most similar in object properties to the query can be identified. These objects can then be displayed via an output 26.

What is decisive is that this identification of objects with the system 10 described here also works if the data sets with object properties for the various objects are not uniform, in particular not complete, or the recorded object properties, in particular the recorded physical parameters, are represented imprecisely or deviate from one another . This allows the data sets with object properties to come from different, not strictly coordinated sources. Nevertheless, a meaningful query to identify suitable objects is possible. This allows suitable objects to be identified automatically without the need for time-consuming manual maintenance of the data records and without a detailed definition of the contents of a query.

The comparison unit 24 is designed to determine a similarity value for each typified representation of a query, in particular the Euclidean distance to the typified representations of each of the data sets with object properties. From the values of similarity thus determined ten, the comparison unit 24 generates a similarity matrix 60 with the dimension m×n, where m is the number of typed representations of the query and n is the number of typed representations of the data sets with object properties. The similarity matrix 60 can then be used to identify those objects whose associated typed representations of the data sets with object properties have the greatest similarity, in particular the smallest Euclidean distance, to the typed representations of the query.

1. 1. Es wird nur der höchste Wert der Ähnlichkeitsmatrix 60 herangezogen
2. 2. Der Mittelwert oder Median einer Zeile oder Spalte der Ähnlichkeitsmatrix 60 wird herangezogen
3. 3. Der Mittelwert oder Median der Top-N Ähnlichkeitswerte (beispielsweise der 5 höchsten Werte) wird verwendet
4. 4. Der Mittelwert oder Median der gesamten Matrix wird verwendet.

Finally, the selection and output unit 26 is provided for identifying objects that match the query. This is designed to form a representation value from the similarity matrix 60 on the basis of the similarity matrix 60 . For example, this can be done in one of four ways:

1. 1. Only the highest value of the similarity matrix 60 is used
2. 2. The mean or median of a row or column of the similarity matrix 60 is taken
3. 3. The mean or median of the top-N similarity values (e.g. the 5 highest values) is used
4. 4. The mean or median of the entire matrix is used.

This representation value is checked against a threshold set in the system and if the representation value is above or equal to the threshold, the record in question and the object (or person) associated with it are identified as matching the query and a list of matching objects is provided at the output.

The comparisons carried out on the basis of the typed representations of the queries or the corresponding interpretable data representations with the data sets with object properties or their typed representations can also be evaluated using a corresponding ranking. For this purpose, data sets with object properties are assigned to the respective query on the basis of vector spaces that represent a respective query. The more central a data set with object properties is in the vector space defined by the interpretable representation of the query, the higher the ranking. This is an example in 9 for a three-dimensional parameter or vector space. Such a three-dimensional parameter or vector space results when the individual objects are each characterized by three parameters such as age, size and condition. Each object is then represented by a three-dimensional feature vector, which can be represented by a point in the three-dimensional parameter or vector space. The as a cuboid in 10 The three-dimensional parameter space or vector space shown can represent a query, for example. Some objects fall within the three-dimensional parameter or vector space defined by the query, while other objects do not match the query and fall outside the three-dimensional parameter or vector space defined by the query.

In a respective vector space representing a query, each part of the query (e.g. each parameter defined in the query) represents a vector dimension. For example, material density, date of manufacture and condition each represent a vector dimension. This results in a three-dimensional parameter space. The values of the individual dimensions must be converted to a uniform scale. If each value is to have the same value for ranking, the highest ranking value is assigned to the middle of the vector space. However, weightings can also be assigned for the individual dimensions. In this case, the point of the highest ranking value is no longer in the middle of the vector space, but is shifted in the vector space.

As explained at the outset, both the first neural network for generating typed representations of a query and the second neural network for generating typed representations of the respective data sets with object properties can be transformers, in particular sparse transformers. 10 shows such a transformer used in the system according to FIG 1 can be provided instead of the autoencoder 14 and 20 shown there as an example.

The Transformer 90 also has an encoder-decoder architecture. In fact, a plurality of encoders 92 and a plurality of decoders 94 are provided. As is fundamentally known to a person skilled in the art, each encoder 92 of a transformer has a self-attention layer 96 and a feedforward layer 98 in each case. Each decoder 94 also has a self-attention layer 100 and a feedforward layer 102 . An encoder-decoder attention layer 104 is provided between the self-attention layer 100 of a decoder 94 and its feedforward layer 102 .

The input layer of the first encoder 92 is formed by an embedding layer 106 . For a respective query is therefore first of all an embedding of the query components (input embedding) and a coding of the position (position encoding). An input data set formed in this way is then passed through the various encoders 92 and processed. The output value of the last encoder 92 is then given to all encoder-decoder attention layers 104 of all decoders 94 .

In known transformers, the outputs of all decoders are processed into a single output tensor, for example using a softmax function. Deviating from this, in order to obtain different typed representations of a query or a respective data set with object properties, the output tensors generated by each individual decoder 94 are taken out of neural network 14 or 20 as different typed representations of a query and fed to comparison unit 26. For this reason, in the schematic representation of a transformer in 9 the layers following the last decoder 94 are not further shown.

With regard to the modeling, i.e. the training of the neural networks 14 and 20, it should be noted that the neural network 14 used to generate typed representations of the query and the neural network 20 used to generate the typed representations of the data sets with object properties initially trained with identical training data sets can become. The neural network 20 can then be retrained to form the typified representations of the data sets with object properties.

Preferably, the first neural network 14 and the second neural network 20 have a similar or identical topology. If the topology of the two neural networks 14 and 20 is identical, the two neural networks 14 and 20 usually still differ in the model they embody, which is represented by the weights generated in the individual nodes as a result of the training. It is known, for example, that each node 30 of a preceding layer passes on its output value to all nodes of the following layer (at least in the case of a fully connected neural network), so that a node of a subsequent layer receives the output values of all nodes of the preceding layer as input values become. These input values are weighted differently in the respective node, the different weights being the result of the training of the neural network and embodying a model together with the topology of the neural network (i.e. its structure from nodes, layers and connections).

11 shows another representation of a transformer.

Reference List

10

system

12

input interface

14

Autoencoder/Transformer

16

typed representation of the query

18, 18'

Database

20

Autoencoder/Transformer

22

typed representations of object properties

24

comparison unit

26

Selection and output unit

30, 30'

node

32

input layer

34, 34'

hidden layers

36

output layer

38

encoders

40

bottleneck

42

decoder

50

Sparse autoencoders

56

feature vectors

60

similarity matrix

92

encoders

94

decoder

96, 100

Self Attention Layer

98, 102

feedforward layer

104

Encoder Decoder Attention Layer

106

embedding layer

Claims (12)

Method with the steps: - Detecting a plurality of physically measurable properties of a number M of objects or individuals, each object or individual being individually characterized by values of the plurality of physically measurable properties detected for this object or individual - Forming a number m corresponding to the number of Data sets with object properties, each of which contains recorded values for at least a part of a plurality of physically measurable properties relating to the respective object or individual - forming a query for selecting a subset of the objects or individuals, the query containing values for at least part of the plurality of physically measurable properties - transforming the query using an autoencoder or a transformer into a number n of typed representations of the query, - transforming each the m data sets with object properties by means of an autoencoder or a transformer into a number of m typed representations of the data sets with object properties, and - determining a similarity value for each of the m typed representations of the data sets with object properties by determining the similarity between a typed representation of the query and a respective one of the m typed representations of the datasets with object properties. procedure after claim 1 , additionally comprising: setting up a similarity matrix containing the determined similarity values. procedure after claim 1 or 2 , wherein determining a similarity value for each of the m typed representations of the object property datasets by determining the similarity between a typed representation of the query and a respective one of the m typed representations of the object property datasets forming a value of the Euclidean distance between the selection feature vectors with the object feature vectors representing a respective data set. procedure after claim 3 , where the Euclidean distance is the respective similarity value. procedure after claim 2 and 4 , wherein the similarity matrix contains values for the Euclidean distance between the selection feature vectors and the object feature vectors representing a respective data set. procedure after claim 1 or 2 , wherein a similarity value is determined for each of the m typed representations of the data sets with object properties by forming a vector space representing the respective query and determining the position given by the object properties, which are defined by a respective data set with object properties, in the vector space representing the respective query . procedure after claim 6 where the proximity of the position given by the object properties to a center of the vector space representing the respective query is the respective similarity value. Procedure with the steps: - detecting a plurality of physically measurable properties of a number M of objects or individuals, each object or individual being individually characterized by values of the plurality of physically measurable properties detected for this object or individual - Forming a number m corresponding to the number of data sets with object properties, each of which contains detected values for at least a part of a plurality of physically measurable properties of the respective object or individual - forming a query to select a subset of the objects or individuals, the query including values for at least a portion of the plurality of physically measurable properties - Transforming the query using an autoencoder or a transformer into a number n of typed representations of the query, - transforming the typed representations of the query into interpretable data representations of the query, and - determining a similarity score for each of the object property records by determining the similarity between an interpretable data representation of the query and a respective one of the object property records. procedure according to claim 8 , characterized in that the typed representations of the query are mapped to an interpretable data representation of the typed query by means of a further decoder or by deterministic mapping and are thus converted into a query format that can be applied directly to the data sets with object properties. System for identifying objects, with - an input interface for entering or receiving a query, - a first neural network connected to the input interface, which is configured and trained to convert a query into at least one typed representation, preferably a plurality of typed representations of the query, - an access to a database containing data sets with object properties, - a second neural network connected to the access to the database, which is used to convert a respective data set with object properties into at least one typed representation of the data set with object properties, preferably a plurality of typed representations of the data set with object properties is configured and trained, the number of nodes of the output layer or layers of the second neural network being equal to the number of nodes of the out output layer or the output layers of the first neural network, so that the typified representations of the query and the respective data set with object properties have the same dimension, - a comparison unit for determining the similarities of typified representations of the query generated by means of the first neural network with by means of the second neural network generated typed representations of the data sets with object properties, and - a selection and output unit for identifying a query matching objects based on the similarities determined by the comparison unit and for displaying identified matching objects. system according to claim 10 , In which the first neural network (14) and the second neural network (20) are each designed as a sparse autoencoder. system according to claim 10 , In which the first neural network (14) and the second neural network (20) are each designed as a transformer (90), in particular as a sparse transformer.

Images