WO2023099481A1

WO2023099481A1 - System and method for identifying objects

Info

Publication number: WO2023099481A1
Application number: PCT/EP2022/083683
Authority: WO
Inventors: Sebastian NIEHAUS; Alberto Merola; Janis REINELT
Original assignee: Aicura Medical Gmbh
Priority date: 2021-11-30
Filing date: 2022-11-29
Publication date: 2023-06-08

Abstract

The invention relates to a system for identifying objects. The system comprises at least a first neural network, in particular an autoencoder or transformer, preferably a sparse autoencoder or sparse transformer for converting a query into at least one typed representation, preferably a plurality of typed representations of the query. The first neural network is trained such that a query is transformed into more abstract, typed representations of the query and the data sets representing the properties of the objects. In addition, the system comprises a second neural network for converting a query into at least one typed representation, preferably a plurality of typed representations, of the data sets having object properties. The second neural network is trained such that it transforms each data set having object properties into a more abstract, typed representation of each data set representing the properties of the objects.

Description

System and method for identifying objects

The invention relates to a system and method for identifying objects.

It is known to represent items in a database. In a database, the objects are characterized by their properties, i.e. the database typically contains entries for several properties of a respective object. With the help of a database, for example, spare parts in a spare parts store can be identified and found without having to search the store "by hand".

However, there is still a problem if, when searching for a spare part, it is not known which properties of a spare part are recorded in the database or how the properties are recorded and represented or what the structure of the database is.

If then, in addition, the warehouse in which the searched item or items may be located is not accessible to the person searching and the database itself is also inaccessible to a person searching. the search problem increases and can traditionally only be solved by an authorized, qualified employee on site. The aim is to solve this problem using technical means without granting a searcher access to the database, so that the security and integrity of the database is maintained in any case.

Objects, for example items, but also people are typically characterized by a large number of properties, at least some of which are physical in nature and measurable. Such characteristics may be size, color, age, weight, composition, or the like. Other properties that are not readily measurable, but can also belong to the attributes of an object, are, for example, the history of the object. An example of this can be the manufacturing history of a complex component that is successively manufactured in different stages from several basic elements and basic components to form a more complex component. Such a production history is also an attribute of an object and can be relevant, for example, in connection with quality control. For example, batch numbers for a product are regularly recorded in the chemical industry in order to be able to trace how the batch to which the product belongs was manufactured in the event of problems. One problem is that the large number of properties and attributes of different objects is typically not recorded completely or not uniformly. For example, used automotive spare parts can be described both by the type of spare part, e.g. "tailgate" or by the manufacturer's part number. If only "tailgate" is specified, typically the vehicle to which the tailgate is to belong must be described in more detail. With regard to the chosen example, the color also often plays a role. This can, for example, be indicated roughly as light green or with the manufacturer's specification "spring green" or with a color code from the manufacturer. A light green tailgate may be a spring green tailgate. However, the latter is not certain if there are several light green shades for vehicles from the same manufacturer. Other properties of such an object are, for example, the equipment such as a closing sensor, rear window cleaning or a drive for opening and closing the tailgate.

If many objects of a similar or the same type are recorded in a database with their properties, the problem arises that the data recorded for a single object can be incomplete or inaccurate. Therefore, it is difficult to use an automated query to identify those objects that best match the query. A precisely formulated query therefore does not necessarily lead to the identification of the objects that best match the query, because the description of the objects in the database may be in terms of the format of the information or the required Accuracy may vary from query. If, for example, the target objects that potentially best match a query are to be identified from a thousand objects without subsequently re-entering the object properties, a problem usually arises.

As mentioned at the beginning, the problem could be solved by means of a manual triage. For security, regulatory, or other confidentiality reasons, manual review may not be possible or desirable. If queries are carried out by third parties from a separate network or the Internet, this problem is exacerbated because the lack of manual screening means that the queries cannot be corrected. A specific reformulation of the query is therefore not possible, so that a specific search for objects is impossible for outsiders.

The aim is to enable outsiders to search without giving outsiders direct access to the warehouse or the database.

A query is typically composed of a plurality of attributes or object properties which objects to be selected should have if possible. For an outsider, however, it is not known which attributes of the object he is looking for are possibly recorded in the database - and how possibly recorded attributes are represented.

Ideally, the objects are described exactly by such attributes or values of such object properties as are also named in the query. However, the values of the recorded object properties are often incomplete (i.e. the object properties are incompletely recorded in the sense of the query) or have an incorrect format, e.g. because they are recorded by values in different units or on different scales or in different languages or terms.

Accordingly, it is desirable to capture the properties and attributes of objects as accurately and completely as possible. This requires precise specifications and, for example, precise measurement methods. The latter is always difficult to implement when the information on the properties of the objects comes from different sources - for example from different institutions or people or using different devices. After capturing the properties and attributes of objects, they must be represented in some form. it is usual to describe the properties and attributes of objects with words and numbers or other codes and to combine these representations of the properties (words, number codes, etc.) of a respective object into a data record. Also the data sets as a kind of container for these representations of the properties of a respective object can have different formats, just like the representations of the properties contained in them. For example, when it comes to properties that can be measured and expressed in numbers and associated units, such as size, weight or volume, the measurement can vary in accuracy, the representation in numbers and units can vary (number of decimal places, unit used) and their representation and formatting in a file can differ widely for two identical objects, even if the measurements and the representations are free of errors. Describing properties in natural language words increases the variety of ways to name a specific property and represent it in datasets.

Recorded properties and attributes of objects can thus be converted into machine-readable descriptions. Usually these must be unique for a deterministic query. However, this is often not the case in practice.

Another goal is therefore to ensure good usability even for databases that contain less well-maintained data.

In order to enable the search for objects in an inaccessible store or to enable a search via a database without direct access to those in the database, a system is proposed which has an interrogation terminal and a first server connected to it, and one from the interrogation terminal and the first server spatially separate second server and a database which is at least temporarily connected to the second server for querying, but is inaccessible to the query terminal and the first server, with a preferably encrypted connection at least temporarily between the first server and the second server consists. The first server has at least one first neural network, in particular an autoencoder or the transformer, preferably a sparse autoencoder or sparse transformer for converting a query into at least one typed representation, preferably a plurality of typed representations of the query. The first neural network is trained to transform a query into more abstract, typed representations of the query. The second server has a second neural network for converting the data sets with object properties into at least one typed representation, preferably a plurality of typed representations, of the data sets with object properties. The second neural network is trained in such a way that it converts a respective data set with object properties into a more abstract, typified representation of the respective data set representing the properties of the objects.

The system thus has two separate servers, of which the second server has access to the properties of the objects to be queried (e.g. components) and the first is used to formulate the typified query and allows the query results to be evaluated. The connection between the two servers is preferably implemented as an encrypted connection (e.g. SSH). The query is made on the first server and the typed representation is generated. These are transmitted to the second server. On the second server, the typified representation is compared with the typified representations of the objects, in particular compared, and the result of the comparison or comparison is played back to the first server and the query terminal.

It is particularly preferred if the respective neural network, ie in particular the (sparse) autoencoder or (sparse) transformer, is configured in such a way that it generates a plurality of typified representations of a query or of a respective data set with object properties. In the case of an autoencoder, this can be done, for example, by means of a varying drop-out in the individual hidden layers, or in the case of a transformer, by decoupling the outputs of the individual decoders (see below).

In addition, the system has a comparison unit that is configured to compare the typed representation or the typed representations of a query (or multiple typed representations of a query) with the typed representations of the properties of the objects and to generate corresponding similarity values. A selection and output unit connected to the comparison unit is designed to identify those objects whose typed representations of their properties are most similar to the typed representation of the query. The comparison unit is preferably designed to determine the similarity of typified representations by determining the Euclidean distance. The respective Euclidean distance between one of the typified representations of the query and a typified representation of a respective data set with object properties is then a similarity value formed by the comparison unit. The typed representations of the object property records and the typed representations of the query can be in the form of Tensors are present, which are generated by the second or the first neural network. To this end, the first and the second neural network preferably have one or more output layers with an identical number of nodes.

Another possibility for comparing the typified representations of the objects with the respective typified query are so-called distance measures such as the Mahalanobis distance or the Wasserstein distance. The Mahalanobis distance measures the distance between a point p and a distribution D, and the Wasserstein distance measures the distance between two distributions D1 and D2. If an object or a query is now described by means of several typified representations, then these representations (vectors) can be converted into a parameterized distribution D. This distribution can then be used to calculate the distance to a single representation (using Mahalanobis distance) or a number of representations/distribution (using Wasserstein distance).

Alternatively, the query's typed representations can be converted to queries in a query format that is directly applicable to the object property records. The comparison unit then determines the similarity between such transformed queries and the original object property records - rather than with typed representations of the object property records.

For such a transformation of the queries, in a first step they are converted into typed representations of the query as described here. The typed representations of the query obtained in this way are then - in a second step - mapped to an interpretable data representation of the typed query by means of a further decoder or by deterministic mapping and thus converted into a query format that can be applied directly to the data sets with object properties. Such an interpretable data representation can be implemented as a JSON file, for example. In this JSON file, the key terms (e.g. the names of the parameters such as "year of construction" etc.) are named in natural language.

The interpretable data representation of the typed query can then be applied directly to the object property records. By "interpretable" here is meant that the data representation is understandable to humans and can be read to the underlying query or data to be selected - and not in abstract form. A first approach described here consists in converting both the queries and the data sets with object properties into typed representations, each of which has an abstract format. The typed representations of the queries can then be compared to the typed representations of the object property records to determine, through this comparison, those object property records that best match the particular query.

The second approach described here is to first convert only the queries into typed representations of the queries and then to convert the typed representations of the queries into interpretable data representations that are both understandable to humans and applied directly to the datasets with object properties can become. To determine the object property records that best match the query at hand, the interpretable data representations of the queries can then be compared to the object property records to identify the object property records that best match the query.

The quality of the mapping of the typed representations to interpretable data representations (the representation mapping) is preferably provided with a ranking in order to determine those interpretable data representations from among all the possible data representations that best match the respective typed representation. Since both the typified representations and the interpretable data representations can be represented as vectors composed of values for a large number of parameters, it is possible to determine a measure of similarity between the vectors, e.g. their Euclidean distance, and to calculate the ranking based on the similarity measure. to do.

A comparison unit similar to that described above can be provided for this ranking. Accordingly, a typified representation of a query can be compared with different interpretable data representations of queries, and the most suitable interpretable data representation can be determined and selected in each case.

If the parameters (e.g. age, size, year of construction, etc.) in the queries are not defined by a single value but by a range of values, each query spans a multidimensional vector space. to which, for example, a center is assigned can be. In this case, the positions of the centers in the multidimensional vector space can be compared with each other. The ranking is then the higher, the closer the mean values derived from the typified representation of the query and from the interpretable data representation are in the multidimensional vector space.

The comparisons carried out on the basis of the typed representations of the queries or the corresponding interpretable data representations with the data sets with object properties or their typed representations can also be evaluated by means of a corresponding ranking. The same method as for ranking the representation mappings can be used for this. For this purpose, data sets with object properties are assigned to the respective query on the basis of vector spaces that represent a respective query. The more central a data set with object properties is in the vector space defined by the interpretable representation of the query, the higher the ranking.

In a respective vector space representing a query, each part of the query (e.g. each parameter defined in the query) represents a vector dimension. For example, material density, date of manufacture and condition each represent a vector dimension. This results in a three-dimensional parameter or vector space (the terms parameter space and vector space are used synonymously here). The values of the individual dimensions must be converted to a uniform scale. If each value is to have the same value for ranking, the highest ranking value is assigned to the middle of the vector space. However, weightings can also be assigned for the individual dimensions. In this case, the point of the highest ranking value is no longer in the middle of the vector space, but is shifted in the vector space.

The representations - be it the typified representations or the interpretable data representations - are stored in a structured data standard. This can be a JSON format, for example. A format according to the FHIR standard is also suitable for medical data. In addition to the representations of the parameters that define the query or the object properties, the representations can also contain values that represent the quality of the mapping of the resource

If a query or a respective data set with object properties is converted into several typed representations, each typed representation can of a data set with object properties are compared with the different typed representations of the query. With, for example, n different representations of a query and m representations of the data sets with object properties, a matrix of relative similarity values results, the matrix having the dimension n×m.

A system according to the invention thus has the following components: an input interface for entering or receiving a query, e.g. in the form of a query data record, a first neural network connected to the input interface, which is used to convert a query into at least one typed representation, preferably several typed representations of the query is configured and trained, access to a database containing data sets with object properties, a second neural network connected to the access to the database, which is used to convert a respective data set with object properties into at least one typed representation of the data set with object properties, preferably a plurality of typed representations of the data set is configured and trained with object properties, the number of nodes in the output layer or layers of the second neural network corresponding to the number of nodes in the output layer or layers of the first neural network, so that the typified representations of the query and the respective data set with Object properties have the same dimension, a comparison unit for determining the similarities of generated by means of the first neural network typified representations of the query with generated by means of the second neural network typified representations of the data sets with object properties, and a selection and output unit for identifying objects that match a query Based on the similarities determined by the comparison unit and for displaying identified matching objects. The system is divided into a first server, which is at least temporarily connected to a query terminal, a second server, which is spatially separate from the first server, with a preferably encrypted connection existing at least temporarily between the first server and the second server, and a database , which is at least temporarily connected to the second server and inaccessible to the query terminal and the first server,

A corresponding method according to the invention comprises the following steps:

Recording a plurality of physically measurable properties of a number M of objects or individuals, each object or individual being individually characterized by values of the plurality of physically measurable properties recorded for this object or individual

Forming a number m corresponding to the number of data sets with object properties, each of which contains detected values for at least a part of a plurality of physically measurable properties of the respective object or the individual

forming a query to select a subset of the objects or individuals, the query including values for at least a portion of the plurality of physically measurable properties

Transforming the query using an autoencoder or a transformer into a number n of typed representations of the query,

transforming each of the m object property data sets into a number m of typed representations of the object property data sets by means of an autoencoder or a transformer, and

determining a similarity score for each of the m typed representations of the object property records by determining the similarity between a typed representation of the query with a respective one of the m typed representations of the object property records.

The solution allows queries to be carried out without knowing the representations of the objects being searched for, so that a querying person does not need to have any knowledge of the structure of a database or the type and form of the recorded properties of the objects represented. The querying person thus no longer needs direct access to the database, and the representations of objects actually contained in the database remain hidden from the querying person.

This solution also includes the idea that the problem mentioned can not only be solved by more precisely detecting the properties and attributes of the objects, but that it is also possible to achieve good results on the basis of imprecisely or incompletely detected properties and attributes by the data records representing the properties and attributes are converted into a more abstract, typed representation by means of an autoencoder or transformer and the queries are also first converted into a more abstract, typed representation, and then based on a comparison of the typed representation of the query with the typed representations of the Objects to be able to identify one or more objects matching the query.

The system preferably has an autoencoder or a transformer as the first neural network, the input layer of which is supplied with a representation of a query.

The system preferably has an autoencoder or a transformer as the second neural network, to the input layer of which data sets representing properties of the objects are supplied.

The autoencoder or transformer forming the first neural network is trained with training data to convert a specific query into a plurality of typified representations of the query. The autoencoder or transformer forming the second neural network is trained with training data to convert a data set representing respective properties of the objects into a typified representation of the data set. This then allows the query's typed representations to be compared to each of the typed representations of the various object property datasets to identify, among the plurality of objects, those objects whose typed representations whose properties most closely resemble the query's typed representation.

Both the typed representation of the query and the typed representations of the properties of the objects can be represented as tensors generated by the autoencoder or transformer and provided to their respective output layer. The similarity of tensors representing the typed representation of the datasets with object properties to a tensor representing a typed representation of the query can be determined, for example, by determining the Euclidean distance between the tensor representing the typed query and the various tensors representing the typed properties of the objects. A desired number of objects can then be identified for which the Euclidean distance between the tensor describing the respective object and the tensor describing the typed query is the smallest.

The Euclidean distance can also be calculated separately for different subdivisions of the tensor (e.g. for each quarter of the tensor). In this way it can be determined which parts of the tensor have a particularly high similarity. Using an explainability algorithm (XAI) such as Gradient Weighted Class Activation Mapping (Grad-CAM), the partial representation (the part of the tensor) of the objects can be calculated back and identified which properties or input parameters of the object are relevant for the generation of the are partial tensors. This means that not only can the query be transmitted as to how many objects meet the conditions of the query generated, but also which properties exactly apply or which lie in the border areas of the queries. This information can be transmitted from the second server to the first server.

Autoencoders typically have an encoder-decoder architecture with an input layer and an output layer and a plurality of hidden layers in between. As is known, each layer is composed of a large number of nodes, whereby the number of nodes of the input layer must match the representation of the query data set. A property of the autoencoder is that the number of nodes, starting from the input layer, first decreases layer by layer and finally increases again layer by layer, so that a bottleneck forms between the input layer and the output layer. The presented method and the presented system solve the problem of identifying "suitable" objects/persons in relation to a query criterion, i.e. finding objects/persons that match a requirement profile, even if the query criterion or the data for the queried Attributes are not consistent or incomplete because, for example, data records for the individual objects/persons from which the selection is to be made do not contain the requested attributes completely or in a consistent form.

Areas of application are, for example, finding suitable recycling material or finding suitable used spare parts, e.g. car spare parts, batteries or circuit boards with electronic components.

In these use cases, there is often the problem of incomplete or inaccurate data sets. For example, the color of a used replacement part may be generally "black" or manufacturer specific "night black" or specific color code "AB201"b. Inaccurate or missing information about the time of manufacture (model year), inaccurate or incomplete information about the condition (e.g. current battery capacity), etc. is also common.

The invention will now be explained in more detail using exemplary embodiments with reference to the figures. From the figures shows

Fig. 1: an inventive system for identifying objects in a

query have held object properties;

2: an alternative representation of the system according to the invention for identifying objects;

3 shows part of the system from FIG. 1 or 2, in which a query with object properties is converted into a typified representation of the query with the aid of an autoencoder;

4 shows a part of the system from FIG. 1 or 2, in which, with the aid of an autoencoder, data sets with object properties can be converted into typed representations of these object properties; Fig. 5: an example where the autoencoder for converting a query or data sets with object properties into corresponding typed representations is a sparse autoencoder;

Fig. 6: an example in which the autoencoder to convert queries or

records with object properties into corresponding typed representations is only the encoder part of an autoencoder;

7 shows an example in which the autoencoder for converting a query or data sets with object properties into corresponding typed representations is a multi-head autoencoder, in which an encoder of an autoencoder is linked to different decoders;

8 shows an example of an autoencoder for converting data sets with object properties into corresponding typified representations of the object properties using the encoder part of an autoencoder;

Fig. 9: a schematic sketch of a comparison unit and a selection and

Output unit of the system from FIG. 1;

10: a sketch of a three-dimensional parameter space—spanned by the parameter values of a query—and objects located therein on the basis of the object properties; and

11 shows an example of a transformer for converting queries or data sets with object properties into a plurality of typed representations of the query or a respective data set with object properties; and

Fig. 12: another representation of a transformer.

A system 10 for identifying objects that match a query comprises a query terminal 2 and a first server 4 connected to it, as well as a second server 6 spatially separate from the query terminal 2 and the first server 4, and a database 18, which is at least temporarily used for queries with the second server 6 is connected, but for the query terminal 2 and the first server 4 is inaccessible, wherein between the first server 4 and the second server 6 at least temporarily a preferred wise encrypted connection exists; see figure 1 . The first server 4 has at least one first neural network 14, in particular an autoencoder or transformer, preferably a sparse autoencoder or sparse transformer, for converting a query into at least one typified representation of the query. The first neural network 14 is trained to transform a query into more abstract, typed representations of the query.

The second server 6 has a second neural network 20 for converting the data sets with object properties into at least one typed representation, preferably a plurality of typed representations, of the data sets with object properties. The second neural network 20 is trained in such a way that it converts a respective data set with object properties into a more abstract, typified representation of the respective data set representing the properties of the objects.

The system thus has two separate servers 4, 6, of which the second server 6 has access to the properties of the objects to be queried (e.g. components) and the first is used to formulate the typified query and allows the query results to be evaluated. The connection between the two servers is preferably implemented as an encrypted connection (e.g. SSH). The query is made on the first server and the typed representation is generated. These are transmitted to the second server. On the second server, the typified representation is compared with the typified representations of the objects, in particular compared, and the result of the comparison or comparison is played back to the first server and the query terminal.

The query terminal offers an input interface 12 for a query and the first server 4 offers a first autoencoder or transformer 14 for converting the query into a typed representation 16 of the query. The system 10 also has a database 18 with data sets representing object properties and a second server. a second autoencoder or transformer 20 for converting the data sets with object properties into typed representations 22 of the data sets with object properties and a comparison unit 24 for determining the similarity of typed representations 16 of the query with typed representations 22 of the object properties, as well as a selection and output unit 26 for comprises displaying identified matching objects; see figure 2.

The selection and output unit 26 is preferably connected to the first server 4 or the query terminal 2 to the result supplied by the comparison unit 24 - identified objects or number of identified objects - to be able to display a querying person via the query terminal 2.

An autoencoder suitable for converting a query into a typed representation of the query typically has an encoder-decoder structure formed by nodes 30 organized in layers. These layers include an input layer 32, several hidden layers 34, and an output layer 36; see FIG. 3. The nodes 30 of successive layers are connected to one another, so that a node 30 of a subsequent layer is supplied with the output values of some or all of the nodes 30 of a preceding layer. The output values of the nodes of a previous layer thus form the input values for a node in a subsequent layer. A node is designed to process the input values in a weighted manner, for example to add them together in order to form an output value in this way, which is then forwarded to the nodes of the subsequent layer.

An autoencoder has the property that the number of nodes in the layers that follow the input layer initially decreases. This portion of the autoencoder forms an encoder 38. At least a middle layer of the autoencoder 14 is a layer with the fewest number of nodes and forms a bottleneck 40 Nodes of the input layer 32 of the autoencoder 14 are given. In a decoder part 42 the number of nodes per layer increases again successively up to the output layer 36 .

An autoencoder is a neural network whose properties are determined by training. During training, the autoencoder is supplied with a large number of training data. The training data is first converted into more abstract presentations by the encoder of the autoencoder and then reconstructed as precisely as possible by the decoder part of the autoencoder. In this case, a reconstruction loss (reconstruction loss) between the data that is given to the input layer of the autoencoder and the data that the autoencoder outputs to the output layer should be as small as possible. Due to the training with a large number of data sets, which, for example, represent different queries, weights arise in the nodes due to the requirement to keep the reconstruction loss as low as possible, which lead to heavily scattering parts of the training data sets given on the input layer are weighted less than those components of the training data sets that scatter less. This gives the autoencoder the ability to convert any query into a typed representation of the query. This typed representation of the query can be completely abstract and hardly understandable for a user. However, the decoder typically generates a typed representation of a query that is understandable to a user and has been cleaned of random inaccuracies, for example.

In order to avoid overfitting of the trained autoencoder 14, the autoencoder 14 is varied during training, e.g. by randomly deactivating (dropping out) some nodes of the hidden layers 34 during training.

Instead of a conventional autoencoder 14 or 20, a sparse autoencoder 50 can be used to generate the typified representations of either the query or the data sets with object properties, as is shown in FIG. 4 by way of example. Compared to the autoencoder 14 or 20, the sparse autoencoder 50 lacks the characteristic bottleneck between encoder 38 and decoder 42. Rather, in a sparse autoencoder 50 all layers can contain the same number of nodes, i.e. both the input layer 32 and the Output layer 36 and the intervening hidden layers 34 have the same number of nodes 30 . In order to still achieve the desired effect, for example for a specific query given to the input layer 32 of the sparse autoencoder 50 to generate a typified representation in which arbitrary features are weighted less than central features, some nodes are removed during the training of the sparse autoencoder 30' of the hidden layers 34' are deactivated. To this end, the sparse autoencoder 50 can have additional bias nodes for each layer. It should be noted that the structure (topology) of the sparse autoencoder 50 can be similar to that of the autoencoder 14. That is, even with the sparse autoencoder 50, hidden layers 34' can have fewer nodes 30 than the input layer 32 and the output layer 36.

It should be noted that by appropriately deactivating individual nodes in a sparse autoencoder, a query can be converted into different typed representations, as is shown in FIG.

Instead of providing a full autoencoder 14 or 20 or a sparse autoencoder 50 to generate the typed representations of the query and/or the object property records, only the encoder 38 of an autoencoder including the bottleneck 40 can be used. Make the knots of the bottle neck 40 then already represents the output layer. Such an example is shown in FIG. With such a neural network, a query or data sets with object properties can be converted into feature vectors 56, which likewise represent typified representations, for example of a query or a respective one with object properties. The example in FIG. 5 shows the conversion of a query into a plurality of feature vectors by means of the encoder 38. Data sets with object properties can also be converted into corresponding feature vectors in the same way. This is not shown.

FIG. 6 illustrates that the autoencoder 18' can also have a multihead architecture, in which an encoder 38 is connected to a large number of decoders 42, which are each trained differently. In this way it is also possible, for example, to convert a query or also data sets with object properties into a plurality of typed representations.

Figures 7 and 8 illustrate how each object property data set from a plurality of object property data sets can be converted into typed representations of these object property data sets by means of an autoencoder 20 (Figure 7). FIG. 8 illustrates analogously to FIG. 4 that each data set with object properties from a large number of data sets with object properties can be converted into feature vectors by means of the encoder part 38 of an autoencoder, which represent a typified representation of the respective data set with object properties.

After, on the one hand, a query has been converted into one or more typed representations and, on the other hand, the data sets with object properties have each been converted into one or more typed representations, the similarities between the typed representation(s) of the query and the typed representation(s) of the respective data sets with object properties to be determined. This is done using the comparison unit 24 shown in FIG. 2. FIG. 9 shows the comparison unit 24 in a more detailed representation.

After determining the similarities between, for example, a representation of the query and the representations of the object property records, those associated objects that are most similar in object properties to the query can be identified. These objects can then be displayed via an output 26. What is decisive is that this identification of objects with the system 10 described here also works if the data sets with object properties for the various objects are not uniform, in particular not complete, or the recorded object properties, in particular the recorded physical parameters, are represented imprecisely or deviate from one another . This allows the data sets with object properties to come from different, not strictly coordinated sources. Nevertheless, a meaningful query to identify suitable objects is possible. This allows suitable objects to be identified automatically without the need for time-consuming manual maintenance of the data records and without a detailed definition of the contents of a query.

The comparison unit 24 is designed to determine a similarity value for each typified representation of a query, in particular the Euclidean distance to the typified representations of each of the data sets with object properties. From the similarity values determined in this way, the comparison unit 24 generates a similarity matrix 60 with the dimension m×n, where m is the number of typed representations of the query and n is the number of typed representations of the data records with object properties. The similarity matrix 60 can then be used to identify those objects whose associated typed representations of the data sets with object properties have the greatest similarity, in particular the smallest Euclidean distance, to the typed representations of the query.

Finally, the selection and output unit 26 is provided for identifying objects that match the query. This is designed to form a representation value from the similarity matrix 60 on the basis of the similarity matrix 60 . For example, this can be done in one of four ways:

1 . Only the highest value of the similarity matrix 60 is used

2. The mean or median of a row or column of the similarity matrix 60 is taken

3. The mean or median of the top-N similarity values (e.g. the 5 highest values) is used

4. The mean or median of the entire matrix is used. This representation value is checked against a threshold set in the system and if the representation value is above or equal to the threshold, the record in question and the object (or person) associated with it are identified as matching the query and a list of matching objects is provided at the output.

The comparisons carried out on the basis of the typed representations of the queries or the corresponding interpretable data representations with the data sets with object properties or their typed representations can also be evaluated using a corresponding ranking. For this purpose, data sets with object properties are assigned to the respective query on the basis of vector spaces that represent a respective query. The more central a data set with object properties is in the vector space defined by the interpretable representation of the query, the higher the ranking. This is sketched as an example in FIG. 10 for a three-dimensional parameter or vector space. Such a three-dimensional parameter or vector space results when the individual objects are each characterized by three parameters such as age, size and condition. Each object is then represented by a three-dimensional feature vector, which can be represented by a point in the three-dimensional parameter or vector space. The three-dimensional parameter space or vector space shown as a cuboid in FIG. 11 can represent a query, for example. Some objects fall within the three-dimensional parameter or vector space defined by the query, while other objects do not match the query and fall outside the three-dimensional parameter or vector space defined by the query.

In a respective vector space representing a query, each part of the query (ie, for example, each parameter defined in the query) represents a vector dimension. For example, material density, date of manufacture and condition each represent a vector dimension. This results in a three-dimensional parameter space. The values of the individual dimensions must be converted to a uniform scale. If each value is to have the same value for ranking, the highest ranking value is assigned to the middle of the vector space. However, weightings can also be assigned for the individual dimensions. In this case, the point of the highest ranking value is no longer in the middle of the vector space, but is shifted in the vector space. As explained at the outset, both the first neural network for generating typed representations of a query and the second neural network for generating typed representations of the respective data sets with object properties can be transformers, in particular sparse transformers. FIG. 11 shows such a transformer, which can be provided in the system according to FIG. 1 instead of the autoencoders 14 and 20 shown there by way of example.

The Transformer 90 also has an encoder-decoder architecture. In fact, a plurality of encoders 92 and a plurality of decoders 94 are provided. As is fundamentally known to a person skilled in the art, each encoder 92 of a transformer has a self-attention layer 96 and a feedforward layer 98 in each case. Each decoder 94 also has a self-attention layer 100 and a feedforward layer 102 . An encoder-decoder attention layer 104 is provided between the self-attention layer 100 of a decoder 94 and its feedforward layer 102 .

The input layer of the first encoder 92 is formed by an embedding layer 106 . For each query, the query components are first embedded (input embedding) and the position is coded (position encoding). An input data set formed in this way is then passed through the various encoders 92 and processed. The output value of the last encoder 92 is then given to all encoder-decoder attention layers 104 of all decoders 94 .

In known transformers, the outputs of all decoders are processed into a single output tensor, for example using a softmax function. Deviating from this, in order to obtain different typed representations of a query or a respective data set with object properties, the output tensors generated by each individual decoder 94 are taken out of neural network 14 or 20 as different typed representations of a query and fed to comparison unit 26. For this reason, the layers following the last decoder 94 are not shown further in the schematic representation of a transformer in FIG.

With regard to the modeling, i.e. the training of the neural networks 14 and 20, it should be noted that the neural network 14 used to generate typed representations of the query and the neural network 20 used to generate the typed representations of the data sets with object properties initially trained with identical training data sets can become. Subsequently, the neural network 20 for Forming the typified representations of the datasets with object properties can be retrained.

Preferably, the first neural network 14 and the second neural network 20 have a similar or identical topology. If the topology of the two neural networks 14 and 20 is identical, the two neural networks 14 and 20 usually still differ in the model they embody, which is represented by the weights generated in the individual nodes as a result of the training. It is known, for example, that each node 30 of a preceding layer passes on its output value to all nodes of the following layer (at least in the case of a fully connected neural network), so that a node of a subsequent layer receives the output values of all nodes of the preceding layer as input values become. These input values are weighted differently in the respective node, the different weights being the result of the training of the neural network and together with the topology of the neural network (i.e. its structure of nodes, layers and connections) embodying a model.

Figure 12 shows another representation of a transformer.

reference list

2 query terminal

4 first server to create typed queries

6 second servers for creating typed representations

10 systems

12 input interface

14 auto encoders/transformers

16 typed representation of the query

18, 18' database

20 auto encoders/transformers

22 typed representations of object properties

24 comparison unit

26 selection and output unit

30, 30' knots

32 input layer

34, 34' hidden layers

36 output layer

38 encoders

40 bottle neck

42 decoders

50 sparse auto encoders

56 feature vectors

60 Similarity Matrix

92 encoders

94 decoders

96, 100 Self-Attention Layer

98, 102 feedforward layer

104 Encoder Decoder Attention Layer

106 embedding layer

Claims

- 24 - claims

1. System (10) for identifying and/or locating objects, with a first server (4) which is at least temporarily connected to a query terminal (2), a second server (6) spatially separated from the first server (4). is separated, wherein between the first server (4) and the second server (6) there is at least temporarily a preferably encrypted connection and a database (18) which is at least temporarily connected to the second server (6) and for the query terminal (2) and the first server (4) is inaccessible, the first server (4) having an input interface for entering or receiving a query, and a first neural network connected to the input interface for converting a query into at least one typed representation, preferably a plurality of typed representations Representations of the query is configured and trained, wherein the second server (6) has access to the database (18) containing data sets with object properties, a second neural network connected to the access to the database (18) for converting a respective data set with object properties is configured and trained in at least one typed representation of the data set with object properties, preferably several typed representations of the data set with object properties, the number of nodes of the output layer or layers of the second neural network being equal to the number of nodes of the output layer or layers of the first neural network, so that the typed representations of the query and the respective data set with object properties have the same dimension, a comparison unit for determining the similarities of generated by means of the first neural network typed representations of the query with generated by means of the second neural network typed representations of the data sets with object properties, and a selection and an output unit for identifying objects that match a query based on the similarities determined by the comparison unit and for displaying identified matching objects. System according to Claim 1, in which the first neural network (14) and the second neural network (20) are each designed as a sparse autoencoder. System according to Claim 1, in which the first neural network (14) and the second neural network (20) are each designed as a transformer (90), in particular as a sparse transformer. Procedure with the steps:

forming a query to select a subset of the objects or individuals, the query including values for at least a portion of the plurality of physically measurable properties Transforming the query using an autoencoder or a transformer into a number n of typed representations of the query,

determining a similarity score for each of the m typed representations of the object property records by determining the similarity between a typed representation of the query and a respective one of the m typed representations of the object property records. The method of claim 4, further comprising: constructing a similarity matrix containing the determined similarity values. The method of claim 4 or 5, wherein determining a similarity value for each of the m typed representations of the object property data sets by determining the similarity between a typed representation of the query and d a respective one of the m typed representations of the object property data sets includes forming a value of the Euclidean Distance between the selection feature vectors with the object feature vectors representing a respective data set. Method according to claim 6, wherein the Euclidean distance is the respective similarity value. Method according to Claims 5 and 7, in which the similarity matrix contains values for the Euclidean distance between the selection feature vectors and the object feature vectors representing a respective data set. The method of claim 4 or 5, wherein determining a similarity value for each of the m typed representations of the object property datasets by forming a vector space representing the respective query and determining the position in the given by the object properties defined by a respective object property dataset the vector space representing the respective query takes place. - 27 - Method according to claim 9, wherein the proximity of the position given by the object properties to a center of the vector space representing the respective query is the respective similarity value. Procedure with the steps:

transforming the typed representations of the query into interpretable data representations of the query, and

determining a similarity score for each of the object property records by determining the similarity between an interpretable data representation of the query and a respective one of the object property records. Method according to claim 11, characterized in that the typed representations of the query are mapped to an interpretable data representation of the typed query by means of a further decoder or by deterministic mapping and are thus converted into a query format which can be applied directly to the data sets with object properties.