CN112668597A

CN112668597A - Feature comparison method, device and equipment

Info

Publication number: CN112668597A
Application number: CN201910980095.0A
Authority: CN
Inventors: 罗仕胜; 陈益新; 李二连
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2019-10-15
Filing date: 2019-10-15
Publication date: 2021-04-16
Anticipated expiration: 2039-10-15
Also published as: CN112668597B

Abstract

The application provides a feature comparison method, a device and equipment, wherein the method comprises the following steps: determining at least one target GPU for processing the characteristics to be detected from all GPUs; sending the features to be detected to a target GPU, so that the target GPU sends model information of a plurality of first feature models in a first feature library to a CPU (central processing unit) according to the similarity between the features to be detected and each first feature model in the first feature library; selecting a plurality of second feature models from a second feature library according to the model information sent by the target GPU; determining at least one target feature model from the plurality of second feature models according to the similarity between the features to be detected and each selected second feature model; the number of feature points of the first feature model in the first feature library is smaller than the number of feature points of the second feature model in the second feature library. According to the technical scheme, the number of the feature models loaded by the GPU is increased in a mode of loading the small feature models by the GPU, and the comparison precision is guaranteed.

Description

Feature comparison method, device and equipment

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, and a device for comparing features.

Background

Machine learning is a way to realize artificial intelligence, is a multi-field cross subject, and relates to a plurality of subjects such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. Machine learning is used to study how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to improve their performance. Machine learning focuses more on algorithm design, so that a computer can automatically learn rules from data and predict unknown data by using the rules.

In the related technology of machine learning, feature comparison is used as an effective means and widely applied to the fields of face recognition, human body recognition, vehicle recognition and the like. The characteristic comparison means that the characteristics to be detected are compared with the characteristic models in the characteristic library one by one, and then the characteristic models meeting the requirements are screened from the characteristic library.

Because the number of the feature models in the feature library is very large, when the features to be detected are compared with the feature models in the feature library one by one, the computation amount is very large, so that the computation speed of feature comparison is very slow, and therefore, how to improve the computation speed of feature comparison becomes a problem to be solved at present.

Disclosure of Invention

The application provides a feature comparison method, a feature comparison device and a feature comparison device, so that the number of feature models which can be loaded by a GPU is increased in a mode of loading small feature models by the GPU, and comparison accuracy is guaranteed.

The application provides a feature comparison method, which is applied to a CPU (central processing unit), and the method comprises the following steps:

determining at least one target GPU for processing the characteristics to be detected from all GPUs;

sending the feature to be detected to the target GPU, so that the target GPU sends model information of a plurality of first feature models in a first feature library to the CPU according to the similarity between the feature to be detected and each first feature model in the first feature library;

selecting a plurality of second feature models from a second feature library according to the model information sent by the target GPU; determining at least one target feature model from the plurality of second feature models according to the similarity between the feature to be detected and each selected second feature model; the number of feature points of the first feature model in the first feature library is smaller than the number of feature points of the second feature model in the second feature library.

The application provides a device is compared to characteristic, the device includes: the determining module is used for determining at least one target GPU for processing the characteristics to be detected from all GPUs;

the sending module is used for sending the features to be detected to the target GPU so that the target GPU sends the model information of the first feature models in the first feature library to the CPU according to the similarity between the features to be detected and each first feature model in the first feature library;

the processing module is used for selecting a plurality of second feature models from a second feature library according to the model information sent by the target GPU; determining at least one target feature model from the plurality of second feature models according to the similarity between the feature to be detected and each selected second feature model;

the number of the feature points of the first feature model in the first feature library is smaller than the number of the feature points of the second feature model in the second feature library.

The application provides a device is compared to characteristic, includes: a CPU and a machine-readable storage medium storing machine-executable instructions executable by the CPU;

the CPU is configured to execute machine executable instructions to perform the following steps:

The application provides a feature comparison device, which is characterized by comprising a CPU and a GPU, wherein:

the CPU is used for determining at least one target GPU for processing the characteristics to be detected from all GPUs; sending the features to be detected to the target GPU; the GPU is used for sending the model information of the plurality of first feature models in the first feature library to the CPU according to the similarity between the feature to be detected and each first feature model in the first feature library;

the CPU is further used for selecting a plurality of second feature models from a second feature library according to the model information sent by the target GPU; determining at least one target feature model from the plurality of second feature models according to the similarity between the feature to be detected and each selected second feature model;

As can be seen from the above technical solutions, in the embodiment of the present application, a feature comparison process can be implemented based on the GPU and the CPU, and the number of feature models in the GPU can be increased by loading the first feature library (including the first feature model with a smaller number of feature points) in the GPU and loading the second feature library (including the second feature model with a larger number of feature points) in the specified storage medium. When the GPU performs feature comparison on the features to be detected, a small amount of precision loss occurs because the number of feature points of the first feature model is small. And the GPU outputs a plurality of comparison results to the CPU, so that the CPU performs characteristic comparison on the characteristics to be detected based on the second characteristic models corresponding to the comparison results, and accurate characteristic comparison results are obtained due to the fact that the number of characteristic points of the second characteristic models is large. Obviously, in the above manner, by loading a small model (i.e., a first feature model with a small number of feature points) on the GPU and loading a large model (i.e., a second feature model with a large number of feature points) on the specified storage medium, the CPU computation amount is reduced, the comparison accuracy is ensured, and the capacity and concurrency of the feature library are improved (concurrency refers to how many features to be detected can be compared with the feature models per second on average).

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments of the present application or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings of the embodiments of the present application.

FIG. 1 is a flow chart of a feature comparison method in one embodiment of the present application;

FIG. 2 is a flow chart of a method of feature alignment in another embodiment of the present application;

FIG. 3 is a schematic illustration of the loading of a feature library in one embodiment of the present application;

FIG. 4 is a flow chart of a method of feature alignment in another embodiment of the present application;

FIG. 5 is a block diagram of a feature matching apparatus according to an embodiment of the present application;

fig. 6 is a structural diagram of a feature comparison device in an embodiment of the present application.

Detailed Description

The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein is meant to encompass any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Depending on the context, moreover, the word "if" as used may be interpreted as "at … …" or "when … …" or "in response to a determination".

The feature comparison is widely applied to the fields of face recognition, human body recognition, vehicle recognition and the like. The characteristic comparison means that the characteristics to be detected are compared with the characteristic models in the characteristic library one by one, and then the characteristic models meeting the requirements are screened from the characteristic library. Because the number of the feature models in the feature library is very large, when the feature to be detected is compared with the feature models in the feature library one by one, the operation speed is very slow, the operation amount is very large, and how to improve the operation speed of feature comparison becomes a problem to be solved at present.

In one implementation, since the feature comparison is substantially a linear operation between the feature model in the feature library and the feature to be detected, by increasing the main frequency of a Central Processing Unit (CPU) or using a multi-core CPU, the feature to be detected can be compared with a plurality of feature models in the feature library at the same time, thereby increasing the linear operation speed. However, the improvement of the CPU dominant frequency and the multi-core CPU technology have reached development bottlenecks, and it is difficult to further improve the CPU dominant frequency or adopt a CPU with more cores.

In another implementation, feature comparison may be implemented by a GPU (Graphics Processing Unit), and since a large number of GPUs may be deployed, each GPU may perform feature comparison between features to be detected and a plurality of feature models in a feature library, thereby increasing linear operation speed. However, the storage space (e.g., video memory) of the GPU is relatively small (e.g., 1-24GB), so that all the feature models in the feature library cannot be stored, only a part of the feature models in the feature library can be stored, and then the GPU cannot perform feature comparison on the features to be detected by using all the feature models, and thus an accurate feature comparison result cannot be obtained.

In view of the above problems, in the embodiment of the present application, feature comparison is implemented based on a GPU and a CPU, and a small model (referred to as a first feature model with a small number of feature points in this embodiment) is loaded on the GPU and a large model (referred to as a second feature model with a large number of feature points in this embodiment) is loaded on a specified storage medium, so that the CPU computation amount is reduced, the comparison accuracy is ensured, and the capacity and concurrence of a feature library are improved.

The technical solutions of the embodiments of the present application will be described in detail below with reference to specific embodiments.

The embodiment of the application provides a feature comparison method, which can be applied to feature comparison equipment, wherein the feature comparison equipment can comprise a CPU (single-core CPU or multi-core CPU) and a plurality of GPUs. The feature comparison device may be any type of device, such as a personal computer, a server, an imaging device, a storage device, a notebook computer, and the like, without limitation, as long as the device includes a CPU and multiple GPUs. Referring to fig. 1, a schematic flow chart of the feature comparison method is shown, and the feature comparison method may include:

in step 101, the CPU determines at least one target GPU for processing the features to be detected from all GPUs, that is, the number of the target GPUs may be one or multiple.

In an example, before step 101, the CPU may obtain a first feature model and a second feature model corresponding to each sample feature, where the number of feature points of the first feature model corresponding to the sample feature is smaller than the number of feature points of the second feature model corresponding to the sample feature. The model identification of the first feature model corresponding to the sample feature has a mapping relation with the model identification of the second feature model corresponding to the sample feature.

Determining the number M of GPUs (graphics processing units) required to be occupied by all the first feature models according to the total number N of all the first feature models and a set threshold K; grouping all GPUs, wherein the number of the GPUs in each GPU group is more than or equal to M; all the first feature models are grouped to obtain M first feature libraries, and the M first feature libraries are stored in the GPUs in each GPU group, namely each GPU group can store the M first feature libraries, and each GPU stores one first feature library. The number of the first feature models in each first feature library is less than or equal to K, and K is the maximum value of the number of the first feature models allowed to be stored by the GPU.

All the second feature models are grouped into a second feature library, and the second feature library is stored in a specified storage medium (such as a memory or a high-speed storage area, and the specified storage medium is not limited).

In one example, the CPU determines at least one target GPU for processing the feature to be detected from all GPUs, which may include but is not limited to: and if the number of the GPU groups is one, determining each GPU in the GPU groups as a target GPU. Or if the number of the GPU groups is at least two, selecting the GPU group with the lightest load from all the GPU groups according to the load condition of each GPU group, and determining all the GPUs in the GPU group with the lightest load as the target GPU. Or, if the number of the GPU groups is at least two, arbitrarily selecting one GPU group from all the GPU groups, and determining all the GPUs in the selected GPU group as the target GPU. Or, if one GPU can store all the first feature models (i.e., M is 1), one GPU is arbitrarily selected from all the GPUs, and the selected one GPU is determined as the target GPU.

And 102, the CPU sends the features to be detected to the target GPU, so that the target GPU sends the model information of the plurality of first feature models in the first feature library to the CPU according to the similarity between the features to be detected and each first feature model in the first feature library. Wherein the model information of the first feature model may comprise a model identification of the first feature model. Or, the model information of the first feature model may include a model identifier of the first feature model, and a similarity between the first feature model and the feature to be detected.

Specifically, if at least two target GPUs exist, the CPU sends the features to be detected to each target GPU. Each target GPU sends the model information of the plurality of first feature models in the first feature library to the CPU according to the similarity between the feature to be detected and each first feature model in the first feature library, namely, the CPU can receive the model information of the plurality of first feature models sent by each target GPU.

103, selecting a plurality of second feature models from a second feature library by the CPU according to the model information sent by the target GPU; and determining at least one target feature model from the plurality of second feature models according to the similarity between the features to be detected and each selected second feature model. The number of the feature points of the first feature model in the first feature library is smaller than the number of the feature points of the second feature model in the second feature library.

The process of selecting, by the CPU, a plurality of second feature models from the second feature library may include, but is not limited to:

in the first mode, if the model information includes a model identifier, and a mapping relationship exists between the model identifier of the first feature model and the model identifier of the second feature model corresponding to the same sample feature, the CPU selects the second feature model corresponding to the model identifier sent by the target GPU from the second feature library based on the mapping relationship between the model identifier of the first feature model and the model identifier of the second feature model.

If the model information comprises model identifications and similarity, and the model identifications of the first feature models and the model identifications of the second feature models corresponding to the same sample feature have a mapping relation, the CPU selects target model identifications from the model identifications of the first feature models sent by the target GPU according to the similarity of the first feature models sent by the target GPU (namely the similarity between the first feature models and the features to be detected); and selecting a second characteristic model corresponding to the target model identification from the second characteristic library based on the mapping relation between the model identification of the first characteristic model and the model identification of the second characteristic model.

In the second mode, the CPU may sort the model identifications of the plurality of first feature models sent by the target GPU based on the similarity of the plurality of first feature models sent by the target GPU; and selecting a specified number of model identifications from the model identifications of the first feature models sent by the target GPU as target model identifications based on the sequencing result. For example, when the models are sorted in the order of similarity from high to low, a specified number of model identifications sorted at the top may be selected as the target model identifications. When the models are sorted in the order of similarity from low to high, a specified number of model identifications sorted later may be selected as the target model identifications.

Further, the manner of determining the specified number includes, but is not limited to: the specified number is determined according to the processing capability of the CPU, for example, the greater the processing capability of the CPU, the worse the processing capability of the CPU, and the smaller the specified number. Alternatively, the specified number may be determined according to a concurrent output situation of the GPU, for example, if the GPU outputs model information of S first feature models to the CPU, the specified number may be equal to S.

The storage process of the feature model is described below with reference to specific embodiments.

For example, a sample feature library may be created using a conventional modeling algorithm, and the sample feature library may include a large number of sample features, and the creation process is not limited thereto. For example, in the field of face recognition, a sample feature library for face recognition is established, which includes a large number of face recognition features. In the field of human body identification, a sample feature library for human body identification is established, and the sample feature library comprises a large number of human body identification features. In the field of vehicle identification, a sample feature library for vehicle identification is established, which includes a large number of vehicle identification features. Of course, the above is only an example of the sample feature library, and is not limited thereto.

In this embodiment, for each sample feature in the sample feature library (for example, all features used for identifying the same target, and the same target, such as the same face, the same human body, the same vehicle, and the like, can be identified by the features), the sample feature may correspond to the first feature model and the second feature model.

For each sample feature, the number of feature points of the first feature model corresponding to the sample feature is smaller than the number of feature points of the second feature model corresponding to the sample feature, that is, the complexity of the first feature model corresponding to the sample feature is smaller than the complexity of the second feature model corresponding to the sample feature, and the complexity may refer to the number of feature points of the feature model. For example, for a certain sample feature, the sample feature may be represented by 100 feature points, or may be represented by 10 feature points, so that the feature model based on 100 feature points is referred to as a second feature model, and the feature model based on 10 feature points is referred to as a first feature model. Obviously, the number of the feature points of the second feature model is greater than that of the feature points of the first feature model, so that the second feature model describes the sample features more accurately.

The feature model is a string of binary data describing the target through an algorithm, and when the number of the feature points is larger, the feature model is longer, that is, the data amount of the feature model is larger, and the storage space occupied by the feature model is larger. When the number of feature points is smaller, the feature model is shorter, that is, the data size of the feature model is smaller, and the storage space occupied by the feature model is smaller. In summary, since the number of the feature points of the second feature model is greater than the number of the feature points of the first feature model, the data volume of the second feature model is greater than the data volume of the first feature model, and the storage space occupied by the second feature model is greater than the storage space occupied by the first feature model.

For example, for each sample feature in the sample feature library, the first feature model and the second feature model are obtained based on the sample feature, that is, the first feature model and the second feature model are both obtained based on the sample feature, and the obtaining manner is not limited. For example, this sample feature may be represented by 100 feature points, and a feature model based on 100 feature points is referred to as a second feature model, and this sample feature may be represented by 10 feature points, and a feature model based on 10 feature points is referred to as a first feature model.

Or, for each sample feature in the sample feature library, obtaining a second feature model based on the sample feature, and obtaining a first feature model based on the second feature model, that is, the second feature model is obtained based on the sample feature, and the first feature model is obtained based on the second feature model, which is not limited to this obtaining manner. For example, this sample feature may be represented by 100 feature points, and a feature model based on the 100 feature points is referred to as a second feature model. 10 feature points are selected from the 100 feature points of the second feature model, the sample feature is represented by the 10 feature points, and the feature model based on the 10 feature points is referred to as a first feature model.

Of course, the above manner is only one example of obtaining the first feature model and the second feature model, and is not limited to this, as long as the first feature model and the second feature model corresponding to each sample feature can be obtained, and the number of feature points of the first feature model is smaller than the number of feature points of the second feature model. In practical applications, the first feature model may also be referred to as a small model, and the second feature model may also be referred to as a large model.

For example, for each sample feature in the sample feature library, the first feature model corresponding to the sample feature has a model identifier, and the model identifier is used to uniquely represent the first feature model, that is, the model identifiers corresponding to different first feature models may be different. In addition, the second feature model corresponding to the sample feature has a model identifier, and the model identifier is used to uniquely represent the second feature model, that is, the model identifiers corresponding to different second feature models may be different.

For each sample feature, the model identifier of the first feature model corresponding to the sample feature has a mapping relation with the model identifier of the second feature model corresponding to the sample feature. For example, the model identifier of the first feature model is the same as the model identifier of the second feature model, which is not limited to this, as long as the two feature models have a mapping relationship, and the corresponding second feature model can be known through the model identifier of the first feature model.

For example, a first feature model 1 and a second feature model 1 corresponding to the sample feature 1 are obtained, and a model identifier (e.g., a1) of the first feature model 1 has a mapping relation with a model identifier (e.g., a1 or B1) of the second feature model 1. The first feature model 2 and the second feature model 2 corresponding to the sample feature 2 are obtained, a model identifier (e.g., a2) of the first feature model 2 and a model identifier (e.g., a2 or B2) of the second feature model 2 have a mapping relationship, and so on, and subsequently, taking ten thousand sample features as an example, ten thousand first feature models and ten thousand second feature models can be obtained, and of course, the number of sample features is much greater than ten thousand, which is not limited.

Based on the above application scenario, referring to fig. 2, a schematic flow chart of the feature comparison method is shown.

Step 201, the CPU obtains a first feature model and a second feature model corresponding to each sample feature.

For each sample feature in the sample feature library, a first feature model and a second feature model corresponding to the sample feature may be obtained, the number of feature points of the first feature model corresponding to the sample feature is smaller than the number of feature points of the second feature model corresponding to the sample feature, and a model identifier of the first feature model corresponding to the sample feature and a model identifier of the second feature model corresponding to the sample feature have a mapping relationship.

For example, assuming that 10000 sample features are included in the sample feature library, the CPU may acquire 10000 first feature models (subsequently denoted as first feature model 1-first feature model 10000) and 10000 second feature models (subsequently denoted as second feature model 1-second feature model 10000).

In step 202, the CPU groups all the second feature models into a second feature library and stores the second feature library into a designated storage medium, that is, the designated storage medium stores all the second feature models.

For example, the CPU groups the second feature models 1 to 10000 into a second feature library (i.e., a set including these second feature models), and stores the second feature library into a specified storage medium, so that the specified storage medium includes the second feature models 1 to 10000 in the second feature library.

Among others, the specified storage media may include, but are not limited to: a memory, a high-speed storage area, which may be, for example, a Solid State Drive (SSD), etc., without limitation.

Step 203, the CPU determines the number M of GPUs that all the first feature models need to occupy according to the total number N of all the first feature models and a set threshold K, where K is the maximum value of the number of first feature models stored by the GPU, that is, the maximum value of the number of first feature models that can be stored by the GPU video memory.

In one possible embodiment, the CPU divides the total number N of all the first feature models by the set threshold K and rounds up to obtain an intermediate value. The number M of GPUs that all the first feature models need to occupy may be greater than or equal to the intermediate value. For example, assuming that the total number N of all the first feature models is 10000, and the threshold K is 2500, the value obtained by dividing 10000 by 2500 is rounded up to obtain an intermediate value of 4, and the GPU number M may be 4, or may be other values greater than 4, such as 5, 6, and the like, which is not limited thereto.

In another possible embodiment, the CPU rounds up the total number N of all the first feature models divided by the set threshold K to obtain an intermediate value. The number M of GPUs that all the first feature models need to occupy may be greater than or equal to the intermediate value, and the total number of all the GPUs is an integer multiple of the number M of GPUs. For example, assuming that the total number N of all the first feature models is 10000 and the threshold K is set to 2500, the division of 10000 by 2500 is rounded up to obtain an intermediate value of 4. Assuming that the total number of all GPUs is 8, the GPU number M may be 4 or 8. Assuming that the total number of all GPUs is 10, the GPU number M may be 5 or 10. Assuming that the total number of all GPUs is 12, the GPU number M may be 4, 6 or 12.

In the above embodiment, the threshold K is set to be the maximum value of the first feature model number stored by the GPU. The set threshold K may be a threshold configured empirically, or may be determined based on the size of the first feature model and the size of the available storage space in the GPU video memory, for example, the value obtained by dividing the size of the available storage space in the GPU video memory by the size of the first feature model is taken upward, and the value after the upward value is taken may be the set threshold K.

And step 204, grouping all the GPUs by the CPU, wherein the number of the GPUs in each GPU group is more than or equal to M. For convenience of description, in this embodiment, the number of GPUs in each GPU group is equal to M as an example.

For example, assuming that the number M of GPUs is 4, all the GPUs include GPU1, GPU2, GPU3, GPU4, GPU5, GPU6, GPU7, and GPU8, based on which GPU1, GPU2, GPU3, and GPU4 may be grouped into GPU group 1, and GPU5, GPU6, GPU7, and GPU8 may be grouped into GPU group 2.

Step 205, the CPU groups all the first feature models to obtain M first feature libraries, and stores the M first feature libraries to the GPUs in each GPU group, that is, to the GPU video memory of the GPU. That is, each GPU group can store M first feature libraries, and each GPU may store one first feature library, and the number of first feature models in each first feature library may be less than or equal to K.

Assuming that M is 4 and K is 2500, the first feature models 1 to 10000 are grouped into 4 first feature libraries, and the number of the first feature models in each first feature library is 2500. For example, first feature library 1 comprises first feature model 1-first feature model 2500, first feature library 2 comprises first feature model 2501-first feature model 5000, first feature library 3 comprises first feature model 5001-first feature model 7500, and first feature library 4 comprises first feature model 7501-first feature model 10000. Up to this point, 4 first feature libraries were obtained, each of which is a set comprising 2500 first feature models.

GPU group 1 includes GPU1, GPU2, GPU3, and GPU4, and thus, the first feature library 1 is stored to the GPU memory of GPU1, the GPU memory of GPU1 includes 2500 first feature models in the first feature library 1, and GPU1 has access to the first feature models in the first feature library 1. The first feature library 2 is stored to the GPU cache of the GPU2, the GPU cache of the GPU2 includes 2500 first feature models in the first feature library 2, and the GPU2 has access to the first feature models in the first feature library 2. The first feature library 3 is stored to the GPU cache of the GPU3, the GPU cache of the GPU3 includes 2500 first feature models in the first feature library 3, and the GPU3 has access to the first feature models in the first feature library 3. The first feature library 4 is stored to the GPU cache of the GPU4, the GPU cache of the GPU4 includes 2500 first feature models in the first feature library 4, and the GPU4 has access to the first feature models in the first feature library 4.

The GPU group 2 comprises a GPU5, a GPU6, a GPU7 and a GPU8, the first feature library 1 is stored in a GPU video memory of the GPU5, the first feature library 2 is stored in a GPU video memory of the GPU6, the first feature library 3 is stored in a GPU video memory of the GPU7, and the first feature library 4 is stored in a GPU video memory of the GPU 8.

Referring to fig. 3, the CPU stores the second feature library to a specified storage medium. The CPU stores the first feature library 1 in the GPU video memory of the GPU1 in the GPU group 1, loads the first feature library 2 in the GPU video memory of the GPU2 in the GPU group 1, loads the first feature library 3 in the GPU video memory of the GPU3 in the GPU group 1, and loads the first feature library 4 in the GPU video memory of the GPU4 in the GPU group 1. The CPU loads the first feature library 1 into the GPU video memory of the GPU5 in the GPU group 2. The first feature library 2 is loaded into the GPU video memory of GPU6 in GPU group 2. The first feature library 3 is loaded into the GPU video memory of GPU7 in GPU group 2. The first feature library 4 is loaded into the GPU video memory of GPU8 in GPU group 2.

The following describes a feature comparison process of features to be detected with reference to specific embodiments. Referring to fig. 4, a schematic flow chart of a feature comparison method is shown, where the feature comparison method may include:

step 401, the CPU obtains features to be detected, and determines a target GPU for processing the features to be detected from all GPUs, where the number of the target GPUs may be one or more.

For example, if the number of GPU groups is multiple, the GPU group with the lightest load may be selected from all the GPU groups according to the load condition of each GPU group, and all the GPUs in the GPU group with the lightest load are determined as the target GPU. For example, the CPU selects a GPU group with the lightest load from the GPU group 1 and the GPU group 2 according to the load conditions of the GPU group 1 and the GPU group 2, and determines the GPU1, the GPU2, the GPU3, and the GPU4 in the GPU group 1 as the target GPU assuming that the GPU group with the lightest load is the GPU group 1.

Step 402, the CPU sends the features to be detected to the target GPU.

Illustratively, if the comparison task of the features to be detected has a high real-time requirement, the CPU directly outputs the features to be detected to each target GPU. And if the comparison task of the features to be detected has lower real-time requirement, the CPU adds the comparison task of the features to be detected into the processing queue. And the CPU sequentially processes the comparison tasks in the processing queue according to the sequence, and outputs the characteristics to be detected to each target GPU when the comparison tasks of the characteristics to be detected are processed. When processing the comparison tasks in the processing queue, the CPU may process a plurality of comparison tasks at a time in a batch manner, that is, output a plurality of features to be detected to the target GPU at a time.

Step 403, the target GPU selects a plurality of first feature models from the first feature library according to the similarity between the feature to be detected and each first feature model in the first feature library, and sends model information of the plurality of first feature models to the CPU. Wherein the model information of the first feature model may comprise a model identification of the first feature model. Or, the model information of the first feature model may include a model identifier of the first feature model, and a similarity between the first feature model and the feature to be detected.

For example, after receiving the feature to be detected, the GPU1 determines the similarity between the feature to be detected and each first feature model (e.g., the first feature model 1-the first feature model 2500) in the first feature library 1, and this determination method is not limited. And selecting S first feature models with high similarity from the first feature library according to the sequence of the similarity from high to low. The value of S can be configured according to experience, and since the number of feature points of the first feature model is small, the value of S can be large instead of small values such as 2 and 3, and certainly, the value of S cannot be too large in order to avoid consuming a large amount of resources of the CPU. For example, S may be 100, and the S first feature models with high similarity may be, for example, first feature model 1 — first feature model 100.

The GPU1 may then send the model information (e.g., model identification, or model identification and similarity) for the S first feature models to the CPU. For example, the model information of the first feature model 1 (such as the model identifier of the first feature model 1, or the similarity between the model identifier of the first feature model 1 and the feature to be detected of the first feature model 1), the model information of the first feature model 2, and so on.

After receiving the features to be detected, the GPU2 refers to the GPU1 in a processing manner, and may also send model information of the S first feature models, that is, model information of the S first feature models in the first feature library 2 to the CPU. Similarly, the GPU3 may also send the model information of the S first feature models, i.e., the model information of the S first feature models in the first feature library 3, to the CPU. The GPU4 may also send the CPU model information of the S first feature models, i.e., the model information of the S first feature models in the first feature library 4.

In step 404, the CPU selects a plurality of second feature models from the second feature library according to the model information sent by the target GPU (i.e., the model information of the first feature models sent by the target GPUs, respectively).

In a first mode, the model information of the first feature model includes a model identifier of the first feature model. And the CPU selects a second characteristic model corresponding to the model identifier sent by the target GPU from a second characteristic library based on the mapping relation between the model identifier of the first characteristic model and the model identifier of the second characteristic model.

For example, the GPU1 sends a first feature model 1 to a model identifier of the first feature model 100 to the CPU, the CPU selects a second feature model 1 corresponding to the model identifier of the first feature model 1 from the second feature library, selects a second feature model 2 corresponding to the model identifier of the first feature model 2 from the second feature library, and so on, and finally selects the second feature model 1 to the second feature model 100 from the second feature library.

The GPU2 sends the first feature model 2501 — the model identification of the first feature model 2600 to the CPU, and the CPU selects the second feature model corresponding to these model identifications from the second feature library, that is, selects the second feature model 2501 — the second feature model 2600 from the second feature library. The GPU3 sends the model identification of the first feature model 5001-first feature model 5100 to the CPU, which selects the second feature model 5001-second feature model 5100 from the second feature library. The GPU3 sends the model identification of the first feature model 7501-first feature model 7600 to the CPU, which selects the second feature model 7501-second feature model 7600 from the second feature library.

To this end, the CPU selects 400 second feature models from the second feature library, and performs subsequent processing based on the 400 second feature models.

In a second mode, the model information of the first feature model includes a model identifier of the first feature model, and a similarity between the first feature model and the feature to be detected. And the CPU selects a target model identifier from the model identifiers of the plurality of first characteristic models sent by the target GPU according to the similarity of the plurality of first characteristic models sent by the target GPU. And selecting a second characteristic model corresponding to the target model identification from the second characteristic library based on the mapping relation between the model identification of the first characteristic model and the model identification of the second characteristic model.

Illustratively, for a process of selecting a target model identifier from the model identifiers of the plurality of first feature models, the model identifiers of the plurality of first feature models sent by the target GPU are sorted based on the similarity of the plurality of first feature models sent by the target GPU; and selecting a specified number of model identifications from the model identifications of the first feature models sent by the target GPU as target model identifications based on the sequencing result.

For example, the model identifications of the plurality of first feature models sent by the target GPU may be sorted in order of high similarity, and a specified number of model identifications sorted top may be selected as the target model identifications. Wherein, the determination manner of the specified quantity includes but is not limited to: the specified number is determined according to the processing capability of the CPU, for example, the greater the processing capability of the CPU, the worse the processing capability of the CPU, and the smaller the specified number. Alternatively, the specified number may be determined according to a concurrent output situation of the GPU, for example, if the GPU outputs model information of S first feature models to the CPU, the specified number may be equal to S.

For example, the GPU1 outputs model identifications of the S first feature models, similarities between the S first feature models and the features to be detected to the CPU; the GPU2 outputs model identifications of the S first feature models and the similarity between the S first feature models and the features to be detected to the CPU; the GPU3 outputs model identifications of the S first feature models and the similarity between the S first feature models and the features to be detected to the CPU; the GPU4 outputs the model identifications of the S first feature models, the similarities between the S first feature models and the features to be detected to the CPU.

Based on the similarity corresponding to the model identifications of the 4S first feature models, the CPU may sort the model identifications of the 4S first feature models in order from high to low in similarity, and select a specified number of model identifications sorted in the top order from the model identifications of the 4S first feature models as target model identifications, where the specified number may be S, that is, select S model identifications sorted in the top order from the 4S model identifications as target model identifications. Of course, the above manner is only an example, and is not limited thereto.

Assuming that the target model identifier is a model identifier of the first feature model 1-the first feature model 100, the CPU selects a second feature model 1 corresponding to the model identifier of the first feature model 1 from the second feature library, selects a second feature model 2 corresponding to the model identifier of the first feature model 2 from the second feature library, and so on, and finally selects the second feature model 1-the second feature model 100 from the second feature library. At this point, 100 second feature models are selected from the second feature library, and subsequent processing is performed based on the 100 second feature models.

In summary, based on the first or second manner, the CPU may select a plurality of second feature models from the second feature library, and the number of the selected second feature models is far smaller than the total number of the second feature models. For example, 100 second feature models are selected from the second feature library, and the total number of the second feature models is 10000.

Step 405, based on a plurality of second feature models selected from the second feature library, the CPU determines at least one target feature model from the plurality of second feature models according to the similarity between the feature to be detected and each selected second feature model, where the target feature model is the final feature comparison result.

For example, the CPU determines the similarity between the feature to be detected and each of the selected second feature models (e.g., 100 second feature models), and the determination method is not limited. And sequencing each second feature model according to the sequence of the similarity from high to low, and selecting T second feature models which are sequenced at the top, namely T second feature models with high similarity, wherein the T second feature models are target feature models.

The value of T may be configured empirically to indicate that T feature comparison results are finally output, for example, T may be 1, 2, 3, and the like, which is not limited thereto. And if T is 3, the CPU selects 3 second feature models ranked in the front, namely the feature comparison result is the model identification of the 3 second feature models ranked in the front.

In the above embodiment, the CPU selects T second feature models ranked in the top from a plurality of second feature models (e.g., 100 second feature models), instead of selecting T second feature models ranked in the top from all second feature models (e.g., 10000 second feature models) in the second feature library, and since the number of 100 second feature models is much smaller than the total number of second feature models, in the above manner, by loading a small model with a smaller number of feature points on the GPU and loading a large model with a larger number of feature points on a designated storage medium, while reducing the CPU computation amount, the alignment accuracy is ensured, and the capacity and concurrence of the feature library are improved (concurrence means how many features to be detected can be compared with feature models per second on average).

Based on the same application concept as the method, the embodiment of the present application further provides a feature comparison apparatus, as shown in fig. 5, which is a structural diagram of the feature comparison apparatus, and the apparatus includes: a determining module 51, configured to determine, from all GPUs, at least one target GPU for processing the features to be detected;

a sending module 52, configured to send the feature to be detected to the target GPU, so that the target GPU sends model information of the plurality of first feature models in the first feature library to the CPU according to the similarity between the feature to be detected and each first feature model in the first feature library;

a processing module 53, configured to select multiple second feature models from a second feature library according to the model information sent by the target GPU; determining at least one target feature model from the plurality of second feature models according to the similarity between the feature to be detected and each selected second feature model;

In an example, the processing module 53 is specifically configured to, when selecting a plurality of second feature models from a second feature library according to the model information sent by the target GPU:

if the model information comprises model identifications and a mapping relation exists between the model identification of the first characteristic model and the model identification of the second characteristic model corresponding to the same sample characteristic, selecting a second characteristic model corresponding to the model identification sent by the target GPU from a second characteristic library based on the mapping relation between the model identification of the first characteristic model and the model identification of the second characteristic model; or if the model information comprises model identifications and similarity, and the model identifications of the first feature models and the model identifications of the second feature models corresponding to the same sample feature have a mapping relation, selecting a target model identification from the model identifications of the first feature models sent by the target GPU according to the similarity of the first feature models sent by the target GPU; and selecting a second characteristic model corresponding to the target model identification from a second characteristic library based on the mapping relation between the model identification of the first characteristic model and the model identification of the second characteristic model.

In an example, the processing module 53 is specifically configured to, when selecting the target model identifier from the model identifiers of the plurality of first feature models sent by the target GPU according to the similarity of the plurality of first feature models sent by the target GPU: based on the similarity of the plurality of first feature models sent by the target GPU, sorting the model identifications of the plurality of first feature models sent by the target GPU; and selecting a specified number of model identifications from the model identifications of the first feature models sent by the target GPU as target model identifications based on the sequencing result.

The determining module 51 is further configured to obtain a first feature model and a second feature model corresponding to each sample feature; determining the number M of GPUs (graphics processing units) required to be occupied by all the first feature models according to the total number N of all the first feature models and a set threshold K; grouping all GPUs, wherein the number of the GPUs in each GPU group is more than or equal to M; grouping all the first feature models to obtain M first feature libraries, storing the M first feature libraries to GPUs in each GPU group, wherein each GPU stores one first feature library, the number of the first feature models in each first feature library is less than or equal to K, and K is the maximum value of the number of the first feature models stored by the GPU; all the second feature models are grouped into a second feature library, and the second feature library is stored to a specified storage medium.

The determining module 51 is specifically configured to, when determining at least one target GPU for processing the features to be detected from all GPUs: selecting a GPU group with the lightest load from all GPU groups according to the load condition of each GPU group; and determining all GPUs in the GPU group with the lightest load as the target GPU.

Based on the same application concept as the method, a feature comparison device is further provided in the embodiment of the present application, and from a hardware level, a schematic diagram of a hardware architecture of the feature comparison device provided in the embodiment of the present application may be as shown in fig. 6. The feature mapping apparatus may include: a CPU61, a machine-readable storage medium 62, and a GPU, the machine-readable storage medium 62 storing machine-executable instructions executable by the CPU 61; the CPU61 is configured to execute machine-executable instructions to implement the methods disclosed in the above examples of the present application. For example, the CPU61 is configured to execute machine-executable instructions to perform the following steps:

Based on the same application concept as the method, embodiments of the present application further provide a machine-readable storage medium, where several computer instructions are stored on the machine-readable storage medium, and when the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

For example, the computer instructions, when executed by a processor, enable the following steps:

The machine-readable storage medium may be, for example, any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: a RAM (random Access Memory), a volatile Memory, a non-volatile Memory, a flash Memory, a storage drive (e.g., a hard drive), a solid state drive, any type of storage disk (e.g., an optical disk, a dvd, etc.), or similar storage medium, or a combination thereof.

Based on the same application concept as the method, the embodiment of the present application further provides a feature comparison device, which may include but is not limited to a CPU and a GPU, wherein:

the CPU is used for determining at least one target GPU for processing the characteristics to be detected from all GPUs; sending the features to be detected to the target GPU;

the GPU is used for sending the model information of the plurality of first feature models in the first feature library to the CPU according to the similarity between the feature to be detected and each first feature model in the first feature library;

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Furthermore, these computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A feature comparison method is applied to a CPU, and comprises the following steps:

2. The method according to claim 1, wherein before determining at least one target GPU from all GPUs for processing the feature to be detected, the method further comprises:

acquiring a first characteristic model and a second characteristic model corresponding to each sample characteristic;

determining the number M of GPUs (graphics processing units) required to be occupied by all the first feature models according to the total number N of all the first feature models and a set threshold K; grouping all GPUs, wherein the number of the GPUs in each GPU group is more than or equal to M; grouping all the first feature models to obtain M first feature libraries, storing the M first feature libraries to GPUs in each GPU group, wherein each GPU stores one first feature library, the number of the first feature models in each first feature library is less than or equal to K, and K is the maximum value of the number of the first feature models stored by the GPU;

all second feature models are grouped into a second feature library, and the second feature library is stored to a designated storage medium.

3. The method of claim 2,

the determining of at least one target GPU for processing the features to be detected from all GPUs comprises the following steps:

selecting a GPU group with the lightest load from all GPU groups according to the load condition of each GPU group;

and determining all GPUs in the GPU group with the lightest load as the target GPU.

4. The method of claim 2,

the model information includes model identification, a mapping relationship exists between the model identification of the first feature model and the model identification of the second feature model corresponding to the same sample feature, and the selecting of the plurality of second feature models from the second feature library according to the model information sent by the target GPU includes:

and selecting a second feature model corresponding to the model identifier sent by the target GPU from a second feature library based on the mapping relation between the model identifier of the first feature model and the model identifier of the second feature model.

5. The method of claim 2,

the model information includes model identification and similarity, a mapping relationship exists between the model identification of the first feature model and the model identification of the second feature model corresponding to the same sample feature, and the selecting of the plurality of second feature models from the second feature library according to the model information sent by the target GPU includes:

selecting a target model identifier from the model identifiers of the plurality of first characteristic models sent by the target GPU according to the similarity of the plurality of first characteristic models sent by the target GPU;

and selecting a second characteristic model corresponding to the target model identification from a second characteristic library based on the mapping relation between the model identification of the first characteristic model and the model identification of the second characteristic model.

6. The method of claim 5,

selecting a target model identifier from the model identifiers of the plurality of first feature models sent by the target GPU according to the similarity of the plurality of first feature models sent by the target GPU, wherein the selecting comprises the following steps:

based on the similarity of the plurality of first feature models sent by the target GPU, sorting the model identifications of the plurality of first feature models sent by the target GPU;

and selecting a specified number of model identifications from the model identifications of the first feature models sent by the target GPU as target model identifications based on the sequencing result.

7. A feature matching apparatus, comprising:

the determining module is used for determining at least one target GPU for processing the characteristics to be detected from all GPUs;

8. The apparatus according to claim 7, wherein the processing module, when selecting the plurality of second feature models from the second feature library according to the model information sent by the target GPU, is specifically configured to:

if the model information comprises model identifications and a mapping relation exists between the model identification of the first characteristic model and the model identification of the second characteristic model corresponding to the same sample characteristic, selecting a second characteristic model corresponding to the model identification sent by the target GPU from a second characteristic library based on the mapping relation between the model identification of the first characteristic model and the model identification of the second characteristic model; alternatively, the first and second electrodes may be,

if the model information comprises model identifications and similarity, and the model identifications of the first characteristic models and the model identifications of the second characteristic models corresponding to the same sample characteristics have a mapping relation, selecting target model identifications from the model identifications of the first characteristic models sent by the target GPU according to the similarity of the first characteristic models sent by the target GPU; and selecting a second characteristic model corresponding to the target model identification from a second characteristic library based on the mapping relation between the model identification of the first characteristic model and the model identification of the second characteristic model.

9. A feature comparison device, comprising: a CPU and a machine-readable storage medium storing machine-executable instructions executable by the CPU;

10. A feature comparison device, comprising a CPU and a GPU, wherein: