CN112232360A - Image retrieval model optimization method, image retrieval device and storage medium - Google Patents
Image retrieval model optimization method, image retrieval device and storage medium Download PDFInfo
- Publication number
- CN112232360A CN112232360A CN202011066003.7A CN202011066003A CN112232360A CN 112232360 A CN112232360 A CN 112232360A CN 202011066003 A CN202011066003 A CN 202011066003A CN 112232360 A CN112232360 A CN 112232360A
- Authority
- CN
- China
- Prior art keywords
- image
- sample
- feature
- model
- image feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005457 optimization Methods 0.000 title claims abstract description 83
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000012549 training Methods 0.000 claims abstract description 121
- 238000000605 extraction Methods 0.000 claims abstract description 108
- 238000012545 processing Methods 0.000 claims description 34
- 238000010606 normalization Methods 0.000 claims description 30
- 238000004590 computer program Methods 0.000 claims description 29
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 26
- 230000009466 transformation Effects 0.000 claims description 25
- 230000006870 function Effects 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 6
- 238000003062 neural network model Methods 0.000 description 6
- 239000013598 vector Substances 0.000 description 6
- 239000010410 layer Substances 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000005094 computer simulation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007850 degeneration Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application relates to an image retrieval model optimization method, an image retrieval device and a storage medium, wherein the method obtains an image feature set corresponding to a training data set by obtaining a first image feature of each sample image in the training data set, performs image feature extraction on any sample image in the training data set through an image retrieval model to be optimized to obtain a second image feature of the sample image, performs image feature extraction on the sample image through an image feature extraction model to obtain a third image feature of the sample image, determines the loss of model optimization according to the second image feature of the sample image, the third image feature and the image feature set corresponding to the training data set, obtains an optimized image retrieval model according to the loss, and enables the image retrieval model to be optimized to really learn the relation information between the sample images, and the optimization of the image retrieval model to be optimized is completed, so that the image retrieval model with better performance is obtained.
Description
Technical Field
The present application relates to the field of deep learning technologies, and in particular, to an image retrieval model optimization method, an image retrieval device, and a storage medium.
Background
With the rise of deep learning technology, the development of the field of Image retrieval (Image retrieval) is greatly promoted, and for extracting Image features, a neural network has irreplaceable advantages at present, so that the neural network is increasingly used in the field of Image retrieval which is extremely dependent on the quality of the Image features.
In the related art, knowledge in some metric learning fields is also increasingly used to improve image retrieval models. It is intuitively understood that for image samples of the same class, the distance in feature space should be closer, and for image samples of different classes, the distance in feature space should be farther. However, the current methods for improving the image retrieval model are usually limited to the number of single batch (samples), and the concerned feature relationship is limited to the current iteration (number of iterations) of the network and the current batch. However, since the image retrieval task usually has a large number of categories and a small number of image samples of each category, the image retrieval model improved by the current method is not global.
Disclosure of Invention
In view of the above, it is necessary to provide an image retrieval model optimization method, an image retrieval device, and a storage medium, which can have globality, for solving the problem that the conventional method for improving the image retrieval model does not have globality.
A method of image retrieval model optimization, the method comprising:
acquiring a first image feature of each sample image in a training data set to obtain an image feature set corresponding to the training data set, wherein the training data set comprises a plurality of sample images;
for any sample image in the training data set, carrying out image feature extraction through an image retrieval model to be optimized to obtain a second image feature of the sample image;
performing image feature extraction on the sample image through an image feature extraction model to obtain a third image feature of the sample image, wherein the image feature extraction model is obtained after momentum updating is performed on the basis of the image retrieval model to be optimized;
determining the loss of model optimization according to the second image feature and the third image feature of the sample image and the image feature set corresponding to the training data set;
and carrying out back propagation on the image retrieval model to be optimized according to the loss to obtain the optimized image retrieval model.
In one embodiment, the determining a loss of model optimization according to the second image feature, the third image feature and the image feature set corresponding to the training data set of the sample image includes: according to the third image feature of the sample image, updating the first image feature corresponding to the sample image in the image feature set to obtain an updated image feature set; determining a loss of model optimization according to the updated image feature set and the second image features of the sample image.
In one embodiment, the determining a loss of model optimization according to the updated image feature set and the second image feature of the sample image includes: acquiring positive example features and negative example features corresponding to the sample image based on the updated image feature set; and calculating the loss of model optimization according to the positive example feature and the negative example feature corresponding to the sample image and the second image feature of the sample image.
In one embodiment, the obtaining positive and negative example features corresponding to the sample image based on the updated image feature set includes: acquiring an updated first image feature corresponding to the sample image in the updated image feature set, and determining the updated first image feature as a normal feature corresponding to the sample image; and acquiring other first image features except the positive example features corresponding to the sample image in the updated image feature set, and determining the first image features as the negative example features corresponding to the sample image.
In one embodiment, the calculating a loss of model optimization according to the positive example feature, the negative example feature and the second image feature of the sample image includes: according to the positive example feature, the negative example feature and the second image corresponding to the sample imageCharacterized by calculating the model-optimized loss using an information noise convergence estimation loss function
Wherein q is a second image feature corresponding to the sample image, t+Is the corresponding positive example feature of the sample image, ti-And the number of the ith negative example features corresponding to the sample image is K, and tau is a hyperparameter.
In one embodiment, the obtaining the first image feature of each sample image in the training data set to obtain an image feature set corresponding to the training data set includes: performing image feature extraction on each sample image in the training data set by adopting an initial image feature extraction model to obtain a first image feature of each sample image; based on the first image features of each sample image in the training data set, an image feature set corresponding to the training data set is generated.
In one embodiment, the performing, by using an image retrieval model to be optimized, image feature extraction on any sample image in the training data set to obtain a second image feature of the sample image includes: for any sample image in the training data set, inputting the sample image into an image retrieval model to be optimized to obtain a second original image characteristic corresponding to the sample image; and carrying out affine transformation and norm normalization processing on the second original image characteristics to obtain the correspondingly processed second image characteristics.
In one embodiment, the performing, by an image feature extraction model, image feature extraction on the sample image to obtain a third image feature of the sample image includes: momentum updating is carried out on the image feature extraction model based on the image retrieval model to be optimized, and an updated image feature extraction model is obtained; inputting the sample image into an image feature extraction model after momentum updating to obtain a third original image feature corresponding to the sample image; and carrying out affine transformation and norm normalization processing on the third original image characteristics to obtain correspondingly processed third image characteristics.
An image retrieval method, the method comprising:
acquiring an image to be retrieved;
and inputting the image to be retrieved into the optimized image retrieval model obtained by the image retrieval model optimization method to obtain a corresponding target image.
An image retrieval model optimization apparatus, the apparatus comprising:
the image feature set acquisition module is used for acquiring a first image feature of each sample image in a training data set to obtain an image feature set corresponding to the training data set, wherein the training data set comprises a plurality of sample images;
the second image feature extraction module is used for extracting image features of any sample image in the training data set through an image retrieval model to be optimized to obtain second image features of the sample image;
the third image feature extraction module is used for extracting image features of the sample image through an image feature extraction model to obtain third image features of the sample image, and the image feature extraction model is obtained after momentum updating is carried out on the basis of the image retrieval model to be optimized;
the loss obtaining module is used for determining the loss of model optimization according to the second image characteristic and the third image characteristic of the sample image and the image characteristic set corresponding to the training data set;
and the model optimization module is used for performing back propagation on the image retrieval model to be optimized according to the loss to obtain the optimized image retrieval model.
An image retrieval apparatus, the apparatus comprising:
the image to be retrieved acquiring module is used for acquiring an image to be retrieved;
and the retrieval processing module is used for inputting the image to be retrieved into the optimized image retrieval model obtained by the image retrieval model optimizing method to obtain a corresponding target image.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the method as described above when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method as set forth above.
The image retrieval model optimization method, the image retrieval device and the storage medium obtain the image feature set corresponding to the training data set by obtaining the first image feature of each sample image in the training data set, extract the image feature of any sample image in the training data set by the image retrieval model to be optimized to obtain the second image feature of the sample image, extract the image feature of the sample image by the image feature extraction model to obtain the third image feature of the sample image, determine the loss of model optimization according to the second image feature of the sample image, the third image feature and the image feature set corresponding to the training data set, perform back propagation on the image retrieval model to be optimized according to the loss to obtain the optimized image retrieval model, thereby enabling the image retrieval model to be optimized to really learn the relationship information between the sample images, and the optimization of the image retrieval model to be optimized is completed, so that the image retrieval model with better performance is obtained.
Drawings
FIG. 1 is a schematic flow chart diagram illustrating a method for optimizing an image retrieval model according to one embodiment;
FIG. 2 is a flowchart illustrating the step of obtaining a feature set of an image according to one embodiment;
FIG. 3 is a flowchart illustrating the step of obtaining second image features in one embodiment;
FIG. 4 is a flowchart illustrating the step of obtaining a third image feature according to one embodiment;
FIG. 5 is a schematic flow chart diagram illustrating the loss step of determining model optimization in one embodiment;
FIG. 6 is a schematic flow chart diagram illustrating the penalty step of computational model optimization in one embodiment;
FIG. 7 is a flowchart illustrating a method for optimizing an image retrieval model according to another embodiment;
FIG. 8 is a flowchart illustrating an image retrieval method according to one embodiment;
FIG. 9 is a block diagram showing the configuration of an image search model optimizing apparatus according to an embodiment;
FIG. 10 is a block diagram showing the configuration of an image search device according to an embodiment;
FIG. 11 is a diagram illustrating an internal structure of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In one embodiment, there is provided an image retrieval model optimization method, as shown in fig. 1, including the following steps:
The training data set comprises a plurality of sample images, and is sample data used for optimizing an image retrieval model to be optimized so that retrieval effect can be improved. And the image retrieval model to be optimized may be any neural network model for image retrieval in conventional techniques. The first image feature is an image feature result obtained by performing image feature extraction on a sample image in the training data set. In this embodiment, when a certain image retrieval model needs to be optimized, a training data set for model optimization needs to be obtained first, and then image feature extraction is performed on each sample image in the training data set, so as to obtain first image features corresponding to each sample image, and an image feature set corresponding to the training data set is obtained based on the first image features corresponding to each sample image in the training data set one to one, that is, the image feature set includes the first image features of each sample image.
And step 120, for any sample image in the training data set, performing image feature extraction through an image retrieval model to be optimized to obtain a second image feature of the sample image.
The image retrieval model to be optimized may be any neural network model for performing image retrieval in the conventional technology, for example, the neural network model may be a neural network model for performing a graph search in a search engine (Google, hundredth), a neural network model for performing similar commodity search in an e-commerce website (naobao, Amazon, ebay), a neural network model for performing similar content recommendation in a social platform (Pinterest), and the like. And the second image characteristic is an image characteristic result obtained after the sample image passes through the image retrieval model to be optimized. Specifically, in this embodiment, for any sample image in the training data set, the sample image is input into the image retrieval model to be optimized, so as to obtain the second image feature corresponding to the sample image.
And step 130, performing image feature extraction on the sample image through the image feature extraction model to obtain a third image feature of the sample image.
The image feature extraction model is obtained by momentum updating based on the image retrieval model to be optimized. And the third image characteristic is an image characteristic result obtained after the sample image passes through the image characteristic extraction model. Specifically, for any sample image in the training data set (i.e., the sample image in step 120), the sample image is input into an image feature extraction model obtained by momentum update based on the image retrieval model to be optimized, so as to obtain a third image feature corresponding to the sample image.
And 140, determining the loss of model optimization according to the second image characteristic and the third image characteristic of the sample image and the image characteristic set corresponding to the training data set.
The loss of model optimization represents the relationship information between the sample images, and is used for optimizing the image retrieval model to be optimized. In this embodiment, the loss of model optimization is determined by the second image feature and the third image feature corresponding to any sample image in the training data set and the image feature set corresponding to the training data set, and then the model parameter of the image retrieval model to be optimized is adjusted based on the loss of model optimization, so that the image retrieval model to be optimized can really learn the relationship information between the sample images, thereby completing the optimization of the image retrieval model to be optimized and improving the retrieval effect of the optimized image retrieval model.
And 150, reversely propagating the image retrieval model to be optimized according to the loss to obtain the optimized image retrieval model.
Specifically, after the loss of the model optimization is obtained through the above steps, the image retrieval model to be optimized may be subjected to one-time back propagation according to the loss, and the calculation of one-time iterative training of the image retrieval model to be optimized is completed, so as to update the model parameters of the image retrieval model to be optimized. It can be understood that, for each sample image in the training data set, the processing may be performed according to the method in steps 120 to 150, so that the image retrieval model to be optimized can really learn the relationship information between the sample images, so as to complete the optimization of the image retrieval model to be optimized, and improve the retrieval effect of the optimized image retrieval model.
The image retrieval model optimization method comprises the steps of obtaining an image feature set corresponding to a training data set by obtaining a first image feature of each sample image in the training data set, carrying out image feature extraction on any sample image in the training data set through an image retrieval model to be optimized to obtain a second image feature of the sample image, carrying out image feature extraction on the sample image through an image feature extraction model to obtain a third image feature of the sample image, determining the loss of model optimization according to the second image feature and the third image feature of the sample image and the image feature set corresponding to the training data set, carrying out back propagation on the image retrieval model to be optimized according to the loss to obtain the optimized image retrieval model, and enabling the image retrieval model to be optimized to really learn the relation information between the sample images, and the optimization of the image retrieval model to be optimized is completed, so that the image retrieval model with better performance is obtained.
In an embodiment, as shown in fig. 2, in step 110, obtaining a first image feature of each sample image in the training data set to obtain an image feature set corresponding to the training data set may specifically include the following steps:
and step 111, performing image feature extraction on each sample image in the training data set by using an initial image feature extraction model to obtain a first image feature of each sample image.
In this embodiment, the structure of the image retrieval model to be optimized is given, and the image retrieval model to be optimized is initialized based on ImageNet (a large visual database for visual object recognition software research) pre-training parameters, so as to obtain the image retrieval model to be optimized, and the initial image feature extraction model is the same as the image retrieval model to be optimized. In this embodiment, each sample image in the training data set is input into the initial image feature extraction model, so as to obtain a first image feature corresponding to each sample image.
Specifically, image feature extraction is performed on each sample image in the training data set respectively, so that first image features corresponding to each sample image are obtained, and an image feature set corresponding to the training data set is obtained based on the first image features corresponding to each sample image in the training data set one to one.
In the above embodiment, an initial image feature extraction model is adopted to perform image feature extraction on each sample image in the training data set to obtain the first image feature of each sample image, and an image feature set corresponding to the training data set is generated based on the first image feature of each sample image in the training data set.
In an embodiment, as shown in fig. 3, in step 120, for any sample image in the training data set, image feature extraction is performed through an image retrieval model to be optimized to obtain a second image feature of the sample image, which may specifically include the following steps:
and step 121, inputting the sample image into an image retrieval model to be optimized for any sample image in the training data set to obtain a second original image characteristic corresponding to the sample image.
The image retrieval model to be optimized generally comprises a backbone network and a full-connection layer norm normalization part, and can be updated according to normal gradient. And the second original image characteristic is the output of the sample image after passing through the backbone network of the image retrieval model to be optimized, namely the sample image passes through the backbone network of the image retrieval model to be optimized and is not subjected to full-connection layer and norm normalization processing to obtain the original image characteristic. Specifically, for any sample image in the training data set, the sample image is input into the image retrieval model to be optimized, so that a second original image feature corresponding to the sample image is obtained.
And step 122, performing affine transformation and norm normalization processing on the second original image characteristics to obtain correspondingly processed second image characteristics.
Affine transformation, also called affine mapping, is a process of performing linear transformation on one vector space and then translating the vector space into another vector space in geometry. The norm normalization processing can be processed in a L2 norm normalization mode, L2 norm normalization is to divide each dimension data X1, X2, …, xn of the vector X by | | | | |2 (namely L2 norm) to obtain a new vector, the influence of units between features can be eliminated through L2 norm normalization processing, and the convergence speed of the learning model is improved. In this embodiment, the dimension of the second original image feature output through the backbone network of the image retrieval model to be optimized is made to coincide with the dimension of the first original image feature output through the initial image feature extraction model by performing affine transformation on the second original image feature. And further carrying out norm normalization processing on the second original image characteristics after affine transformation, thereby obtaining the second image characteristics after corresponding processing.
In the above embodiment, for any sample image in the training data set, the second original image feature corresponding to the sample image is obtained by inputting the sample image into the image retrieval model to be optimized, and affine transformation and norm normalization processing are performed on the second original image feature to obtain the second image feature after corresponding processing, so that the image retrieval model to be optimized can learn the relationship information between the sample images.
In an embodiment, as shown in fig. 4, in step 130, performing image feature extraction on the sample image through an image feature extraction model to obtain a third image feature of the sample image, which may specifically include the following steps:
and 131, momentum updating is carried out on the image feature extraction model based on the image retrieval model to be optimized, and the updated image feature extraction model is obtained.
The initial image feature extraction model is the same as the initial image retrieval model to be optimized in a same mode and comprises a backbone network and a full-connection layer norm normalization part, specifically, the image feature extraction model and the initial image retrieval model to be optimized adopt the same network structure and initialization parameters, the image retrieval model to be optimized can be updated according to normal gradient, and the image feature extraction model can adopt the image retrieval model to be optimizedThe number of iterations (iteration) is updated by a running average (i.e., momentum update). Specifically, if the model parameter of the image retrieval model to be optimized is θqThe model parameter of the image feature extraction model before momentum updating is thetakThen thetak+1＝mθk+(1-m)θqWherein, thetak+1And extracting model parameters of the model for the image features after momentum updating, wherein m is the momentum size which can be specifically a super parameter preset before network training. In this embodiment, momentum update is performed on the image feature extraction model based on the image retrieval model to be optimized, so as to obtain an updated image feature extraction model.
Similarly, the third original image feature is output of the sample image after passing through the backbone network of the momentum-updated image feature extraction model, that is, the original image feature obtained by the sample image after passing through the backbone network of the momentum-updated image feature extraction model without being subjected to full-link layer and norm normalization processing. Specifically, the sample image (i.e., the sample image determined in step 121) is input into the momentum-updated image feature extraction model, so as to obtain a third original image feature corresponding to the sample image.
And step 133, performing affine transformation and norm normalization processing on the third original image feature to obtain a correspondingly processed third image feature.
Specifically, affine transformation and norm normalization processing are performed on the third original image feature, so that a correspondingly processed third image feature is obtained. The affine transformation and norm normalization processing may adopt the same method as that in step 122, which is not described again in this embodiment.
In the above embodiment, the momentum of the image feature extraction model is updated based on the image retrieval model to be optimized to obtain an updated image feature extraction model, the sample image is input into the momentum-updated image feature extraction model to obtain a third original image feature corresponding to the sample image, affine transformation and norm normalization processing are performed on the third original image feature to obtain a third image feature which has the same dimensionality as the first image feature and is subjected to corresponding processing, so that the image retrieval model to be optimized can learn the relationship features between the sample images.
In an embodiment, as shown in fig. 5, in step 140, determining a loss of model optimization according to the second image feature and the third image feature of the sample image and the image feature set corresponding to the training data set may specifically include the following steps:
and step 141, updating the first image feature corresponding to the sample image in the image feature set according to the third image feature of the sample image to obtain an updated image feature set.
The image feature set comprises first image features of each sample image, and the first image features are obtained by performing image feature extraction on each sample image in the training data set by adopting an initial image feature extraction model. And the third image characteristic is obtained by adopting an image characteristic extraction model after momentum updating to extract the image characteristic of a certain sample image in the training data set. Therefore, in this embodiment, based on the third image feature of a certain sample image, the first image feature corresponding to the sample image in the image feature set is updated, that is, the first image feature corresponding to the sample image in the image feature set is updated to the third image feature of the sample image, so as to obtain an updated image feature set, thereby ensuring consistency and slowness of features in the image feature set.
And 142, determining the loss of model optimization according to the updated image feature set and the second image feature of the sample image.
Specifically, the loss of model optimization is determined according to the updated image feature set and the second image features of the sample images, and then the model parameters of the image retrieval model to be optimized are adjusted based on the loss of model optimization, so that the image retrieval model to be optimized can really learn the relationship information between the sample images, and the optimization of the image retrieval model to be optimized is completed.
In an embodiment, as shown in fig. 6, in step 142, determining a loss of model optimization according to the updated image feature set and the second image feature of the sample image may specifically include the following steps:
The positive example feature refers to a first image feature corresponding to any sample image in the training data set in the updated image feature set, and the negative example feature is other first image features except the positive example feature corresponding to the sample image in the updated image feature set. That is, for any sample image in the training data set, the corresponding positive example feature is the first image feature corresponding to the sample image in the updated image feature set, and the negative example feature is all the first image features except the positive example feature corresponding to the sample image in the updated image feature set. Based on the positive example feature and the negative example feature corresponding to any sample image in the training data set can be determined.
Specifically, the present embodiment may adopt an information Noise convergence Estimation Loss function (info Noise-convergence Estimation Loss) to calculate the Loss of model optimization, wherein the information Noise convergence Estimation Loss function
Where q is the second image feature corresponding to the sample image (i.e., the sample image determined in step 121), and t is+For the corresponding positive example feature of the sample image, ti-And K is the number of the negative example features, and tau is a settable hyper-parameter.
As shown in fig. 7, the image retrieval model optimization method in the present application is further described below based on fig. 7, wherein a backbone network (i.e., a backbone network) takes googlenet as an example, and a pre-training parameter on Imagenet is given for initialization, and the specific scheme includes the following steps:
1. as shown in fig. 7, model _ q (701 in the figure) is an image retrieval model to be optimized, which includes two parts, namely, a backbone _ q (googlelenet) and a head _ q (single-layer full-connected layer plus L2 norm normalization), and model _ k is an image feature extraction model, which may also be called a momentum encoder (momentum encoder), and is a network which is the same as model _ q, including two parts, namely, backbone _ k and head _ k. Initially, the model _ k is the same initialization parameter as the model _ q, the model _ q is updated according to the normal gradient, and the model _ k is directly updated by the running average of each iteration number (iteration) of the model _ q parameter, and the specific updating method is shown as follows:
θk+1＝mθk+(1-m)θqwherein, thetak+1Model parameters, θ, for the momentum updated model _ kkModel parameters, θ, for model _ k before momentum updateqThe model parameter is model _ q, and m is the momentum size, which may be a hyper-parameter preset before network training.
2. Acquiring a training data set for model optimization, assuming that the training data set contains N sample images, and inputting each sample image into an initial model _ k to obtain a corresponding first image feature, namely f1,f2,…,fNBased on the N image features, obtaining a corresponding image feature set fiAnd the size of the Memory Bank is N multiplied by D, wherein N is the number of elements in the set, and D is a characteristic dimension.
3. For model _ q, for any sample image m in the training dataset, it outputs the corresponding feature (i.e. the second original image feature) through background _ q, if it is marked as fmThen f ism∈RdAnd d is the characteristic dimension.
In the present embodiment, by pairing fmAffine transformation is performed so that fmHaving a and fiSame feature dimensionIn this embodiment, a specific affine transformation method is as follows:
qm′＝Wfm+ b; wherein W ∈ Rd×DW represents a parameter matrix of affine transformation, b is an offset, d is fmD is a pair fmDimension of object feature for affine transformation, fmIs the second original image characteristic obtained after the sample image m is output by model _ q, qmIs' as a pair of fmAnd carrying out affine transformation to obtain corresponding image characteristics.
And then q ism' normalization processing of L2 norm is carried out (affine transformation and normalization processing of L2 norm are both completed by head _ q in the figure), so that a second image characteristic after corresponding processing, namely q, is obtained. The L2 norm normalization processing procedure is as follows:
4. Similarly, for the model _ k network, the same sample image m outputs a feature k (i.e., a third original image feature) through a backbone _ k and a head _ k, and if the feature k is recorded as k, the corresponding feature stored in the memory bank of the sample image m is updated according to the index (index) of the sample image m, so that the updated memory bank is obtained.
5. For sample image m, it corresponds to the first image feature t in Memory Banki+For positive example characteristics (for ease of explanation, t is used in the following formula)+Representation) and other first image features in Memory Bank
The negative example feature corresponding to the sample image m is obtained (for the sake of convenience, t is used in the following formulai-Expressed), where K is the number of negative example features.
6. For the sample image m, the infoNCE loss corresponding to the sample image is obtained based on the positive and negative features in the corresponding Memory Bank obtained as follows:
Based on this, for an arbitrary sample image n, info loss is obtained based on the positive and negative case features in the Memory Bank:
7. And the Loss feedback updates the network parameters of the model _ q according to the gradient.
The image retrieval model optimization method provided by the embodiment stores feature vectors of a large number of sample images by introducing the Memory Bank, solves the problem of insufficient number of sample images in a single Bank, extracts relationship information among samples by using infoNCE loss which is excellent in representation and learning tasks, ensures feature multiplexing by using the Memory Bank, provides a large number of negative samples, ensures consistency and degeneration of features in the Memory Bank by using a momentum encoder, and overcomes the defect that the features in the original Memory Bank come from different iterations and different samples, thereby greatly improving the effect of optimizing the image retrieval model.
In one embodiment, as shown in fig. 8, there is provided an image retrieval method including the steps of:
The image to be retrieved is an image to be subjected to image retrieval, and specifically, the image retrieval includes, but is not limited to, image retrieval performed by searching images in a search engine, image retrieval performed by searching similar commodities in an e-commerce website, image retrieval performed by recommending similar contents in a social platform, and the like.
And step 820, inputting the image to be retrieved into the image retrieval model to obtain a corresponding target image.
The image retrieval model is an optimized image retrieval model obtained by the image retrieval model optimization method described in fig. 1 to 7. It can be understood that, after the image retrieval model is obtained by the image retrieval model optimization method described in fig. 1 to fig. 7, the image retrieval model may be trained based on a scene related to image retrieval in practical application, so that features in a specific scene can be learned, and the image retrieval is performed on the image to be retrieved by the trained image retrieval model, so as to obtain a target image whose similarity to the image to be retrieved reaches a target value, where the target value may be set according to the practical application scene.
Since the image retrieval model for performing image retrieval is obtained by the image retrieval model optimization method described in fig. 1 to 7, the image retrieval model can really learn the relationship characteristics between sample images, so as to achieve the effect of optimizing the image retrieval model, and when the image retrieval model is trained and used for a specific image retrieval task, the effect of the specific image retrieval task can be improved.
It should be understood that although the various steps in the flow charts of fig. 1-8 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-8 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 9, there is provided an image retrieval model optimization apparatus including: an image feature set obtaining module 901, a second image feature extracting module 902, a third image feature extracting module 903, a loss obtaining module 904, and a model optimizing module 905, wherein:
an image feature set obtaining module 901, configured to obtain a first image feature of each sample image in a training data set, to obtain an image feature set corresponding to the training data set, where the training data set includes a plurality of sample images;
a second image feature extraction module 902, configured to perform image feature extraction on any sample image in the training data set through an image retrieval model to be optimized, to obtain a second image feature of the sample image;
a third image feature extraction module 903, configured to perform image feature extraction on the sample image through an image feature extraction model to obtain a third image feature of the sample image, where the image feature extraction model is obtained after momentum update is performed on the basis of the image retrieval model to be optimized;
a loss obtaining module 904, configured to determine a loss of model optimization according to the second image feature and the third image feature of the sample image and the image feature set corresponding to the training data set;
and the model optimization module 905 is configured to perform back propagation on the image retrieval model to be optimized according to the loss to obtain an optimized image retrieval model.
In one embodiment, the loss acquisition module includes: the updating unit is used for updating the first image feature corresponding to the sample image in the image feature set according to the third image feature of the sample image to obtain an updated image feature set; and the loss determining unit is used for determining the loss of model optimization according to the updated image feature set and the second image feature of the sample image.
In one embodiment, the loss determination unit includes: the characteristic determining subunit is used for acquiring positive example characteristics and negative example characteristics corresponding to the sample image based on the updated image characteristic set; and the loss calculation subunit is used for calculating the loss of model optimization according to the positive example feature and the negative example feature corresponding to the sample image and the second image feature of the sample image.
In one embodiment, the feature determination subunit is specifically configured to: acquiring an updated first image feature corresponding to the sample image in the updated image feature set, and determining the updated first image feature as a normal feature corresponding to the sample image; and acquiring other first image features except the positive example features corresponding to the sample image in the updated image feature set, and determining the first image features as the negative example features corresponding to the sample image.
In one embodiment, the loss calculation subunit is specifically configured to: calculating the model optimization loss by adopting an information noise convergence estimation loss function according to the positive example feature, the negative example feature and the second image feature corresponding to the sample image, wherein the information noise convergence estimation loss function
Wherein q is a second image feature corresponding to the sample image, t+Is the corresponding positive example feature of the sample image, ti-And the number of the ith negative example features corresponding to the sample image is K, and tau is a hyperparameter.
In one embodiment, the image feature set acquisition module is specifically configured to: performing image feature extraction on each sample image in the training data set by adopting an initial image feature extraction model to obtain a first image feature of each sample image; based on the first image features of each sample image in the training data set, an image feature set corresponding to the training data set is generated.
In one embodiment, the second image feature extraction module is specifically configured to: for any sample image in the training data set, inputting the sample image into an image retrieval model to be optimized to obtain a second original image characteristic corresponding to the sample image; and carrying out affine transformation and norm normalization processing on the second original image characteristics to obtain the correspondingly processed second image characteristics.
In one embodiment, the third image feature extraction module is specifically configured to: momentum updating is carried out on the image feature extraction model based on the image retrieval model to be optimized, and an updated image feature extraction model is obtained; inputting the sample image into an image feature extraction model after momentum updating to obtain a third original image feature corresponding to the sample image; and carrying out affine transformation and norm normalization processing on the third original image characteristics to obtain correspondingly processed third image characteristics.
For specific limitations of the image retrieval model optimization device, reference may be made to the above limitations of the image retrieval model optimization method, which are not described herein again. The respective blocks in the image retrieval model optimization apparatus described above may be wholly or partially implemented by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, as shown in fig. 10, an image retrieval apparatus is provided, which includes an image to be retrieved acquisition module 1001 and a retrieval processing module 1002, wherein:
an image to be retrieved acquisition module 1001 configured to acquire an image to be retrieved;
the retrieval processing module 1002 is configured to input the image to be retrieved into the optimized image retrieval model obtained by the image retrieval model optimization method described above, so as to obtain a corresponding target image.
For specific limitations of the image retrieval apparatus, reference may be made to the above limitations of the image retrieval method, which are not described herein again. The modules in the image retrieval device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 11. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing the image feature set corresponding to the training data set. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an image retrieval model optimization method or to implement an image retrieval method.
Those skilled in the art will appreciate that the architecture shown in fig. 11 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring a first image feature of each sample image in a training data set to obtain an image feature set corresponding to the training data set, wherein the training data set comprises a plurality of sample images;
for any sample image in the training data set, carrying out image feature extraction through an image retrieval model to be optimized to obtain a second image feature of the sample image;
performing image feature extraction on the sample image through an image feature extraction model to obtain a third image feature of the sample image, wherein the image feature extraction model is obtained after momentum updating is performed on the basis of the image retrieval model to be optimized;
determining the loss of model optimization according to the second image feature and the third image feature of the sample image and the image feature set corresponding to the training data set;
and carrying out back propagation on the image retrieval model to be optimized according to the loss to obtain the optimized image retrieval model.
In one embodiment, the processor, when executing the computer program, further performs the steps of: according to the third image feature of the sample image, updating the first image feature corresponding to the sample image in the image feature set to obtain an updated image feature set; determining a loss of model optimization according to the updated image feature set and the second image features of the sample image.
In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring positive example features and negative example features corresponding to the sample image based on the updated image feature set; and calculating the loss of model optimization according to the positive example feature and the negative example feature corresponding to the sample image and the second image feature of the sample image.
In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring an updated first image feature corresponding to the sample image in the updated image feature set, and determining the updated first image feature as a normal feature corresponding to the sample image; and acquiring other first image features except the positive example features corresponding to the sample image in the updated image feature set, and determining the first image features as the negative example features corresponding to the sample image.
In one embodiment, the processor, when executing the computer program, further performs the steps of: calculating the model optimization loss by adopting an information noise convergence estimation loss function according to the positive example feature, the negative example feature and the second image feature corresponding to the sample image, wherein the information noise convergence estimation loss function
Wherein q is a second image feature corresponding to the sample image, t+Is the corresponding positive example feature of the sample image, ti-And the number of the ith negative example features corresponding to the sample image is K, and tau is a hyperparameter.
In one embodiment, the processor, when executing the computer program, further performs the steps of: performing image feature extraction on each sample image in the training data set by adopting an initial image feature extraction model to obtain a first image feature of each sample image; based on the first image features of each sample image in the training data set, an image feature set corresponding to the training data set is generated.
In one embodiment, the processor, when executing the computer program, further performs the steps of: for any sample image in the training data set, inputting the sample image into an image retrieval model to be optimized to obtain a second original image characteristic corresponding to the sample image; and carrying out affine transformation and norm normalization processing on the second original image characteristics to obtain the correspondingly processed second image characteristics.
In one embodiment, the processor, when executing the computer program, further performs the steps of: momentum updating is carried out on the image feature extraction model based on the image retrieval model to be optimized, and an updated image feature extraction model is obtained; inputting the sample image into an image feature extraction model after momentum updating to obtain a third original image feature corresponding to the sample image; and carrying out affine transformation and norm normalization processing on the third original image characteristics to obtain correspondingly processed third image characteristics.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring an image to be retrieved;
and inputting the image to be retrieved into the optimized image retrieval model obtained by the image retrieval model optimization method to obtain a corresponding target image.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a first image feature of each sample image in a training data set to obtain an image feature set corresponding to the training data set, wherein the training data set comprises a plurality of sample images;
for any sample image in the training data set, carrying out image feature extraction through an image retrieval model to be optimized to obtain a second image feature of the sample image;
performing image feature extraction on the sample image through an image feature extraction model to obtain a third image feature of the sample image, wherein the image feature extraction model is obtained after momentum updating is performed on the basis of the image retrieval model to be optimized;
determining the loss of model optimization according to the second image feature and the third image feature of the sample image and the image feature set corresponding to the training data set;
and carrying out back propagation on the image retrieval model to be optimized according to the loss to obtain the optimized image retrieval model.
In one embodiment, the computer program when executed by the processor further performs the steps of: according to the third image feature of the sample image, updating the first image feature corresponding to the sample image in the image feature set to obtain an updated image feature set; determining a loss of model optimization according to the updated image feature set and the second image features of the sample image.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring positive example features and negative example features corresponding to the sample image based on the updated image feature set; and calculating the loss of model optimization according to the positive example feature and the negative example feature corresponding to the sample image and the second image feature of the sample image.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring an updated first image feature corresponding to the sample image in the updated image feature set, and determining the updated first image feature as a normal feature corresponding to the sample image; and acquiring other first image features except the positive example features corresponding to the sample image in the updated image feature set, and determining the first image features as the negative example features corresponding to the sample image.
In one embodiment, the computer program when executed by the processor further performs the steps of: calculating the model optimization loss by adopting an information noise convergence estimation loss function according to the positive example feature, the negative example feature and the second image feature corresponding to the sample image, wherein the information noise convergence estimation loss function
Wherein q is a second image feature corresponding to the sample image, t+Is the corresponding positive example feature of the sample image, ti-The ith negative example characteristic corresponding to the sample imageK is the number of the negative example characteristics, and tau is a hyperparameter.
In one embodiment, the computer program when executed by the processor further performs the steps of: performing image feature extraction on each sample image in the training data set by adopting an initial image feature extraction model to obtain a first image feature of each sample image; based on the first image features of each sample image in the training data set, an image feature set corresponding to the training data set is generated.
In one embodiment, the computer program when executed by the processor further performs the steps of: for any sample image in the training data set, inputting the sample image into an image retrieval model to be optimized to obtain a second original image characteristic corresponding to the sample image; and carrying out affine transformation and norm normalization processing on the second original image characteristics to obtain the correspondingly processed second image characteristics.
In one embodiment, the computer program when executed by the processor further performs the steps of: momentum updating is carried out on the image feature extraction model based on the image retrieval model to be optimized, and an updated image feature extraction model is obtained; inputting the sample image into an image feature extraction model after momentum updating to obtain a third original image feature corresponding to the sample image; and carrying out affine transformation and norm normalization processing on the third original image characteristics to obtain correspondingly processed third image characteristics.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring an image to be retrieved;
and inputting the image to be retrieved into the optimized image retrieval model obtained by the image retrieval model optimization method to obtain a corresponding target image.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (13)
1. An image retrieval model optimization method, the method comprising:
acquiring a first image feature of each sample image in a training data set to obtain an image feature set corresponding to the training data set, wherein the training data set comprises a plurality of sample images;
for any sample image in the training data set, carrying out image feature extraction through an image retrieval model to be optimized to obtain a second image feature of the sample image;
performing image feature extraction on the sample image through an image feature extraction model to obtain a third image feature of the sample image, wherein the image feature extraction model is obtained after momentum updating is performed on the basis of the image retrieval model to be optimized;
determining the loss of model optimization according to the second image feature and the third image feature of the sample image and the image feature set corresponding to the training data set;
and carrying out back propagation on the image retrieval model to be optimized according to the loss to obtain the optimized image retrieval model.
2. The method of claim 1, wherein determining a loss of model optimization based on the second image feature, the third image feature of the sample image, and the set of image features corresponding to the training dataset comprises:
according to the third image feature of the sample image, updating the first image feature corresponding to the sample image in the image feature set to obtain an updated image feature set;
determining a loss of model optimization according to the updated image feature set and the second image features of the sample image.
3. The method of claim 2, wherein determining a loss of model optimization from the updated set of image features and the second image features of the sample image comprises:
acquiring positive example features and negative example features corresponding to the sample image based on the updated image feature set;
and calculating the loss of model optimization according to the positive example feature and the negative example feature corresponding to the sample image and the second image feature of the sample image.
4. The method of claim 3, wherein obtaining positive and negative example features corresponding to the sample image based on the updated set of image features comprises:
acquiring an updated first image feature corresponding to the sample image in the updated image feature set, and determining the updated first image feature as a normal feature corresponding to the sample image;
and acquiring other first image features except the positive example features corresponding to the sample image in the updated image feature set, and determining the first image features as the negative example features corresponding to the sample image.
5. The method of claim 4, wherein calculating a loss of model optimization based on the positive and negative side features corresponding to the sample image and the second image feature of the sample image comprises:
calculating the model optimization loss by adopting an information noise convergence estimation loss function according to the positive example feature, the negative example feature and the second image feature corresponding to the sample image, wherein the information noise convergence estimation loss function
Wherein q is a second image feature corresponding to the sample image, t+Is the corresponding positive example feature of the sample image, ti-And the number of the ith negative example features corresponding to the sample image is K, and tau is a hyperparameter.
6. The method of any one of claims 1 to 5, wherein the obtaining the first image feature of each sample image in the training data set to obtain an image feature set corresponding to the training data set comprises:
performing image feature extraction on each sample image in the training data set by adopting an initial image feature extraction model to obtain a first image feature of each sample image;
based on the first image features of each sample image in the training data set, an image feature set corresponding to the training data set is generated.
7. The method according to any one of claims 1 to 5, wherein the obtaining, for any one sample image in the training data set, a second image feature of the sample image by performing image feature extraction through an image retrieval model to be optimized comprises:
for any sample image in the training data set, inputting the sample image into an image retrieval model to be optimized to obtain a second original image characteristic corresponding to the sample image;
and carrying out affine transformation and norm normalization processing on the second original image characteristics to obtain the correspondingly processed second image characteristics.
8. The method according to any one of claims 1 to 5, wherein the performing image feature extraction on the sample image through an image feature extraction model to obtain a third image feature of the sample image comprises:
momentum updating is carried out on the image feature extraction model based on the image retrieval model to be optimized, and an updated image feature extraction model is obtained;
inputting the sample image into an image feature extraction model after momentum updating to obtain a third original image feature corresponding to the sample image;
and carrying out affine transformation and norm normalization processing on the third original image characteristics to obtain correspondingly processed third image characteristics.
9. An image retrieval method, characterized in that the method comprises:
acquiring an image to be retrieved;
inputting the image to be retrieved into the optimized image retrieval model obtained by the image retrieval model optimization method according to any one of claims 1 to 8, and obtaining a corresponding target image.
10. An image retrieval model optimization apparatus, characterized in that the apparatus comprises:
the image feature set acquisition module is used for acquiring a first image feature of each sample image in a training data set to obtain an image feature set corresponding to the training data set, wherein the training data set comprises a plurality of sample images;
the second image feature extraction module is used for extracting image features of any sample image in the training data set through an image retrieval model to be optimized to obtain second image features of the sample image;
the third image feature extraction module is used for extracting image features of the sample image through an image feature extraction model to obtain third image features of the sample image, and the image feature extraction model is obtained after momentum updating is carried out on the basis of the image retrieval model to be optimized;
the loss obtaining module is used for determining the loss of model optimization according to the second image characteristic and the third image characteristic of the sample image and the image characteristic set corresponding to the training data set;
and the model optimization module is used for performing back propagation on the image retrieval model to be optimized according to the loss to obtain the optimized image retrieval model.
11. An image retrieval apparatus, characterized in that the apparatus comprises:
the image to be retrieved acquiring module is used for acquiring an image to be retrieved;
a retrieval processing module, configured to input the image to be retrieved into the optimized image retrieval model obtained by the image retrieval model optimization method according to any one of claims 1 to 8, so as to obtain a corresponding target image.
12. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 8.
13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011066003.7A CN112232360A (en) | 2020-09-30 | 2020-09-30 | Image retrieval model optimization method, image retrieval device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011066003.7A CN112232360A (en) | 2020-09-30 | 2020-09-30 | Image retrieval model optimization method, image retrieval device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112232360A true CN112232360A (en) | 2021-01-15 |
Family
ID=74119335
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011066003.7A Pending CN112232360A (en) | 2020-09-30 | 2020-09-30 | Image retrieval model optimization method, image retrieval device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112232360A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113920404A (en) * | 2021-11-09 | 2022-01-11 | 北京百度网讯科技有限公司 | Training method, image processing method, device, electronic device and storage medium |
| WO2022161357A1 (en) * | 2021-01-29 | 2022-08-04 | 北京有竹居网络技术有限公司 | Data augmentation-based training sample acquisition method and apparatus, and electronic device |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108898185A (en) * | 2018-07-03 | 2018-11-27 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating image recognition model |
| CN109685121A (en) * | 2018-12-11 | 2019-04-26 | 中国科学院苏州纳米技术与纳米仿生研究所 | Training method, image search method, the computer equipment of image encrypting algorithm |
| CN109829894A (en) * | 2019-01-09 | 2019-05-31 | 平安科技（深圳）有限公司 | Parted pattern training method, OCT image dividing method, device, equipment and medium |
| CN110209867A (en) * | 2019-06-05 | 2019-09-06 | 腾讯科技（深圳）有限公司 | Training method, device, equipment and the storage medium of image encrypting algorithm |
| CN110516737A (en) * | 2019-08-26 | 2019-11-29 | 南京人工智能高等研究院有限公司 | Method and apparatus for generating image recognition model |
| CN111008294A (en) * | 2018-10-08 | 2020-04-14 | 阿里巴巴集团控股有限公司 | Traffic image processing and image retrieval method and device |
| CN111079632A (en) * | 2019-12-12 | 2020-04-28 | 上海眼控科技股份有限公司 | Training method and device of text detection model, computer equipment and storage medium |
| US20200160512A1 (en) * | 2018-11-16 | 2020-05-21 | Boe Technology Group Co., Ltd. | Method, client, server and system for detecting tongue image, and tongue imager |
| CN111275060A (en) * | 2018-12-04 | 2020-06-12 | 北京嘀嘀无限科技发展有限公司 | Recognition model updating processing method and device, electronic equipment and storage medium |
-
2020
- 2020-09-30 CN CN202011066003.7A patent/CN112232360A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108898185A (en) * | 2018-07-03 | 2018-11-27 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating image recognition model |
| CN111008294A (en) * | 2018-10-08 | 2020-04-14 | 阿里巴巴集团控股有限公司 | Traffic image processing and image retrieval method and device |
| US20200160512A1 (en) * | 2018-11-16 | 2020-05-21 | Boe Technology Group Co., Ltd. | Method, client, server and system for detecting tongue image, and tongue imager |
| CN111275060A (en) * | 2018-12-04 | 2020-06-12 | 北京嘀嘀无限科技发展有限公司 | Recognition model updating processing method and device, electronic equipment and storage medium |
| CN109685121A (en) * | 2018-12-11 | 2019-04-26 | 中国科学院苏州纳米技术与纳米仿生研究所 | Training method, image search method, the computer equipment of image encrypting algorithm |
| CN109829894A (en) * | 2019-01-09 | 2019-05-31 | 平安科技（深圳）有限公司 | Parted pattern training method, OCT image dividing method, device, equipment and medium |
| CN110209867A (en) * | 2019-06-05 | 2019-09-06 | 腾讯科技（深圳）有限公司 | Training method, device, equipment and the storage medium of image encrypting algorithm |
| CN110516737A (en) * | 2019-08-26 | 2019-11-29 | 南京人工智能高等研究院有限公司 | Method and apparatus for generating image recognition model |
| CN111079632A (en) * | 2019-12-12 | 2020-04-28 | 上海眼控科技股份有限公司 | Training method and device of text detection model, computer equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 杜凯旋;王亮;王勇;车向红;: "主动学习和卷积神经网络的地图图片识别方法", 测绘科学, no. 07, 20 July 2020 (2020-07-20) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022161357A1 (en) * | 2021-01-29 | 2022-08-04 | 北京有竹居网络技术有限公司 | Data augmentation-based training sample acquisition method and apparatus, and electronic device |
| CN113920404A (en) * | 2021-11-09 | 2022-01-11 | 北京百度网讯科技有限公司 | Training method, image processing method, device, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109492674B (en) | Generation method and device of SSD (solid State disk) framework for target detection | |
| CN113902926A (en) | General image target detection method and device based on self-attention mechanism | |
| US20230316733A1 (en) | Video behavior recognition method and apparatus, and computer device and storage medium | |
| CN113505797B (en) | Model training method and device, computer equipment and storage medium | |
| CN112232397A (en) | Knowledge distillation method and device of image classification model and computer equipment | |
| CN114549913B (en) | Semantic segmentation method and device, computer equipment and storage medium | |
| CN113344016A (en) | Deep migration learning method and device, electronic equipment and storage medium | |
| CN113065525A (en) | Age recognition model training method, face age recognition method and related device | |
| CN110188422B (en) | Method and device for extracting feature vector of node based on network data | |
| CN112232360A (en) | Image retrieval model optimization method, image retrieval device and storage medium | |
| CN112749737A (en) | Image classification method and device, electronic equipment and storage medium | |
| CN114155388B (en) | Image recognition method and device, computer equipment and storage medium | |
| CN114255381A (en) | Training method of image recognition model, image recognition method, device and medium | |
| CN117315090A (en) | Cross-modal style learning-based image generation method and device | |
| US12112524B2 (en) | Image augmentation method, electronic device and readable storage medium | |
| US20210042625A1 (en) | Performance of neural networks using learned specialized transformation functions | |
| CN116975347A (en) | Image generation model training method and related device | |
| CN111091198A (en) | Data processing method and device | |
| CN113743448B (en) | Model training data acquisition method, model training method and device | |
| CN115758271A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN112633285B (en) | Domain adaptation method, domain adaptation device, electronic equipment and storage medium | |
| CN116150462A (en) | Vector construction method and device for target object and computer equipment | |
| CN113591637B (en) | Training method and device for alignment model, computer equipment and storage medium | |
| CN113569887B (en) | Picture recognition model training and picture recognition method, device and storage medium | |
| CN113792163B (en) | Multimedia recommendation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |