CN114549938B - Model training method, image information management method, image recognition method and device - Google Patents

Abstract

The invention discloses a model training method, an image information management method, an image recognition method and a device. The model training method includes: randomly initializing an initial image processing model; the initial image processing model includes a feature extraction branch and a prototype building branch; fixing the parameters of the prototype building branch, and inputting the training image set into the initial image processing model to obtain the first image processing model. An output result; determine a loss function according to the first output result, and use the loss function to update the initialization parameters of the feature extraction branch; fix the updated parameters of the feature extraction branch, input the training image set into the initial image processing model, and obtain the second Output results; determine a loss function according to the second output result, and use the loss function to update the initialization parameters of the prototype construction branch; build a commodity fingerprint extraction model according to the updated parameters of the feature extraction branch, so that the commodity fingerprint output by the commodity fingerprint extraction model has better differentiation.

Description

Model training method, image information management method, image recognition method and device

technical field

The present invention relates to the technical field of image processing, in particular to a model training method, an image information management method, an image recognition method and a device.

Background technique

Commodity fingerprint technology is a technology that represents the uniqueness of commodities and their relationships through digital coding. Through commodity fingerprint modeling technology, an effective representation of commodity pictures can be obtained without manual identification of commodities. Therefore, commodity fingerprint modeling technology can quickly Promote it to applications such as product classification and similar product retrieval.

At present, when using the commodity fingerprint matching technology for commodity identification, it is necessary to store the reference fingerprints of each commodity in the commodity fingerprint database, so that the target commodity fingerprint can be searched and identified in the complete fingerprint database. However, in practical application scenarios, commodities in the fast-selling industry tend to be of various types and bring forth new ones quickly, and the commodities are easily affected by the environment such as oblique shooting, posture and lighting, which makes the scale of the benchmark fingerprints to be stored in the commodity fingerprint database huge. It takes up a lot of memory, and at the same time, the fingerprint matching process of the product takes more time to traverse the fingerprint database, and the identification efficiency is low.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve at least one of the technical problems existing in the prior art, and to provide a model training method, an image information management method, an image recognition method and a device, which can improve the distinguishability of commodity fingerprints and reduce the number of stored fingerprints at the same time. , the technical solution is as follows:

In a first aspect, the present invention provides a model training method, comprising:

Randomly initialize the parameters of the initial image processing model; wherein, the initial image processing model includes a feature extraction branch and a prototype building branch;

Fix the initialization parameters of the prototype construction branch, input the training image set into the initial image processing model, and obtain the first output result;

Determine the value of the loss function according to the first output result, and use the value of the loss function to update the initialization parameter of the feature extraction branch;

Fixing the updated parameters of the feature extraction branch, and inputting the training image set into the initial image processing model after updating the parameters of the feature extraction branch to obtain a second output result;

Determine the value of the loss function according to the second output result, and use the value of the loss function to update the initialization parameter of the prototype building branch;

A commodity fingerprint extraction model is constructed according to the updated parameters of the feature extraction branch.

As a further improvement, both the first output result and the second output result include the commodity fingerprint output by the feature extraction branch and the fingerprint prototype output by the prototype building branch.

As a further improvement, the class loss is defined as:

In the formula, f _x represents the commodity fingerprint output by the feature extraction branch, e represents the fingerprint prototype output by the prototype construction branch, T represents the temperature coefficient, and N represents the number of image samples;

The intra-class and inter-class loss is defined as:

表示边界阈值，Z _p 表示第一输出结果或第二输出结果中与所述特征提取分支输出的商品指纹类别相同的结果，Z _n 表示第一输出结果或第二输出结果中与所述特征提取分支输出的商品指纹类别不相同的结果。In the formula,

represents the boundary threshold , Z _p represents the first output result or the second output result that is the same as the product fingerprint category output by the feature extraction branch, Z _n represents the first output result or the second output result that is the same as the feature extraction result The product fingerprint categories output by the branch are not the same result.

In a second aspect, the present invention also provides an image information management method, comprising:

Obtaining a scene image set, and using an image detection model to obtain the full amount of commodity images in the scene image set;

The commodity fingerprints in the commodity images are extracted according to the commodity fingerprint extraction model to obtain a commodity fingerprint set; wherein,

The commodity fingerprint extraction model is a model trained by the model training method described in the first aspect;

Determine the commodity category of the commodity fingerprint set, and use a clustering algorithm to compress commodity fingerprints corresponding to each commodity category;

Store the compressed commodity fingerprints in the commodity fingerprint database.

As a further improvement, the use of a clustering algorithm to compress the commodity fingerprints corresponding to each commodity category is specifically: determining the commodity category of each commodity fingerprint in the commodity fingerprint set; according to a preset quantity parameter N, using clustering The algorithm compresses the commodity fingerprints under each commodity category into N clusters in turn, and sets the cluster centers of the N clusters as the target commodity fingerprints of the corresponding commodity category; among them, N is a non-zero natural number; The target commodity fingerprint of the commodity category is stored in the commodity fingerprint database.

In a third aspect, the present invention also provides an image recognition method, comprising:

acquiring a target commodity image, and extracting the target commodity fingerprint of the target commodity image by using a commodity fingerprint extraction model;

Based on the target commodity fingerprint traversing the benchmark fingerprints in the commodity fingerprint database, the commodity category corresponding to the benchmark fingerprint with the highest similarity with the target commodity fingerprint is determined as the identification result; wherein,

The commodity fingerprint extraction model is a model trained by the model training method described in the first aspect;

The commodity fingerprint database is obtained by the image information management method according to the second aspect.

In a fourth aspect, the present invention also provides a model training device, comprising:

an initialization module for randomly initializing parameters of an initial image processing model; wherein, the initial image processing model includes a feature extraction branch and a prototype building branch;

The first training module is used to fix the initialization parameters of the prototype construction branch, input the training image set into the initial image processing model, and obtain the first output result;

Determine the value of the loss function according to the first output result, and use the value of the loss function to update the initialization parameter of the feature extraction branch;

The second training module is used to fix the updated parameters of the feature extraction branch, and input the training image set into the initial image processing model after the updated parameters of the feature extraction branch to obtain a second output result;

Determine the value of the loss function according to the second output result, and use the value of the loss function to update the initialization parameter of the prototype building branch;

A determination module, configured to construct a commodity fingerprint extraction model according to the updated parameters of the feature extraction branch.

As a further improvement, both the first output result and the second output result include the commodity fingerprint output by the feature extraction branch and the fingerprint prototype output by the prototype building branch.

Meanwhile, the present invention provides a data processing device, comprising a processor, the processor is coupled with a memory, the memory stores a program, the program is executed by the processor, so that the data processing device executes the first aspect The model training method, the image information management method described in the second aspect, or the image recognition method described in the third aspect.

The present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the model training method described in the first aspect above, or the image described in the second aspect. An information management method, or the image recognition method described in the third aspect.

Compared with the prior art, the technical solutions provided by the present invention have at least the following beneficial effects:

1. By training the dual-branch model including the feature extraction branch and the prototype construction branch, it can ensure that the commodity fingerprint output by the constructed commodity fingerprint extraction model has better distinguishability;

2. By using the dual-branch commodity fingerprint extraction model and clustering method to extract and compress commodity images of commodity images in application scenarios, the storage scale of commodity fingerprint database can be effectively reduced on the premise of ensuring commodity recognition rate, thereby reducing The matching search time of commodity fingerprints improves the efficiency of commodity fingerprint identification.

Description of drawings

In order to illustrate the technical solutions of the present invention more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, which are common in the art. As far as technical personnel are concerned, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic flowchart of a model training method provided by an embodiment of the present invention;

2 is a schematic flowchart of an image information management method provided by an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a model training apparatus provided by an embodiment of the present invention.

Detailed ways

This part will describe the specific embodiments of the present invention in detail, and the preferred embodiments of the present invention are shown in the accompanying drawings. Each technical feature and overall technical solution of the invention should not be construed as limiting the protection scope of the invention.

As shown in FIG. 1 , in a first aspect, an embodiment of the present invention provides a model training method, including the following steps S101 to S104.

S101: Randomly initialize parameters of an initial image processing model; wherein, the initial image processing model includes a feature extraction branch and a prototype building branch.

S102: Fix the initialization parameters of the prototype construction branch, input the training image set into the initial image processing model, and obtain a first output result; determine the value of the loss function according to the first output result, and use the loss function value to update the initialization parameters of the feature extraction branch.

The training image set used in this embodiment is specifically a commodity image set, and each commodity image is obtained by segmenting a large number of collected real images of fast-selling scenes such as shelves, end racks, and refrigerators. real images for image segmentation.

S103: Fix the updated parameters of the feature extraction branch, input the training image set into the initial image processing model after the updated parameters of the feature extraction branch, and obtain a second output result; determine the The value of the loss function, using the value of the loss function to update the initialization parameters of the prototype building branch.

S104: Build a commodity fingerprint extraction model according to the updated parameters of the feature extraction branch.

It should be noted that both the first output result and the second output result include the commodity fingerprint output by the feature extraction branch and the fingerprint prototype output by the prototype building branch.

Specifically, the loss function includes two parts: category loss and intra-class loss, where category loss is used to describe the similarity between commodity fingerprints and fingerprint prototypes.

Specifically, the class loss is defined as:

In the formula, f _x represents the commodity fingerprint output by the feature extraction branch, e represents the fingerprint prototype output by the prototype construction branch, T represents the temperature coefficient, and N represents the number of image samples;

The intra-class and inter-class loss is defined as:

表示边界阈值，Z _p 表示第一输出结果或第二输出结果中与所述特征提取分支输出的商品指纹类别相同的结果，Z _n 表示第一输出结果或第二输出结果中与所述特征提取分支输出的商品指纹类别不相同的结果。In the formula,

represents the boundary threshold , Z _p represents the first output result or the second output result that is the same as the product fingerprint category output by the feature extraction branch, Z _n represents the first output result or the second output result that is the same as the feature extraction result The product fingerprint categories output by the branch are not the same result.

In this embodiment, the loss function can be specifically expressed as:

Among them, α represents the weight coefficient.

Exemplarily, after acquiring the training image set, first randomly initialize the feature extraction branch and the prototype building branch in the initial image processing model, and then repeat the following training steps until the model converges.

The training image is input into the initial image processing model, and the product fingerprint f _x is output through the feature extraction branch. First, keep the parameters of the prototype construction branch unchanged, and train the feature extraction branch. Specifically, the parameters of the feature extraction branch are updated through the back-propagation of the loss function until the iteration ends.

Further, keeping the updated parameters of the feature extraction branch unchanged, and updating the parameters of the prototype construction branch based on the back-propagation of the above loss function.

Specifically, when each epoch is completed, the learning rate of the initial image processing model is exponentially decayed.

After completing the above training steps, save the feature extraction branch parameters and set them as the final commodity fingerprint extraction model.

The above-mentioned embodiments of the present invention train the dual-branch model including the feature extraction branch and the prototype construction branch, so that the fingerprint distribution of the same category of commodities under different environmental influences can be surrounded by the same fingerprint prototype, and the prototype distance of different categories of commodities can be widened at the same time. It is ensured that the commodity fingerprint output by the constructed commodity fingerprint extraction model has better distinguishability, and can better identify similar commodities and unseen commodities.

As shown in FIG. 2, in a second aspect, an embodiment of the present invention further provides an image information management method, including the following steps S201 to S203.

S201: Acquire a scene image set, and use an image detection model to acquire a full amount of commodity images in the scene image set.

It can be understood that the scene image set specifically includes real images of fast-selling scenes such as shelves, end racks, and refrigerators, and the real images are segmented by the commodity detection model to obtain the full amount of commodity images in the scene image set.

S202: Extract commodity fingerprints in the commodity image according to the commodity fingerprint extraction model to obtain a commodity fingerprint set. Wherein, the commodity fingerprint extraction model is obtained by training the model training method described in the above embodiment.

S203: Determine the commodity category of the commodity fingerprint set, use a clustering algorithm to compress commodity fingerprints corresponding to each commodity category, and store the compressed commodity fingerprints in a commodity fingerprint database.

In an example, when using the clustering algorithm to compress the commodity fingerprints corresponding to each commodity category, the commodity category of each commodity fingerprint in the commodity fingerprint set can be determined first; The commodity fingerprints under the commodity category are compressed into N clusters, and the cluster centers of the N clusters are set as the target commodity fingerprints of the corresponding commodity categories; among them, N is a non-zero natural number; finally, the target commodity fingerprints of each commodity category are set. Stored in the commodity fingerprint database.

Specifically, the clustering algorithm used may be the K-means algorithm.

Exemplarily, when the commodity fingerprint set S _i of the i -th category is extracted by using the commodity fingerprint extraction model, the commodity fingerprint set S _i can be clustered into N clusters by the K-means algorithm, and the centers of the N clusters can be divided into N clusters. As the N commodity fingerprints of the i -th category commodity.

It should be noted that the quantity parameter N is a value smaller than the number of commodity fingerprints in the commodity fingerprint set S _i , and different commodity categories can determine the corresponding quantity parameter according to whether the outer packaging of the commodity has multiple different surfaces.

Specifically, the N cluster centers can be calculated by the following formula:

，y表示商品类别，M表示第i类别商品第N个类簇的商品图像数量。in,

, y represents the product category, and M represents the number of product images in the Nth cluster of the i -th product.

Further, N commodity fingerprints and their corresponding commodity categories i are written into the commodity fingerprint database, until commodity fingerprints of all commodity categories are stored in the commodity fingerprint database.

The above-mentioned embodiments of the present invention extract and compress commodity images under application scenarios by using a dual-branch commodity fingerprint extraction model and a clustering method, so that under the premise of ensuring the commodity recognition rate, fewer fingerprints can be used to characterize each scenario. Therefore, the storage scale of the number of reference fingerprints in the commodity fingerprint database is effectively reduced, the matching search time of commodity fingerprints is reduced, and the efficiency of commodity fingerprint identification is improved.

In a third aspect, an embodiment of the present invention also provides an image recognition method, which specifically includes: acquiring a target commodity image, extracting a target commodity fingerprint of the target commodity image by using a commodity fingerprint extraction model, and then traversing the commodity fingerprint database based on the target commodity fingerprint. The reference fingerprint, the reference fingerprint with the highest similarity with the target commodity fingerprint is determined as the identification result.

Wherein, the commodity fingerprint extraction model is a model trained by the model training method described in the first aspect, and the commodity fingerprint database is obtained by the image information management method described in the second aspect.

Specifically, when the target product image is a known product, the product fingerprint can be extracted directly from the product image, and the cosine distance is used as the metric function to extract the product corresponding to the reference fingerprint with the largest cosine similarity in the product fingerprint database. category as the final recognition result.

On the other hand, when the target product image is an unseen product, M images of unseen products are randomly selected for marking, that is, the product category label is set; Fingerprints and their corresponding commodity categories are added to the commodity fingerprint database.

Further, the commodity category corresponding to the reference fingerprint with the largest cosine similarity is taken from the commodity fingerprint database as the final identification result.

Referring to FIG. 3 , another embodiment of the present invention further provides a model training apparatus, including an initialization module 101 , a first training module 102 , a second training module 103 and a determination module 104 .

The initialization module 101 is used to randomly initialize the parameters of the initial image processing model; wherein, the initial image processing model includes a feature extraction branch and a prototype building branch.

The first training module 102 is configured to fix the initialization parameters of the prototype building branch, input the training image set to the initial image processing model, and obtain a first output result.

The value of the loss function is determined according to the first output result, and the initialization parameter of the feature extraction branch is updated by using the value of the loss function.

The second training module 103 is configured to fix the updated parameters of the feature extraction branch, and input the training image set into the initial image processing model after the updated parameters of the feature extraction branch to obtain a second output result.

The value of the loss function is determined according to the second output result, and the initialization parameter of the prototype building branch is updated by using the value of the loss function.

The determining module 104 is configured to construct a commodity fingerprint extraction model according to the updated parameters of the feature extraction branch.

Since the information exchange and execution process among the modules in the above-mentioned device are based on the same concept as the embodiment of the model training method of the present invention, the specific content can be found in the description in the method embodiment of the present invention, and will not be repeated here.

The present invention provides a data processing device, comprising a processor, the processor is coupled with a memory, the memory stores a program, and the program is executed by the processor, so that the data processing device executes the above-mentioned first aspect. The model training method described in the second aspect, the image information management method described in the second aspect, or the image recognition method described in the third aspect.

The present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the model training method described in the first aspect or the image information described in the second aspect The management method, or the image recognition method described in the third aspect.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer monitorable storage medium. , may include the flow of the above-mentioned method embodiments. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

Claims (9)

1. A method of model training, comprising:

carrying out random initialization on parameters of the initial image processing model; wherein the initial image processing model comprises a feature extraction branch and a prototype construction branch;

fixing the initialization parameters of the prototype building branch, and inputting a training image set to the initial image processing model to obtain a first output result;

determining a value of a loss function according to the first output result, and updating the initialization parameter of the feature extraction branch by using the value of the loss function; the loss function comprises a class loss and an intra-class inter-class loss; wherein,

the class loss is defined as

(ii) a In the formula,f _x a fingerprint of the commodity representing the output of the feature extraction branch,ea fingerprint prototype representing the prototype-building branch output,< , >the inner product is represented by the sum of the two,ithe commodity type is represented, T represents a temperature coefficient, and N represents the number of image samples;

the intra-class inter-class loss is defined as

(ii) a In the formula,

indicating a boundary threshold，z _p The result which is the same as the commodity fingerprint category output by the characteristic extraction branch in the first output result or the second output result is represented,z _n the result which is different from the commodity fingerprint category output by the characteristic extraction branch in the first output result or the second output result is represented;

fixing the updated parameters of the feature extraction branch, and inputting the training image set into the initial image processing model after the parameters of the feature extraction branch are updated to obtain a second output result;

determining the value of the loss function according to the second output result, and updating the initialization parameter of the prototype construction branch by using the value of the loss function;

and constructing a commodity fingerprint extraction model according to the parameters updated by the feature extraction branches.

2. The model training method of claim 1, wherein the first output result and the second output result each comprise:

and the commodity fingerprint output by the branch is extracted by the characteristic extraction module and the prototype of the fingerprint output by the prototype construction module.

3. An image information management method characterized by comprising:

acquiring a scene image set, and acquiring a full commodity image in the scene image set by using an image detection model;

extracting the commodity fingerprint in the commodity image according to a commodity fingerprint extraction model to obtain a commodity fingerprint set; wherein,

the commodity fingerprint extraction model is a model trained by the model training method according to any one of claims 1-2;

determining the commodity category of the commodity fingerprint set, and compressing the commodity fingerprint corresponding to each commodity category by using a clustering algorithm;

and storing the compressed commodity fingerprint into a commodity fingerprint database.

4. The image information management method according to claim 3, wherein the compressing of the commodity fingerprint corresponding to each of the commodity categories by using a clustering algorithm is specifically:

determining the commodity category of each commodity fingerprint in the commodity fingerprint set;

according to a preset quantity parameter N, sequentially compressing the commodity fingerprints under each commodity category into N categories by using a clustering algorithm, and setting the cluster centers of the N categories as target commodity fingerprints corresponding to the commodity categories; wherein N is a non-zero natural number;

and storing the target commodity fingerprint of each commodity category in a commodity fingerprint database.

5. An image recognition method, comprising:

acquiring a target commodity image, and extracting a target commodity fingerprint of the target commodity image by using a commodity fingerprint extraction model;

traversing the reference fingerprints in the commodity fingerprint database based on the target commodity fingerprint, and determining the commodity category corresponding to the reference fingerprint with the highest similarity with the target commodity fingerprint as an identification result; wherein,

the commodity fingerprint extraction model is a model trained by the model training method according to any one of claims 1-2;

the commodity fingerprint database is obtained by the image information management method according to any one of claims 3 to 4.

6. A model training apparatus, comprising:

the initialization module is used for carrying out random initialization on the parameters of the initial image processing model; wherein the initial image processing model comprises a feature extraction branch and a prototype construction branch;

the first training module is used for fixing the initialization parameters of the prototype building branch, inputting a training image set to the initial image processing model and obtaining a first output result;

determining a value of a loss function according to the first output result, and updating the initialization parameter of the feature extraction branch by using the value of the loss function; the loss function comprises a class loss and an intra-class inter-class loss; wherein,

the class loss is defined as

(ii) a In the formula,f _x a fingerprint of the commodity representing the output of the feature extraction branch,ea fingerprint prototype representing the prototype-building branch output,< , >the inner product is represented by the sum of the two,ithe commodity type is represented, T represents a temperature coefficient, and N represents the number of image samples;

the intra-class inter-class loss is defined as

(ii) a In the formula,

indicating a boundary threshold，z _p The result which is the same as the commodity fingerprint category output by the characteristic extraction branch in the first output result or the second output result is represented,z _n the result which is different from the commodity fingerprint category output by the characteristic extraction branch in the first output result or the second output result is represented;

the second training module is used for fixing the updated parameters of the feature extraction branch, and inputting the training image set into the initial image processing model after the parameters of the feature extraction branch are updated to obtain a second output result;

determining the value of the loss function according to the second output result, and updating the initialization parameter of the prototype construction branch by using the value of the loss function;

and the determining module is used for constructing a commodity fingerprint extraction model according to the parameters updated by the feature extraction branches.

7. The model training apparatus of claim 6, wherein the first output result and the second output result each comprise:

and the commodity fingerprint output by the branch is extracted by the characteristic extraction module and the prototype of the fingerprint output by the prototype construction module.

8. A data processing apparatus, characterized by comprising:

a processor coupled to a memory, the memory storing a program for execution by the processor to cause the data processing apparatus to perform the model training method of any one of claims 1 to 2, or the image information management method of any one of claims 3 to 4, or the image recognition method of claim 5.

9. A computer storage medium storing computer instructions for performing the model training method according to any one of claims 1 to 2, or the image information management method according to any one of claims 3 to 4, or the image recognition method according to claim 5.

Images